From: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
To: "dalias@libc.org" <dalias@libc.org>
Cc: "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"suzuki.poulose@arm.com" <suzuki.poulose@arm.com>,
"Szabolcs.Nagy@arm.com" <Szabolcs.Nagy@arm.com>,
"musl@lists.openwall.com" <musl@lists.openwall.com>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"linux-riscv@lists.infradead.org"
<linux-riscv@lists.infradead.org>,
"kvmarm@lists.linux.dev" <kvmarm@lists.linux.dev>,
"corbet@lwn.net" <corbet@lwn.net>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
"broonie@kernel.org" <broonie@kernel.org>,
"oliver.upton@linux.dev" <oliver.upton@linux.dev>,
"palmer@dabbelt.com" <palmer@dabbelt.com>,
"debug@rivosinc.com" <debug@rivosinc.com>,
"aou@eecs.berkeley.edu" <aou@eecs.berkeley.edu>,
"shuah@kernel.org" <shuah@kernel.org>,
"arnd@arndb.de" <arnd@arndb.de>,
"maz@kernel.org" <maz@kernel.org>,
"oleg@redhat.com" <oleg@redhat.com>,
"fweimer@redhat.com" <fweimer@redhat.com>,
"keescook@chromium.org" <keescook@chromium.org>,
"james.morse@arm.com" <james.morse@arm.com>,
"ebiederm@xmission.com" <ebiederm@xmission.com>,
"will@kernel.org" <will@kernel.org>,
"brauner@kernel.org" <brauner@kernel.org>,
"hjl.tools@gmail.com" <hjl.tools@gmail.com>,
"linux-kselftest@vger.kernel.org"
<linux-kselftest@vger.kernel.org>,
"paul.walmsley@sifive.com" <paul.walmsley@sifive.com>,
"ardb@kernel.org" <ardb@kernel.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"thiago.bauermann@linaro.org" <thiago.bauermann@linaro.org>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"sorear@fastmail.com" <sorear@fastmail.com>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>
Subject: Re: [musl] Re: [PATCH v8 00/38] arm64/gcs: Provide support for GCS in userspace
Date: Wed, 21 Feb 2024 00:35:48 +0000 [thread overview]
Message-ID: <a57d6c7eada4b9a7c35addbc8556f5b53a0c3e6f.camel@intel.com> (raw)
In-Reply-To: <20240220235415.GP4163@brightrain.aerifal.cx>
On Tue, 2024-02-20 at 18:54 -0500, dalias@libc.org wrote:
> On Tue, Feb 20, 2024 at 11:30:22PM +0000, Edgecombe, Rick P wrote:
> > On Tue, 2024-02-20 at 13:57 -0500, Rich Felker wrote:
> > > On Tue, Feb 20, 2024 at 06:41:05PM +0000, Edgecombe, Rick P
> > > wrote:
> > > > Hmm, could the shadow stack underflow onto the real stack then?
> > > > Not
> > > > sure how bad that is. INCSSP (incrementing the SSP register on
> > > > x86)
> > > > loops are not rare so it seems like something that could
> > > > happen.
> > >
> > > Shadow stack underflow should fault on attempt to access
> > > non-shadow-stack memory as shadow-stack, no?
> >
> > Maybe I'm misunderstanding. I thought the proposal included
> > allowing
> > shadow stack access to convert normal address ranges to shadow
> > stack,
> > and normal writes to convert shadow stack to normal.
>
> As I understood the original discussion of the proposal on IRC, it
> was
> only one-way (from shadow to normal). Unless I'm missing something,
> making it one-way is necessary to catch situations where the shadow
> stack would become compromised.
The original post here:
https://lore.kernel.org/lkml/22a53b78-10d7-4a5a-a01e-b2f3a8c22e94@app.fastmail.com/
...has:
"Shadow stack faults on non-shadow stack pages, if flexible shadow
stack handling is in effect, cause the affected page to become a shadow
stack page. When this happens, the page filled with invalid address
tokens."
...and:
"Faults from non-shadow-stack accesses to a shadow-stack page which was
created by the previous paragraph will cause the page to revert to non-
shadow-stack usage, with or without clearing."
I see Stefan has clarified in another response. So I'll go try to
figure it out.
>
> > > > Shadow stacks currently have automatic guard gaps to try to
> > > > prevent
> > > > one
> > > > thread from overflowing onto another thread's shadow stack.
> > > > This
> > > > would
> > > > somewhat opens that up, as the stack guard gaps are usually
> > > > maintained
> > > > by userspace for new threads. It would have to be thought
> > > > through
> > > > if
> > > > these could still be enforced with checking at additional
> > > > spots.
> > >
> > > I would think the existing guard pages would already do that if a
> > > thread's shadow stack is contiguous with its own data stack.
> >
> > The difference is that the kernel provides the guard gaps, where
> > this
> > would rely on userspace to do it. It is not a showstopper either.
> >
> > I think my biggest question on this is how does it change the
> > capability for two threads to share a shadow stack. It might
> > require
> > some special rules around the syscall that writes restore tokens.
> > So
> > I'm not sure. It probably needs a POC.
>
> Why would they be sharing a shadow stack?
The guard gap was introduced originally based on a suggestion that
overflowing a shadow stack onto an adjacent shadow stack could cause
corruption that could be used by an attacker to work around the
protection. There wasn't any concrete demonstrated attacks or
suggestion that all the protection was moot.
But when we talk about capabilities for converting memory to shadow
stack with simple memory accesses, and syscalls that can write restore
token to shadow stacks, it's not immediately clear to me that it
wouldn't open up something like that. Like if two restore tokens were
written to a shadow stack, or two shadow stacks were adjacent with
normal memory between them that later got converted to shadow stack.
Those sorts of scenarios, but I won't lean on those specific examples.
Sorry for being hand wavy. It's just where I'm at, at this point.
>
> > > From the musl side, I have always looked at the entirely of
> > > shadow
> > > stack stuff with very heavy skepticism, and anything that breaks
> > > existing interface contracts, introduced places where apps can
> > > get
> > > auto-killed because a late resource allocation fails, or requires
> > > applications to code around the existence of something that
> > > should be
> > > an implementation detail, is a non-starter. To even consider
> > > shadow
> > > stack support, it must truely be fully non-breaking.
> >
> > The manual assembly stack switching and JIT code in the apps needs
> > to
> > be updated. I don't think there is a way around it.
>
> Indeed, I'm not talking about programs with JIT/manual stack-
> switching
> asm, just anything using existing APIs for control of stack --
> pthread_setstack, makecontext, sigaltstack, etc.
Then I think WRSS might fit your requirements better than what glibc
did. It was considered a reduced security mode that made libc's job
much easier and had better compatibility, but the last discussion was
to try to do it without WRSS.
>
> > I agree though that the late allocation failures are not great.
> > Mark is
> > working on clone3 support which should allow moving the shadow
> > stack
> > allocation to happen in userspace with the normal stack. Even for
> > riscv
> > though, doesn't it need to update a new register in stack
> > switching?
>
> If clone is called with signals masked, it's probably not necessary
> for the kernel to set the shadow stack register as part of clone3.
So you would want a mode of clone3 that basically leaves the shadow
stack bits alone? Mark was driving that effort, but it doesn't seem
horrible to me on first impression. If it would open up the possibility
of musl support.
>
> > BTW, x86 shadow stack has a mode where the shadow stack is writable
> > with a special instruction (WRSS). It enables the SSP to be set
> > arbitrarily by writing restore tokens. We discussed this as an
> > option
> > to make the existing longjmp() and signal stuff work more
> > transparently
> > for glibc.
> >
> >
> >
> > BTW, when I talk about "not supporting" I don't mean the app should
> > crash. I mean it should instead run normally, just without shadow
> > stack
> > enabled. Not sure if that was clear. Since shadow stack is not
> > essential for an application to function, it is only security
> > hardening
> > on top.
> >
> > Although determining if an application supports shadow stack has
> > turned
> > out to be difficult in practice. Handling dlopen() is especially
> > hard.
>
> One reasonable thing to do, that might be preferable to
> overengineered
> solutions, is to disable shadow-stack process-wide if an interface
> incompatible with it is used (sigaltstack, pthread_create with an
> attribute setup using pthread_attr_setstack, makecontext, etc.), as
> well as if an incompatible library is is dlopened.
I think it would be an interesting approach to determining
compatibility. On x86 there has been cases of binaries getting
mismarked as supporting shadow stack. So an automated way of filtering
some of those out would be very useful I think. I guess the dynamic
linker could determine this based on some list of functions?
The dlopen() bit gets complicated though. You need to disable shadow
stack for all threads, which presumably the kernel could be coaxed into
doing. But those threads might be using shadow stack instructions
(INCSSP, RSTORSSP, etc). These are a collection of instructions that
allow limited control of the SSP. When shadow stack gets disabled,
these suddenly turn into #UD generating instructions. So any other
threads executing those instructions when shadow stack got disabled
would be in for a nasty surprise.
Glibc's permissive mode (that disables shadow stack when dlopen()ing a
DSO that doesn't support shadow stack) is quite limited because of
this. There was a POC for working around it, but I'll stop there for
now, to not spam you with the details. I'm not sure of arm and risc-v
details on this specific corner, but for x86.
> This is much more
> acceptable than continuing to run with shadow stacks managed sloppily
> by the kernel and async killing the process on OOM, and is probably
> *more compatible* with apps than changing the minimum stack size
> requirements out from under them.
Yep.
>
> The place where it's really needed to be able to allocate the shadow
> stack synchronously under userspace control, in order to harden
> normal
> applications that aren't doing funny things, is in pthread_create
> without a caller-provided stack.
Yea most apps don't do anything too tricky. Mostly shadow stack "just
works". But it's no excuse to just crash for the others.
next prev parent reply other threads:[~2024-02-21 0:35 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-03 12:25 [PATCH v8 00/38] arm64/gcs: Provide support for GCS in userspace Mark Brown
2024-02-03 12:25 ` [PATCH v8 01/38] arm64/mm: Restructure arch_validate_flags() for extensibility Mark Brown
2024-02-03 12:25 ` [PATCH v8 02/38] prctl: arch-agnostic prctl for shadow stack Mark Brown
2024-02-03 12:25 ` [PATCH v8 03/38] mman: Add map_shadow_stack() flags Mark Brown
2024-02-03 12:25 ` [PATCH v8 04/38] arm64: Document boot requirements for Guarded Control Stacks Mark Brown
2024-02-03 12:25 ` [PATCH v8 05/38] arm64/gcs: Document the ABI " Mark Brown
2024-02-03 12:25 ` [PATCH v8 06/38] arm64/sysreg: Add definitions for architected GCS caps Mark Brown
2024-02-03 12:25 ` [PATCH v8 07/38] arm64/gcs: Add manual encodings of GCS instructions Mark Brown
2024-02-03 12:25 ` [PATCH v8 08/38] arm64/gcs: Provide put_user_gcs() Mark Brown
2024-02-03 12:25 ` [PATCH v8 09/38] arm64/cpufeature: Runtime detection of Guarded Control Stack (GCS) Mark Brown
2024-02-03 12:25 ` [PATCH v8 10/38] arm64/mm: Allocate PIE slots for EL0 guarded control stack Mark Brown
2024-02-03 12:25 ` [PATCH v8 11/38] mm: Define VM_SHADOW_STACK for arm64 when we support GCS Mark Brown
2024-02-03 12:25 ` [PATCH v8 12/38] arm64/mm: Map pages for guarded control stack Mark Brown
2024-02-03 12:25 ` [PATCH v8 13/38] KVM: arm64: Manage GCS registers for guests Mark Brown
2024-02-05 9:46 ` Marc Zyngier
2024-02-05 12:35 ` Mark Brown
2024-02-05 15:34 ` Marc Zyngier
2024-02-05 16:58 ` Mark Brown
2024-02-03 12:25 ` [PATCH v8 14/38] arm64/gcs: Allow GCS usage at EL0 and EL1 Mark Brown
2024-02-03 12:25 ` [PATCH v8 15/38] arm64/idreg: Add overrride for GCS Mark Brown
2024-02-03 12:25 ` [PATCH v8 16/38] arm64/hwcap: Add hwcap " Mark Brown
2024-02-03 12:25 ` [PATCH v8 17/38] arm64/traps: Handle GCS exceptions Mark Brown
2024-02-03 12:25 ` [PATCH v8 18/38] arm64/mm: Handle GCS data aborts Mark Brown
2024-02-03 12:25 ` [PATCH v8 19/38] arm64/gcs: Context switch GCS state for EL0 Mark Brown
2024-02-03 12:25 ` [PATCH v8 20/38] arm64/gcs: Ensure that new threads have a GCS Mark Brown
2024-02-20 2:02 ` Thiago Jung Bauermann
2024-02-21 18:16 ` Mark Brown
2024-02-03 12:25 ` [PATCH v8 21/38] arm64/gcs: Implement shadow stack prctl() interface Mark Brown
2024-02-03 12:25 ` [PATCH v8 22/38] arm64/mm: Implement map_shadow_stack() Mark Brown
2024-02-03 12:25 ` [PATCH v8 23/38] arm64/signal: Set up and restore the GCS context for signal handlers Mark Brown
2024-02-20 2:03 ` Thiago Jung Bauermann
2024-02-03 12:25 ` [PATCH v8 24/38] arm64/signal: Expose GCS state in signal frames Mark Brown
2024-02-03 12:25 ` [PATCH v8 25/38] arm64/ptrace: Expose GCS via ptrace and core files Mark Brown
2024-02-03 12:25 ` [PATCH v8 26/38] arm64: Add Kconfig for Guarded Control Stack (GCS) Mark Brown
2024-02-03 12:25 ` [PATCH v8 27/38] kselftest/arm64: Verify the GCS hwcap Mark Brown
2024-02-03 12:25 ` [PATCH v8 28/38] kselftest/arm64: Add GCS as a detected feature in the signal tests Mark Brown
2024-02-03 12:25 ` [PATCH v8 29/38] kselftest/arm64: Add framework support for GCS to signal handling tests Mark Brown
2024-02-03 12:25 ` [PATCH v8 30/38] kselftest/arm64: Allow signals tests to specify an expected si_code Mark Brown
2024-02-03 12:25 ` [PATCH v8 31/38] kselftest/arm64: Always run signals tests with GCS enabled Mark Brown
2024-02-03 12:25 ` [PATCH v8 32/38] kselftest/arm64: Add very basic GCS test program Mark Brown
2024-02-03 12:25 ` [PATCH v8 33/38] kselftest/arm64: Add a GCS test program built with the system libc Mark Brown
2024-02-20 2:15 ` Thiago Jung Bauermann
2024-02-21 19:20 ` Mark Brown
2024-02-22 19:11 ` Mark Brown
2024-02-23 2:24 ` Thiago Jung Bauermann
2024-02-27 16:14 ` Mark Brown
2024-02-27 19:08 ` Thiago Jung Bauermann
2024-02-29 21:45 ` Mark Brown
2024-02-29 22:13 ` Thiago Jung Bauermann
2024-02-03 12:26 ` [PATCH v8 34/38] kselftest/arm64: Add test coverage for GCS mode locking Mark Brown
2024-02-03 12:26 ` [PATCH v8 35/38] selftests/arm64: Add GCS signal tests Mark Brown
2024-02-20 2:17 ` Thiago Jung Bauermann
2024-02-03 12:26 ` [PATCH v8 36/38] kselftest/arm64: Add a GCS stress test Mark Brown
2024-02-03 12:26 ` [PATCH v8 37/38] kselftest/arm64: Enable GCS for the FP stress tests Mark Brown
2024-02-03 12:26 ` [PATCH v8 38/38] kselftest: Provide shadow stack enable helpers for arm64 Mark Brown
2024-02-20 2:00 ` [PATCH v8 00/38] arm64/gcs: Provide support for GCS in userspace Thiago Jung Bauermann
2024-02-20 16:36 ` Stefan O'Rear
2024-02-20 18:41 ` Edgecombe, Rick P
2024-02-20 18:57 ` [musl] " Rich Felker
2024-02-20 23:30 ` Edgecombe, Rick P
2024-02-20 23:54 ` dalias
2024-02-21 0:35 ` Edgecombe, Rick P [this message]
2024-02-21 0:44 ` Mark Brown
2024-02-21 1:27 ` dalias
2024-02-21 2:11 ` Edgecombe, Rick P
2024-02-21 4:18 ` Edgecombe, Rick P
2024-02-21 13:53 ` Mark Brown
2024-02-21 14:58 ` dalias
2024-02-21 17:36 ` Mark Brown
2024-02-21 17:57 ` dalias
2024-02-21 18:12 ` Edgecombe, Rick P
2024-02-21 18:30 ` dalias
2024-02-21 18:53 ` Edgecombe, Rick P
2024-02-21 19:06 ` dalias
2024-02-21 19:22 ` Edgecombe, Rick P
2024-02-21 20:18 ` H.J. Lu
2024-02-21 20:25 ` H.J. Lu
2024-02-21 21:12 ` H.J. Lu
2024-02-21 20:18 ` dalias
2024-02-22 13:57 ` Mark Brown
2024-02-21 18:32 ` Mark Brown
2024-02-21 19:10 ` dalias
2024-03-02 14:57 ` Szabolcs Nagy
2024-03-02 15:05 ` H.J. Lu
2024-03-14 14:03 ` Mark Brown
2024-02-20 23:59 ` Stefan O'Rear
2024-02-21 0:40 ` Mark Brown
2024-02-21 4:30 ` Edgecombe, Rick P
2024-02-20 20:14 ` Mark Brown
2024-02-20 23:30 ` Edgecombe, Rick P
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a57d6c7eada4b9a7c35addbc8556f5b53a0c3e6f.camel@intel.com \
--to=rick.p.edgecombe@intel.com \
--cc=Szabolcs.Nagy@arm.com \
--cc=akpm@linux-foundation.org \
--cc=aou@eecs.berkeley.edu \
--cc=ardb@kernel.org \
--cc=arnd@arndb.de \
--cc=brauner@kernel.org \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=dalias@libc.org \
--cc=debug@rivosinc.com \
--cc=ebiederm@xmission.com \
--cc=fweimer@redhat.com \
--cc=hjl.tools@gmail.com \
--cc=james.morse@arm.com \
--cc=keescook@chromium.org \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=maz@kernel.org \
--cc=musl@lists.openwall.com \
--cc=oleg@redhat.com \
--cc=oliver.upton@linux.dev \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=shuah@kernel.org \
--cc=sorear@fastmail.com \
--cc=suzuki.poulose@arm.com \
--cc=thiago.bauermann@linaro.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).