From: Ard Biesheuvel <ardb+git@google.com>
To: linux-kernel@vger.kernel.org
Cc: Ard Biesheuvel <ardb@kernel.org>,
Kevin Loughlin <kevinloughlin@google.com>,
Tom Lendacky <thomas.lendacky@amd.com>,
Dionna Glaze <dionnaglaze@google.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
Andy Lutomirski <luto@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
Nathan Chancellor <nathan@kernel.org>,
Nick Desaulniers <ndesaulniers@google.com>,
Justin Stitt <justinstitt@google.com>,
Brian Gerst <brgerst@gmail.com>,
linux-arch@vger.kernel.org, llvm@lists.linux.dev
Subject: [PATCH v2 05/17] x86/startup_64: Simplify virtual switch on primary boot
Date: Thu, 25 Jan 2024 12:28:24 +0100 [thread overview]
Message-ID: <20240125112818.2016733-24-ardb+git@google.com> (raw)
In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com>
From: Ard Biesheuvel <ardb@kernel.org>
The secondary startup code is used on the primary boot path as well, but
in this case, the initial part runs from a 1:1 mapping, until an
explicit cross-jump is made to the kernel virtual mapping of the same
code.
On the secondary boot path, this jump is pointless as the code already
executes from the mapping targeted by the jump. So combine this
cross-jump with the jump from startup_64() into the common boot path.
This simplifies the execution flow, and clearly separates code that runs
from a 1:1 mapping from code that runs from the kernel virtual mapping.
Note that this requires a page table switch, so hoist the CR3 assignment
into startup_64() as well.
Given that the secondary startup code does not require a special
placement inside the executable, move it to the .text section.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/x86/kernel/head_64.S | 41 +++++++++-----------
1 file changed, 19 insertions(+), 22 deletions(-)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 2d361e0ac74e..399241dcdbb5 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -39,7 +39,6 @@ L4_START_KERNEL = l4_index(__START_KERNEL_map)
L3_START_KERNEL = pud_index(__START_KERNEL_map)
- .text
__HEAD
.code64
SYM_CODE_START_NOALIGN(startup_64)
@@ -128,9 +127,19 @@ SYM_CODE_START_NOALIGN(startup_64)
call sev_verify_cbit
#endif
- jmp 1f
+ /*
+ * Switch to early_top_pgt which still has the identity mappings
+ * present.
+ */
+ movq %rax, %cr3
+
+ /* Branch to the common startup code at its kernel virtual address */
+ movq $common_startup_64, %rax
+ ANNOTATE_RETPOLINE_SAFE
+ jmp *%rax
SYM_CODE_END(startup_64)
+ .text
SYM_CODE_START(secondary_startup_64)
UNWIND_HINT_END_OF_STACK
ANNOTATE_NOENDBR
@@ -176,8 +185,15 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL)
#ifdef CONFIG_AMD_MEM_ENCRYPT
addq sme_me_mask(%rip), %rax
#endif
+ /*
+ * Switch to the init_top_pgt here, away from the trampoline_pgd and
+ * unmap the identity mapped ranges.
+ */
+ movq %rax, %cr3
-1:
+SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL)
+ UNWIND_HINT_END_OF_STACK
+ ANNOTATE_NOENDBR // above
/*
* Define a mask of CR4 bits to preserve. PAE and LA57 cannot be
@@ -195,17 +211,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL)
*/
orl $X86_CR4_MCE, %edx
#endif
-
- /*
- * Switch to new page-table
- *
- * For the boot CPU this switches to early_top_pgt which still has the
- * identity mappings present. The secondary CPUs will switch to the
- * init_top_pgt here, away from the trampoline_pgd and unmap the
- * identity mapped ranges.
- */
- movq %rax, %cr3
-
/*
* Do a global TLB flush after the CR3 switch to make sure the TLB
* entries from the identity mapping are flushed.
@@ -216,14 +221,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL)
movq %rcx, %cr4
jc 0b
- /* Ensure I am executing from virtual addresses */
- movq $1f, %rax
- ANNOTATE_RETPOLINE_SAFE
- jmp *%rax
-1:
- UNWIND_HINT_END_OF_STACK
- ANNOTATE_NOENDBR // above
-
#ifdef CONFIG_SMP
/*
* For parallel boot, the APIC ID is read from the APIC, and then
--
2.43.0.429.g432eaa2c6b-goog
next prev parent reply other threads:[~2024-01-25 11:32 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-25 11:28 [PATCH v2 00/17] x86: Confine early 1:1 mapped startup code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 01/17] x86/startup_64: Drop long return to initial_code pointer Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 02/17] x86/startup_64: Simplify calculation of initial page table address Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 03/17] x86/startup_64: Simplify CR4 handling in startup code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 04/17] x86/startup_64: Drop global variables to keep track of LA57 state Ard Biesheuvel
2024-01-25 11:28 ` Ard Biesheuvel [this message]
2024-01-25 11:28 ` [PATCH v2 06/17] x86/head64: Replace pointer fixups with PIE codegen Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 07/17] x86/head64: Simplify GDT/IDT initialization code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 08/17] asm-generic: Add special .pi.text section for position independent code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 09/17] x86: Move return_thunk to __pitext section Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 10/17] x86/head64: Move early startup code into __pitext Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 11/17] modpost: Warn about calls from __pitext into other text sections Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 12/17] x86/coco: Make cc_set_mask() static inline Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 13/17] x86/sev: Make all code reachable from 1:1 mapping __pitext Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 14/17] x86/sev: Avoid WARN() in early code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 15/17] x86/sev: Use PIC codegen for early SEV startup code Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 16/17] x86/sev: Drop inline asm LEA instructions for RIP-relative references Ard Biesheuvel
2024-01-25 20:46 ` Kevin Loughlin
2024-01-25 23:24 ` Ard Biesheuvel
2024-01-25 11:28 ` [PATCH v2 17/17] x86/startup_64: Don't bother setting up GS before the kernel is mapped Ard Biesheuvel
2024-01-25 22:23 ` [PATCH v2 00/17] x86: Confine early 1:1 mapped startup code Nathan Chancellor
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240125112818.2016733-24-ardb+git@google.com \
--to=ardb+git@google.com \
--cc=ardb@kernel.org \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=brgerst@gmail.com \
--cc=dave.hansen@linux.intel.com \
--cc=dionnaglaze@google.com \
--cc=justinstitt@google.com \
--cc=kevinloughlin@google.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).