From: Peter Gonda <pgonda@google.com>
To: Ashish Kalra <Ashish.Kalra@amd.com>
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"H. Peter Anvin" <hpa@zytor.com>,
"Radim Krčmář" <rkrcmar@redhat.com>,
"Joerg Roedel" <joro@8bytes.org>, "Borislav Petkov" <bp@suse.de>,
"Lendacky, Thomas" <thomas.lendacky@amd.com>,
x86@kernel.org, "kvm list" <kvm@vger.kernel.org>,
linux-kernel@vger.kernel.org,
"Steve Rutherford" <srutherford@google.com>,
"Sean Christopherson" <seanjc@google.com>,
venu.busireddy@oracle.com, "Singh,
Brijesh" <brijesh.singh@amd.com>
Subject: Re: [PATCH v11 00/13] Add AMD SEV guest live migration support
Date: Mon, 5 Apr 2021 09:17:07 -0600 [thread overview]
Message-ID: <CAMkAt6oWF23YFiOGW_h+iyhjkaAp6uaMNjYKDXNxvNCWR=vyWw@mail.gmail.com> (raw)
In-Reply-To: <cover.1617302792.git.ashish.kalra@amd.com>
Could this patch set include support for the SEND_CANCEL command?
On Mon, Apr 5, 2021 at 8:20 AM Ashish Kalra <Ashish.Kalra@amd.com> wrote:
>
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> The series add support for AMD SEV guest live migration commands. To protect the
> confidentiality of an SEV protected guest memory while in transit we need to
> use the SEV commands defined in SEV API spec [1].
>
> SEV guest VMs have the concept of private and shared memory. Private memory
> is encrypted with the guest-specific key, while shared memory may be encrypted
> with hypervisor key. The commands provided by the SEV FW are meant to be used
> for the private memory only. The patch series introduces a new hypercall.
> The guest OS can use this hypercall to notify the page encryption status.
> If the page is encrypted with guest specific-key then we use SEV command during
> the migration. If page is not encrypted then fallback to default.
>
> The patch adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
> userspace exit functionality as a common interface from the guest back to the
> VMM and passing on the guest shared/unencrypted page information to the
> userspace VMM/Qemu. Qemu can consult this information during migration to know
> whether the page is encrypted.
>
> This section descibes how the SEV live migration feature is negotiated
> between the host and guest, the host indicates this feature support via
> KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
> sets a UEFI enviroment variable indicating OVMF support for live
> migration, the guest kernel also detects the host support for this
> feature via cpuid and in case of an EFI boot verifies if OVMF also
> supports this feature by getting the UEFI enviroment variable and if it
> set then enables live migration feature on host by writing to a custom
> MSR, if not booted under EFI, then it simply enables the feature by
> again writing to the custom MSR. The MSR is also handled by the
> userspace VMM/Qemu.
>
> A branch containing these patches is available here:
> https://github.com/AMDESE/linux/tree/sev-migration-v11
>
> [1] https://developer.amd.com/wp-content/resources/55766.PDF
>
> Changes since v10:
> - Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
> userspace exit functionality as a common interface from the guest back to the
> KVM and passing on the guest shared/unencrypted region information to the
> userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
> memory regions information anymore.
> - Remove implicit enabling of SEV live migration feature for an SEV
> guest, now this is explicitly in control of the userspace VMM/Qemu.
> - Custom MSR handling is also now moved into userspace VMM/Qemu.
> - As KVM does not maintain the guest shared memory region information
> anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
> regions without support from userspace VMM/Qemu.
>
> Changes since v9:
> - Transitioning from page encryption bitmap to the shared pages list
> to keep track of guest's shared/unencrypted memory regions.
> - Move back to marking the complete _bss_decrypted section as
> decrypted in the shared pages list.
> - Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
> for guest to query for host-side support for SEV live migration
> and to enable the SEV live migration feature, to avoid
> #ifdefs in code
> - Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
> - Invoke a new function handle_unencrypted_region() from
> sev_dbg_crypt() to bypass unencrypted guest memory regions.
>
> Changes since v8:
> - Rebasing to kvm next branch.
> - Fixed and added comments as per review feedback on v8 patches.
> - Removed implicitly enabling live migration for incoming VMs in
> in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
> - Adds support for bypassing unencrypted guest memory regions for
> DBG_DECRYPT API calls, guest memory region encryption status in
> sev_dbg_decrypt() is referenced using the page encryption bitmap.
>
> Changes since v7:
> - Removed the hypervisor specific hypercall/paravirt callback for
> SEV live migration and moved back to calling kvm_sev_hypercall3
> directly.
> - Fix build errors as
> Reported-by: kbuild test robot <lkp@intel.com>, specifically fixed
> build error when CONFIG_HYPERVISOR_GUEST=y and
> CONFIG_AMD_MEM_ENCRYPT=n.
> - Implicitly enabled live migration for incoming VM(s) to handle
> A->B->C->... VM migrations.
> - Fixed Documentation as per comments on v6 patches.
> - Fixed error return path in sev_send_update_data() as per comments
> on v6 patches.
>
> Changes since v6:
> - Rebasing to mainline and refactoring to the new split SVM
> infrastructre.
> - Move to static allocation of the unified Page Encryption bitmap
> instead of the dynamic resizing of the bitmap, the static allocation
> is done implicitly by extending kvm_arch_commit_memory_region() callack
> to add svm specific x86_ops which can read the userspace provided memory
> region/memslots and calculate the amount of guest RAM managed by the KVM
> and grow the bitmap.
> - Fixed KVM_SET_PAGE_ENC_BITMAP ioctl to set the whole bitmap instead
> of simply clearing specific bits.
> - Removed KVM_PAGE_ENC_BITMAP_RESET ioctl, which is now performed using
> KVM_SET_PAGE_ENC_BITMAP.
> - Extended guest support for enabling Live Migration feature by adding a
> check for UEFI environment variable indicating OVMF support for Live
> Migration feature and additionally checking for KVM capability for the
> same feature. If not booted under EFI, then we simply check for KVM
> capability.
> - Add hypervisor specific hypercall for SEV live migration by adding
> a new paravirt callback as part of x86_hyper_runtime.
> (x86 hypervisor specific runtime callbacks)
> - Moving MSR handling for MSR_KVM_SEV_LIVE_MIG_EN into svm/sev code
> and adding check for SEV live migration enabled by guest in the
> KVM_GET_PAGE_ENC_BITMAP ioctl.
> - Instead of the complete __bss_decrypted section, only specific variables
> such as hv_clock_boot and wall_clock are marked as decrypted in the
> page encryption bitmap
>
> Changes since v5:
> - Fix build errors as
> Reported-by: kbuild test robot <lkp@intel.com>
>
> Changes since v4:
> - Host support has been added to extend KVM capabilities/feature bits to
> include a new KVM_FEATURE_SEV_LIVE_MIGRATION, which the guest can
> query for host-side support for SEV live migration and a new custom MSR
> MSR_KVM_SEV_LIVE_MIG_EN is added for guest to enable the SEV live
> migration feature.
> - Ensure that _bss_decrypted section is marked as decrypted in the
> page encryption bitmap.
> - Fixing KVM_GET_PAGE_ENC_BITMAP ioctl to return the correct bitmap
> as per the number of pages being requested by the user. Ensure that
> we only copy bmap->num_pages bytes in the userspace buffer, if
> bmap->num_pages is not byte aligned we read the trailing bits
> from the userspace and copy those bits as is. This fixes guest
> page(s) corruption issues observed after migration completion.
> - Add kexec support for SEV Live Migration to reset the host's
> page encryption bitmap related to kernel specific page encryption
> status settings before we load a new kernel by kexec. We cannot
> reset the complete page encryption bitmap here as we need to
> retain the UEFI/OVMF firmware specific settings.
>
> Changes since v3:
> - Rebasing to mainline and testing.
> - Adding a new KVM_PAGE_ENC_BITMAP_RESET ioctl, which resets the
> page encryption bitmap on a guest reboot event.
> - Adding a more reliable sanity check for GPA range being passed to
> the hypercall to ensure that guest MMIO ranges are also marked
> in the page encryption bitmap.
>
> Changes since v2:
> - reset the page encryption bitmap on vcpu reboot
>
> Changes since v1:
> - Add support to share the page encryption between the source and target
> machine.
> - Fix review feedbacks from Tom Lendacky.
> - Add check to limit the session blob length.
> - Update KVM_GET_PAGE_ENC_BITMAP icotl to use the base_gfn instead of
> the memory slot when querying the bitmap.
>
> Ashish Kalra (5):
> KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall
> KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature &
> Custom MSR.
> EFI: Introduce the new AMD Memory Encryption GUID.
> x86/kvm: Add guest support for detecting and enabling SEV Live
> Migration feature.
> x86/kvm: Add kexec support for SEV Live Migration.
>
> Brijesh Singh (8):
> KVM: SVM: Add KVM_SEV SEND_START command
> KVM: SVM: Add KVM_SEND_UPDATE_DATA command
> KVM: SVM: Add KVM_SEV_SEND_FINISH command
> KVM: SVM: Add support for KVM_SEV_RECEIVE_START command
> KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command
> KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command
> KVM: x86: Add AMD SEV specific Hypercall3
> mm: x86: Invoke hypercall when page encryption status is changed
>
> .../virt/kvm/amd-memory-encryption.rst | 120 ++++
> Documentation/virt/kvm/api.rst | 18 +
> Documentation/virt/kvm/cpuid.rst | 5 +
> Documentation/virt/kvm/hypercalls.rst | 15 +
> Documentation/virt/kvm/msr.rst | 12 +
> arch/x86/include/asm/kvm_host.h | 2 +
> arch/x86/include/asm/kvm_para.h | 12 +
> arch/x86/include/asm/mem_encrypt.h | 8 +
> arch/x86/include/asm/paravirt.h | 10 +
> arch/x86/include/asm/paravirt_types.h | 2 +
> arch/x86/include/uapi/asm/kvm_para.h | 4 +
> arch/x86/kernel/kvm.c | 76 +++
> arch/x86/kernel/paravirt.c | 1 +
> arch/x86/kvm/cpuid.c | 3 +-
> arch/x86/kvm/svm/sev.c | 514 ++++++++++++++++++
> arch/x86/kvm/svm/svm.c | 24 +
> arch/x86/kvm/svm/svm.h | 2 +
> arch/x86/kvm/vmx/vmx.c | 1 +
> arch/x86/kvm/x86.c | 12 +
> arch/x86/mm/mem_encrypt.c | 98 +++-
> arch/x86/mm/pat/set_memory.c | 7 +
> include/linux/efi.h | 1 +
> include/linux/psp-sev.h | 8 +-
> include/uapi/linux/kvm.h | 47 ++
> include/uapi/linux/kvm_para.h | 1 +
> 25 files changed, 997 insertions(+), 6 deletions(-)
>
> --
> 2.17.1
>
next prev parent reply other threads:[~2021-04-05 15:17 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-05 14:20 [PATCH v11 00/13] Add AMD SEV guest live migration support Ashish Kalra
2021-04-05 14:21 ` [PATCH v11 01/13] KVM: SVM: Add KVM_SEV SEND_START command Ashish Kalra
2021-04-05 14:23 ` [PATCH v11 02/13] KVM: SVM: Add KVM_SEND_UPDATE_DATA command Ashish Kalra
2021-04-05 14:23 ` [PATCH v11 03/13] KVM: SVM: Add KVM_SEV_SEND_FINISH command Ashish Kalra
2021-04-05 14:24 ` [PATCH v11 04/13] KVM: SVM: Add support for KVM_SEV_RECEIVE_START command Ashish Kalra
2021-04-05 14:24 ` [PATCH v11 05/13] KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command Ashish Kalra
2021-04-05 14:25 ` [PATCH v11 06/13] KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command Ashish Kalra
2021-04-05 14:26 ` [PATCH v11 07/13] KVM: x86: Add AMD SEV specific Hypercall3 Ashish Kalra
2021-04-05 14:28 ` [PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall Ashish Kalra
2021-04-05 20:42 ` Steve Rutherford
2021-04-06 6:22 ` Ashish Kalra
2021-04-06 15:11 ` Sean Christopherson
2021-04-06 15:20 ` Sean Christopherson
2021-04-06 18:14 ` Ashish Kalra
2021-04-06 15:48 ` Sean Christopherson
2021-04-06 16:07 ` Ashish Kalra
2021-04-06 20:14 ` Steve Rutherford
2021-04-06 20:27 ` Sean Christopherson
2021-04-07 14:01 ` Ashish Kalra
2021-04-05 14:29 ` [PATCH v11 09/13] mm: x86: Invoke hypercall when page encryption status is changed Ashish Kalra
2021-04-05 14:30 ` [PATCH v11 10/13] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR Ashish Kalra
2021-04-06 1:39 ` Steve Rutherford
2021-04-06 13:26 ` Ashish Kalra
2021-04-06 13:47 ` Paolo Bonzini
2021-04-06 13:59 ` Ashish Kalra
2021-04-06 19:41 ` Steve Rutherford
2021-04-05 14:30 ` [PATCH v11 11/13] EFI: Introduce the new AMD Memory Encryption GUID Ashish Kalra
2021-04-05 14:31 ` [PATCH v11 12/13] x86/kvm: Add guest support for detecting and enabling SEV Live Migration feature Ashish Kalra
2021-04-05 14:35 ` [PATCH v11 13/13] x86/kvm: Add kexec support for SEV Live Migration Ashish Kalra
2021-04-05 15:17 ` Peter Gonda [this message]
2021-04-05 18:27 ` [PATCH v11 00/13] Add AMD SEV guest live migration support Steve Rutherford
2021-04-06 13:48 ` Paolo Bonzini
2021-04-06 1:43 ` Steve Rutherford
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMkAt6oWF23YFiOGW_h+iyhjkaAp6uaMNjYKDXNxvNCWR=vyWw@mail.gmail.com' \
--to=pgonda@google.com \
--cc=Ashish.Kalra@amd.com \
--cc=bp@suse.de \
--cc=brijesh.singh@amd.com \
--cc=hpa@zytor.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=seanjc@google.com \
--cc=srutherford@google.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=venu.busireddy@oracle.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).