Linux-arch Archive mirror
 help / color / mirror / Atom feed
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
To: <linux-pm@vger.kernel.org>, <loongarch@lists.linux.dev>,
	<linux-acpi@vger.kernel.org>, <linux-arch@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>,
	<linux-arm-kernel@lists.infradead.org>, <kvmarm@lists.linux.dev>,
	<x86@kernel.org>, Russell King <linux@armlinux.org.uk>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Miguel Luis <miguel.luis@oracle.com>,
	James Morse <james.morse@arm.com>,
	Salil Mehta <salil.mehta@huawei.com>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>
Cc: <linuxarm@huawei.com>, <justin.he@arm.com>, <jianyong.wu@arm.com>
Subject: [PATCH v5 17/18] arm64: document virtual CPU hotplug's expectations
Date: Fri, 12 Apr 2024 15:37:18 +0100	[thread overview]
Message-ID: <20240412143719.11398-18-Jonathan.Cameron@huawei.com> (raw)
In-Reply-To: <20240412143719.11398-1-Jonathan.Cameron@huawei.com>

From: James Morse <james.morse@arm.com>

Add a description of physical and virtual CPU hotplug, explain the
differences and elaborate on what is required in ACPI for a working
virtual hotplug system.

Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

---
v5: No change.
---
 Documentation/arch/arm64/cpu-hotplug.rst | 79 ++++++++++++++++++++++++
 Documentation/arch/arm64/index.rst       |  1 +
 2 files changed, 80 insertions(+)

diff --git a/Documentation/arch/arm64/cpu-hotplug.rst b/Documentation/arch/arm64/cpu-hotplug.rst
new file mode 100644
index 000000000000..76ba8d932c72
--- /dev/null
+++ b/Documentation/arch/arm64/cpu-hotplug.rst
@@ -0,0 +1,79 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. _cpuhp_index:
+
+====================
+CPU Hotplug and ACPI
+====================
+
+CPU hotplug in the arm64 world is commonly used to describe the kernel taking
+CPUs online/offline using PSCI. This document is about ACPI firmware allowing
+CPUs that were not available during boot to be added to the system later.
+
+``possible`` and ``present`` refer to the state of the CPU as seen by linux.
+
+
+CPU Hotplug on physical systems - CPUs not present at boot
+----------------------------------------------------------
+
+Physical systems need to mark a CPU that is ``possible`` but not ``present`` as
+being ``present``. An example would be a dual socket machine, where the package
+in one of the sockets can be replaced while the system is running.
+
+This is not supported.
+
+In the arm64 world CPUs are not a single device but a slice of the system.
+There are no systems that support the physical addition (or removal) of CPUs
+while the system is running, and ACPI is not able to sufficiently describe
+them.
+
+e.g. New CPUs come with new caches, but the platform's cache toplogy is
+described in a static table, the PPTT. How caches are shared between CPUs is
+not discoverable, and must be described by firmware.
+
+e.g. The GIC redistributor for each CPU must be accessed by the driver during
+boot to discover the system wide supported features. ACPI's MADT GICC
+structures can describe a redistributor associated with a disabled CPU, but
+can't describe whether the redistributor is accessible, only that it is not
+'always on'.
+
+arm64's ACPI tables assume that everything described is ``present``.
+
+
+CPU Hotplug on virtual systems - CPUs not enabled at boot
+---------------------------------------------------------
+
+Virtual systems have the advantage that all the properties the system will
+ever have can be described at boot. There are no power-domain considerations
+as such devices are emulated.
+
+CPU Hotplug on virtual systems is supported. It is distinct from physical
+CPU Hotplug as all resources are described as ``present``, but CPUs may be
+marked as disabled by firmware. Only the CPU's online/offline behaviour is
+influenced by firmware. An example is where a virtual machine boots with a
+single CPU, and additional CPUs are added once a cloud orchestrator deploys
+the workload.
+
+For a virtual machine, the VMM (e.g. Qemu) plays the part of firmware.
+
+Virtual hotplug is implemented as a firmware policy affecting which CPUs can be
+brought online. Firmware can enforce its policy via PSCI's return codes. e.g.
+``DENIED``.
+
+The ACPI tables must describe all the resources of the virtual machine. CPUs
+that firmware wishes to disable either from boot (or later) should not be
+``enabled`` in the MADT GICC structures, but should have the ``online capable``
+bit set, to indicate they can be enabled later. The boot CPU must be marked as
+``enabled``.  The 'always on' GICR structure must be used to describe the
+redistributors.
+
+CPUs described as ``online capable`` but not ``enabled`` can be set to enabled
+by the DSDT's Processor object's _STA method. On virtual systems the _STA method
+must always report the CPU as ``present``. Changes to the firmware policy can
+be notified to the OS via device-check or eject-request.
+
+CPUs described as ``enabled`` in the static table, should not have their _STA
+modified dynamically by firmware. Soft-restart features such as kexec will
+re-read the static properties of the system from these static tables, and
+may malfunction if these no longer describe the running system. Linux will
+re-discover the dynamic properties of the system from the _STA method later
+during boot.
diff --git a/Documentation/arch/arm64/index.rst b/Documentation/arch/arm64/index.rst
index d08e924204bf..78544de0a8a9 100644
--- a/Documentation/arch/arm64/index.rst
+++ b/Documentation/arch/arm64/index.rst
@@ -13,6 +13,7 @@ ARM64 Architecture
     asymmetric-32bit
     booting
     cpu-feature-registers
+    cpu-hotplug
     elf_hwcaps
     hugetlbpage
     kdump
-- 
2.39.2


  parent reply	other threads:[~2024-04-12 14:46 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-12 14:37 [PATCH v5 00/18] ACPI/arm64: add support for virtual cpu hotplug Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 01/18] cpu: Do not warn on arch_register_cpu() returning -EPROBE_DEFER Jonathan Cameron
2024-04-12 17:42   ` Rafael J. Wysocki
2024-04-22  3:53   ` Gavin Shan
2024-04-12 14:37 ` [PATCH v5 02/18] ACPI: processor: Set the ACPI_COMPANION for the struct cpu instance Jonathan Cameron
2024-04-12 18:10   ` Rafael J. Wysocki
2024-04-15 15:48     ` Jonathan Cameron
2024-04-15 16:16       ` Rafael J. Wysocki
2024-04-15 16:19         ` Rafael J. Wysocki
2024-04-15 16:50           ` Jonathan Cameron
2024-04-15 17:34             ` Jonathan Cameron
2024-04-15 17:41               ` Rafael J. Wysocki
2024-04-16 17:35                 ` Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 03/18] ACPI: processor: Register deferred CPUs from acpi_processor_get_info() Jonathan Cameron
2024-04-12 18:30   ` Rafael J. Wysocki
2024-04-12 20:16     ` Russell King (Oracle)
2024-04-12 20:54       ` Thomas Gleixner
2024-04-12 21:52         ` Russell King (Oracle)
2024-04-12 23:23           ` Thomas Gleixner
2024-04-15  8:45             ` Jonathan Cameron
2024-04-15  9:16               ` Jonathan Cameron
2024-04-15  9:31                 ` Jonathan Cameron
2024-04-15 11:57                 ` Jonathan Cameron
2024-04-15 11:37               ` Rafael J. Wysocki
2024-04-15 11:56                 ` Jonathan Cameron
2024-04-15 12:04                   ` Rafael J. Wysocki
2024-04-15 12:23                     ` Jonathan Cameron
2024-04-16 17:41                       ` Jonathan Cameron
2024-04-16 19:02                         ` Rafael J. Wysocki
2024-04-17 10:39                           ` Jonathan Cameron
2024-04-15 12:37                     ` Salil Mehta
2024-04-15 12:41                       ` Rafael J. Wysocki
2024-04-15 11:51         ` Salil Mehta
2024-04-15 12:51           ` Rafael J. Wysocki
2024-04-15 15:31             ` Salil Mehta
2024-04-15 16:38               ` Rafael J. Wysocki
2024-04-17 15:01                 ` Salil Mehta
2024-04-17 16:19                   ` Rafael J. Wysocki
2024-04-15 10:52     ` Jonathan Cameron
2024-04-15 11:11       ` Jonathan Cameron
2024-04-15 11:52       ` Rafael J. Wysocki
2024-04-15 11:07     ` Salil Mehta
2024-04-16 14:00   ` Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 04/18] ACPI: Rename acpi_processor_hotadd_init and remove pre-processor guards Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 05/18] ACPI: utils: Add an acpi_sta_enabled() helper and use it in acpi_processor_make_present() Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 06/18] ACPI: scan: Add parameter to allow defering some actions in acpi_scan_check_and_detach Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 07/18] ACPI: Add post_eject to struct acpi_scan_handler for cpu hotplug Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 08/18] ACPI: convert acpi_processor_post_eject() to use IS_ENABLED() Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 09/18] ACPI: Check _STA present bit before making CPUs not present Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 10/18] ACPI: Warn when the present bit changes but the feature is not enabled Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 11/18] arm64: acpi: Move get_cpu_for_acpi_id() to a header Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 12/18] irqchip/gic-v3: Don't return errors from gic_acpi_match_gicc() Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 13/18] irqchip/gic-v3: Add support for ACPI's disabled but 'online capable' CPUs Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 14/18] arm64: psci: Ignore DENIED CPUs Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 15/18] arm64: arch_register_cpu() variant to allow checking of ACPI _STA Jonathan Cameron
2024-04-12 14:37 ` [PATCH v5 16/18] ACPI: add support to (un)register CPUs based on the _STA enabled bit Jonathan Cameron
2024-04-12 14:37 ` Jonathan Cameron [this message]
2024-04-12 14:37 ` [PATCH v5 18/18] cpumask: Add enabled cpumask for present CPUs that can be brought online Jonathan Cameron

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240412143719.11398-18-Jonathan.Cameron@huawei.com \
    --to=jonathan.cameron@huawei.com \
    --cc=catalin.marinas@arm.com \
    --cc=james.morse@arm.com \
    --cc=jean-philippe@linaro.org \
    --cc=jianyong.wu@arm.com \
    --cc=justin.he@arm.com \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-acpi@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=linux@armlinux.org.uk \
    --cc=linuxarm@huawei.com \
    --cc=loongarch@lists.linux.dev \
    --cc=miguel.luis@oracle.com \
    --cc=rafael@kernel.org \
    --cc=salil.mehta@huawei.com \
    --cc=will@kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).