From: Feng Wu <feng.wu@intel.com>
To: xen-devel@lists.xen.org
Cc: kevin.tian@intel.com, keir@xen.org, george.dunlap@eu.citrix.com,
andrew.cooper3@citrix.com, jbeulich@suse.com,
yang.z.zhang@intel.com, feng.wu@intel.com
Subject: [v3 11/15] Update IRTE according to guest interrupt config changes
Date: Wed, 24 Jun 2015 13:18:25 +0800 [thread overview]
Message-ID: <1435123109-10481-12-git-send-email-feng.wu@intel.com> (raw)
In-Reply-To: <1435123109-10481-1-git-send-email-feng.wu@intel.com>
When guest changes its interrupt configuration (such as, vector, etc.)
for direct-assigned devices, we need to update the associated IRTE
with the new guest vector, so external interrupts from the assigned
devices can be injected to guests without VM-Exit.
For lowest-priority interrupts, we use vector-hashing mechamisn to find
the destination vCPU. This follows the hardware behavior, since modern
Intel CPUs use vector hashing to handle the lowest-priority interrupt.
For multicast/broadcast vCPU, we cannot handle it via interrupt posting,
still use interrupt remapping.
Signed-off-by: Feng Wu <feng.wu@intel.com>
---
v3:
- Use bitmap to store the all the possible destination vCPUs of an
interrupt, then trying to find the right destination from the bitmap
- Typo and some small changes
xen/drivers/passthrough/io.c | 96 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 95 insertions(+), 1 deletion(-)
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index 9b77334..18e24e1 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -26,6 +26,7 @@
#include <asm/hvm/iommu.h>
#include <asm/hvm/support.h>
#include <xen/hvm/irq.h>
+#include <asm/io_apic.h>
static DEFINE_PER_CPU(struct list_head, dpci_list);
@@ -199,6 +200,78 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci)
xfree(dpci);
}
+/*
+ * The purpose of this routine is to find the right destination vCPU for
+ * an interrupt which will be delivered by VT-d posted-interrupt. There
+ * are several cases as below:
+ *
+ * - For lowest-priority interrupts, we find the destination vCPU from the
+ * guest vector using vector-hashing mechanism and return true. This follows
+ * the hardware behavior, since modern Intel CPUs use vector hashing to
+ * handle the lowest-priority interrupt.
+ * - Otherwise, for single destination interrupt, it is straightforward to
+ * find the destination vCPU and return true.
+ * - For multicast/broadcast vCPU, we cannot handle it via interrupt posting,
+ * so return false.
+ *
+ * Here is the details about the vector-hashing mechanism:
+ * 1. For lowest-priority interrupts, store all the possible destination
+ * vCPUs in an array.
+ * 2. Use "gvec % max number of destination vCPUs" to find the right
+ * destination vCPU in the array for the lowest-priority interrupt.
+ */
+static struct vcpu *pi_find_dest_vcpu(struct domain *d, uint8_t dest_id,
+ uint8_t dest_mode, uint8_t delivery_mode,
+ uint8_t gvec)
+{
+ unsigned long *dest_vcpu_bitmap = NULL;
+ unsigned int dest_vcpu_num = 0, idx = 0;
+ int size = (d->max_vcpus + BITS_PER_LONG - 1) / BITS_PER_LONG;
+ struct vcpu *v, *dest = NULL;
+ int i;
+
+ dest_vcpu_bitmap = xzalloc_array(unsigned long, size);
+ if ( !dest_vcpu_bitmap )
+ {
+ dprintk(XENLOG_G_INFO,
+ "dom%d: failed to allocate memory\n", d->domain_id);
+ return NULL;
+ }
+
+ for_each_vcpu ( d, v )
+ {
+ if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0,
+ dest_id, dest_mode) )
+ continue;
+
+ __set_bit(v->vcpu_id, dest_vcpu_bitmap);
+ dest_vcpu_num++;
+ }
+
+ if ( delivery_mode == dest_LowestPrio )
+ {
+ if ( dest_vcpu_num != 0 )
+ {
+ for ( i = 0; i <= gvec % dest_vcpu_num; i++)
+ idx = find_next_bit(dest_vcpu_bitmap, d->max_vcpus, idx) + 1;
+ idx--;
+
+ BUG_ON(idx >= d->max_vcpus || idx < 0);
+ dest = d->vcpu[idx];
+ }
+ }
+ else if ( dest_vcpu_num == 1 )
+ {
+ idx = find_first_bit(dest_vcpu_bitmap, d->max_vcpus);
+ BUG_ON(idx >= d->max_vcpus || idx < 0);
+ dest = d->vcpu[idx];
+ }
+
+ xfree(dest_vcpu_bitmap);
+
+ return dest;
+}
+
int pt_irq_create_bind(
struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind)
{
@@ -257,7 +330,7 @@ int pt_irq_create_bind(
{
case PT_IRQ_TYPE_MSI:
{
- uint8_t dest, dest_mode;
+ uint8_t dest, dest_mode, delivery_mode;
int dest_vcpu_id;
if ( !(pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) )
@@ -330,11 +403,32 @@ int pt_irq_create_bind(
/* Calculate dest_vcpu_id for MSI-type pirq migration. */
dest = pirq_dpci->gmsi.gflags & VMSI_DEST_ID_MASK;
dest_mode = !!(pirq_dpci->gmsi.gflags & VMSI_DM_MASK);
+ delivery_mode = (pirq_dpci->gmsi.gflags >> GFLAGS_SHIFT_DELIV_MODE) &
+ VMSI_DELIV_MASK;
dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id;
spin_unlock(&d->event_lock);
if ( dest_vcpu_id >= 0 )
hvm_migrate_pirqs(d->vcpu[dest_vcpu_id]);
+
+ /* Use interrupt posting if it is supported */
+ if ( iommu_intpost )
+ {
+ struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode,
+ delivery_mode, pirq_dpci->gmsi.gvec);
+
+ if ( !vcpu )
+ dprintk(XENLOG_G_WARNING,
+ "dom%u: failed to find the dest vCPU for PI, guest "
+ "vector:0x%x use software way to deliver the "
+ " interrupts.\n", d->domain_id, pirq_dpci->gmsi.gvec);
+ else if ( pi_update_irte( vcpu, info, pirq_dpci->gmsi.gvec ) != 0 )
+ dprintk(XENLOG_G_WARNING,
+ "%pv: failed to update PI IRTE, guest vector:0x%x "
+ "use software way to deliver the interrupts.\n",
+ vcpu, pirq_dpci->gmsi.gvec);
+ }
+
break;
}
--
2.1.0
next prev parent reply other threads:[~2015-06-24 5:18 UTC|newest]
Thread overview: 157+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-24 5:18 [v3 00/15] Add VT-d Posted-Interrupts support Feng Wu
2015-06-24 5:18 ` [v3 01/15] Vt-d Posted-intterrupt (PI) design Feng Wu
2015-06-24 6:15 ` Meng Xu
2015-06-24 6:19 ` Wu, Feng
2015-07-08 7:21 ` Tian, Kevin
2015-07-08 7:29 ` Wu, Feng
2015-06-24 5:18 ` [v3 02/15] Add helper macro for X86_FEATURE_CX16 feature detection Feng Wu
2015-06-24 17:31 ` Andrew Cooper
2015-07-08 7:23 ` Tian, Kevin
2015-06-24 5:18 ` [v3 03/15] Add cmpxchg16b support for x86-64 Feng Wu
2015-06-24 18:35 ` Andrew Cooper
2015-07-08 7:06 ` Wu, Feng
2015-07-08 8:12 ` Jan Beulich
2015-07-08 8:33 ` Wu, Feng
2015-07-08 8:43 ` Jan Beulich
2015-07-08 8:50 ` Wu, Feng
2015-07-08 8:50 ` Andrew Cooper
2015-07-10 12:57 ` Jan Beulich
2015-06-24 5:18 ` [v3 04/15] iommu: Add iommu_intpost to control VT-d Posted-Interrupts feature Feng Wu
2015-06-25 9:06 ` Andrew Cooper
2015-06-25 9:47 ` Wu, Feng
2015-06-25 10:16 ` Andrew Cooper
2015-06-25 12:47 ` Wu, Feng
2015-07-08 7:30 ` Tian, Kevin
2015-06-24 5:18 ` [v3 05/15] vt-d: VT-d Posted-Interrupts feature detection Feng Wu
2015-06-25 10:21 ` Andrew Cooper
2015-06-25 13:02 ` Wu, Feng
2015-07-08 7:32 ` Tian, Kevin
2015-07-08 8:00 ` Wu, Feng
2015-06-24 5:18 ` [v3 06/15] vmx: Extend struct pi_desc to support VT-d Posted-Interrupts Feng Wu
2015-06-29 15:04 ` Andrew Cooper
2015-07-08 7:48 ` Tian, Kevin
2015-07-10 13:08 ` Jan Beulich
2015-07-15 2:40 ` Wu, Feng
2015-07-15 8:20 ` Jan Beulich
2015-07-15 8:26 ` Wu, Feng
2015-07-15 8:36 ` Jan Beulich
2015-07-15 8:43 ` Wu, Feng
2015-07-15 9:28 ` Jan Beulich
2015-07-15 9:30 ` Wu, Feng
2015-07-15 3:13 ` Wu, Feng
2015-06-24 5:18 ` [v3 07/15] vmx: Initialize VT-d Posted-Interrupts Descriptor Feng Wu
2015-06-29 15:32 ` Andrew Cooper
2015-06-30 1:46 ` Wu, Feng
2015-06-30 2:32 ` Dario Faggioli
2015-07-08 7:53 ` Tian, Kevin
2015-06-24 5:18 ` [v3 08/15] Suppress posting interrupts when 'SN' is set Feng Wu
2015-06-29 15:41 ` Andrew Cooper
2015-06-30 1:48 ` Wu, Feng
2015-07-08 9:06 ` Tian, Kevin
2015-07-08 10:11 ` Wu, Feng
2015-07-08 11:31 ` Tian, Kevin
2015-07-08 11:58 ` Wu, Feng
2015-07-10 13:20 ` Jan Beulich
2015-06-24 5:18 ` [v3 09/15] vt-d: Extend struct iremap_entry to support VT-d Posted-Interrupts Feng Wu
2015-06-29 16:04 ` Andrew Cooper
2015-06-30 1:52 ` Wu, Feng
2015-07-08 9:10 ` Tian, Kevin
2015-07-10 13:27 ` Jan Beulich
2015-06-24 5:18 ` [v3 10/15] vt-d: Add API to update IRTE when VT-d PI is used Feng Wu
2015-06-29 16:22 ` Andrew Cooper
2015-07-08 9:59 ` Tian, Kevin
2015-07-08 10:12 ` Wu, Feng
2015-07-10 14:01 ` Jan Beulich
2015-07-15 6:04 ` Wu, Feng
2015-07-15 8:24 ` Jan Beulich
2015-07-15 8:38 ` Wu, Feng
2015-07-15 8:46 ` Jan Beulich
2015-07-15 8:55 ` Wu, Feng
2015-07-15 9:32 ` Jan Beulich
2015-06-24 5:18 ` Feng Wu [this message]
2015-06-29 16:46 ` [v3 11/15] Update IRTE according to guest interrupt config changes Andrew Cooper
2015-07-08 10:22 ` Tian, Kevin
2015-07-08 10:31 ` Wu, Feng
2015-07-08 11:46 ` Tian, Kevin
2015-07-08 11:52 ` Wu, Feng
2015-07-08 11:54 ` Tian, Kevin
2015-07-10 14:23 ` Jan Beulich
2015-06-24 5:18 ` [v3 12/15] vmx: posted-interrupt handling when vCPU is blocked Feng Wu
2015-06-29 17:07 ` Andrew Cooper
2015-07-08 10:36 ` Wu, Feng
2015-07-08 10:48 ` Jan Beulich
[not found] ` <559181F9.6020106@citrix.com>
2015-06-30 2:51 ` Fwd: " Dario Faggioli
2015-06-30 2:59 ` Wu, Feng
2015-06-30 9:46 ` Dario Faggioli
2015-06-30 10:11 ` Andrew Cooper
2015-07-01 13:26 ` Dario Faggioli
2015-07-02 4:27 ` Wu, Feng
2015-07-02 8:30 ` Dario Faggioli
2015-07-02 8:58 ` Wu, Feng
2015-07-02 10:09 ` Dario Faggioli
2015-07-02 10:41 ` Wu, Feng
2015-07-02 10:30 ` Andrew Cooper
2015-07-02 10:56 ` Wu, Feng
2015-07-02 12:04 ` Dario Faggioli
2015-07-02 12:10 ` Wu, Feng
2015-07-02 12:16 ` Andrew Cooper
2015-07-02 12:38 ` Dario Faggioli
2015-07-02 12:59 ` Andrew Cooper
2015-07-03 1:33 ` Wu, Feng
2015-07-02 4:25 ` Wu, Feng
2015-07-08 11:00 ` Tian, Kevin
2015-07-08 11:02 ` Wu, Feng
2015-07-08 12:46 ` Jan Beulich
2015-07-08 13:09 ` Andrew Cooper
2015-07-08 22:49 ` Tian, Kevin
2015-07-09 7:25 ` Jan Beulich
2015-07-10 6:21 ` Wu, Feng
2015-07-10 6:32 ` Jan Beulich
2015-07-10 7:29 ` Wu, Feng
2015-07-10 8:49 ` Jan Beulich
2015-07-10 8:57 ` Wu, Feng
2015-07-08 22:31 ` Tian, Kevin
2015-06-24 5:18 ` [v3 13/15] vmx: Properly handle notification event when vCPU is running Feng Wu
2015-07-08 11:03 ` Tian, Kevin
2015-07-10 14:40 ` Jan Beulich
2015-06-24 5:18 ` [v3 14/15] Update Posted-Interrupts Descriptor during vCPU scheduling Feng Wu
[not found] ` <55918214.4030102@citrix.com>
2015-06-30 2:58 ` Fwd: " Dario Faggioli
2015-07-02 4:32 ` Wu, Feng
2015-07-02 4:34 ` Wu, Feng
2015-07-02 8:20 ` Dario Faggioli
2015-07-09 3:09 ` Wu, Feng
2015-07-09 8:18 ` Dario Faggioli
2015-07-09 11:19 ` George Dunlap
2015-07-09 11:29 ` George Dunlap
2015-07-09 11:38 ` Wu, Feng
2015-07-09 12:42 ` Dario Faggioli
2015-07-10 0:07 ` Wu, Feng
2015-07-10 12:40 ` Dario Faggioli
2015-07-10 13:47 ` Konrad Rzeszutek Wilk
2015-07-10 13:59 ` Dario Faggioli
2015-07-09 12:53 ` George Dunlap
2015-07-09 13:44 ` Jan Beulich
2015-07-09 14:18 ` Dario Faggioli
2015-07-09 14:27 ` George Dunlap
2015-07-09 14:47 ` Dario Faggioli
2015-07-10 5:59 ` Wu, Feng
2015-07-10 6:22 ` Jan Beulich
2015-07-10 11:05 ` Dario Faggioli
2015-07-14 5:44 ` Wu, Feng
2015-07-14 14:08 ` Wu, Feng
2015-07-14 14:54 ` Jan Beulich
2015-07-14 15:20 ` Dario Faggioli
2015-07-14 16:41 ` George Dunlap
2015-07-14 16:02 ` Dario Faggioli
2015-07-15 0:54 ` Wu, Feng
2015-07-17 7:46 ` Wu, Feng
2015-07-17 10:13 ` Dario Faggioli
2015-07-17 22:57 ` Wu, Feng
2015-07-18 13:43 ` Dario Faggioli
2015-07-10 0:15 ` Wu, Feng
2015-07-08 11:24 ` Tian, Kevin
2015-07-10 14:48 ` Jan Beulich
2015-06-24 5:18 ` [v3 15/15] Add a command line parameter for VT-d posted-interrupts Feng Wu
2015-07-08 11:25 ` Tian, Kevin
-- strict thread matches above, loose matches on Subject: below --
2015-07-08 11:05 [v3 11/15] Update IRTE according to guest interrupt config changes Wu, Feng
2015-07-08 11:47 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1435123109-10481-12-git-send-email-feng.wu@intel.com \
--to=feng.wu@intel.com \
--cc=andrew.cooper3@citrix.com \
--cc=george.dunlap@eu.citrix.com \
--cc=jbeulich@suse.com \
--cc=keir@xen.org \
--cc=kevin.tian@intel.com \
--cc=xen-devel@lists.xen.org \
--cc=yang.z.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.