From: "Wu, Feng" <feng.wu@intel.com>
To: "Tian, Kevin" <kevin.tian@intel.com>,
"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
Cc: "Wu, Feng" <feng.wu@intel.com>,
"george.dunlap@eu.citrix.com" <george.dunlap@eu.citrix.com>,
"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
"jbeulich@suse.com" <jbeulich@suse.com>,
"Zhang, Yang Z" <yang.z.zhang@intel.com>,
"keir@xen.org" <keir@xen.org>
Subject: Re: [v3 12/15] vmx: posted-interrupt handling when vCPU is blocked
Date: Wed, 8 Jul 2015 11:02:55 +0000 [thread overview]
Message-ID: <E959C4978C3B6342920538CF579893F0025F8E01@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D1262B8C4D@SHSMSX101.ccr.corp.intel.com>
> -----Original Message-----
> From: Tian, Kevin
> Sent: Wednesday, July 08, 2015 7:00 PM
> To: Wu, Feng; xen-devel@lists.xen.org
> Cc: keir@xen.org; jbeulich@suse.com; andrew.cooper3@citrix.com; Zhang,
> Yang Z; george.dunlap@eu.citrix.com
> Subject: RE: [v3 12/15] vmx: posted-interrupt handling when vCPU is blocked
>
> > From: Wu, Feng
> > Sent: Wednesday, June 24, 2015 1:18 PM
> >
> > This patch includes the following aspects:
> > - Add a global vector to wake up the blocked vCPU
> > when an interrupt is being posted to it (This
> > part was sugguested by Yang Zhang <yang.z.zhang@intel.com>).
> > - Adds a new per-vCPU tasklet to wakeup the blocked
> > vCPU. It can be used in the case vcpu_unblock
> > cannot be called directly.
> > - Define two per-cpu variables:
> > * pi_blocked_vcpu:
> > A list storing the vCPUs which were blocked on this pCPU.
> >
> > * pi_blocked_vcpu_lock:
> > The spinlock to protect pi_blocked_vcpu.
> >
> > Signed-off-by: Feng Wu <feng.wu@intel.com>
> > ---
> > v3:
> > - This patch is generated by merging the following three patches in v2:
> > [RFC v2 09/15] Add a new per-vCPU tasklet to wakeup the blocked vCPU
> > [RFC v2 10/15] vmx: Define two per-cpu variables
> > [RFC v2 11/15] vmx: Add a global wake-up vector for VT-d
> Posted-Interrupts
> > - rename 'vcpu_wakeup_tasklet' to 'pi_vcpu_wakeup_tasklet'
> > - Move the definition of 'pi_vcpu_wakeup_tasklet' to 'struct arch_vmx_struct'
> > - rename 'vcpu_wakeup_tasklet_handler' to
> 'pi_vcpu_wakeup_tasklet_handler'
> > - Make pi_wakeup_interrupt() static
> > - Rename 'blocked_vcpu_list' to 'pi_blocked_vcpu_list'
> > - move 'pi_blocked_vcpu_list' to 'struct arch_vmx_struct'
> > - Rename 'blocked_vcpu' to 'pi_blocked_vcpu'
> > - Rename 'blocked_vcpu_lock' to 'pi_blocked_vcpu_lock'
> >
> > xen/arch/x86/hvm/vmx/vmcs.c | 3 +++
> > xen/arch/x86/hvm/vmx/vmx.c | 54
> > ++++++++++++++++++++++++++++++++++++++
> > xen/include/asm-x86/hvm/hvm.h | 1 +
> > xen/include/asm-x86/hvm/vmx/vmcs.h | 5 ++++
> > xen/include/asm-x86/hvm/vmx/vmx.h | 5 ++++
> > 5 files changed, 68 insertions(+)
> >
> > diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
> > index 11dc1b5..0c5ce3f 100644
> > --- a/xen/arch/x86/hvm/vmx/vmcs.c
> > +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> > @@ -631,6 +631,9 @@ int vmx_cpu_up(void)
> > if ( cpu_has_vmx_vpid )
> > vpid_sync_all();
> >
> > + INIT_LIST_HEAD(&per_cpu(pi_blocked_vcpu, cpu));
> > + spin_lock_init(&per_cpu(pi_blocked_vcpu_lock, cpu));
> > +
> > return 0;
> > }
> >
> > diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
> > index b94ef6a..7db6009 100644
> > --- a/xen/arch/x86/hvm/vmx/vmx.c
> > +++ b/xen/arch/x86/hvm/vmx/vmx.c
> > @@ -82,7 +82,20 @@ static int vmx_msr_read_intercept(unsigned int msr,
> uint64_t
> > *msr_content);
> > static int vmx_msr_write_intercept(unsigned int msr, uint64_t
> msr_content);
> > static void vmx_invlpg_intercept(unsigned long vaddr);
> >
> > +/*
> > + * We maintian a per-CPU linked-list of vCPU, so in PI wakeup handler we
> > + * can find which vCPU should be waken up.
> > + */
> > +DEFINE_PER_CPU(struct list_head, pi_blocked_vcpu);
> > +DEFINE_PER_CPU(spinlock_t, pi_blocked_vcpu_lock);
> > +
> > uint8_t __read_mostly posted_intr_vector;
> > +uint8_t __read_mostly pi_wakeup_vector;
> > +
> > +static void pi_vcpu_wakeup_tasklet_handler(unsigned long arg)
> > +{
> > + vcpu_unblock((struct vcpu *)arg);
> > +}
> >
> > static int vmx_domain_initialise(struct domain *d)
> > {
> > @@ -148,11 +161,19 @@ static int vmx_vcpu_initialise(struct vcpu *v)
> > if ( v->vcpu_id == 0 )
> > v->arch.user_regs.eax = 1;
> >
> > + tasklet_init(
> > + &v->arch.hvm_vmx.pi_vcpu_wakeup_tasklet,
> > + pi_vcpu_wakeup_tasklet_handler,
> > + (unsigned long)v);
> > +
> > + INIT_LIST_HEAD(&v->arch.hvm_vmx.pi_blocked_vcpu_list);
> > +
> > return 0;
> > }
> >
> > static void vmx_vcpu_destroy(struct vcpu *v)
> > {
> > + tasklet_kill(&v->arch.hvm_vmx.pi_vcpu_wakeup_tasklet);
> > /*
> > * There are cases that domain still remains in log-dirty mode when it
> is
> > * about to be destroyed (ex, user types 'xl destroy <dom>'), in which
> case
> > @@ -1848,6 +1869,33 @@ static struct hvm_function_table __initdata
> > vmx_function_table = {
> > .enable_msr_exit_interception = vmx_enable_msr_exit_interception,
> > };
> >
> > +/*
> > + * Handle VT-d posted-interrupt when VCPU is blocked.
> > + */
> > +static void pi_wakeup_interrupt(struct cpu_user_regs *regs)
> > +{
> > + struct arch_vmx_struct *vmx;
> > + unsigned int cpu = smp_processor_id();
> > +
> > + spin_lock(&per_cpu(pi_blocked_vcpu_lock, cpu));
> > +
> > + /*
> > + * FIXME: The length of the list depends on how many
> > + * vCPU is current blocked on this specific pCPU.
> > + * This may hurt the interrupt latency if the list
> > + * grows to too many entries.
> > + */
>
> let's go with this linked list first until a real issue is identified.
>
> > + list_for_each_entry(vmx, &per_cpu(pi_blocked_vcpu, cpu),
> > + pi_blocked_vcpu_list)
> > + if ( vmx->pi_desc.on )
> > + tasklet_schedule(&vmx->pi_vcpu_wakeup_tasklet);
>
> Not sure where the vcpu is removed from the list (possibly in later patch).
> But at least removing vcpu from the list at this point should be safe and
> right way to go. IIRC Andrew and other guys raised similar concern earlier. :-)
Thanks for the comments!
Yes, though it will be remove eventually in another patch, we need to remove
the vcpu right here to avoid kick the vCPU for multiple times.
Thanks,
Feng
>
> Thanks
> Kevin
next prev parent reply other threads:[~2015-07-08 11:02 UTC|newest]
Thread overview: 155+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-24 5:18 [v3 00/15] Add VT-d Posted-Interrupts support Feng Wu
2015-06-24 5:18 ` [v3 01/15] Vt-d Posted-intterrupt (PI) design Feng Wu
2015-06-24 6:15 ` Meng Xu
2015-06-24 6:19 ` Wu, Feng
2015-07-08 7:21 ` Tian, Kevin
2015-07-08 7:29 ` Wu, Feng
2015-06-24 5:18 ` [v3 02/15] Add helper macro for X86_FEATURE_CX16 feature detection Feng Wu
2015-06-24 17:31 ` Andrew Cooper
2015-07-08 7:23 ` Tian, Kevin
2015-06-24 5:18 ` [v3 03/15] Add cmpxchg16b support for x86-64 Feng Wu
2015-06-24 18:35 ` Andrew Cooper
2015-07-08 7:06 ` Wu, Feng
2015-07-08 8:12 ` Jan Beulich
2015-07-08 8:33 ` Wu, Feng
2015-07-08 8:43 ` Jan Beulich
2015-07-08 8:50 ` Wu, Feng
2015-07-08 8:50 ` Andrew Cooper
2015-07-10 12:57 ` Jan Beulich
2015-06-24 5:18 ` [v3 04/15] iommu: Add iommu_intpost to control VT-d Posted-Interrupts feature Feng Wu
2015-06-25 9:06 ` Andrew Cooper
2015-06-25 9:47 ` Wu, Feng
2015-06-25 10:16 ` Andrew Cooper
2015-06-25 12:47 ` Wu, Feng
2015-07-08 7:30 ` Tian, Kevin
2015-06-24 5:18 ` [v3 05/15] vt-d: VT-d Posted-Interrupts feature detection Feng Wu
2015-06-25 10:21 ` Andrew Cooper
2015-06-25 13:02 ` Wu, Feng
2015-07-08 7:32 ` Tian, Kevin
2015-07-08 8:00 ` Wu, Feng
2015-06-24 5:18 ` [v3 06/15] vmx: Extend struct pi_desc to support VT-d Posted-Interrupts Feng Wu
2015-06-29 15:04 ` Andrew Cooper
2015-07-08 7:48 ` Tian, Kevin
2015-07-10 13:08 ` Jan Beulich
2015-07-15 2:40 ` Wu, Feng
2015-07-15 8:20 ` Jan Beulich
2015-07-15 8:26 ` Wu, Feng
2015-07-15 8:36 ` Jan Beulich
2015-07-15 8:43 ` Wu, Feng
2015-07-15 9:28 ` Jan Beulich
2015-07-15 9:30 ` Wu, Feng
2015-07-15 3:13 ` Wu, Feng
2015-06-24 5:18 ` [v3 07/15] vmx: Initialize VT-d Posted-Interrupts Descriptor Feng Wu
2015-06-29 15:32 ` Andrew Cooper
2015-06-30 1:46 ` Wu, Feng
2015-06-30 2:32 ` Dario Faggioli
2015-07-08 7:53 ` Tian, Kevin
2015-06-24 5:18 ` [v3 08/15] Suppress posting interrupts when 'SN' is set Feng Wu
2015-06-29 15:41 ` Andrew Cooper
2015-06-30 1:48 ` Wu, Feng
2015-07-08 9:06 ` Tian, Kevin
2015-07-08 10:11 ` Wu, Feng
2015-07-08 11:31 ` Tian, Kevin
2015-07-08 11:58 ` Wu, Feng
2015-07-10 13:20 ` Jan Beulich
2015-06-24 5:18 ` [v3 09/15] vt-d: Extend struct iremap_entry to support VT-d Posted-Interrupts Feng Wu
2015-06-29 16:04 ` Andrew Cooper
2015-06-30 1:52 ` Wu, Feng
2015-07-08 9:10 ` Tian, Kevin
2015-07-10 13:27 ` Jan Beulich
2015-06-24 5:18 ` [v3 10/15] vt-d: Add API to update IRTE when VT-d PI is used Feng Wu
2015-06-29 16:22 ` Andrew Cooper
2015-07-08 9:59 ` Tian, Kevin
2015-07-08 10:12 ` Wu, Feng
2015-07-10 14:01 ` Jan Beulich
2015-07-15 6:04 ` Wu, Feng
2015-07-15 8:24 ` Jan Beulich
2015-07-15 8:38 ` Wu, Feng
2015-07-15 8:46 ` Jan Beulich
2015-07-15 8:55 ` Wu, Feng
2015-07-15 9:32 ` Jan Beulich
2015-06-24 5:18 ` [v3 11/15] Update IRTE according to guest interrupt config changes Feng Wu
2015-06-29 16:46 ` Andrew Cooper
2015-07-08 10:22 ` Tian, Kevin
2015-07-08 10:31 ` Wu, Feng
2015-07-08 11:46 ` Tian, Kevin
2015-07-08 11:52 ` Wu, Feng
2015-07-08 11:54 ` Tian, Kevin
2015-07-10 14:23 ` Jan Beulich
2015-06-24 5:18 ` [v3 12/15] vmx: posted-interrupt handling when vCPU is blocked Feng Wu
2015-06-29 17:07 ` Andrew Cooper
2015-07-08 10:36 ` Wu, Feng
2015-07-08 10:48 ` Jan Beulich
[not found] ` <559181F9.6020106@citrix.com>
2015-06-30 2:51 ` Fwd: " Dario Faggioli
2015-06-30 2:59 ` Wu, Feng
2015-06-30 9:46 ` Dario Faggioli
2015-06-30 10:11 ` Andrew Cooper
2015-07-01 13:26 ` Dario Faggioli
2015-07-02 4:27 ` Wu, Feng
2015-07-02 8:30 ` Dario Faggioli
2015-07-02 8:58 ` Wu, Feng
2015-07-02 10:09 ` Dario Faggioli
2015-07-02 10:41 ` Wu, Feng
2015-07-02 10:30 ` Andrew Cooper
2015-07-02 10:56 ` Wu, Feng
2015-07-02 12:04 ` Dario Faggioli
2015-07-02 12:10 ` Wu, Feng
2015-07-02 12:16 ` Andrew Cooper
2015-07-02 12:38 ` Dario Faggioli
2015-07-02 12:59 ` Andrew Cooper
2015-07-03 1:33 ` Wu, Feng
2015-07-02 4:25 ` Wu, Feng
2015-07-08 11:00 ` Tian, Kevin
2015-07-08 11:02 ` Wu, Feng [this message]
2015-07-08 12:46 ` Jan Beulich
2015-07-08 13:09 ` Andrew Cooper
2015-07-08 22:49 ` Tian, Kevin
2015-07-09 7:25 ` Jan Beulich
2015-07-10 6:21 ` Wu, Feng
2015-07-10 6:32 ` Jan Beulich
2015-07-10 7:29 ` Wu, Feng
2015-07-10 8:49 ` Jan Beulich
2015-07-10 8:57 ` Wu, Feng
2015-07-08 22:31 ` Tian, Kevin
2015-06-24 5:18 ` [v3 13/15] vmx: Properly handle notification event when vCPU is running Feng Wu
2015-07-08 11:03 ` Tian, Kevin
2015-07-10 14:40 ` Jan Beulich
2015-06-24 5:18 ` [v3 14/15] Update Posted-Interrupts Descriptor during vCPU scheduling Feng Wu
[not found] ` <55918214.4030102@citrix.com>
2015-06-30 2:58 ` Fwd: " Dario Faggioli
2015-07-02 4:32 ` Wu, Feng
2015-07-02 4:34 ` Wu, Feng
2015-07-02 8:20 ` Dario Faggioli
2015-07-09 3:09 ` Wu, Feng
2015-07-09 8:18 ` Dario Faggioli
2015-07-09 11:19 ` George Dunlap
2015-07-09 11:29 ` George Dunlap
2015-07-09 11:38 ` Wu, Feng
2015-07-09 12:42 ` Dario Faggioli
2015-07-10 0:07 ` Wu, Feng
2015-07-10 12:40 ` Dario Faggioli
2015-07-10 13:47 ` Konrad Rzeszutek Wilk
2015-07-10 13:59 ` Dario Faggioli
2015-07-09 12:53 ` George Dunlap
2015-07-09 13:44 ` Jan Beulich
2015-07-09 14:18 ` Dario Faggioli
2015-07-09 14:27 ` George Dunlap
2015-07-09 14:47 ` Dario Faggioli
2015-07-10 5:59 ` Wu, Feng
2015-07-10 6:22 ` Jan Beulich
2015-07-10 11:05 ` Dario Faggioli
2015-07-14 5:44 ` Wu, Feng
2015-07-14 14:08 ` Wu, Feng
2015-07-14 14:54 ` Jan Beulich
2015-07-14 15:20 ` Dario Faggioli
2015-07-14 16:41 ` George Dunlap
2015-07-14 16:02 ` Dario Faggioli
2015-07-15 0:54 ` Wu, Feng
2015-07-17 7:46 ` Wu, Feng
2015-07-17 10:13 ` Dario Faggioli
2015-07-17 22:57 ` Wu, Feng
2015-07-18 13:43 ` Dario Faggioli
2015-07-10 0:15 ` Wu, Feng
2015-07-08 11:24 ` Tian, Kevin
2015-07-10 14:48 ` Jan Beulich
2015-06-24 5:18 ` [v3 15/15] Add a command line parameter for VT-d posted-interrupts Feng Wu
2015-07-08 11:25 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E959C4978C3B6342920538CF579893F0025F8E01@SHSMSX104.ccr.corp.intel.com \
--to=feng.wu@intel.com \
--cc=andrew.cooper3@citrix.com \
--cc=george.dunlap@eu.citrix.com \
--cc=jbeulich@suse.com \
--cc=keir@xen.org \
--cc=kevin.tian@intel.com \
--cc=xen-devel@lists.xen.org \
--cc=yang.z.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.