* Nested posted interrupts
@ 2017-04-18 19:56 Jim Mattson
2017-04-19 9:10 ` Paolo Bonzini
2017-04-21 11:50 ` Wincy Van
0 siblings, 2 replies; 9+ messages in thread
From: Jim Mattson @ 2017-04-18 19:56 UTC (permalink / raw
To: kvm list
In commit 705699a139948 ("KVM: nVMX: Enable nested posted interrupt
processing"), nested posted interrupts are implemented such that
vmcs02 uses the same notification vector as vmcs01. I don't understand
how this can work.
Suppose that VCPU0 is in L2, with posted interrupts enabled (for
simplicity, let's say that the notification vector in vmcs12 is 0xf1,
the same as in vmcs01 and vmcs02). Furthermore, suppose that L2 is
halted, with the pin-based vm-execution control for
"external-interrupt exiting" enabled.
Now, suppose that VCPU1 is in L1, and it wants to send an IPI (say
0xc7) to VCPU0 (which it doesn't know is in L2...and shouldn't have to
know). vmx_deliver_posted_interrupt will set the PIR and ON bits in
VCPU0's vmcs01 posted interrupt descriptor and then it will send a
0xf1 IPI to the pCPU running VCPU0.
Hardware will recognize that 0xf1 is the notification vector for
vmcs02, and it will process vmcs02's posted interrupt descriptor.
Finding nothing of interest there, it will return to the halt state,
without a VM-exit.
Something should trigger L0 to synthesize an external interrupt
VM-exit from L2 to L1 for vector 0xc7. However, I don't see how this
happens.
What am I missing?
Thanks,
--jim
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-18 19:56 Nested posted interrupts Jim Mattson
@ 2017-04-19 9:10 ` Paolo Bonzini
2017-04-19 15:33 ` Jim Mattson
2017-04-21 11:50 ` Wincy Van
1 sibling, 1 reply; 9+ messages in thread
From: Paolo Bonzini @ 2017-04-19 9:10 UTC (permalink / raw
To: Jim Mattson, kvm list
On 18/04/2017 21:56, Jim Mattson wrote:
> In commit 705699a139948 ("KVM: nVMX: Enable nested posted interrupt
> processing"), nested posted interrupts are implemented such that
> vmcs02 uses the same notification vector as vmcs01. I don't understand
> how this can work.
>
> Suppose that VCPU0 is in L2, with posted interrupts enabled (for
> simplicity, let's say that the notification vector in vmcs12 is 0xf1,
> the same as in vmcs01 and vmcs02). Furthermore, suppose that L2 is
> halted, with the pin-based vm-execution control for
> "external-interrupt exiting" enabled.
>
> Now, suppose that VCPU1 is in L1, and it wants to send an IPI (say
> 0xc7) to VCPU0 (which it doesn't know is in L2...and shouldn't have to
> know). vmx_deliver_posted_interrupt will set the PIR and ON bits in
> VCPU0's vmcs01 posted interrupt descriptor and then it will send a
> 0xf1 IPI to the pCPU running VCPU0.
>
> Hardware will recognize that 0xf1 is the notification vector for
> vmcs02, and it will process vmcs02's posted interrupt descriptor.
> Finding nothing of interest there, it will return to the halt state,
> without a VM-exit.
>
> Something should trigger L0 to synthesize an external interrupt
> VM-exit from L2 to L1 for vector 0xc7. However, I don't see how this
> happens.
>
> What am I missing?
Nothing, I think you're right.
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-19 9:10 ` Paolo Bonzini
@ 2017-04-19 15:33 ` Jim Mattson
2017-04-19 15:47 ` Paolo Bonzini
2017-04-21 2:51 ` Wincy Van
0 siblings, 2 replies; 9+ messages in thread
From: Jim Mattson @ 2017-04-19 15:33 UTC (permalink / raw
To: Paolo Bonzini; +Cc: kvm list
I think the easiest solution is to allocate a second fixed
notification vector for vmcs02. If the L2 vector arrives while vmcs01
is active, we suffer an unnecessary VM-exit, but at least if the L1
vector arrives while vmcs02 is active, we will know to synthesize a
VM-exit from L2 to L1 (if vmcs01's PI descriptor indicates that there
is a virtual interrupt to be delivered).
On Wed, Apr 19, 2017 at 2:10 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 18/04/2017 21:56, Jim Mattson wrote:
>> In commit 705699a139948 ("KVM: nVMX: Enable nested posted interrupt
>> processing"), nested posted interrupts are implemented such that
>> vmcs02 uses the same notification vector as vmcs01. I don't understand
>> how this can work.
>>
>> Suppose that VCPU0 is in L2, with posted interrupts enabled (for
>> simplicity, let's say that the notification vector in vmcs12 is 0xf1,
>> the same as in vmcs01 and vmcs02). Furthermore, suppose that L2 is
>> halted, with the pin-based vm-execution control for
>> "external-interrupt exiting" enabled.
>>
>> Now, suppose that VCPU1 is in L1, and it wants to send an IPI (say
>> 0xc7) to VCPU0 (which it doesn't know is in L2...and shouldn't have to
>> know). vmx_deliver_posted_interrupt will set the PIR and ON bits in
>> VCPU0's vmcs01 posted interrupt descriptor and then it will send a
>> 0xf1 IPI to the pCPU running VCPU0.
>>
>> Hardware will recognize that 0xf1 is the notification vector for
>> vmcs02, and it will process vmcs02's posted interrupt descriptor.
>> Finding nothing of interest there, it will return to the halt state,
>> without a VM-exit.
>>
>> Something should trigger L0 to synthesize an external interrupt
>> VM-exit from L2 to L1 for vector 0xc7. However, I don't see how this
>> happens.
>>
>> What am I missing?
>
> Nothing, I think you're right.
>
> Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-19 15:33 ` Jim Mattson
@ 2017-04-19 15:47 ` Paolo Bonzini
2017-04-21 2:51 ` Wincy Van
1 sibling, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2017-04-19 15:47 UTC (permalink / raw
To: Jim Mattson; +Cc: kvm list
On 19/04/2017 17:33, Jim Mattson wrote:
> I think the easiest solution is to allocate a second fixed
> notification vector for vmcs02. If the L2 vector arrives while vmcs01
> is active, we suffer an unnecessary VM-exit, but at least if the L1
> vector arrives while vmcs02 is active, we will know to synthesize a
> VM-exit from L2 to L1 (if vmcs01's PI descriptor indicates that there
> is a virtual interrupt to be delivered).
For IPIs it'd be enough to use kvm_vcpu_kick instead of
kvm_vcpu_trigger_posted_interrupt when sending an IPI other than the PI
vector.
Basically we have three cases:
- !is_guest_mode, set PIR/ON and either trigger posted interrupt or kick
- is_guest_mode, PI vector, set PIR/ON and trigger either posted
interrupt or kick (kicking may be necessary when L2 runs under HLT
activity state)
- is_guest_mode, non PI vector, just kick. kvm_vcpu_running's call to
kvm_x86_ops->check_nested_events should handle it just fine:
vmx_check_nested_events
kvm_cpu_has_interrupt
kvm_apic_has_interrupt
apic_has_interrupt_for_ppr
vmx_sync_pir_to_irr
nested_vmx_vmexit
On the other hand, I'm not sure how to handle VT-d posted interrupts
this way. They would cause a posted interrupt vector IPI, which is then
ignored as described in your previous email.
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-19 15:33 ` Jim Mattson
2017-04-19 15:47 ` Paolo Bonzini
@ 2017-04-21 2:51 ` Wincy Van
2017-04-21 7:57 ` Paolo Bonzini
1 sibling, 1 reply; 9+ messages in thread
From: Wincy Van @ 2017-04-21 2:51 UTC (permalink / raw
To: Jim Mattson; +Cc: Paolo Bonzini, kvm list
On Wed, Apr 19, 2017 at 11:33 PM, Jim Mattson <jmattson@google.com> wrote:
> I think the easiest solution is to allocate a second fixed
> notification vector for vmcs02. If the L2 vector arrives while vmcs01
> is active, we suffer an unnecessary VM-exit, but at least if the L1
> vector arrives while vmcs02 is active, we will know to synthesize a
> VM-exit from L2 to L1 (if vmcs01's PI descriptor indicates that there
> is a virtual interrupt to be delivered).
>
I also prefer this solution.
Assume that the new PI vector is pi_vec02, and the old vector is pi_vec01.
We just need to use pi_vec02 and pi_vec01 to set POSTED_INTR_VECTOR
field in prepare_vmcs02/prepare_vmcs12.
Then, use the corresponding pi_vec01/pi_vec02 to trigger PI when dest
vcpu ! is_guest_mode / is_guest_mode.
If dest vcpu is_guest_mode, when L1 is sending IPI, we are going to
use pi_vec01, and a nested vmexit
will be triggered in vmx_check_nested_events.
Paolo, I think may be the VT-d PI will also be handled in this solution ?
Thanks
Wincy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-21 2:51 ` Wincy Van
@ 2017-04-21 7:57 ` Paolo Bonzini
2017-04-21 11:46 ` Wincy Van
0 siblings, 1 reply; 9+ messages in thread
From: Paolo Bonzini @ 2017-04-21 7:57 UTC (permalink / raw
To: Wincy Van, Jim Mattson; +Cc: kvm list
On 21/04/2017 04:51, Wincy Van wrote:
> On Wed, Apr 19, 2017 at 11:33 PM, Jim Mattson <jmattson@google.com> wrote:
>> I think the easiest solution is to allocate a second fixed
>> notification vector for vmcs02. If the L2 vector arrives while vmcs01
>> is active, we suffer an unnecessary VM-exit, but at least if the L1
>> vector arrives while vmcs02 is active, we will know to synthesize a
>> VM-exit from L2 to L1 (if vmcs01's PI descriptor indicates that there
>> is a virtual interrupt to be delivered).
>>
> I also prefer this solution.
>
> Assume that the new PI vector is pi_vec02, and the old vector is pi_vec01.
> We just need to use pi_vec02 and pi_vec01 to set POSTED_INTR_VECTOR
> field in prepare_vmcs02/prepare_vmcs12.
> Then, use the corresponding pi_vec01/pi_vec02 to trigger PI when dest
> vcpu ! is_guest_mode / is_guest_mode.
>
> If dest vcpu is_guest_mode, when L1 is sending IPI, we are going to
> use pi_vec01, and a nested vmexit
> will be triggered in vmx_check_nested_events.
>
> Paolo, I think may be the VT-d PI will also be handled in this solution ?
Yes, it would. I thought more about the VT-d PI and I didn't find an
efficient way to fix it with one vector. Wincy, would you like to write
the patch?
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-21 7:57 ` Paolo Bonzini
@ 2017-04-21 11:46 ` Wincy Van
0 siblings, 0 replies; 9+ messages in thread
From: Wincy Van @ 2017-04-21 11:46 UTC (permalink / raw
To: Paolo Bonzini; +Cc: Jim Mattson, kvm list
On Fri, Apr 21, 2017 at 3:57 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 21/04/2017 04:51, Wincy Van wrote:
>> On Wed, Apr 19, 2017 at 11:33 PM, Jim Mattson <jmattson@google.com> wrote:
>>> I think the easiest solution is to allocate a second fixed
>>> notification vector for vmcs02. If the L2 vector arrives while vmcs01
>>> is active, we suffer an unnecessary VM-exit, but at least if the L1
>>> vector arrives while vmcs02 is active, we will know to synthesize a
>>> VM-exit from L2 to L1 (if vmcs01's PI descriptor indicates that there
>>> is a virtual interrupt to be delivered).
>>>
>> I also prefer this solution.
>>
>> Assume that the new PI vector is pi_vec02, and the old vector is pi_vec01.
>> We just need to use pi_vec02 and pi_vec01 to set POSTED_INTR_VECTOR
>> field in prepare_vmcs02/prepare_vmcs12.
>> Then, use the corresponding pi_vec01/pi_vec02 to trigger PI when dest
>> vcpu ! is_guest_mode / is_guest_mode.
>>
>> If dest vcpu is_guest_mode, when L1 is sending IPI, we are going to
>> use pi_vec01, and a nested vmexit
>> will be triggered in vmx_check_nested_events.
>>
>> Paolo, I think may be the VT-d PI will also be handled in this solution ?
>
> Yes, it would. I thought more about the VT-d PI and I didn't find an
> efficient way to fix it with one vector. Wincy, would you like to write
> the patch?
>
Certainly, will do it next week.
Thanks,
Wincy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-18 19:56 Nested posted interrupts Jim Mattson
2017-04-19 9:10 ` Paolo Bonzini
@ 2017-04-21 11:50 ` Wincy Van
2017-04-21 15:04 ` Jim Mattson
1 sibling, 1 reply; 9+ messages in thread
From: Wincy Van @ 2017-04-21 11:50 UTC (permalink / raw
To: Jim Mattson; +Cc: kvm list
On Wed, Apr 19, 2017 at 3:56 AM, Jim Mattson <jmattson@google.com> wrote:
> In commit 705699a139948 ("KVM: nVMX: Enable nested posted interrupt
> processing"), nested posted interrupts are implemented such that
> vmcs02 uses the same notification vector as vmcs01. I don't understand
> how this can work.
>
> Suppose that VCPU0 is in L2, with posted interrupts enabled (for
> simplicity, let's say that the notification vector in vmcs12 is 0xf1,
> the same as in vmcs01 and vmcs02). Furthermore, suppose that L2 is
> halted, with the pin-based vm-execution control for
> "external-interrupt exiting" enabled.
>
> Now, suppose that VCPU1 is in L1, and it wants to send an IPI (say
> 0xc7) to VCPU0 (which it doesn't know is in L2...and shouldn't have to
> know). vmx_deliver_posted_interrupt will set the PIR and ON bits in
> VCPU0's vmcs01 posted interrupt descriptor and then it will send a
> 0xf1 IPI to the pCPU running VCPU0.
>
> Hardware will recognize that 0xf1 is the notification vector for
> vmcs02, and it will process vmcs02's posted interrupt descriptor.
> Finding nothing of interest there, it will return to the halt state,
> without a VM-exit.
>
> Something should trigger L0 to synthesize an external interrupt
> VM-exit from L2 to L1 for vector 0xc7. However, I don't see how this
> happens.
>
> What am I missing?
>
Jim, thank you for pointing out the mistake :-)
Wincy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested posted interrupts
2017-04-21 11:50 ` Wincy Van
@ 2017-04-21 15:04 ` Jim Mattson
0 siblings, 0 replies; 9+ messages in thread
From: Jim Mattson @ 2017-04-21 15:04 UTC (permalink / raw
To: Wincy Van; +Cc: kvm list
Thanks for volunteering to fix it!
On Fri, Apr 21, 2017 at 4:50 AM, Wincy Van <fanwenyi0529@gmail.com> wrote:
> On Wed, Apr 19, 2017 at 3:56 AM, Jim Mattson <jmattson@google.com> wrote:
>> In commit 705699a139948 ("KVM: nVMX: Enable nested posted interrupt
>> processing"), nested posted interrupts are implemented such that
>> vmcs02 uses the same notification vector as vmcs01. I don't understand
>> how this can work.
>>
>> Suppose that VCPU0 is in L2, with posted interrupts enabled (for
>> simplicity, let's say that the notification vector in vmcs12 is 0xf1,
>> the same as in vmcs01 and vmcs02). Furthermore, suppose that L2 is
>> halted, with the pin-based vm-execution control for
>> "external-interrupt exiting" enabled.
>>
>> Now, suppose that VCPU1 is in L1, and it wants to send an IPI (say
>> 0xc7) to VCPU0 (which it doesn't know is in L2...and shouldn't have to
>> know). vmx_deliver_posted_interrupt will set the PIR and ON bits in
>> VCPU0's vmcs01 posted interrupt descriptor and then it will send a
>> 0xf1 IPI to the pCPU running VCPU0.
>>
>> Hardware will recognize that 0xf1 is the notification vector for
>> vmcs02, and it will process vmcs02's posted interrupt descriptor.
>> Finding nothing of interest there, it will return to the halt state,
>> without a VM-exit.
>>
>> Something should trigger L0 to synthesize an external interrupt
>> VM-exit from L2 to L1 for vector 0xc7. However, I don't see how this
>> happens.
>>
>> What am I missing?
>>
>
> Jim, thank you for pointing out the mistake :-)
>
> Wincy
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-04-21 19:34 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-04-18 19:56 Nested posted interrupts Jim Mattson
2017-04-19 9:10 ` Paolo Bonzini
2017-04-19 15:33 ` Jim Mattson
2017-04-19 15:47 ` Paolo Bonzini
2017-04-21 2:51 ` Wincy Van
2017-04-21 7:57 ` Paolo Bonzini
2017-04-21 11:46 ` Wincy Van
2017-04-21 11:50 ` Wincy Van
2017-04-21 15:04 ` Jim Mattson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.