From: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: izumi.taku@jp.fujitsu.com, qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] [RFC v9 14/18] vfio: improve vfio_pci_hot_reset to support more case
Date: Thu, 18 Jun 2015 18:27:24 +0800 [thread overview]
Message-ID: <55829D0C.3050008@cn.fujitsu.com> (raw)
In-Reply-To: <1434554607.5628.26.camel@redhat.com>
On 06/17/2015 11:23 PM, Alex Williamson wrote:
> On Wed, 2015-06-17 at 14:28 +0800, Chen Fan wrote:
>> On 06/16/2015 10:08 PM, Alex Williamson wrote:
>>> On Tue, 2015-06-16 at 16:10 +0800, Chen Fan wrote:
>>>> On 06/10/2015 05:24 AM, Alex Williamson wrote:
>>>>> On Tue, 2015-06-09 at 11:37 +0800, Chen Fan wrote:
>>>>>> the vfio_pci_hot_reset differentiate the single and multi in-used
>>>>>> devices for reset. but sometimes we own the group without any devices,
>>>>>> that also should support hot reset.
>>>>> Nope, did you try it? It can be done, but the group still needs to be
>>>>> connected to a container for isolation.
>>>> I'm sorry for that. because I have no such host in hand. but I think if
>>>> we can keep connect container for each affected group, we also able
>>>> to use this method to do host bus reset.
>>> All you need is a dual-port card with isolation, which includes all
>>> Intel 1G NICs (igb & e1000e) as of the quirks that are currently in
>>> linux-next to be pushed for v4.2. Intel 10G NICs are already quirked
>>> upstream. There are certainly ways to fake isolation for testing as
>>> well. Thanks,
>> I just have a Intel Corporation 82576 dual-port card, but how can I fake
>> isolation group for this card in linux-next kernel? can you tell me the
>> document link?
> If you're running linux-next and have the card installed under a root
> port that provides isolation then each port should be in a separate
> iommu group. Nearly all Intel PCH root ports should have quirks to
> enable isolation. If you're installing it in a processor root port
> slot, you need to use a Xeon E5 or better CPU or else the lack of
> isolation at the root port will negate the isolation capabilities of the
> endpoint. Thanks,
Hi Alex,
I had test the case with isolation groups in latest linux-next,
I can see the dual-port devices in host looks like:
#readlink /sys/bus/pci/devices/0000\:06\:00.0/iommu_group
../../../../kernel/iommu_groups/30
#readlink /sys/bus/pci/devices/0000\:06\:00.1/iommu_group
../../../../kernel/iommu_groups/31
I used my v10 qemu code that added affected groups to VM to
test the case with qemu command:
qemu-system-x86_64 -M q35 -device
ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1
-device vfio-pci,host=06:00.0,bus=bridge1,aer=true --enable-kvm
then I used to emulate the aer with aer-inject in host. I could find the aer
recovery successful in guest. and the 6:00.0 NIC can reuse by
network-manage.
but when I re-binding the dual devices to host. the host show error:
Jun 18 19:13:36 TX300I kernel: [<ffffffff81666913>] dump_stack+0x45/0x57
Jun 18 19:13:36 TX300I kernel: [<ffffffff8107986a>]
warn_slowpath_common+0x8a/0xc0
Jun 18 19:13:36 TX300I kernel: [<ffffffff810798f5>]
warn_slowpath_fmt+0x55/0x70
Jun 18 19:13:36 TX300I kernel: [<ffffffff81326818>] bad_io_access+0x38/0x40
Jun 18 19:13:36 TX300I kernel: [<ffffffff81326a57>] pci_iounmap+0x27/0x40
Jun 18 19:13:36 TX300I kernel: [<ffffffffa01254ad>]
igb_probe+0xafd/0x1280 [igb]
Jun 18 19:13:36 TX300I kernel: [<ffffffff8134a6d5>]
local_pci_probe+0x45/0xa0
Jun 18 19:13:36 TX300I kernel: [<ffffffff8134b884>] ?
pci_match_device+0xf4/0x120
Jun 18 19:13:36 TX300I kernel: [<ffffffff8134b9d9>]
pci_device_probe+0xe9/0x130
Jun 18 19:13:36 TX300I kernel: [<ffffffff8143547f>]
driver_probe_device+0x14f/0x420
Jun 18 19:13:36 TX300I kernel: [<ffffffff8143389c>] bind_store+0xdc/0x120
Jun 18 19:13:36 TX300I kernel: [<ffffffff81432f74>] drv_attr_store+0x24/0x30
Jun 18 19:13:36 TX300I kernel: [<ffffffff8126de9a>] sysfs_kf_write+0x3a/0x50
Jun 18 19:13:36 TX300I kernel: [<ffffffff8126d520>]
kernfs_fop_write+0x120/0x170
Jun 18 19:13:36 TX300I kernel: [<ffffffff811f13a7>] __vfs_write+0x37/0x100
Jun 18 19:13:36 TX300I kernel: [<ffffffff811f40d8>] ?
__sb_start_write+0x58/0x110
Jun 18 19:13:36 TX300I kernel: [<ffffffff811f1aa9>] vfs_write+0xa9/0x190
Jun 18 19:13:36 TX300I kernel: [<ffffffff81023556>] ?
do_audit_syscall_entry+0x66/0x70
Jun 18 19:13:36 TX300I kernel: [<ffffffff811f28a5>] SyS_write+0x55/0xc0
Jun 18 19:13:36 TX300I kernel: [<ffffffff8166d72e>]
entry_SYSCALL_64_fastpath+0x12/0x71
Jun 18 19:13:36 TX300I kernel: ---[ end trace b312cb051751fac4 ]---
Jun 18 19:13:36 TX300I kernel: igb: probe of 0000:06:00.1 failed with
error -5
the 06:00.0 device can be initialized by igb driver. but the affected
06:00.1 can't be initialized by igb driver.
Is not the 06:00.1 device state initialized by reset ?
Thanks,
Chen
>
> Alex
>
>>>>>> Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
>>>>>> ---
>>>>>> hw/vfio/pci.c | 11 +++++++++++
>>>>>> 1 file changed, 11 insertions(+)
>>>>>>
>>>>>> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
>>>>>> index a4e8658..6507f39 100644
>>>>>> --- a/hw/vfio/pci.c
>>>>>> +++ b/hw/vfio/pci.c
>>>>>> @@ -3398,6 +3398,7 @@ static int vfio_pci_hot_reset(VFIOPCIDevice *vdev, bool single)
>>>>>> PCIHostDeviceAddress host;
>>>>>> VFIOPCIDevice *tmp;
>>>>>> VFIODevice *vbasedev_iter;
>>>>>> + bool found;
>>>>>>
>>>>>> host.domain = devices[i].segment;
>>>>>> host.bus = devices[i].bus;
>>>>>> @@ -3427,6 +3428,7 @@ static int vfio_pci_hot_reset(VFIOPCIDevice *vdev, bool single)
>>>>>> goto out;
>>>>>> }
>>>>>>
>>>>>> + found = false;
>>>>>> /* Prep dependent devices for reset and clear our marker. */
>>>>>> QLIST_FOREACH(vbasedev_iter, &group->device_list, next) {
>>>>>> if (vbasedev_iter->type != VFIO_DEVICE_TYPE_PCI) {
>>>>>> @@ -3438,12 +3440,21 @@ static int vfio_pci_hot_reset(VFIOPCIDevice *vdev, bool single)
>>>>>> ret = -EINVAL;
>>>>>> goto out_single;
>>>>>> }
>>>>>> + found = true;
>>>>>> vfio_pci_pre_reset(tmp);
>>>>>> tmp->vbasedev.needs_reset = false;
>>>>>> multi = true;
>>>>>> break;
>>>>>> }
>>>>>> }
>>>>>> +
>>>>>> + /*
>>>>>> + * If we own the group but does not own the device, we also
>>>>>> + * should call hot reset with multi.
>>>>>> + */
>>>>>> + if (!single && !found) {
>>>>>> + multi = true;
>>>>>> + }
>>>>>> }
>>>>>>
>>>>>> if (!single && !multi) {
>>>>>
>>>>>
>>>
>>>
>>> .
>>>
>
>
> .
>
next prev parent reply other threads:[~2015-06-18 10:29 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-09 3:37 [Qemu-devel] [RFC v9 00/18] vfio-pci: pass the aer error to guest Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 01/18] vfio: extract vfio_get_hot_reset_info as a single function Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 02/18] vfio: squeeze out vfio_pci_do_hot_reset for support bus reset Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 03/18] pcie: modify the capability size assert Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 04/18] vfio: make the 4 bytes aligned for capability size Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 05/18] vfio: add pcie extanded capability support Chen Fan
2015-06-09 21:22 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 06/18] aer: impove pcie_aer_init to support vfio device Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 07/18] vfio: add aer support for " Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 08/18] vfio: improve vfio_get_group to support adding group without devices Chen Fan
2015-06-09 21:23 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 09/18] vfio: add ref for group to support own affected groups Chen Fan
2015-06-09 21:23 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 10/18] get all affected groups for each device support aer Chen Fan
2015-06-09 21:22 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 11/18] vfio: add check host bus reset is support or not Chen Fan
2015-06-09 21:23 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 12/18] pci: add bus reset_notifiers callbacks for host bus reset Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 13/18] vfio: add sec_bus_reset notifier to notify physical bus reset is needed Chen Fan
2015-06-09 21:23 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 14/18] vfio: improve vfio_pci_hot_reset to support more case Chen Fan
2015-06-09 21:24 ` Alex Williamson
2015-06-16 8:10 ` Chen Fan
2015-06-16 14:08 ` Alex Williamson
2015-06-17 6:28 ` Chen Fan
2015-06-17 15:23 ` Alex Williamson
2015-06-18 10:27 ` Chen Fan [this message]
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 15/18] vfio: do hot bus reset when do virtual secondary bus reset Chen Fan
2015-06-09 21:23 ` Alex Williamson
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 16/18] pcie_aer: expose pcie_aer_msg() interface Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 17/18] vfio-pci: pass the aer error to guest Chen Fan
2015-06-09 3:37 ` [Qemu-devel] [RFC v9 18/18] vfio: add 'aer' property to expose aercap Chen Fan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55829D0C.3050008@cn.fujitsu.com \
--to=chen.fan.fnst@cn.fujitsu.com \
--cc=alex.williamson@redhat.com \
--cc=izumi.taku@jp.fujitsu.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.