From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754977AbbKYFjc (ORCPT ); Wed, 25 Nov 2015 00:39:32 -0500 Received: from mail-ig0-f174.google.com ([209.85.213.174]:33529 "EHLO mail-ig0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751291AbbKYFjY (ORCPT ); Wed, 25 Nov 2015 00:39:24 -0500 MIME-Version: 1.0 In-Reply-To: <20151124230551-mutt-send-email-mst@redhat.com> References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> Date: Tue, 24 Nov 2015 21:39:23 -0800 Message-ID: Subject: Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver From: Alexander Duyck To: "Michael S. Tsirkin" Cc: Lan Tianyu , a.motakis@virtualopensystems.com, Alex Williamson , b.reynal@virtualopensystems.com, Bjorn Helgaas , Carolyn Wyborny , "Skidmore, Donald C" , eddie.dong@intel.com, nrupal.jani@intel.com, Alexander Graf , kvm@vger.kernel.org, Paolo Bonzini , qemu-devel@nongnu.org, "Tantilov, Emil S" , Or Gerlitz , "Rustad, Mark D" , Eric Auger , intel-wired-lan , Jeff Kirsher , "Brandeburg, Jesse" , "Ronciak, John" , linux-api@vger.kernel.org, "linux-kernel@vger.kernel.org" , Mitch Williams , Netdev , "Nelson, Shannon" , Wei Yang , zajec5@gmail.com Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Nov 24, 2015 at 1:20 PM, Michael S. Tsirkin wrote: > On Tue, Nov 24, 2015 at 09:38:18PM +0800, Lan Tianyu wrote: >> This patch is to add migration support for ixgbevf driver. Using >> faked PCI migration capability table communicates with Qemu to >> share migration status and mailbox irq vector index. >> >> Qemu will notify VF via sending MSIX msg to trigger mailbox >> vector during migration and store migration status in the >> PCI_VF_MIGRATION_VMM_STATUS regs in the new capability table. >> The mailbox irq will be triggered just befoe stop-and-copy stage >> and after migration on the target machine. >> >> VF driver will put down net when detect migration and tell >> Qemu it's ready for migration via writing PCI_VF_MIGRATION_VF_STATUS >> reg. After migration, put up net again. >> >> Qemu will in charge of migrating PCI config space regs and MSIX config. >> >> The patch is to dedicate on the normal case that net traffic works >> when mailbox irq is enabled. For other cases(such as the driver >> isn't loaded, adapter is suspended or closed), mailbox irq won't be >> triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP >> reg. These case will be resolved later. >> >> Signed-off-by: Lan Tianyu > > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? The tracking of dirty pages still needs to be addressed unless the interface is being downed before migration even starts which based on other comments I am assuming is not the case. I still feel that having a means of marking a page as being dirty when it is unmapped would be the best way to go. That way you only have to update the DMA API instead of messing with each and every driver trying to add code to force the page to be dirtied. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32967) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1Snd-0000ah-Gg for qemu-devel@nongnu.org; Wed, 25 Nov 2015 00:39:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a1Snc-0007Fh-KK for qemu-devel@nongnu.org; Wed, 25 Nov 2015 00:39:25 -0500 Received: from mail-ig0-x232.google.com ([2607:f8b0:4001:c05::232]:32847) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1Snc-0007Fd-Eu for qemu-devel@nongnu.org; Wed, 25 Nov 2015 00:39:24 -0500 Received: by igcmv3 with SMTP id mv3so66124976igc.0 for ; Tue, 24 Nov 2015 21:39:24 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <20151124230551-mutt-send-email-mst@redhat.com> References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> Date: Tue, 24 Nov 2015 21:39:23 -0800 Message-ID: From: Alexander Duyck Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Wei Yang , "Tantilov, Emil S" , kvm@vger.kernel.org, qemu-devel@nongnu.org, "Brandeburg, Jesse" , "Rustad, Mark D" , Carolyn Wyborny , Eric Auger , "Skidmore, Donald C" , zajec5@gmail.com, Alexander Graf , intel-wired-lan , Jeff Kirsher , Or Gerlitz , Mitch Williams , nrupal.jani@intel.com, Bjorn Helgaas , a.motakis@virtualopensystems.com, Lan Tianyu , b.reynal@virtualopensystems.com, linux-api@vger.kernel.org, "Nelson, Shannon" , eddie.dong@intel.com, Alex Williamson , "linux-kernel@vger.kernel.org" , "Ronciak, John" , Netdev , Paolo Bonzini On Tue, Nov 24, 2015 at 1:20 PM, Michael S. Tsirkin wrote: > On Tue, Nov 24, 2015 at 09:38:18PM +0800, Lan Tianyu wrote: >> This patch is to add migration support for ixgbevf driver. Using >> faked PCI migration capability table communicates with Qemu to >> share migration status and mailbox irq vector index. >> >> Qemu will notify VF via sending MSIX msg to trigger mailbox >> vector during migration and store migration status in the >> PCI_VF_MIGRATION_VMM_STATUS regs in the new capability table. >> The mailbox irq will be triggered just befoe stop-and-copy stage >> and after migration on the target machine. >> >> VF driver will put down net when detect migration and tell >> Qemu it's ready for migration via writing PCI_VF_MIGRATION_VF_STATUS >> reg. After migration, put up net again. >> >> Qemu will in charge of migrating PCI config space regs and MSIX config. >> >> The patch is to dedicate on the normal case that net traffic works >> when mailbox irq is enabled. For other cases(such as the driver >> isn't loaded, adapter is suspended or closed), mailbox irq won't be >> triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP >> reg. These case will be resolved later. >> >> Signed-off-by: Lan Tianyu > > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? The tracking of dirty pages still needs to be addressed unless the interface is being downed before migration even starts which based on other comments I am assuming is not the case. I still feel that having a means of marking a page as being dirty when it is unmapped would be the best way to go. That way you only have to update the DMA API instead of messing with each and every driver trying to add code to force the page to be dirtied. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Duyck Subject: Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver Date: Tue, 24 Nov 2015 21:39:23 -0800 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: Lan Tianyu , a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org, Alex Williamson , b.reynal-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org, Bjorn Helgaas , Carolyn Wyborny , "Skidmore, Donald C" , eddie.dong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, nrupal.jani-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, Alexander Graf , kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Paolo Bonzini , qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org, "Tantilov, Emil S" , Or Gerlitz , "Rustad, Mark D" , Eric Auger , intel-wired-lan , Jeff Kirsher , "Brandeburg, Jesse" , "Ronciak, John" , linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" Return-path: In-Reply-To: <20151124230551-mutt-send-email-mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: netdev.vger.kernel.org On Tue, Nov 24, 2015 at 1:20 PM, Michael S. Tsirkin wrote: > On Tue, Nov 24, 2015 at 09:38:18PM +0800, Lan Tianyu wrote: >> This patch is to add migration support for ixgbevf driver. Using >> faked PCI migration capability table communicates with Qemu to >> share migration status and mailbox irq vector index. >> >> Qemu will notify VF via sending MSIX msg to trigger mailbox >> vector during migration and store migration status in the >> PCI_VF_MIGRATION_VMM_STATUS regs in the new capability table. >> The mailbox irq will be triggered just befoe stop-and-copy stage >> and after migration on the target machine. >> >> VF driver will put down net when detect migration and tell >> Qemu it's ready for migration via writing PCI_VF_MIGRATION_VF_STATUS >> reg. After migration, put up net again. >> >> Qemu will in charge of migrating PCI config space regs and MSIX config. >> >> The patch is to dedicate on the normal case that net traffic works >> when mailbox irq is enabled. For other cases(such as the driver >> isn't loaded, adapter is suspended or closed), mailbox irq won't be >> triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP >> reg. These case will be resolved later. >> >> Signed-off-by: Lan Tianyu > > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? The tracking of dirty pages still needs to be addressed unless the interface is being downed before migration even starts which based on other comments I am assuming is not the case. I still feel that having a means of marking a page as being dirty when it is unmapped would be the best way to go. That way you only have to update the DMA API instead of messing with each and every driver trying to add code to force the page to be dirtied. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Duyck Subject: Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver Date: Tue, 24 Nov 2015 21:39:23 -0800 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: In-Reply-To: <20151124230551-mutt-send-email-mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: "Michael S. Tsirkin" Cc: Lan Tianyu , a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org, Alex Williamson , b.reynal-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org, Bjorn Helgaas , Carolyn Wyborny , "Skidmore, Donald C" , eddie.dong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, nrupal.jani-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, Alexander Graf , kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Paolo Bonzini , qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org, "Tantilov, Emil S" , Or Gerlitz , "Rustad, Mark D" , Eric Auger , intel-wired-lan , Jeff Kirsher , "Brandeburg, Jesse" , "Ronciak, John" , linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: linux-api@vger.kernel.org On Tue, Nov 24, 2015 at 1:20 PM, Michael S. Tsirkin wrote: > On Tue, Nov 24, 2015 at 09:38:18PM +0800, Lan Tianyu wrote: >> This patch is to add migration support for ixgbevf driver. Using >> faked PCI migration capability table communicates with Qemu to >> share migration status and mailbox irq vector index. >> >> Qemu will notify VF via sending MSIX msg to trigger mailbox >> vector during migration and store migration status in the >> PCI_VF_MIGRATION_VMM_STATUS regs in the new capability table. >> The mailbox irq will be triggered just befoe stop-and-copy stage >> and after migration on the target machine. >> >> VF driver will put down net when detect migration and tell >> Qemu it's ready for migration via writing PCI_VF_MIGRATION_VF_STATUS >> reg. After migration, put up net again. >> >> Qemu will in charge of migrating PCI config space regs and MSIX config. >> >> The patch is to dedicate on the normal case that net traffic works >> when mailbox irq is enabled. For other cases(such as the driver >> isn't loaded, adapter is suspended or closed), mailbox irq won't be >> triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP >> reg. These case will be resolved later. >> >> Signed-off-by: Lan Tianyu > > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? The tracking of dirty pages still needs to be addressed unless the interface is being downed before migration even starts which based on other comments I am assuming is not the case. I still feel that having a means of marking a page as being dirty when it is unmapped would be the best way to go. That way you only have to update the DMA API instead of messing with each and every driver trying to add code to force the page to be dirtied. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Duyck Date: Tue, 24 Nov 2015 21:39:23 -0800 Subject: [Intel-wired-lan] [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver In-Reply-To: <20151124230551-mutt-send-email-mst@redhat.com> References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <1448372298-28386-4-git-send-email-tianyu.lan@intel.com> <20151124230551-mutt-send-email-mst@redhat.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: intel-wired-lan@osuosl.org List-ID: On Tue, Nov 24, 2015 at 1:20 PM, Michael S. Tsirkin wrote: > On Tue, Nov 24, 2015 at 09:38:18PM +0800, Lan Tianyu wrote: >> This patch is to add migration support for ixgbevf driver. Using >> faked PCI migration capability table communicates with Qemu to >> share migration status and mailbox irq vector index. >> >> Qemu will notify VF via sending MSIX msg to trigger mailbox >> vector during migration and store migration status in the >> PCI_VF_MIGRATION_VMM_STATUS regs in the new capability table. >> The mailbox irq will be triggered just befoe stop-and-copy stage >> and after migration on the target machine. >> >> VF driver will put down net when detect migration and tell >> Qemu it's ready for migration via writing PCI_VF_MIGRATION_VF_STATUS >> reg. After migration, put up net again. >> >> Qemu will in charge of migrating PCI config space regs and MSIX config. >> >> The patch is to dedicate on the normal case that net traffic works >> when mailbox irq is enabled. For other cases(such as the driver >> isn't loaded, adapter is suspended or closed), mailbox irq won't be >> triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP >> reg. These case will be resolved later. >> >> Signed-off-by: Lan Tianyu > > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? The tracking of dirty pages still needs to be addressed unless the interface is being downed before migration even starts which based on other comments I am assuming is not the case. I still feel that having a means of marking a page as being dirty when it is unmapped would be the best way to go. That way you only have to update the DMA API instead of messing with each and every driver trying to add code to force the page to be dirtied. - Alex