From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752192AbbKZDPn (ORCPT ); Wed, 25 Nov 2015 22:15:43 -0500 Received: from mga11.intel.com ([192.55.52.93]:8641 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751651AbbKZDPk (ORCPT ); Wed, 25 Nov 2015 22:15:40 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,345,1444719600"; d="scan'208";a="859593403" From: "Dong, Eddie" To: Alexander Duyck , "Lan, Tianyu" CC: "a.motakis@virtualopensystems.com" , Alex Williamson , "b.reynal@virtualopensystems.com" , "Bjorn Helgaas" , "Wyborny, Carolyn" , "Skidmore, Donald C" , "Jani, Nrupal" , Alexander Graf , "kvm@vger.kernel.org" , Paolo Bonzini , "qemu-devel@nongnu.org" , "Tantilov, Emil S" , "Or Gerlitz" , "Rustad, Mark D" , "Michael S. Tsirkin" , Eric Auger , intel-wired-lan , "Kirsher, Jeffrey T" , "Brandeburg, Jesse" , "Ronciak, John" , "linux-api@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "Vick, Matthew" , "Williams, Mitch A" , Netdev , "Nelson, Shannon" , Wei Yang , "zajec5@gmail.com" Subject: RE: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC Thread-Topic: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC Thread-Index: AQHRJ5Z/TMVKeo2E6U6s/FYSG47paJ6toQ8Q Date: Thu, 26 Nov 2015 03:15:35 +0000 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <5654722D.4010409@gmail.com> <56552888.90108@intel.com> <56556F98.5060507@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id tAQ3GEVb000995 > On Wed, Nov 25, 2015 at 12:21 AM, Lan Tianyu wrote: > > On 2015年11月25日 13:30, Alexander Duyck wrote: > >> No, what I am getting at is that you can't go around and modify the > >> configuration space for every possible device out there. This > >> solution won't scale. > > > > > > PCI config space regs are emulation by Qemu and so We can find the > > free PCI config space regs for the faked PCI capability. Its position > > can be not permanent. > > Yes, but do you really want to edit every driver on every OS that you plan to > support this on. What about things like direct assignment of regular Ethernet > ports? What you really need is a solution that will work generically on any > existing piece of hardware out there. The fundamental assumption of this patch series is to modify the driver in guest to self-emulate or track the device state, so that the migration may be possible. I don't think we can modify OS, without modifying the drivers, even using the PCIe hotplug mechanism. In the meantime, modifying Windows OS is a big challenge given that only Microsoft can do. While, modifying driver is relatively simple and manageable to device vendors, if the device vendor want to support state-clone based migration. Thx Eddie {.n++%ݶw{.n+{G{ayʇڙ,jfhz_(階ݢj"mG?&~iOzv^m ?I From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58755) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1n2A-0004gw-HR for qemu-devel@nongnu.org; Wed, 25 Nov 2015 22:15:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a1n25-0003pZ-Hv for qemu-devel@nongnu.org; Wed, 25 Nov 2015 22:15:46 -0500 Received: from mga03.intel.com ([134.134.136.65]:56417) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1n25-0003pK-80 for qemu-devel@nongnu.org; Wed, 25 Nov 2015 22:15:41 -0500 From: "Dong, Eddie" Date: Thu, 26 Nov 2015 03:15:35 +0000 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <5654722D.4010409@gmail.com> <56552888.90108@intel.com> <56556F98.5060507@intel.com> In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [Qemu-devel] [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexander Duyck , "Lan, Tianyu" Cc: Wei Yang , "Tantilov, Emil S" , "kvm@vger.kernel.org" , "Michael S. Tsirkin" , "qemu-devel@nongnu.org" , "Brandeburg, Jesse" , "Rustad, Mark D" , "Wyborny, Carolyn" , Eric Auger , "Skidmore, Donald C" , "zajec5@gmail.com" , Alexander Graf , "Vick, Matthew" , intel-wired-lan , "Kirsher, Jeffrey T" , Or Gerlitz , "Williams, Mitch A" , "Jani, Nrupal" , Bjorn Helgaas , "a.motakis@virtualopensystems.com" , "b.reynal@virtualopensystems.com" , "linux-api@vger.kernel.org" , "Nelson, Shannon" , Alex Williamson , "linux-kernel@vger.kernel.org" , "Ronciak, John" , Netdev , Paolo Bonzini PiBPbiBXZWQsIE5vdiAyNSwgMjAxNSBhdCAxMjoyMSBBTSwgTGFuIFRpYW55dSA8dGlhbnl1Lmxh bkBpbnRlbC5jb20+IHdyb3RlOg0KPiA+IE9uIDIwMTXlubQxMeaciDI15pelIDEzOjMwLCBBbGV4 YW5kZXIgRHV5Y2sgd3JvdGU6DQo+ID4+IE5vLCB3aGF0IEkgYW0gZ2V0dGluZyBhdCBpcyB0aGF0 IHlvdSBjYW4ndCBnbyBhcm91bmQgYW5kIG1vZGlmeSB0aGUNCj4gPj4gY29uZmlndXJhdGlvbiBz cGFjZSBmb3IgZXZlcnkgcG9zc2libGUgZGV2aWNlIG91dCB0aGVyZS4gIFRoaXMNCj4gPj4gc29s dXRpb24gd29uJ3Qgc2NhbGUuDQo+ID4NCj4gPg0KPiA+IFBDSSBjb25maWcgc3BhY2UgcmVncyBh cmUgZW11bGF0aW9uIGJ5IFFlbXUgYW5kIHNvIFdlIGNhbiBmaW5kIHRoZQ0KPiA+IGZyZWUgUENJ IGNvbmZpZyBzcGFjZSByZWdzIGZvciB0aGUgZmFrZWQgUENJIGNhcGFiaWxpdHkuIEl0cyBwb3Np dGlvbg0KPiA+IGNhbiBiZSBub3QgcGVybWFuZW50Lg0KPiANCj4gWWVzLCBidXQgZG8geW91IHJl YWxseSB3YW50IHRvIGVkaXQgZXZlcnkgZHJpdmVyIG9uIGV2ZXJ5IE9TIHRoYXQgeW91IHBsYW4g dG8NCj4gc3VwcG9ydCB0aGlzIG9uLiAgV2hhdCBhYm91dCB0aGluZ3MgbGlrZSBkaXJlY3QgYXNz aWdubWVudCBvZiByZWd1bGFyIEV0aGVybmV0DQo+IHBvcnRzPyAgV2hhdCB5b3UgcmVhbGx5IG5l ZWQgaXMgYSBzb2x1dGlvbiB0aGF0IHdpbGwgd29yayBnZW5lcmljYWxseSBvbiBhbnkNCj4gZXhp c3RpbmcgcGllY2Ugb2YgaGFyZHdhcmUgb3V0IHRoZXJlLg0KDQpUaGUgZnVuZGFtZW50YWwgYXNz dW1wdGlvbiBvZiB0aGlzIHBhdGNoIHNlcmllcyBpcyB0byBtb2RpZnkgdGhlIGRyaXZlciBpbiBn dWVzdCB0byBzZWxmLWVtdWxhdGUgb3IgdHJhY2sgdGhlIGRldmljZSBzdGF0ZSwgc28gdGhhdCB0 aGUgbWlncmF0aW9uIG1heSBiZSBwb3NzaWJsZS4NCkkgZG9uJ3QgdGhpbmsgd2UgY2FuIG1vZGlm eSBPUywgd2l0aG91dCBtb2RpZnlpbmcgdGhlIGRyaXZlcnMsIGV2ZW4gdXNpbmcgdGhlIFBDSWUg aG90cGx1ZyBtZWNoYW5pc20uICANCkluIHRoZSBtZWFudGltZSwgbW9kaWZ5aW5nIFdpbmRvd3Mg T1MgaXMgYSBiaWcgY2hhbGxlbmdlIGdpdmVuIHRoYXQgb25seSBNaWNyb3NvZnQgY2FuIGRvLiBX aGlsZSwgbW9kaWZ5aW5nIGRyaXZlciBpcyByZWxhdGl2ZWx5IHNpbXBsZSBhbmQgbWFuYWdlYWJs ZSB0byBkZXZpY2UgdmVuZG9ycywgaWYgdGhlIGRldmljZSB2ZW5kb3Igd2FudCB0byBzdXBwb3J0 IHN0YXRlLWNsb25lIGJhc2VkIG1pZ3JhdGlvbi4NCg0KVGh4IEVkZGllDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Dong, Eddie" Subject: RE: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC Date: Thu, 26 Nov 2015 03:15:35 +0000 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <5654722D.4010409@gmail.com> <56552888.90108@intel.com> <56556F98.5060507@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: "a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org" , Alex Williamson , "b.reynal-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org" , "Bjorn Helgaas" , "Wyborny, Carolyn" , "Skidmore, Donald C" , "Jani, Nrupal" , Alexander Graf , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Paolo Bonzini , "qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org" , "Tantilov, Emil S" , "Or Gerlitz" , "Rustad, Mark D" , "Michael S. Tsirkin" , Eric Auger , intel-wired-lan , "Kirsher, Jeffrey T" , "Brandeburg, Jesse" , "Lan, Tianyu" Return-path: In-Reply-To: Content-Language: en-US Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: netdev.vger.kernel.org PiBPbiBXZWQsIE5vdiAyNSwgMjAxNSBhdCAxMjoyMSBBTSwgTGFuIFRpYW55dSA8dGlhbnl1Lmxh bkBpbnRlbC5jb20+IHdyb3RlOg0KPiA+IE9uIDIwMTXlubQxMeaciDI15pelIDEzOjMwLCBBbGV4 YW5kZXIgRHV5Y2sgd3JvdGU6DQo+ID4+IE5vLCB3aGF0IEkgYW0gZ2V0dGluZyBhdCBpcyB0aGF0 IHlvdSBjYW4ndCBnbyBhcm91bmQgYW5kIG1vZGlmeSB0aGUNCj4gPj4gY29uZmlndXJhdGlvbiBz cGFjZSBmb3IgZXZlcnkgcG9zc2libGUgZGV2aWNlIG91dCB0aGVyZS4gIFRoaXMNCj4gPj4gc29s dXRpb24gd29uJ3Qgc2NhbGUuDQo+ID4NCj4gPg0KPiA+IFBDSSBjb25maWcgc3BhY2UgcmVncyBh cmUgZW11bGF0aW9uIGJ5IFFlbXUgYW5kIHNvIFdlIGNhbiBmaW5kIHRoZQ0KPiA+IGZyZWUgUENJ IGNvbmZpZyBzcGFjZSByZWdzIGZvciB0aGUgZmFrZWQgUENJIGNhcGFiaWxpdHkuIEl0cyBwb3Np dGlvbg0KPiA+IGNhbiBiZSBub3QgcGVybWFuZW50Lg0KPiANCj4gWWVzLCBidXQgZG8geW91IHJl YWxseSB3YW50IHRvIGVkaXQgZXZlcnkgZHJpdmVyIG9uIGV2ZXJ5IE9TIHRoYXQgeW91IHBsYW4g dG8NCj4gc3VwcG9ydCB0aGlzIG9uLiAgV2hhdCBhYm91dCB0aGluZ3MgbGlrZSBkaXJlY3QgYXNz aWdubWVudCBvZiByZWd1bGFyIEV0aGVybmV0DQo+IHBvcnRzPyAgV2hhdCB5b3UgcmVhbGx5IG5l ZWQgaXMgYSBzb2x1dGlvbiB0aGF0IHdpbGwgd29yayBnZW5lcmljYWxseSBvbiBhbnkNCj4gZXhp c3RpbmcgcGllY2Ugb2YgaGFyZHdhcmUgb3V0IHRoZXJlLg0KDQpUaGUgZnVuZGFtZW50YWwgYXNz dW1wdGlvbiBvZiB0aGlzIHBhdGNoIHNlcmllcyBpcyB0byBtb2RpZnkgdGhlIGRyaXZlciBpbiBn dWVzdCB0byBzZWxmLWVtdWxhdGUgb3IgdHJhY2sgdGhlIGRldmljZSBzdGF0ZSwgc28gdGhhdCB0 aGUgbWlncmF0aW9uIG1heSBiZSBwb3NzaWJsZS4NCkkgZG9uJ3QgdGhpbmsgd2UgY2FuIG1vZGlm eSBPUywgd2l0aG91dCBtb2RpZnlpbmcgdGhlIGRyaXZlcnMsIGV2ZW4gdXNpbmcgdGhlIFBDSWUg aG90cGx1ZyBtZWNoYW5pc20uICANCkluIHRoZSBtZWFudGltZSwgbW9kaWZ5aW5nIFdpbmRvd3Mg T1MgaXMgYSBiaWcgY2hhbGxlbmdlIGdpdmVuIHRoYXQgb25seSBNaWNyb3NvZnQgY2FuIGRvLiBX aGlsZSwgbW9kaWZ5aW5nIGRyaXZlciBpcyByZWxhdGl2ZWx5IHNpbXBsZSBhbmQgbWFuYWdlYWJs ZSB0byBkZXZpY2UgdmVuZG9ycywgaWYgdGhlIGRldmljZSB2ZW5kb3Igd2FudCB0byBzdXBwb3J0 IHN0YXRlLWNsb25lIGJhc2VkIG1pZ3JhdGlvbi4NCg0KVGh4IEVkZGllDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Dong, Eddie" Subject: RE: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC Date: Thu, 26 Nov 2015 03:15:35 +0000 Message-ID: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <5654722D.4010409@gmail.com> <56552888.90108@intel.com> <56556F98.5060507@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: Content-Language: en-US Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Alexander Duyck , "Lan, Tianyu" Cc: "a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org" , Alex Williamson , "b.reynal-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org" , Bjorn Helgaas , "Wyborny, Carolyn" , "Skidmore, Donald C" , "Jani, Nrupal" , Alexander Graf , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Paolo Bonzini , "qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org" , "Tantilov, Emil S" , Or Gerlitz , "Rustad, Mark D" , "Michael S. Tsirkin" , Eric Auger , intel-wired-lan , "Kirsher, Jeffrey T" , "Brandeburg, Jesse" List-Id: linux-api@vger.kernel.org PiBPbiBXZWQsIE5vdiAyNSwgMjAxNSBhdCAxMjoyMSBBTSwgTGFuIFRpYW55dSA8dGlhbnl1Lmxh bkBpbnRlbC5jb20+IHdyb3RlOg0KPiA+IE9uIDIwMTXlubQxMeaciDI15pelIDEzOjMwLCBBbGV4 YW5kZXIgRHV5Y2sgd3JvdGU6DQo+ID4+IE5vLCB3aGF0IEkgYW0gZ2V0dGluZyBhdCBpcyB0aGF0 IHlvdSBjYW4ndCBnbyBhcm91bmQgYW5kIG1vZGlmeSB0aGUNCj4gPj4gY29uZmlndXJhdGlvbiBz cGFjZSBmb3IgZXZlcnkgcG9zc2libGUgZGV2aWNlIG91dCB0aGVyZS4gIFRoaXMNCj4gPj4gc29s dXRpb24gd29uJ3Qgc2NhbGUuDQo+ID4NCj4gPg0KPiA+IFBDSSBjb25maWcgc3BhY2UgcmVncyBh cmUgZW11bGF0aW9uIGJ5IFFlbXUgYW5kIHNvIFdlIGNhbiBmaW5kIHRoZQ0KPiA+IGZyZWUgUENJ IGNvbmZpZyBzcGFjZSByZWdzIGZvciB0aGUgZmFrZWQgUENJIGNhcGFiaWxpdHkuIEl0cyBwb3Np dGlvbg0KPiA+IGNhbiBiZSBub3QgcGVybWFuZW50Lg0KPiANCj4gWWVzLCBidXQgZG8geW91IHJl YWxseSB3YW50IHRvIGVkaXQgZXZlcnkgZHJpdmVyIG9uIGV2ZXJ5IE9TIHRoYXQgeW91IHBsYW4g dG8NCj4gc3VwcG9ydCB0aGlzIG9uLiAgV2hhdCBhYm91dCB0aGluZ3MgbGlrZSBkaXJlY3QgYXNz aWdubWVudCBvZiByZWd1bGFyIEV0aGVybmV0DQo+IHBvcnRzPyAgV2hhdCB5b3UgcmVhbGx5IG5l ZWQgaXMgYSBzb2x1dGlvbiB0aGF0IHdpbGwgd29yayBnZW5lcmljYWxseSBvbiBhbnkNCj4gZXhp c3RpbmcgcGllY2Ugb2YgaGFyZHdhcmUgb3V0IHRoZXJlLg0KDQpUaGUgZnVuZGFtZW50YWwgYXNz dW1wdGlvbiBvZiB0aGlzIHBhdGNoIHNlcmllcyBpcyB0byBtb2RpZnkgdGhlIGRyaXZlciBpbiBn dWVzdCB0byBzZWxmLWVtdWxhdGUgb3IgdHJhY2sgdGhlIGRldmljZSBzdGF0ZSwgc28gdGhhdCB0 aGUgbWlncmF0aW9uIG1heSBiZSBwb3NzaWJsZS4NCkkgZG9uJ3QgdGhpbmsgd2UgY2FuIG1vZGlm eSBPUywgd2l0aG91dCBtb2RpZnlpbmcgdGhlIGRyaXZlcnMsIGV2ZW4gdXNpbmcgdGhlIFBDSWUg aG90cGx1ZyBtZWNoYW5pc20uICANCkluIHRoZSBtZWFudGltZSwgbW9kaWZ5aW5nIFdpbmRvd3Mg T1MgaXMgYSBiaWcgY2hhbGxlbmdlIGdpdmVuIHRoYXQgb25seSBNaWNyb3NvZnQgY2FuIGRvLiBX aGlsZSwgbW9kaWZ5aW5nIGRyaXZlciBpcyByZWxhdGl2ZWx5IHNpbXBsZSBhbmQgbWFuYWdlYWJs ZSB0byBkZXZpY2UgdmVuZG9ycywgaWYgdGhlIGRldmljZSB2ZW5kb3Igd2FudCB0byBzdXBwb3J0 IHN0YXRlLWNsb25lIGJhc2VkIG1pZ3JhdGlvbi4NCg0KVGh4IEVkZGllDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dong, Eddie Date: Thu, 26 Nov 2015 03:15:35 +0000 Subject: [Intel-wired-lan] [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC In-Reply-To: References: <1448372298-28386-1-git-send-email-tianyu.lan@intel.com> <5654722D.4010409@gmail.com> <56552888.90108@intel.com> <56556F98.5060507@intel.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: intel-wired-lan@osuosl.org List-ID: > On Wed, Nov 25, 2015 at 12:21 AM, Lan Tianyu wrote: > > On 2015?11?25? 13:30, Alexander Duyck wrote: > >> No, what I am getting at is that you can't go around and modify the > >> configuration space for every possible device out there. This > >> solution won't scale. > > > > > > PCI config space regs are emulation by Qemu and so We can find the > > free PCI config space regs for the faked PCI capability. Its position > > can be not permanent. > > Yes, but do you really want to edit every driver on every OS that you plan to > support this on. What about things like direct assignment of regular Ethernet > ports? What you really need is a solution that will work generically on any > existing piece of hardware out there. The fundamental assumption of this patch series is to modify the driver in guest to self-emulate or track the device state, so that the migration may be possible. I don't think we can modify OS, without modifying the drivers, even using the PCIe hotplug mechanism. In the meantime, modifying Windows OS is a big challenge given that only Microsoft can do. While, modifying driver is relatively simple and manageable to device vendors, if the device vendor want to support state-clone based migration. Thx Eddie