From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZK0s5-00051O-Lh for qemu-devel@nongnu.org; Tue, 28 Jul 2015 05:08:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZK0s2-0004gD-GA for qemu-devel@nongnu.org; Tue, 28 Jul 2015 05:08:25 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53743) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZK0s2-0004fx-8K for qemu-devel@nongnu.org; Tue, 28 Jul 2015 05:08:22 -0400 Date: Tue, 28 Jul 2015 10:08:15 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20150728090815.GC2247@work-vm> References: <1434450415-11339-1-git-send-email-dgilbert@redhat.com> <1434450415-11339-43-git-send-email-dgilbert@redhat.com> <20150728061547.GL12267@grmbl.mre> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150728061547.GL12267@grmbl.mre> Subject: Re: [Qemu-devel] [PATCH v7 42/42] Inhibit ballooning during postcopy List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Amit Shah Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, liang.z.li@intel.com, qemu-devel@nongnu.org, luis@cs.umu.se, pbonzini@redhat.com, david@gibson.dropbear.id.au * Amit Shah (amit.shah@redhat.com) wrote: > On (Tue) 16 Jun 2015 [11:26:55], Dr. David Alan Gilbert (git) wrote: > > From: "Dr. David Alan Gilbert" > > > > Postcopy detects accesses to pages that haven't been transferred yet > > using userfaultfd, and it causes exceptions on pages that are 'not > > present'. > > Ballooning also causes pages to be marked as 'not present' when the > > guest inflates the balloon. > > Potentially a balloon could be inflated to discard pages that are > > currently inflight during postcopy and that may be arriving at about > > the same time. > > > > To avoid this confusion, disable ballooning during postcopy. > > > > When disabled we drop balloon requests from the guest. Since ballooning > > is generally initiated by the host, the management system should avoid > > initiating any balloon instructions to the guest during migration, > > although it's not possible to know how long it would take a guest to > > process a request made prior to the start of migration. > > > > Queueing the requests until after migration would be nice, but is > > non-trivial, since the set of inflate/deflate requests have to > > be compared with the state of the page to know what the final > > outcome is allowed to be. > > I didn't track the previous discussion, but there were plans to have > guest-initiated balloon requests for cases where the guest wants to > co-operate with hosts and return any free mem available We don't > currently have guests that do this, but we also don't want to have a > dependency between the host and guest -- they should be independent. > > This approach here seems the simplest possible, short of maintaining > another bitmap for the duration of postcopy which indicates > guest-freed memory pages which postcopy should not populate, after > receiving them at the dest (this sounds better to me than queuing up > guest requests). > > The downside here is that the guest offered some memory back, and we > don't use it. The guest also doesn't use it -- so it's a double loss, > of sorts. > > Thoughts? I don't have a problem with this current approach, but if > we could get something better, that'll be good too. It needs something like that bitmap, but it would take quite a bit of care to manage the interaction between: a) The guest emitting balloon notifications b) Pages being received from the source c) Destination use of that page we also have to think what to do with a page that's been ballooned after reception of the source page; the madvise(dontneed) that's used normally would cause userfault to fire again, and we can't allow that. (We could make it the same as receiving a zero page). But then we would also have to cope with the source sending us a page after the destination has ballooned it and make sure to discard that (I suspect there are further ordering examples that have to also be considered). Dave > > Amit -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK