From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43181) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZUWlo-0004iP-UE for qemu-devel@nongnu.org; Wed, 26 Aug 2015 05:13:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZUWln-0006Il-T1 for qemu-devel@nongnu.org; Wed, 26 Aug 2015 05:13:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60189) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZUWln-0006IT-OQ for qemu-devel@nongnu.org; Wed, 26 Aug 2015 05:13:23 -0400 Date: Wed, 26 Aug 2015 10:13:18 +0100 From: Stefan Hajnoczi Message-ID: <20150826091318.GB30715@stefanha-thinkpad.redhat.com> References: <1433776886-27239-1-git-send-email-vsementsov@virtuozzo.com> <20150611200632.GA29013@stefanha-thinkpad.redhat.com> <557AAD4B.10805@openvz.org> <20150612103656.GD623@stefanha-thinkpad.redhat.com> <55DD5C0C.7030209@virtuozzo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55DD5C0C.7030209@virtuozzo.com> Subject: Re: [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy Cc: kwolf@redhat.com, qemu-devel@nongnu.org, pbonzini@redhat.com, "Denis V. Lunev" , jsnow@redhat.com On Wed, Aug 26, 2015 at 09:26:20AM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 12.06.2015 13:36, Stefan Hajnoczi wrote: > >On Fri, Jun 12, 2015 at 12:58:35PM +0300, Denis V. Lunev wrote: > >>On 11/06/15 23:06, Stefan Hajnoczi wrote: > >>>The load/store API is not scalable when bitmaps are 1 MB or larger. > >>> > >>>For example, a 500 GB disk image with 64 KB granularity requires a 1 MB > >>>bitmap. If a guest has several disk images of this size, then multiple > >>>megabytes must be read to start the guest and written out to shut down > >>>the guest. > >>> > >>>By comparison, the L1 table for the 500 GB disk image is less than 8 KB. > >>> > >>>I think something like qcow2-cache.c or metabitmaps should be used to > >>>lazily read/write persistent bitmaps. That way only small portions need > >>>to be read/written at a time. > >>> > >>>Stefan > >>for the first iteration we could open the image, start tracking, > >>read bitmap as one entity in the background and or read > >>and collected data. > >> > >>partial read could be done in the next step > >Making bitmap load/store fully lazy will require changes to the > >load/store API, so it's worth thinking about a little upfront. > >Otherwise there will be a lot of code churn when the fully lazy patches > >are posted. As a reviewer it's in my interest to only spend time > >reviewing the final version instead of code that gets thrown out :-), > >but I understand. > > > >If you can make the read lazy to some extent that's a good start. > That way we can improve load performance, but what about store? > > I see two solutions: > 1) meta bitmaps (already mentioned) > 2) Always (optionally?) have two bitmaps instead one: backing, which should > be equal to the bitmap, already stored to the image, and active delta. This > can be used instead of meta bitmaps in migration too. > > difference: > with meta bitmaps we have double time overhead for writing to the bitmap > (which is more often operations as I think), > with second approach we have double overhead for read from the bitmap (but > for backup, we can *or* these two bitmaps once, and it can be done fast, > using the power of HBitmap). And of course double ram overhead.. Meta bitmaps seem like a good idea since they are already needed for live migration.