From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52933) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZUUAV-00062H-5X for qemu-devel@nongnu.org; Wed, 26 Aug 2015 02:26:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZUUAQ-0004Ge-5G for qemu-devel@nongnu.org; Wed, 26 Aug 2015 02:26:43 -0400 Received: from mx2.parallels.com ([199.115.105.18]:36940) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZUUAP-0004Fo-Vs for qemu-devel@nongnu.org; Wed, 26 Aug 2015 02:26:38 -0400 Message-ID: <55DD5C0C.7030209@virtuozzo.com> Date: Wed, 26 Aug 2015 09:26:20 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1433776886-27239-1-git-send-email-vsementsov@virtuozzo.com> <20150611200632.GA29013@stefanha-thinkpad.redhat.com> <557AAD4B.10805@openvz.org> <20150612103656.GD623@stefanha-thinkpad.redhat.com> In-Reply-To: <20150612103656.GD623@stefanha-thinkpad.redhat.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , "Denis V. Lunev" Cc: kwolf@redhat.com, jsnow@redhat.com, qemu-devel@nongnu.org, pbonzini@redhat.com On 12.06.2015 13:36, Stefan Hajnoczi wrote: > On Fri, Jun 12, 2015 at 12:58:35PM +0300, Denis V. Lunev wrote: >> On 11/06/15 23:06, Stefan Hajnoczi wrote: >>> The load/store API is not scalable when bitmaps are 1 MB or larger. >>> >>> For example, a 500 GB disk image with 64 KB granularity requires a 1 MB >>> bitmap. If a guest has several disk images of this size, then multiple >>> megabytes must be read to start the guest and written out to shut down >>> the guest. >>> >>> By comparison, the L1 table for the 500 GB disk image is less than 8 KB. >>> >>> I think something like qcow2-cache.c or metabitmaps should be used to >>> lazily read/write persistent bitmaps. That way only small portions need >>> to be read/written at a time. >>> >>> Stefan >> for the first iteration we could open the image, start tracking, >> read bitmap as one entity in the background and or read >> and collected data. >> >> partial read could be done in the next step > Making bitmap load/store fully lazy will require changes to the > load/store API, so it's worth thinking about a little upfront. > Otherwise there will be a lot of code churn when the fully lazy patches > are posted. As a reviewer it's in my interest to only spend time > reviewing the final version instead of code that gets thrown out :-), > but I understand. > > If you can make the read lazy to some extent that's a good start. That way we can improve load performance, but what about store? I see two solutions: 1) meta bitmaps (already mentioned) 2) Always (optionally?) have two bitmaps instead one: backing, which should be equal to the bitmap, already stored to the image, and active delta. This can be used instead of meta bitmaps in migration too. difference: with meta bitmaps we have double time overhead for writing to the bitmap (which is more often operations as I think), with second approach we have double overhead for read from the bitmap (but for backup, we can *or* these two bitmaps once, and it can be done fast, using the power of HBitmap). And of course double ram overhead.. -- Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.