From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36919) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2i12-0001BW-2J for qemu-devel@nongnu.org; Wed, 10 Jun 2015 11:34:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z2i0y-0005oB-RC for qemu-devel@nongnu.org; Wed, 10 Jun 2015 11:34:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41089) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2i0y-0005mu-JV for qemu-devel@nongnu.org; Wed, 10 Jun 2015 11:34:04 -0400 Date: Wed, 10 Jun 2015 17:34:00 +0200 From: Kevin Wolf Message-ID: <20150610153400.GG4899@noname.str.redhat.com> References: <1433776886-27239-1-git-send-email-vsementsov@virtuozzo.com> <1433776886-27239-2-git-send-email-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1433776886-27239-2-git-send-email-vsementsov@virtuozzo.com> Subject: Re: [Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy Cc: qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy , stefanha@redhat.com, pbonzini@redhat.com, den@openvz.org, jsnow@redhat.com Am 08.06.2015 um 17:21 hat Vladimir Sementsov-Ogievskiy geschrieben: > From: Vladimir Sementsov-Ogievskiy > > Persistent dirty bitmaps will be saved into qcow2 files. It may be used > as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for > other drives (there may be qcow2 file with zero disk size but with > several dirty bitmaps for other drives). > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > docs/specs/qcow2.txt | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 66 insertions(+) > > diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt > index 121dfc8..0fffba2 100644 > --- a/docs/specs/qcow2.txt > +++ b/docs/specs/qcow2.txt > @@ -123,6 +123,7 @@ be stored. Each extension has a structure like the following: > 0x00000000 - End of the header extension area > 0xE2792ACA - Backing file format name > 0x6803f857 - Feature name table > + 0x23852875 - Dirty bitmaps > other - Unknown header extension, can be safely > ignored > > @@ -166,6 +167,19 @@ the header extension data. Each entry look like this: > terminated if it has full length) > > > +== Dirty bitmaps == > + > +Dirty bitmaps is an optional header extension. It provides a possibility of > +storing dirty bitmaps in qcow2 image. The fields are: > + > + 0 - 3: nb_dirty_bitmaps > + Number of dirty bitmaps contained in the image > + > + 4 - 11: dirty_bitmaps_offset > + Offset into the image file at which the dirty bitmaps table > + starts. Must be aligned to a cluster boundary. > + > + > == Host cluster management == You need to use a compatibility flag because for old qemu versions, the dirty bitmaps (and associated metadata) are leaked clusters and qemu-img check would "repair" them by resetting the refcount to 0. At second sight, I see that your patches add an autoclear flag. Presumably the contents of the dirty bitmaps is outdated when you accessed the image with an older version, so this seems right. We just need to document it. > qcow2 manages the allocation of host clusters by maintaining a reference count > @@ -360,3 +374,55 @@ Snapshot table entry: > > variable: Padding to round up the snapshot table entry size to the > next multiple of 8. > + > + > +== Dirty bitmaps == > + > +The feature supports storing several dirty bitmaps in the qcow2 file. > + > +=== Cluster mapping === > + > +Dirty bitmaps are stored using a ONE-level structure for the mapping of > +bitmaps to host clusters. There is only an L1 table. > + > +The L1 table has a variable size (stored in the Bitmap table entry) and may > +use multiple clusters, however it must be contiguous in the image file. > + > +Given an offset into the bitmap, the offset into the image file can be > +obtained as follows: > + > + offset = l1_table[offset / cluster_size] + (offset % cluster_size) > + > +L1 table entry: > + > + Bit 0 - 61: Standard cluster descriptor > + > + 62 - 63: Reserved Stefan already mentioned that we don't have a "L1" when there is only one level, and that you shouldn't reuse the cluster descriptors from L2 tables. > +=== Bitmap table === > + > +A directory of all bitmaps is stored in the bitmap table, a contiguous area in > +the image file, whose starting offset and length are given by the header fields > +dirty_bitmaps_offset and nb_dirty_bitmaps. The entries of the bitmap table have > +variable length, depending on the length of name and extra data. > + > +Bitmap table entry: > + > + Byte 0 - 7: Offset into the image file at which the L1 table for the > + bitmap starts. Must be aligned to a cluster boundary. > + > + 8 - 11: Number of entries in the L1 table of the bitmap Worth using 64 bits here? This can only cover 4 * 512 GB = 2 TB for the smallest possible cluster size. Though it's 65536 * 512 = 32 PB for the default, which might be enough for a while. > + 12 - 15: Bitmap granularity in bytes > + > + 16 - 23: Bitmap size in sectors Please don't use sectors, that's a meaningless unit. Bytes is better. > + 24 - 25: Size of the bitmap name We should use a smaller limit than the possible 64k to avoid too large memory allocations. Nobody needs really long bitmap names. > + > + variable: The name of the bitmap (not null terminated) > + > + variable: Padding to round up the bitmap table entry size to the > + next multiple of 8. Kevin