From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38478) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2Mw2-0004Cq-Ls for qemu-devel@nongnu.org; Tue, 09 Jun 2015 13:03:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z2Mvw-0003AP-Ld for qemu-devel@nongnu.org; Tue, 09 Jun 2015 13:03:34 -0400 Received: from mail-wi0-x22d.google.com ([2a00:1450:400c:c05::22d]:33040) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z2Mvw-0003AI-7l for qemu-devel@nongnu.org; Tue, 09 Jun 2015 13:03:28 -0400 Received: by wiwd19 with SMTP id d19so24743294wiw.0 for ; Tue, 09 Jun 2015 10:03:27 -0700 (PDT) Date: Tue, 9 Jun 2015 18:03:25 +0100 From: Stefan Hajnoczi Message-ID: <20150609170325.GI3181@stefanha-thinkpad.redhat.com> References: <1433776886-27239-1-git-send-email-vsementsov@virtuozzo.com> <1433776886-27239-2-git-send-email-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rCb8EA+9TsBVtA92" Content-Disposition: inline In-Reply-To: <1433776886-27239-2-git-send-email-vsementsov@virtuozzo.com> Subject: Re: [Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy Cc: kwolf@redhat.com, qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy , stefanha@redhat.com, den@openvz.org, pbonzini@redhat.com, jsnow@redhat.com --rCb8EA+9TsBVtA92 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 08, 2015 at 06:21:19PM +0300, Vladimir Sementsov-Ogievskiy wrot= e: > From: Vladimir Sementsov-Ogievskiy >=20 > Persistent dirty bitmaps will be saved into qcow2 files. It may be used > as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for > other drives (there may be qcow2 file with zero disk size but with > several dirty bitmaps for other drives). >=20 > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > docs/specs/qcow2.txt | 66 ++++++++++++++++++++++++++++++++++++++++++++++= ++++++ > 1 file changed, 66 insertions(+) >=20 > diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt > index 121dfc8..0fffba2 100644 > --- a/docs/specs/qcow2.txt > +++ b/docs/specs/qcow2.txt > @@ -123,6 +123,7 @@ be stored. Each extension has a structure like the fo= llowing: > 0x00000000 - End of the header extension area > 0xE2792ACA - Backing file format name > 0x6803f857 - Feature name table > + 0x23852875 - Dirty bitmaps > other - Unknown header extension, can be sa= fely > ignored > =20 > @@ -166,6 +167,19 @@ the header extension data. Each entry look like this: > terminated if it has full length) > =20 > =20 > +=3D=3D Dirty bitmaps =3D=3D > + > +Dirty bitmaps is an optional header extension. It provides a possibility= of > +storing dirty bitmaps in qcow2 image. The fields are: > + > + 0 - 3: nb_dirty_bitmaps > + Number of dirty bitmaps contained in the image Is there a maximum? > + > + 4 - 11: dirty_bitmaps_offset > + Offset into the image file at which the dirty bitmaps= table > + starts. Must be aligned to a cluster boundary. The autoclear feature bit is undocumented. > =3D=3D Host cluster management =3D=3D > =20 > qcow2 manages the allocation of host clusters by maintaining a reference= count > @@ -360,3 +374,55 @@ Snapshot table entry: > =20 > variable: Padding to round up the snapshot table entry size to= the > next multiple of 8. > + > + > +=3D=3D Dirty bitmaps =3D=3D > + > +The feature supports storing several dirty bitmaps in the qcow2 file. > + > +=3D=3D=3D Cluster mapping =3D=3D=3D > + > +Dirty bitmaps are stored using a ONE-level structure for the mapping of > +bitmaps to host clusters. There is only an L1 table. > + > +The L1 table has a variable size (stored in the Bitmap table entry) and = may > +use multiple clusters, however it must be contiguous in the image file. The use of "L1 table" could be confusing. The refcount metadata uses "refcount table" and "refcount block" to describe a one-level table. > + > +Given an offset into the bitmap, the offset into the image file can be > +obtained as follows: > + > + offset =3D l1_table[offset / cluster_size] + (offset % cluster_size) It might help to add granularity to this formula. Instead of "offset", "bit_number" or "bitnr" might be clearer since "offset" means something different in other parts of the document. > + > +L1 table entry: > + > + Bit 0 - 61: Standard cluster descriptor > + > + 62 - 63: Reserved Do you really want to use the standard cluster descriptor with it's zero bit? Since bitmaps don't honor backing files there doesn't seem much point in using the zero bit, things are simpler if just bits 9-55 are contain the host cluster offset and 0 means the cluster is unallocated. By honoring the zero bit there are three states: 1. Zero bit set, read zeroes 2. Zero bit not set, host cluster offset !=3D 0, bits valid 3. Zero bit not set, host cluster offset =3D=3D 0, unallocated State 1 is not useful. > +=3D=3D=3D Bitmap table =3D=3D=3D > + > +A directory of all bitmaps is stored in the bitmap table, a contiguous a= rea in > +the image file, whose starting offset and length are given by the header= fields > +dirty_bitmaps_offset and nb_dirty_bitmaps. The entries of the bitmap tab= le have > +variable length, depending on the length of name and extra data. > + > +Bitmap table entry: > + > + Byte 0 - 7: Offset into the image file at which the L1 table for= the > + bitmap starts. Must be aligned to a cluster boundary. > + > + 8 - 11: Number of entries in the L1 table of the bitmap > + > + 12 - 15: Bitmap granularity in bytes > + > + 16 - 23: Bitmap size in sectors > + > + 24 - 25: Size of the bitmap name > + > + variable: The name of the bitmap (not null terminated) > + > + variable: Padding to round up the bitmap table entry size to t= he > + next multiple of 8. > + > +The fields "size", "granularity" and "name" are corresponding with the f= ields > +in struct BdrvDirtyBitmap. Referring to the internals of a C struct in QEMU is not appropriate for a file format specification. Please document the fields fully including their constraints, minimums, maximums, etc. --rCb8EA+9TsBVtA92 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVdxxdAAoJEJykq7OBq3PIeywIAJhR9vrVfhFEu4VVnaB/gFy9 oicO+Jtoi+FRkjIB4PxzA0pryFxo4Pin61u8LXjTja4OAKrJM9LOW0T7nepUCqeA lPzLO70vQ3F+TUwHYPr+Ma9nedQ5jCg3lqAZ++zkLF3kYaMWGKbH8ROdQvCmHLIV KXBl9WaP+y9ubnSNuP/zhNq9AHG3MCnGzWurXtZocki6aGQAzDSUm3yQQ26Kpktv 0fjomn1JzMvkvzJFZTR0pM+STvGZ0Vn3YAJj7MDh3T4nxam2IbQj92uG5t62vXCT uQrkR2XlzIxMTd+cRpNusCc0eu2by+dm0nJym02hCCYVLIsseLdXxA2l3bKhr5g= =tH0S -----END PGP SIGNATURE----- --rCb8EA+9TsBVtA92--