From: Stefan Hajnoczi <stefanha@redhat.com>
To: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Cc: kwolf@redhat.com, qemu-devel@nongnu.org, pbonzini@redhat.com,
"Denis V. Lunev" <den@openvz.org>,
jsnow@redhat.com
Subject: Re: [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps
Date: Wed, 26 Aug 2015 10:13:18 +0100 [thread overview]
Message-ID: <20150826091318.GB30715@stefanha-thinkpad.redhat.com> (raw)
In-Reply-To: <55DD5C0C.7030209@virtuozzo.com>
On Wed, Aug 26, 2015 at 09:26:20AM +0300, Vladimir Sementsov-Ogievskiy wrote:
> On 12.06.2015 13:36, Stefan Hajnoczi wrote:
> >On Fri, Jun 12, 2015 at 12:58:35PM +0300, Denis V. Lunev wrote:
> >>On 11/06/15 23:06, Stefan Hajnoczi wrote:
> >>>The load/store API is not scalable when bitmaps are 1 MB or larger.
> >>>
> >>>For example, a 500 GB disk image with 64 KB granularity requires a 1 MB
> >>>bitmap. If a guest has several disk images of this size, then multiple
> >>>megabytes must be read to start the guest and written out to shut down
> >>>the guest.
> >>>
> >>>By comparison, the L1 table for the 500 GB disk image is less than 8 KB.
> >>>
> >>>I think something like qcow2-cache.c or metabitmaps should be used to
> >>>lazily read/write persistent bitmaps. That way only small portions need
> >>>to be read/written at a time.
> >>>
> >>>Stefan
> >>for the first iteration we could open the image, start tracking,
> >>read bitmap as one entity in the background and or read
> >>and collected data.
> >>
> >>partial read could be done in the next step
> >Making bitmap load/store fully lazy will require changes to the
> >load/store API, so it's worth thinking about a little upfront.
> >Otherwise there will be a lot of code churn when the fully lazy patches
> >are posted. As a reviewer it's in my interest to only spend time
> >reviewing the final version instead of code that gets thrown out :-),
> >but I understand.
> >
> >If you can make the read lazy to some extent that's a good start.
> That way we can improve load performance, but what about store?
>
> I see two solutions:
> 1) meta bitmaps (already mentioned)
> 2) Always (optionally?) have two bitmaps instead one: backing, which should
> be equal to the bitmap, already stored to the image, and active delta. This
> can be used instead of meta bitmaps in migration too.
>
> difference:
> with meta bitmaps we have double time overhead for writing to the bitmap
> (which is more often operations as I think),
> with second approach we have double overhead for read from the bitmap (but
> for backup, we can *or* these two bitmaps once, and it can be done fast,
> using the power of HBitmap). And of course double ram overhead..
Meta bitmaps seem like a good idea since they are already needed for
live migration.
next prev parent reply other threads:[~2015-08-26 9:13 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-08 15:21 [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps Vladimir Sementsov-Ogievskiy
2015-06-08 15:21 ` [Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification Vladimir Sementsov-Ogievskiy
2015-06-09 16:01 ` John Snow
2015-06-09 17:03 ` Stefan Hajnoczi
2015-06-10 8:19 ` Vladimir Sementsov-Ogievskiy
2015-06-10 8:49 ` Vladimir Sementsov-Ogievskiy
2015-06-10 13:00 ` Eric Blake
2015-06-11 10:16 ` Vladimir Sementsov-Ogievskiy
2015-06-10 13:24 ` Stefan Hajnoczi
2015-06-11 10:19 ` Vladimir Sementsov-Ogievskiy
2015-06-11 13:03 ` Stefan Hajnoczi
2015-06-11 16:21 ` John Snow
2015-06-12 10:28 ` Stefan Hajnoczi
2015-06-12 15:19 ` John Snow
2015-06-10 15:34 ` Kevin Wolf
2015-06-11 10:25 ` Vladimir Sementsov-Ogievskiy
2015-06-11 16:30 ` John Snow
2015-06-12 8:33 ` Kevin Wolf
2015-08-24 10:46 ` Vladimir Sementsov-Ogievskiy
2015-08-24 13:30 ` Vladimir Sementsov-Ogievskiy
2015-08-24 14:08 ` Vladimir Sementsov-Ogievskiy
2015-08-24 14:04 ` Vladimir Sementsov-Ogievskiy
2015-08-31 22:21 ` Eric Blake
2015-08-31 22:24 ` John Snow
2015-06-08 15:21 ` [Qemu-devel] [PATCH 2/8] qcow2: add dirty-bitmaps feature Vladimir Sementsov-Ogievskiy
2015-06-09 16:52 ` Stefan Hajnoczi
2015-06-10 14:30 ` Stefan Hajnoczi
2015-06-12 19:02 ` John Snow
2015-06-15 14:42 ` Stefan Hajnoczi
2015-06-23 17:57 ` John Snow
2015-06-24 9:39 ` Stefan Hajnoczi
2015-08-14 17:14 ` Vladimir Sementsov-Ogievskiy
2015-08-26 9:09 ` Stefan Hajnoczi
2015-06-11 23:04 ` John Snow
2015-06-15 14:05 ` Vladimir Sementsov-Ogievskiy
2015-06-15 16:53 ` John Snow
2015-06-12 21:55 ` John Snow
2015-08-26 13:15 ` Vladimir Sementsov-Ogievskiy
2015-08-26 14:14 ` Vladimir Sementsov-Ogievskiy
2015-08-27 12:43 ` Vladimir Sementsov-Ogievskiy
2015-06-08 15:21 ` [Qemu-devel] [PATCH 3/8] block: store persistent dirty bitmaps Vladimir Sementsov-Ogievskiy
2015-06-08 15:21 ` [Qemu-devel] [PATCH 4/8] block: add bdrv_load_dirty_bitmap Vladimir Sementsov-Ogievskiy
2015-06-09 16:01 ` Stefan Hajnoczi
2015-06-10 22:33 ` John Snow
2015-06-11 10:41 ` Vladimir Sementsov-Ogievskiy
2015-06-08 15:21 ` [Qemu-devel] [PATCH 5/8] qcow2: add qcow2_dirty_bitmap_delete_all Vladimir Sementsov-Ogievskiy
2015-06-08 15:21 ` [Qemu-devel] [PATCH 6/8] qcow2: add autoclear bit for dirty bitmaps Vladimir Sementsov-Ogievskiy
2015-06-09 15:49 ` Stefan Hajnoczi
2015-06-09 15:50 ` Stefan Hajnoczi
2015-08-27 7:45 ` Vladimir Sementsov-Ogievskiy
2015-08-31 11:06 ` Vladimir Sementsov-Ogievskiy
2015-08-31 22:39 ` Eric Blake
2015-08-31 22:50 ` Eric Blake
2015-06-10 23:42 ` John Snow
2015-06-11 8:35 ` Kevin Wolf
2015-06-11 10:49 ` Vladimir Sementsov-Ogievskiy
2015-06-11 16:36 ` John Snow
2015-06-08 15:21 ` [Qemu-devel] [PATCH 7/8] qemu: command line option " Vladimir Sementsov-Ogievskiy
2015-06-11 20:57 ` John Snow
2015-06-12 21:49 ` John Snow
2015-06-08 15:21 ` [Qemu-devel] [PATCH 8/8] iotests: test internal persistent dirty bitmap Vladimir Sementsov-Ogievskiy
2015-06-09 16:17 ` Eric Blake
2015-06-10 15:27 ` [Qemu-devel] [PATCH v2 RFC 0/8] block: persistent dirty bitmaps Stefan Hajnoczi
2015-06-11 11:22 ` Vladimir Sementsov-Ogievskiy
2015-06-11 13:14 ` Stefan Hajnoczi
2015-06-11 20:06 ` Stefan Hajnoczi
2015-06-12 9:58 ` Denis V. Lunev
2015-06-12 10:36 ` Stefan Hajnoczi
2015-08-26 6:26 ` Vladimir Sementsov-Ogievskiy
2015-08-26 9:13 ` Stefan Hajnoczi [this message]
2015-06-12 19:34 ` John Snow
2015-06-17 14:29 ` Vladimir Sementsov-Ogievskiy
2015-06-24 0:21 ` John Snow
2015-07-08 12:24 ` Vladimir Sementsov-Ogievskiy
2015-07-08 15:21 ` John Snow
2015-08-27 10:08 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150826091318.GB30715@stefanha-thinkpad.redhat.com \
--to=stefanha@redhat.com \
--cc=den@openvz.org \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.