From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53070) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z4o5B-0005Rw-TM for qemu-devel@nongnu.org; Tue, 16 Jun 2015 06:27:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z4o58-0000VK-AS for qemu-devel@nongnu.org; Tue, 16 Jun 2015 06:27:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54771) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z4o58-0000Uz-3s for qemu-devel@nongnu.org; Tue, 16 Jun 2015 06:27:02 -0400 From: "Dr. David Alan Gilbert (git)" Date: Tue, 16 Jun 2015 11:26:13 +0100 Message-Id: <1434450415-11339-1-git-send-email-dgilbert@redhat.com> Subject: [Qemu-devel] [PATCH v7 00/42] Postcopy implementation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, liang.z.li@intel.com, luis@cs.umu.se, amit.shah@redhat.com, pbonzini@redhat.com, david@gibson.dropbear.id.au From: "Dr. David Alan Gilbert" This is the 7th cut of my version of postcopy; it is designed for use with the Linux kernel additions posted by Andrea Arcangeli here: git clone --reference linux -b userfault21 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git Note this API is slightly different from the last version; but the code is now in the linux-mm tree, the API is getting stable and the kernel code is just getting fixes now. This qemu series can be found at: https://github.com/orbitfp7/qemu.git on the wp3-postcopy-v7 tag. It addresses most of the previous review comments, but there are still one or two I'm working. As with v6, the userfaultfd.h isn't included in the tree, so you'll need a userfaultfd.h header (and syscall define) from Andrea's kernel. This work has been partially funded by the EU Orbit project: see http://www.orbitproject.eu/about/ v7 updated to Andrea's userfault 21 interface Don't restart the source after an error if we entered postcopy (thanks to Li Liang for reporting) Killed off dead local_tmp_page (thanks to Christoph Seifert for spotting that) Made sure the incoming page mutex is only initialised once Moved request alignment from destination to source; destination now just verifies the request is aligned Got rid of the 'dup' on the return-fd Just use the same fd and make the forward path own the fd Reworked the consumption of request pages off the queue to handle hosts with larger pages more cleanly Disallowed enabling postcopy+compression This is probably fixable, but it needs reworking of the decompression threads to place the pages atomically Disallow postcopy+RDMA This is probably trickier to fix; RDMA drops the received data straight into memory using DMA; we would need to find a way to stop that. I can see a way to make it so that the precopy phase still uses RDMA and then stops using it later, but that would make it harder when a real fix came along. Minor fixups from review 6 of the smaller patches from v6 are now already in head TODO Testing on machines with hps!=tps Tidy up the hps!=tps code on the receive side Dr. David Alan Gilbert (42): Start documenting how postcopy works. Provide runtime Target page information Init page sizes in qtest qemu_ram_block_from_host Add qemu_get_buffer_less_copy to avoid copies some of the time Add wrapper for setting blocking status on a QEMUFile ram_debug_dump_bitmap: Dump a migration bitmap as text migrate_init: Call from savevm Rename save_live_complete to save_live_complete_precopy Return path: Open a return path on QEMUFile for sockets Return path: socket_writev_buffer: Block even on non-blocking fd's Migration commands Return path: Control commands Return path: Send responses from destination to source Return path: Source handling of return path Rework loadvm path for subloops Add migration-capability boolean for postcopy-ram. Add wrappers and handlers for sending/receiving the postcopy-ram migration messages. MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Modify save_live_pending for postcopy postcopy: OS support test migrate_start_postcopy: Command to trigger transition to postcopy MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Add qemu_savevm_state_complete_postcopy Postcopy: Maintain sentmap and calculate discard postcopy: Incoming initialisation postcopy: ram_enable_notify to switch on userfault Postcopy: Postcopy startup in migration thread Postcopy end in migration_thread Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Page request: Process incoming page request Page request: Consume pages off the post-copy queue postcopy_ram.c: place_page and helpers Postcopy: Use helpers to map pages during migration Don't sync dirty bitmaps in postcopy Host page!=target page: Cleanup bitmaps Postcopy; Handle userfault requests Start up a postcopy/listener thread ready for incoming page data postcopy: Wire up loadvm_postcopy_handle_ commands End of migration for postcopy Disable mlock around incoming postcopy Inhibit ballooning during postcopy balloon.c | 11 + docs/migration.txt | 167 +++++++ exec.c | 64 ++- hmp-commands.hx | 15 + hmp.c | 7 + hmp.h | 1 + hw/ppc/spapr.c | 2 +- hw/virtio/virtio-balloon.c | 4 +- include/exec/cpu-all.h | 2 - include/exec/cpu-common.h | 3 + include/migration/migration.h | 112 ++++- include/migration/postcopy-ram.h | 88 ++++ include/migration/qemu-file.h | 10 + include/migration/vmstate.h | 8 +- include/qemu/typedefs.h | 3 + include/sysemu/balloon.h | 2 + include/sysemu/sysemu.h | 44 +- migration/Makefile.objs | 2 +- migration/block.c | 9 +- migration/migration.c | 736 ++++++++++++++++++++++++++++-- migration/postcopy-ram.c | 706 +++++++++++++++++++++++++++++ migration/qemu-file-unix.c | 110 ++++- migration/qemu-file.c | 74 ++++ migration/ram.c | 933 ++++++++++++++++++++++++++++++++++++--- migration/savevm.c | 776 +++++++++++++++++++++++++++++--- qapi-schema.json | 18 +- qmp-commands.hx | 19 + qtest.c | 1 + trace-events | 76 +++- 29 files changed, 3788 insertions(+), 215 deletions(-) create mode 100644 include/migration/postcopy-ram.h create mode 100644 migration/postcopy-ram.c -- 2.4.3