From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: [PATCH v3 00/27] Libxl migration v2 Date: Mon, 13 Jul 2015 13:01:19 +0100 Message-ID: <1436788907-1921-1-git-send-email-andrew.cooper3@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Xen-devel Cc: Wei Liu , Ian Campbell , Wen Congyang , Andrew Cooper , Ian Jackson , Yang Hongyang List-Id: xen-devel@lists.xenproject.org This series adds support for the libxl migration v2 stream, and untangles the existing layering violations of the toolstack and qemu records. It can be found on the branch "libxl-migv2-v3" git://xenbits.xen.org/people/andrewcoop/xen.git http://xenbits.xen.org/git-http/people/andrewcoop/xen.git No major changes over v2, but a lot of minor changes following review. At the end of the series, legacy migration is no longer used. The Remus code is untested by me. All other combinations of suspend/migrate/resume have been tested with PV and HVM guests (qemu-trad and qemu-upstream), including 32 -> 64 bit migration (which was the underlying bug causing us to write migration v2 in the first place). Anyway, thoughts/comments welcome. Please test! ~Andrew Summary of Acks/Modified/New from v2 A bsd-sys-queue-h-seddery: Massage `offsetof' A tools/libxc: Always compile the compat qemu variables into xc_sr_context A tools/libxl: Introduce ROUNDUP() AM tools/libxl: Introduce libxl__kill() A tools/libxl: Stash all restore parameters in domain_create_state A tools/libxl: Split libxl__domain_create_state.restore_fd in two M tools/libxl: Extra management APIs for the save helper N tools/libxl: Add save_helper_state pointers to libxl__xc_domain_{save,restore}() A tools/xl: Mandatory flag indicating the format of the migration stream A docs: Libxl migration v2 stream specification A tools/python: Libxc migration v2 infrastructure A tools/python: Libxl migration v2 infrastructure A tools/python: Other migration infrastructure A tools/python: Verification utility for v2 stream spec compliance A tools/python: Conversion utility for legacy migration streams A tools/libxl: Migration v2 stream format M tools/libxl: Infrastructure for reading a libxl migration v2 stream M tools/libxl: Infrastructure to convert a legacy stream M tools/libxl: Convert a legacy stream if needed M tools/libxc+libxl+xl: Restore v2 streams M tools/libxl: Infrastructure for writing a v2 stream M tools/libxc+libxl+xl: Save v2 streams A docs/libxl: Introduce CHECKPOINT_END to support migration v2 remus streams AM tools/libxl: Write checkpoint records into the stream M tools/libx{c,l}: Introduce restore_callbacks.checkpoint() M tools/libxl: Handle checkpoint records in a libxl migration v2 stream A tools/libxc: Drop all XG_LIBXL_HVM_COMPAT code from libxc A tools/libxl: Drop all knowledge of toolstack callbacks Andrew Cooper (24): tools/libxc: Always compile the compat qemu variables into xc_sr_context tools/libxl: Introduce ROUNDUP() tools/libxl: Introduce libxl__kill() tools/libxl: Stash all restore parameters in domain_create_state tools/libxl: Split libxl__domain_create_state.restore_fd in two tools/libxl: Extra management APIs for the save helper tools/libxl: Add save_helper_state pointers to libxl__xc_domain_{save,restore}() tools/xl: Mandatory flag indicating the format of the migration stream docs: Libxl migration v2 stream specification tools/python: Libxc migration v2 infrastructure tools/python: Libxl migration v2 infrastructure tools/python: Other migration infrastructure tools/python: Verification utility for v2 stream spec compliance tools/python: Conversion utility for legacy migration streams tools/libxl: Infrastructure to convert a legacy stream tools/libxl: Convert a legacy stream if needed tools/libxc+libxl+xl: Restore v2 streams tools/libxc+libxl+xl: Save v2 streams docs/libxl: Introduce CHECKPOINT_END to support migration v2 remus streams tools/libxl: Write checkpoint records into the stream tools/libx{c,l}: Introduce restore_callbacks.checkpoint() tools/libxl: Handle checkpoint records in a libxl migration v2 stream tools/libxc: Drop all XG_LIBXL_HVM_COMPAT code from libxc tools/libxl: Drop all knowledge of toolstack callbacks Ian Jackson (1): bsd-sys-queue-h-seddery: Massage `offsetof' Ross Lagerwall (3): tools/libxl: Migration v2 stream format tools/libxl: Infrastructure for reading a libxl migration v2 stream tools/libxl: Infrastructure for writing a v2 stream docs/specs/libxl-migration-stream.pandoc | 217 ++++++ tools/include/xen-external/bsd-sys-queue-h-seddery | 2 + tools/libxc/Makefile | 2 - tools/libxc/include/xenguest.h | 7 + tools/libxc/xc_sr_common.h | 12 +- tools/libxc/xc_sr_restore.c | 71 +- tools/libxc/xc_sr_restore_x86_hvm.c | 124 ---- tools/libxc/xc_sr_save_x86_hvm.c | 36 - tools/libxl/Makefile | 7 + tools/libxl/libxl.h | 19 + tools/libxl/libxl_aoutils.c | 15 + tools/libxl/libxl_convert_callout.c | 158 ++++ tools/libxl/libxl_create.c | 91 ++- tools/libxl/libxl_dom.c | 107 +-- tools/libxl/libxl_internal.h | 172 ++++- tools/libxl/libxl_no_convert_callout.c | 35 + tools/libxl/libxl_save_callout.c | 117 ++- tools/libxl/libxl_save_helper.c | 33 +- tools/libxl/libxl_save_msgs_gen.pl | 9 +- tools/libxl/libxl_sr_stream_format.h | 58 ++ tools/libxl/libxl_stream_read.c | 768 ++++++++++++++++++++ tools/libxl/libxl_stream_write.c | 525 +++++++++++++ tools/libxl/libxl_types.idl | 1 + tools/libxl/xl_cmdimpl.c | 9 +- tools/python/Makefile | 4 + tools/python/scripts/convert-legacy-stream | 678 +++++++++++++++++ tools/python/scripts/verify-stream-v2 | 174 +++++ tools/python/setup.py | 1 + tools/python/xen/migration/legacy.py | 279 +++++++ tools/python/xen/migration/libxc.py | 446 ++++++++++++ tools/python/xen/migration/libxl.py | 199 +++++ tools/python/xen/migration/public.py | 21 + tools/python/xen/migration/tests.py | 54 ++ tools/python/xen/migration/verify.py | 37 + tools/python/xen/migration/xl.py | 12 + 35 files changed, 4071 insertions(+), 429 deletions(-) create mode 100644 docs/specs/libxl-migration-stream.pandoc create mode 100644 tools/libxl/libxl_convert_callout.c create mode 100644 tools/libxl/libxl_no_convert_callout.c create mode 100644 tools/libxl/libxl_sr_stream_format.h create mode 100644 tools/libxl/libxl_stream_read.c create mode 100644 tools/libxl/libxl_stream_write.c create mode 100755 tools/python/scripts/convert-legacy-stream create mode 100755 tools/python/scripts/verify-stream-v2 create mode 100644 tools/python/xen/migration/__init__.py create mode 100644 tools/python/xen/migration/legacy.py create mode 100644 tools/python/xen/migration/libxc.py create mode 100644 tools/python/xen/migration/libxl.py create mode 100644 tools/python/xen/migration/public.py create mode 100644 tools/python/xen/migration/tests.py create mode 100644 tools/python/xen/migration/verify.py create mode 100644 tools/python/xen/migration/xl.py -- 1.7.10.4