From: Juan Quintela <quintela@redhat.com>
To: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: "Jaggi, Manish" <Manish.Jaggi@cavium.com>,
Auger Eric <eric.auger@redhat.com>,
"drjones@redhat.com" <drjones@redhat.com>,
"peter.maydell@linaro.org qemu-devel@nongnu.org"
<qemu-devel@nongnu.org>, Anthony Liguori <aliguori@us.ibm.com>
Subject: Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids
Date: Fri, 31 Aug 2018 11:41:03 +0200 [thread overview]
Message-ID: <87d0tyoouo.fsf@trasno.org> (raw)
In-Reply-To: <20180829135814.GE2412@work-vm> (David Alan Gilbert's message of "Wed, 29 Aug 2018 14:58:15 +0100")
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Jaggi, Manish (Manish.Jaggi@cavium.com) wrote:
>
>> Just to add what happens in ARM64 case, qemu running on Machine A sends cpu state information to Machine B.
>> This state contains MIDR value, and so Processor ID value is compared in KVM and not in qemu (correcting myself).
>>
>> IIRC, Peter/Eric please point if there is something incorrect in the below flow...
>>
>> (Machine B)
>> target/arm/machine.c: cpu_post_load()
>> - updates cpu->cpreg_values[i] : which includes MIDR (processor ID register)
>>
>> - calls write_list_to_kvmstate(cpu, KVM_PUT_FULL_STATE)
>>
>> target/arm/kvm.c: write_list_to_kvmstate
>> - calls => kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
>>
>> => and it eventually lands up IIRC in Linux code in
>>
>> => arch/arm64/kvm/sys_regs.c : set_invariant_sys_reg(u64 id, void __user *uaddr)
>> /* This is what we mean by invariant: you can't change it. */
>> if (r->val != val)
>> return -EINVAL;
>> Note: MIDR_EL1 is invariant register.
>> result: Migration fails on Machine B.
>>
>> A few points:
>> - qemu on arm64 is invoked with -machine virt and -cpu as host. So we don't explicitly define which cpu.
>
> Note that even on x86 we don't guarantee as much about '-cpu host', what
> we expect to work for migration is that if you pick a '-cpu amodel'
> and both hosts support the feature flags required by 'amodel' then it
> should work.
I really think that the right approach here is not using -cpu host. You
do the full work, create a model as David says, and be sure that you car
run that model on both cpus. It is a lot of work, but it is the only
way to make sure that this is going to work long term.
>> - In case Machine A and Machine B have almost same Core and the
>> delta may-not have any effect on qemu operation, migration should
>> work by just looking into whitelist.
>> whitelist can be given as a parameter for qemu on machine B.
>>
>> qemu-system-aarch64 -whitelist <ids separated by commas>
>>
>> (This is my proposal)
>>
>> - So in cpu_post_load (Machine B) qemu can lookup whitelist and
>> replace the MIDR with the one at Machine B.
>> Sounds good?
>>
>> - Juan raised a point about clock speed, I am not sure it will have
>> any effect on arm since qemu is run with -cpu host param.
>> I could be wrong here, Peter/Eric can you please correct me...
>
> Clock speed is only really a problem for things like timestamp counters;
> some cores let you scale them; for those that don't then yes it's a bit
> odd.
This was just one example that I thought on top of my head. Other thing
that I remember was that at some point, if you migrated to a cpu with
hyperthreading, you lost half of your performance counters. I still
think that you need to create a proper model, and work from there.
Later, Juan.
prev parent reply other threads:[~2018-08-31 9:41 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-21 6:39 [Qemu-devel] Live Migration between machines with different processor ids Mjaggi Oss
2018-08-23 11:18 ` [Qemu-devel] [Query] " Jaggi, Manish
2018-08-23 14:29 ` Juan Quintela
2018-08-24 9:24 ` Jaggi, Manish
2018-08-28 17:27 ` Dr. David Alan Gilbert
2018-08-29 12:40 ` Jaggi, Manish
2018-08-29 13:16 ` Andrew Jones
2018-08-31 9:52 ` Jaggi, Manish
2018-08-31 11:11 ` Andrew Jones
2018-09-04 9:16 ` Jaggi, Manish
2018-09-04 9:54 ` Andrew Jones
2018-09-04 10:27 ` Juan Quintela
2018-09-04 10:32 ` Dr. David Alan Gilbert
2018-09-04 12:17 ` Peter Maydell
2018-09-05 11:46 ` Jaggi, Manish
2018-09-05 12:20 ` Andrew Jones
2018-09-05 12:42 ` Jaggi, Manish
2018-09-05 13:17 ` Andrew Jones
2018-08-29 13:58 ` Dr. David Alan Gilbert
2018-08-31 9:41 ` Juan Quintela [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87d0tyoouo.fsf@trasno.org \
--to=quintela@redhat.com \
--cc=Manish.Jaggi@cavium.com \
--cc=aliguori@us.ibm.com \
--cc=dgilbert@redhat.com \
--cc=drjones@redhat.com \
--cc=eric.auger@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.