* [Qemu-devel] incorrect memory size inside vm
@ 2015-06-17 13:35 Vasiliy Tolstov
2015-06-17 14:09 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 13:35 UTC (permalink / raw)
To: libvir-list; +Cc: qemu-devel
Hi. I have issue with incorrect memory side inside vm. I'm try utilize
memory balloon (not memory hotplug, because i have guest without
memory hotplug (may be)).
When domain started with static memory all works fine, but then i'm
specify in libvirt
memory = 16384 , maxMemory = 16384 and currentMemory = 1024, guest in
f/rpoc/meminfo says that have only 603608 Kb memory. Then i set memory
via virsh setmem to 2Gb, guest see only 1652184 Kb memory.
software versions
libvirt: 1.2.10
qemu: 2.3.0
Guest OS: centos 6.
qemu.log:
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/bin/kvm
-name 26543 -S -machine pc-i440fx-1.7,accel=kvm,usb=off -m 1024
-realtime mlock=off -smp 1,maxcpus=4,sockets=4,cores=1,threads=1 -uuid
4521fb01-c2ca-4269-d2d6-0000035fd910 -no-user-config -nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/26543.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-scsi-pci,id=scsi0,num_queues=1,bus=pci.0,addr=0x4 -device
virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
file=/dev/vg4/26543,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,discard=unmap,aio=native,iops=5000
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
-drive if=none,id=drive-scsi0-0-1-0,readonly=on,format=raw -device
scsi-cd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0
-netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=52 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:34:f7,bus=pci.0,addr=0x3,rombar=0
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/26543.agent,server,nowait
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
-device usb-mouse,id=input0 -device usb-kbd,id=input1 -vnc
[::]:8,password -device VGA,id=video0,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object
rng-random,id=rng0,filename=/dev/random -device
virtio-rng-pci,rng=rng0,max-bytes=1024,period=2000,bus=pci.0,addr=0x7
-msg timestamp=on
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 13:35 [Qemu-devel] incorrect memory size inside vm Vasiliy Tolstov
@ 2015-06-17 14:09 ` Andrey Korolyov
2015-06-17 15:33 ` Vasiliy Tolstov
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-17 14:09 UTC (permalink / raw)
To: Vasiliy Tolstov; +Cc: libvir-list, qemu-devel@nongnu.org
On Wed, Jun 17, 2015 at 4:35 PM, Vasiliy Tolstov <v.tolstov@selfip.ru> wrote:
> Hi. I have issue with incorrect memory side inside vm. I'm try utilize
> memory balloon (not memory hotplug, because i have guest without
> memory hotplug (may be)).
>
> When domain started with static memory all works fine, but then i'm
> specify in libvirt
> memory = 16384 , maxMemory = 16384 and currentMemory = 1024, guest in
> f/rpoc/meminfo says that have only 603608 Kb memory. Then i set memory
> via virsh setmem to 2Gb, guest see only 1652184 Kb memory.
>
> software versions
> libvirt: 1.2.10
> qemu: 2.3.0
> Guest OS: centos 6.
>
> qemu.log:
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
> HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/bin/kvm
> -name 26543 -S -machine pc-i440fx-1.7,accel=kvm,usb=off -m 1024
> -realtime mlock=off -smp 1,maxcpus=4,sockets=4,cores=1,threads=1 -uuid
> 4521fb01-c2ca-4269-d2d6-0000035fd910 -no-user-config -nodefaults
> -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/26543.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
> -no-shutdown -boot strict=on -device
> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> virtio-scsi-pci,id=scsi0,num_queues=1,bus=pci.0,addr=0x4 -device
> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
> file=/dev/vg4/26543,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,discard=unmap,aio=native,iops=5000
> -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
> -drive if=none,id=drive-scsi0-0-1-0,readonly=on,format=raw -device
> scsi-cd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0
> -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=52 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:34:f7,bus=pci.0,addr=0x3,rombar=0
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -chardev
> socket,id=charchannel0,path=/var/lib/libvirt/qemu/26543.agent,server,nowait
> -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
> -device usb-mouse,id=input0 -device usb-kbd,id=input1 -vnc
> [::]:8,password -device VGA,id=video0,bus=pci.0,addr=0x2 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object
> rng-random,id=rng0,filename=/dev/random -device
> virtio-rng-pci,rng=rng0,max-bytes=1024,period=2000,bus=pci.0,addr=0x7
> -msg timestamp=on
>
> --
> Vasiliy Tolstov,
> e-mail: v.tolstov@selfip.ru
>
The rest of visible memory is eaten by reserved kernel areas, for us
this was a main reason to switch to a hotplug a couple of years ago.
You would not be able to scale a VM by an order of magnitude with
regular balloon mechanism without mentioned impact, unfortunately.
Igor Mammedov posted hotplug-related patches for 2.6.32 a while ago,
though RHEL6 never adopted them by some reason.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 14:09 ` Andrey Korolyov
@ 2015-06-17 15:33 ` Vasiliy Tolstov
2015-06-17 15:38 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 15:33 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: libvir-list, qemu-devel@nongnu.org
2015-06-17 17:09 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
> The rest of visible memory is eaten by reserved kernel areas, for us
> this was a main reason to switch to a hotplug a couple of years ago.
> You would not be able to scale a VM by an order of magnitude with
> regular balloon mechanism without mentioned impact, unfortunately.
> Igor Mammedov posted hotplug-related patches for 2.6.32 a while ago,
> though RHEL6 never adopted them by some reason.
Hmm.. Thanks for info, from what version of kernel memory hotplug works?
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 15:33 ` Vasiliy Tolstov
@ 2015-06-17 15:38 ` Andrey Korolyov
2015-06-17 16:26 ` Vasiliy Tolstov
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-17 15:38 UTC (permalink / raw)
To: Vasiliy Tolstov; +Cc: libvir-list, qemu-devel@nongnu.org
On Wed, Jun 17, 2015 at 6:33 PM, Vasiliy Tolstov <v.tolstov@selfip.ru> wrote:
> 2015-06-17 17:09 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
>> The rest of visible memory is eaten by reserved kernel areas, for us
>> this was a main reason to switch to a hotplug a couple of years ago.
>> You would not be able to scale a VM by an order of magnitude with
>> regular balloon mechanism without mentioned impact, unfortunately.
>> Igor Mammedov posted hotplug-related patches for 2.6.32 a while ago,
>> though RHEL6 never adopted them by some reason.
>
>
> Hmm.. Thanks for info, from what version of kernel memory hotplug works?
>
> --
> Vasiliy Tolstov,
> e-mail: v.tolstov@selfip.ru
Currently QEMU memory hotplug should work with 3.8 and onwards.
Mentioned patches are an adaptation for an older frankenkernel of 3.8`
functionality.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 15:38 ` Andrey Korolyov
@ 2015-06-17 16:26 ` Vasiliy Tolstov
2015-06-17 21:21 ` Vasiliy Tolstov
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 16:26 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: libvir-list, qemu-devel@nongnu.org
2015-06-17 18:38 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
> Currently QEMU memory hotplug should work with 3.8 and onwards.
> Mentioned patches are an adaptation for an older frankenkernel of 3.8`
> functionality.
This is band news =( i have debian wheezy that have old kernel...
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 16:26 ` Vasiliy Tolstov
@ 2015-06-17 21:21 ` Vasiliy Tolstov
2015-06-17 22:40 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 21:21 UTC (permalink / raw)
To: Vasiliy Tolstov; +Cc: libvir-list, Andrey Korolyov, qemu-devel@nongnu.org
2015-06-17 19:26 GMT+03:00 Vasiliy Tolstov <v.tolstov@selfip.ru>:
> This is band news =( i have debian wheezy that have old kernel...
Does it possible to get proper results with balloon ? For example by
patching qemu or something like this?
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 21:21 ` Vasiliy Tolstov
@ 2015-06-17 22:40 ` Andrey Korolyov
2015-06-17 22:44 ` Vasiliy Tolstov
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-17 22:40 UTC (permalink / raw)
To: Vasiliy Tolstov; +Cc: libvir-list, qemu-devel@nongnu.org
On Thu, Jun 18, 2015 at 12:21 AM, Vasiliy Tolstov <v.tolstov@selfip.ru> wrote:
> 2015-06-17 19:26 GMT+03:00 Vasiliy Tolstov <v.tolstov@selfip.ru>:
>> This is band news =( i have debian wheezy that have old kernel...
>
>
> Does it possible to get proper results with balloon ? For example by
> patching qemu or something like this?
>
>
Yes, but I`m afraid that I don`t fully understand why do you need this
when pure hotplug mechanism is available, aside may be nice memory
stats from balloon and easy-to-use deflation. Just populate a couple
of static dimms with small enough 'base' e820 memory and use balloon
on this setup, you`ll get the reserved memory footprint as small as it
would be in setup with equal overall amount of memory populated via
BIOS. For example, you may use -m 128 ... {handful amount of memory
placed in memory slots} setup to achieve the thing you want.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 22:40 ` Andrey Korolyov
@ 2015-06-17 22:44 ` Vasiliy Tolstov
2015-06-17 22:52 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 22:44 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: libvir-list, qemu-devel@nongnu.org
2015-06-18 1:40 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
>
> Yes, but I`m afraid that I don`t fully understand why do you need this
> when pure hotplug mechanism is available, aside may be nice memory
> stats from balloon and easy-to-use deflation. Just populate a couple
> of static dimms with small enough 'base' e820 memory and use balloon
> on this setup, you`ll get the reserved memory footprint as small as it
> would be in setup with equal overall amount of memory populated via
> BIOS. For example, you may use -m 128 ... {handful amount of memory
> placed in memory slots} setup to achieve the thing you want.
I have debian wheezy guests with 3.4 kernels (or 3.2..) and many
others like 32 centos 6, opensue , ubuntu, and other.
Does memory hotplug works with this distros (kernels)?
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 22:44 ` Vasiliy Tolstov
@ 2015-06-17 22:52 ` Andrey Korolyov
2015-06-17 22:57 ` Vasiliy Tolstov
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-17 22:52 UTC (permalink / raw)
To: Vasiliy Tolstov; +Cc: libvir-list, qemu-devel@nongnu.org
On Thu, Jun 18, 2015 at 1:44 AM, Vasiliy Tolstov <v.tolstov@selfip.ru> wrote:
> 2015-06-18 1:40 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
>>
>> Yes, but I`m afraid that I don`t fully understand why do you need this
>> when pure hotplug mechanism is available, aside may be nice memory
>> stats from balloon and easy-to-use deflation. Just populate a couple
>> of static dimms with small enough 'base' e820 memory and use balloon
>> on this setup, you`ll get the reserved memory footprint as small as it
>> would be in setup with equal overall amount of memory populated via
>> BIOS. For example, you may use -m 128 ... {handful amount of memory
>> placed in memory slots} setup to achieve the thing you want.
>
>
> I have debian wheezy guests with 3.4 kernels (or 3.2..) and many
> others like 32 centos 6, opensue , ubuntu, and other.
> Does memory hotplug works with this distros (kernels)?
>
Whoosh... technically it is possible but it would be an incompatible
fork for the upstreams for both SeaBIOS and Qemu, because the generic
way of plugging DIMMs in is available down to at least generic 2.6.32.
Except may be Centos where broken kabi would bring great consequences,
it may be better to just provide a backport repository with newer
kernels, but it doesn`t sound very optimistic. For the history
records, the initial hotplug support proposal provided by Vasilis
Liaskovitis a couple of years ago worked in an exact way you are
suggesting to, but its resurrection would mean emulator and rom code
alteration, as I said above.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 22:52 ` Andrey Korolyov
@ 2015-06-17 22:57 ` Vasiliy Tolstov
2015-06-18 15:05 ` Piotr Rybicki
0 siblings, 1 reply; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-17 22:57 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: libvir-list, qemu-devel@nongnu.org
2015-06-18 1:52 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
> Whoosh... technically it is possible but it would be an incompatible
> fork for the upstreams for both SeaBIOS and Qemu, because the generic
> way of plugging DIMMs in is available down to at least generic 2.6.32.
> Except may be Centos where broken kabi would bring great consequences,
> it may be better to just provide a backport repository with newer
> kernels, but it doesn`t sound very optimistic. For the history
> records, the initial hotplug support proposal provided by Vasilis
> Liaskovitis a couple of years ago worked in an exact way you are
> suggesting to, but its resurrection would mean emulator and rom code
> alteration, as I said above.
Ok, i'm try to build latest libvirt and check all oses for memory
hotplug support =).
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-17 22:57 ` Vasiliy Tolstov
@ 2015-06-18 15:05 ` Piotr Rybicki
2015-06-18 21:23 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Piotr Rybicki @ 2015-06-18 15:05 UTC (permalink / raw)
To: andrey; +Cc: libvir-list, qemu-devel, v.tolstov
W dniu 2015-06-18 o 00:57, Vasiliy Tolstov pisze:
> 2015-06-18 1:52 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
>> Whoosh... technically it is possible but it would be an incompatible
>> fork for the upstreams for both SeaBIOS and Qemu, because the generic
>> way of plugging DIMMs in is available down to at least generic 2.6.32.
>> Except may be Centos where broken kabi would bring great consequences,
>> it may be better to just provide a backport repository with newer
>> kernels, but it doesn`t sound very optimistic. For the history
>> records, the initial hotplug support proposal provided by Vasilis
>> Liaskovitis a couple of years ago worked in an exact way you are
>> suggesting to, but its resurrection would mean emulator and rom code
>> alteration, as I said above.
>
>
> Ok, i'm try to build latest libvirt and check all oses for memory
> hotplug support =).
>
Hi guys.
I'm actually investigating mem waste issue at my lab. I'm using libvirt
+ qemu on gentoo.
# libvirtd -v
2015-06-18 14:50:56.619+0000: 11720: info : libvirt version: 1.2.16
# qemu-x86_64 -version
qemu-x86_64 version 2.3.0, Copyright (c) 2003-2008 Fabrice Bellard
# uname -a
Linux vms06 3.18.14-gentoo #1 SMP Wed Jun 17 14:55:27 CEST 2015 x86_64
AMD Opteron(tm) Processor 6380 AuthenticAMD GNU/Linux
When not using dimm (static or hotplugged) - only 'main' memory, waste
is huge, especially if one define big limit.
my test with only 'plain' memory:
define domain.xml with different mem setting, and see mem sizes (virsh
domstats DOMAIN --balloon from host and 'free' command from guest)
libvirt max: 2GB, curr: 2GB
system total: 2001
balloon.current=2097152
balloon.maximum=2097152
libvirt max: 4GB, curr: 4GB
system total: 3953
balloon.current=4194304
balloon.maximum=4194304
libvirt max: 4GB, curr: 2GB
system total: 1905
balloon.current=2097152
balloon.maximum=4194304
libvirt max: 8GB, curr: 8GB
system total: 7985
balloon.current=8388608
balloon.maximum=8388608
libvirt max: 8GB, curr: 4GB
system total: 3889
balloon.current=4194304
balloon.maximum=8388608
libvirt max: 8GB, curr: 2GB
system total: 1841
balloon.current=2097152
balloon.maximum=8388608
libvirt max: 16GB, curr: 16GB
system total: 16049
balloon.current=16777216
balloon.maximum=16777216
libvirt max: 16GB, curr: 8GB
system total: 7857
balloon.current=8388608
balloon.maximum=16777216
libvirt max: 16GB, curr: 4GB
system total: 3761
balloon.current=4194304
balloon.maximum=16777216
libvirt max: 16GB, curr: 2GB
system total: 1713
balloon.current=2097152
balloon.maximum=16777216
As You can see, when one set 16GB max mem and define current mem to 2GB,
guest only see 1713 MB. When You set 2GB max mem and leave 2GB mem,
guest see 2001. So there are 288 MB wasted.
Later, i tried to define 15 static 1GB dimms and 1GB 'main' memory, and
later on decreased guest memory via balloon to 1GB, and there was no
waste in guest.
But, when i checked RES size of qemu process on host, It was about
5,5GB! In contrast, when using only 'main' memory, results were as
expected (res mem for qemu was amount ram for guest + qemu itself).
Do You see similar results at Your side?
Best regards
Piotr Rybicki
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-18 15:05 ` Piotr Rybicki
@ 2015-06-18 21:23 ` Andrey Korolyov
2015-06-19 11:14 ` Piotr Rybicki
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-18 21:23 UTC (permalink / raw)
To: Piotr Rybicki; +Cc: libvir-list, qemu-devel@nongnu.org, Vasiliy Tolstov
> Do You see similar results at Your side?
>
> Best regards
Would you mind to share you argument set to an emulator? As far as I
understood you are using plain ballooning with most results from above
for which those numbers are expected. The case with 5+gig memory
consumption for deflated 1G guest looks like a bug with mixed
dimm/balloon configuration if you are tried against latest qemu, so
please describe a setup a bit more verbosely too.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-18 21:23 ` Andrey Korolyov
@ 2015-06-19 11:14 ` Piotr Rybicki
2015-06-19 12:01 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Piotr Rybicki @ 2015-06-19 11:14 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: libvir-list, qemu-devel@nongnu.org, Vasiliy Tolstov
Hello.
Actually it was my mistake.
After some time using memory in guest (find /, cp bigfine, etc), res
size of qemu process shrinks to expected value.
Sorry for disturbing.
Now i don't see any memory waste in guest and host when using 'base'
memory + 'dimm' memory.
Although i have one issue.
When i start qemu via libvirt with 16GB mem and 1G current mem:
(...)
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
(...)
Qemu starts, and balloon can't free memmory, so guest doesn't boot (It
hangs or is looped in ' virtio_balloon virtio2: Out of puff! Can't get 1
pages').
I think this is because dimm memmory is not yet onlined, and balloon
already tries to shring memory in guest.
Best regards
Piotr Rybicki
W dniu 2015-06-18 o 23:23, Andrey Korolyov pisze:
>> Do You see similar results at Your side?
>>
>> Best regards
>
> Would you mind to share you argument set to an emulator? As far as I
> understood you are using plain ballooning with most results from above
> for which those numbers are expected. The case with 5+gig memory
> consumption for deflated 1G guest looks like a bug with mixed
> dimm/balloon configuration if you are tried against latest qemu, so
> please describe a setup a bit more verbosely too.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-19 11:14 ` Piotr Rybicki
@ 2015-06-19 12:01 ` Andrey Korolyov
2015-06-19 12:02 ` Vasiliy Tolstov
2015-06-22 13:30 ` Piotr Rybicki
0 siblings, 2 replies; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-19 12:01 UTC (permalink / raw)
To: Piotr Rybicki; +Cc: qemu-devel@nongnu.org, Vasiliy Tolstov
On Fri, Jun 19, 2015 at 2:14 PM, Piotr Rybicki
<piotr.rybicki@innervision.pl> wrote:
> Hello.
>
> Actually it was my mistake.
> After some time using memory in guest (find /, cp bigfine, etc), res size of
> qemu process shrinks to expected value.
>
> Sorry for disturbing.
> Now i don't see any memory waste in guest and host when using 'base' memory
> + 'dimm' memory.
>
> Although i have one issue.
>
> When i start qemu via libvirt with 16GB mem and 1G current mem:
> (...)
> <memory unit='KiB'>16777216</memory>
> <currentMemory unit='KiB'>1048576</currentMemory>
> (...)
>
> Qemu starts, and balloon can't free memmory, so guest doesn't boot (It hangs
> or is looped in ' virtio_balloon virtio2: Out of puff! Can't get 1 pages').
> I think this is because dimm memmory is not yet onlined, and balloon already
> tries to shring memory in guest.
>
> Best regards
> Piotr Rybicki
>
>
- libvir@
Please don`t top-post in technical mailing lists. Do you have a
crashkernel-reserved area there in the boot arguments? What
distro/guest kernel are running in this guest and what is dimm
configuration?
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-19 12:01 ` Andrey Korolyov
@ 2015-06-19 12:02 ` Vasiliy Tolstov
2015-06-22 13:30 ` Piotr Rybicki
1 sibling, 0 replies; 20+ messages in thread
From: Vasiliy Tolstov @ 2015-06-19 12:02 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: qemu-devel@nongnu.org, Piotr Rybicki
2015-06-19 15:01 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
>
> Please don`t top-post in technical mailing lists. Do you have a
> crashkernel-reserved area there in the boot arguments? What
> distro/guest kernel are running in this guest and what is dimm
> configuration?
disabling crashkernel in my case save is about 130-150 Mb,
--
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-19 12:01 ` Andrey Korolyov
2015-06-19 12:02 ` Vasiliy Tolstov
@ 2015-06-22 13:30 ` Piotr Rybicki
2015-06-22 15:58 ` Andrey Korolyov
1 sibling, 1 reply; 20+ messages in thread
From: Piotr Rybicki @ 2015-06-22 13:30 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: qemu-devel@nongnu.org, Vasiliy Tolstov
W dniu 2015-06-19 o 14:01, Andrey Korolyov pisze:
> On Fri, Jun 19, 2015 at 2:14 PM, Piotr Rybicki
> <piotr.rybicki@innervision.pl> wrote:
>> Hello.
>>
>> Actually it was my mistake.
>> After some time using memory in guest (find /, cp bigfine, etc), res size of
>> qemu process shrinks to expected value.
>>
>> Sorry for disturbing.
>> Now i don't see any memory waste in guest and host when using 'base' memory
>> + 'dimm' memory.
>>
>> Although i have one issue.
>>
>> When i start qemu via libvirt with 16GB mem and 1G current mem:
>> (...)
>> <memory unit='KiB'>16777216</memory>
>> <currentMemory unit='KiB'>1048576</currentMemory>
>> (...)
>>
>> Qemu starts, and balloon can't free memmory, so guest doesn't boot (It hangs
>> or is looped in ' virtio_balloon virtio2: Out of puff! Can't get 1 pages').
>> I think this is because dimm memmory is not yet onlined, and balloon already
>> tries to shring memory in guest.
>>
>> Best regards
>> Piotr Rybicki
>>
>>
> - libvir@
>
> Please don`t top-post in technical mailing lists. Do you have a
> crashkernel-reserved area there in the boot arguments? What
> distro/guest kernel are running in this guest and what is dimm
> configuration?
>
Hi.
No, I don't have a crashkernel reserved area.
But I've solved my problem. I load virtio-balloon device as a module
(previously it was kernel built-in), and everything is working as expected.
Thank You for Your attention.
Best regards
Piotr Rybicki
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-22 13:30 ` Piotr Rybicki
@ 2015-06-22 15:58 ` Andrey Korolyov
0 siblings, 0 replies; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-22 15:58 UTC (permalink / raw)
To: Piotr Rybicki; +Cc: qemu-devel@nongnu.org, Vasiliy Tolstov
On Mon, Jun 22, 2015 at 4:30 PM, Piotr Rybicki
<piotr.rybicki@innervision.pl> wrote:
>
>
> W dniu 2015-06-19 o 14:01, Andrey Korolyov pisze:
>
>> On Fri, Jun 19, 2015 at 2:14 PM, Piotr Rybicki
>> <piotr.rybicki@innervision.pl> wrote:
>>>
>>> Hello.
>>>
>>> Actually it was my mistake.
>>> After some time using memory in guest (find /, cp bigfine, etc), res size
>>> of
>>> qemu process shrinks to expected value.
>>>
>>> Sorry for disturbing.
>>> Now i don't see any memory waste in guest and host when using 'base'
>>> memory
>>> + 'dimm' memory.
>>>
>>> Although i have one issue.
>>>
>>> When i start qemu via libvirt with 16GB mem and 1G current mem:
>>> (...)
>>> <memory unit='KiB'>16777216</memory>
>>> <currentMemory unit='KiB'>1048576</currentMemory>
>>> (...)
>>>
>>> Qemu starts, and balloon can't free memmory, so guest doesn't boot (It
>>> hangs
>>> or is looped in ' virtio_balloon virtio2: Out of puff! Can't get 1
>>> pages').
>>> I think this is because dimm memmory is not yet onlined, and balloon
>>> already
>>> tries to shring memory in guest.
>>>
>>> Best regards
>>> Piotr Rybicki
>>>
>>>
>> - libvir@
>>
>> Please don`t top-post in technical mailing lists. Do you have a
>> crashkernel-reserved area there in the boot arguments? What
>> distro/guest kernel are running in this guest and what is dimm
>> configuration?
>>
>
> Hi.
>
> No, I don't have a crashkernel reserved area.
>
> But I've solved my problem. I load virtio-balloon device as a module
> (previously it was kernel built-in), and everything is working as expected.
>
> Thank You for Your attention.
>
> Best regards
> Piotr Rybicki
Thanks Piotr, the lack of the host-side memory shrinkage after balloon
deflation is interesting anyway, hopefully you may share guest` dmesg
bits from compiled-in balloon to check visible signs of the issue
(balloon should be initialized *before* DIMMs are being onlined, so
the behavior of the balloon itself it not clear at least for me).
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
@ 2015-06-23 12:32 Piotr Rybicki
2015-06-23 18:59 ` Andrey Korolyov
0 siblings, 1 reply; 20+ messages in thread
From: Piotr Rybicki @ 2015-06-23 12:32 UTC (permalink / raw)
To: andrey; +Cc: qemu-devel
> Thanks Piotr, the lack of the host-side memory shrinkage after balloon
> deflation is interesting anyway, hopefully you may share guest` dmesg
> bits from compiled-in balloon to check visible signs of the issue
It is really hard to see dmesg output via vnc console, when kernel is sending so many messages looped :(
> (balloon should be initialized *before* DIMMs are being onlined, so
> the behavior of the balloon itself it not clear at least for me).
Why balloon should be initialized before DIMMs onlined?
DIMMS are onlined as soon as udev starts (i have udev rule to online mem, cpu), so system has to start init process. I don't see any option to online DIMMs before booting reaches init process.
With balloon compiled-in, it starts to shrink memmory (for example to shrink 8GB, when 1GB 'base' mem is defined, and 10GB in DIMMs), when there is only 1GB 'base' memory. This leads to problem
With balloon loaded as a module, shrinking starts after init has begun (because system has to at least mount filesystem where balloon module is placed to load)
Well, i can't proove this with dmesg, but with balloon as module - system is booting and working as expected. With balloon compiled-in - system does not boot saying there is no free mem for balloon to shrink it.
Best regards
Piotr Rybicki
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-23 12:32 Piotr Rybicki
@ 2015-06-23 18:59 ` Andrey Korolyov
2015-06-23 22:10 ` Piotr Rybicki
0 siblings, 1 reply; 20+ messages in thread
From: Andrey Korolyov @ 2015-06-23 18:59 UTC (permalink / raw)
To: Piotr Rybicki; +Cc: qemu-devel@nongnu.org
On Tue, Jun 23, 2015 at 3:32 PM, Piotr Rybicki
<piotr.rybicki@innervision.pl> wrote:
>> Thanks Piotr, the lack of the host-side memory shrinkage after balloon
>> deflation is interesting anyway, hopefully you may share guest` dmesg
>> bits from compiled-in balloon to check visible signs of the issue
>
>
> It is really hard to see dmesg output via vnc console, when kernel is
> sending so many messages looped :(
>
>> (balloon should be initialized *before* DIMMs are being onlined, so
>> the behavior of the balloon itself it not clear at least for me).
>
>
> Why balloon should be initialized before DIMMs onlined?
For compiled-in balloon, its initialization should be done earlier
than the udev, because the kernel is not switched to init execution
yet.
>
> DIMMS are onlined as soon as udev starts (i have udev rule to online mem,
> cpu), so system has to start init process. I don't see any option to online
> DIMMs before booting reaches init process.
>
> With balloon compiled-in, it starts to shrink memmory (for example to shrink
> 8GB, when 1GB 'base' mem is defined, and 10GB in DIMMs), when there is only
> 1GB 'base' memory. This leads to problem
>
> With balloon loaded as a module, shrinking starts after init has begun
> (because system has to at least mount filesystem where balloon module is
> placed to load)
>
> Well, i can't proove this with dmesg, but with balloon as module - system is
> booting and working as expected. With balloon compiled-in - system does not
> boot saying there is no free mem for balloon to shrink it.
Thanks, this is an unexpected behavior, because you are starting with
small amount of total memory exposed to a balloon at the beginning and
there should be no issues except if the inflation races with plugging
in new DIMMs (just a wild guess), dmesg output will be helpful there
as I pointed earlier.
>
> Best regards
> Piotr Rybicki
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] incorrect memory size inside vm
2015-06-23 18:59 ` Andrey Korolyov
@ 2015-06-23 22:10 ` Piotr Rybicki
0 siblings, 0 replies; 20+ messages in thread
From: Piotr Rybicki @ 2015-06-23 22:10 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: qemu-devel@nongnu.org
> Thanks, this is an unexpected behavior, because you are starting with
> small amount of total memory exposed to a balloon at the beginning and
> there should be no issues except if the inflation races with plugging
> in new DIMMs (just a wild guess), dmesg output will be helpful there
> as I pointed earlier.
Try to start qemu with small amount of 'base' memory (like 1GB), and add
(even have static defined, not hotplugged after boot) 8GB in DIMMs but
do not online it.
My guest see 1GB RAM, balloon see 9GB. At this state, try to shrink
guest RAM to 8GB. My guest becomes unresposive and kernel messages loops
witch similar string.
So, perhaps balloon shoud consider only onlined DIMM RAM and report 1GB
in my example.
Best regards
Piotr Rybicki
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2015-06-23 22:10 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-17 13:35 [Qemu-devel] incorrect memory size inside vm Vasiliy Tolstov
2015-06-17 14:09 ` Andrey Korolyov
2015-06-17 15:33 ` Vasiliy Tolstov
2015-06-17 15:38 ` Andrey Korolyov
2015-06-17 16:26 ` Vasiliy Tolstov
2015-06-17 21:21 ` Vasiliy Tolstov
2015-06-17 22:40 ` Andrey Korolyov
2015-06-17 22:44 ` Vasiliy Tolstov
2015-06-17 22:52 ` Andrey Korolyov
2015-06-17 22:57 ` Vasiliy Tolstov
2015-06-18 15:05 ` Piotr Rybicki
2015-06-18 21:23 ` Andrey Korolyov
2015-06-19 11:14 ` Piotr Rybicki
2015-06-19 12:01 ` Andrey Korolyov
2015-06-19 12:02 ` Vasiliy Tolstov
2015-06-22 13:30 ` Piotr Rybicki
2015-06-22 15:58 ` Andrey Korolyov
-- strict thread matches above, loose matches on Subject: below --
2015-06-23 12:32 Piotr Rybicki
2015-06-23 18:59 ` Andrey Korolyov
2015-06-23 22:10 ` Piotr Rybicki
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.