From: "Michael S. Tsirkin" <mst@redhat.com>
To: Igor Mammedov <imammedo@redhat.com>
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, pbonzini@redhat.com
Subject: Re: [PATCH 3/5] vhost: support upto 509 memory regions
Date: Wed, 17 Jun 2015 12:11:09 +0200 [thread overview]
Message-ID: <20150617110711-mutt-send-email-mst@redhat.com> (raw)
In-Reply-To: <20150617105421.71751f44@nial.brq.redhat.com>
On Wed, Jun 17, 2015 at 10:54:21AM +0200, Igor Mammedov wrote:
> On Wed, 17 Jun 2015 09:39:06 +0200
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, Jun 17, 2015 at 09:28:02AM +0200, Igor Mammedov wrote:
> > > On Wed, 17 Jun 2015 08:34:26 +0200
> > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > >
> > > > On Wed, Jun 17, 2015 at 12:00:56AM +0200, Igor Mammedov wrote:
> > > > > On Tue, 16 Jun 2015 23:14:20 +0200
> > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > >
> > > > > > On Tue, Jun 16, 2015 at 06:33:37PM +0200, Igor Mammedov wrote:
> > > > > > > since commit
> > > > > > > 1d4e7e3 kvm: x86: increase user memory slots to 509
> > > > > > >
> > > > > > > it became possible to use a bigger amount of memory
> > > > > > > slots, which is used by memory hotplug for
> > > > > > > registering hotplugged memory.
> > > > > > > However QEMU crashes if it's used with more than ~60
> > > > > > > pc-dimm devices and vhost-net since host kernel
> > > > > > > in module vhost-net refuses to accept more than 65
> > > > > > > memory regions.
> > > > > > >
> > > > > > > Increase VHOST_MEMORY_MAX_NREGIONS from 65 to 509
> > > > > >
> > > > > > It was 64, not 65.
> > > > > >
> > > > > > > to match KVM_USER_MEM_SLOTS fixes issue for vhost-net.
> > > > > > >
> > > > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > >
> > > > > > Still thinking about this: can you reorder this to
> > > > > > be the last patch in the series please?
> > > > > sure
> > > > >
> > > > > >
> > > > > > Also - 509?
> > > > > userspace memory slots in terms of KVM, I made it match
> > > > > KVM's allotment of memory slots for userspace side.
> > > >
> > > > Maybe KVM has its reasons for this #. I don't see
> > > > why we need to match this exactly.
> > > np, I can cap it at safe 300 slots but it's unlikely that it
> > > would take cut off 1 extra hop since it's capped by QEMU
> > > at 256+[initial fragmented memory]
> >
> > But what's the point? We allocate 32 bytes per slot.
> > 300*32 = 9600 which is more than 8K, so we are doing
> > an order-3 allocation anyway.
> > If we could cap it at 8K (256 slots) that would make sense
> > since we could avoid wasting vmalloc space.
> 256 is amount of hotpluggable slots and there is no way
> to predict how initial memory would be fragmented
> (i.e. amount of slots it would take), if we guess wrong
> we are back to square one with crashing userspace.
> So I'd stay consistent with KVM's limit 509 since
> it's only limit, i.e. not actual amount of allocated slots.
>
> > I'm still not very happy with the whole approach,
> > giving userspace ability allocate 4 whole pages
> > of kernel memory like this.
> I'm working in parallel so that userspace won't take so
> many slots but it won't prevent its current versions
> crashing due to kernel limitation.
Right but at least it's not a regression. If we promise userspace to
support a ton of regions, we can't take it back later, and I'm concerned
about the memory usage.
I think it's already safe to merge the binary lookup patches, and maybe
cache and vmalloc, so that the remaining patch will be small.
>
> > > > > > I think if we are changing this, it'd be nice to
> > > > > > create a way for userspace to discover the support
> > > > > > and the # of regions supported.
> > > > > That was my first idea before extending KVM's memslots
> > > > > to teach kernel to tell qemu this number so that QEMU
> > > > > at least would be able to check if new memory slot could
> > > > > be added but I was redirected to a more simple solution
> > > > > of just extending vs everdoing things.
> > > > > Currently QEMU supports upto ~250 memslots so 509
> > > > > is about twice high we need it so it should work for near
> > > > > future
> > > >
> > > > Yes but old kernels are still around. Would be nice if you
> > > > can detect them.
> > > >
> > > > > but eventually we might still teach kernel and QEMU
> > > > > to make things more robust.
> > > >
> > > > A new ioctl would be easy to add, I think it's a good
> > > > idea generally.
> > > I can try to do something like this on top of this series.
> > >
> > > >
> > > > > >
> > > > > >
> > > > > > > ---
> > > > > > > drivers/vhost/vhost.c | 2 +-
> > > > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> > > > > > > index 99931a0..6a18c92 100644
> > > > > > > --- a/drivers/vhost/vhost.c
> > > > > > > +++ b/drivers/vhost/vhost.c
> > > > > > > @@ -30,7 +30,7 @@
> > > > > > > #include "vhost.h"
> > > > > > >
> > > > > > > enum {
> > > > > > > - VHOST_MEMORY_MAX_NREGIONS = 64,
> > > > > > > + VHOST_MEMORY_MAX_NREGIONS = 509,
> > > > > > > VHOST_MEMORY_F_LOG = 0x1,
> > > > > > > };
> > > > > > >
> > > > > > > --
> > > > > > > 1.8.3.1
next prev parent reply other threads:[~2015-06-17 10:11 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-16 16:33 [PATCH 0/5] vhost: support upto 509 memory regions Igor Mammedov
2015-06-16 16:33 ` [PATCH 1/5] vhost: use binary search instead of linear in find_region() Igor Mammedov
2015-06-16 21:07 ` Michael S. Tsirkin
2015-06-16 21:13 ` Igor Mammedov
2015-06-16 16:33 ` [PATCH 2/5] vhost: extend memory regions allocation to vmalloc Igor Mammedov
2015-06-16 16:33 ` [PATCH 3/5] vhost: support upto 509 memory regions Igor Mammedov
2015-06-16 21:14 ` Michael S. Tsirkin
2015-06-16 22:00 ` Igor Mammedov
2015-06-17 6:34 ` Michael S. Tsirkin
2015-06-17 7:28 ` Igor Mammedov
2015-06-17 7:39 ` Michael S. Tsirkin
2015-06-17 8:54 ` Igor Mammedov
2015-06-17 10:11 ` Michael S. Tsirkin [this message]
2015-06-17 10:37 ` Igor Mammedov
2015-06-17 10:46 ` Michael S. Tsirkin
2015-06-17 11:48 ` Igor Mammedov
2015-06-17 11:51 ` Michael S. Tsirkin
2015-06-17 12:23 ` Igor Mammedov
2015-06-17 13:13 ` Michael S. Tsirkin
2015-06-17 13:20 ` Paolo Bonzini
2015-06-17 14:32 ` Michael S. Tsirkin
2015-06-17 15:12 ` Igor Mammedov
2015-06-17 15:38 ` Michael S. Tsirkin
2015-06-17 16:09 ` Igor Mammedov
2015-06-17 16:30 ` Michael S. Tsirkin
2015-06-17 16:31 ` Paolo Bonzini
2015-06-17 16:34 ` Michael S. Tsirkin
2015-06-17 16:38 ` Paolo Bonzini
2015-06-17 16:41 ` Michael S. Tsirkin
2015-06-17 16:47 ` Paolo Bonzini
2015-06-17 17:32 ` Igor Mammedov
2015-06-17 19:11 ` Michael S. Tsirkin
2015-06-17 17:30 ` Igor Mammedov
2015-06-18 9:12 ` Igor Mammedov
2015-06-18 9:50 ` Michael S. Tsirkin
2015-06-18 10:03 ` Paolo Bonzini
2015-06-18 11:39 ` Igor Mammedov
2015-06-18 11:41 ` Michael S. Tsirkin
2015-06-18 11:50 ` Paolo Bonzini
2015-06-18 13:19 ` Michael S. Tsirkin
2015-06-18 13:46 ` Paolo Bonzini
2015-06-18 14:47 ` Michael S. Tsirkin
2015-06-18 15:54 ` Igor Mammedov
2015-06-18 16:02 ` Paolo Bonzini
2015-06-19 7:56 ` Michael S. Tsirkin
2015-06-19 7:57 ` Paolo Bonzini
2015-06-19 8:05 ` Michael S. Tsirkin
2015-06-19 8:52 ` Paolo Bonzini
2015-06-19 10:14 ` Michael S. Tsirkin
2015-06-19 10:44 ` Paolo Bonzini
2015-06-19 13:34 ` Michael S. Tsirkin
2015-06-19 13:34 ` Michael S. Tsirkin
2015-06-19 15:19 ` Paolo Bonzini
2015-06-19 15:19 ` Paolo Bonzini
2015-06-19 16:20 ` Michael S. Tsirkin
2015-06-19 16:20 ` Michael S. Tsirkin
2015-06-19 16:26 ` Paolo Bonzini
2015-06-19 16:26 ` Paolo Bonzini
2015-06-19 16:33 ` Michael S. Tsirkin
2015-06-19 16:33 ` Michael S. Tsirkin
2015-06-19 16:44 ` Paolo Bonzini
2015-06-19 16:44 ` Paolo Bonzini
2015-06-22 7:10 ` Igor Mammedov
2015-06-22 7:10 ` Igor Mammedov
2015-06-22 9:45 ` Paolo Bonzini
2015-06-22 9:45 ` Paolo Bonzini
2015-06-19 16:45 ` Michael S. Tsirkin
2015-06-19 16:45 ` Michael S. Tsirkin
2015-06-19 16:50 ` Paolo Bonzini
2015-06-19 16:50 ` Paolo Bonzini
2015-06-18 12:02 ` Igor Mammedov
2015-06-17 8:53 ` Paolo Bonzini
2015-06-16 16:33 ` [PATCH 4/5] vhost: add per VQ memory region caching Igor Mammedov
2015-06-16 16:33 ` [PATCH 5/5] vhost: translate_desc: optimization for desc.len < region size Igor Mammedov
2015-06-16 21:11 ` Michael S. Tsirkin
2015-06-16 21:16 ` [PATCH 0/5] vhost: support upto 509 memory regions Michael S. Tsirkin
2015-06-16 22:19 ` Igor Mammedov
2015-06-17 6:31 ` Michael S. Tsirkin
2015-06-17 7:33 ` Igor Mammedov
2015-06-17 7:40 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150617110711-mutt-send-email-mst@redhat.com \
--to=mst@redhat.com \
--cc=imammedo@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.