From: Xuewei Niu <niuxuewei97@gmail.com>
To: sgarzare@redhat.com
Cc: fupan.lfp@antgroup.com, mst@redhat.com,
niuxuewei.nxw@antgroup.com, niuxuewei97@gmail.com,
parav@nvidia.com, stefanha@redhat.com,
virtio-comment@lists.linux.dev
Subject: Re: [PATCH v6 RESEND] virtio-vsock: Add support for multi devices
Date: Thu, 10 Apr 2025 16:58:54 +0800 [thread overview]
Message-ID: <20250410085854.3127511-1-niuxuewei.nxw@antgroup.com> (raw)
In-Reply-To: <CAGxU2F4m4gO0cRz9kqHOmRG4fKDjUMOnNaes6_dik84T5bNsxQ@mail.gmail.com>
> On Thu, 10 Apr 2025 at 05:06, Xuewei Niu <niuxuewei97@gmail.com> wrote:
> >
> > > On Wed, 9 Apr 2025 at 08:56, Xuewei Niu <niuxuewei97@gmail.com> wrote:
> > > [...]
> > > > >
> > > > > The idea is to inform the guest which addresses are reachable by the
> > > > > device, so the guest can easily decide which device to use. I'm
> > > > > talking about the destination, so CID_HOST(2), CID_HYPERVISOS(0) or a
> > > > > sibling VM (CID >=3).
> > > > >
> > > > > >
> > > > > > We have had the guest_cid field in the config space. The guest knows all
> > > > > > devices present in the VM.
> > > > >
> > > > > Okay, but how can the guest figure out from this information which
> > > > > device to use to talk to the hypervisor or an application in the host?
> > > > >
> > > > > >
> > > > > > If the app tries to bind a random CID, it will fail since the driver can't
> > > > > > find the device by the CID.
> > > > >
> > > > > I'm not talking about the source CID on which to do bind() (which I
> > > > > honestly don't like), but I'm talking about the destination CID on
> > > > > which to do connect().
> > > > >
> > > > > > > so that the guest knows which device to use depending on the destination
> > > > > > > CID.
> > > > > >
> > > > > > Yes, this is what I was describing in the previous comment. The message
> > > > > > will be directed to the device by the destination CID.
> > > > >
> > > > > Sorry, I don't understand how you do this without having an
> > > > > information from the device about what addresses it supports. Can you
> > > > > elaborate a bit?
> > > >
> > > > Thanks for your explanation. So things you were talking about are as
> > > > follows:
> > > >
> > > > 1) guest app as a server: the app MUST do `bind()` to a CID that is
> > > > available in current VM.
> > > > 2) guest app as a client: the guest driver picks a device and uses the
> > > > device's CID as src CID, so that the guest app don't need to do `bind()`,
> > > > but only do `connect()`.
> > > >
> > > > The key point is who takes responsibility for picking a device:
> > > >
> > > > 1) I prefer the guest app to do such thing: do `bind()` to pick one, then
> > > > do `connect()`.
> > >
> > > Why?
> > >
> > > This implies that users must be informed in some way that they must
> > > use a certain CID to communicate with the VMM and another to
> > > communicate with the host application, when they could just as well
> > > use CID_HYPERVISOS(0) or CID_HOST(2) for that and everything would be
> > > transparent.
> >
> > Sorry, I didn't express it clearly. I just listed my original idea and your
> > idea.
> >
> > What types of services can be considered as hypervisor services?
>
> TSI.
> The vsock used to implement it end up in the VMM (i.e. libkrun), right?
>
> > In our
> > case, the kata-agent is a control service for Kata, so could we consider it
> > as a hypervisor service?
>
> Nope, kata-agent is clearly a host application.
Make sense for me.
> > How does the driver know which device is for hypervisor? A possible way is
> > to add a feature, like `VIRTIO_VSOCK_F_HYPERVISOR`? If it is possible, I
> > think the CID_HYPERVISOR can still be used in the case of 3+ devices to
> > maintain the same behavior.
>
> Or adding a bitfield in the config space, where the device set several
> flags depending on what kind of CIDs is able to handle.
> They don't need to be negotiated, IMHO. Just advertised by the device.
Will do in the v7.
Thanks,
Xuewei
> > > > 2) You prefer the guest driver: only do `connect()`, and the guest driver
> > > > picks one according to the dst CID.
> > > >
> > > > Am I right?
> > >
> > > Yes, I prefer to keep things simple.
> > >
> > > Obviously what I said works if we only have 2 devices (or at most 3
> > > where one supports only CID_HYPERVISOR, the other only CID_HOST, and
> > > the third CID >=3).
> > > As I anticipated, if we want to support more, then we have to
> > > necessarily go with the original option of this proposal with the
> > > default device, etc.
> > >
> > > That's why IMO we should support both.
> > >
> > > >
> > > > I'm open to both ideas, but I have some concerns:
> > > >
> > > > 1) Two devices are in the different namespaces, e.g. host kernel(vhost) and
> > > > hybrid vsock(vhost-user), which might cause two same CIDs (e.g.
> > > > VMADDR_CID_HOST). If that happened, the driver can't distinguish them.
> > > > Instead, we can avoid this by letting the guest app pick a device.
> > >
> > > Yep, in this case you need that, but how the user knows which source CID
> > > to use?
> >
> > The CID should be confirmed in advance, and users should ensure that there
> > are no conflicts with CIDs.
>
> Exactly, this is kind of against what we call zero-config for vsock.
>
> We could look at it the same way as ports, though, where the user has
> to know which one to use to reach a service, so I'm not completely
> against that.
>
> >
> > > Again I'm not saying we should not implement this proposal, I'm saying
> > > that we should also add the other option in order to keep vsock simple
> > > when you have 2 devices, one used to talk with the VMM and the other
> > > to talk with applications.
> >
> > Got it. Thanks for clarifying this.
> >
> > > > 2) What if the number of VMs is too large? For instance, 1,000 VMs (1,000
> > > > CIDs) will need at least 8000B of config space. (Hmm, it looks like an
> > > > extreme example, I don't know if it will happen in real world.)
> > >
> > > I didn't get sorry, can you elaborate a bit?
> >
> > I misunderstood. I thought you are trying to have an array to save all
> > supported dst CIDs in the case of 3+ devices. Please just ignore this.
> >
> > > Anyway, for me we can also go ahead with this proposal and add the
> > > other one later.
> > > I would like to keep the guest simple, and IMO hiding the complexity
> > > in the driver is not bad for your use case where one device talks to
> > > the VMM and the other talks to applications in the host.
> >
> > Fair enough. I am very glad to see that we can move forward with this
> > feature ;)
>
> :-)
>
> Thanks,
> Stefano
next prev parent reply other threads:[~2025-04-10 8:59 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-24 6:43 [PATCH v6 RESEND] virtio-vsock: Add support for multi devices Xuewei Niu
2025-03-24 13:51 ` Stefano Garzarella
2025-03-25 3:19 ` Xuewei Niu
2025-03-26 8:50 ` Stefano Garzarella
2025-03-26 10:00 ` Xuewei Niu
2025-03-26 10:32 ` Stefano Garzarella
2025-03-26 10:36 ` Stefano Garzarella
2025-03-26 2:59 ` Xuewei Niu
2025-03-26 9:03 ` Stefano Garzarella
2025-03-27 8:18 ` Xuewei Niu
2025-03-31 6:18 ` Xuewei Niu
2025-04-01 11:15 ` Stefano Garzarella
2025-04-07 2:17 ` Xuewei Niu
2025-04-08 13:34 ` Stefano Garzarella
2025-04-09 6:55 ` Xuewei Niu
2025-04-09 9:34 ` Stefano Garzarella
2025-04-10 3:05 ` Xuewei Niu
2025-04-10 7:21 ` Stefano Garzarella
2025-04-10 8:58 ` Xuewei Niu [this message]
2025-04-10 10:38 ` Stefano Garzarella
2025-04-10 10:47 ` Xuewei Niu
2025-04-10 10:49 ` Stefano Garzarella
2025-04-10 13:47 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250410085854.3127511-1-niuxuewei.nxw@antgroup.com \
--to=niuxuewei97@gmail.com \
--cc=fupan.lfp@antgroup.com \
--cc=mst@redhat.com \
--cc=niuxuewei.nxw@antgroup.com \
--cc=parav@nvidia.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=virtio-comment@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).