From: Stefano Garzarella <sgarzare@redhat.com>
To: Xuewei Niu <niuxuewei97@gmail.com>
Cc: fupan.lfp@antgroup.com, mst@redhat.com,
niuxuewei.nxw@antgroup.com, parav@nvidia.com,
stefanha@redhat.com, virtio-comment@lists.linux.dev
Subject: Re: [PATCH v6 RESEND] virtio-vsock: Add support for multi devices
Date: Thu, 10 Apr 2025 09:21:04 +0200 [thread overview]
Message-ID: <CAGxU2F4m4gO0cRz9kqHOmRG4fKDjUMOnNaes6_dik84T5bNsxQ@mail.gmail.com> (raw)
In-Reply-To: <20250410030547.3108952-1-niuxuewei.nxw@antgroup.com>
On Thu, 10 Apr 2025 at 05:06, Xuewei Niu <niuxuewei97@gmail.com> wrote:
>
> > On Wed, 9 Apr 2025 at 08:56, Xuewei Niu <niuxuewei97@gmail.com> wrote:
> > [...]
> > > >
> > > > The idea is to inform the guest which addresses are reachable by the
> > > > device, so the guest can easily decide which device to use. I'm
> > > > talking about the destination, so CID_HOST(2), CID_HYPERVISOS(0) or a
> > > > sibling VM (CID >=3).
> > > >
> > > > >
> > > > > We have had the guest_cid field in the config space. The guest knows all
> > > > > devices present in the VM.
> > > >
> > > > Okay, but how can the guest figure out from this information which
> > > > device to use to talk to the hypervisor or an application in the host?
> > > >
> > > > >
> > > > > If the app tries to bind a random CID, it will fail since the driver can't
> > > > > find the device by the CID.
> > > >
> > > > I'm not talking about the source CID on which to do bind() (which I
> > > > honestly don't like), but I'm talking about the destination CID on
> > > > which to do connect().
> > > >
> > > > > > so that the guest knows which device to use depending on the destination
> > > > > > CID.
> > > > >
> > > > > Yes, this is what I was describing in the previous comment. The message
> > > > > will be directed to the device by the destination CID.
> > > >
> > > > Sorry, I don't understand how you do this without having an
> > > > information from the device about what addresses it supports. Can you
> > > > elaborate a bit?
> > >
> > > Thanks for your explanation. So things you were talking about are as
> > > follows:
> > >
> > > 1) guest app as a server: the app MUST do `bind()` to a CID that is
> > > available in current VM.
> > > 2) guest app as a client: the guest driver picks a device and uses the
> > > device's CID as src CID, so that the guest app don't need to do `bind()`,
> > > but only do `connect()`.
> > >
> > > The key point is who takes responsibility for picking a device:
> > >
> > > 1) I prefer the guest app to do such thing: do `bind()` to pick one, then
> > > do `connect()`.
> >
> > Why?
> >
> > This implies that users must be informed in some way that they must
> > use a certain CID to communicate with the VMM and another to
> > communicate with the host application, when they could just as well
> > use CID_HYPERVISOS(0) or CID_HOST(2) for that and everything would be
> > transparent.
>
> Sorry, I didn't express it clearly. I just listed my original idea and your
> idea.
>
> What types of services can be considered as hypervisor services?
TSI.
The vsock used to implement it end up in the VMM (i.e. libkrun), right?
> In our
> case, the kata-agent is a control service for Kata, so could we consider it
> as a hypervisor service?
Nope, kata-agent is clearly a host application.
>
> How does the driver know which device is for hypervisor? A possible way is
> to add a feature, like `VIRTIO_VSOCK_F_HYPERVISOR`? If it is possible, I
> think the CID_HYPERVISOR can still be used in the case of 3+ devices to
> maintain the same behavior.
Or adding a bitfield in the config space, where the device set several
flags depending on what kind of CIDs is able to handle.
They don't need to be negotiated, IMHO. Just advertised by the device.
>
> > > 2) You prefer the guest driver: only do `connect()`, and the guest driver
> > > picks one according to the dst CID.
> > >
> > > Am I right?
> >
> > Yes, I prefer to keep things simple.
> >
> > Obviously what I said works if we only have 2 devices (or at most 3
> > where one supports only CID_HYPERVISOR, the other only CID_HOST, and
> > the third CID >=3).
> > As I anticipated, if we want to support more, then we have to
> > necessarily go with the original option of this proposal with the
> > default device, etc.
> >
> > That's why IMO we should support both.
> >
> > >
> > > I'm open to both ideas, but I have some concerns:
> > >
> > > 1) Two devices are in the different namespaces, e.g. host kernel(vhost) and
> > > hybrid vsock(vhost-user), which might cause two same CIDs (e.g.
> > > VMADDR_CID_HOST). If that happened, the driver can't distinguish them.
> > > Instead, we can avoid this by letting the guest app pick a device.
> >
> > Yep, in this case you need that, but how the user knows which source CID
> > to use?
>
> The CID should be confirmed in advance, and users should ensure that there
> are no conflicts with CIDs.
Exactly, this is kind of against what we call zero-config for vsock.
We could look at it the same way as ports, though, where the user has
to know which one to use to reach a service, so I'm not completely
against that.
>
> > Again I'm not saying we should not implement this proposal, I'm saying
> > that we should also add the other option in order to keep vsock simple
> > when you have 2 devices, one used to talk with the VMM and the other
> > to talk with applications.
>
> Got it. Thanks for clarifying this.
>
> > > 2) What if the number of VMs is too large? For instance, 1,000 VMs (1,000
> > > CIDs) will need at least 8000B of config space. (Hmm, it looks like an
> > > extreme example, I don't know if it will happen in real world.)
> >
> > I didn't get sorry, can you elaborate a bit?
>
> I misunderstood. I thought you are trying to have an array to save all
> supported dst CIDs in the case of 3+ devices. Please just ignore this.
>
> > Anyway, for me we can also go ahead with this proposal and add the
> > other one later.
> > I would like to keep the guest simple, and IMO hiding the complexity
> > in the driver is not bad for your use case where one device talks to
> > the VMM and the other talks to applications in the host.
>
> Fair enough. I am very glad to see that we can move forward with this
> feature ;)
:-)
Thanks,
Stefano
next prev parent reply other threads:[~2025-04-10 7:21 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-24 6:43 [PATCH v6 RESEND] virtio-vsock: Add support for multi devices Xuewei Niu
2025-03-24 13:51 ` Stefano Garzarella
2025-03-25 3:19 ` Xuewei Niu
2025-03-26 8:50 ` Stefano Garzarella
2025-03-26 10:00 ` Xuewei Niu
2025-03-26 10:32 ` Stefano Garzarella
2025-03-26 10:36 ` Stefano Garzarella
2025-03-26 2:59 ` Xuewei Niu
2025-03-26 9:03 ` Stefano Garzarella
2025-03-27 8:18 ` Xuewei Niu
2025-03-31 6:18 ` Xuewei Niu
2025-04-01 11:15 ` Stefano Garzarella
2025-04-07 2:17 ` Xuewei Niu
2025-04-08 13:34 ` Stefano Garzarella
2025-04-09 6:55 ` Xuewei Niu
2025-04-09 9:34 ` Stefano Garzarella
2025-04-10 3:05 ` Xuewei Niu
2025-04-10 7:21 ` Stefano Garzarella [this message]
2025-04-10 8:58 ` Xuewei Niu
2025-04-10 10:38 ` Stefano Garzarella
2025-04-10 10:47 ` Xuewei Niu
2025-04-10 10:49 ` Stefano Garzarella
2025-04-10 13:47 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAGxU2F4m4gO0cRz9kqHOmRG4fKDjUMOnNaes6_dik84T5bNsxQ@mail.gmail.com \
--to=sgarzare@redhat.com \
--cc=fupan.lfp@antgroup.com \
--cc=mst@redhat.com \
--cc=niuxuewei.nxw@antgroup.com \
--cc=niuxuewei97@gmail.com \
--cc=parav@nvidia.com \
--cc=stefanha@redhat.com \
--cc=virtio-comment@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).