* [PATCH v7 0/4] Implement vdpasim suspend operation
@ 2022-08-10 17:15 Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 1/4] vdpa: Add " Eugenio Pérez
` (4 more replies)
0 siblings, 5 replies; 19+ messages in thread
From: Eugenio Pérez @ 2022-08-10 17:15 UTC (permalink / raw)
To: kvm, Michael S. Tsirkin, linux-kernel, Jason Wang, virtualization,
netdev
Cc: dinang, martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar,
ecree.xilinx, martinh, Stefano Garzarella, pabloc, habetsm.xilinx,
lvivier, Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand,
Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min,
Dan Carpenter, Christophe JAILLET
Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
that backend feature and userspace can effectively suspend the device.
This is a must before getting virtqueue indexes (base) for live migration,
since the device could modify them after userland gets them. There are
individual ways to perform that action for some devices
(VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
way to perform it for any vhost device (and, in particular, vhost-vdpa).
After a successful return of ioctl the device must not process more virtqueue
descriptors. The device can answer to read or writes of config fields as if it
were not suspended. In particular, writing to "queue_enable" with a value of 1
will not make the device start processing virtqueue buffers.
In the future, we will provide features similar to
VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
Applied on top of [1] branch after removing the old commits.
Comments are welcome.
v7:
* Remove ioctl leftover argument and update doc accordingly.
v6:
* Remove the resume operation, making the ioctl simpler. We can always add
another ioctl for VM_STOP/VM_RESUME operation later.
* s/stop/suspend/ to differentiate more from reset.
* Clarify scope of the suspend operation.
v5:
* s/not stop/resume/ in doc.
v4:
* Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
v3:
* s/VHOST_STOP/VHOST_VDPA_STOP/
* Add documentation and requirements of the ioctl above its definition.
v2:
* Replace raw _F_STOP with BIT_ULL(_F_STOP).
* Fix obtaining of stop ioctl arg (it was not obtained but written).
* Add stop to vdpa_sim_blk.
[1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
Eugenio Pérez (4):
vdpa: Add suspend operation
vhost-vdpa: introduce SUSPEND backend feature bit
vhost-vdpa: uAPI to suspend the device
vdpa_sim: Implement suspend vdpa op
drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 +++++++++++
drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
drivers/vhost/vdpa.c | 35 +++++++++++++++++++++++++++-
include/linux/vdpa.h | 4 ++++
include/uapi/linux/vhost.h | 9 +++++++
include/uapi/linux/vhost_types.h | 2 ++
8 files changed, 70 insertions(+), 1 deletion(-)
--
2.31.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v7 1/4] vdpa: Add suspend operation
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
@ 2022-08-10 17:15 ` Eugenio Pérez
2022-08-11 8:27 ` Michael S. Tsirkin
2022-08-10 17:15 ` [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit Eugenio Pérez
` (3 subsequent siblings)
4 siblings, 1 reply; 19+ messages in thread
From: Eugenio Pérez @ 2022-08-10 17:15 UTC (permalink / raw)
To: kvm, Michael S. Tsirkin, linux-kernel, Jason Wang, virtualization,
netdev
Cc: dinang, martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar,
ecree.xilinx, martinh, Stefano Garzarella, pabloc, habetsm.xilinx,
lvivier, Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand,
Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min,
Dan Carpenter, Christophe JAILLET
This operation is optional: It it's not implemented, backend feature bit
will not be exposed.
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
include/linux/vdpa.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
index 7b4a13d3bd91..d282f464d2f1 100644
--- a/include/linux/vdpa.h
+++ b/include/linux/vdpa.h
@@ -218,6 +218,9 @@ struct vdpa_map_file {
* @reset: Reset device
* @vdev: vdpa device
* Returns integer: success (0) or error (< 0)
+ * @suspend: Suspend or resume the device (optional)
+ * @vdev: vdpa device
+ * Returns integer: success (0) or error (< 0)
* @get_config_size: Get the size of the configuration space includes
* fields that are conditional on feature bits.
* @vdev: vdpa device
@@ -319,6 +322,7 @@ struct vdpa_config_ops {
u8 (*get_status)(struct vdpa_device *vdev);
void (*set_status)(struct vdpa_device *vdev, u8 status);
int (*reset)(struct vdpa_device *vdev);
+ int (*suspend)(struct vdpa_device *vdev);
size_t (*get_config_size)(struct vdpa_device *vdev);
void (*get_config)(struct vdpa_device *vdev, unsigned int offset,
void *buf, unsigned int len);
--
2.31.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 1/4] vdpa: Add " Eugenio Pérez
@ 2022-08-10 17:15 ` Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device Eugenio Pérez
` (2 subsequent siblings)
4 siblings, 0 replies; 19+ messages in thread
From: Eugenio Pérez @ 2022-08-10 17:15 UTC (permalink / raw)
To: kvm, Michael S. Tsirkin, linux-kernel, Jason Wang, virtualization,
netdev
Cc: dinang, martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar,
ecree.xilinx, martinh, Stefano Garzarella, pabloc, habetsm.xilinx,
lvivier, Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand,
Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min,
Dan Carpenter, Christophe JAILLET
Userland knows if it can suspend the device or not by checking this feature
bit.
It's only offered if the vdpa driver backend implements the suspend()
operation callback, and to offer it or userland to ack it if the backend
does not offer that callback is an error.
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220623160738.632852-3-eperezma@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/vhost/vdpa.c | 16 +++++++++++++++-
include/uapi/linux/vhost_types.h | 2 ++
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 23dcbfdfa13b..3d636e192061 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -347,6 +347,14 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v,
return 0;
}
+static bool vhost_vdpa_can_suspend(const struct vhost_vdpa *v)
+{
+ struct vdpa_device *vdpa = v->vdpa;
+ const struct vdpa_config_ops *ops = vdpa->config;
+
+ return ops->suspend;
+}
+
static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep)
{
struct vdpa_device *vdpa = v->vdpa;
@@ -577,7 +585,11 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
if (cmd == VHOST_SET_BACKEND_FEATURES) {
if (copy_from_user(&features, featurep, sizeof(features)))
return -EFAULT;
- if (features & ~VHOST_VDPA_BACKEND_FEATURES)
+ if (features & ~(VHOST_VDPA_BACKEND_FEATURES |
+ BIT_ULL(VHOST_BACKEND_F_SUSPEND)))
+ return -EOPNOTSUPP;
+ if ((features & BIT_ULL(VHOST_BACKEND_F_SUSPEND)) &&
+ !vhost_vdpa_can_suspend(v))
return -EOPNOTSUPP;
vhost_set_backend_features(&v->vdev, features);
return 0;
@@ -628,6 +640,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
break;
case VHOST_GET_BACKEND_FEATURES:
features = VHOST_VDPA_BACKEND_FEATURES;
+ if (vhost_vdpa_can_suspend(v))
+ features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND);
if (copy_to_user(featurep, &features, sizeof(features)))
r = -EFAULT;
break;
diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h
index 634cee485abb..1bdd6e363f4c 100644
--- a/include/uapi/linux/vhost_types.h
+++ b/include/uapi/linux/vhost_types.h
@@ -161,5 +161,7 @@ struct vhost_vdpa_iova_range {
* message
*/
#define VHOST_BACKEND_F_IOTLB_ASID 0x3
+/* Device can be suspended */
+#define VHOST_BACKEND_F_SUSPEND 0x4
#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 1/4] vdpa: Add " Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit Eugenio Pérez
@ 2022-08-10 17:15 ` Eugenio Pérez
2022-08-11 8:29 ` Michael S. Tsirkin
2022-08-10 17:15 ` [PATCH v7 4/4] vdpa_sim: Implement suspend vdpa op Eugenio Pérez
2022-08-10 19:19 ` Michael S. Tsirkin
4 siblings, 1 reply; 19+ messages in thread
From: Eugenio Pérez @ 2022-08-10 17:15 UTC (permalink / raw)
To: kvm, Michael S. Tsirkin, linux-kernel, Jason Wang, virtualization,
netdev
Cc: dinang, martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar,
ecree.xilinx, martinh, Stefano Garzarella, pabloc, habetsm.xilinx,
lvivier, Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand,
Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min,
Dan Carpenter, Christophe JAILLET
The ioctl adds support for suspending the device from userspace.
This is a must before getting virtqueue indexes (base) for live migration,
since the device could modify them after userland gets them. There are
individual ways to perform that action for some devices
(VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
way to perform it for any vhost device (and, in particular, vhost-vdpa).
After a successful return of the ioctl call the device must not process
more virtqueue descriptors. The device can answer to read or writes of
config fields as if it were not suspended. In particular, writing to
"queue_enable" with a value of 1 will not make the device start
processing buffers of the virtqueue.
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220623160738.632852-4-eperezma@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
v7: Delete argument to ioctl, unused
---
drivers/vhost/vdpa.c | 19 +++++++++++++++++++
include/uapi/linux/vhost.h | 9 +++++++++
2 files changed, 28 insertions(+)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 3d636e192061..7fa671ac4bdf 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vdpa *v, u32 __user *argp)
return 0;
}
+/* After a successful return of ioctl the device must not process more
+ * virtqueue descriptors. The device can answer to read or writes of config
+ * fields as if it were not suspended. In particular, writing to "queue_enable"
+ * with a value of 1 will not make the device start processing buffers.
+ */
+static long vhost_vdpa_suspend(struct vhost_vdpa *v)
+{
+ struct vdpa_device *vdpa = v->vdpa;
+ const struct vdpa_config_ops *ops = vdpa->config;
+
+ if (!ops->suspend)
+ return -EOPNOTSUPP;
+
+ return ops->suspend(vdpa);
+}
+
static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
void __user *argp)
{
@@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
case VHOST_VDPA_GET_VQS_COUNT:
r = vhost_vdpa_get_vqs_count(v, argp);
break;
+ case VHOST_VDPA_SUSPEND:
+ r = vhost_vdpa_suspend(v);
+ break;
default:
r = vhost_dev_ioctl(&v->vdev, cmd, argp);
if (r == -ENOIOCTLCMD)
diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
index cab645d4a645..f9f115a7c75b 100644
--- a/include/uapi/linux/vhost.h
+++ b/include/uapi/linux/vhost.h
@@ -171,4 +171,13 @@
#define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \
struct vhost_vring_state)
+/* Suspend a device so it does not process virtqueue requests anymore
+ *
+ * After the return of ioctl the device must preserve all the necessary state
+ * (the virtqueue vring base plus the possible device specific states) that is
+ * required for restoring in the future. The device must not change its
+ * configuration after that point.
+ */
+#define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D)
+
#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v7 4/4] vdpa_sim: Implement suspend vdpa op
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
` (2 preceding siblings ...)
2022-08-10 17:15 ` [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device Eugenio Pérez
@ 2022-08-10 17:15 ` Eugenio Pérez
2022-08-10 19:19 ` Michael S. Tsirkin
4 siblings, 0 replies; 19+ messages in thread
From: Eugenio Pérez @ 2022-08-10 17:15 UTC (permalink / raw)
To: kvm, Michael S. Tsirkin, linux-kernel, Jason Wang, virtualization,
netdev
Cc: dinang, martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar,
ecree.xilinx, martinh, Stefano Garzarella, pabloc, habetsm.xilinx,
lvivier, Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand,
Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min,
Dan Carpenter, Christophe JAILLET
Implement suspend operation for vdpa_sim devices, so vhost-vdpa will
offer that backend feature and userspace can effectively suspend the
device.
This is a must before get virtqueue indexes (base) for live migration,
since the device could modify them after userland gets them. There are
individual ways to perform that action for some devices
(VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
way to perform it for any vhost device (and, in particular, vhost-vdpa).
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220623160738.632852-5-eperezma@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
v7: Delete leftover resume code
---
drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 ++++++++++++++
drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
4 files changed, 21 insertions(+)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 0f2865899647..79a50edf8998 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -107,6 +107,7 @@ static void vdpasim_do_reset(struct vdpasim *vdpasim)
for (i = 0; i < vdpasim->dev_attr.nas; i++)
vhost_iotlb_reset(&vdpasim->iommu[i]);
+ vdpasim->running = true;
spin_unlock(&vdpasim->iommu_lock);
vdpasim->features = 0;
@@ -505,6 +506,17 @@ static int vdpasim_reset(struct vdpa_device *vdpa)
return 0;
}
+static int vdpasim_suspend(struct vdpa_device *vdpa)
+{
+ struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
+
+ spin_lock(&vdpasim->lock);
+ vdpasim->running = false;
+ spin_unlock(&vdpasim->lock);
+
+ return 0;
+}
+
static size_t vdpasim_get_config_size(struct vdpa_device *vdpa)
{
struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
@@ -694,6 +706,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = {
.get_status = vdpasim_get_status,
.set_status = vdpasim_set_status,
.reset = vdpasim_reset,
+ .suspend = vdpasim_suspend,
.get_config_size = vdpasim_get_config_size,
.get_config = vdpasim_get_config,
.set_config = vdpasim_set_config,
@@ -726,6 +739,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = {
.get_status = vdpasim_get_status,
.set_status = vdpasim_set_status,
.reset = vdpasim_reset,
+ .suspend = vdpasim_suspend,
.get_config_size = vdpasim_get_config_size,
.get_config = vdpasim_get_config,
.set_config = vdpasim_set_config,
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h
index 622782e92239..061986f30911 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.h
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h
@@ -66,6 +66,7 @@ struct vdpasim {
u32 generation;
u64 features;
u32 groups;
+ bool running;
/* spinlock to synchronize iommu table */
spinlock_t iommu_lock;
};
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
index 42d401d43911..bcdb1982c378 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
@@ -204,6 +204,9 @@ static void vdpasim_blk_work(struct work_struct *work)
if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK))
goto out;
+ if (!vdpasim->running)
+ goto out;
+
for (i = 0; i < VDPASIM_BLK_VQ_NUM; i++) {
struct vdpasim_virtqueue *vq = &vdpasim->vqs[i];
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
index 5125976a4df8..886449e88502 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
@@ -154,6 +154,9 @@ static void vdpasim_net_work(struct work_struct *work)
spin_lock(&vdpasim->lock);
+ if (!vdpasim->running)
+ goto out;
+
if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK))
goto out;
--
2.31.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v7 0/4] Implement vdpasim suspend operation
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
@ 2022-08-10 19:19 ` Michael S. Tsirkin
2022-08-10 17:15 ` [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit Eugenio Pérez
` (3 subsequent siblings)
4 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-10 19:19 UTC (permalink / raw)
To: Eugenio Pérez
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Piotr.Uminski, martinh, Xie Yongji, dinang,
habetsm.xilinx, Longpeng, Dan Carpenter, lvivier, netdev,
linux-kernel, ecree.xilinx, hanand, martinpo, gautam.dawar,
Christophe JAILLET, Zhu Lingshan
On Wed, Aug 10, 2022 at 07:15:08PM +0200, Eugenio Pérez wrote:
> Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
> that backend feature and userspace can effectively suspend the device.
>
> This is a must before getting virtqueue indexes (base) for live migration,
> since the device could modify them after userland gets them. There are
> individual ways to perform that action for some devices
> (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> way to perform it for any vhost device (and, in particular, vhost-vdpa).
>
> After a successful return of ioctl the device must not process more virtqueue
> descriptors. The device can answer to read or writes of config fields as if it
> were not suspended. In particular, writing to "queue_enable" with a value of 1
> will not make the device start processing virtqueue buffers.
>
> In the future, we will provide features similar to
> VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
>
> Applied on top of [1] branch after removing the old commits.
Except, I can't really do this without invaliding all testing.
Can't you post an incremental patch?
> Comments are welcome.
>
> v7:
> * Remove ioctl leftover argument and update doc accordingly.
> v6:
> * Remove the resume operation, making the ioctl simpler. We can always add
> another ioctl for VM_STOP/VM_RESUME operation later.
> * s/stop/suspend/ to differentiate more from reset.
> * Clarify scope of the suspend operation.
>
> v5:
> * s/not stop/resume/ in doc.
>
> v4:
> * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
>
> v3:
> * s/VHOST_STOP/VHOST_VDPA_STOP/
> * Add documentation and requirements of the ioctl above its definition.
>
> v2:
> * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> * Fix obtaining of stop ioctl arg (it was not obtained but written).
> * Add stop to vdpa_sim_blk.
>
> [1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
>
> Eugenio Pérez (4):
> vdpa: Add suspend operation
> vhost-vdpa: introduce SUSPEND backend feature bit
> vhost-vdpa: uAPI to suspend the device
> vdpa_sim: Implement suspend vdpa op
>
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 +++++++++++
> drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
> drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
> drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
> drivers/vhost/vdpa.c | 35 +++++++++++++++++++++++++++-
> include/linux/vdpa.h | 4 ++++
> include/uapi/linux/vhost.h | 9 +++++++
> include/uapi/linux/vhost_types.h | 2 ++
> 8 files changed, 70 insertions(+), 1 deletion(-)
>
> --
> 2.31.1
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 0/4] Implement vdpasim suspend operation
@ 2022-08-10 19:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-10 19:19 UTC (permalink / raw)
To: Eugenio Pérez
Cc: kvm, linux-kernel, Jason Wang, virtualization, netdev, dinang,
martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar, ecree.xilinx,
martinh, Stefano Garzarella, pabloc, habetsm.xilinx, lvivier,
Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand, Parav Pandit,
Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min, Dan Carpenter,
Christophe JAILLET
On Wed, Aug 10, 2022 at 07:15:08PM +0200, Eugenio Pérez wrote:
> Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
> that backend feature and userspace can effectively suspend the device.
>
> This is a must before getting virtqueue indexes (base) for live migration,
> since the device could modify them after userland gets them. There are
> individual ways to perform that action for some devices
> (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> way to perform it for any vhost device (and, in particular, vhost-vdpa).
>
> After a successful return of ioctl the device must not process more virtqueue
> descriptors. The device can answer to read or writes of config fields as if it
> were not suspended. In particular, writing to "queue_enable" with a value of 1
> will not make the device start processing virtqueue buffers.
>
> In the future, we will provide features similar to
> VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
>
> Applied on top of [1] branch after removing the old commits.
Except, I can't really do this without invaliding all testing.
Can't you post an incremental patch?
> Comments are welcome.
>
> v7:
> * Remove ioctl leftover argument and update doc accordingly.
> v6:
> * Remove the resume operation, making the ioctl simpler. We can always add
> another ioctl for VM_STOP/VM_RESUME operation later.
> * s/stop/suspend/ to differentiate more from reset.
> * Clarify scope of the suspend operation.
>
> v5:
> * s/not stop/resume/ in doc.
>
> v4:
> * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
>
> v3:
> * s/VHOST_STOP/VHOST_VDPA_STOP/
> * Add documentation and requirements of the ioctl above its definition.
>
> v2:
> * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> * Fix obtaining of stop ioctl arg (it was not obtained but written).
> * Add stop to vdpa_sim_blk.
>
> [1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
>
> Eugenio Pérez (4):
> vdpa: Add suspend operation
> vhost-vdpa: introduce SUSPEND backend feature bit
> vhost-vdpa: uAPI to suspend the device
> vdpa_sim: Implement suspend vdpa op
>
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 +++++++++++
> drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
> drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
> drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
> drivers/vhost/vdpa.c | 35 +++++++++++++++++++++++++++-
> include/linux/vdpa.h | 4 ++++
> include/uapi/linux/vhost.h | 9 +++++++
> include/uapi/linux/vhost_types.h | 2 ++
> 8 files changed, 70 insertions(+), 1 deletion(-)
>
> --
> 2.31.1
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 0/4] Implement vdpasim suspend operation
2022-08-10 19:19 ` Michael S. Tsirkin
(?)
@ 2022-08-11 5:35 ` Eugenio Perez Martin
-1 siblings, 0 replies; 19+ messages in thread
From: Eugenio Perez Martin @ 2022-08-11 5:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: kvm list, linux-kernel, Jason Wang, virtualization, netdev,
Dinan Gunawardena, Martin Porter, Wu Zongyong, Uminski, Piotr,
Dawar, Gautam, ecree.xilinx, Martin Petrus Hubertus Habets,
Stefano Garzarella, Pablo Cascon Katchadourian, habetsm.xilinx,
Laurent Vivier, Zhu Lingshan, Kamde, Tanuj, Longpeng, Cindy Lu,
Harpreet Singh Anand, Parav Pandit, Si-Wei Liu, Eli Cohen,
Xie Yongji, Zhang Min, Dan Carpenter, Christophe JAILLET
On Wed, Aug 10, 2022 at 9:20 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Aug 10, 2022 at 07:15:08PM +0200, Eugenio Pérez wrote:
> > Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer
> > that backend feature and userspace can effectively suspend the device.
> >
> > This is a must before getting virtqueue indexes (base) for live migration,
> > since the device could modify them after userland gets them. There are
> > individual ways to perform that action for some devices
> > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> > way to perform it for any vhost device (and, in particular, vhost-vdpa).
> >
> > After a successful return of ioctl the device must not process more virtqueue
> > descriptors. The device can answer to read or writes of config fields as if it
> > were not suspended. In particular, writing to "queue_enable" with a value of 1
> > will not make the device start processing virtqueue buffers.
> >
> > In the future, we will provide features similar to
> > VHOST_USER_GET_INFLIGHT_FD so the device can save pending operations.
> >
> > Applied on top of [1] branch after removing the old commits.
>
> Except, I can't really do this without invaliding all testing.
> Can't you post an incremental patch?
>
Oops, sorry.
I can send it for doc and internal code. But is it ok to remove an
ioctl arg with incremental patches?
Thanks!
> > Comments are welcome.
> >
> > v7:
> > * Remove ioctl leftover argument and update doc accordingly.
>
> > v6:
> > * Remove the resume operation, making the ioctl simpler. We can always add
> > another ioctl for VM_STOP/VM_RESUME operation later.
> > * s/stop/suspend/ to differentiate more from reset.
> > * Clarify scope of the suspend operation.
> >
> > v5:
> > * s/not stop/resume/ in doc.
> >
> > v4:
> > * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too.
> >
> > v3:
> > * s/VHOST_STOP/VHOST_VDPA_STOP/
> > * Add documentation and requirements of the ioctl above its definition.
> >
> > v2:
> > * Replace raw _F_STOP with BIT_ULL(_F_STOP).
> > * Fix obtaining of stop ioctl arg (it was not obtained but written).
> > * Add stop to vdpa_sim_blk.
> >
> > [1] git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git
> >
> > Eugenio Pérez (4):
> > vdpa: Add suspend operation
> > vhost-vdpa: introduce SUSPEND backend feature bit
> > vhost-vdpa: uAPI to suspend the device
> > vdpa_sim: Implement suspend vdpa op
> >
> > drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 +++++++++++
> > drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +
> > drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++
> > drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++
> > drivers/vhost/vdpa.c | 35 +++++++++++++++++++++++++++-
> > include/linux/vdpa.h | 4 ++++
> > include/uapi/linux/vhost.h | 9 +++++++
> > include/uapi/linux/vhost_types.h | 2 ++
> > 8 files changed, 70 insertions(+), 1 deletion(-)
> >
> > --
> > 2.31.1
> >
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
2022-08-10 17:15 ` [PATCH v7 1/4] vdpa: Add " Eugenio Pérez
@ 2022-08-11 8:27 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 8:27 UTC (permalink / raw)
To: Eugenio Pérez
Cc: kvm, linux-kernel, Jason Wang, virtualization, netdev, dinang,
martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar, ecree.xilinx,
martinh, Stefano Garzarella, pabloc, habetsm.xilinx, lvivier,
Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand, Parav Pandit,
Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min, Dan Carpenter,
Christophe JAILLET
On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> This operation is optional: It it's not implemented, backend feature bit
> will not be exposed.
>
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
What is this message id doing here?
> ---
> include/linux/vdpa.h | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
> index 7b4a13d3bd91..d282f464d2f1 100644
> --- a/include/linux/vdpa.h
> +++ b/include/linux/vdpa.h
> @@ -218,6 +218,9 @@ struct vdpa_map_file {
> * @reset: Reset device
> * @vdev: vdpa device
> * Returns integer: success (0) or error (< 0)
> + * @suspend: Suspend or resume the device (optional)
> + * @vdev: vdpa device
> + * Returns integer: success (0) or error (< 0)
> * @get_config_size: Get the size of the configuration space includes
> * fields that are conditional on feature bits.
> * @vdev: vdpa device
> @@ -319,6 +322,7 @@ struct vdpa_config_ops {
> u8 (*get_status)(struct vdpa_device *vdev);
> void (*set_status)(struct vdpa_device *vdev, u8 status);
> int (*reset)(struct vdpa_device *vdev);
> + int (*suspend)(struct vdpa_device *vdev);
> size_t (*get_config_size)(struct vdpa_device *vdev);
> void (*get_config)(struct vdpa_device *vdev, unsigned int offset,
> void *buf, unsigned int len);
> --
> 2.31.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
@ 2022-08-11 8:27 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 8:27 UTC (permalink / raw)
To: Eugenio Pérez
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Piotr.Uminski, martinh, Xie Yongji, dinang,
habetsm.xilinx, Longpeng, Dan Carpenter, lvivier, netdev,
linux-kernel, ecree.xilinx, hanand, martinpo, gautam.dawar,
Christophe JAILLET, Zhu Lingshan
On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> This operation is optional: It it's not implemented, backend feature bit
> will not be exposed.
>
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
What is this message id doing here?
> ---
> include/linux/vdpa.h | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
> index 7b4a13d3bd91..d282f464d2f1 100644
> --- a/include/linux/vdpa.h
> +++ b/include/linux/vdpa.h
> @@ -218,6 +218,9 @@ struct vdpa_map_file {
> * @reset: Reset device
> * @vdev: vdpa device
> * Returns integer: success (0) or error (< 0)
> + * @suspend: Suspend or resume the device (optional)
> + * @vdev: vdpa device
> + * Returns integer: success (0) or error (< 0)
> * @get_config_size: Get the size of the configuration space includes
> * fields that are conditional on feature bits.
> * @vdev: vdpa device
> @@ -319,6 +322,7 @@ struct vdpa_config_ops {
> u8 (*get_status)(struct vdpa_device *vdev);
> void (*set_status)(struct vdpa_device *vdev, u8 status);
> int (*reset)(struct vdpa_device *vdev);
> + int (*suspend)(struct vdpa_device *vdev);
> size_t (*get_config_size)(struct vdpa_device *vdev);
> void (*get_config)(struct vdpa_device *vdev, unsigned int offset,
> void *buf, unsigned int len);
> --
> 2.31.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device
2022-08-10 17:15 ` [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device Eugenio Pérez
@ 2022-08-11 8:29 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 8:29 UTC (permalink / raw)
To: Eugenio Pérez
Cc: kvm, linux-kernel, Jason Wang, virtualization, netdev, dinang,
martinpo, Wu Zongyong, Piotr.Uminski, gautam.dawar, ecree.xilinx,
martinh, Stefano Garzarella, pabloc, habetsm.xilinx, lvivier,
Zhu Lingshan, tanuj.kamde, Longpeng, lulu, hanand, Parav Pandit,
Si-Wei Liu, Eli Cohen, Xie Yongji, Zhang Min, Dan Carpenter,
Christophe JAILLET
On Wed, Aug 10, 2022 at 07:15:11PM +0200, Eugenio Pérez wrote:
> The ioctl adds support for suspending the device from userspace.
>
> This is a must before getting virtqueue indexes (base) for live migration,
> since the device could modify them after userland gets them. There are
> individual ways to perform that action for some devices
> (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> way to perform it for any vhost device (and, in particular, vhost-vdpa).
>
> After a successful return of the ioctl call the device must not process
> more virtqueue descriptors. The device can answer to read or writes of
> config fields as if it were not suspended. In particular, writing to
> "queue_enable" with a value of 1 will not make the device start
> processing buffers of the virtqueue.
>
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> Message-Id: <20220623160738.632852-4-eperezma@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
You are not supposed to include upstream maintainer's signoff
like this.
> ---
> v7: Delete argument to ioctl, unused
> ---
> drivers/vhost/vdpa.c | 19 +++++++++++++++++++
> include/uapi/linux/vhost.h | 9 +++++++++
> 2 files changed, 28 insertions(+)
>
> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> index 3d636e192061..7fa671ac4bdf 100644
> --- a/drivers/vhost/vdpa.c
> +++ b/drivers/vhost/vdpa.c
> @@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vdpa *v, u32 __user *argp)
> return 0;
> }
>
> +/* After a successful return of ioctl the device must not process more
> + * virtqueue descriptors. The device can answer to read or writes of config
> + * fields as if it were not suspended. In particular, writing to "queue_enable"
> + * with a value of 1 will not make the device start processing buffers.
> + */
> +static long vhost_vdpa_suspend(struct vhost_vdpa *v)
> +{
> + struct vdpa_device *vdpa = v->vdpa;
> + const struct vdpa_config_ops *ops = vdpa->config;
> +
> + if (!ops->suspend)
> + return -EOPNOTSUPP;
> +
> + return ops->suspend(vdpa);
> +}
> +
> static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
> void __user *argp)
> {
> @@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
> case VHOST_VDPA_GET_VQS_COUNT:
> r = vhost_vdpa_get_vqs_count(v, argp);
> break;
> + case VHOST_VDPA_SUSPEND:
> + r = vhost_vdpa_suspend(v);
> + break;
> default:
> r = vhost_dev_ioctl(&v->vdev, cmd, argp);
> if (r == -ENOIOCTLCMD)
> diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
> index cab645d4a645..f9f115a7c75b 100644
> --- a/include/uapi/linux/vhost.h
> +++ b/include/uapi/linux/vhost.h
> @@ -171,4 +171,13 @@
> #define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \
> struct vhost_vring_state)
>
> +/* Suspend a device so it does not process virtqueue requests anymore
> + *
> + * After the return of ioctl the device must preserve all the necessary state
> + * (the virtqueue vring base plus the possible device specific states) that is
> + * required for restoring in the future. The device must not change its
> + * configuration after that point.
> + */
> +#define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D)
> +
> #endif
> --
> 2.31.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device
@ 2022-08-11 8:29 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 8:29 UTC (permalink / raw)
To: Eugenio Pérez
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Piotr.Uminski, martinh, Xie Yongji, dinang,
habetsm.xilinx, Longpeng, Dan Carpenter, lvivier, netdev,
linux-kernel, ecree.xilinx, hanand, martinpo, gautam.dawar,
Christophe JAILLET, Zhu Lingshan
On Wed, Aug 10, 2022 at 07:15:11PM +0200, Eugenio Pérez wrote:
> The ioctl adds support for suspending the device from userspace.
>
> This is a must before getting virtqueue indexes (base) for live migration,
> since the device could modify them after userland gets them. There are
> individual ways to perform that action for some devices
> (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> way to perform it for any vhost device (and, in particular, vhost-vdpa).
>
> After a successful return of the ioctl call the device must not process
> more virtqueue descriptors. The device can answer to read or writes of
> config fields as if it were not suspended. In particular, writing to
> "queue_enable" with a value of 1 will not make the device start
> processing buffers of the virtqueue.
>
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> Message-Id: <20220623160738.632852-4-eperezma@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
You are not supposed to include upstream maintainer's signoff
like this.
> ---
> v7: Delete argument to ioctl, unused
> ---
> drivers/vhost/vdpa.c | 19 +++++++++++++++++++
> include/uapi/linux/vhost.h | 9 +++++++++
> 2 files changed, 28 insertions(+)
>
> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> index 3d636e192061..7fa671ac4bdf 100644
> --- a/drivers/vhost/vdpa.c
> +++ b/drivers/vhost/vdpa.c
> @@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vdpa *v, u32 __user *argp)
> return 0;
> }
>
> +/* After a successful return of ioctl the device must not process more
> + * virtqueue descriptors. The device can answer to read or writes of config
> + * fields as if it were not suspended. In particular, writing to "queue_enable"
> + * with a value of 1 will not make the device start processing buffers.
> + */
> +static long vhost_vdpa_suspend(struct vhost_vdpa *v)
> +{
> + struct vdpa_device *vdpa = v->vdpa;
> + const struct vdpa_config_ops *ops = vdpa->config;
> +
> + if (!ops->suspend)
> + return -EOPNOTSUPP;
> +
> + return ops->suspend(vdpa);
> +}
> +
> static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
> void __user *argp)
> {
> @@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
> case VHOST_VDPA_GET_VQS_COUNT:
> r = vhost_vdpa_get_vqs_count(v, argp);
> break;
> + case VHOST_VDPA_SUSPEND:
> + r = vhost_vdpa_suspend(v);
> + break;
> default:
> r = vhost_dev_ioctl(&v->vdev, cmd, argp);
> if (r == -ENOIOCTLCMD)
> diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
> index cab645d4a645..f9f115a7c75b 100644
> --- a/include/uapi/linux/vhost.h
> +++ b/include/uapi/linux/vhost.h
> @@ -171,4 +171,13 @@
> #define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \
> struct vhost_vring_state)
>
> +/* Suspend a device so it does not process virtqueue requests anymore
> + *
> + * After the return of ioctl the device must preserve all the necessary state
> + * (the virtqueue vring base plus the possible device specific states) that is
> + * required for restoring in the future. The device must not change its
> + * configuration after that point.
> + */
> +#define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D)
> +
> #endif
> --
> 2.31.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device
2022-08-11 8:29 ` Michael S. Tsirkin
(?)
@ 2022-08-11 9:58 ` Eugenio Perez Martin
-1 siblings, 0 replies; 19+ messages in thread
From: Eugenio Perez Martin @ 2022-08-11 9:58 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: kvm list, linux-kernel, Jason Wang, virtualization, netdev,
Dinan Gunawardena, Martin Porter, Wu Zongyong, Uminski, Piotr,
Dawar, Gautam, ecree.xilinx, Martin Petrus Hubertus Habets,
Stefano Garzarella, Pablo Cascon Katchadourian, habetsm.xilinx,
Laurent Vivier, Zhu Lingshan, Kamde, Tanuj, Longpeng, Cindy Lu,
Harpreet Singh Anand, Parav Pandit, Si-Wei Liu, Eli Cohen,
Xie Yongji, Zhang Min, Dan Carpenter, Christophe JAILLET
On Thu, Aug 11, 2022 at 10:29 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Aug 10, 2022 at 07:15:11PM +0200, Eugenio Pérez wrote:
> > The ioctl adds support for suspending the device from userspace.
> >
> > This is a must before getting virtqueue indexes (base) for live migration,
> > since the device could modify them after userland gets them. There are
> > individual ways to perform that action for some devices
> > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no
> > way to perform it for any vhost device (and, in particular, vhost-vdpa).
> >
> > After a successful return of the ioctl call the device must not process
> > more virtqueue descriptors. The device can answer to read or writes of
> > config fields as if it were not suspended. In particular, writing to
> > "queue_enable" with a value of 1 will not make the device start
> > processing buffers of the virtqueue.
> >
> > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > Message-Id: <20220623160738.632852-4-eperezma@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> You are not supposed to include upstream maintainer's signoff
> like this.
>
I'm very sorry, I modified the commits in your vhost branch and I left
the signoff (and message-id) lines by mistake.
> > ---
> > v7: Delete argument to ioctl, unused
> > ---
> > drivers/vhost/vdpa.c | 19 +++++++++++++++++++
> > include/uapi/linux/vhost.h | 9 +++++++++
> > 2 files changed, 28 insertions(+)
> >
> > diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> > index 3d636e192061..7fa671ac4bdf 100644
> > --- a/drivers/vhost/vdpa.c
> > +++ b/drivers/vhost/vdpa.c
> > @@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vdpa *v, u32 __user *argp)
> > return 0;
> > }
> >
> > +/* After a successful return of ioctl the device must not process more
> > + * virtqueue descriptors. The device can answer to read or writes of config
> > + * fields as if it were not suspended. In particular, writing to "queue_enable"
> > + * with a value of 1 will not make the device start processing buffers.
> > + */
> > +static long vhost_vdpa_suspend(struct vhost_vdpa *v)
> > +{
> > + struct vdpa_device *vdpa = v->vdpa;
> > + const struct vdpa_config_ops *ops = vdpa->config;
> > +
> > + if (!ops->suspend)
> > + return -EOPNOTSUPP;
> > +
> > + return ops->suspend(vdpa);
> > +}
> > +
> > static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
> > void __user *argp)
> > {
> > @@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
> > case VHOST_VDPA_GET_VQS_COUNT:
> > r = vhost_vdpa_get_vqs_count(v, argp);
> > break;
> > + case VHOST_VDPA_SUSPEND:
> > + r = vhost_vdpa_suspend(v);
> > + break;
> > default:
> > r = vhost_dev_ioctl(&v->vdev, cmd, argp);
> > if (r == -ENOIOCTLCMD)
> > diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
> > index cab645d4a645..f9f115a7c75b 100644
> > --- a/include/uapi/linux/vhost.h
> > +++ b/include/uapi/linux/vhost.h
> > @@ -171,4 +171,13 @@
> > #define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \
> > struct vhost_vring_state)
> >
> > +/* Suspend a device so it does not process virtqueue requests anymore
> > + *
> > + * After the return of ioctl the device must preserve all the necessary state
> > + * (the virtqueue vring base plus the possible device specific states) that is
> > + * required for restoring in the future. The device must not change its
> > + * configuration after that point.
> > + */
> > +#define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D)
> > +
> > #endif
> > --
> > 2.31.1
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
2022-08-11 8:27 ` Michael S. Tsirkin
@ 2022-08-11 10:15 ` Dan Carpenter
-1 siblings, 0 replies; 19+ messages in thread
From: Dan Carpenter @ 2022-08-11 10:15 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eugenio Pérez, kvm, linux-kernel, Jason Wang, virtualization,
netdev, dinang, martinpo, Wu Zongyong, Piotr.Uminski,
gautam.dawar, ecree.xilinx, martinh, Stefano Garzarella, pabloc,
habetsm.xilinx, lvivier, Zhu Lingshan, tanuj.kamde, Longpeng,
lulu, hanand, Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji,
Zhang Min, Christophe JAILLET
On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > This operation is optional: It it's not implemented, backend feature bit
> > will not be exposed.
> >
> > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> What is this message id doing here?
>
I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
the thread.
Linus has complained (rough remembering) that everyone is using the
Link: tag for links to the patch itself. It's supposed to be for Links
to bugzilla or to the spec or whatever. Extra information, too much to
put in the commit message. Now the Link tag is useless because it either
points to the patch or to a bugzilla. Depend on what you want it to do,
it *always* points to the opposite thing.
But I can't remember what people settled on as the alternative to use
to link to lore...
In theory, we should be able to figure out the link to lore automatically
and there have been a couple projects which tried to do this but they
can't make it work 100%. Maintainers massage and reformat the patches
too much before applying them.
regards,
dan carpenter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
@ 2022-08-11 10:15 ` Dan Carpenter
0 siblings, 0 replies; 19+ messages in thread
From: Dan Carpenter @ 2022-08-11 10:15 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Xie Yongji, Piotr.Uminski, martinh,
Eugenio Pérez, dinang, habetsm.xilinx, Longpeng, lvivier,
netdev, linux-kernel, ecree.xilinx, hanand, martinpo,
gautam.dawar, Christophe JAILLET, Zhu Lingshan
On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > This operation is optional: It it's not implemented, backend feature bit
> > will not be exposed.
> >
> > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> What is this message id doing here?
>
I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
the thread.
Linus has complained (rough remembering) that everyone is using the
Link: tag for links to the patch itself. It's supposed to be for Links
to bugzilla or to the spec or whatever. Extra information, too much to
put in the commit message. Now the Link tag is useless because it either
points to the patch or to a bugzilla. Depend on what you want it to do,
it *always* points to the opposite thing.
But I can't remember what people settled on as the alternative to use
to link to lore...
In theory, we should be able to figure out the link to lore automatically
and there have been a couple projects which tried to do this but they
can't make it work 100%. Maintainers massage and reformat the patches
too much before applying them.
regards,
dan carpenter
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
2022-08-11 10:15 ` Dan Carpenter
@ 2022-08-11 11:23 ` Michael S. Tsirkin
-1 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 11:23 UTC (permalink / raw)
To: Dan Carpenter
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Xie Yongji, Piotr.Uminski, martinh,
Eugenio Pérez, dinang, habetsm.xilinx, Longpeng, lvivier,
netdev, linux-kernel, ecree.xilinx, hanand, martinpo,
gautam.dawar, Christophe JAILLET, Zhu Lingshan
On Thu, Aug 11, 2022 at 01:15:08PM +0300, Dan Carpenter wrote:
> On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> > On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > > This operation is optional: It it's not implemented, backend feature bit
> > > will not be exposed.
> > >
> > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >
> > What is this message id doing here?
> >
>
> I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
> the thread.
Yes it makes sense in git. But I don't see what it does in this patch
posted to the list. It seems to refer to the previous version of the
patch here. Which is ok I guess but better called out e.g.
Previous-version: <20220623160738.632852-2-eperezma@redhat.com>
> Linus has complained (rough remembering) that everyone is using the
> Link: tag for links to the patch itself. It's supposed to be for Links
> to bugzilla or to the spec or whatever. Extra information, too much to
> put in the commit message. Now the Link tag is useless because it either
> points to the patch or to a bugzilla. Depend on what you want it to do,
> it *always* points to the opposite thing.
>
> But I can't remember what people settled on as the alternative to use
> to link to lore...
>
> In theory, we should be able to figure out the link to lore automatically
> and there have been a couple projects which tried to do this but they
> can't make it work 100%. Maintainers massage and reformat the patches
> too much before applying them.
>
> regards,
> dan carpenter
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
@ 2022-08-11 11:23 ` Michael S. Tsirkin
0 siblings, 0 replies; 19+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 11:23 UTC (permalink / raw)
To: Dan Carpenter
Cc: Eugenio Pérez, kvm, linux-kernel, Jason Wang, virtualization,
netdev, dinang, martinpo, Wu Zongyong, Piotr.Uminski,
gautam.dawar, ecree.xilinx, martinh, Stefano Garzarella, pabloc,
habetsm.xilinx, lvivier, Zhu Lingshan, tanuj.kamde, Longpeng,
lulu, hanand, Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji,
Zhang Min, Christophe JAILLET
On Thu, Aug 11, 2022 at 01:15:08PM +0300, Dan Carpenter wrote:
> On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> > On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > > This operation is optional: It it's not implemented, backend feature bit
> > > will not be exposed.
> > >
> > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >
> > What is this message id doing here?
> >
>
> I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
> the thread.
Yes it makes sense in git. But I don't see what it does in this patch
posted to the list. It seems to refer to the previous version of the
patch here. Which is ok I guess but better called out e.g.
Previous-version: <20220623160738.632852-2-eperezma@redhat.com>
> Linus has complained (rough remembering) that everyone is using the
> Link: tag for links to the patch itself. It's supposed to be for Links
> to bugzilla or to the spec or whatever. Extra information, too much to
> put in the commit message. Now the Link tag is useless because it either
> points to the patch or to a bugzilla. Depend on what you want it to do,
> it *always* points to the opposite thing.
>
> But I can't remember what people settled on as the alternative to use
> to link to lore...
>
> In theory, we should be able to figure out the link to lore automatically
> and there have been a couple projects which tried to do this but they
> can't make it work 100%. Maintainers massage and reformat the patches
> too much before applying them.
>
> regards,
> dan carpenter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
2022-08-11 11:23 ` Michael S. Tsirkin
@ 2022-08-11 12:40 ` Dan Carpenter
-1 siblings, 0 replies; 19+ messages in thread
From: Dan Carpenter @ 2022-08-11 12:40 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: tanuj.kamde, kvm, virtualization, Wu Zongyong, pabloc, Eli Cohen,
Zhang Min, lulu, Xie Yongji, Piotr.Uminski, martinh,
Eugenio Pérez, dinang, habetsm.xilinx, Longpeng, lvivier,
netdev, linux-kernel, ecree.xilinx, hanand, martinpo,
gautam.dawar, Christophe JAILLET, Zhu Lingshan
On Thu, Aug 11, 2022 at 07:23:44AM -0400, Michael S. Tsirkin wrote:
> On Thu, Aug 11, 2022 at 01:15:08PM +0300, Dan Carpenter wrote:
> > On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> > > On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > > > This operation is optional: It it's not implemented, backend feature bit
> > > > will not be exposed.
> > > >
> > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > > > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >
> > > What is this message id doing here?
> > >
> >
> > I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
> > the thread.
>
> Yes it makes sense in git. But I don't see what it does in this patch
> posted to the list.
Ah. Okay. I misunderstood.
> It seems to refer to the previous version of the
> patch here. Which is ok I guess but better called out e.g.
>
> Previous-version: <20220623160738.632852-2-eperezma@redhat.com>
>
Let's not go over board with storing previous threads. That seems like
a headache...
regards,
dan carpenter
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v7 1/4] vdpa: Add suspend operation
@ 2022-08-11 12:40 ` Dan Carpenter
0 siblings, 0 replies; 19+ messages in thread
From: Dan Carpenter @ 2022-08-11 12:40 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eugenio Pérez, kvm, linux-kernel, Jason Wang, virtualization,
netdev, dinang, martinpo, Wu Zongyong, Piotr.Uminski,
gautam.dawar, ecree.xilinx, martinh, Stefano Garzarella, pabloc,
habetsm.xilinx, lvivier, Zhu Lingshan, tanuj.kamde, Longpeng,
lulu, hanand, Parav Pandit, Si-Wei Liu, Eli Cohen, Xie Yongji,
Zhang Min, Christophe JAILLET
On Thu, Aug 11, 2022 at 07:23:44AM -0400, Michael S. Tsirkin wrote:
> On Thu, Aug 11, 2022 at 01:15:08PM +0300, Dan Carpenter wrote:
> > On Thu, Aug 11, 2022 at 04:27:32AM -0400, Michael S. Tsirkin wrote:
> > > On Wed, Aug 10, 2022 at 07:15:09PM +0200, Eugenio Pérez wrote:
> > > > This operation is optional: It it's not implemented, backend feature bit
> > > > will not be exposed.
> > > >
> > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > > > Message-Id: <20220623160738.632852-2-eperezma@redhat.com>
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >
> > > What is this message id doing here?
> > >
> >
> > I like the Message-Id tag. It means you can `b4 mbox <mesg-id>` and get
> > the thread.
>
> Yes it makes sense in git. But I don't see what it does in this patch
> posted to the list.
Ah. Okay. I misunderstood.
> It seems to refer to the previous version of the
> patch here. Which is ok I guess but better called out e.g.
>
> Previous-version: <20220623160738.632852-2-eperezma@redhat.com>
>
Let's not go over board with storing previous threads. That seems like
a headache...
regards,
dan carpenter
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2022-08-11 12:41 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-10 17:15 [PATCH v7 0/4] Implement vdpasim suspend operation Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 1/4] vdpa: Add " Eugenio Pérez
2022-08-11 8:27 ` Michael S. Tsirkin
2022-08-11 8:27 ` Michael S. Tsirkin
2022-08-11 10:15 ` Dan Carpenter
2022-08-11 10:15 ` Dan Carpenter
2022-08-11 11:23 ` Michael S. Tsirkin
2022-08-11 11:23 ` Michael S. Tsirkin
2022-08-11 12:40 ` Dan Carpenter
2022-08-11 12:40 ` Dan Carpenter
2022-08-10 17:15 ` [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit Eugenio Pérez
2022-08-10 17:15 ` [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device Eugenio Pérez
2022-08-11 8:29 ` Michael S. Tsirkin
2022-08-11 8:29 ` Michael S. Tsirkin
2022-08-11 9:58 ` Eugenio Perez Martin
2022-08-10 17:15 ` [PATCH v7 4/4] vdpa_sim: Implement suspend vdpa op Eugenio Pérez
2022-08-10 19:19 ` [PATCH v7 0/4] Implement vdpasim suspend operation Michael S. Tsirkin
2022-08-10 19:19 ` Michael S. Tsirkin
2022-08-11 5:35 ` Eugenio Perez Martin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.