* [PATCH] virtio_pmem: Add freeze/restore callbacks
@ 2024-08-15 0:46 Philip Chen
2024-08-24 2:06 ` Philip Chen
0 siblings, 1 reply; 6+ messages in thread
From: Philip Chen @ 2024-08-15 0:46 UTC (permalink / raw)
To: Pankaj Gupta, Dan Williams, Vishal Verma, Dave Jiang, Ira Weiny
Cc: virtualization, nvdimm, linux-kernel, Philip Chen
Add basic freeze/restore PM callbacks to support hibernation (S4):
- On freeze, delete vq and quiesce the device to prepare for
snapshotting.
- On restore, re-init vq and mark DRIVER_OK.
Signed-off-by: Philip Chen <philipchen@chromium.org>
---
drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
index c9b97aeabf85..2396d19ce549 100644
--- a/drivers/nvdimm/virtio_pmem.c
+++ b/drivers/nvdimm/virtio_pmem.c
@@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
virtio_reset_device(vdev);
}
+static int virtio_pmem_freeze(struct virtio_device *vdev)
+{
+ vdev->config->del_vqs(vdev);
+ virtio_reset_device(vdev);
+
+ return 0;
+}
+
+static int virtio_pmem_restore(struct virtio_device *vdev)
+{
+ int ret;
+
+ ret = init_vq(vdev->priv);
+ if (ret) {
+ dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
+ return ret;
+ }
+ virtio_device_ready(vdev);
+
+ return 0;
+}
+
static unsigned int features[] = {
VIRTIO_PMEM_F_SHMEM_REGION,
};
@@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
.validate = virtio_pmem_validate,
.probe = virtio_pmem_probe,
.remove = virtio_pmem_remove,
+ .freeze = virtio_pmem_freeze,
+ .restore = virtio_pmem_restore,
};
module_virtio_driver(virtio_pmem_driver);
--
2.46.0.76.ge559c4bf1a-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_pmem: Add freeze/restore callbacks
2024-08-15 0:46 [PATCH] virtio_pmem: Add freeze/restore callbacks Philip Chen
@ 2024-08-24 2:06 ` Philip Chen
2024-09-04 17:53 ` Ira Weiny
0 siblings, 1 reply; 6+ messages in thread
From: Philip Chen @ 2024-08-24 2:06 UTC (permalink / raw)
To: Pankaj Gupta, Dan Williams, Vishal Verma, Dave Jiang, Ira Weiny
Cc: virtualization, nvdimm, linux-kernel
Hi maintainers,
Can anyone let me know if this patch makes sense?
Any comment/feedback is appreciated.
Thanks in advance!
On Wed, Aug 14, 2024 at 5:46 PM Philip Chen <philipchen@chromium.org> wrote:
>
> Add basic freeze/restore PM callbacks to support hibernation (S4):
> - On freeze, delete vq and quiesce the device to prepare for
> snapshotting.
> - On restore, re-init vq and mark DRIVER_OK.
>
> Signed-off-by: Philip Chen <philipchen@chromium.org>
> ---
> drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
> diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
> index c9b97aeabf85..2396d19ce549 100644
> --- a/drivers/nvdimm/virtio_pmem.c
> +++ b/drivers/nvdimm/virtio_pmem.c
> @@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
> virtio_reset_device(vdev);
> }
>
> +static int virtio_pmem_freeze(struct virtio_device *vdev)
> +{
> + vdev->config->del_vqs(vdev);
> + virtio_reset_device(vdev);
> +
> + return 0;
> +}
> +
> +static int virtio_pmem_restore(struct virtio_device *vdev)
> +{
> + int ret;
> +
> + ret = init_vq(vdev->priv);
> + if (ret) {
> + dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
> + return ret;
> + }
> + virtio_device_ready(vdev);
> +
> + return 0;
> +}
> +
> static unsigned int features[] = {
> VIRTIO_PMEM_F_SHMEM_REGION,
> };
> @@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
> .validate = virtio_pmem_validate,
> .probe = virtio_pmem_probe,
> .remove = virtio_pmem_remove,
> + .freeze = virtio_pmem_freeze,
> + .restore = virtio_pmem_restore,
> };
>
> module_virtio_driver(virtio_pmem_driver);
> --
> 2.46.0.76.ge559c4bf1a-goog
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_pmem: Add freeze/restore callbacks
2024-08-24 2:06 ` Philip Chen
@ 2024-09-04 17:53 ` Ira Weiny
2024-09-09 23:52 ` Philip Chen
0 siblings, 1 reply; 6+ messages in thread
From: Ira Weiny @ 2024-09-04 17:53 UTC (permalink / raw)
To: Philip Chen, Pankaj Gupta, Dan Williams, Vishal Verma, Dave Jiang,
Ira Weiny
Cc: virtualization, nvdimm, linux-kernel
Philip Chen wrote:
> Hi maintainers,
>
> Can anyone let me know if this patch makes sense?
> Any comment/feedback is appreciated.
> Thanks in advance!
I'm not an expert on virtio but the code looks ok on the surface. I've
discussed this with Dan a bit and virtio-pmem is not heavily tested.
Based on our discussion [1] I wonder if there is a way we can recreate this
with QEMU to incorporate into our testing?
Ira
[1] https://lore.kernel.org/lkml/CA+cxXhnb2i5O7_BiOfKLth5Zwp5T62d6F6c39vnuT53cUkU_uw@mail.gmail.com/
>
> On Wed, Aug 14, 2024 at 5:46 PM Philip Chen <philipchen@chromium.org> wrote:
> >
> > Add basic freeze/restore PM callbacks to support hibernation (S4):
> > - On freeze, delete vq and quiesce the device to prepare for
> > snapshotting.
> > - On restore, re-init vq and mark DRIVER_OK.
> >
> > Signed-off-by: Philip Chen <philipchen@chromium.org>
> > ---
> > drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
> > 1 file changed, 24 insertions(+)
> >
> > diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
> > index c9b97aeabf85..2396d19ce549 100644
> > --- a/drivers/nvdimm/virtio_pmem.c
> > +++ b/drivers/nvdimm/virtio_pmem.c
> > @@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
> > virtio_reset_device(vdev);
> > }
> >
> > +static int virtio_pmem_freeze(struct virtio_device *vdev)
> > +{
> > + vdev->config->del_vqs(vdev);
> > + virtio_reset_device(vdev);
> > +
> > + return 0;
> > +}
> > +
> > +static int virtio_pmem_restore(struct virtio_device *vdev)
> > +{
> > + int ret;
> > +
> > + ret = init_vq(vdev->priv);
> > + if (ret) {
> > + dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
> > + return ret;
> > + }
> > + virtio_device_ready(vdev);
> > +
> > + return 0;
> > +}
> > +
> > static unsigned int features[] = {
> > VIRTIO_PMEM_F_SHMEM_REGION,
> > };
> > @@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
> > .validate = virtio_pmem_validate,
> > .probe = virtio_pmem_probe,
> > .remove = virtio_pmem_remove,
> > + .freeze = virtio_pmem_freeze,
> > + .restore = virtio_pmem_restore,
> > };
> >
> > module_virtio_driver(virtio_pmem_driver);
> > --
> > 2.46.0.76.ge559c4bf1a-goog
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_pmem: Add freeze/restore callbacks
2024-09-04 17:53 ` Ira Weiny
@ 2024-09-09 23:52 ` Philip Chen
2024-09-23 16:47 ` Philip Chen
2024-09-25 7:30 ` Pankaj Gupta
0 siblings, 2 replies; 6+ messages in thread
From: Philip Chen @ 2024-09-09 23:52 UTC (permalink / raw)
To: Ira Weiny
Cc: Pankaj Gupta, Dan Williams, Vishal Verma, Dave Jiang,
virtualization, nvdimm, linux-kernel
Hi
On Wed, Sep 4, 2024 at 10:54 AM Ira Weiny <ira.weiny@intel.com> wrote:
>
> Philip Chen wrote:
> > Hi maintainers,
> >
> > Can anyone let me know if this patch makes sense?
> > Any comment/feedback is appreciated.
> > Thanks in advance!
>
> I'm not an expert on virtio but the code looks ok on the surface. I've
> discussed this with Dan a bit and virtio-pmem is not heavily tested.
Thanks for your comments.
I think this specific patch is not heavily involved with virtio spec details.
This patch simply provides the basic freeze/restore PM callbacks for
virtio_pmem, like people already did for the other virtio drivers.
>
> Based on our discussion [1] I wonder if there is a way we can recreate this
> with QEMU to incorporate into our testing?
Yes, these are how I test on crosvm, but I believe the same steps can
be applied to QEMU:
(1) Set pm_test_level to TEST_PLATFORM (build time change)
(2) Write something to pmem
(3) Make the device go through a freeze/restore cycle by writing
`disk` to `/sys/power/state`
(4) Validate the data written to pmem in (2) is still preserved
Note:
(a) The freeze/restore PM routines are sometimes called as the backup
for suspend/resume PM routines in a suspend/resume cycle.
In this case, we can also test freeze/restore PM routines with
suspend/resume: i.e. skip (1) and write `mem` to `sys/power/state` in
(3).
(b) I also tried to set up QEMU for testing. But QEMU crashes when I
try to freeze the device even without applying this patch.
Since the issue seems to be irrelevant to pmem and most likely a QEMU
setup problem on my end, I didn't spend more time enabling QEMU.
>
> Ira
>
> [1] https://lore.kernel.org/lkml/CA+cxXhnb2i5O7_BiOfKLth5Zwp5T62d6F6c39vnuT53cUkU_uw@mail.gmail.com/
>
> >
> > On Wed, Aug 14, 2024 at 5:46 PM Philip Chen <philipchen@chromium.org> wrote:
> > >
> > > Add basic freeze/restore PM callbacks to support hibernation (S4):
> > > - On freeze, delete vq and quiesce the device to prepare for
> > > snapshotting.
> > > - On restore, re-init vq and mark DRIVER_OK.
> > >
> > > Signed-off-by: Philip Chen <philipchen@chromium.org>
> > > ---
> > > drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
> > > 1 file changed, 24 insertions(+)
> > >
> > > diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
> > > index c9b97aeabf85..2396d19ce549 100644
> > > --- a/drivers/nvdimm/virtio_pmem.c
> > > +++ b/drivers/nvdimm/virtio_pmem.c
> > > @@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
> > > virtio_reset_device(vdev);
> > > }
> > >
> > > +static int virtio_pmem_freeze(struct virtio_device *vdev)
> > > +{
> > > + vdev->config->del_vqs(vdev);
> > > + virtio_reset_device(vdev);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int virtio_pmem_restore(struct virtio_device *vdev)
> > > +{
> > > + int ret;
> > > +
> > > + ret = init_vq(vdev->priv);
> > > + if (ret) {
> > > + dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
> > > + return ret;
> > > + }
> > > + virtio_device_ready(vdev);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > static unsigned int features[] = {
> > > VIRTIO_PMEM_F_SHMEM_REGION,
> > > };
> > > @@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
> > > .validate = virtio_pmem_validate,
> > > .probe = virtio_pmem_probe,
> > > .remove = virtio_pmem_remove,
> > > + .freeze = virtio_pmem_freeze,
> > > + .restore = virtio_pmem_restore,
> > > };
> > >
> > > module_virtio_driver(virtio_pmem_driver);
> > > --
> > > 2.46.0.76.ge559c4bf1a-goog
> > >
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_pmem: Add freeze/restore callbacks
2024-09-09 23:52 ` Philip Chen
@ 2024-09-23 16:47 ` Philip Chen
2024-09-25 7:30 ` Pankaj Gupta
1 sibling, 0 replies; 6+ messages in thread
From: Philip Chen @ 2024-09-23 16:47 UTC (permalink / raw)
To: Ira Weiny
Cc: Pankaj Gupta, Dan Williams, Vishal Verma, Dave Jiang,
virtualization, nvdimm, linux-kernel
Hi maintainers,
Are there any other concerns I should address for this patch?
On Mon, Sep 9, 2024 at 4:52 PM Philip Chen <philipchen@chromium.org> wrote:
>
> Hi
>
> On Wed, Sep 4, 2024 at 10:54 AM Ira Weiny <ira.weiny@intel.com> wrote:
> >
> > Philip Chen wrote:
> > > Hi maintainers,
> > >
> > > Can anyone let me know if this patch makes sense?
> > > Any comment/feedback is appreciated.
> > > Thanks in advance!
> >
> > I'm not an expert on virtio but the code looks ok on the surface. I've
> > discussed this with Dan a bit and virtio-pmem is not heavily tested.
>
> Thanks for your comments.
> I think this specific patch is not heavily involved with virtio spec details.
> This patch simply provides the basic freeze/restore PM callbacks for
> virtio_pmem, like people already did for the other virtio drivers.
>
> >
> > Based on our discussion [1] I wonder if there is a way we can recreate this
> > with QEMU to incorporate into our testing?
>
> Yes, these are how I test on crosvm, but I believe the same steps can
> be applied to QEMU:
> (1) Set pm_test_level to TEST_PLATFORM (build time change)
> (2) Write something to pmem
> (3) Make the device go through a freeze/restore cycle by writing
> `disk` to `/sys/power/state`
> (4) Validate the data written to pmem in (2) is still preserved
>
> Note:
> (a) The freeze/restore PM routines are sometimes called as the backup
> for suspend/resume PM routines in a suspend/resume cycle.
> In this case, we can also test freeze/restore PM routines with
> suspend/resume: i.e. skip (1) and write `mem` to `sys/power/state` in
> (3).
> (b) I also tried to set up QEMU for testing. But QEMU crashes when I
> try to freeze the device even without applying this patch.
> Since the issue seems to be irrelevant to pmem and most likely a QEMU
> setup problem on my end, I didn't spend more time enabling QEMU.
>
>
>
> >
> > Ira
> >
> > [1] https://lore.kernel.org/lkml/CA+cxXhnb2i5O7_BiOfKLth5Zwp5T62d6F6c39vnuT53cUkU_uw@mail.gmail.com/
> >
> > >
> > > On Wed, Aug 14, 2024 at 5:46 PM Philip Chen <philipchen@chromium.org> wrote:
> > > >
> > > > Add basic freeze/restore PM callbacks to support hibernation (S4):
> > > > - On freeze, delete vq and quiesce the device to prepare for
> > > > snapshotting.
> > > > - On restore, re-init vq and mark DRIVER_OK.
> > > >
> > > > Signed-off-by: Philip Chen <philipchen@chromium.org>
> > > > ---
> > > > drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
> > > > 1 file changed, 24 insertions(+)
> > > >
> > > > diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
> > > > index c9b97aeabf85..2396d19ce549 100644
> > > > --- a/drivers/nvdimm/virtio_pmem.c
> > > > +++ b/drivers/nvdimm/virtio_pmem.c
> > > > @@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
> > > > virtio_reset_device(vdev);
> > > > }
> > > >
> > > > +static int virtio_pmem_freeze(struct virtio_device *vdev)
> > > > +{
> > > > + vdev->config->del_vqs(vdev);
> > > > + virtio_reset_device(vdev);
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static int virtio_pmem_restore(struct virtio_device *vdev)
> > > > +{
> > > > + int ret;
> > > > +
> > > > + ret = init_vq(vdev->priv);
> > > > + if (ret) {
> > > > + dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
> > > > + return ret;
> > > > + }
> > > > + virtio_device_ready(vdev);
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > static unsigned int features[] = {
> > > > VIRTIO_PMEM_F_SHMEM_REGION,
> > > > };
> > > > @@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
> > > > .validate = virtio_pmem_validate,
> > > > .probe = virtio_pmem_probe,
> > > > .remove = virtio_pmem_remove,
> > > > + .freeze = virtio_pmem_freeze,
> > > > + .restore = virtio_pmem_restore,
> > > > };
> > > >
> > > > module_virtio_driver(virtio_pmem_driver);
> > > > --
> > > > 2.46.0.76.ge559c4bf1a-goog
> > > >
> >
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] virtio_pmem: Add freeze/restore callbacks
2024-09-09 23:52 ` Philip Chen
2024-09-23 16:47 ` Philip Chen
@ 2024-09-25 7:30 ` Pankaj Gupta
1 sibling, 0 replies; 6+ messages in thread
From: Pankaj Gupta @ 2024-09-25 7:30 UTC (permalink / raw)
To: Philip Chen
Cc: Ira Weiny, Dan Williams, Vishal Verma, Dave Jiang, virtualization,
nvdimm, linux-kernel, Michael S . Tsirkin
+CC MST
> > Philip Chen wrote:
> > > Hi maintainers,
> > >
> > > Can anyone let me know if this patch makes sense?
> > > Any comment/feedback is appreciated.
> > > Thanks in advance!
> >
> > I'm not an expert on virtio but the code looks ok on the surface. I've
> > discussed this with Dan a bit and virtio-pmem is not heavily tested.
>
> Thanks for your comments.
> I think this specific patch is not heavily involved with virtio spec details.
> This patch simply provides the basic freeze/restore PM callbacks for
> virtio_pmem, like people already did for the other virtio drivers.
>
> >
> > Based on our discussion [1] I wonder if there is a way we can recreate this
> > with QEMU to incorporate into our testing?
>
> Yes, these are how I test on crosvm, but I believe the same steps can
> be applied to QEMU:
> (1) Set pm_test_level to TEST_PLATFORM (build time change)
> (2) Write something to pmem
> (3) Make the device go through a freeze/restore cycle by writing
> `disk` to `/sys/power/state`
> (4) Validate the data written to pmem in (2) is still preserved
>
> Note:
> (a) The freeze/restore PM routines are sometimes called as the backup
> for suspend/resume PM routines in a suspend/resume cycle.
> In this case, we can also test freeze/restore PM routines with
> suspend/resume: i.e. skip (1) and write `mem` to `sys/power/state` in
> (3).
> (b) I also tried to set up QEMU for testing. But QEMU crashes when I
> try to freeze the device even without applying this patch.
> Since the issue seems to be irrelevant to pmem and most likely a QEMU
> setup problem on my end, I didn't spend more time enabling QEMU.
Thanks for the work!
Did you check why QEMU was crashing, maybe because we did not support
freeze/restore before
so some missing NULL check? Any back trace you got?
If it works in crossVM, it should work in Qemu as well?
Thanks,
Pankaj
>
>
>
> >
> > Ira
> >
> > [1] https://lore.kernel.org/lkml/CA+cxXhnb2i5O7_BiOfKLth5Zwp5T62d6F6c39vnuT53cUkU_uw@mail.gmail.com/
> >
> > >
> > > On Wed, Aug 14, 2024 at 5:46 PM Philip Chen <philipchen@chromium.org> wrote:
> > > >
> > > > Add basic freeze/restore PM callbacks to support hibernation (S4):
> > > > - On freeze, delete vq and quiesce the device to prepare for
> > > > snapshotting.
> > > > - On restore, re-init vq and mark DRIVER_OK.
> > > >
> > > > Signed-off-by: Philip Chen <philipchen@chromium.org>
> > > > ---
> > > > drivers/nvdimm/virtio_pmem.c | 24 ++++++++++++++++++++++++
> > > > 1 file changed, 24 insertions(+)
> > > >
> > > > diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
> > > > index c9b97aeabf85..2396d19ce549 100644
> > > > --- a/drivers/nvdimm/virtio_pmem.c
> > > > +++ b/drivers/nvdimm/virtio_pmem.c
> > > > @@ -143,6 +143,28 @@ static void virtio_pmem_remove(struct virtio_device *vdev)
> > > > virtio_reset_device(vdev);
> > > > }
> > > >
> > > > +static int virtio_pmem_freeze(struct virtio_device *vdev)
> > > > +{
> > > > + vdev->config->del_vqs(vdev);
> > > > + virtio_reset_device(vdev);
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static int virtio_pmem_restore(struct virtio_device *vdev)
> > > > +{
> > > > + int ret;
> > > > +
> > > > + ret = init_vq(vdev->priv);
> > > > + if (ret) {
> > > > + dev_err(&vdev->dev, "failed to initialize virtio pmem's vq\n");
> > > > + return ret;
> > > > + }
> > > > + virtio_device_ready(vdev);
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > static unsigned int features[] = {
> > > > VIRTIO_PMEM_F_SHMEM_REGION,
> > > > };
> > > > @@ -155,6 +177,8 @@ static struct virtio_driver virtio_pmem_driver = {
> > > > .validate = virtio_pmem_validate,
> > > > .probe = virtio_pmem_probe,
> > > > .remove = virtio_pmem_remove,
> > > > + .freeze = virtio_pmem_freeze,
> > > > + .restore = virtio_pmem_restore,
> > > > };
> > > >
> > > > module_virtio_driver(virtio_pmem_driver);
> > > > --
> > > > 2.46.0.76.ge559c4bf1a-goog
> > > >
> >
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-09-25 7:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-15 0:46 [PATCH] virtio_pmem: Add freeze/restore callbacks Philip Chen
2024-08-24 2:06 ` Philip Chen
2024-09-04 17:53 ` Ira Weiny
2024-09-09 23:52 ` Philip Chen
2024-09-23 16:47 ` Philip Chen
2024-09-25 7:30 ` Pankaj Gupta
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).