Re: [RFC PATCH 3/3] nvme: add the "debug" host driver

From: Chaitanya Kulkarni <chaitanyak@nvidia.com>
To: "Javier González" <javier@javigon.com>
Cc: Damien Le Moal <damien.lemoal@opensource.wdc.com>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Mikulas Patocka <mpatocka@redhat.com>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	Keith Busch <kbusch@kernel.org>,
	Adam Manzanares <a.manzanares@samsung.com>,
	"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
	"dm-devel@redhat.com" <dm-devel@redhat.com>,
	"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Jens Axboe <axboe@kernel.dk>,
	"msnitzer@redhat.com >> msnitzer@redhat.com"
	<msnitzer@redhat.com>, Bart Van Assche <bvanassche@acm.org>,
	"martin.petersen@oracle.com >> Martin K. Petersen" 
	<martin.petersen@oracle.com>,
	"roland@purestorage.com" <roland@purestorage.com>,
	Hannes Reinecke <hare@suse.de>, Christoph Hellwig <hch@lst.de>,
	"Frederick.Knight@netapp.com" <Frederick.Knight@netapp.com>,
	"zach.brown@ni.com" <zach.brown@ni.com>,
	"osandov@fb.com" <osandov@fb.com>,
	"lsf-pc@lists.linux-foundation.org" 
	<lsf-pc@lists.linux-foundation.org>,
	"djwong@kernel.org" <djwong@kernel.org>,
	"josef@toxicpanda.com" <josef@toxicpanda.com>,
	"clm@fb.com" <clm@fb.com>, "dsterba@suse.com" <dsterba@suse.com>,
	"tytso@mit.edu" <tytso@mit.edu>, "jack@suse.com" <jack@suse.com>,
	Kanchan Joshi <joshi.k@samsung.com>
Subject: Re: [RFC PATCH 3/3] nvme: add the "debug" host driver
Date: Fri, 4 Feb 2022 09:58:03 +0000	[thread overview]
Message-ID: <4d5410a5-93c3-d73c-6aeb-2c1c7f940963@nvidia.com> (raw)
In-Reply-To: <20220204082445.hczdiy2uhxfi3x2g@ArmHalley.local>

On 2/4/22 12:24 AM, Javier González wrote:
> On 04.02.2022 07:58, Chaitanya Kulkarni wrote:
>> On 2/3/22 22:28, Damien Le Moal wrote:
>>> On 2/4/22 12:12, Chaitanya Kulkarni wrote:
>>>>
>>>>>>> One can instantiate scsi devices with qemu by using fake scsi 
>>>>>>> devices,
>>>>>>> but one can also just use scsi_debug to do the same. I see both 
>>>>>>> efforts
>>>>>>> as desirable, so long as someone mantains this.
>>>>>>>
>>>>
>>>> Why do you think both efforts are desirable ?
>>>
>>> When testing code using the functionality, it is far easier to get said
>>> functionality doing a simple "modprobe" rather than having to setup a
>>> VM. C.f. running blktests or fstests.
>>>
>>
>> agree on simplicity but then why do we have QEMU implementations for
>> the NVMe features (e.g. ZNS, NVMe Simple Copy) ? we can just build
>> memoery backed NVMeOF test target for NVMe controller features.
>>
>> Also, recognizing the simplicity I proposed initially NVMe ZNS
>> fabrics based emulation over QEMU (I think I still have initial state
>> machine implementation code for ZNS somewhere), those were "nacked" for
>> the right reason, since we've decided go with QEMU and use that as a
>> primary platform for testing, so I failed to understand what has
>> changed.. since given that QEMU already supports NVMe simple copy ...
> 
> I was not part of this conversation, but as I see it each approach give
> a benefit. QEMU is fantastic for compliance testing and I am not sure
> you get the same level of command analysis anywhere else; at least not
> without writing dedicated code for this in a target.
> 
> This said, when we want to test for race conditions, QEMU is very slow.

Can you please elaborate the scenario and numbers for slowness of QEMU?

For race conditions testing we can build error injection framework
around the code implementation which present in kernel everywhere.

> For a software-only solution, we have experimented with something
> similar to the nvme-debug code tha Mikulas is proposing. Adam pointed to
> the nvme-loop target as an alternative and this seems to work pretty
> nicely. I do not believe there should be many changes to support copy
> offload using this.
> 

If QEMU is so incompetent then we need to add every big feature into
the NVMeOF test target so that we can test it better ? is that what
you are proposing ? since if we implement one feature, it will be
hard to nack any new features that ppl will come up with
same rationale "with QEMU being slow and hard to test race
conditions etc .."

and if that is the case why we don't have ZNS NVMeOF target
memory backed emulation ? Isn't that a bigger and more
complicated feature than Simple Copy where controller states
are involved with AENs ?

ZNS kernel code testing is also done on QEMU, I've also fixed
bugs in the ZNS kernel code which are discovered on QEMU and I've not
seen any issues with that. Given that simple copy feature is way smaller
than ZNS it will less likely to suffer from slowness and etc (listed
above) in QEMU.

my point is if we allow one, we will be opening floodgates and we need 
to be careful not to bloat the code unless it is _absolutely
necessary_ which I don't think it is based on the simple copy
specification.

> So in my view having both is not replication and it gives more
> flexibility for validation, which I believe it is always good.
> 

-ck