From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 491FAC433FE for ; Fri, 4 Feb 2022 19:42:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234499AbiBDTmF (ORCPT ); Fri, 4 Feb 2022 14:42:05 -0500 Received: from mail-lf1-f43.google.com ([209.85.167.43]:35804 "EHLO mail-lf1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231869AbiBDTmF (ORCPT ); Fri, 4 Feb 2022 14:42:05 -0500 Received: by mail-lf1-f43.google.com with SMTP id i34so14688673lfv.2; Fri, 04 Feb 2022 11:42:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=ow88Lu4aUs6K7UkGLeI5Z43Nbv5HpcgNARMLv9ghlkU=; b=TpFi/w8V+E7pbO9sIvLpXZCpdWezLN3gUSIfpae0rAfXunnHQoI1jO+McdOkoyyvry rpUeXbR2S4FMUcIK2ROcUGKWUUt29F/xn7gfPQugn7xPgfsgy2RjXGSLTqvEFqxQizUs UemRWsnEtkd4NE/9MnMJJY+2kzZrILdV0UEahM7z9pM9c2PBRwlaavtJq04TKtp2RKMi SfNZHW788cW54l6at6TZuezDjBfN360XlNenjVWgmSNXGdnmUCR4lwh/F+hZxzOJfoTa zTYtOv5OsrlN8LnEyXtPgrSrhoEO0tFX8dw7kzuvq0hNgelyDruS6BhYPwSZgM8gE7FG /67g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ow88Lu4aUs6K7UkGLeI5Z43Nbv5HpcgNARMLv9ghlkU=; b=PklOu0iEVkLTWJvT2BKPgl42XNmg1i6rzSwU4dwKuon9cmM/2i+/NxrEJDRjsDjAOY rBbHssziydgFD1CnjQ3D8OWirS60tAOEHDA/cbJrIoBBJMQ+txS+1QR9Boq+1o/sXMeB aWKQWx5KSIEz7M7NMKIL4zXRpxZNEHkbHKgaz0y/13bCf6JGV+vy3hved4o/L63Qpe7Y dzbKtJtfXoHpqT3+bQYjDXxy6laVebBaRkH8JYWwfWBch+rD1mwP0bH7e8WUskVi75oE QV6w9DxixIo1nhjdEzbjZZatS2RxAwHLczAPKrfLPXkxBLUeEbZ5jc1J1SuGJGHPaRSj 25iQ== X-Gm-Message-State: AOAM530ojHWL0nvKdUebubKSDVT3otEdHG1JD3wpc4rfjGzZ9Vmo+gFY jrve144V538Ar+zpxnCv/UIW/tPuvfA287vbrQI= X-Google-Smtp-Source: ABdhPJyHIm2s/AwXxjYIOHGmKgAxztwbsh5UaTIz/wUfKedoZaCqO6GHsMXWPEigismScFC9Hc7Iu0UaLWeROhbCTFw= X-Received: by 2002:ac2:4853:: with SMTP id 19mr346407lfy.563.1644003723415; Fri, 04 Feb 2022 11:42:03 -0800 (PST) MIME-Version: 1.0 References: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> In-Reply-To: From: Nitesh Shetty Date: Sat, 5 Feb 2022 01:11:51 +0530 Message-ID: Subject: Re: [RFC PATCH 0/3] NVMe copy offload patches To: Mikulas Patocka Cc: =?UTF-8?Q?Javier_Gonz=C3=A1lez?= , Chaitanya Kulkarni , "linux-block@vger.kernel.org" , "linux-scsi@vger.kernel.org" , "dm-devel@redhat.com" , "linux-nvme@lists.infradead.org" , linux-fsdevel , Jens Axboe , "msnitzer@redhat.com >> msnitzer@redhat.com" , Bart Van Assche , "martin.petersen@oracle.com >> Martin K. Petersen" , "roland@purestorage.com" , Hannes Reinecke , "kbus @imap.gmail.com>> Keith Busch" , Christoph Hellwig , "Frederick.Knight@netapp.com" , "zach.brown@ni.com" , "osandov@fb.com" , "lsf-pc@lists.linux-foundation.org" , "djwong@kernel.org" , "josef@toxicpanda.com" , "clm@fb.com" , "dsterba@suse.com" , "tytso@mit.edu" , "jack@suse.com" , Kanchan Joshi , arnav.dawn@samsung.com, Nitesh Shetty Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Wed, Feb 2, 2022 at 12:23 PM Mikulas Patocka wrote= : > > Hi > > Here I'm submitting the first version of NVMe copy offload patches as a > request for comment. They use the token-based approach as we discussed on > the phone call. > > The first patch adds generic copy offload support to the block layer - it > adds two new bio types (REQ_OP_COPY_READ_TOKEN and > REQ_OP_COPY_WRITE_TOKEN) and a new ioctl BLKCOPY and a kernel function > blkdev_issue_copy. > > The second patch adds copy offload support to the NVMe subsystem. > > The third patch implements a "nvme-debug" driver - it is similar to > "scsi-debug", it simulates a nvme host controller, it keeps data in memor= y > and it supports copy offload according to NVMe Command Set Specification > 1.0a. (there are no hardware or software implementations supporting copy > offload so far, so I implemented it in nvme-debug) > > TODO: > * implement copy offload in device mapper linear target > * implement copy offload in software NVMe target driver We had a series that adds these two elements https://github.com/nitesh-shetty/linux_copy_offload/tree/main/v1 Overall series supports =E2=80=93 1. Multi-source/destination interface(yes, it does add complexity but GC use-case needs it) 2. Copy-emulation at block-layer 3. Dm-linear and dm-kcopyd support (for cases not requiring split) 4. Nvmet support (for block and file backend) These patches definitely need more feedback. If links are hard to read, we can send another RFC instead. But before that it would be great to have = your inputs on the path forward. But before that it would be great to have your inputs on the path forward. PS: The payload-scheme in your series is particularly interesting and simplifying plumbing, so you might notice that above patches borrow that > * make it possible to complete REQ_OP_COPY_WRITE_TOKEN bios asynchronousl= y Patch[0] support asynchronous copy write,if multi dst/src payload is sent. [0] https://github.com/nitesh-shetty/linux_copy_offload/blob/main/v1/0003-b= lock-Add-copy-offload-support-infrastructure.patch -- Nitesh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2357DC433EF for ; Mon, 7 Feb 2022 09:34:16 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-673-mfbtN_ZHN3Ck-t_pb4ClYQ-1; Mon, 07 Feb 2022 04:34:11 -0500 X-MC-Unique: mfbtN_ZHN3Ck-t_pb4ClYQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8789384BA42; Mon, 7 Feb 2022 09:34:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 66E7773164; Mon, 7 Feb 2022 09:34:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3AA851802E34; Mon, 7 Feb 2022 09:34:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 214JgC1j007267 for ; Fri, 4 Feb 2022 14:42:12 -0500 Received: by smtp.corp.redhat.com (Postfix) id 99A7D2166BA3; Fri, 4 Feb 2022 19:42:12 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 954B62166B1A for ; Fri, 4 Feb 2022 19:42:09 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6FCFB10AF7C4 for ; Fri, 4 Feb 2022 19:42:09 +0000 (UTC) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-NDA4JWlmNOylSLJAV1G-Gg-1; Fri, 04 Feb 2022 14:42:05 -0500 X-MC-Unique: NDA4JWlmNOylSLJAV1G-Gg-1 Received: by mail-lf1-f44.google.com with SMTP id o12so14549946lfg.12; Fri, 04 Feb 2022 11:42:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ow88Lu4aUs6K7UkGLeI5Z43Nbv5HpcgNARMLv9ghlkU=; b=uW/6lSXyd9OOES9uhStPTDBFZwpw5sb+0q7GnuZdV8g0zCBOU/vrKeQkczcx8owxRV 3k613gFamWufgO+gjRxQFInFCZDYLGzBSrzfDJIz4lsZLSKVbPvdCGHeldg9cqsWXRMM TNdv0Sq5YpzpRqFSD0qZbpotSUHNYpCR5dZ3Qx6vG+2Cb83iTQp0RWoJGH7iX1e672tY 9b9Jt63j4zIKVEoZoYEtxaVvZiHq96iyQGQkv3rf/RkYfbwVPDRu+qXDSKI62g76ZXVl 2bs2mqxvxxGMVn5f64uksJZhZ7RrI7qsr5gRByrAvigHn9ohuxIUKxyxAmDenZsvbN8G fx7A== X-Gm-Message-State: AOAM530UuScytpWMKm67LuRVM1zAZwuFIoQlgdySo9mZgn3S6nO19/XZ JXuHVRfKUv2LxcoZnn9JxhpxliIseLMAgpKguhVjk2H9lnGfAa7e X-Google-Smtp-Source: ABdhPJyHIm2s/AwXxjYIOHGmKgAxztwbsh5UaTIz/wUfKedoZaCqO6GHsMXWPEigismScFC9Hc7Iu0UaLWeROhbCTFw= X-Received: by 2002:ac2:4853:: with SMTP id 19mr346407lfy.563.1644003723415; Fri, 04 Feb 2022 11:42:03 -0800 (PST) MIME-Version: 1.0 References: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> In-Reply-To: From: Nitesh Shetty Date: Sat, 5 Feb 2022 01:11:51 +0530 Message-ID: To: Mikulas Patocka X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 214JgC1j007267 X-loop: dm-devel@redhat.com X-Mailman-Approved-At: Mon, 07 Feb 2022 04:33:18 -0500 Cc: "djwong@kernel.org" , "linux-nvme@lists.infradead.org" , "clm@fb.com" , "dm-devel@redhat.com" , "osandov@fb.com" , =?UTF-8?Q?Javier_Gonz=C3=A1lez?= , Bart Van Assche , "linux-scsi@vger.kernel.org" , Christoph Hellwig , "roland@purestorage.com" , Nitesh Shetty , "zach.brown@ni.com" , Chaitanya Kulkarni , "msnitzer@redhat.com >> msnitzer@redhat.com" , "josef@toxicpanda.com" , "linux-block@vger.kernel.org" , "dsterba@suse.com" , "kbus @imap.gmail.com>> Keith Busch" , "Frederick.Knight@netapp.com" , Jens Axboe , "tytso@mit.edu" , Kanchan Joshi , "martin.petersen@oracle.com >> Martin K. Petersen" , arnav.dawn@samsung.com, "jack@suse.com" , linux-fsdevel , "lsf-pc@lists.linux-foundation.org" Subject: Re: [dm-devel] [RFC PATCH 0/3] NVMe copy offload patches X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gV2VkLCBGZWIgMiwgMjAyMiBhdCAxMjoyMyBQTSBNaWt1bGFzIFBhdG9ja2EgPG1wYXRvY2th QHJlZGhhdC5jb20+IHdyb3RlOgo+Cj4gSGkKPgo+IEhlcmUgSSdtIHN1Ym1pdHRpbmcgdGhlIGZp cnN0IHZlcnNpb24gb2YgTlZNZSBjb3B5IG9mZmxvYWQgcGF0Y2hlcyBhcyBhCj4gcmVxdWVzdCBm b3IgY29tbWVudC4gVGhleSB1c2UgdGhlIHRva2VuLWJhc2VkIGFwcHJvYWNoIGFzIHdlIGRpc2N1 c3NlZCBvbgo+IHRoZSBwaG9uZSBjYWxsLgo+Cj4gVGhlIGZpcnN0IHBhdGNoIGFkZHMgZ2VuZXJp YyBjb3B5IG9mZmxvYWQgc3VwcG9ydCB0byB0aGUgYmxvY2sgbGF5ZXIgLSBpdAo+IGFkZHMgdHdv IG5ldyBiaW8gdHlwZXMgKFJFUV9PUF9DT1BZX1JFQURfVE9LRU4gYW5kCj4gUkVRX09QX0NPUFlf V1JJVEVfVE9LRU4pIGFuZCBhIG5ldyBpb2N0bCBCTEtDT1BZIGFuZCBhIGtlcm5lbCBmdW5jdGlv bgo+IGJsa2Rldl9pc3N1ZV9jb3B5Lgo+Cj4gVGhlIHNlY29uZCBwYXRjaCBhZGRzIGNvcHkgb2Zm bG9hZCBzdXBwb3J0IHRvIHRoZSBOVk1lIHN1YnN5c3RlbS4KPgo+IFRoZSB0aGlyZCBwYXRjaCBp bXBsZW1lbnRzIGEgIm52bWUtZGVidWciIGRyaXZlciAtIGl0IGlzIHNpbWlsYXIgdG8KPiAic2Nz aS1kZWJ1ZyIsIGl0IHNpbXVsYXRlcyBhIG52bWUgaG9zdCBjb250cm9sbGVyLCBpdCBrZWVwcyBk YXRhIGluIG1lbW9yeQo+IGFuZCBpdCBzdXBwb3J0cyBjb3B5IG9mZmxvYWQgYWNjb3JkaW5nIHRv IE5WTWUgQ29tbWFuZCBTZXQgU3BlY2lmaWNhdGlvbgo+IDEuMGEuICh0aGVyZSBhcmUgbm8gaGFy ZHdhcmUgb3Igc29mdHdhcmUgaW1wbGVtZW50YXRpb25zIHN1cHBvcnRpbmcgY29weQo+IG9mZmxv YWQgc28gZmFyLCBzbyBJIGltcGxlbWVudGVkIGl0IGluIG52bWUtZGVidWcpCj4KPiBUT0RPOgo+ ICogaW1wbGVtZW50IGNvcHkgb2ZmbG9hZCBpbiBkZXZpY2UgbWFwcGVyIGxpbmVhciB0YXJnZXQK PiAqIGltcGxlbWVudCBjb3B5IG9mZmxvYWQgaW4gc29mdHdhcmUgTlZNZSB0YXJnZXQgZHJpdmVy CgpXZSBoYWQgYSBzZXJpZXMgdGhhdCBhZGRzIHRoZXNlIHR3byBlbGVtZW50cwpodHRwczovL2dp dGh1Yi5jb20vbml0ZXNoLXNoZXR0eS9saW51eF9jb3B5X29mZmxvYWQvdHJlZS9tYWluL3YxCgpP dmVyYWxsIHNlcmllcyBzdXBwb3J0cyDigJMKMS4gICAgTXVsdGktc291cmNlL2Rlc3RpbmF0aW9u IGludGVyZmFjZSh5ZXMsIGl0IGRvZXMgYWRkIGNvbXBsZXhpdHkKYnV0IEdDIHVzZS1jYXNlIG5l ZWRzIGl0KQoyLiAgICBDb3B5LWVtdWxhdGlvbiBhdCBibG9jay1sYXllcgozLiAgICBEbS1saW5l YXIgYW5kIGRtLWtjb3B5ZCBzdXBwb3J0IChmb3IgY2FzZXMgbm90IHJlcXVpcmluZyBzcGxpdCkK NC4gICAgTnZtZXQgc3VwcG9ydCAoZm9yIGJsb2NrIGFuZCBmaWxlIGJhY2tlbmQpCgpUaGVzZSBw YXRjaGVzIGRlZmluaXRlbHkgbmVlZCBtb3JlIGZlZWRiYWNrLiBJZiBsaW5rcyBhcmUgaGFyZCB0 byByZWFkLAp3ZSBjYW4gc2VuZCBhbm90aGVyIFJGQyBpbnN0ZWFkLiBCdXQgYmVmb3JlIHRoYXQg aXQgd291bGQgYmUgZ3JlYXQgdG8gaGF2ZSB5b3VyCmlucHV0cyBvbiB0aGUgcGF0aCBmb3J3YXJk LgpCdXQgYmVmb3JlIHRoYXQgaXQgd291bGQgYmUgZ3JlYXQgdG8gaGF2ZSB5b3VyIGlucHV0cyBv biB0aGUgcGF0aCBmb3J3YXJkLgoKUFM6IFRoZSBwYXlsb2FkLXNjaGVtZSBpbiB5b3VyIHNlcmll cyBpcyBwYXJ0aWN1bGFybHkgaW50ZXJlc3RpbmcgYW5kCnNpbXBsaWZ5aW5nIHBsdW1iaW5nLCBz byB5b3UgbWlnaHQgbm90aWNlIHRoYXQgYWJvdmUgcGF0Y2hlcyBib3Jyb3cgdGhhdAoKPiAqIG1h a2UgaXQgcG9zc2libGUgdG8gY29tcGxldGUgUkVRX09QX0NPUFlfV1JJVEVfVE9LRU4gYmlvcyBh c3luY2hyb25vdXNseQpQYXRjaFswXSBzdXBwb3J0IGFzeW5jaHJvbm91cyBjb3B5IHdyaXRlLGlm IG11bHRpIGRzdC9zcmMgcGF5bG9hZCBpcyBzZW50LgoKWzBdIGh0dHBzOi8vZ2l0aHViLmNvbS9u aXRlc2gtc2hldHR5L2xpbnV4X2NvcHlfb2ZmbG9hZC9ibG9iL21haW4vdjEvMDAwMy1ibG9jay1B ZGQtY29weS1vZmZsb2FkLXN1cHBvcnQtaW5mcmFzdHJ1Y3R1cmUucGF0Y2gKCi0tIE5pdGVzaAoK Ci0tCmRtLWRldmVsIG1haWxpbmcgbGlzdApkbS1kZXZlbEByZWRoYXQuY29tCmh0dHBzOi8vbGlz dG1hbi5yZWRoYXQuY29tL21haWxtYW4vbGlzdGluZm8vZG0tZGV2ZWw=