From: "John Groves" <john@jagalactic.com>
To: "Dan Williams" <dan.j.williams@intel.com>,
"John Groves" <jgroves@micron.com>,
"John Groves" <john@jagalactic.com>
Cc: "Vishal Verma" <vishal.l.verma@intel.com>,
"Dave Jiang" <dave.jiang@intel.com>,
"nvdimm@lists.linux.dev" <nvdimm@lists.linux.dev>,
"linux-cxl@vger.kernel.org" <linux-cxl@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"John Groves" <john@groves.net>
Subject: [PATCH RFC 0/4] dev_dax_iomap: introduce iomap for /dev/dax
Date: Wed, 6 Dec 2023 21:02:58 +0000 [thread overview]
Message-ID: <0100018c40f0e6d8-e9683548-4dde-4320-8ff3-3fd4103c7274-000000@email.amazonses.com> (raw)
In-Reply-To: 20231206210252.52107-1-john@jagalactic.com
From: John Groves <john@groves.net>
This patch set is not intended to be merged; I'm hoping to get some
clarification as to the correct approach (especialy from Dan).
This work is related to famfs, which is a dax file system for shared
fabric-attached memory (FAM). Famfs is "coming soon" as an RFC, but
the concept and requirements were presented at LPC 2023. See
https://lpc.events/event/17/contributions/1455/ and
https://www.youtube.com/watch?v=aA_DgO95gLo. My expectation is that
a future (fully working) version of this patch will be folded into the
famfs
patches.
Unlike the current set of fs-dax file systems, famfs does not need a block
(pmem) device, and should really run on a /dev/dax character device since
that's how sharable fabric-attached cxl memory will surface. But
/dev/dax character devices are missing some functionality that is provided
by the block /dev/pmem driver - specifically struct dax_operations pointer
in struct dax_device.
This patch, when CONFIG_DEV_DAX_IOMAP=y, populates dax_dev->ops for
character dax devices. The added operations differ (currently) from
/dev/pmem's dev_dax->ops in that they don't use memremap but instead
provide a physical address in response to the dev_dax->direct_access()
method.
The dax_operations are direct_access() (which resolves a dax dev offset
to an address), zero_page_range() and recovery_write(). I'm not sure yet
how to test the latter two, but the direct_access() method works in
conjunciton with famfs - but only for mmaped files.
But Posix reads fail. Specifically dax_iomap_iter() calls
dax_copy_to_iter(), which declines to copy the data for some reason in
one of the lower level copy_to_user variants. I've tried to isolate the
reason for the failure with a VM under gdb, but no luck there yet. I'm
wondering if there is some flag or attribute that needs to be applied to
these addresses/pages somehow to allow this to work.
The failing copy_to_user() differs from the same path with pmem fs-dax,
in that pmem does a memremap (which I think generates one contiguous
range, even if the device had more than one range - is this right, and
does this mean it's consuming some of the vmap/vmalloc range?)
I spent some time attempting a memremap, but I haven't figured out the
magic for that. However, I like the simplicity of resolving to phys if
that's not a non-starter for some reason.
I hope this is enough context for a meaningful review and suggestions as to
what a working dev_dax->dax_operations implementation should look like.
Thanks for any tips!
John Groves (4):
Add add_dax_ops() func for fs-dax to provide dax holder_ops
Temporary hacks due to linkage issues
Add dax_operations to /dev/dax struct dax_device
Add CONFIG_DEV_DAX_IOMAP kernel build parameter
drivers/dax/Kconfig | 6 ++
drivers/dax/bus.c | 155 ++++++++++++++++++++++++++++++++++++++++++++
drivers/dax/super.c | 16 +++++
include/linux/dax.h | 5 ++
4 files changed, 182 insertions(+)
--
2.40.1
next parent reply other threads:[~2023-12-06 21:02 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20231206210252.52107-1-john@jagalactic.com>
2023-12-06 21:02 ` John Groves [this message]
[not found] ` <20231206210252.52107-2-john@jagalactic.com>
2023-12-06 21:03 ` [PATCH RFC 1/4] dev_dax_iomap: Add add_dax_ops() func for fs-dax to provide dax holder_ops John Groves
[not found] ` <20231206210252.52107-3-john@jagalactic.com>
2023-12-06 21:03 ` [PATCH RFC 2/4] dev_dax_iomap: Temporary hacks due to linkage issues John Groves
[not found] ` <20231206210252.52107-4-john@jagalactic.com>
2023-12-06 21:03 ` [PATCH RFC 3/4] dev_dax_iomap: Add dax_operations to /dev/dax struct dax_device John Groves
[not found] ` <20231206210252.52107-5-john@jagalactic.com>
2023-12-06 21:03 ` [PATCH RFC 4/4] dev_dax_iomap: Add CONFIG_DEV_DAX_IOMAP kernel build parameter John Groves
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0100018c40f0e6d8-e9683548-4dde-4320-8ff3-3fd4103c7274-000000@email.amazonses.com \
--to=john@jagalactic.com \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=jgroves@micron.com \
--cc=john@groves.net \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nvdimm@lists.linux.dev \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).