From: David Howells <dhowells@redhat.com>
To: Steve French <smfrench@gmail.com>
Cc: David Howells <dhowells@redhat.com>,
Jeff Layton <jlayton@kernel.org>,
Matthew Wilcox <willy@infradead.org>,
Paulo Alcantara <pc@manguebit.com>,
Shyam Prasad N <sprasad@microsoft.com>,
Tom Talpey <tom@talpey.com>,
Christian Brauner <christian@brauner.io>,
netfs@lists.linux.dev, linux-cifs@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org
Subject: [PATCH v7 00/16] netfs, cifs: Delegate high-level I/O to netfslib
Date: Tue, 30 Apr 2024 15:09:12 +0100 [thread overview]
Message-ID: <20240430140930.262762-1-dhowells@redhat.com> (raw)
Hi Steve,
Here are patches to convert cifs to use the netfslib library. I've tested
them with and without a cache. There appears to be a signifcant
performance improvement in buffered writeback (around 50% throughput rate
with fio tests).
The patches remove around 2000 lines from CIFS.
Notes:
(1) CIFS is made to use unbuffered I/O for unbuffered caching modes and
write-through caching for cache=strict.
(2) Various cifs fallocate() function implementations have issues that
aren't easily fixed without enhanced protocol support.
(3) It should be possible to turn on multipage folio support in CIFS now.
(4) The then-unused CIFS code is removed in three patches, not one, to
avoid the git patch generator from producing confusing patches in
which it thinks code is being moved around rather than just being
removed.
The patches can be found here also:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=cifs-netfs
Changes
=======
ver #7)
- Rebased on -rc6 to absorb some fixes that have gone upstream, including
a fix for a leak of the fscache volume cookie.
- Added a patch to turn on multipage folio support in cifs.
ver #6)
- The netfslib write helpers got rewritten and, consequently, some changes
were made here.
- Rearranged the patch order a little to put the smaller changes first.
- Use a different way to invalidate that doesn't use ->launder_folio().
- Attempt to open the handle on the server READ+WRITE, even if the user
asks for O_WRONLY, if the mount was with "-o fsc" so that we can fill in
the gaps in partial folio writes; if we can't fall back to WRITE-only
and disable caching.
- Fixed the cookie key to match the key used by iget5_locked() to avoid
"Duplicate cookie" errors.
- Made add_credits_and_wake_if() clear the number of returned credits,
allowing it to be called multiple times in cleanup/error handling.
- Made ->async_writev() not return an error directly, but always set it on
the subreq.
- Fixed error handling from writethrough writing.
ver #5)
- Rebased to -rc3 plus SteveF's for-next branch as netfslib is now
upstream, as are a couple of patches from this series.
- Replace the ->replay bool Shyam added with a flag on the netfs
subrequest. This is tested by the code, but not currently set (see
above).
ver #4)
- Slimmed down the branch:
- Split the cifs-related patches off to a separate branch (cifs-netfs)
- Deferred the content-encryption to the in-progress ceph changes.
- Deferred the use-PG_writeback rather than PG_fscache patch
- Rebased on a later linux-next with afs-rotation patches.
ver #3)
- Moved the fscache module into netfslib to avoid export cycles.
- Fixed a bunch of bugs.
- Got CIFS to pass as much of xfstests as possible.
- Added a patch to make 9P use all the helpers.
- Added a patch to stop using PG_fscache, but rather dirty pages on
reading and have writepages write to the cache.
ver #2)
- Folded the addition of NETFS_RREQ_NONBLOCK/BLOCKED into first patch that
uses them.
- Folded addition of rsize member into first user.
- Don't set rsize in ceph (yet) and set it in kafs to 256KiB. cifs sets
it dynamically.
- Moved direct_bv next to direct_bv_count in struct netfs_io_request and
labelled it with a __counted_by().
- Passed flags into netfs_xa_store_and_mark() rather than two bools.
- Removed netfs_set_up_buffer() as it wasn't used.
David
Link: https://lore.kernel.org/r/20231213152350.431591-1-dhowells@redhat.com/ [1]
Link: https://lore.kernel.org/r/20231013160423.2218093-1-dhowells@redhat.com/ # v1
Link: https://lore.kernel.org/r/20231117211544.1740466-1-dhowells@redhat.com/ # v2
Link: https://lore.kernel.org/r/20231207212206.1379128-1-dhowells@redhat.com/ # v3
Link: https://lore.kernel.org/r/20231213154139.432922-1-dhowells@redhat.com/ # v4
Link: https://lore.kernel.org/r/20240205225726.3104808-1-dhowells@redhat.com/ # v5
David Howells (16):
cifs: Use alternative invalidation to using launder_folio
cifs: Replace cifs_readdata with a wrapper around netfs_io_subrequest
cifs: Replace cifs_writedata with a wrapper around netfs_io_subrequest
cifs: Use more fields from netfs_io_subrequest
cifs: Make wait_mtu_credits take size_t args
cifs: Replace the writedata replay bool with a netfs sreq flag
cifs: Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c
cifs: Set zero_point in the copy_file_range() and remap_file_range()
cifs: Add mempools for cifs_io_request and cifs_io_subrequest structs
cifs: Make add_credits_and_wake_if() clear deducted credits
cifs: Implement netfslib hooks
cifs: Cut over to using netfslib
cifs: Remove some code that's no longer used, part 1
cifs: Remove some code that's no longer used, part 2
cifs: Remove some code that's no longer used, part 3
cifs: Enable large folio support
fs/netfs/buffered_write.c | 6 +
fs/netfs/io.c | 7 +-
fs/smb/client/Kconfig | 1 +
fs/smb/client/cifsfs.c | 124 +-
fs/smb/client/cifsfs.h | 11 +-
fs/smb/client/cifsglob.h | 65 +-
fs/smb/client/cifsproto.h | 12 +-
fs/smb/client/cifssmb.c | 120 +-
fs/smb/client/file.c | 2898 ++++++----------------------------
fs/smb/client/fscache.c | 109 --
fs/smb/client/fscache.h | 54 -
fs/smb/client/inode.c | 45 +-
fs/smb/client/smb2ops.c | 10 +-
fs/smb/client/smb2pdu.c | 186 ++-
fs/smb/client/smb2proto.h | 5 +-
fs/smb/client/trace.h | 144 +-
fs/smb/client/transport.c | 17 +-
include/linux/netfs.h | 1 +
include/trace/events/netfs.h | 1 +
19 files changed, 886 insertions(+), 2930 deletions(-)
next reply other threads:[~2024-04-30 14:09 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-30 14:09 David Howells [this message]
2024-04-30 14:09 ` [PATCH v7 01/16] cifs: Use alternative invalidation to using launder_folio David Howells
2024-04-30 14:09 ` [PATCH v7 02/16] cifs: Replace cifs_readdata with a wrapper around netfs_io_subrequest David Howells
2024-04-30 14:09 ` [PATCH v7 03/16] cifs: Replace cifs_writedata " David Howells
2024-04-30 14:09 ` [PATCH v7 04/16] cifs: Use more fields from netfs_io_subrequest David Howells
2024-04-30 14:09 ` [PATCH v7 05/16] cifs: Make wait_mtu_credits take size_t args David Howells
2024-04-30 14:09 ` [PATCH v7 06/16] cifs: Replace the writedata replay bool with a netfs sreq flag David Howells
2024-04-30 14:09 ` [PATCH v7 07/16] cifs: Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c David Howells
2024-04-30 14:09 ` [PATCH v7 08/16] cifs: Set zero_point in the copy_file_range() and remap_file_range() David Howells
2024-04-30 14:09 ` [PATCH v7 09/16] cifs: Add mempools for cifs_io_request and cifs_io_subrequest structs David Howells
2024-04-30 14:09 ` [PATCH v7 10/16] cifs: Make add_credits_and_wake_if() clear deducted credits David Howells
2024-04-30 14:09 ` [PATCH v7 11/16] cifs: Implement netfslib hooks David Howells
2024-04-30 14:09 ` [PATCH v7 12/16] cifs: Cut over to using netfslib David Howells
2024-04-30 14:09 ` [PATCH v7 13/16] cifs: Remove some code that's no longer used, part 1 David Howells
2024-04-30 14:09 ` [PATCH v7 14/16] cifs: Remove some code that's no longer used, part 2 David Howells
2024-04-30 14:09 ` [PATCH v7 15/16] cifs: Remove some code that's no longer used, part 3 David Howells
2024-04-30 14:09 ` [PATCH v7 16/16] cifs: Enable large folio support David Howells
2024-04-30 14:47 ` [PATCH v7 00/16] netfs, cifs: Delegate high-level I/O to netfslib David Howells
2024-04-30 14:58 ` Steve French
2024-05-02 14:13 ` Christian Brauner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240430140930.262762-1-dhowells@redhat.com \
--to=dhowells@redhat.com \
--cc=christian@brauner.io \
--cc=jlayton@kernel.org \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=netfs@lists.linux.dev \
--cc=pc@manguebit.com \
--cc=smfrench@gmail.com \
--cc=sprasad@microsoft.com \
--cc=tom@talpey.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).