From: "Pankaj Raghav (Samsung)" <kernel@pankajraghav.com>
To: willy@infradead.org, djwong@kernel.org, brauner@kernel.org,
david@fromorbit.com, chandan.babu@oracle.com,
akpm@linux-foundation.org
Cc: linux-fsdevel@vger.kernel.org, hare@suse.de,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
linux-xfs@vger.kernel.org, mcgrof@kernel.org,
gost.dev@samsung.com, p.raghav@samsung.com
Subject: [PATCH v4 00/11] enable bs > ps in XFS
Date: Thu, 25 Apr 2024 13:37:35 +0200 [thread overview]
Message-ID: <20240425113746.335530-1-kernel@pankajraghav.com> (raw)
From: Pankaj Raghav <p.raghav@samsung.com>
This is the fourth version of the series that enables block size > page size
(Large Block Size) in XFS. The context and motivation can be seen in cover
letter of the RFC v1[1]. We also recorded a talk about this effort at LPC [3],
if someone would like more context on this effort.
This series does not split a folio during truncation even though we have
an API to do so due to some issues with writeback. While it is not a
blocker, this feature can be added as a future improvement once we
get the base patches upstream (See patch 7).
A lot of emphasis has been put on testing using kdevops. The testing has
been split into regression and progression.
Regression testing:
In regression testing, we ran the whole test suite to check for
*regression on existing profiles due to the page cache changes.
No regression was found with the patches added on top.
Progression testing:
For progression testing, we tested for 8k, 16k, 32k and 64k block sizes.
To compare it with existing support, an ARM VM with 64k base page system
(without our patches) was used as a reference to check for actual failures
due to LBS support in a 4k base page size system.
There are some tests that assumes block size < page size that needs to
be fixed. I have a tree with fixes for xfstests here [6], which I will be
sending soon to the list. Already a part of this has been upstreamed to
fstest.
No new failures were found with the LBS support.
We've done some preliminary performance tests with fio on XFS on 4k block
size against pmem and NVMe with buffered IO and Direct IO on vanilla
Vs + these patches applied, and detected no regressions.
We also wrote an eBPF tool called blkalgn [7] to see if IO sent to the device
is aligned and at least filesystem block size in length.
Git tree:
https://github.com/linux-kdevops/linux/tree/large-block-minorder-6.9-rc4
[1] https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/
[2] https://lore.kernel.org/linux-xfs/20240213093713.1753368-1-kernel@pankajraghav.com/
[3] https://www.youtube.com/watch?v=ar72r5Xf7x4
[4] https://github.com/linux-kdevops/kdevops/blob/master/docs/xfs-bugs.md
489 non-critical issues and 55 critical issues. We've determined and reported
that the 55 critical issues have all fall into 5 common XFS asserts or hung
tasks and 2 memory management asserts.
[5] https://lore.kernel.org/linux-xfs/fe7fec1c-3b08-430f-9c95-ea76b237acf4@samsung.com/
[6] https://github.com/linux-kdevops/fstests/tree/lbs-fixes
[7] https://github.com/iovisor/bcc/pull/4813
Changes since v3:
- Cap the PTE range to i_size for LBS configuration in folio_map_range()
- Added Chinners kvmalloc xattr patches
- Moved Hannes patches before adding the minorder patches to avoid
confusion.
- Added mapping_set_folio_order_range().
- Return EINVAL instead EAGAIN in split_huge_page_to_list_to_order()
Changes since v2:
- Simplified the filemap and readahead changes. (Thanks willy)
- Removed DEFINE_READAHEAD_ALIGN.
- Added minorder support to readahead_expand().
Changes since v1:
- Round up to nearest min nr pages in ra_init
- Calculate index in filemap_create instead of doing in
filemap_get_pages
- Remove unnecessary BUG_ONs in the delete path
- Use check_shl_overflow instead of check_mul_overflow
- Cast to uint32_t instead of unsigned long in xfs_stat_blksize
Changes since RFC v2:
- Move order 1 patch above the 1st patch
- Remove order == 1 conditional in `fs: Allow fine-grained control of
folio sizes`. This fixed generic/630 that was reported in the previous version.
- Hide the max order and expose `mapping_set_folio_min_order` instead.
- Add new helper mapping_start_index_align and DEFINE_READAHEAD_ALIGN
- don't call `page_cache_ra_order` with min order in do_mmap_sync_readahead
- simplify ondemand readahead with only aligning the start index at the end
- Don't cap ra_pages based on bdi->io_pages
- use `checked_mul_overflow` while calculating bytes in validate_fsb
- Remove config lbs option
- Add a warning while mounting a LBS kernel
- Add Acked-by and Reviewed-by from Hannes and Darrick.
Changes since RFC v1:
- Added willy's patch to enable order-1 folios.
- Unified common page cache effort from Hannes LBS work.
- Added a new helper min_nrpages and added CONFIG_THP for enabling mapping_large_folio_support
- Don't split a folio if it has minorder set. Remove the old code where we set extra pins if it has that requirement.
- Split the code in XFS between the validation of mapping count. Put the icache code changes with enabling bs > ps.
- Added CONFIG_XFS_LBS option
- align the index in do_read_cache_folio()
- Removed truncate changes
- Fixed generic/091 with iomap changes to iomap_dio_zero function.
- Took care of folio truncation scenario in page_cache_ra_unbounded() that happens after read_pages if a folio was found.
- Sqaushed and moved commits around
- Rebased on top of v6.8-rc4
Dave Chinner (1):
xfs: use kvmalloc for xattr buffers
Hannes Reinecke (1):
readahead: rework loop in page_cache_ra_unbounded()
Luis Chamberlain (1):
filemap: allocate mapping_min_order folios in the page cache
Matthew Wilcox (Oracle) (1):
fs: Allow fine-grained control of folio sizes
Pankaj Raghav (7):
readahead: allocate folios with mapping_min_order in readahead
mm: do not split a folio if it has minimum folio order requirement
filemap: cap PTE range to be created to i_size in folio_map_range()
iomap: fix iomap_dio_zero() for fs bs > system page size
xfs: expose block size in stat
xfs: make the calculation generic in xfs_sb_validate_fsb_count()
xfs: enable block size larger than page size support
fs/iomap/direct-io.c | 13 +++-
fs/xfs/libxfs/xfs_attr_leaf.c | 15 ++---
fs/xfs/libxfs/xfs_ialloc.c | 5 ++
fs/xfs/libxfs/xfs_shared.h | 3 +
fs/xfs/xfs_icache.c | 6 +-
fs/xfs/xfs_iops.c | 2 +-
fs/xfs/xfs_mount.c | 10 ++-
fs/xfs/xfs_super.c | 10 +--
include/linux/pagemap.h | 116 ++++++++++++++++++++++++++++------
mm/filemap.c | 29 ++++++---
mm/huge_memory.c | 9 +++
mm/readahead.c | 94 +++++++++++++++++++++------
12 files changed, 242 insertions(+), 70 deletions(-)
base-commit: 0bbac3facb5d6cc0171c45c9873a2dc96bea9680
--
2.34.1
next reply other threads:[~2024-04-25 11:37 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-25 11:37 Pankaj Raghav (Samsung) [this message]
2024-04-25 11:37 ` [PATCH v4 01/11] readahead: rework loop in page_cache_ra_unbounded() Pankaj Raghav (Samsung)
2024-04-25 11:37 ` [PATCH v4 02/11] fs: Allow fine-grained control of folio sizes Pankaj Raghav (Samsung)
2024-04-25 18:07 ` Hannes Reinecke
2024-04-26 15:09 ` Darrick J. Wong
2024-04-25 11:37 ` [PATCH v4 03/11] filemap: allocate mapping_min_order folios in the page cache Pankaj Raghav (Samsung)
2024-04-25 19:04 ` Hannes Reinecke
2024-04-26 15:12 ` Darrick J. Wong
2024-04-28 20:59 ` Pankaj Raghav (Samsung)
2024-04-25 11:37 ` [PATCH v4 04/11] readahead: allocate folios with mapping_min_order in readahead Pankaj Raghav (Samsung)
2024-04-25 18:53 ` Matthew Wilcox
2024-04-25 11:37 ` [PATCH v4 05/11] mm: do not split a folio if it has minimum folio order requirement Pankaj Raghav (Samsung)
2024-04-25 20:10 ` Matthew Wilcox
2024-04-26 0:47 ` Luis Chamberlain
2024-04-26 23:46 ` Luis Chamberlain
2024-04-28 0:57 ` Luis Chamberlain
2024-04-29 3:56 ` Luis Chamberlain
2024-04-29 14:29 ` Zi Yan
2024-04-30 0:31 ` Luis Chamberlain
2024-04-30 0:49 ` Luis Chamberlain
2024-04-30 2:43 ` Zi Yan
2024-04-30 19:27 ` Luis Chamberlain
2024-05-01 4:13 ` Matthew Wilcox
2024-05-01 14:28 ` Matthew Wilcox
2024-04-26 15:49 ` Pankaj Raghav (Samsung)
2024-04-25 11:37 ` [PATCH v4 06/11] filemap: cap PTE range to be created to i_size in folio_map_range() Pankaj Raghav (Samsung)
2024-04-25 20:24 ` Matthew Wilcox
2024-04-26 12:54 ` Pankaj Raghav (Samsung)
2024-04-25 11:37 ` [PATCH v4 07/11] iomap: fix iomap_dio_zero() for fs bs > system page size Pankaj Raghav (Samsung)
2024-04-26 6:22 ` Christoph Hellwig
2024-04-26 11:43 ` Pankaj Raghav (Samsung)
2024-04-27 5:12 ` Christoph Hellwig
2024-04-29 21:02 ` Pankaj Raghav (Samsung)
2024-04-27 3:26 ` Matthew Wilcox
2024-04-27 4:52 ` Christoph Hellwig
2024-04-25 11:37 ` [PATCH v4 08/11] xfs: use kvmalloc for xattr buffers Pankaj Raghav (Samsung)
2024-04-26 15:18 ` Darrick J. Wong
2024-04-28 21:06 ` Pankaj Raghav (Samsung)
2024-04-25 11:37 ` [PATCH v4 09/11] xfs: expose block size in stat Pankaj Raghav (Samsung)
2024-04-26 15:15 ` Darrick J. Wong
2024-04-25 11:37 ` [PATCH v4 10/11] xfs: make the calculation generic in xfs_sb_validate_fsb_count() Pankaj Raghav (Samsung)
2024-04-26 15:16 ` Darrick J. Wong
2024-04-25 11:37 ` [PATCH v4 11/11] xfs: enable block size larger than page size support Pankaj Raghav (Samsung)
2024-04-26 15:18 ` Darrick J. Wong
2024-04-27 4:42 ` [PATCH v4 00/11] enable bs > ps in XFS Ritesh Harjani
2024-04-27 5:05 ` Darrick J. Wong
2024-04-29 20:39 ` Pankaj Raghav (Samsung)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240425113746.335530-1-kernel@pankajraghav.com \
--to=kernel@pankajraghav.com \
--cc=akpm@linux-foundation.org \
--cc=brauner@kernel.org \
--cc=chandan.babu@oracle.com \
--cc=david@fromorbit.com \
--cc=djwong@kernel.org \
--cc=gost.dev@samsung.com \
--cc=hare@suse.de \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
--cc=mcgrof@kernel.org \
--cc=p.raghav@samsung.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.