* [GIT PULLBOMB v13.4] xfs: directory parent pointers
@ 2024-04-24 3:30 Darrick J. Wong
2024-04-24 3:33 ` [GIT PULL 1/9] xfs: shrink struct xfs_da_args Darrick J. Wong
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:30 UTC (permalink / raw
To: Chandan Babu R; +Cc: linux-xfs, Christoph Hellwig
Hi Chandan,
Christoph and I have finished reviewing parent pointers and think it's
ready to be merged, so please accept these pull requests against today's
for-next branch.
For anyone following along at home, the xfsprogs patches are here:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub-6.10_2024-04-23
--D
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 1/9] xfs: shrink struct xfs_da_args
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
@ 2024-04-24 3:33 ` Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 2/9] xfs: improve extended attribute validation Darrick J. Wong
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:33 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 6a94b1acda7e7262418e23f906c12a2b08b69d12:
xfs: reinstate delalloc for RT inodes (if sb_rextsize == 1) (2024-04-22 18:00:50 +0530)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/shrink-dirattr-args-6.10_2024-04-23
for you to fetch changes up to cda60317ac57add7a0a2865aa29afbc6caad3e9a:
xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700)
----------------------------------------------------------------
xfs: shrink struct xfs_da_args [v13.4 1/9]
Let's clean out some unused flags and fields from struct xfs_da_args.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (5):
xfs: remove XFS_DA_OP_REMOVE
xfs: remove XFS_DA_OP_NOTIME
xfs: remove xfs_da_args.attr_flags
xfs: make attr removal an explicit operation
xfs: rearrange xfs_da_args a bit to use less space
fs/xfs/libxfs/xfs_attr.c | 31 ++++++++++++++++---------------
fs/xfs/libxfs/xfs_attr.h | 11 +++++++++--
fs/xfs/libxfs/xfs_da_btree.h | 29 +++++++++++++----------------
fs/xfs/scrub/attr.c | 1 -
fs/xfs/scrub/attr_repair.c | 3 +--
fs/xfs/xfs_acl.c | 17 +++++++++--------
fs/xfs/xfs_ioctl.c | 19 ++++++++++---------
fs/xfs/xfs_iops.c | 2 +-
fs/xfs/xfs_trace.h | 7 +------
fs/xfs/xfs_xattr.c | 22 ++++++++++++++++++----
fs/xfs/xfs_xattr.h | 3 ++-
11 files changed, 80 insertions(+), 65 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 2/9] xfs: improve extended attribute validation
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
2024-04-24 3:33 ` [GIT PULL 1/9] xfs: shrink struct xfs_da_args Darrick J. Wong
@ 2024-04-24 3:34 ` Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 3/9] xfs: Parent Pointers Darrick J. Wong
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:34 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit cda60317ac57add7a0a2865aa29afbc6caad3e9a:
xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/improve-attr-validation-6.10_2024-04-23
for you to fetch changes up to ea0b3e814741fb64e7785b564ea619578058e0b0:
xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700)
----------------------------------------------------------------
xfs: improve extended attribute validation [v13.4 2/9]
Prior to introducing parent pointer extended attributes, let's spend
some time cleaning up the attr code and strengthening the validation
that it performs on attrs coming in from the disk.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (14):
xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
xfs: fix missing check for invalid attr flags
xfs: check shortform attr entry flags specifically
xfs: restructure xfs_attr_complete_op a bit
xfs: use helpers to extract xattr op from opflags
xfs: validate recovered name buffers when recovering xattr items
xfs: always set args->value in xfs_attri_item_recover
xfs: use local variables for name and value length in _attri_commit_pass2
xfs: refactor name/length checks in xfs_attri_validate
xfs: refactor name/value iovec validation in xlog_recover_attri_commit_pass2
xfs: enforce one namespace per attribute
fs/xfs/libxfs/xfs_attr.c | 37 ++++++-
fs/xfs/libxfs/xfs_attr.h | 9 +-
fs/xfs/libxfs/xfs_attr_leaf.c | 7 +-
fs/xfs/libxfs/xfs_da_format.h | 5 +
fs/xfs/scrub/attr.c | 34 ++++--
fs/xfs/scrub/attr_repair.c | 4 +-
fs/xfs/xfs_attr_item.c | 242 +++++++++++++++++++++++++++++++++---------
fs/xfs/xfs_attr_list.c | 18 +++-
fs/xfs/xfs_mount.c | 16 +++
fs/xfs/xfs_mount.h | 6 +-
fs/xfs/xfs_xattr.c | 3 +-
11 files changed, 304 insertions(+), 77 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 3/9] xfs: Parent Pointers
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
2024-04-24 3:33 ` [GIT PULL 1/9] xfs: shrink struct xfs_da_args Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 2/9] xfs: improve extended attribute validation Darrick J. Wong
@ 2024-04-24 3:34 ` Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 4/9] xfs: scrubbing for parent pointers Darrick J. Wong
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:34 UTC (permalink / raw
To: chandanbabu, djwong
Cc: allison.henderson, catherine.hoang, darrick.wong, dchinner, hch,
linux-xfs, mark.tinguely
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit ea0b3e814741fb64e7785b564ea619578058e0b0:
xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/pptrs-6.10_2024-04-23
for you to fetch changes up to 67ac7091e35bd34b75c0ec77331b53ca052e0cb3:
xfs: enable parent pointers (2024-04-23 07:47:01 -0700)
----------------------------------------------------------------
xfs: Parent Pointers [v13.4 3/9]
This is the latest parent pointer attributes for xfs. The goal of this
patch set is to add a parent pointer attribute to each inode. The
attribute name containing the parent inode, generation, and directory
offset, while the attribute value contains the file name. This feature
will enable future optimizations for online scrub, shrink, nfs handles,
verity, or any other feature that could make use of quickly deriving an
inodes path from the mount point.
Directory parent pointers are stored as namespaced extended attributes
of a file. Because parent pointers are an indivisible tuple of
(dirent_name, parent_ino, parent_gen) we cannot use the usual attr name
lookup functions to find a parent pointer. This is solvable by
introducing a new lookup mode that checks both the name and the value of
the xattr.
Therefore, introduce this new name-value lookup mode that's gated on the
XFS_ATTR_PARENT namespace. This requires the introduction of new
opcodes for the extended attribute update log intent items, which
actually means that parent pointers (itself an INCOMPAT feature) does
not depend on the LOGGED_XATTRS log incompat feature bit.
To reduce collisions on the dirent names of parent pointers, introduce a
new attr hash mode that is the dir2 namehash of the dirent name xor'd
with the parent inode number.
At this point, Allison has moved on to other things, so I've merged her
patchset into djwong-dev for merging.
Updates since v12 [djwong]:
Rebase on 6.9-rc and update the online fsck design document.
Redesign the ondisk format to use the name-value lookups to get us back
to the point where the attr is (dirent_name -> parent_ino/gen).
Updates since v11 [djwong]:
Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair
prototypes. Merge with djwong-dev and make online repair actually work.
Updates since v10 [djwong]:
Merge in the ondisk format changes to get rid of the diroffset conflicts
with the parent pointer repair code, rebase the entire series with the
attr vlookup changes first, and merge all the other random fixes.
Updates since v9:
Reordered patches 2 and 3 to be 6 and 7
xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent
xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc
reworked ioctl to use pptr listenty and flex array
NEW
xfs: don't remove the attr fork when parent pointers are enabled
NEW
directory lookups should return diroffsets too
NEW
xfs: move/add parent pointer validators to xfs_parent
Updates since v8:
xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail
Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers
removed unused xfs_parent_name_irec and xfs_init_parent_name_irec
xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas
xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates
xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free
xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once
Fix xfs_rename to disallow reservationless quotas
Fix double unlock on dqattach fail
Move parent frees to out_release_wip
xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename
Questions comments and feedback appreciated!
Thanks all!
Allison
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Allison Henderson (15):
xfs: add parent pointer support to attribute code
xfs: define parent pointer ondisk extended attribute format
xfs: Expose init_xattrs in xfs_create_tmpfile
xfs: add parent pointer validator functions
xfs: extend transaction reservations for parent attributes
xfs: parent pointer attribute creation
xfs: add parent attributes to link
xfs: add parent attributes to symlink
xfs: remove parent pointers in unlink
xfs: Add parent pointers to rename
xfs: Add parent pointers to xfs_cross_rename
xfs: don't return XFS_ATTR_PARENT attributes via listxattr
xfs: pass the attr value to put_listent when possible
xfs: don't remove the attr fork when parent pointers are enabled
xfs: add a incompat feature bit for parent pointers
Christoph Hellwig (1):
xfs: check the flags earlier in xfs_attr_match
Darrick J. Wong (14):
xfs: rearrange xfs_attr_match parameters
xfs: move xfs_attr_defer_add to xfs_attr_item.c
xfs: create a separate hashname function for extended attributes
xfs: allow xattr matching on name and value for parent pointers
xfs: refactor xfs_is_using_logged_xattrs checks in attr item recovery
xfs: create attr log item opcodes and formats for parent pointers
xfs: record inode generation in xattr update log intent items
xfs: create a hashname function for parent pointers
xfs: move handle ioctl code to xfs_handle.c
xfs: split out handle management helpers a bit
xfs: add parent pointer ioctls
xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
xfs: drop compatibility minimum log size computations for reflink
xfs: enable parent pointers
fs/xfs/Makefile | 3 +
fs/xfs/libxfs/xfs_attr.c | 92 ++--
fs/xfs/libxfs/xfs_attr.h | 23 +-
fs/xfs/libxfs/xfs_attr_leaf.c | 87 +++-
fs/xfs/libxfs/xfs_attr_sf.h | 1 +
fs/xfs/libxfs/xfs_da_btree.h | 4 +
fs/xfs/libxfs/xfs_da_format.h | 25 +-
fs/xfs/libxfs/xfs_format.h | 4 +-
fs/xfs/libxfs/xfs_fs.h | 79 +++-
fs/xfs/libxfs/xfs_log_format.h | 25 +-
fs/xfs/libxfs/xfs_log_rlimit.c | 46 ++
fs/xfs/libxfs/xfs_ondisk.h | 6 +
fs/xfs/libxfs/xfs_parent.c | 293 +++++++++++++
fs/xfs/libxfs/xfs_parent.h | 99 +++++
fs/xfs/libxfs/xfs_sb.c | 4 +
fs/xfs/libxfs/xfs_trans_resv.c | 326 +++++++++++---
fs/xfs/libxfs/xfs_trans_space.c | 121 +++++
fs/xfs/libxfs/xfs_trans_space.h | 25 +-
fs/xfs/scrub/attr.c | 15 +-
fs/xfs/scrub/dir_repair.c | 2 +-
fs/xfs/scrub/orphanage.c | 5 +-
fs/xfs/scrub/parent_repair.c | 3 +-
fs/xfs/scrub/symlink_repair.c | 2 +-
fs/xfs/scrub/tempfile.c | 2 +-
fs/xfs/xfs_attr_item.c | 331 ++++++++++++--
fs/xfs/xfs_attr_item.h | 10 +
fs/xfs/xfs_attr_list.c | 13 +-
fs/xfs/xfs_export.c | 2 +-
fs/xfs/xfs_export.h | 2 +
fs/xfs/xfs_handle.c | 952 ++++++++++++++++++++++++++++++++++++++++
fs/xfs/xfs_handle.h | 33 ++
fs/xfs/xfs_inode.c | 218 +++++++--
fs/xfs/xfs_inode.h | 2 +-
fs/xfs/xfs_ioctl.c | 595 +------------------------
fs/xfs/xfs_ioctl.h | 28 --
fs/xfs/xfs_ioctl32.c | 1 +
fs/xfs/xfs_iops.c | 15 +-
fs/xfs/xfs_super.c | 14 +
fs/xfs/xfs_symlink.c | 30 +-
fs/xfs/xfs_trace.c | 1 +
fs/xfs/xfs_trace.h | 95 +++-
fs/xfs/xfs_xattr.c | 5 +
42 files changed, 2807 insertions(+), 832 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_parent.c
create mode 100644 fs/xfs/libxfs/xfs_parent.h
create mode 100644 fs/xfs/libxfs/xfs_trans_space.c
create mode 100644 fs/xfs/xfs_handle.c
create mode 100644 fs/xfs/xfs_handle.h
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 4/9] xfs: scrubbing for parent pointers
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (2 preceding siblings ...)
2024-04-24 3:34 ` [GIT PULL 3/9] xfs: Parent Pointers Darrick J. Wong
@ 2024-04-24 3:34 ` Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 5/9] xfs: online repair " Darrick J. Wong
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:34 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: allison.henderson, catherine.hoang, hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 67ac7091e35bd34b75c0ec77331b53ca052e0cb3:
xfs: enable parent pointers (2024-04-23 07:47:01 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-pptrs-6.10_2024-04-23
for you to fetch changes up to 59a2af9086f0d60fc8de7346da67db7d764c7221:
xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700)
----------------------------------------------------------------
xfs: scrubbing for parent pointers [v13.4 4/9]
Teach online fsck to use parent pointers to assist in checking
directories, parent pointers, extended attributes, and link counts.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (7):
xfs: revert commit 44af6c7e59b12
xfs: check dirents have parent pointers
xfs: deferred scrub of dirents
xfs: scrub parent pointers
xfs: deferred scrub of parent pointers
xfs: walk directory parent pointers to determine backref count
xfs: check parent pointer xattrs when scrubbing
fs/xfs/Makefile | 2 +-
fs/xfs/libxfs/xfs_parent.c | 22 ++
fs/xfs/libxfs/xfs_parent.h | 5 +
fs/xfs/scrub/attr.c | 27 +-
fs/xfs/scrub/common.h | 1 +
fs/xfs/scrub/dir.c | 342 ++++++++++++++++++++-
fs/xfs/scrub/nlinks.c | 85 +++++-
fs/xfs/scrub/nlinks_repair.c | 2 +
fs/xfs/scrub/parent.c | 685 +++++++++++++++++++++++++++++++++++++++++++
fs/xfs/scrub/readdir.c | 78 +++++
fs/xfs/scrub/readdir.h | 3 +
fs/xfs/scrub/trace.c | 1 +
fs/xfs/scrub/trace.h | 65 ++++
13 files changed, 1307 insertions(+), 11 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 5/9] xfs: online repair for parent pointers
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (3 preceding siblings ...)
2024-04-24 3:34 ` [GIT PULL 4/9] xfs: scrubbing for parent pointers Darrick J. Wong
@ 2024-04-24 3:34 ` Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 6/9] xfs: detect and correct directory tree problems Darrick J. Wong
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:34 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: allison.henderson, catherine.hoang, hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 59a2af9086f0d60fc8de7346da67db7d764c7221:
xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-pptrs-6.10_2024-04-23
for you to fetch changes up to 327ed702d84034879572812f580cb769848af7ae:
xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700)
----------------------------------------------------------------
xfs: online repair for parent pointers [v13.4 5/9]
This series implements online repair for directory parent pointer
metadata. The checking half is fairly straightforward -- for each
outgoing directory link (forward or backwards), grab the inode at the
other end, and confirm that there's a corresponding link. If we can't
grab an inode or lock it, we'll save that link for a slower loop that
cycles all the locks, confirms the continued existence of the link, and
rechecks the link if it's actually still there.
Repairs are a bit more involved -- for directories, we walk the entire
filesystem to rebuild the dirents from parent pointer information.
Parent pointer repairs do the same walk but rebuild the pptrs from the
dirent information, but with the added twist that it duplicates all the
xattrs so that it can use the atomic extent swapping code to commit the
repairs atomically.
This introduces an added twist to the xattr repair code -- we use dirent
hooks to detect a colliding update to the pptr data while we're not
holding the ILOCKs; if one is detected, we restart the xattr salvaging
process but this time hold all the ILOCKs until the end of the scan.
For offline repair, the phase6 directory connectivity scan generates an
index of all the expected parent pointers in the filesystem. Then it
walks each file and compares the parent pointers attached to that file
against the index generated, and resyncs the results as necessary.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (16):
xfs: remove some boilerplate from xfs_attr_set
xfs: make the reserved block permission flag explicit in xfs_attr_set
xfs: salvage parent pointers when rebuilding xattr structures
xfs: add raw parent pointer apis to support repair
xfs: repair directories by scanning directory parent pointers
xfs: implement live updates for directory repairs
xfs: replay unlocked parent pointer updates that accrue during xattr repair
xfs: repair directory parent pointers by scanning for dirents
xfs: implement live updates for parent pointer repairs
xfs: remove pointless unlocked assertion
xfs: split xfs_bmap_add_attrfork into two pieces
xfs: add a per-leaf block callback to xchk_xattr_walk
xfs: actually rebuild the parent pointer xattrs
xfs: adapt the orphanage code to handle parent pointers
xfs: repair link count of nondirectories after rebuilding parent pointers
xfs: inode repair should ensure there's an attr fork to store parent pointers
fs/xfs/libxfs/xfs_attr.c | 76 ++-
fs/xfs/libxfs/xfs_attr.h | 3 +-
fs/xfs/libxfs/xfs_bmap.c | 38 +-
fs/xfs/libxfs/xfs_bmap.h | 3 +-
fs/xfs/libxfs/xfs_dir2.c | 2 +-
fs/xfs/libxfs/xfs_dir2.h | 2 +-
fs/xfs/libxfs/xfs_parent.c | 64 +++
fs/xfs/libxfs/xfs_parent.h | 6 +
fs/xfs/scrub/attr.c | 2 +-
fs/xfs/scrub/attr_repair.c | 484 +++++++++++++++-
fs/xfs/scrub/attr_repair.h | 4 +
fs/xfs/scrub/dir_repair.c | 567 +++++++++++++++++-
fs/xfs/scrub/findparent.c | 12 +-
fs/xfs/scrub/findparent.h | 10 +-
fs/xfs/scrub/inode_repair.c | 41 ++
fs/xfs/scrub/listxattr.c | 10 +-
fs/xfs/scrub/listxattr.h | 4 +-
fs/xfs/scrub/nlinks.c | 3 +-
fs/xfs/scrub/orphanage.c | 38 ++
fs/xfs/scrub/orphanage.h | 3 +
fs/xfs/scrub/parent.c | 7 +-
fs/xfs/scrub/parent_repair.c | 1307 +++++++++++++++++++++++++++++++++++++++++-
fs/xfs/scrub/scrub.c | 2 +
fs/xfs/scrub/trace.h | 153 +++++
fs/xfs/xfs_xattr.c | 26 +-
25 files changed, 2743 insertions(+), 124 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 6/9] xfs: detect and correct directory tree problems
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (4 preceding siblings ...)
2024-04-24 3:34 ` [GIT PULL 5/9] xfs: online repair " Darrick J. Wong
@ 2024-04-24 3:35 ` Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 7/9] xfs: vectorize scrub kernel calls Darrick J. Wong
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:35 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 327ed702d84034879572812f580cb769848af7ae:
xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-directory-tree-6.10_2024-04-23
for you to fetch changes up to 3f31406aef493b3f19020909d29974e28253f91c:
xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700)
----------------------------------------------------------------
xfs: detect and correct directory tree problems [v13.4 6/9]
Historically, checking the tree-ness of the directory tree structure has
not been complete. Cycles of subdirectories break the tree properties,
as do subdirectories with multiple parents. It's easy enough for DFS to
detect problems as long as one of the participants is reachable from the
root, but this technique cannot find unconnected cycles.
Directory parent pointers change that, because we can discover all of
these problems from a simple walk from a subdirectory towards the root.
For each child we start with, if the walk terminates without reaching
the root, we know the path is disconnected and ought to be attached to
the lost and found. If we find ourselves, we know this is a cycle and
can delete an incoming edge. If we find multiple paths to the root, we
know to delete an incoming edge.
Even better, once we've finished walking paths, we've identified the
good ones and know which other path(s) to remove.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (4):
xfs: teach online scrub to find directory tree structure problems
xfs: invalidate dirloop scrub path data when concurrent updates happen
xfs: report directory tree corruption in the health information
xfs: fix corruptions in the directory tree
fs/xfs/Makefile | 2 +
fs/xfs/libxfs/xfs_fs.h | 4 +-
fs/xfs/libxfs/xfs_health.h | 4 +-
fs/xfs/scrub/common.h | 1 +
fs/xfs/scrub/dirtree.c | 985 ++++++++++++++++++++++++++++++++++++++++++
fs/xfs/scrub/dirtree.h | 178 ++++++++
fs/xfs/scrub/dirtree_repair.c | 821 +++++++++++++++++++++++++++++++++++
fs/xfs/scrub/health.c | 1 +
fs/xfs/scrub/ino_bitmap.h | 37 ++
fs/xfs/scrub/orphanage.c | 6 +
fs/xfs/scrub/orphanage.h | 8 +
fs/xfs/scrub/repair.h | 4 +
fs/xfs/scrub/scrub.c | 7 +
fs/xfs/scrub/scrub.h | 1 +
fs/xfs/scrub/stats.c | 1 +
fs/xfs/scrub/trace.c | 4 +
fs/xfs/scrub/trace.h | 272 +++++++++++-
fs/xfs/scrub/xfarray.h | 1 +
fs/xfs/xfs_health.c | 1 +
fs/xfs/xfs_inode.c | 2 +-
fs/xfs/xfs_inode.h | 1 +
21 files changed, 2337 insertions(+), 4 deletions(-)
create mode 100644 fs/xfs/scrub/dirtree.c
create mode 100644 fs/xfs/scrub/dirtree.h
create mode 100644 fs/xfs/scrub/dirtree_repair.c
create mode 100644 fs/xfs/scrub/ino_bitmap.h
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 7/9] xfs: vectorize scrub kernel calls
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (5 preceding siblings ...)
2024-04-24 3:35 ` [GIT PULL 6/9] xfs: detect and correct directory tree problems Darrick J. Wong
@ 2024-04-24 3:35 ` Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 8/9] xfs: reduce iget overhead in scrub Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 9/9] xfs: minor fixes to online repair Darrick J. Wong
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:35 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 3f31406aef493b3f19020909d29974e28253f91c:
xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/vectorized-scrub-6.10_2024-04-23
for you to fetch changes up to c77b37584c2d1054452853e47e42c7350b8fe687:
xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700)
----------------------------------------------------------------
xfs: vectorize scrub kernel calls [v13.4 7/9]
Create a vectorized version of the metadata scrub and repair ioctl, and
adapt xfs_scrub to use that. This mitigates the impact of system call
overhead on xfs_scrub runtime.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (3):
xfs: reduce the rate of cond_resched calls inside scrub
xfs: move xfs_ioc_scrub_metadata to scrub.c
xfs: introduce vectored scrub mode
fs/xfs/libxfs/xfs_fs.h | 33 +++++++++
fs/xfs/scrub/common.h | 25 -------
fs/xfs/scrub/scrub.c | 177 ++++++++++++++++++++++++++++++++++++++++++++++-
fs/xfs/scrub/scrub.h | 64 +++++++++++++++++
fs/xfs/scrub/trace.h | 79 ++++++++++++++++++++-
fs/xfs/scrub/xfarray.c | 10 +--
fs/xfs/scrub/xfarray.h | 3 +
fs/xfs/scrub/xfile.c | 2 +-
fs/xfs/scrub/xfs_scrub.h | 6 +-
fs/xfs/xfs_ioctl.c | 26 +------
10 files changed, 366 insertions(+), 59 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 8/9] xfs: reduce iget overhead in scrub
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (6 preceding siblings ...)
2024-04-24 3:35 ` [GIT PULL 7/9] xfs: vectorize scrub kernel calls Darrick J. Wong
@ 2024-04-24 3:35 ` Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 9/9] xfs: minor fixes to online repair Darrick J. Wong
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:35 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit c77b37584c2d1054452853e47e42c7350b8fe687:
xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/reduce-scrub-iget-overhead-6.10_2024-04-23
for you to fetch changes up to 4ad350ac58627bfe81f71f43f6738e36b4eb75c6:
xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700)
----------------------------------------------------------------
xfs: reduce iget overhead in scrub [v13.4 8/9]
This patchset looks to reduce iget overhead in two ways: First, a
previous patch conditionally set DONTCACHE on inodes during xchk_irele
on the grounds that we knew better at irele time if an inode should be
dropped. Unfortunately, over time that patch morphed into a call to
d_mark_dontcache, which resulted in inodes being dropped even if they
were referenced by the dcache. This actually caused *more* recycle
overhead than if we'd simply called xfs_iget to set DONTCACHE only on
misses.
The second patch reduces the cost of untrusted iget for a vectored scrub
call by having the scrubv code maintain a separate refcount to the inode
so that the cache will always hit.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (2):
xfs: use dontcache for grabbing inodes during scrub
xfs: only iget the file once when doing vectored scrub-by-handle
fs/xfs/scrub/common.c | 12 +++---------
fs/xfs/scrub/iscan.c | 13 +++++++++++--
fs/xfs/scrub/scrub.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
fs/xfs/scrub/scrub.h | 7 +++++++
4 files changed, 66 insertions(+), 11 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT PULL 9/9] xfs: minor fixes to online repair
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
` (7 preceding siblings ...)
2024-04-24 3:35 ` [GIT PULL 8/9] xfs: reduce iget overhead in scrub Darrick J. Wong
@ 2024-04-24 3:35 ` Darrick J. Wong
8 siblings, 0 replies; 10+ messages in thread
From: Darrick J. Wong @ 2024-04-24 3:35 UTC (permalink / raw
To: chandanbabu, djwong; +Cc: hch, linux-xfs
Hi Chandan,
Please pull this branch with changes for xfs for 6.10-rc1.
As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts. Please let me know if you
encounter any problems.
--D
The following changes since commit 4ad350ac58627bfe81f71f43f6738e36b4eb75c6:
xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-fixes-6.10_2024-04-23
for you to fetch changes up to 5e1c7d0b29f7e05b01e448d2579a469cf3a0d350:
xfs: invalidate dentries for a file before moving it to the orphanage (2024-04-23 16:55:19 -0700)
----------------------------------------------------------------
xfs: minor fixes to online repair [v13.4 9/9]
Here are some miscellaneous bug fixes for the online repair code.
This has been running on the djcloud for months with no problems. Enjoy!
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
----------------------------------------------------------------
Darrick J. Wong (4):
xfs: drop the scrub file's iolock when transaction allocation fails
xfs: fix iunlock calls in xrep_adoption_trans_alloc
xfs: exchange-range for repairs is no longer dynamic
xfs: invalidate dentries for a file before moving it to the orphanage
fs/xfs/scrub/attr_repair.c | 3 +++
fs/xfs/scrub/dir_repair.c | 3 +++
fs/xfs/scrub/nlinks_repair.c | 4 +++-
fs/xfs/scrub/orphanage.c | 49 ++++++++++++++++++-----------------------
fs/xfs/scrub/parent_repair.c | 10 ++++++---
fs/xfs/scrub/rtsummary_repair.c | 10 ++++-----
fs/xfs/scrub/scrub.c | 8 +++----
fs/xfs/scrub/scrub.h | 7 ------
fs/xfs/scrub/symlink_repair.c | 3 +++
fs/xfs/scrub/tempexch.h | 1 -
fs/xfs/scrub/tempfile.c | 24 ++------------------
fs/xfs/scrub/trace.h | 3 ---
12 files changed, 49 insertions(+), 76 deletions(-)
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2024-04-24 3:36 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-24 3:30 [GIT PULLBOMB v13.4] xfs: directory parent pointers Darrick J. Wong
2024-04-24 3:33 ` [GIT PULL 1/9] xfs: shrink struct xfs_da_args Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 2/9] xfs: improve extended attribute validation Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 3/9] xfs: Parent Pointers Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 4/9] xfs: scrubbing for parent pointers Darrick J. Wong
2024-04-24 3:34 ` [GIT PULL 5/9] xfs: online repair " Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 6/9] xfs: detect and correct directory tree problems Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 7/9] xfs: vectorize scrub kernel calls Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 8/9] xfs: reduce iget overhead in scrub Darrick J. Wong
2024-04-24 3:35 ` [GIT PULL 9/9] xfs: minor fixes to online repair Darrick J. Wong
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.