From: "Darrick J. Wong" <djwong@kernel.org>
To: Christoph Hellwig <hch@infradead.org>
Cc: linux-xfs@vger.kernel.org
Subject: [PATCHBOMB v13.1] xfs: directory parent pointers
Date: Tue, 9 Apr 2024 17:36:46 -0700 [thread overview]
Message-ID: <20240410003646.GS6390@frogsfrogsfrogs> (raw)
Hi everyone,
Christoph and I have been working to get the directory parent pointers
patchset into shape for merging in the next kernel cycle. This v13
release contains what I hope are the last ondisk format changes -- we've
gone back to parent pointers being xattrs attached to the child, wherein
the attr name is the dirent name, and the attr value is a handle to the
parent directory.
We've solved the pptr lookup uniqueness problem by forcing all
XFS_ATTR_PARENT attr lookups to be done on the name and value; avoided
namehash collisions on container farms by adjusting the hash function
slightly; and avoided the entire log incompat feature flag mess by
defining a permanent incompat feature for parent pointers and using
totally separate attr log item opcodes.
With that, I think this is finally ready to go.
Full versions are here:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub
--D
next reply other threads:[~2024-04-10 0:36 UTC|newest]
Thread overview: 234+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-10 0:36 Darrick J. Wong [this message]
2024-04-10 0:44 ` [PATCHSET v13.1 1/9] xfs: design documentation for online fsck, part 2 Darrick J. Wong
2024-04-10 0:46 ` [PATCH 1/4] docs: update the parent pointers documentation to the final version Darrick J. Wong
2024-04-10 4:40 ` Christoph Hellwig
2024-04-10 0:46 ` [PATCH 2/4] docs: update online directory and parent pointer repair sections Darrick J. Wong
2024-04-10 4:40 ` Christoph Hellwig
2024-04-10 0:47 ` [PATCH 3/4] docs: update offline parent pointer repair strategy Darrick J. Wong
2024-04-10 4:40 ` Christoph Hellwig
2024-04-10 0:47 ` [PATCH 4/4] docs: describe xfs directory tree online fsck Darrick J. Wong
2024-04-10 4:40 ` Christoph Hellwig
2024-04-10 0:44 ` [PATCHSET v13.1 2/9] xfs: retain ILOCK during directory updates Darrick J. Wong
2024-04-10 0:47 ` [PATCH 1/7] xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 Darrick J. Wong
2024-04-10 4:41 ` Christoph Hellwig
2024-04-10 0:48 ` [PATCH 2/7] xfs: Increase XFS_QM_TRANS_MAXDQS " Darrick J. Wong
2024-04-10 4:41 ` Christoph Hellwig
2024-04-10 0:48 ` [PATCH 3/7] xfs: Hold inode locks in xfs_ialloc Darrick J. Wong
2024-04-10 4:41 ` Christoph Hellwig
2024-04-10 0:48 ` [PATCH 4/7] xfs: Hold inode locks in xfs_trans_alloc_dir Darrick J. Wong
2024-04-10 4:41 ` Christoph Hellwig
2024-04-10 0:48 ` [PATCH 5/7] xfs: Hold inode locks in xfs_rename Darrick J. Wong
2024-04-10 4:42 ` Christoph Hellwig
2024-04-10 0:49 ` [PATCH 6/7] xfs: don't pick up IOLOCK during rmapbt repair scan Darrick J. Wong
2024-04-10 4:42 ` Christoph Hellwig
2024-04-10 0:49 ` [PATCH 7/7] xfs: unlock new repair tempfiles after creation Darrick J. Wong
2024-04-10 4:42 ` Christoph Hellwig
2024-04-10 0:44 ` [PATCHSET v13.1 3/9] xfs: shrink struct xfs_da_args Darrick J. Wong
2024-04-10 0:49 ` [PATCH 1/4] xfs: remove XFS_DA_OP_REMOVE Darrick J. Wong
2024-04-10 4:43 ` Christoph Hellwig
2024-04-10 0:49 ` [PATCH 2/4] xfs: remove XFS_DA_OP_NOTIME Darrick J. Wong
2024-04-10 4:44 ` Christoph Hellwig
2024-04-10 0:50 ` [PATCH 3/4] xfs: rename xfs_da_args.attr_flags Darrick J. Wong
2024-04-10 5:01 ` Christoph Hellwig
2024-04-10 20:55 ` Darrick J. Wong
2024-04-11 0:00 ` Darrick J. Wong
2024-04-11 3:26 ` Christoph Hellwig
2024-04-11 4:15 ` Darrick J. Wong
2024-04-10 0:50 ` [PATCH 4/4] xfs: rearrange xfs_da_args a bit to use less space Darrick J. Wong
2024-04-10 5:02 ` Christoph Hellwig
2024-04-10 20:56 ` Darrick J. Wong
2024-04-10 0:45 ` [PATCHSET v13.1 4/9] xfs: improve extended attribute validation Darrick J. Wong
2024-04-10 0:50 ` [PATCH 01/12] xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf Darrick J. Wong
2024-04-10 5:04 ` Christoph Hellwig
2024-04-10 20:58 ` Darrick J. Wong
2024-04-10 0:50 ` [PATCH 02/12] xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery Darrick J. Wong
2024-04-10 5:04 ` Christoph Hellwig
2024-04-10 0:51 ` [PATCH 03/12] xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available Darrick J. Wong
2024-04-10 5:05 ` Christoph Hellwig
2024-04-10 0:51 ` [PATCH 04/12] xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2 Darrick J. Wong
2024-04-10 5:05 ` Christoph Hellwig
2024-04-10 0:51 ` [PATCH 05/12] xfs: fix missing check for invalid attr flags Darrick J. Wong
2024-04-10 5:07 ` Christoph Hellwig
2024-04-10 21:04 ` Darrick J. Wong
2024-04-10 0:51 ` [PATCH 06/12] xfs: restructure xfs_attr_complete_op a bit Darrick J. Wong
2024-04-10 5:07 ` Christoph Hellwig
2024-04-10 0:52 ` [PATCH 07/12] xfs: use helpers to extract xattr op from opflags Darrick J. Wong
2024-04-10 5:07 ` Christoph Hellwig
2024-04-10 0:52 ` [PATCH 08/12] xfs: validate recovered name buffers when recovering xattr items Darrick J. Wong
2024-04-10 5:08 ` Christoph Hellwig
2024-04-10 0:52 ` [PATCH 09/12] xfs: always set args->value in xfs_attri_item_recover Darrick J. Wong
2024-04-10 5:08 ` Christoph Hellwig
2024-04-10 0:52 ` [PATCH 10/12] xfs: use local variables for name and value length in _attri_commit_pass2 Darrick J. Wong
2024-04-10 5:08 ` Christoph Hellwig
2024-04-10 0:53 ` [PATCH 11/12] xfs: refactor name/length checks in xfs_attri_validate Darrick J. Wong
2024-04-10 5:09 ` Christoph Hellwig
2024-04-10 0:53 ` [PATCH 12/12] xfs: enforce one namespace per attribute Darrick J. Wong
2024-04-10 5:09 ` Christoph Hellwig
2024-04-10 0:45 ` [PATCHSET v13.1 5/9] xfs: Parent Pointers Darrick J. Wong
2024-04-10 0:53 ` [PATCH 01/32] xfs: rearrange xfs_attr_match parameters Darrick J. Wong
2024-04-10 5:10 ` Christoph Hellwig
2024-04-10 0:54 ` [PATCH 02/32] xfs: check the flags earlier in xfs_attr_match Darrick J. Wong
2024-04-10 0:54 ` [PATCH 03/32] xfs: move xfs_attr_defer_add to xfs_attr_item.c Darrick J. Wong
2024-04-10 5:11 ` Christoph Hellwig
2024-04-10 0:54 ` [PATCH 04/32] xfs: create a separate hashname function for extended attributes Darrick J. Wong
2024-04-10 5:11 ` Christoph Hellwig
2024-04-10 0:54 ` [PATCH 05/32] xfs: add parent pointer support to attribute code Darrick J. Wong
2024-04-10 5:11 ` Christoph Hellwig
2024-04-10 0:55 ` [PATCH 06/32] xfs: define parent pointer ondisk extended attribute format Darrick J. Wong
2024-04-10 5:12 ` Christoph Hellwig
2024-04-10 0:55 ` [PATCH 07/32] xfs: allow xattr matching on name and value for local/sf attrs Darrick J. Wong
2024-04-10 5:16 ` Christoph Hellwig
2024-04-10 21:13 ` Darrick J. Wong
2024-04-11 3:28 ` Christoph Hellwig
2024-04-10 0:55 ` [PATCH 08/32] xfs: allow logged xattr operations if parent pointers are enabled Darrick J. Wong
2024-04-10 5:18 ` Christoph Hellwig
2024-04-10 21:18 ` Darrick J. Wong
2024-04-10 0:55 ` [PATCH 09/32] xfs: log parent pointer xattr removal operations Darrick J. Wong
2024-04-10 5:18 ` Christoph Hellwig
2024-04-10 0:56 ` [PATCH 10/32] xfs: log parent pointer xattr setting operations Darrick J. Wong
2024-04-10 0:56 ` [PATCH 11/32] xfs: log parent pointer xattr replace operations Darrick J. Wong
2024-04-10 5:26 ` Christoph Hellwig
2024-04-10 23:07 ` Darrick J. Wong
2024-04-11 3:35 ` Christoph Hellwig
2024-04-10 0:56 ` [PATCH 12/32] xfs: record inode generation in xattr update log intent items Darrick J. Wong
2024-04-10 5:27 ` Christoph Hellwig
2024-04-10 0:56 ` [PATCH 13/32] xfs: Expose init_xattrs in xfs_create_tmpfile Darrick J. Wong
2024-04-10 5:28 ` Christoph Hellwig
2024-04-10 0:57 ` [PATCH 14/32] xfs: add parent pointer validator functions Darrick J. Wong
2024-04-10 5:31 ` Christoph Hellwig
2024-04-10 18:53 ` Darrick J. Wong
2024-04-11 3:25 ` Christoph Hellwig
2024-04-10 0:57 ` [PATCH 15/32] xfs: extend transaction reservations for parent attributes Darrick J. Wong
2024-04-10 5:31 ` Christoph Hellwig
2024-04-10 0:57 ` [PATCH 16/32] xfs: create a hashname function for parent pointers Darrick J. Wong
2024-04-10 5:33 ` Christoph Hellwig
2024-04-10 21:39 ` Darrick J. Wong
2024-04-10 0:57 ` [PATCH 17/32] xfs: parent pointer attribute creation Darrick J. Wong
2024-04-10 5:44 ` Christoph Hellwig
2024-04-10 21:50 ` Darrick J. Wong
2024-04-10 0:58 ` [PATCH 18/32] xfs: add parent attributes to link Darrick J. Wong
2024-04-10 5:45 ` Christoph Hellwig
2024-04-10 0:58 ` [PATCH 19/32] xfs: add parent attributes to symlink Darrick J. Wong
2024-04-10 5:45 ` Christoph Hellwig
2024-04-10 0:58 ` [PATCH 20/32] xfs: remove parent pointers in unlink Darrick J. Wong
2024-04-10 5:45 ` Christoph Hellwig
2024-04-10 0:58 ` [PATCH 21/32] xfs: Add parent pointers to rename Darrick J. Wong
2024-04-10 5:46 ` Christoph Hellwig
2024-04-10 0:59 ` [PATCH 22/32] xfs: Add parent pointers to xfs_cross_rename Darrick J. Wong
2024-04-10 5:46 ` Christoph Hellwig
2024-04-10 0:59 ` [PATCH 23/32] xfs: Filter XFS_ATTR_PARENT for getfattr Darrick J. Wong
2024-04-10 5:51 ` Christoph Hellwig
2024-04-10 21:58 ` Darrick J. Wong
2024-04-11 3:29 ` Christoph Hellwig
2024-04-10 0:59 ` [PATCH 24/32] xfs: pass the attr value to put_listent when possible Darrick J. Wong
2024-04-10 5:51 ` Christoph Hellwig
2024-04-10 1:00 ` [PATCH 25/32] xfs: move handle ioctl code to xfs_handle.c Darrick J. Wong
2024-04-10 5:52 ` Christoph Hellwig
2024-04-10 1:00 ` [PATCH 26/32] xfs: split out handle management helpers a bit Darrick J. Wong
2024-04-10 5:56 ` Christoph Hellwig
2024-04-10 22:01 ` Darrick J. Wong
2024-04-10 1:00 ` [PATCH 27/32] xfs: Add parent pointer ioctls Darrick J. Wong
2024-04-10 6:04 ` Christoph Hellwig
2024-04-10 23:34 ` Darrick J. Wong
2024-04-12 17:39 ` Darrick J. Wong
2024-04-14 5:18 ` Christoph Hellwig
2024-04-15 19:40 ` Darrick J. Wong
2024-04-16 4:47 ` Christoph Hellwig
2024-04-16 16:50 ` Darrick J. Wong
2024-04-16 16:54 ` Christoph Hellwig
2024-04-16 18:52 ` Darrick J. Wong
2024-04-16 19:01 ` Christoph Hellwig
2024-04-16 19:07 ` Darrick J. Wong
2024-04-16 19:14 ` Christoph Hellwig
2024-04-17 5:22 ` Darrick J. Wong
2024-04-17 5:29 ` Christoph Hellwig
2024-04-17 5:55 ` Darrick J. Wong
2024-04-10 1:00 ` [PATCH 28/32] xfs: don't remove the attr fork when parent pointers are enabled Darrick J. Wong
2024-04-10 6:04 ` Christoph Hellwig
2024-04-10 1:01 ` [PATCH 29/32] xfs: Add the parent pointer support to the superblock version 5 Darrick J. Wong
2024-04-10 6:05 ` Christoph Hellwig
2024-04-10 22:06 ` Darrick J. Wong
2024-04-10 1:01 ` [PATCH 30/32] xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res Darrick J. Wong
2024-04-10 6:05 ` Christoph Hellwig
2024-04-10 1:01 ` [PATCH 31/32] xfs: drop compatibility minimum log size computations for reflink Darrick J. Wong
2024-04-10 6:06 ` Christoph Hellwig
2024-04-10 1:01 ` [PATCH 32/32] xfs: enable parent pointers Darrick J. Wong
2024-04-10 6:06 ` Christoph Hellwig
2024-04-10 22:11 ` Darrick J. Wong
2024-04-10 0:45 ` [PATCHSET v13.1 6/9] xfs: scrubbing for " Darrick J. Wong
2024-04-10 1:02 ` [PATCH 1/7] xfs: check dirents have " Darrick J. Wong
2024-04-10 6:12 ` Christoph Hellwig
2024-04-10 1:02 ` [PATCH 2/7] xfs: deferred scrub of dirents Darrick J. Wong
2024-04-10 6:13 ` Christoph Hellwig
2024-04-10 1:02 ` [PATCH 3/7] xfs: scrub parent pointers Darrick J. Wong
2024-04-10 6:13 ` Christoph Hellwig
2024-04-10 1:02 ` [PATCH 4/7] xfs: deferred scrub of " Darrick J. Wong
2024-04-10 6:14 ` Christoph Hellwig
2024-04-10 1:03 ` [PATCH 5/7] xfs: walk directory parent pointers to determine backref count Darrick J. Wong
2024-04-10 6:14 ` Christoph Hellwig
2024-04-10 1:03 ` [PATCH 6/7] xfs: check parent pointer xattrs when scrubbing Darrick J. Wong
2024-04-10 6:14 ` Christoph Hellwig
2024-04-10 1:03 ` [PATCH 7/7] xfs: salvage parent pointers when rebuilding xattr structures Darrick J. Wong
2024-04-10 6:15 ` Christoph Hellwig
2024-04-10 0:45 ` [PATCHSET v13.1 7/9] xfs: online repair for parent pointers Darrick J. Wong
2024-04-10 1:03 ` [PATCH 01/14] xfs: add xattr setname and removename functions for internal users Darrick J. Wong
2024-04-10 6:18 ` Christoph Hellwig
2024-04-10 22:18 ` Darrick J. Wong
2024-04-11 3:32 ` Christoph Hellwig
2024-04-11 4:30 ` Darrick J. Wong
2024-04-11 4:50 ` Christoph Hellwig
2024-04-10 1:04 ` [PATCH 02/14] xfs: add raw parent pointer apis to support repair Darrick J. Wong
2024-04-10 6:18 ` Christoph Hellwig
2024-04-10 1:04 ` [PATCH 03/14] xfs: repair directories by scanning directory parent pointers Darrick J. Wong
2024-04-10 6:19 ` Christoph Hellwig
2024-04-10 1:04 ` [PATCH 04/14] xfs: implement live updates for directory repairs Darrick J. Wong
2024-04-10 6:19 ` Christoph Hellwig
2024-04-10 1:04 ` [PATCH 05/14] xfs: replay unlocked parent pointer updates that accrue during xattr repair Darrick J. Wong
2024-04-10 6:19 ` Christoph Hellwig
2024-04-10 1:05 ` [PATCH 06/14] xfs: repair directory parent pointers by scanning for dirents Darrick J. Wong
2024-04-10 6:20 ` Christoph Hellwig
2024-04-10 1:05 ` [PATCH 07/14] xfs: implement live updates for parent pointer repairs Darrick J. Wong
2024-04-10 6:20 ` Christoph Hellwig
2024-04-10 1:05 ` [PATCH 08/14] xfs: remove pointless unlocked assertion Darrick J. Wong
2024-04-10 6:20 ` Christoph Hellwig
2024-04-10 1:06 ` [PATCH 09/14] xfs: split xfs_bmap_add_attrfork into two pieces Darrick J. Wong
2024-04-10 6:21 ` Christoph Hellwig
2024-04-10 1:06 ` [PATCH 10/14] xfs: add a per-leaf block callback to xchk_xattr_walk Darrick J. Wong
2024-04-10 6:22 ` Christoph Hellwig
2024-04-10 1:06 ` [PATCH 11/14] xfs: actually rebuild the parent pointer xattrs Darrick J. Wong
2024-04-10 6:22 ` Christoph Hellwig
2024-04-10 1:06 ` [PATCH 12/14] xfs: adapt the orphanage code to handle parent pointers Darrick J. Wong
2024-04-10 6:23 ` Christoph Hellwig
2024-04-10 1:07 ` [PATCH 13/14] xfs: repair link count of nondirectories after rebuilding " Darrick J. Wong
2024-04-10 6:22 ` Christoph Hellwig
2024-04-10 1:07 ` [PATCH 14/14] xfs: inode repair should ensure there's an attr fork to store " Darrick J. Wong
2024-04-10 6:24 ` Christoph Hellwig
2024-04-10 0:46 ` [PATCHSET v13.1 8/9] xfs: detect and correct directory tree problems Darrick J. Wong
2024-04-10 1:07 ` [PATCH 1/4] xfs: teach online scrub to find directory tree structure problems Darrick J. Wong
2024-04-10 7:21 ` Christoph Hellwig
2024-04-10 1:07 ` [PATCH 2/4] xfs: invalidate dirloop scrub path data when concurrent updates happen Darrick J. Wong
2024-04-10 7:21 ` Christoph Hellwig
2024-04-10 1:08 ` [PATCH 3/4] xfs: report directory tree corruption in the health information Darrick J. Wong
2024-04-10 7:23 ` Christoph Hellwig
2024-04-10 1:08 ` [PATCH 4/4] xfs: fix corruptions in the directory tree Darrick J. Wong
2024-04-10 7:23 ` Christoph Hellwig
2024-04-10 0:46 ` [PATCHSET v13.1 9/9] xfs: vectorize scrub kernel calls Darrick J. Wong
2024-04-10 1:08 ` [PATCH 1/3] xfs: reduce the rate of cond_resched calls inside scrub Darrick J. Wong
2024-04-10 14:55 ` Christoph Hellwig
2024-04-10 22:19 ` Darrick J. Wong
2024-04-10 1:08 ` [PATCH 2/3] xfs: introduce vectored scrub mode Darrick J. Wong
2024-04-10 15:00 ` Christoph Hellwig
2024-04-11 0:59 ` Darrick J. Wong
2024-04-11 3:38 ` Christoph Hellwig
2024-04-11 4:31 ` Darrick J. Wong
2024-04-10 1:09 ` [PATCH 3/3] xfs: only iget the file once when doing vectored scrub-by-handle Darrick J. Wong
2024-04-10 15:12 ` Christoph Hellwig
2024-04-11 1:15 ` Darrick J. Wong
2024-04-11 3:49 ` Christoph Hellwig
2024-04-11 4:41 ` Darrick J. Wong
2024-04-11 4:52 ` Christoph Hellwig
2024-04-11 4:56 ` Darrick J. Wong
2024-04-11 5:02 ` Christoph Hellwig
2024-04-11 5:21 ` Darrick J. Wong
2024-04-11 14:02 ` Christoph Hellwig
2024-04-12 0:21 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240410003646.GS6390@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=hch@infradead.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).