From: Chandan Babu R <chandanbabu@kernel.org>
To: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Cc: akpm@linux-foundation.org, "Darrick J. Wong" <djwong@kernel.org>,
linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev,
linux-xfs@vger.kernel.org, linux-mm@kvack.org,
dan.j.williams@intel.com, willy@infradead.org, jack@suse.cz,
mcgrof@kernel.org
Subject: Re: [PATCH v15] mm, pmem, xfs: Introduce MF_MEM_PRE_REMOVE for unbind
Date: Mon, 23 Oct 2023 12:10:10 +0530 [thread overview]
Message-ID: <87msw9zvpk.fsf@debian-BULLSEYE-live-builder-AMD64> (raw)
In-Reply-To: <20231020154009.GS3195650@frogsfrogsfrogs>
On Fri, Oct 20, 2023 at 08:40:09 AM -0700, Darrick J. Wong wrote:
> On Fri, Oct 20, 2023 at 03:26:32PM +0530, Chandan Babu R wrote:
>> On Thu, Sep 28, 2023 at 06:32:27 PM +0800, Shiyang Ruan wrote:
>> > ====
>> > Changes since v14:
>> > 1. added/fixed code comments per Dan's comments
>> > ====
>> >
>> > Now, if we suddenly remove a PMEM device(by calling unbind) which
>> > contains FSDAX while programs are still accessing data in this device,
>> > e.g.:
>> > ```
>> > $FSSTRESS_PROG -d $SCRATCH_MNT -n 99999 -p 4 &
>> > # $FSX_PROG -N 1000000 -o 8192 -l 500000 $SCRATCH_MNT/t001 &
>> > echo "pfn1.1" > /sys/bus/nd/drivers/nd_pmem/unbind
>> > ```
>> > it could come into an unacceptable state:
>> > 1. device has gone but mount point still exists, and umount will fail
>> > with "target is busy"
>> > 2. programs will hang and cannot be killed
>> > 3. may crash with NULL pointer dereference
>> >
>> > To fix this, we introduce a MF_MEM_PRE_REMOVE flag to let it know that we
>> > are going to remove the whole device, and make sure all related processes
>> > could be notified so that they could end up gracefully.
>> >
>> > This patch is inspired by Dan's "mm, dax, pmem: Introduce
>> > dev_pagemap_failure()"[1]. With the help of dax_holder and
>> > ->notify_failure() mechanism, the pmem driver is able to ask filesystem
>> > on it to unmap all files in use, and notify processes who are using
>> > those files.
>> >
>> > Call trace:
>> > trigger unbind
>> > -> unbind_store()
>> > -> ... (skip)
>> > -> devres_release_all()
>> > -> kill_dax()
>> > -> dax_holder_notify_failure(dax_dev, 0, U64_MAX, MF_MEM_PRE_REMOVE)
>> > -> xfs_dax_notify_failure()
>> > `-> freeze_super() // freeze (kernel call)
>> > `-> do xfs rmap
>> > ` -> mf_dax_kill_procs()
>> > ` -> collect_procs_fsdax() // all associated processes
>> > ` -> unmap_and_kill()
>> > ` -> invalidate_inode_pages2_range() // drop file's cache
>> > `-> thaw_super() // thaw (both kernel & user call)
>> >
>> > Introduce MF_MEM_PRE_REMOVE to let filesystem know this is a remove
>> > event. Use the exclusive freeze/thaw[2] to lock the filesystem to prevent
>> > new dax mapping from being created. Do not shutdown filesystem directly
>> > if configuration is not supported, or if failure range includes metadata
>> > area. Make sure all files and processes(not only the current progress)
>> > are handled correctly. Also drop the cache of associated files before
>> > pmem is removed.
>> >
>> > [1]: https://lore.kernel.org/linux-mm/161604050314.1463742.14151665140035795571.stgit@dwillia2-desk3.amr.corp.intel.com/
>> > [2]: https://lore.kernel.org/linux-xfs/169116275623.3187159.16862410128731457358.stg-ugh@frogsfrogsfrogs/
>> >
>> > Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com>
>> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>> > Acked-by: Dan Williams <dan.j.williams@intel.com>
>>
>> Hi Andrew,
>>
>> Shiyang had indicated that this patch has been added to
>> akpm/mm-hotfixes-unstable branch. However, I don't see the patch listed in
>> that branch.
>>
>> I am about to start collecting XFS patches for v6.7 cycle. Please let me know
>> if you have any objections with me taking this patch via the XFS tree.
>
> V15 was dropped from his tree on 28 Sept., you might as well pull it
> into your own tree for 6.7. It's been testing fine on my trees for the
> past 3 weeks.
>
> https://lore.kernel.org/mm-commits/20230928172815.EE6AFC433C8@smtp.kernel.org/
Shiyang, this patch does not apply cleanly on v6.6-rc7. Can you please rebase
the patch on v6.6-rc7 and send it to the mailing list?
--
Chandan
next prev parent reply other threads:[~2023-10-23 6:42 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-29 8:16 [PATCH v12 0/2] mm, pmem, xfs: Introduce MF_MEM_REMOVE for unbind Shiyang Ruan
2023-06-29 8:16 ` [PATCH v12 1/2] xfs: fix the calculation for "end" and "length" Shiyang Ruan
2023-06-29 8:16 ` [PATCH v12 2/2] mm, pmem, xfs: Introduce MF_MEM_REMOVE for unbind Shiyang Ruan
2023-06-29 12:02 ` kernel test robot
2023-07-14 9:07 ` Shiyang Ruan
2023-07-14 14:18 ` Darrick J. Wong
2023-07-20 1:50 ` Shiyang Ruan
2023-07-29 10:01 ` Shiyang Ruan
2023-07-29 15:15 ` Darrick J. Wong
2023-07-29 15:15 ` Darrick J. Wong
2023-07-31 9:36 ` Shiyang Ruan
2023-08-01 3:25 ` Darrick J. Wong
2023-08-03 10:44 ` Shiyang Ruan
2023-08-08 0:31 ` Dan Williams
2023-08-23 8:36 ` Shiyang Ruan
2023-08-23 8:17 ` [PATCH v13] mm, pmem, xfs: Introduce MF_MEM_PRE_REMOVE " Shiyang Ruan
2023-08-23 23:36 ` Darrick J. Wong
2023-08-24 9:41 ` Shiyang Ruan
2023-08-24 23:57 ` Darrick J. Wong
2023-08-25 3:52 ` Shiyang Ruan
2023-08-26 0:17 ` Darrick J. Wong
2023-08-28 6:57 ` [PATCH v14] " Shiyang Ruan
2023-08-30 15:34 ` Darrick J. Wong
2023-09-27 8:17 ` Dan Williams
2023-09-27 9:18 ` Shiyang Ruan
2023-09-28 10:32 ` [PATCH v15] " Shiyang Ruan
2023-09-29 18:31 ` Dan Williams
2023-10-01 1:43 ` kernel test robot
2023-10-02 11:57 ` Shiyang Ruan
2023-10-20 9:56 ` Chandan Babu R
2023-10-20 15:40 ` Darrick J. Wong
2023-10-23 6:40 ` Chandan Babu R [this message]
2023-10-23 7:26 ` Shiyang Ruan
2023-10-23 12:21 ` Chandan Babu R
2023-10-23 7:20 ` [PATCH v15.1] " Shiyang Ruan
2024-01-11 22:24 ` [PATCH v12 0/2] mm, pmem, xfs: Introduce MF_MEM_REMOVE " Bill O'Donnell
2024-01-12 1:56 ` Shiyang Ruan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87msw9zvpk.fsf@debian-BULLSEYE-live-builder-AMD64 \
--to=chandanbabu@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=dan.j.williams@intel.com \
--cc=djwong@kernel.org \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
--cc=mcgrof@kernel.org \
--cc=nvdimm@lists.linux.dev \
--cc=ruansy.fnst@fujitsu.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).