From: John Garry <john.g.garry@oracle.com>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-xfs@vger.kernel.org, ojaswin@linux.ibm.com, ritesh.list@gmail.com
Subject: Re: [PATCH 1/3] xfs: simplify extent allocation alignment
Date: Wed, 3 Apr 2024 09:49:06 +0100 [thread overview]
Message-ID: <d17cbb52-0d30-40c9-b700-6617c50fa86c@oracle.com> (raw)
In-Reply-To: <Zgx4BhcW9/6XAiq9@dread.disaster.area>
On 02/04/2024 22:26, Dave Chinner wrote:
>> And then this gives:
>> args->minlen=48 blen=64
> Which is perfectly reasonable - it fits the bounds specified just
> fine, and we'll get a 64 block allocation if that free space is
> exactly aligned. Otherwise we'll get a 48 block allocation.
>
>> But xfs_alloc_vextent_start_ag() -> xfs_alloc_vextent_iterate_ags() does not
>> seem to find something suitable.
> Entirely possible. The AGFL might have needed refilling, so there
> really wasn't enough blocks remaining for an aligned allocation to
> take place after doing that. That's a real ENOSPC condition, despite
> the best length sampling indicating that the allocation could be
> done.
>
> Remember, bestlen sampling is racy - it does not hold the AGF
> locked from the point of sampling to the point of allocation.
ok, I assumed that some lock was held.
> Hence
> when we finally go to do the allocation after setting it all up, we
> might have raced with another allocation that took the free space
> sampled during the bestlen pass and so then the allocation fails
> despite the setup saying it should succeed.
My test is single threaded, so I did not think that would be an issue.
>
> Fundamentally, if the filesystem's best free space length is the
> same size as the requested allocation,*failure is expected* and the
> fallback attempts progressively remove restrictions (such as
> alignment) to allow sub-optimal extents to be allocated down to
> minlen in size. Forced alignment turns off these fallbacks, so you
> are going to see hard ENOSPC errors the moment the filesystem runs
> out of contiguous free space extents large enough to hold aligned
> allocations.
>
> This can happen a -long- way away from a real enospc condition -
> depending on alignment constraints, you can start seeing this sort
> of behaviour (IIRC my math correctly) at around 70% full. The larger
> the alignment and the older the filesystem, the earlier this sort of
> ENOSPC event will occur.
For this scenario, statvfs returns - as a sample - f_blocks=73216,
f_bfree=19950, f_bavail=19950
So ~27% free.
>
> Use `xfs_spaceman -c 'freesp'` to dump the free space extent size
> histogram. That will quickly tell you if there is no remaining free
> extents large enough for an aligned 48 block allocation to succeed.
> With an alignment of 16 blocks, this requires at least a 63 block
> free space extent to succeed.
# xfs_spaceman -c 'freesp' /root/mnt2/
from to extents blocks pct
4 7 4 25 0.10
16 31 90 1440 5.77
32 63 12 400 1.60
64 127 1 64 0.26
512 1023 1 640 2.56
16384 22400 1 22390 89.71
>
> IOWs, we should expect ENOSPC to occur long before the filesystem
> reports that it is out of space when we are doing forced alignment
> allocation.
For my test, once ENOSPC occurs and statvfs tells us more than 10% space
free, we exit as something seems wrong. As you say, deducing an error
for this condition may not be the proper thing to do.
I do also note that if I then manually attempt to write the same data to
that same empty file after the test exits, it succeeds. So something
racy. I also notice that the FSB range we scan in
xfs_alloc_ag_vextent_size() -> xfs_alloc_compute_aligned() ->
xfs_extent_busy_trim() returns busy=1 when ENOSPC occurs - I have not
checked that further.
Thanks,
John
next prev parent reply other threads:[~2024-04-03 8:49 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-04 13:04 [PATCH v2 00/14] block atomic writes for XFS John Garry
2024-03-04 13:04 ` [PATCH v2 01/14] block: Add blk_validate_atomic_write_op_size() John Garry
2024-03-04 13:04 ` [PATCH v2 02/14] fs: xfs: Don't use low-space allocator for alignment > 1 John Garry
2024-03-04 22:15 ` Dave Chinner
2024-03-05 13:36 ` John Garry
2024-03-04 13:04 ` [PATCH v2 03/14] fs: xfs: Introduce FORCEALIGN inode flag John Garry
2024-03-04 13:04 ` [PATCH v2 04/14] fs: xfs: Make file data allocations observe the 'forcealign' flag John Garry
2024-03-05 0:44 ` Dave Chinner
2024-03-05 15:22 ` John Garry
2024-03-05 22:18 ` Dave Chinner
2024-03-06 5:20 ` [RFC PATCH 0/3] xfs: forced extent alignment Dave Chinner
2024-03-06 5:20 ` [PATCH 1/3] xfs: simplify extent allocation alignment Dave Chinner
2024-03-13 11:03 ` John Garry
2024-03-20 4:35 ` Dave Chinner
2024-03-26 16:08 ` John Garry
2024-04-02 5:58 ` Dave Chinner
2024-04-02 7:49 ` John Garry
2024-04-02 15:11 ` John Garry
2024-04-02 21:26 ` Dave Chinner
2024-04-03 8:49 ` John Garry [this message]
2024-04-02 23:44 ` Dave Chinner
2024-04-03 11:30 ` John Garry
2024-03-06 5:20 ` [PATCH 2/3] xfs: make EOF allocation simpler Dave Chinner
2024-03-06 5:20 ` [PATCH 3/3] xfs: introduce forced allocation alignment Dave Chinner
2024-03-06 11:46 ` [RFC PATCH 0/3] xfs: forced extent alignment John Garry
2024-03-06 17:52 ` John Garry
2024-03-06 20:54 ` Dave Chinner
2024-03-13 18:32 ` John Garry
2024-03-06 9:41 ` [PATCH v2 04/14] fs: xfs: Make file data allocations observe the 'forcealign' flag John Garry
2024-03-04 13:04 ` [PATCH v2 05/14] fs: xfs: Enable file data forcealign feature John Garry
2024-03-04 13:04 ` [PATCH v2 06/14] fs: xfs: Do not free EOF blocks for forcealign John Garry
2024-03-06 21:07 ` Dave Chinner
2024-03-07 11:38 ` John Garry
2024-03-04 13:04 ` [PATCH v2 07/14] fs: iomap: Sub-extent zeroing John Garry
2024-03-06 21:14 ` Dave Chinner
2024-03-07 11:51 ` John Garry
2024-03-04 13:04 ` [PATCH v2 08/14] fs: xfs: " John Garry
2024-03-06 22:00 ` Dave Chinner
2024-03-07 12:57 ` John Garry
2024-03-04 13:04 ` [PATCH v2 09/14] fs: Add FS_XFLAG_ATOMICWRITES flag John Garry
2024-03-04 13:04 ` [PATCH v2 10/14] fs: iomap: Atomic write support John Garry
2024-03-04 13:04 ` [PATCH v2 11/14] fs: xfs: Support FS_XFLAG_ATOMICWRITES for forcealign John Garry
2024-03-06 21:43 ` Dave Chinner
2024-03-07 12:42 ` John Garry
2024-03-04 13:04 ` [PATCH v2 12/14] fs: xfs: Support atomic write for statx John Garry
2024-03-06 21:31 ` Dave Chinner
2024-03-07 10:35 ` John Garry
2024-03-04 13:04 ` [PATCH v2 13/14] fs: xfs: Validate atomic writes John Garry
2024-03-06 21:22 ` Dave Chinner
2024-03-07 10:19 ` John Garry
2024-03-04 13:04 ` [PATCH v2 14/14] fs: xfs: Support setting FMODE_CAN_ATOMIC_WRITE John Garry
2024-03-06 21:33 ` Dave Chinner
2024-03-07 11:55 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d17cbb52-0d30-40c9-b700-6617c50fa86c@oracle.com \
--to=john.g.garry@oracle.com \
--cc=david@fromorbit.com \
--cc=linux-xfs@vger.kernel.org \
--cc=ojaswin@linux.ibm.com \
--cc=ritesh.list@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).