From: John Garry <john.g.garry@oracle.com>
To: "Darrick J. Wong" <djwong@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
martin.petersen@oracle.com, david@fromorbit.com,
himanshu.madhani@oracle.com
Subject: Re: [PATCH 2/4] readv.2: Document RWF_ATOMIC flag
Date: Tue, 24 Oct 2023 13:30:03 +0100 [thread overview]
Message-ID: <0262fed7-a70f-8782-628f-2e9ded0108f8@oracle.com> (raw)
In-Reply-To: <20231009174438.GE21283@frogsfrogsfrogs>
On 09/10/2023 18:44, Darrick J. Wong wrote:
> On Fri, Sep 29, 2023 at 09:37:15AM +0000, John Garry wrote:
>> From: Himanshu Madhani <himanshu.madhani@oracle.com>
>>
>> Add RWF_ATOMIC flag description for pwritev2().
>>
>> Signed-off-by: Himanshu Madhani <himanshu.madhani@oracle.com>
>> #jpg: complete rewrite
>> Signed-off-by: John Garry <john.g.garry@oracle.com>
>> ---
>> man2/readv.2 | 45 +++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 45 insertions(+)
>>
>> diff --git a/man2/readv.2 b/man2/readv.2
>> index fa9b0e4e44a2..ff09f3bc9792 100644
>> --- a/man2/readv.2
>> +++ b/man2/readv.2
>> @@ -193,6 +193,51 @@ which provides lower latency, but may use additional resources.
>> .B O_DIRECT
>> flag.)
>> .TP
>> +.BR RWF_ATOMIC " (since Linux 6.7)"
>> +Allows block-based filesystems to indicate that write operations will be issued
>
> "Require regular file write operations to be issued with torn write
> protection."
ok
>
>> +with torn-write protection. Torn-write protection means that for a power or any
>> +other hardware failure, all or none of the data from the write will be stored,
>> +but never a mix of old and new data. This flag is meaningful only for
>> +.BR pwritev2 (),
>> +and its effect applies only to the data range written by the system call.
>> +The total write length must be power-of-2 and must be sized between
>> +stx_atomic_write_unit_min and stx_atomic_write_unit_max, both inclusive. The
>> +write must be at a natural offset within the file with respect to the total
>
> What is a "natural" offset?
I really meant naturally-aligned offset
> That should be defined with more
> specificity. Does that mean that the position of a XX-KiB write must
> also be aligned to XX-KiB?
Yes
> e.g. a 32K untorn write can only start at a
> multiple of 32K?
Correct
> What if the device supports untorn writes between 4K
> and 64K, does that mean I /cannot/ issue a 32K untorn write at offset
> 48K?
Correct
Do you think that an example would help?
>
>> +write length. Torn-write protection only works with
>> +.B O_DIRECT
>> +flag, i.e. buffered writes are not supported. To guarantee consistency from
>> +the write between a file's in-core state with the storage device,
>> +.BR fdatasync (2)
>> +or
>> +.BR fsync (2)
>> +or
>> +.BR open (2)
>> +and
>> +.B O_SYNC
>> +or
>> +.B O_DSYNC
>> +or
>> +.B pwritev2 ()
>> +flag
>> +.B RWF_SYNC
>> +or
>> +.B RWF_DSYNC
>> +is required.
>
> I'm starting to think that this manpage shouldn't be restating
> durability information here.
>
> "Application programs with data or file integrity completion
> requirements must configure synchronous writes with the DSYNC
> or SYNC flags, as explained above."
ok
>
>> +For when regular files are opened with
>> +.BR open (2)
>> +but without
>> +.B O_SYNC
>> +or
>> +.B O_DSYNC
>> +and the
>> +.BR pwritev2()
>> +call is made without
>> +.B RWF_SYNC
>> +or
>> +.BR RWF_DSYNC
>> +set, the range metadata must already be flushed to storage and the data range
>> +must not be in unwritten state, shared, a preallocation, or a hole.
>
> I think that we can drop all of these flags requirements, since the
> contiguous small space allocation requirement means that the fs can
> provide all-or-nothing writes even if metadata updates are needed:
>
> If the file range is allocated and marked unwritten (i.e. a
> preallocation), the ioend will clear the unwritten bit from the file
> mapping atomically. After a crash, the application sees either zeroes
> or all the data that was written.
>
> If the file range is shared, the ioend will map the COW staging extent
> into the file atomically. After a crash, the application sees either
> the old contents from the old blocks, or the new contents from the new
> blocks.
>
> If the file range is a sparse hole, the directio setup will allocate
> space and create an unwritten mapping before issuing the write bio. The
> rest of the process works the same as preallocations and has the same
> behaviors.
>
> If the file range is allocated and was previously written, the write is
> issued and that's all that's needed from the fs. After a crash, reads
> of the storage device produce the old contents or the new contents.
>
> Summarizing:
>
> An (ATOMIC|SYNC) request provides the strongest guarantees (data
> will not be torn, and all file metadata updates are persisted before
> the write is returned to userspace. Programs see either the old data or
> the new data, even if there's a crash.
>
> (ATOMIC|DSYNC) is less strong -- data will not be torn, and any file
> updates for just that region are persisted before the write is returned.
>
> (ATOMIC) is the least strong -- data will not be torn. Neither the
> filesystem nor the device make guarantees that anything ended up on
> stable storage, but if it does, programs see either the old data or the
> new data.
>
Will respond to later mail in thread.
> Maybe we should rename the whole UAPI s/atomic/untorn/...
> > --D
>
>> +.TP
>> .BR RWF_SYNC " (since Linux 4.7)"
>> .\" commit e864f39569f4092c2b2bc72c773b6e486c7e3bd9
>> Provide a per-write equivalent of the
>> --
>> 2.31.1
>>
Thanks,
John
next prev parent reply other threads:[~2023-10-24 12:30 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-29 9:37 [PATCH 0/4] man2: Document RWF_ATOMIC John Garry
2023-09-29 9:37 ` [PATCH 1/4] statx.2: Document STATX_WRITE_ATOMIC John Garry
2023-09-29 9:37 ` [PATCH 2/4] readv.2: Document RWF_ATOMIC flag John Garry
2023-10-03 19:25 ` Bart Van Assche
2023-10-04 8:47 ` John Garry
2023-10-04 17:36 ` Bart Van Assche
2023-10-04 22:48 ` Dave Chinner
2023-10-09 17:44 ` Darrick J. Wong
2023-10-09 20:39 ` Dave Chinner
2023-10-09 21:05 ` Darrick J. Wong
2023-10-24 12:35 ` John Garry
2023-10-24 15:39 ` Darrick J. Wong
2023-10-24 12:30 ` John Garry [this message]
2023-10-24 15:39 ` Darrick J. Wong
2023-09-29 9:37 ` [PATCH 3/4] man2/open.2: Document RWF_ATOMIC John Garry
2023-09-29 9:37 ` [PATCH 4/4] io_submit.2: " John Garry
2023-10-09 17:45 ` Darrick J. Wong
2023-10-24 11:51 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0262fed7-a70f-8782-628f-2e9ded0108f8@oracle.com \
--to=john.g.garry@oracle.com \
--cc=david@fromorbit.com \
--cc=djwong@kernel.org \
--cc=himanshu.madhani@oracle.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=martin.petersen@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).