* cp -a leaves some compressed data.
@ 2020-05-16 16:32 A L
2020-05-16 17:18 ` Filipe Manana
0 siblings, 1 reply; 5+ messages in thread
From: A L @ 2020-05-16 16:32 UTC (permalink / raw
To: linux-btrfs
Dear all,
I did some testing on copying files with the +c (compression) xattrs set.
As far as I can tell, 'cp - a' only sets any xattrs after copying the data. This means that a compressed file should end up without compression, but still with the +c xattr set. However this is not entirely true. Some small amount of data is still getting compressed.
I would like to understand why.
Here is a small test case:
File test-comp.sh:
#!/bin/bash
mkdir -p test test/a test/b
chattr +c test/a
touch test/a/foo
dd if=/dev/zero of=test/a/foo bs=1024 count=1M
cp -a test/a test/b/
Now check the output with the compsize tool:
# compsize test/a
Type Perc Disk Usage Uncompressed Referenced
TOTAL 3% 32M 1.0G 1.0G
zlib 3% 32M 1.0G 1.0G
# compsize test/b
Type Perc Disk Usage Uncompressed Referenced
TOTAL 63% 652M 1.0G 1.0G
none 100% 640M 640M 640M
zlib 3% 12M 384M 384M
/mnt/test #
As you see, the copy ended up with 384M compressed data. When running this test several times, the amount changes between runs.
I did an strace too see what was going on. It is clear that the setfxattr() is called after all the data was written to the file.
# strace -s8 -xx cp -av a/foo b/
execve("\x2f\x62\x69\x6e\x2f\x63\x70", ["\x63\x70", "\x2d\x61\x76", "\x61\x2f
\x66\x6f\x6f", "\x62\x2f"], 0x7fff9d6acb68 /* 44 vars */) = 0
brk(NULL) = 0x556e7cf7c000
access("\x2f\x65\x74\x63\x2f\x6c\x64\x2e\x73\x6f\x2e\x70\x72\x65\x6c\x6f\x61\
x64", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "\x2f\x65\x74\x63\x2f\x6c\x64\x2e\x73\x6f\x2e\x63\x61\x63\x6
8\x65", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=122526, ...}) = 0
mmap(NULL, 122526, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5f04d8b000
close(3) = 0
openat(AT_FDCWD, "\x2f\x6c\x69\x62\x36\x34\x2f\x6c\x69\x62\x61\x63\x6c\x2e\x7
3\x6f\x2e\x31", O_RDONLY|O_CLOEXEC) = 3
read(3, "\x7f\x45\x4c\x46\x02\x01\x01\x00"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=39240, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x
7f5f04d89000
mmap(NULL, 41568, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5f04d7e00
0
mmap(0x7f5f04d80000, 20480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DE
NYWRITE, 3, 0x2000) = 0x7f5f04d80000
mmap(0x7f5f04d85000, 8192, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3,
0x7000) = 0x7f5f04d85000
mmap(0x7f5f04d87000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DE
NYWRITE, 3, 0x8000) = 0x7f5f04d87000
close(3) = 0
openat(AT_FDCWD, "\x2f\x6c\x69\x62\x36\x34\x2f\x6c\x69\x62\x61\x74\x74\x72\x2
e\x73\x6f\x2e\x31", O_RDONLY|O_CLOEXEC) = 3
read(3, "\x7f\x45\x4c\x46\x02\x01\x01\x00"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=26720, ...}) = 0
mmap(NULL, 29016, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5f04d7600
0
mmap(0x7f5f04d78000, 12288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DE
NYWRITE, 3, 0x2000) = 0x7f5f04d78000
write(1, "\x27\x61\x2f\x66\x6f\x6f\x27\x20"..., 19'a/foo' -> 'b/foo'
) = 19
openat(AT_FDCWD, "\x61\x2f\x66\x6f\x6f", O_RDONLY|O_NOFOLLOW) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1048576, ...}) = 0
openat(AT_FDCWD, "\x62\x2f\x66\x6f\x6f", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f5f0440f000
read(3, "\x00\x00\x00\x00\x00\x00\x00\x00"..., 131072) = 131072
write(4, "\x00\x00\x00\x00\x00\x00\x00\x00"..., 131072) = 131072
...
snip
...
read(3, "", 131072) = 0
utimensat(4, NULL, [{tv_sec=1589642969, tv_nsec=260647830} /* 2020-05-16T17:2
9:29.260647830+0200 */, {tv_sec=1589643713, tv_nsec=971537549} /* 2020-05-16T
17:41:53.971537549+0200 */], 0) = 0
flistxattr(3, NULL, 0) = 18
flistxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., 18) = 18
openat(AT_FDCWD, "\x2f\x65\x74\x63\x2f\x78\x61\x74\x74\x72\x2e\x63\x6f\x6e\x6
6", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=642, ...}) = 0
read(5, "\x23\x20\x2f\x65\x74\x63\x2f\x78"..., 4096) = 642
read(5, "", 4096) = 0
close(5) = 0
openat(AT_FDCWD, "\x2f\x75\x73\x72\x2f\x6c\x69\x62\x36\x34\x2f\x67\x63\x6f\x6
e\x76\x2f\x67\x63\x6f\x6e\x76\x2d\x6d\x6f\x64\x75\x6c\x65\x73\x2e\x63\x61\x63
\x68\x65", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=26988, ...}) = 0
mmap(NULL, 26988, PROT_READ, MAP_SHARED, 5, 0) = 0x7f5f04da2000
close(5) = 0
fgetxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., NULL, 0) = 4
fgetxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., "\x7a\x6c\x69\x62", 4) =
4
fsetxattr(4, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., "\x7a\x6c\x69\x62", 4, 0)
= 0
fgetxattr(3, "\x73\x79\x73\x74\x65\x6d\x2e\x70"..., 0x7fff8daf2580, 132) = -1
ENODATA (No data available)
fstat(3, {st_mode=S_IFREG|0644, st_size=1048576, ...}) = 0
fsetxattr(4, "\x73\x79\x73\x74\x65\x6d\x2e\x70"..., "\x02\x00\x00\x00\x01\x00
\x06\x00"..., 28, 0) = 0
close(4) = 0
close(3) = 0
munmap(0x7f5f0440f000, 139264) = 0
lseek(0, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
close(0) = 0
close(1) = 0
close(2) = 0
exit_group(0) = ?
+++ exited with 0 +++
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cp -a leaves some compressed data.
2020-05-16 16:32 cp -a leaves some compressed data A L
@ 2020-05-16 17:18 ` Filipe Manana
2020-05-16 20:01 ` A L
0 siblings, 1 reply; 5+ messages in thread
From: Filipe Manana @ 2020-05-16 17:18 UTC (permalink / raw
To: A L; +Cc: linux-btrfs
On Sat, May 16, 2020 at 5:51 PM A L <mail@lechevalier.se> wrote:
>
> Dear all,
>
> I did some testing on copying files with the +c (compression) xattrs set.
>
> As far as I can tell, 'cp - a' only sets any xattrs after copying the data. This means that a compressed file should end up without compression, but still with the +c xattr set. However this is not entirely true. Some small amount of data is still getting compressed.
>
> I would like to understand why.
As discussed on the mailing list:
cp copies the xattr only after copying the file data. Since the data
is written to the destination using buffered IO, it is possible that
while copying the data the system flushes dirty pages for whatever
reason (due to memory pressure, someone called sync(2), etc) - this
data will not be compressed since the file doesn't have yet the
compression xattr. If the remaining data is flushed after cp finishes,
then that data can end up compressed, since the file has the
compression xattr at that point. Typically for small files, all the
data ends up getting flushed after cp finishes, so we don't see any
surprising behaviour.
I'll look into changing 'cp''s behaviour to copy xattrs before file
data next week, unless you or someone else is interested in doing it.
Thanks.
>
> Here is a small test case:
>
> File test-comp.sh:
> #!/bin/bash
> mkdir -p test test/a test/b
> chattr +c test/a
> touch test/a/foo
> dd if=/dev/zero of=test/a/foo bs=1024 count=1M
> cp -a test/a test/b/
>
> Now check the output with the compsize tool:
>
> # compsize test/a
> Type Perc Disk Usage Uncompressed Referenced
> TOTAL 3% 32M 1.0G 1.0G
> zlib 3% 32M 1.0G 1.0G
>
> # compsize test/b
> Type Perc Disk Usage Uncompressed Referenced
> TOTAL 63% 652M 1.0G 1.0G
> none 100% 640M 640M 640M
> zlib 3% 12M 384M 384M
> /mnt/test #
>
>
> As you see, the copy ended up with 384M compressed data. When running this test several times, the amount changes between runs.
>
> I did an strace too see what was going on. It is clear that the setfxattr() is called after all the data was written to the file.
>
>
>
> # strace -s8 -xx cp -av a/foo b/
> execve("\x2f\x62\x69\x6e\x2f\x63\x70", ["\x63\x70", "\x2d\x61\x76", "\x61\x2f
> \x66\x6f\x6f", "\x62\x2f"], 0x7fff9d6acb68 /* 44 vars */) = 0
> brk(NULL) = 0x556e7cf7c000
> access("\x2f\x65\x74\x63\x2f\x6c\x64\x2e\x73\x6f\x2e\x70\x72\x65\x6c\x6f\x61\
> x64", R_OK) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "\x2f\x65\x74\x63\x2f\x6c\x64\x2e\x73\x6f\x2e\x63\x61\x63\x6
> 8\x65", O_RDONLY|O_CLOEXEC) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=122526, ...}) = 0
> mmap(NULL, 122526, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5f04d8b000
> close(3) = 0
> openat(AT_FDCWD, "\x2f\x6c\x69\x62\x36\x34\x2f\x6c\x69\x62\x61\x63\x6c\x2e\x7
> 3\x6f\x2e\x31", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\x7f\x45\x4c\x46\x02\x01\x01\x00"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=39240, ...}) = 0
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x
> 7f5f04d89000
> mmap(NULL, 41568, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5f04d7e00
> 0
> mmap(0x7f5f04d80000, 20480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DE
> NYWRITE, 3, 0x2000) = 0x7f5f04d80000
> mmap(0x7f5f04d85000, 8192, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3,
> 0x7000) = 0x7f5f04d85000
> mmap(0x7f5f04d87000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DE
> NYWRITE, 3, 0x8000) = 0x7f5f04d87000
> close(3) = 0
> openat(AT_FDCWD, "\x2f\x6c\x69\x62\x36\x34\x2f\x6c\x69\x62\x61\x74\x74\x72\x2
> e\x73\x6f\x2e\x31", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\x7f\x45\x4c\x46\x02\x01\x01\x00"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=26720, ...}) = 0
> mmap(NULL, 29016, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5f04d7600
> 0
> mmap(0x7f5f04d78000, 12288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DE
> NYWRITE, 3, 0x2000) = 0x7f5f04d78000
> write(1, "\x27\x61\x2f\x66\x6f\x6f\x27\x20"..., 19'a/foo' -> 'b/foo'
> ) = 19
> openat(AT_FDCWD, "\x61\x2f\x66\x6f\x6f", O_RDONLY|O_NOFOLLOW) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=1048576, ...}) = 0
> openat(AT_FDCWD, "\x62\x2f\x66\x6f\x6f", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
> fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
> fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
> mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f5f0440f000
> read(3, "\x00\x00\x00\x00\x00\x00\x00\x00"..., 131072) = 131072
> write(4, "\x00\x00\x00\x00\x00\x00\x00\x00"..., 131072) = 131072
> ...
> snip
> ...
> read(3, "", 131072) = 0
> utimensat(4, NULL, [{tv_sec=1589642969, tv_nsec=260647830} /* 2020-05-16T17:2
> 9:29.260647830+0200 */, {tv_sec=1589643713, tv_nsec=971537549} /* 2020-05-16T
> 17:41:53.971537549+0200 */], 0) = 0
> flistxattr(3, NULL, 0) = 18
> flistxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., 18) = 18
> openat(AT_FDCWD, "\x2f\x65\x74\x63\x2f\x78\x61\x74\x74\x72\x2e\x63\x6f\x6e\x6
> 6", O_RDONLY) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=642, ...}) = 0
> read(5, "\x23\x20\x2f\x65\x74\x63\x2f\x78"..., 4096) = 642
> read(5, "", 4096) = 0
> close(5) = 0
> openat(AT_FDCWD, "\x2f\x75\x73\x72\x2f\x6c\x69\x62\x36\x34\x2f\x67\x63\x6f\x6
> e\x76\x2f\x67\x63\x6f\x6e\x76\x2d\x6d\x6f\x64\x75\x6c\x65\x73\x2e\x63\x61\x63
> \x68\x65", O_RDONLY) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=26988, ...}) = 0
> mmap(NULL, 26988, PROT_READ, MAP_SHARED, 5, 0) = 0x7f5f04da2000
> close(5) = 0
> fgetxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., NULL, 0) = 4
> fgetxattr(3, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., "\x7a\x6c\x69\x62", 4) =
> 4
> fsetxattr(4, "\x62\x74\x72\x66\x73\x2e\x63\x6f"..., "\x7a\x6c\x69\x62", 4, 0)
> = 0
> fgetxattr(3, "\x73\x79\x73\x74\x65\x6d\x2e\x70"..., 0x7fff8daf2580, 132) = -1
> ENODATA (No data available)
> fstat(3, {st_mode=S_IFREG|0644, st_size=1048576, ...}) = 0
> fsetxattr(4, "\x73\x79\x73\x74\x65\x6d\x2e\x70"..., "\x02\x00\x00\x00\x01\x00
> \x06\x00"..., 28, 0) = 0
> close(4) = 0
> close(3) = 0
> munmap(0x7f5f0440f000, 139264) = 0
> lseek(0, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> close(0) = 0
> close(1) = 0
> close(2) = 0
> exit_group(0) = ?
> +++ exited with 0 +++
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cp -a leaves some compressed data.
2020-05-16 17:18 ` Filipe Manana
@ 2020-05-16 20:01 ` A L
2020-05-18 16:33 ` Filipe Manana
0 siblings, 1 reply; 5+ messages in thread
From: A L @ 2020-05-16 20:01 UTC (permalink / raw
To: linux-btrfs; +Cc: fdmanana
On 2020-05-16 19:18, Filipe Manana wrote:
> On Sat, May 16, 2020 at 5:51 PM A L <mail@lechevalier.se> wrote:
>> Dear all,
>>
>> I did some testing on copying files with the +c (compression) xattrs set.
>>
>> As far as I can tell, 'cp - a' only sets any xattrs after copying the data. This means that a compressed file should end up without compression, but still with the +c xattr set. However this is not entirely true. Some small amount of data is still getting compressed.
>>
>> I would like to understand why.
> As discussed on the mailing list:
>
> cp copies the xattr only after copying the file data. Since the data
> is written to the destination using buffered IO, it is possible that
> while copying the data the system flushes dirty pages for whatever
> reason (due to memory pressure, someone called sync(2), etc) - this
> data will not be compressed since the file doesn't have yet the
> compression xattr. If the remaining data is flushed after cp finishes,
> then that data can end up compressed, since the file has the
> compression xattr at that point. Typically for small files, all the
> data ends up getting flushed after cp finishes, so we don't see any
> surprising behaviour.
>
> I'll look into changing 'cp''s behaviour to copy xattrs before file
> data next week, unless you or someone else is interested in doing it.
>
> Thanks.
>
Based on what you say, the file operations are happening asynchronous in
the background, rather than synchronous. I guess 'cp' and other tools
like it should issue a 'fsync' call between setting the xattrs and
writing data? Is this specific to Btrfs, or is it a Linux design choice?
Also, thanks for looking into changing cp to do the xattrs before
writing data. I had also asked about this on the coreutils mailing list:
https://lists.gnu.org/archive/html/coreutils/2020-05/msg00011.html
Thanks
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cp -a leaves some compressed data.
2020-05-16 20:01 ` A L
@ 2020-05-18 16:33 ` Filipe Manana
2020-05-19 15:32 ` A L
0 siblings, 1 reply; 5+ messages in thread
From: Filipe Manana @ 2020-05-18 16:33 UTC (permalink / raw
To: A L; +Cc: linux-btrfs
On Sat, May 16, 2020 at 9:01 PM A L <mail@lechevalier.se> wrote:
>
>
> On 2020-05-16 19:18, Filipe Manana wrote:
> > On Sat, May 16, 2020 at 5:51 PM A L <mail@lechevalier.se> wrote:
> >> Dear all,
> >>
> >> I did some testing on copying files with the +c (compression) xattrs set.
> >>
> >> As far as I can tell, 'cp - a' only sets any xattrs after copying the data. This means that a compressed file should end up without compression, but still with the +c xattr set. However this is not entirely true. Some small amount of data is still getting compressed.
> >>
> >> I would like to understand why.
> > As discussed on the mailing list:
> >
> > cp copies the xattr only after copying the file data. Since the data
> > is written to the destination using buffered IO, it is possible that
> > while copying the data the system flushes dirty pages for whatever
> > reason (due to memory pressure, someone called sync(2), etc) - this
> > data will not be compressed since the file doesn't have yet the
> > compression xattr. If the remaining data is flushed after cp finishes,
> > then that data can end up compressed, since the file has the
> > compression xattr at that point. Typically for small files, all the
> > data ends up getting flushed after cp finishes, so we don't see any
> > surprising behaviour.
> >
> > I'll look into changing 'cp''s behaviour to copy xattrs before file
> > data next week, unless you or someone else is interested in doing it.
> >
> > Thanks.
> >
> Based on what you say, the file operations are happening asynchronous in
> the background, rather than synchronous.
What I said is that while in the middle of copying, dirty pages might
be flushed for some reason, in which case the data won't be compressed
since the destination file doesn't have the xattr set yet. The
remaining data will be flushed after the xattr was set, so it will end
up getting compressed.
> I guess 'cp' and other tools
> like it should issue a 'fsync' call between setting the xattrs and
> writing data?
I don't understand what you are trying to say. Are you suggesting the
fsync would help the issue you described (I don't see how), with the
file ending up having compressed and uncompressed extents, or is that
for some other issue you are thinking about?
> Is this specific to Btrfs, or is it a Linux design choice?
Can't tell since I don't understand what is the problem.
>
> Also, thanks for looking into changing cp to do the xattrs before
> writing data. I had also asked about this on the coreutils mailing list:
> https://lists.gnu.org/archive/html/coreutils/2020-05/msg00011.html
Great, thanks. The coreutils folks will deal with it the best way.
>
> Thanks
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: cp -a leaves some compressed data.
2020-05-18 16:33 ` Filipe Manana
@ 2020-05-19 15:32 ` A L
0 siblings, 0 replies; 5+ messages in thread
From: A L @ 2020-05-19 15:32 UTC (permalink / raw
To: linux-btrfs
> I don't understand what you are trying to say. Are you suggesting the
> fsync would help the issue you described (I don't see how), with the
> file ending up having compressed and uncompressed extents, or is that
> for some other issue you are thinking about?
>
>> Is this specific to Btrfs, or is it a Linux design choice?
> Can't tell since I don't understand what is the problem.
>
As a normal end-user, my expectation is that if I 1) first write to a
file, then 2) change its attributes (compression), those attributes
should not affect the data previously written.
But how it is now, depending on how "quick" the system acts in writing
out the data, some of it might be written after the attributes take
affect. I found this non-serial way of writing confusing. It is probably
how it is supposed to be, and my understanding is lacking :)
Thanks again!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-05-19 15:32 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-05-16 16:32 cp -a leaves some compressed data A L
2020-05-16 17:18 ` Filipe Manana
2020-05-16 20:01 ` A L
2020-05-18 16:33 ` Filipe Manana
2020-05-19 15:32 ` A L
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.