* Error while balance after convert from ext4
@ 2015-07-07 21:46 Николай Овчинников
2015-07-08 0:18 ` Duncan
2015-07-08 13:56 ` Chris Murphy
0 siblings, 2 replies; 8+ messages in thread
From: Николай Овчинников @ 2015-07-07 21:46 UTC (permalink / raw
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 5477 bytes --]
While balancing after succsecful converting from ext4 filesystem moved
into read-only mode and balance aborted.
Before balance df says about 6Gb freespace.
Now filesystem can be mounted only in recovery mode with 1 dir and
about 20 zero-sized files.
Please find attached syslog
btrfsck thrown abort
btrfs-image failed
btrfs restore of zero-sized files with different -t num from find-root failed
Is there a chance to recover these files?
The sequence of my actions with timestamp to correlate with syslog:
btrfs-convert /dev/sda7
13:33 mount /dev/sda7 /mnt
btrfs sub del /mnt/ext2_saved
13:35 btrfs balance start /mnt
btrfs balance cancel
umount /mnt
13:40 mount /dev/sda7 /mnt -o compress-force=zlib
btrfs balance start /mnt
13:57 ***Error*** balance stop because fs changed to read-only
14:02 mount /dev/sda7 /mnt -o compress-force=zlib
14:22 mount /dev/sda7 /mnt -o ro,recovery
root@ovchin-eee:/home/ovchin/btrfs-progs# uname -a
Linux ovchin-eee 4.0.0-2-amd64 #1 SMP Debian 4.0.7-1 (2015-07-06)
x86_64 GNU/Linux
root@ovchin-eee:/home/ovchin/btrfs-progs# ./btrfs --version
btrfs-progs v4.1
root@ovchin-eee:/home/ovchin/btrfs-progs# ./btrfs fi show
Label: none uuid: 489fb2ac-7f1a-46da-9fb8-964e483c8081
Total devices 1 FS bytes used 188.43GiB
devid 1 size 201.23GiB used 199.23GiB path /dev/sda7
root@ovchin-eee:/home/ovchin/btrfs-progs# gdb ./btrfsck
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./btrfsck...done.
(gdb) run /dev/sda7
Starting program: /home/ovchin/btrfs-progs/btrfsck /dev/sda7
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Checking filesystem on /dev/sda7
UUID: 489fb2ac-7f1a-46da-9fb8-964e483c8081
checking extents
Program received signal SIGABRT, Aborted.
0x00007ffff6fc6107 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: Нет такого файла или каталога.
(gdb) bt
#0 0x00007ffff6fc6107 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff6fc74e8 in __GI_abort () at abort.c:89
#2 0x000000000042cd25 in add_tree_backref
(extent_cache=0x7fffffffe530, bytenr=7707766784,
parent=0, root=4, found_ref=1) at cmds-check.c:4421
#3 0x00000000004309a2 in add_root_to_pending (buf=0x6dcd00,
extent_cache=0x7fffffffe530,
pending=0x7fffffffe510, seen=0x7fffffffe520, nodes=0x7fffffffe4f0,
objectid=4)
at cmds-check.c:5961
#4 0x0000000000434abb in deal_root_from_list (list=0x7fffffffe200,
root=0x6eaa10,
bits=0x6f1170, bits_nr=1024, pending=0x7fffffffe510, seen=0x7fffffffe520,
reada=0x7fffffffe500, nodes=0x7fffffffe4f0, extent_cache=0x7fffffffe530,
chunk_cache=0x7fffffffe590, dev_cache=0x7fffffffe5a0,
block_group_cache=0x7fffffffe570,
dev_extent_cache=0x7fffffffe540) at cmds-check.c:7803
#5 0x0000000000434f95 in check_chunks_and_extents (root=0x6eaa10) at
cmds-check.c:7973
#6 0x00000000004381a0 in cmd_check (argc=1, argv=0x7fffffffe808) at
cmds-check.c:9402
#7 0x0000000000409dec in main (argc=2, argv=0x7fffffffe808) at btrfs.c:245
oot@ovchin-eee:/home/ovchin/btrfs-progs# ./btrfs-image -c9 -t4
/dev/sda7 ../btrfs.image
checksum verify failed on 36143104 found 79B018CB wanted B0E76697
checksum verify failed on 36143104 found 79B018CB wanted B0E76697
bytenr mismatch, want=36143104, have=13590450109232199243
Error reading metadata block
Error adding block -5
checksum verify failed on 36143104 found 79B018CB wanted B0E76697
checksum verify failed on 36143104 found 79B018CB wanted B0E76697
bytenr mismatch, want=36143104, have=13590450109232199243
Error reading metadata block
Error flushing pending -5
create failed (Success)
root@ovchin-eee:/home/ovchin/btrfs-progs# ./btrfs-image -c9 -t4 -w
/dev/sda7 ../btrfs.image
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
bytenr mismatch, want=38006784, have=671059430232024561
Error reading log block
create failed (Success)
root@ovchin-eee:/home/ovchin/btrfs-progs# btrfs restore --path-regex
^/[^/]*dat$ /dev/sda7 ../rescue/
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
bytenr mismatch, want=38006784, have=671059430232024561
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
checksum verify failed on 38006784 found FD87CEC0 wanted E5015CC7
bytenr mismatch, want=38006784, have=671059430232024561
Error searching -5
Error copying data for ../rescue/cryptkey.dat
[-- Attachment #2: syslog.bz2 --]
[-- Type: application/x-bzip2, Size: 12662 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-07 21:46 Error while balance after convert from ext4 Николай Овчинников
@ 2015-07-08 0:18 ` Duncan
2015-07-08 13:56 ` Chris Murphy
1 sibling, 0 replies; 8+ messages in thread
From: Duncan @ 2015-07-08 0:18 UTC (permalink / raw
To: linux-btrfs
Николай Овчинников posted on Wed, 08 Jul 2015 00:46:48 +0300 as excerpted:
> While balancing after succsecful converting from ext4 filesystem moved
> into read-only mode and balance aborted.
> Before balance df says about 6Gb freespace.
> Now filesystem can be mounted only in recovery mode with 1 dir and about
> 20 zero-sized files.
> Is there a chance to recover these files?
FWIW, there have been several other reports of conversion not going so
well recently. The opinion of at least some users is that apparently,
btrfs has moved on from where it was when the conversion tool was built,
and the conversion tool hasn't kept up, causing problems with the
conversion in more cases, recently. (FWIW, I've never particularly
trusted filesystem conversion tools in the first place. As such, I
always had a strong preference for doing a fresh mkfs, then copying
things over from the old filesystem. That way, the old filesystem
becomes the backup you're restoring to a fresh filesystem from, and it
remains a backup, in case there are issues with the new filesystem, as
well as for all the other reasons, including fat-fingering, that valuing
the data is in practice defined as having a backup.)
Meanwhile, to answer your question, yes, the files can be recovered from
the backups you had if you valued the data. If you didn't have them, the
data obviously wasn't important, and you simply had a bit of help
clearing your cache of unimportant junk-files. =:^)
Seriously, the sysadmin's standard backup rule states that if you don't
have a backup, by definition, you value the data less than the time and
resources necessary to create the backup, no matter any claims to the
contrary. Further, for purposes of this rule a would-be backup that
hasn't been tested restorable isn't yet a backup as a backup isn't
complete until it's tested.
Of course, the fact that btrfs isn't yet fully stable means the rule
applies double, compared to its normal application to a fully stable
filesystem.
And of course, converting a filesystem without a backup is also something
that by definition is never done with data you actually care about, so at
/least/ double again (if not 4X on its own).
Which means the rule applied at 4X-8X strength. You REALLY had a backup
(and at 4-8X strength, probably more than one), or REALLY don't care
about the data.
But, there does remain a glimmer of hope in case you had no backup.
There's the btrfs restore tool you can try. I'm not sure how well it'll
work on an incompletely rebalanced after convert filesystem, but it's
worth a try. You will need somewhere else to store the restored files,
since it works with an unmounted filesystem without writing to it (so at
least it can't make the problem worse). If restore can't find a usable
root on its own, you can try using btrfs-find-root, and feed that (or
rather the corresponding bytenr) to btrfs restore using the -t option.
There's a somewhat dated but still useful writeup on the wiki about it.
https://btrfs.wiki.kernel.org/index.php/Restore
Note that generation and tranaction-id (transid) are different words for
the same thing, btrfs restore -t <bytenr> -l lists the various subtree
roots available from that root (the part of the writeup where it says
select the root with the most available trees in it), there's now a -D
dry-run option that will let you get at least some idea how many files
can be restored, and that the new -m (metadata, ownership/perms info) and
-S (symlinks) options allow restoring these, as they won't be restored by
default.
Again, I'm not sure how well restore will do with a fresh conversion
where the balance died like that, but it's worth a shot, and has a couple
times restored newer versions of files than my backups had, working well
for me, tho with a filesystem that was btrfs all along, not converted.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-07 21:46 Error while balance after convert from ext4 Николай Овчинников
2015-07-08 0:18 ` Duncan
@ 2015-07-08 13:56 ` Chris Murphy
2015-07-08 19:38 ` Николай Овчинников
1 sibling, 1 reply; 8+ messages in thread
From: Chris Murphy @ 2015-07-08 13:56 UTC (permalink / raw
To: Btrfs BTRFS
I've reproduced this bug with a brand new ext4 system, converted with
btrfs-progs 4.1 and a 4.1 kernel. The conversion is OK. And btrfs
check is ok after conversion, after removing the ext2_saved snapshot,
and after defrag. The implosion happens on balance and the fs goes
read only.
https://bugzilla.kernel.org/show_bug.cgi?id=101191
I will double post this as a new thread, since I've found a different
bug with the identical procedure and 4.2rc1.
Chris Murphy
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-08 13:56 ` Chris Murphy
@ 2015-07-08 19:38 ` Николай Овчинников
2015-07-08 23:32 ` Duncan
2015-07-09 0:37 ` Chris Murphy
0 siblings, 2 replies; 8+ messages in thread
From: Николай Овчинников @ 2015-07-08 19:38 UTC (permalink / raw
Cc: Btrfs BTRFS
What about btrfsck and btrfs-image errors?
2015-07-08 16:56 GMT+03:00 Chris Murphy <lists@colorremedies.com>:
> I've reproduced this bug with a brand new ext4 system, converted with
> btrfs-progs 4.1 and a 4.1 kernel. The conversion is OK. And btrfs
> check is ok after conversion, after removing the ext2_saved snapshot,
> and after defrag. The implosion happens on balance and the fs goes
> read only.
>
> https://bugzilla.kernel.org/show_bug.cgi?id=101191
>
> I will double post this as a new thread, since I've found a different
> bug with the identical procedure and 4.2rc1.
>
> Chris Murphy
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-08 19:38 ` Николай Овчинников
@ 2015-07-08 23:32 ` Duncan
2015-07-08 23:58 ` Николай Овчинников
2015-07-09 0:37 ` Chris Murphy
1 sibling, 1 reply; 8+ messages in thread
From: Duncan @ 2015-07-08 23:32 UTC (permalink / raw
To: linux-btrfs
Николай Овчинников posted
on Wed, 08 Jul 2015 22:38:20 +0300 as excerpted:
[Rearranged to standard quote, reply-in-context, format. It makes
replying and following replies on mailing lists FAR easier.]
> 2015-07-08 16:56 GMT+03:00 Chris Murphy <lists@colorremedies.com>:
>> I've reproduced this bug with a brand new ext4 system, converted with
>> btrfs-progs 4.1 and a 4.1 kernel. The conversion is OK. And btrfs check
>> is ok after conversion, after removing the ext2_saved snapshot, and
>> after defrag. The implosion happens on balance and the fs goes read
>> only.
> What about btrfsck and btrfs-image errors?
C Murphy mentioned btrfs check (saying it was fine after conversion).
Btrfsck is now the check subcommand under the primary btrfs tool, with
the standalone btrfsck deprecated and to be removed after a few
transition releases.
But he didn't cover btrfs-image. Valid point/question there.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-08 23:32 ` Duncan
@ 2015-07-08 23:58 ` Николай Овчинников
0 siblings, 0 replies; 8+ messages in thread
From: Николай Овчинников @ 2015-07-08 23:58 UTC (permalink / raw
Cc: Btrfs BTRFS
2015-07-09 2:32 GMT+03:00 Duncan <1i5t5.duncan@cox.net>:
> Николай Овчинников posted
> on Wed, 08 Jul 2015 22:38:20 +0300 as excerpted:
>
>> 2015-07-08 16:56 GMT+03:00 Chris Murphy <lists@colorremedies.com>:
>>> I've reproduced this bug with a brand new ext4 system, converted with
>>> btrfs-progs 4.1 and a 4.1 kernel. The conversion is OK. And btrfs check
>>> is ok after conversion, after removing the ext2_saved snapshot, and
>>> after defrag. The implosion happens on balance and the fs goes read
>>> only.
>
>> What about btrfsck and btrfs-image errors?
>
> C Murphy mentioned btrfs check (saying it was fine after conversion).
> Btrfsck is now the check subcommand under the primary btrfs tool, with
> the standalone btrfsck deprecated and to be removed after a few
> transition releases.
>
> But he didn't cover btrfs-image. Valid point/question there.
>
btrfs check have same error
root@ovchin-eee:/home/ovchin/btrfs-progs# gdb ./btrfs
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./btrfs...done.
(gdb) run check /dev/sda7
Starting program: /home/ovchin/btrfs-progs/btrfs check /dev/sda7
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Checking filesystem on /dev/sda7
UUID: 489fb2ac-7f1a-46da-9fb8-964e483c8081
checking extents
Program received signal SIGABRT, Aborted.
0x00007ffff6fc6107 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: Нет такого файла или каталога.
(gdb) bt
#0 0x00007ffff6fc6107 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff6fc74e8 in __GI_abort () at abort.c:89
#2 0x000000000042cd25 in add_tree_backref
(extent_cache=0x7fffffffe530, bytenr=7707766784,
parent=0, root=4, found_ref=1) at cmds-check.c:4421
#3 0x00000000004309a2 in add_root_to_pending (buf=0x6dcd00,
extent_cache=0x7fffffffe530,
pending=0x7fffffffe510, seen=0x7fffffffe520, nodes=0x7fffffffe4f0,
objectid=4)
at cmds-check.c:5961
#4 0x0000000000434abb in deal_root_from_list (list=0x7fffffffe200,
root=0x6eaa10,
bits=0x6f1170, bits_nr=1024, pending=0x7fffffffe510, seen=0x7fffffffe520,
reada=0x7fffffffe500, nodes=0x7fffffffe4f0, extent_cache=0x7fffffffe530,
chunk_cache=0x7fffffffe590, dev_cache=0x7fffffffe5a0,
block_group_cache=0x7fffffffe570,
dev_extent_cache=0x7fffffffe540) at cmds-check.c:7803
#5 0x0000000000434f95 in check_chunks_and_extents (root=0x6eaa10) at
cmds-check.c:7973
#6 0x00000000004381a0 in cmd_check (argc=1, argv=0x7fffffffe810) at
cmds-check.c:9402
#7 0x0000000000409dec in main (argc=2, argv=0x7fffffffe810) at btrfs.c:245
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-08 19:38 ` Николай Овчинников
2015-07-08 23:32 ` Duncan
@ 2015-07-09 0:37 ` Chris Murphy
2015-07-09 1:55 ` Chris Murphy
1 sibling, 1 reply; 8+ messages in thread
From: Chris Murphy @ 2015-07-09 0:37 UTC (permalink / raw
To: Николай Овчинников
Cc: Btrfs BTRFS
On Wed, Jul 8, 2015 at 1:38 PM, Николай Овчинников <nik.ovchin@gmail.com> wrote:
> What about btrfsck and btrfs-image errors?
btrfs check and btrfs-image both crash, which I consider as a separate
bug. And the crash doesn't produce useful info so I think it's a case
where I need to include either some gdb or valgrind info but I suck at
both and don't really know what to include in such a bug report.
The gist right now is that ext4 to btrfs conversion appears to be
broken, using defaults. I vaguely recall within the last, month (?), a
dev suggested using -n option to disable inlining of small files and
maybe that helped someone. I didn't test that.
--
Chris Murphy
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Error while balance after convert from ext4
2015-07-09 0:37 ` Chris Murphy
@ 2015-07-09 1:55 ` Chris Murphy
0 siblings, 0 replies; 8+ messages in thread
From: Chris Murphy @ 2015-07-09 1:55 UTC (permalink / raw
Cc: Николай Овчинников,
Btrfs BTRFS
I've filed a bug for the btrfs check and btrfs-image crashing/failure
I'm getting. They may be unrelated but I filed them as one bug here.
https://bugzilla.kernel.org/show_bug.cgi?id=101221
Chris Murphy
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2015-07-09 1:56 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-07-07 21:46 Error while balance after convert from ext4 Николай Овчинников
2015-07-08 0:18 ` Duncan
2015-07-08 13:56 ` Chris Murphy
2015-07-08 19:38 ` Николай Овчинников
2015-07-08 23:32 ` Duncan
2015-07-08 23:58 ` Николай Овчинников
2015-07-09 0:37 ` Chris Murphy
2015-07-09 1:55 ` Chris Murphy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.