* trim not working and irreparable errors from btrfsck
@ 2015-06-16 14:18 Christian
2015-06-16 21:14 ` Chris Murphy
0 siblings, 1 reply; 19+ messages in thread
From: Christian @ 2015-06-16 14:18 UTC (permalink / raw)
To: linux-btrfs
Hi,
I have a btrfs partions on my laptop containing / and /home. Recently I
noticed that trim (fstrim) didn't work anymore. It always tells me there
nothing to trim which can not be correct. I then tried to to run btrfsck
--repair and this is the result:
ubuntu@ubuntu:~$ sudo btrfsck /dev/sda3 --repair
enabling repair mode
Fixed 0 roots.
Checking filesystem on /dev/sda3
UUID: 3d52dc93-c89f-453f-965d-8601d11e7710
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
root 257 inode 353819 errors 400, nbytes wrong
root 2260 inode 353819 errors 400, nbytes wrong
root 2262 inode 353819 errors 400, nbytes wrong
root 2264 inode 353819 errors 400, nbytes wrong
root 2266 inode 353819 errors 400, nbytes wrong
root 2268 inode 353819 errors 400, nbytes wrong
root 2270 inode 353819 errors 400, nbytes wrong
root 2273 inode 353819 errors 400, nbytes wrong
root 2275 inode 353819 errors 400, nbytes wrong
root 2277 inode 353819 errors 400, nbytes wrong
root 2279 inode 353819 errors 400, nbytes wrong
root 2281 inode 353819 errors 400, nbytes wrong
root 2283 inode 353819 errors 400, nbytes wrong
root 2285 inode 353819 errors 400, nbytes wrong
root 2287 inode 353819 errors 400, nbytes wrong
root 2289 inode 353819 errors 400, nbytes wrong
root 2291 inode 353819 errors 400, nbytes wrong
root 2293 inode 353819 errors 400, nbytes wrong
root 2295 inode 353819 errors 400, nbytes wrong
root 2297 inode 353819 errors 400, nbytes wrong
root 2299 inode 353819 errors 400, nbytes wrong
root 2301 inode 353819 errors 400, nbytes wrong
root 2303 inode 353819 errors 400, nbytes wrong
root 2305 inode 353819 errors 400, nbytes wrong
root 2317 inode 353819 errors 400, nbytes wrong
root 2320 inode 353819 errors 400, nbytes wrong
root 2326 inode 353819 errors 400, nbytes wrong
root 2556 inode 353819 errors 400, nbytes wrong
root 2574 inode 353819 errors 400, nbytes wrong
root 2592 inode 353819 errors 400, nbytes wrong
root 2601 inode 353819 errors 400, nbytes wrong
root 2617 inode 353819 errors 400, nbytes wrong
root 2620 inode 353819 errors 400, nbytes wrong
root 2621 inode 353819 errors 400, nbytes wrong
root 2624 inode 353819 errors 400, nbytes wrong
root 2625 inode 353819 errors 400, nbytes wrong
root 2626 inode 353819 errors 400, nbytes wrong
root 2627 inode 353819 errors 400, nbytes wrong
root 2628 inode 353819 errors 400, nbytes wrong
root 2629 inode 353819 errors 400, nbytes wrong
root 2630 inode 353819 errors 400, nbytes wrong
root 2631 inode 353819 errors 400, nbytes wrong
root 2632 inode 353819 errors 400, nbytes wrong
found 49406146650 bytes used err is 1
total csum bytes: 117817456
total tree bytes: 792248320
total fs tree bytes: 604520448
total extent tree bytes: 52101120
btree space waste bytes: 180897891
file data blocks allocated: 195314843648
referenced 135991529472
Btrfs v3.17
I get the exact same result every time I run btrfsck --repair. If the
file system irreparable or is there something I can do to save it?
--
//Christian
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-16 14:18 trim not working and irreparable errors from btrfsck Christian
@ 2015-06-16 21:14 ` Chris Murphy
[not found] ` <55816E7B.5040905@gmail.com>
0 siblings, 1 reply; 19+ messages in thread
From: Chris Murphy @ 2015-06-16 21:14 UTC (permalink / raw)
To: Christian; +Cc: Btrfs BTRFS
I experienced the errors 400 problem on an HDD and how I fixed it is
in comment 2 in this bug:
https://bugzilla.kernel.org/show_bug.cgi?id=90071
The gist is to find out what file is affected by finding the
path/filename for the inode with the error and then deleting it.
You'll need to recover the file from backups. Once you do this the
problem is resolved, the file system itself is OK. (At least it was in
my case.)
I don't know that the source of the problem you're having has anything
to do with trim, but I recommend not using discard or fstrim at all
until you isolate what's causing it. There are some trim bugs that
have been fixed in newer kernels that sound like the problem you're
having; and there are definitely some SSDs out there with trim
problems where the wrong sectors get trimmed. It shows up as files
with 512 byte holes of zeros in them. Of course, that's when the bug
affects data. If it affects metadata it can obliterate the file system
beyond repair.
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
[not found] ` <55816E7B.5040905@gmail.com>
@ 2015-06-17 14:22 ` Chris Murphy
2015-06-17 14:33 ` Christian
0 siblings, 1 reply; 19+ messages in thread
From: Chris Murphy @ 2015-06-17 14:22 UTC (permalink / raw)
To: Christian Dysthe; +Cc: Btrfs BTRFS
On Wed, Jun 17, 2015 at 6:56 AM, Christian Dysthe <cdysthe@gmail.com> wrote:
> Hi,
>
> Sorry for asking more about this. I'm not a developer but trying to learn.
> In my case I get several errors like this one:
>
> root 2625 inode 353819 errors 400, nbytes wrong
>
> Is it inode 353819 I should focus on and what is the number after "root", in
> this case 2625?
I'm going to guess it's tree root 2625, which is the same thing as fs
tree, which is the same thing as subvolume. Each subvolume has its own
inodes. So on a given Btrfs volume, an inode number can exist more
than once, but in separate subvolumes. When you use btrfs inspect
inode it will list all files with that inode number, but only the one
in subvol ID 2625 is what you care about deleting and replacing.
--
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-17 14:22 ` Chris Murphy
@ 2015-06-17 14:33 ` Christian
2015-06-17 15:28 ` Chris Murphy
2015-06-18 2:20 ` Paul Jones
0 siblings, 2 replies; 19+ messages in thread
From: Christian @ 2015-06-17 14:33 UTC (permalink / raw)
To: linux-btrfs
On 06/17/2015 10:22 AM, Chris Murphy wrote:
> On Wed, Jun 17, 2015 at 6:56 AM, Christian Dysthe <cdysthe@gmail.com> wrote:
>> Hi,
>>
>> Sorry for asking more about this. I'm not a developer but trying to learn.
>> In my case I get several errors like this one:
>>
>> root 2625 inode 353819 errors 400, nbytes wrong
>>
>> Is it inode 353819 I should focus on and what is the number after "root", in
>> this case 2625?
>
> I'm going to guess it's tree root 2625, which is the same thing as fs
> tree, which is the same thing as subvolume. Each subvolume has its own
> inodes. So on a given Btrfs volume, an inode number can exist more
> than once, but in separate subvolumes. When you use btrfs inspect
> inode it will list all files with that inode number, but only the one
> in subvol ID 2625 is what you care about deleting and replacing.
>
Thanks! Deleting the file for that inode took care of it. No more
errors. Restored it from a backup.
However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
another problem. Is there a way to check if trim works?
--
//Christian
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-17 14:33 ` Christian
@ 2015-06-17 15:28 ` Chris Murphy
2015-06-17 15:40 ` Christian
2015-06-18 2:20 ` Paul Jones
1 sibling, 1 reply; 19+ messages in thread
From: Chris Murphy @ 2015-06-17 15:28 UTC (permalink / raw)
To: Christian; +Cc: Btrfs BTRFS
On Wed, Jun 17, 2015 at 8:33 AM, Christian <cdysthe@gmail.com> wrote:
> On 06/17/2015 10:22 AM, Chris Murphy wrote:
>>
>> On Wed, Jun 17, 2015 at 6:56 AM, Christian Dysthe <cdysthe@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> Sorry for asking more about this. I'm not a developer but trying to
>>> learn.
>>> In my case I get several errors like this one:
>>>
>>> root 2625 inode 353819 errors 400, nbytes wrong
>>>
>>> Is it inode 353819 I should focus on and what is the number after "root",
>>> in
>>> this case 2625?
>>
>>
>> I'm going to guess it's tree root 2625, which is the same thing as fs
>> tree, which is the same thing as subvolume. Each subvolume has its own
>> inodes. So on a given Btrfs volume, an inode number can exist more
>> than once, but in separate subvolumes. When you use btrfs inspect
>> inode it will list all files with that inode number, but only the one
>> in subvol ID 2625 is what you care about deleting and replacing.
>>
> Thanks! Deleting the file for that inode took care of it. No more errors.
> Restored it from a backup.
>
> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
> another problem. Is there a way to check if trim works?
That sounds like maybe your SSD is blacklisted for trim, is all I can
think of. So trim shouldn't be the cause of the problem if it's being
blacklisted. The recent problems appear to be around newer SSDs that
support queue trim and newer kernels that issue queued trim. There
have been some patches related to trim to the kernel, but the
existence of blacklisting and claims of bugs in firmware make it
difficult to test and isolate.
http://techreport.com/news/28473/some-samsung-ssds-may-suffer-from-a-buggy-trim-implementation
--
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-17 15:28 ` Chris Murphy
@ 2015-06-17 15:40 ` Christian
2015-06-17 17:17 ` Austin S Hemmelgarn
0 siblings, 1 reply; 19+ messages in thread
From: Christian @ 2015-06-17 15:40 UTC (permalink / raw)
To: linux-btrfs
On 06/17/2015 11:28 AM, Chris Murphy wrote:
>>
>> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
>> another problem. Is there a way to check if trim works?
>
> That sounds like maybe your SSD is blacklisted for trim, is all I can
> think of. So trim shouldn't be the cause of the problem if it's being
> blacklisted. The recent problems appear to be around newer SSDs that
> support queue trim and newer kernels that issue queued trim. There
> have been some patches related to trim to the kernel, but the
> existence of blacklisting and claims of bugs in firmware make it
> difficult to test and isolate.
>
> http://techreport.com/news/28473/some-samsung-ssds-may-suffer-from-a-buggy-trim-implementation
>
This is an Intel SSD in a Lenovo Thinkpad X1 Carbon. Trim worked until a
few weeks ago and still works for my small ext4 boot partition (just ran
it to check). I will keep looking for a solution. Thanks!
--
//Christian
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-17 15:40 ` Christian
@ 2015-06-17 17:17 ` Austin S Hemmelgarn
2015-06-18 5:25 ` Duncan
2015-06-20 14:11 ` Lutz Euler
0 siblings, 2 replies; 19+ messages in thread
From: Austin S Hemmelgarn @ 2015-06-17 17:17 UTC (permalink / raw)
To: Christian, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]
On 2015-06-17 11:40, Christian wrote:
> On 06/17/2015 11:28 AM, Chris Murphy wrote:
>
>>>
>>> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
>>> another problem. Is there a way to check if trim works?
>>
>> That sounds like maybe your SSD is blacklisted for trim, is all I can
>> think of. So trim shouldn't be the cause of the problem if it's being
>> blacklisted. The recent problems appear to be around newer SSDs that
>> support queue trim and newer kernels that issue queued trim. There
>> have been some patches related to trim to the kernel, but the
>> existence of blacklisting and claims of bugs in firmware make it
>> difficult to test and isolate.
>>
>> http://techreport.com/news/28473/some-samsung-ssds-may-suffer-from-a-buggy-trim-implementation
>>
>>
> This is an Intel SSD in a Lenovo Thinkpad X1 Carbon. Trim worked until a
> few weeks ago and still works for my small ext4 boot partition (just ran
> it to check). I will keep looking for a solution. Thanks!
>
I'm seeing the same issue here, but with a Crucial brand SSD. Somewhat
interestingly, I don't see any issues like this with BTRFS on top of
LVM's thin-provisioning volumes, or with any other filesystems, so I
think it has something to do with how BTRFS is reporting unused space or
how it is submitting the discard requests.
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: trim not working and irreparable errors from btrfsck
2015-06-17 14:33 ` Christian
2015-06-17 15:28 ` Chris Murphy
@ 2015-06-18 2:20 ` Paul Jones
2015-06-18 4:15 ` Chris Murphy
1 sibling, 1 reply; 19+ messages in thread
From: Paul Jones @ 2015-06-18 2:20 UTC (permalink / raw)
To: Christian, linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1801 bytes --]
> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Christian
> Sent: Thursday, 18 June 2015 12:34 AM
> To: linux-btrfs@vger.kernel.org
> Subject: Re: trim not working and irreparable errors from btrfsck
>
> On 06/17/2015 10:22 AM, Chris Murphy wrote:
> > On Wed, Jun 17, 2015 at 6:56 AM, Christian Dysthe <cdysthe@gmail.com>
> wrote:
> >> Hi,
> >>
> >> Sorry for asking more about this. I'm not a developer but trying to learn.
> >> In my case I get several errors like this one:
> >>
> >> root 2625 inode 353819 errors 400, nbytes wrong
> >>
> >> Is it inode 353819 I should focus on and what is the number after
> >> "root", in this case 2625?
> >
> > I'm going to guess it's tree root 2625, which is the same thing as fs
> > tree, which is the same thing as subvolume. Each subvolume has its own
> > inodes. So on a given Btrfs volume, an inode number can exist more
> > than once, but in separate subvolumes. When you use btrfs inspect
> > inode it will list all files with that inode number, but only the one
> > in subvol ID 2625 is what you care about deleting and replacing.
> >
> Thanks! Deleting the file for that inode took care of it. No more errors.
> Restored it from a backup.
>
> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be another
> problem. Is there a way to check if trim works?
I've got the same problem. I've got 2 SSDs with 2 partitions in RAID1, fstrim always works on the 2nd partition but not the first. There are no errors on either filesystem that I know of, but the first one is root so I can't take it offline to run btrfs check.
Paul.
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-18 2:20 ` Paul Jones
@ 2015-06-18 4:15 ` Chris Murphy
2015-06-18 4:19 ` Chris Murphy
0 siblings, 1 reply; 19+ messages in thread
From: Chris Murphy @ 2015-06-18 4:15 UTC (permalink / raw)
To: Paul Jones; +Cc: Christian, linux-btrfs@vger.kernel.org
File a new bug at bugzilla.kernel.org describing this problem. Include
make/model of all involved SSDs, which you can get from smartctl or
hdparm. And then do a strace fstrim on the working and non-working
volumes, saving the output to separate files and attaching them to the
bug report. And then it's probably best to create a new list thread to
post the URL for the bug, since this thread is really about two
problems that may not be related.
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-18 4:15 ` Chris Murphy
@ 2015-06-18 4:19 ` Chris Murphy
0 siblings, 0 replies; 19+ messages in thread
From: Chris Murphy @ 2015-06-18 4:19 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
And you might as well just attach a full dmesg to the bug report too.
Who knows there might be something buried in there that's useful. The
cleanest approach is to reboot and then reproduce all of this with
fstrim on each volume, then capture the dmesg to a file. Then do the
strace fstrim for each volume. But a bunch of kernel message clutter
may be better than no messages at all.
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-17 17:17 ` Austin S Hemmelgarn
@ 2015-06-18 5:25 ` Duncan
2015-08-14 15:11 ` Jeff Mahoney
2015-06-20 14:11 ` Lutz Euler
1 sibling, 1 reply; 19+ messages in thread
From: Duncan @ 2015-06-18 5:25 UTC (permalink / raw)
To: linux-btrfs
Austin S Hemmelgarn posted on Wed, 17 Jun 2015 13:17:22 -0400 as
excerpted:
> On 2015-06-17 11:40, Christian wrote:
>> On 06/17/2015 11:28 AM, Chris Murphy wrote:
>>
>>
>>>> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
>>>> another problem. Is there a way to check if trim works?
>>>
>>> That sounds like maybe your SSD is blacklisted for trim, is all I can
>>> think of. So trim shouldn't be the cause of the problem if it's being
>>> blacklisted. The recent problems appear to be around newer SSDs that
>>> support queue trim and newer kernels that issue queued trim. There
>>> have been some patches related to trim to the kernel, but the
>>> existence of blacklisting and claims of bugs in firmware make it
>>> difficult to test and isolate.
>>>
>>> http://techreport.com/news/28473/some-samsung-ssds-may-suffer-from-a-
buggy-trim-implementation
>>>
>>>
>> This is an Intel SSD in a Lenovo Thinkpad X1 Carbon. Trim worked until
>> a few weeks ago and still works for my small ext4 boot partition (just
>> ran it to check). I will keep looking for a solution. Thanks!
>>
> I'm seeing the same issue here, but with a Crucial brand SSD. Somewhat
> interestingly, I don't see any issues like this with BTRFS on top of
> LVM's thin-provisioning volumes, or with any other filesystems, so I
> think it has something to do with how BTRFS is reporting unused space or
> how it is submitting the discard requests.
FWIW, there's a current btrfs patch in progress that relates to problems
with btrfs trim.
But while I do have SSDs, I purposefully overprovisioned them by nearly
100% (IOW I partitioned only about 55%, the rest is entirely unused), so
trim isn't as critical here as it is for many. I don't use the discard
mount option, and have a systemd timer job setup to automate my fstrims
and don't worry about the output too much, so I haven't been following
the patch progress /that/ closely.
But I do know that recent kernel btrfs trims (either fstrim or discard
mount option triggered) haven't been working as originally intended due
to some bug, and this patch is supposed to fix it.
I'd thus conclude that you're very likely hitting this known issue, and
that either for 4.1 or 4.2 (again, I'm not following progress that
closely, and don't remember for sure if it's in 4.1, altho I've been
running the rcs since rc6 or so), the problem should be fixed as that
patch gets into mainline.
Anyone wishing to investigate further can of course check the list (and/
or possibly the kernel's git log) for discard/trim related patches and
follow the progress once found.
... Actually, just checked myself. Looks like the patches were first
posted on March 30 @ 15:12:17 -0400 or so (that's the time for one of
them). There's one for the discard mount option, and another for FITRIM
(which may or may not be a typo for FSTRIM, I'm not actually sure). Jeff
Mahoney <jeffm@suse.com> author. That should be enough to find the
threads. And I don't see the patches in the late 4.1-rc I'm running so
either my git log search foo is bad or it'll be (at least) 4.2.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: trim not working and irreparable errors from btrfsck
2015-06-17 17:17 ` Austin S Hemmelgarn
2015-06-18 5:25 ` Duncan
@ 2015-06-20 14:11 ` Lutz Euler
2015-06-21 7:21 ` Paul Jones
1 sibling, 1 reply; 19+ messages in thread
From: Lutz Euler @ 2015-06-20 14:11 UTC (permalink / raw)
To: Christian, Paul Jones, Austin S Hemmelgarn; +Cc: linux-btrfs@vger.kernel.org
Hi Christian, Paul and Austin,
Christian wrote:
> However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
> another problem. Is there a way to check if trim works?
Paul wrote:
> I've got the same problem. I've got 2 SSDs with 2 partitions in RAID1,
> fstrim always works on the 2nd partition but not the first. There are
> no errors on either filesystem that I know of, but the first one is
> root so I can't take it offline to run btrfs check.
Austin wrote:
> I'm seeing the same issue here, but with a Crucial brand SSD.
> Somewhat interestingly, I don't see any issues like this with BTRFS on
> top of LVM's thin-provisioning volumes, or with any other filesystems,
> so I think it has something to do with how BTRFS is reporting unused
> space or how it is submitting the discard requests.
Probably you all suffer from the same problem I had a few years ago.
It is a bug in how btrfs implements fstrim.
To check whether you are a victim of this bug simply run:
# btrfs-debug-tree /dev/whatever | grep 'FIRST_CHUNK_TREE CHUNK_ITEM'
where /dev/whatever is a device of your filesystem, and interrupt after
the first several output lines with C-c. (Officially the filesystem
should be unmounted when running btrfs-debug-tree, but that is not
necessary as we only read from it and the relevant data doesn't change
very often.)
You get something like:
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 12947816448)
item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 14021558272)
...
(This output is from an old version of btrfs-progs. I understand newer
version are more verbose, but you should nevertheless easily be able to
interpret the output).
If the first number different from 0 (here, the 12947816448) is larger
than the sum of the sizes of the devices the filesystem consists of,
bingo.
This has been discussed already in the past and there is a patch.
Please see for the patch:
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg40618.html
and for the background:
http://comments.gmane.org/gmane.comp.file-systems.btrfs/15597
Kind regards,
Lutz Euler
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: trim not working and irreparable errors from btrfsck
2015-06-20 14:11 ` Lutz Euler
@ 2015-06-21 7:21 ` Paul Jones
2015-08-13 9:23 ` Marc Joliet
0 siblings, 1 reply; 19+ messages in thread
From: Paul Jones @ 2015-06-21 7:21 UTC (permalink / raw)
To: Lutz Euler, Christian, Austin S Hemmelgarn; +Cc: linux-btrfs@vger.kernel.org
> -----Original Message-----
> From: Lutz Euler [mailto:lutz.euler@freenet.de]
> Sent: Sunday, 21 June 2015 12:11 AM
> To: Christian; Paul Jones; Austin S Hemmelgarn
> Cc: linux-btrfs@vger.kernel.org
> Subject: RE: trim not working and irreparable errors from btrfsck
>
> Hi Christian, Paul and Austin,
>
> Christian wrote:
> > However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
> > another problem. Is there a way to check if trim works?
>
> Paul wrote:
> > I've got the same problem. I've got 2 SSDs with 2 partitions in RAID1,
> > fstrim always works on the 2nd partition but not the first. There are
> > no errors on either filesystem that I know of, but the first one is
> > root so I can't take it offline to run btrfs check.
>
> Austin wrote:
> > I'm seeing the same issue here, but with a Crucial brand SSD.
> > Somewhat interestingly, I don't see any issues like this with BTRFS on
> > top of LVM's thin-provisioning volumes, or with any other filesystems,
> > so I think it has something to do with how BTRFS is reporting unused
> > space or how it is submitting the discard requests.
>
> Probably you all suffer from the same problem I had a few years ago.
> It is a bug in how btrfs implements fstrim.
>
> To check whether you are a victim of this bug simply run:
>
> # btrfs-debug-tree /dev/whatever | grep 'FIRST_CHUNK_TREE
> CHUNK_ITEM'
>
> where /dev/whatever is a device of your filesystem, and interrupt after the
> first several output lines with C-c. (Officially the filesystem should be
> unmounted when running btrfs-debug-tree, but that is not necessary as we
> only read from it and the relevant data doesn't change very often.)
>
> You get something like:
>
> item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
> item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 12947816448)
> item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 14021558272)
> ...
>
> (This output is from an old version of btrfs-progs. I understand newer version
> are more verbose, but you should nevertheless easily be able to interpret
> the output).
>
> If the first number different from 0 (here, the 12947816448) is larger than the
> sum of the sizes of the devices the filesystem consists of, bingo.
>
> This has been discussed already in the past and there is a patch.
>
> Please see for the patch:
> http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg40618.html
>
> and for the background:
> http://comments.gmane.org/gmane.comp.file-systems.btrfs/15597
>
> Kind regards,
>
> Lutz Euler
I tried the test and the numbers I was getting seemed reasonable, however I went ahead and applied the patch anyway. Trim now works correctly!
Thanks,
Paul.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-21 7:21 ` Paul Jones
@ 2015-08-13 9:23 ` Marc Joliet
2015-08-13 23:14 ` Chris Murphy
0 siblings, 1 reply; 19+ messages in thread
From: Marc Joliet @ 2015-08-13 9:23 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 4767 bytes --]
Am Sun, 21 Jun 2015 07:21:03 +0000
schrieb Paul Jones <paul@pauljones.id.au>:
> > -----Original Message-----
> > From: Lutz Euler [mailto:lutz.euler@freenet.de]
> > Sent: Sunday, 21 June 2015 12:11 AM
> > To: Christian; Paul Jones; Austin S Hemmelgarn
> > Cc: linux-btrfs@vger.kernel.org
> > Subject: RE: trim not working and irreparable errors from btrfsck
> >
> > Hi Christian, Paul and Austin,
> >
> > Christian wrote:
> > > However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be
> > > another problem. Is there a way to check if trim works?
> >
> > Paul wrote:
> > > I've got the same problem. I've got 2 SSDs with 2 partitions in RAID1,
> > > fstrim always works on the 2nd partition but not the first. There are
> > > no errors on either filesystem that I know of, but the first one is
> > > root so I can't take it offline to run btrfs check.
> >
> > Austin wrote:
> > > I'm seeing the same issue here, but with a Crucial brand SSD.
> > > Somewhat interestingly, I don't see any issues like this with BTRFS on
> > > top of LVM's thin-provisioning volumes, or with any other filesystems,
> > > so I think it has something to do with how BTRFS is reporting unused
> > > space or how it is submitting the discard requests.
> >
> > Probably you all suffer from the same problem I had a few years ago.
> > It is a bug in how btrfs implements fstrim.
> >
> > To check whether you are a victim of this bug simply run:
> >
> > # btrfs-debug-tree /dev/whatever | grep 'FIRST_CHUNK_TREE
> > CHUNK_ITEM'
> >
> > where /dev/whatever is a device of your filesystem, and interrupt after the
> > first several output lines with C-c. (Officially the filesystem should be
> > unmounted when running btrfs-debug-tree, but that is not necessary as we
> > only read from it and the relevant data doesn't change very often.)
> >
> > You get something like:
> >
> > item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
> > item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 12947816448)
> > item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 14021558272)
> > ...
> >
> > (This output is from an old version of btrfs-progs. I understand newer version
> > are more verbose, but you should nevertheless easily be able to interpret
> > the output).
> >
> > If the first number different from 0 (here, the 12947816448) is larger than the
> > sum of the sizes of the devices the filesystem consists of, bingo.
> >
> > This has been discussed already in the past and there is a patch.
> >
> > Please see for the patch:
> > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg40618.html
> >
> > and for the background:
> > http://comments.gmane.org/gmane.comp.file-systems.btrfs/15597
> >
> > Kind regards,
> >
> > Lutz Euler
>
> I tried the test and the numbers I was getting seemed reasonable, however I went ahead and applied the patch anyway. Trim now works correctly!
>
> Thanks,
> Paul.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
Speaking as a user, since "fstrim -av" still always outputs 0 bytes trimmed
on my system: what's the status of this? Did anybody ever file a bug report?
There was also that other thread, "fstrim not working on one of three BTRFS
filesystems", that also never went anywhere.
I take it from this that my SSD has been running untrimmed for quite a while
now?
(FWIW, queued trim is blocked by my kernel (it's "forced_unqueued"), but fstrim
should still start an unqueued trim, right?)
# uname -a
Linux thetick 4.1.4-gentoo #1 SMP PREEMPT Tue Aug 4 21:58:41 CEST 2015 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux
# btrfs --version
btrfs-progs v4.1.2
# btrfs filesystem show
Label: 'MARCEC_ROOT' uuid: 0267d8b3-a074-460a-832d-5d5fd36bae64
Total devices 1 FS bytes used 56.59GiB
devid 1 size 107.79GiB used 69.03GiB path /dev/sda1
Label: 'MARCEC_STORAGE' uuid: 472c9290-3ff2-4096-9c47-0612d3a52cef
Total devices 2 FS bytes used 597.75GiB
devid 1 size 931.51GiB used 600.03GiB path /dev/sdc
devid 2 size 931.51GiB used 600.03GiB path /dev/sdb
Label: 'MARCEC_BACKUP' uuid: f97b3cda-15e8-418b-bb9b-235391ef2a38
Total devices 1 FS bytes used 807.59GiB
devid 1 size 976.56GiB used 837.06GiB path /dev/sdd2
btrfs-progs v4.1.2
# btrfs filesystem df /
Data, single: total=65.00GiB, used=54.83GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=4.00GiB, used=1.76GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
Greetings
--
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-08-13 9:23 ` Marc Joliet
@ 2015-08-13 23:14 ` Chris Murphy
2015-08-14 8:05 ` Marc Joliet
0 siblings, 1 reply; 19+ messages in thread
From: Chris Murphy @ 2015-08-13 23:14 UTC (permalink / raw)
To: Btrfs BTRFS
On Thu, Aug 13, 2015 at 3:23 AM, Marc Joliet <marcec@gmx.de> wrote:
> Speaking as a user, since "fstrim -av" still always outputs 0 bytes trimmed
> on my system: what's the status of this? Did anybody ever file a bug report?
Since I'm not having this problem with my SSD, I'm not in a position
to provide any meaningful information for such a report.
The bug should whether this problem is reproducible with ext4 and XFS
on the same device, and the complete details of the stacking (if this
is not the full device or partition of it; e.g. if LVM, md, or
encryption is between fs and physical device). And also the bug should
include full dmesg as attachment, and strace of the fstrim command
that results in 0 bytes trimmed. And probably separate bugs for each
make/model of SSD, with the bug including make/model and firmware
version.
Right now I think there's no status because a.) no bug report and b.)
not enough information.
--
Chris Murphy
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-08-13 23:14 ` Chris Murphy
@ 2015-08-14 8:05 ` Marc Joliet
2015-08-14 8:15 ` Marc Joliet
2015-08-14 10:51 ` Paul Jones
0 siblings, 2 replies; 19+ messages in thread
From: Marc Joliet @ 2015-08-14 8:05 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]
Am Thu, 13 Aug 2015 17:14:36 -0600
schrieb Chris Murphy <lists@colorremedies.com>:
> On Thu, Aug 13, 2015 at 3:23 AM, Marc Joliet <marcec@gmx.de> wrote:
>
> > Speaking as a user, since "fstrim -av" still always outputs 0 bytes trimmed
> > on my system: what's the status of this? Did anybody ever file a bug report?
>
> Since I'm not having this problem with my SSD, I'm not in a position
> to provide any meaningful information for such a report.
>
> The bug should whether this problem is reproducible with ext4 and XFS
> on the same device, and the complete details of the stacking (if this
> is not the full device or partition of it; e.g. if LVM, md, or
> encryption is between fs and physical device). And also the bug should
> include full dmesg as attachment, and strace of the fstrim command
> that results in 0 bytes trimmed. And probably separate bugs for each
> make/model of SSD, with the bug including make/model and firmware
> version.
>
> Right now I think there's no status because a.) no bug report and b.)
> not enough information.
I was mainly asking because apparently there *is* a patch that helps some
people affected by this, but nobody ever commented on it. Perhaps there's a
reason for that, but I found it curious. (I see now that it was submitted in
early January, in the thread "[PATCH V2] Btrfs: really fix trim 0 bytes after a
device delete".)
I can open a bug (I mean, that's part of being a user of btrfs at this stage),
I'm just surprised that nobody else has.
BTW, is there a way to tell if the "discard" mount option does anything? I'm
curious about whether it could behave differently.
--
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-08-14 8:05 ` Marc Joliet
@ 2015-08-14 8:15 ` Marc Joliet
2015-08-14 10:51 ` Paul Jones
1 sibling, 0 replies; 19+ messages in thread
From: Marc Joliet @ 2015-08-14 8:15 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 373 bytes --]
Am Fri, 14 Aug 2015 10:05:55 +0200
schrieb Marc Joliet <marcec@gmx.de>:
> (I mean, that's part of being a user of btrfs at this stage)
I meant *being prepared* to file a bug report, not that one constantly has to
file bug reports :) .
--
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: trim not working and irreparable errors from btrfsck
2015-08-14 8:05 ` Marc Joliet
2015-08-14 8:15 ` Marc Joliet
@ 2015-08-14 10:51 ` Paul Jones
1 sibling, 0 replies; 19+ messages in thread
From: Paul Jones @ 2015-08-14 10:51 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Marc Joliet
> Sent: Friday, 14 August 2015 6:06 PM
> To: linux-btrfs@vger.kernel.org
> Subject: Re: trim not working and irreparable errors from btrfsck
>
> Am Thu, 13 Aug 2015 17:14:36 -0600
> schrieb Chris Murphy <lists@colorremedies.com>:
>
> >
> > Right now I think there's no status because a.) no bug report and b.)
> > not enough information.
>
> I was mainly asking because apparently there *is* a patch that helps some
> people affected by this, but nobody ever commented on it. Perhaps there's
> a reason for that, but I found it curious. (I see now that it was submitted in
> early January, in the thread "[PATCH V2] Btrfs: really fix trim 0 bytes after a
> device delete".)
>
> I can open a bug (I mean, that's part of being a user of btrfs at this stage), I'm
> just surprised that nobody else has.
I have to use that patch on one of my systems. I just assumed it was never merged because it wasn't quite ready yet. It seems to work fine for me though.
Paul.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: trim not working and irreparable errors from btrfsck
2015-06-18 5:25 ` Duncan
@ 2015-08-14 15:11 ` Jeff Mahoney
0 siblings, 0 replies; 19+ messages in thread
From: Jeff Mahoney @ 2015-08-14 15:11 UTC (permalink / raw)
To: Duncan, linux-btrfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 6/18/15 1:25 AM, Duncan wrote:
> Austin S Hemmelgarn posted on Wed, 17 Jun 2015 13:17:22 -0400 as
> excerpted:
>
>> On 2015-06-17 11:40, Christian wrote:
>>> On 06/17/2015 11:28 AM, Chris Murphy wrote:
>>>
>>>
>>>>> However, fstrim still gives me "0 B (0 bytes) trimmed, so
>>>>> that may be another problem. Is there a way to check if
>>>>> trim works?
>>>>
>>>> That sounds like maybe your SSD is blacklisted for trim, is
>>>> all I can think of. So trim shouldn't be the cause of the
>>>> problem if it's being blacklisted. The recent problems appear
>>>> to be around newer SSDs that support queue trim and newer
>>>> kernels that issue queued trim. There have been some patches
>>>> related to trim to the kernel, but the existence of
>>>> blacklisting and claims of bugs in firmware make it difficult
>>>> to test and isolate.
>>>>
>>>> http://techreport.com/news/28473/some-samsung-ssds-may-suffer-from-
a-
>
>>>>
buggy-trim-implementation
>>>>
>>>>
>>> This is an Intel SSD in a Lenovo Thinkpad X1 Carbon. Trim
>>> worked until a few weeks ago and still works for my small ext4
>>> boot partition (just ran it to check). I will keep looking for
>>> a solution. Thanks!
>>>
>> I'm seeing the same issue here, but with a Crucial brand SSD.
>> Somewhat interestingly, I don't see any issues like this with
>> BTRFS on top of LVM's thin-provisioning volumes, or with any
>> other filesystems, so I think it has something to do with how
>> BTRFS is reporting unused space or how it is submitting the
>> discard requests.
>
> FWIW, there's a current btrfs patch in progress that relates to
> problems with btrfs trim.
>
> But while I do have SSDs, I purposefully overprovisioned them by
> nearly 100% (IOW I partitioned only about 55%, the rest is entirely
> unused), so trim isn't as critical here as it is for many. I don't
> use the discard mount option, and have a systemd timer job setup to
> automate my fstrims and don't worry about the output too much, so I
> haven't been following the patch progress /that/ closely.
>
> But I do know that recent kernel btrfs trims (either fstrim or
> discard mount option triggered) haven't been working as originally
> intended due to some bug, and this patch is supposed to fix it.
>
> I'd thus conclude that you're very likely hitting this known issue,
> and that either for 4.1 or 4.2 (again, I'm not following progress
> that closely, and don't remember for sure if it's in 4.1, altho
> I've been running the rcs since rc6 or so), the problem should be
> fixed as that patch gets into mainline.
>
> Anyone wishing to investigate further can of course check the list
> (and/ or possibly the kernel's git log) for discard/trim related
> patches and follow the progress once found.
>
> ... Actually, just checked myself. Looks like the patches were
> first posted on March 30 @ 15:12:17 -0400 or so (that's the time
> for one of them). There's one for the discard mount option, and
> another for FITRIM (which may or may not be a typo for FSTRIM, I'm
> not actually sure). Jeff Mahoney <jeffm@suse.com> author. That
> should be enough to find the threads. And I don't see the patches
> in the late 4.1-rc I'm running so either my git log search foo is
> bad or it'll be (at least) 4.2.
It's not a typo. FITRIM is the name of the ioctl that fstrim calls.
The final version of that patch set is ready to go. Mostly. It
probably needs to be re-integrated now. The reason it was delayed for
inclusion is that it makes other bugs more obvious and irrecoverable
since the data is completely gone.
I'm not sure what Chris's timeline for inclusion is.
- -Jeff
- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)
iQIcBAEBAgAGBQJVzgU/AAoJEB57S2MheeWyjSwQALOLfzNLiuAdxBXDnP076Pq7
8m2F2DtTRxuDCwBmnlgZtX3QuWK/J1HVRpAO/aC6WQkOo3uRNFrG4xK45EOTA5hH
VBNtAFooMreicFQq5ZE6i+4yEdV8D4YRSoVn7+GrjL40IjiP8u7HXtDGw0x4ugGI
iVNf3yipaTZtlRcjGt91dfW3w3D8RpjUK3z7RwSOEy3C8GP90omRWVkYV/jcFIo4
hqFMZ77hisRLf1aCFxXlO14ERyMpLPtC3HOBMLHRrdpjPp/f4XnXyFmFA0kbOX8S
dwS9qPRmlnS5Lif2XMXK0a6aA0HK7sN/ghMigAh9t4zHwDkuDpAd6OWVEuCMMpCY
uN2KyuNsjam2DxJHQVulNu1xlS/sGedfh8p66lC29fkB8ZpyGp4fnK1N4MVRdk8R
4o/emRb+vg7CTZ3fvss7Af6w+m22GISO43Q1MWr6Hr1Ll2y0DWL1IaB/zky8sr/5
u6E5RI7DOvbFyC31dGqvh5WQDIPrTxRoDMJL+pSOkF4CM5SM1uHak4IgUqfZ85hr
MXVhRHFmH9UXRTFrkxzAV2wmSNpl2ki2pX5ItB6+c4fMMStb5dynThv27R69xxHf
mn8qZBuwc5iXXsPJ9dUAxTRoquOw9Rd/1fz4S/oLH6xOrtlNlLa2HFour4Ofp16h
3e6CvcV+h4/sz0PYpYSQ
=ZiEh
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2015-08-14 15:12 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-16 14:18 trim not working and irreparable errors from btrfsck Christian
2015-06-16 21:14 ` Chris Murphy
[not found] ` <55816E7B.5040905@gmail.com>
2015-06-17 14:22 ` Chris Murphy
2015-06-17 14:33 ` Christian
2015-06-17 15:28 ` Chris Murphy
2015-06-17 15:40 ` Christian
2015-06-17 17:17 ` Austin S Hemmelgarn
2015-06-18 5:25 ` Duncan
2015-08-14 15:11 ` Jeff Mahoney
2015-06-20 14:11 ` Lutz Euler
2015-06-21 7:21 ` Paul Jones
2015-08-13 9:23 ` Marc Joliet
2015-08-13 23:14 ` Chris Murphy
2015-08-14 8:05 ` Marc Joliet
2015-08-14 8:15 ` Marc Joliet
2015-08-14 10:51 ` Paul Jones
2015-06-18 2:20 ` Paul Jones
2015-06-18 4:15 ` Chris Murphy
2015-06-18 4:19 ` Chris Murphy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.