From: 許耀宗 <yaozhonghsu@gmail.com>
To: linux-bcache@vger.kernel.org, kent.overstreet@gmail.com
Cc: everest@synology.com, norrishsu@synology.com
Subject: bcache: Data is inconsistent when the cache device failed in writearound mode
Date: Thu, 24 Nov 2022 15:57:38 +0800 [thread overview]
Message-ID: <CABeQj8kYqMFVZ3OOAT9svsXg-GHTJC+N6G4_gtGmBwoC0=xseg@mail.gmail.com> (raw)
Hi linux-bcache,
I was evaluating the performance and stability of bcache.
In my test environment, I was using bcache as a read cache in
writearound cache mode running on OS ubuntu 20.04 with linux kernel
5.15.
However, I found that sometimes the data was inconsistent, and here is
how to reproduce:
1. Set up bcache with 1 hard drive and 1 read cache
$ make-bcache -B /dev/sdb -C /dev/sdc
2. Set as writearound mode
$ echo "writearound" > /sys/block/bcache0/bcache/cache_mode
3. Create XFS on /dev/bcache0
$ mkfs.xfs -m reflink=0 -f /dev/bcache0
4. Mount the file system
$ mkdir /srv/node/ -p
$ mount -t xfs /dev/bcache0 /srv/node -o
rw,nofail,noexec,nodev,noatime,nodiratime,inode64,logbsize=256k
5. Prepare a 50G file with random content
$ head -c 50G < /dev/urandom > 50G
6. Get md5sum of this 50G file
$ md5sum 50G
99c91956af4eec68707b329902062bfd
7. Copy the 50G file to the mount point
$ cp 50G /srv/node/
8. During the copy operation, remove the cache device manually
$ echo 1 > /sys/block/sdc/device/delete
9. Wait for the copy operation done, then calculate md5sum of 50G file
in the destination
$ md5sum /srv/node/50G
bb178bbb9c0ca07c202580e3ea9e62c5
The result shows the md5sum of two 50G file are not the same.
As far as I know, no data is written to cache device in writearound
mode. Therefore a failed cache device should not cause any data
inconsistency.
There are some further observations,
1. I can reproduce it with kernel 5.04 / 5.14.
2. I can reproduce it with ubuntu 22.04.
3. I cannot reproduce if the file is not large enough, like 10G and 20G.
Please advise me how to fix it. Thank you.
Sincerely,
Norris Hsu
reply other threads:[~2022-11-24 7:57 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABeQj8kYqMFVZ3OOAT9svsXg-GHTJC+N6G4_gtGmBwoC0=xseg@mail.gmail.com' \
--to=yaozhonghsu@gmail.com \
--cc=everest@synology.com \
--cc=kent.overstreet@gmail.com \
--cc=linux-bcache@vger.kernel.org \
--cc=norrishsu@synology.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).