From: Kuan-Wei Chiu <visitorckw@gmail.com>
To: Matthew Mirvish <matthew@mm12.xyz>
Cc: Kent Overstreet <kent.overstreet@linux.dev>,
Coly Li <colyli@suse.de>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Linux Next Mailing List <linux-next@vger.kernel.org>
Subject: Re: linux-next: manual merge of the refactor-heap tree with the block tree
Date: Fri, 10 May 2024 17:10:59 +0800 [thread overview]
Message-ID: <Zj3kowGa9XzJ0yak@visitorckw-System-Product-Name> (raw)
In-Reply-To: <20240510034618.GA3161190@mm12.xyz>
On Thu, May 09, 2024 at 11:46:18PM -0400, Matthew Mirvish wrote:
> On Fri, May 10, 2024 at 11:07:11AM +0800, Kuan-Wei Chiu wrote:
> > On Thu, May 09, 2024 at 07:16:31PM -0400, Kent Overstreet wrote:
> > > On Fri, May 10, 2024 at 06:44:29AM +0800, Kuan-Wei Chiu wrote:
> > > > On Thu, May 09, 2024 at 03:58:57PM -0400, Kent Overstreet wrote:
> > > > > On Thu, May 09, 2024 at 03:27:45PM +1000, Stephen Rothwell wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > Today's linux-next merge of the refactor-heap tree got conflicts in:
> > > > > >
> > > > > > drivers/md/bcache/bset.c
> > > > > > drivers/md/bcache/bset.h
> > > > > > drivers/md/bcache/btree.c
> > > > > > drivers/md/bcache/writeback.c
> > > > > >
> > > > > > between commit:
> > > > > >
> > > > > > 3a861560ccb3 ("bcache: fix variable length array abuse in btree_iter")
> > > > > >
> > > > > > from the block tree and commit:
> > > > > >
> > > > > > afa5721abaaa ("bcache: Remove heap-related macros and switch to generic min_heap")
> > > > > >
> > > > > > from the refactor-heap tree.
> > > > > >
> > > > > > Ok, these conflicts are too extensive, so I am dropping the refactor-heap
> > > > > > tree for today. I suggest you all get together and sort something out.
> > > > >
> > > > > Coli and Kuan, you guys will need to get this sorted out quick if we
> > > > > want refactor-heap to make the merge window
> > > >
> > > > Hi Coli and Kent,
> > > >
> > > > If I understand correctly, the reported bug is because we attempted to
> > > > point (heap)->data to a dynamically allocated memory , but at that time
> > > > (heap)->data was not a regular pointer but a fixed size array with a
> > > > length of MAX_BSETS.
> > > >
> > > > In my refactor heap patch series, I introduced a preallocated array and
> > > > decided in min_heap_init() whether the data pointer should point to an
> > > > incoming pointer or to the preallocated array. Therefore, I am
> > > > wondering if my patch might have unintentionally fixed this bug?
> > > >
> > > > I am unsure how to reproduce the reported issue. Could you assist me in
> > > > verifying whether my assumption is correct?
> > >
> > > This is a merge conflict, not a runtime. Can you rebase onto Coli's
> > > tree? We'll have to retest.
> >
> > Oh, sorry for the misunderstanding I caused. When I mentioned "bug" [1]
> > earlier, I was referring to the bug addressed in
> > 3a861560ccb3 ("bcache: fix variable length array abuse in btree_iter"),
> > not a merge conflict.
> >
> > Here are the results after the rebase:
> > https://github.com/visitorckw/linux.git refactor-heap
> >
> > [1]: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2039368
>
> The ubuntu kernels build with UBSAN now, and the bug reported is just a
> UBSAN warning. The original implementation's iterator has a fixed size
> sets array that is indexed out of bounds when the iterator is allocated
> on the heap with more space -- the patch restructures it a bit to have a
> single iterator type with a flexible array and then a larger "stack"
> type which embeds the iterator along with the preallocated region.
>
> I took a brief look at the refactor-heap branch but I'm not entirely
> sure what's going on with the new min heaps: in the one place where the
> larger iterators are used (in bch_btree_node_read_done) it doesn't look
> like the heap is ever initialized (perhaps since the old iter_init
> wasn't used here because of the special case it got missed in the
> refactor?) With the new heaps it should be fairly easy to fix though;
> just change the fill_iter mempool to be allocating only the minheap data
> arrays and setup iter->heap.data properly with that instead.
Thank you, Matthew.
Not initializing the heap's data pointer was indeed my mistake.
Following your advice, I made the following modifications to the code
on the refactor-heap branch in my github repo. I hope this time it
works well.
Regards,
Kuan-Wei
diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c
index a2bb86d52ad4..ce9d729bc8ff 100644
--- a/drivers/md/bcache/btree.c
+++ b/drivers/md/bcache/btree.c
@@ -149,19 +149,19 @@ void bch_btree_node_read_done(struct btree *b)
{
const char *err = "bad btree header";
struct bset *i = btree_bset_first(b);
- struct btree_iter *iter;
+ struct btree_iter iter;
/*
* c->fill_iter can allocate an iterator with more memory space
* than static MAX_BSETS.
* See the comment arount cache_set->fill_iter.
*/
- iter = mempool_alloc(&b->c->fill_iter, GFP_NOIO);
- iter->heap.size = b->c->cache->sb.bucket_size / b->c->cache->sb.block_size;
- iter->heap.nr = 0;
+ iter.heap.data = mempool_alloc(&b->c->fill_iter, GFP_NOIO);
+ iter.heap.size = b->c->cache->sb.bucket_size / b->c->cache->sb.block_size;
+ iter.heap.nr = 0;
#ifdef CONFIG_BCACHE_DEBUG
- iter->b = &b->keys;
+ iter.b = &b->keys;
#endif
if (!i->seq)
@@ -199,7 +199,7 @@ void bch_btree_node_read_done(struct btree *b)
if (i != b->keys.set[0].data && !i->keys)
goto err;
- bch_btree_iter_push(iter, i->start, bset_bkey_last(i));
+ bch_btree_iter_push(&iter, i->start, bset_bkey_last(i));
b->written += set_blocks(i, block_bytes(b->c->cache));
}
@@ -211,7 +211,7 @@ void bch_btree_node_read_done(struct btree *b)
if (i->seq == b->keys.set[0].data->seq)
goto err;
- bch_btree_sort_and_fix_extents(&b->keys, iter, &b->c->sort);
+ bch_btree_sort_and_fix_extents(&b->keys, &iter, &b->c->sort);
i = b->keys.set[0].data;
err = "short btree key";
@@ -223,7 +223,7 @@ void bch_btree_node_read_done(struct btree *b)
bch_bset_init_next(&b->keys, write_block(b),
bset_magic(&b->c->cache->sb));
out:
- mempool_free(iter, &b->c->fill_iter);
+ mempool_free(iter.heap.data, &b->c->fill_iter);
return;
err:
set_btree_node_io_error(b);
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index cba09660148a..c6f5592996a8 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1914,8 +1914,7 @@ struct cache_set *bch_cache_set_alloc(struct cache_sb *sb)
INIT_LIST_HEAD(&c->btree_cache_freed);
INIT_LIST_HEAD(&c->data_buckets);
- iter_size = sizeof(struct btree_iter) +
- ((meta_bucket_pages(sb) * PAGE_SECTORS) / sb->block_size) *
+ iter_size = ((meta_bucket_pages(sb) * PAGE_SECTORS) / sb->block_size) *
sizeof(struct btree_iter_set);
c->devices = kcalloc(c->nr_uuids, sizeof(void *), GFP_KERNEL);
next prev parent reply other threads:[~2024-05-10 9:11 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-09 5:27 linux-next: manual merge of the refactor-heap tree with the block tree Stephen Rothwell
2024-05-09 19:58 ` Kent Overstreet
2024-05-09 22:44 ` Kuan-Wei Chiu
2024-05-09 23:16 ` Kent Overstreet
2024-05-10 3:07 ` Kuan-Wei Chiu
2024-05-10 3:46 ` Matthew Mirvish
2024-05-10 9:10 ` Kuan-Wei Chiu [this message]
2024-05-11 19:24 ` Kuan-Wei Chiu
2024-05-14 7:40 ` Bagas Sanjaya
2024-05-21 2:18 ` Stephen Rothwell
2024-05-21 2:44 ` Kuan-Wei Chiu
2024-05-21 13:42 ` Kent Overstreet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zj3kowGa9XzJ0yak@visitorckw-System-Product-Name \
--to=visitorckw@gmail.com \
--cc=colyli@suse.de \
--cc=kent.overstreet@linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=matthew@mm12.xyz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).