From: Kui-Feng Lee <sinquersw@gmail.com>
To: Martin KaFai Lau <martin.lau@linux.dev>,
Kui-Feng Lee <thinker.li@gmail.com>
Cc: kuifeng@meta.com, bpf@vger.kernel.org, ast@kernel.org,
song@kernel.org, kernel-team@meta.com, andrii@kernel.org
Subject: Re: [PATCH bpf-next v2 6/6] selftests/bpf: make sure bpf_testmod handling racing link destroying well.
Date: Thu, 9 May 2024 10:02:07 -0700 [thread overview]
Message-ID: <6c156d6d-02f4-4d9a-aeca-951103894be2@gmail.com> (raw)
In-Reply-To: <a4a5571b-7536-402b-b099-19a9e54524b3@linux.dev>
On 5/8/24 17:04, Martin KaFai Lau wrote:
> On 5/6/24 10:56 PM, Kui-Feng Lee wrote:
>> Subsystems that manage struct_ops objects may attempt to detach a link
>> when
>> the link has been released or is about to be released. The test in
>> this patch demonstrate to developers the correct way to handle this
>> situation using a locking mechanism and atomic64_inc_not_zero().
>>
>> A subsystem must ensure that a link is valid when detaching the link. In
>> order to achieve that, the subsystem may need to obtain a lock to
>> safeguard
>> a table that holds the pointer to the link being detached. However, the
>> subsystem cannot invoke link->ops->detach() while holding the lock
>> because
>> other tasks may be in the process of unregistering, which could lead to a
>> deadlock. This is why atomic64_inc_not_zero() is used to maintain the
>
> Other tasks un-registering in parallel is not the reason for deadlock.
> The deadlock is because the link detach will call unreg() which usually
> will acquire the same lock (the detach_mutex here) and there is lock
> ordering with the update_mutex also. Hence, the link detach must be done
> after releasing the detach_mutex. After releasing the detach_mutex, the
> link is protected by its refcnt.
It is what I mean in the commit log. I will rephrase it to emphasize
holding the same lock.
>
> I think the above should be put as comments in bpf_dummy_do_link_detach
> for the subsystem to reference later.
ok!
>
>> link's validity. (Refer to bpf_dummy_do_link_detach() in the previous
>> patch
>> for more details.)
>>
>> This test make sure the pattern mentioned above work correctly.
>>
>> Signed-off-by: Kui-Feng Lee <thinker.li@gmail.com>
>> ---
>> .../bpf/prog_tests/test_struct_ops_module.c | 44 +++++++++++++++++++
>> 1 file changed, 44 insertions(+)
>>
>> diff --git
>> a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
>> b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
>> index 9f6657b53a93..1e37037cfd8a 100644
>> --- a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
>> +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
>> @@ -292,6 +292,48 @@ static void test_subsystem_detach(void)
>> struct_ops_detach__destroy(skel);
>> }
>> +/* A subsystem detachs a link while the link is going to be free. */
>> +static void test_subsystem_detach_free(void)
>> +{
>> + LIBBPF_OPTS(bpf_test_run_opts, topts,
>> + .data_in = &pkt_v4,
>> + .data_size_in = sizeof(pkt_v4));
>> + struct struct_ops_detach *skel;
>> + struct bpf_link *link = NULL;
>> + int prog_fd;
>> + int err;
>> +
>> + skel = struct_ops_detach__open_and_load();
>> + if (!ASSERT_OK_PTR(skel, "struct_ops_detach_open_and_load"))
>> + return;
>> +
>> + link = bpf_map__attach_struct_ops(skel->maps.testmod_do_detach);
>> + if (!ASSERT_OK_PTR(link, "attach_struct_ops"))
>> + goto cleanup;
>> +
>> + bpf_link__destroy(link);
>> +
>> + prog_fd = bpf_program__fd(skel->progs.start_detach);
>> + if (!ASSERT_GE(prog_fd, 0, "start_detach_fd"))
>> + goto cleanup;
>> +
>> + /* Do detachment from the registered subsystem */
>> + err = bpf_prog_test_run_opts(prog_fd, &topts);
>> + if (!ASSERT_OK(err, "start_detach_run"))
>> + goto cleanup;
>> +
>> + /* The link may have zero refcount value and may have been
>> + * unregistered, so the detachment from the subsystem should fail.
>> + */
>> + ASSERT_EQ(topts.retval, (u32)-ENOENT, "start_detach_run retval");
>> +
>> + /* Sync RCU to make sure the link is freed without any crash */
>> + ASSERT_OK(kern_sync_rcu(), "sync rcu");
>> +
>> +cleanup:
>> + struct_ops_detach__destroy(skel);
>> +}
>> +
>> void serial_test_struct_ops_module(void)
>> {
>> if (test__start_subtest("test_struct_ops_load"))
>> @@ -304,5 +346,7 @@ void serial_test_struct_ops_module(void)
>> test_detach_link();
>> if (test__start_subtest("test_subsystem_detach"))
>> test_subsystem_detach();
>> + if (test__start_subtest("test_subsystem_detach_free"))
>> + test_subsystem_detach_free();
>> }
>
prev parent reply other threads:[~2024-05-09 17:02 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 5:55 [PATCH bpf-next v2 0/6] Notify user space when a struct_ops object is detached/unregistered Kui-Feng Lee
2024-05-07 5:55 ` [PATCH bpf-next v2 1/6] bpf: pass bpf_struct_ops_link to callbacks in bpf_struct_ops Kui-Feng Lee
2024-05-07 5:55 ` [PATCH bpf-next v2 2/6] bpf: enable detaching links of struct_ops objects Kui-Feng Lee
2024-05-08 23:22 ` Martin KaFai Lau
2024-05-09 0:14 ` Kui-Feng Lee
2024-05-09 0:36 ` Martin KaFai Lau
2024-05-09 16:59 ` Kui-Feng Lee
2024-05-09 0:46 ` Kui-Feng Lee
2024-05-07 5:55 ` [PATCH bpf-next v2 3/6] bpf: support epoll from bpf struct_ops links Kui-Feng Lee
2024-05-07 5:55 ` [PATCH bpf-next v2 4/6] selftests/bpf: test struct_ops with epoll Kui-Feng Lee
2024-05-08 23:34 ` Martin KaFai Lau
2024-05-09 0:22 ` Kui-Feng Lee
2024-05-07 5:55 ` [PATCH bpf-next v2 5/6] selftests/bpf: detach a struct_ops link from the subsystem managing it Kui-Feng Lee
2024-05-08 23:50 ` Martin KaFai Lau
2024-05-09 5:50 ` Kui-Feng Lee
2024-05-07 5:56 ` [PATCH bpf-next v2 6/6] selftests/bpf: make sure bpf_testmod handling racing link destroying well Kui-Feng Lee
2024-05-09 0:04 ` Martin KaFai Lau
2024-05-09 17:02 ` Kui-Feng Lee [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6c156d6d-02f4-4d9a-aeca-951103894be2@gmail.com \
--to=sinquersw@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=kernel-team@meta.com \
--cc=kuifeng@meta.com \
--cc=martin.lau@linux.dev \
--cc=song@kernel.org \
--cc=thinker.li@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).