From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6ADCC4338F for ; Tue, 3 Aug 2021 06:29:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B57460EBB for ; Tue, 3 Aug 2021 06:29:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233909AbhHCG32 (ORCPT ); Tue, 3 Aug 2021 02:29:28 -0400 Received: from szxga08-in.huawei.com ([45.249.212.255]:13227 "EHLO szxga08-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230096AbhHCG31 (ORCPT ); Tue, 3 Aug 2021 02:29:27 -0400 Received: from dggeme703-chm.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Gf4hj4X4Mz1CRwp; Tue, 3 Aug 2021 14:29:09 +0800 (CST) Received: from [10.174.179.25] (10.174.179.25) by dggeme703-chm.china.huawei.com (10.1.199.99) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Tue, 3 Aug 2021 14:29:13 +0800 Subject: Re: [PATCH 2/5] mm, memcg: narrow the scope of percpu_charge_mutex To: Roman Gushchin CC: Michal Hocko , , , , , , , , , , , References: <20210729125755.16871-1-linmiaohe@huawei.com> <20210729125755.16871-3-linmiaohe@huawei.com> <4a3c23c4-054c-2896-29c5-8cf9a4deee98@huawei.com> From: Miaohe Lin Message-ID: <95629d91-6ae8-b445-e7fc-b51c888cad59@huawei.com> Date: Tue, 3 Aug 2021 14:29:13 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.179.25] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggeme703-chm.china.huawei.com (10.1.199.99) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2021/8/3 11:40, Roman Gushchin wrote: > On Sat, Jul 31, 2021 at 10:29:52AM +0800, Miaohe Lin wrote: >> On 2021/7/30 14:50, Michal Hocko wrote: >>> On Thu 29-07-21 20:06:45, Roman Gushchin wrote: >>>> On Thu, Jul 29, 2021 at 08:57:52PM +0800, Miaohe Lin wrote: >>>>> Since percpu_charge_mutex is only used inside drain_all_stock(), we can >>>>> narrow the scope of percpu_charge_mutex by moving it here. >>>>> >>>>> Signed-off-by: Miaohe Lin >>>>> --- >>>>> mm/memcontrol.c | 2 +- >>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>> >>>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c >>>>> index 6580c2381a3e..a03e24e57cd9 100644 >>>>> --- a/mm/memcontrol.c >>>>> +++ b/mm/memcontrol.c >>>>> @@ -2050,7 +2050,6 @@ struct memcg_stock_pcp { >>>>> #define FLUSHING_CACHED_CHARGE 0 >>>>> }; >>>>> static DEFINE_PER_CPU(struct memcg_stock_pcp, memcg_stock); >>>>> -static DEFINE_MUTEX(percpu_charge_mutex); >>>>> >>>>> #ifdef CONFIG_MEMCG_KMEM >>>>> static void drain_obj_stock(struct obj_stock *stock); >>>>> @@ -2209,6 +2208,7 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) >>>>> */ >>>>> static void drain_all_stock(struct mem_cgroup *root_memcg) >>>>> { >>>>> + static DEFINE_MUTEX(percpu_charge_mutex); >>>>> int cpu, curcpu; >>>> >>>> It's considered a good practice to protect data instead of code paths. After >>>> the proposed change it becomes obvious that the opposite is done here: the mutex >>>> is used to prevent a simultaneous execution of the code of the drain_all_stock() >>>> function. >>> >>> The purpose of the lock was indeed to orchestrate callers more than any >>> data structure consistency. >>> >>>> Actually we don't need a mutex here: nobody ever sleeps on it. So I'd replace >>>> it with a simple atomic variable or even a single bitfield. Then the change will >>>> be better justified, IMO. >>> >>> Yes, mutex can be replaced by an atomic in a follow up patch. >>> >> >> Thanks for both of you. It's a really good suggestion. What do you mean is something like below？ >> >> diff --git a/mm/memcontrol.c b/mm/memcontrol.c >> index 616d1a72ece3..508a96e80980 100644 >> --- a/mm/memcontrol.c >> +++ b/mm/memcontrol.c >> @@ -2208,11 +2208,11 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) >> */ >> static void drain_all_stock(struct mem_cgroup *root_memcg) >> { >> - static DEFINE_MUTEX(percpu_charge_mutex); >> int cpu, curcpu; >> + static atomic_t drain_all_stocks = ATOMIC_INIT(-1); >> >> /* If someone's already draining, avoid adding running more workers. */ >> - if (!mutex_trylock(&percpu_charge_mutex)) >> + if (!atomic_inc_not_zero(&drain_all_stocks)) >> return; > > It should work, but why not a simple atomic_cmpxchg(&drain_all_stocks, 0, 1) and > initialize it to 0? Maybe it's just my preference, but IMO (0, 1) is easier > to understand than (-1, 0) here. Not a strong opinion though, up to you. > I think this would improve the readability. What you mean is something like below ? Many thanks. diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 616d1a72ece3..6210b1124929 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2208,11 +2208,11 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) */ static void drain_all_stock(struct mem_cgroup *root_memcg) { - static DEFINE_MUTEX(percpu_charge_mutex); int cpu, curcpu; + static atomic_t drainer = ATOMIC_INIT(0); /* If someone's already draining, avoid adding running more workers. */ - if (!mutex_trylock(&percpu_charge_mutex)) + if (atomic_cmpxchg(&drainer, 0, 1) != 0) return; /* * Notify other cpus that system-wide "drain" is running @@ -2244,7 +2244,7 @@ static void drain_all_stock(struct mem_cgroup *root_memcg) } } put_cpu(); - mutex_unlock(&percpu_charge_mutex); + atomic_set(&drainer, 0); } > Thanks! > . > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Miaohe Lin Subject: Re: [PATCH 2/5] mm, memcg: narrow the scope of percpu_charge_mutex Date: Tue, 3 Aug 2021 14:29:13 +0800 Message-ID: <95629d91-6ae8-b445-e7fc-b51c888cad59@huawei.com> References: <20210729125755.16871-1-linmiaohe@huawei.com> <20210729125755.16871-3-linmiaohe@huawei.com> <4a3c23c4-054c-2896-29c5-8cf9a4deee98@huawei.com> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Language: en-US List-ID: Content-Type: text/plain; charset="windows-1252" To: Roman Gushchin Cc: Michal Hocko , hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org, vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org, alexs-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, richard.weiyang-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org On 2021/8/3 11:40, Roman Gushchin wrote: > On Sat, Jul 31, 2021 at 10:29:52AM +0800, Miaohe Lin wrote: >> On 2021/7/30 14:50, Michal Hocko wrote: >>> On Thu 29-07-21 20:06:45, Roman Gushchin wrote: >>>> On Thu, Jul 29, 2021 at 08:57:52PM +0800, Miaohe Lin wrote: >>>>> Since percpu_charge_mutex is only used inside drain_all_stock(), we c= an >>>>> narrow the scope of percpu_charge_mutex by moving it here. >>>>> >>>>> Signed-off-by: Miaohe Lin >>>>> --- >>>>> mm/memcontrol.c | 2 +- >>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>> >>>>> diff --git a/mm/memcontrol.c b/mm/memcontrol.c >>>>> index 6580c2381a3e..a03e24e57cd9 100644 >>>>> --- a/mm/memcontrol.c >>>>> +++ b/mm/memcontrol.c >>>>> @@ -2050,7 +2050,6 @@ struct memcg_stock_pcp { >>>>> #define FLUSHING_CACHED_CHARGE 0 >>>>> }; >>>>> static DEFINE_PER_CPU(struct memcg_stock_pcp, memcg_stock); >>>>> -static DEFINE_MUTEX(percpu_charge_mutex); >>>>> =20 >>>>> #ifdef CONFIG_MEMCG_KMEM >>>>> static void drain_obj_stock(struct obj_stock *stock); >>>>> @@ -2209,6 +2208,7 @@ static void refill_stock(struct mem_cgroup *mem= cg, unsigned int nr_pages) >>>>> */ >>>>> static void drain_all_stock(struct mem_cgroup *root_memcg) >>>>> { >>>>> + static DEFINE_MUTEX(percpu_charge_mutex); >>>>> int cpu, curcpu; >>>> >>>> It's considered a good practice to protect data instead of code paths.= After >>>> the proposed change it becomes obvious that the opposite is done here:= the mutex >>>> is used to prevent a simultaneous execution of the code of the drain_a= ll_stock() >>>> function. >>> >>> The purpose of the lock was indeed to orchestrate callers more than any >>> data structure consistency. >>> =20 >>>> Actually we don't need a mutex here: nobody ever sleeps on it. So I'd = replace >>>> it with a simple atomic variable or even a single bitfield. Then the c= hange will >>>> be better justified, IMO. >>> >>> Yes, mutex can be replaced by an atomic in a follow up patch. >>> >> >> Thanks for both of you. It's a really good suggestion. What do you mean = is something like below=EF=BC=9F >> >> diff --git a/mm/memcontrol.c b/mm/memcontrol.c >> index 616d1a72ece3..508a96e80980 100644 >> --- a/mm/memcontrol.c >> +++ b/mm/memcontrol.c >> @@ -2208,11 +2208,11 @@ static void refill_stock(struct mem_cgroup *memc= g, unsigned int nr_pages) >> */ >> static void drain_all_stock(struct mem_cgroup *root_memcg) >> { >> - static DEFINE_MUTEX(percpu_charge_mutex); >> int cpu, curcpu; >> + static atomic_t drain_all_stocks =3D ATOMIC_INIT(-1); >> >> /* If someone's already draining, avoid adding running more work= ers. */ >> - if (!mutex_trylock(&percpu_charge_mutex)) >> + if (!atomic_inc_not_zero(&drain_all_stocks)) >> return; >=20 > It should work, but why not a simple atomic_cmpxchg(&drain_all_stocks, 0,= 1) and > initialize it to 0? Maybe it's just my preference, but IMO (0, 1) is easi= er > to understand than (-1, 0) here. Not a strong opinion though, up to you. >=20 I think this would improve the readability. What you mean is something like= below ? Many thanks. diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 616d1a72ece3..6210b1124929 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2208,11 +2208,11 @@ static void refill_stock(struct mem_cgroup *memcg, = unsigned int nr_pages) */ static void drain_all_stock(struct mem_cgroup *root_memcg) { - static DEFINE_MUTEX(percpu_charge_mutex); int cpu, curcpu; + static atomic_t drainer =3D ATOMIC_INIT(0); /* If someone's already draining, avoid adding running more workers= . */ - if (!mutex_trylock(&percpu_charge_mutex)) + if (atomic_cmpxchg(&drainer, 0, 1) !=3D 0) return; /* * Notify other cpus that system-wide "drain" is running @@ -2244,7 +2244,7 @@ static void drain_all_stock(struct mem_cgroup *root_m= emcg) } } put_cpu(); - mutex_unlock(&percpu_charge_mutex); + atomic_set(&drainer, 0); } > Thanks! > . >=20