From: Andrea Arcangeli <aarcange@redhat.com>
To: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Hugh Dickins <hughd@google.com>,
Dave Hansen <dave.hansen@intel.com>,
Vlastimil Babka <vbabka@suse.cz>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@suse.cz>,
David Rientjes <rientjes@google.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCHv5 0/7] Fix compound_head() race
Date: Fri, 4 Sep 2015 15:43:53 +0200 [thread overview]
Message-ID: <20150904134353.GD31717@redhat.com> (raw)
In-Reply-To: <1441283758-92774-1-git-send-email-kirill.shutemov@linux.intel.com>
On Thu, Sep 03, 2015 at 03:35:51PM +0300, Kirill A. Shutemov wrote:
> Kirill A. Shutemov (7):
> mm: drop page->slab_page
> slub: use page->rcu_head instead of page->lru plus cast
> zsmalloc: use page->private instead of page->first_page
> mm: pack compound_dtor and compound_order into one word in struct page
> mm: make compound_head() robust
> mm: use 'unsigned int' for page order
> mm: use 'unsigned int' for compound_dtor/compound_order on 64BIT
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
The only other alternative solution that doesn't require finding a bit
zero at the LSB in a field unused in tail pages, is to drop both
PG_head and PG_tail, and reserve 4 bits from page->flags.
This means a net loss of 2 bits from page->flags (and loss of 3 bits
if !CONFIG_PAGEFLAGS_EXTENDED), but then everything becomes simple and
there's no need of finding a LSB field that is guaranteed zero at all
times.
With those 4 bits, you clear them for not compound pages. When you
create a compound page you encode the compound_order in those 4 bits
of page->flags, equal for for all head and tail
pages. compound_order() then becomes atomically available for tail
pages too and compound_order goes away from struct page along with
first_page (and there's no need to add a compound_head).
In PageCompound you read the 4 bits, if they're not all zero it's
compound, otherwise it's not.
In PageHead/Tail, if the 4 bits are all zero it's not head/tail,
otherwise you do the math on the page_to_pfn(page). If the pfn is
naturally aligned against the order encoded in the 4 bits "!(pfn &
(1<<order)-1)" it's a head, otherwise it's a tail.
If it's a tail, for the compound_head then it's just a matter of doing
"return page - (pfn & ((1<<order)-1)" (no need of pfn_to_page).
This leverages the physical natural alignment of compound pages for
all orders. It'd cover up to CONFIG_FORCE_MAX_ZONEORDER == 16
(128MBytes of order 15 with PAGE_SIZE 4kb).
page_to_pfn can actually be replaced with
(&NODE_DATA(page_to_nid(page))->node_mem_map-page) which is faster as
page_to_nid only need to accesses page->flags which is already in
L1. So then it costs only one cacheline access in the pgdat and a sub.
Because of the two (or three) additional bits taken out of page->flags
I doubt it's viable on 32bit, but I thought I'd mention it just in case.
Thanks,
Andrea
WARNING: multiple messages have this Message-ID (diff)
From: Andrea Arcangeli <aarcange@redhat.com>
To: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Hugh Dickins <hughd@google.com>,
Dave Hansen <dave.hansen@intel.com>,
Vlastimil Babka <vbabka@suse.cz>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@suse.cz>,
David Rientjes <rientjes@google.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCHv5 0/7] Fix compound_head() race
Date: Fri, 4 Sep 2015 15:43:53 +0200 [thread overview]
Message-ID: <20150904134353.GD31717@redhat.com> (raw)
In-Reply-To: <1441283758-92774-1-git-send-email-kirill.shutemov@linux.intel.com>
On Thu, Sep 03, 2015 at 03:35:51PM +0300, Kirill A. Shutemov wrote:
> Kirill A. Shutemov (7):
> mm: drop page->slab_page
> slub: use page->rcu_head instead of page->lru plus cast
> zsmalloc: use page->private instead of page->first_page
> mm: pack compound_dtor and compound_order into one word in struct page
> mm: make compound_head() robust
> mm: use 'unsigned int' for page order
> mm: use 'unsigned int' for compound_dtor/compound_order on 64BIT
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
The only other alternative solution that doesn't require finding a bit
zero at the LSB in a field unused in tail pages, is to drop both
PG_head and PG_tail, and reserve 4 bits from page->flags.
This means a net loss of 2 bits from page->flags (and loss of 3 bits
if !CONFIG_PAGEFLAGS_EXTENDED), but then everything becomes simple and
there's no need of finding a LSB field that is guaranteed zero at all
times.
With those 4 bits, you clear them for not compound pages. When you
create a compound page you encode the compound_order in those 4 bits
of page->flags, equal for for all head and tail
pages. compound_order() then becomes atomically available for tail
pages too and compound_order goes away from struct page along with
first_page (and there's no need to add a compound_head).
In PageCompound you read the 4 bits, if they're not all zero it's
compound, otherwise it's not.
In PageHead/Tail, if the 4 bits are all zero it's not head/tail,
otherwise you do the math on the page_to_pfn(page). If the pfn is
naturally aligned against the order encoded in the 4 bits "!(pfn &
(1<<order)-1)" it's a head, otherwise it's a tail.
If it's a tail, for the compound_head then it's just a matter of doing
"return page - (pfn & ((1<<order)-1)" (no need of pfn_to_page).
This leverages the physical natural alignment of compound pages for
all orders. It'd cover up to CONFIG_FORCE_MAX_ZONEORDER == 16
(128MBytes of order 15 with PAGE_SIZE 4kb).
page_to_pfn can actually be replaced with
(&NODE_DATA(page_to_nid(page))->node_mem_map-page) which is faster as
page_to_nid only need to accesses page->flags which is already in
L1. So then it costs only one cacheline access in the pgdat and a sub.
Because of the two (or three) additional bits taken out of page->flags
I doubt it's viable on 32bit, but I thought I'd mention it just in case.
Thanks,
Andrea
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-09-04 13:43 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-03 12:35 [PATCHv5 0/7] Fix compound_head() race Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-03 12:35 ` [PATCHv5 1/7] mm: drop page->slab_page Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-07 5:00 ` Alexander Duyck
2015-09-07 5:00 ` Alexander Duyck
2015-09-07 12:19 ` Kirill A. Shutemov
2015-09-07 12:19 ` Kirill A. Shutemov
2015-09-03 12:35 ` [PATCHv5 2/7] slub: use page->rcu_head instead of page->lru plus cast Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-03 12:35 ` [PATCHv5 3/7] zsmalloc: use page->private instead of page->first_page Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-03 12:35 ` [PATCHv5 4/7] mm: pack compound_dtor and compound_order into one word in struct page Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-10 9:41 ` Vlastimil Babka
2015-09-10 9:41 ` Vlastimil Babka
2015-09-03 12:35 ` [PATCHv5 5/7] mm: make compound_head() robust Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-10 10:54 ` Vlastimil Babka
2015-09-10 10:54 ` Vlastimil Babka
2015-09-11 13:35 ` Kirill A. Shutemov
2015-09-11 13:35 ` Kirill A. Shutemov
2015-09-14 13:31 ` Vlastimil Babka
2015-09-14 13:31 ` Vlastimil Babka
2015-09-03 12:35 ` [PATCHv5 6/7] mm: use 'unsigned int' for page order Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-10 11:04 ` Vlastimil Babka
2015-09-10 11:04 ` Vlastimil Babka
2015-09-03 12:35 ` [PATCHv5 7/7] mm: use 'unsigned int' for compound_dtor/compound_order on 64BIT Kirill A. Shutemov
2015-09-03 12:35 ` Kirill A. Shutemov
2015-09-10 11:34 ` Vlastimil Babka
2015-09-10 11:34 ` Vlastimil Babka
2015-09-04 13:43 ` Andrea Arcangeli [this message]
2015-09-04 13:43 ` [PATCHv5 0/7] Fix compound_head() race Andrea Arcangeli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150904134353.GD31717@redhat.com \
--to=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=dave.hansen@intel.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.cz \
--cc=rientjes@google.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.