From: Michal Nazarewicz <mina86@mina86.com> To: Vlastimil Babka <vbabka@suse.cz>, Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Vlastimil Babka <vbabka@suse.cz>, Minchan Kim <minchan@kernel.org>, Mel Gorman <mgorman@suse.de>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>, Christoph Lameter <cl@linux.com>, Rik van Riel <riel@redhat.com>, David Rientjes <rientjes@google.com> Subject: Re: [PATCH 5/6] mm, compaction: skip compound pages by order in free scanner Date: Fri, 12 Jun 2015 12:18:07 +0200 [thread overview] Message-ID: <xa1tk2v9p6w0.fsf@mina86.com> (raw) In-Reply-To: <1433928754-966-6-git-send-email-vbabka@suse.cz> On Wed, Jun 10 2015, Vlastimil Babka wrote: > The compaction free scanner is looking for PageBuddy() pages and skipping all > others. For large compound pages such as THP or hugetlbfs, we can save a lot > of iterations if we skip them at once using their compound_order(). This is > generally unsafe and we can read a bogus value of order due to a race, but if > we are careful, the only danger is skipping too much. > > When tested with stress-highalloc from mmtests on 4GB system with 1GB hugetlbfs > pages, the vmstat compact_free_scanned count decreased by at least 15%. > > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> > Cc: Minchan Kim <minchan@kernel.org> > Cc: Mel Gorman <mgorman@suse.de> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Michal Nazarewicz <mina86@mina86.com> Acked-by: Michal Nazarewicz <mina86@mina86.com> > Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> > Cc: Christoph Lameter <cl@linux.com> > Cc: Rik van Riel <riel@redhat.com> > Cc: David Rientjes <rientjes@google.com> > --- > mm/compaction.c | 25 +++++++++++++++++++++++++ > 1 file changed, 25 insertions(+) > > diff --git a/mm/compaction.c b/mm/compaction.c > index e37d361..4a14084 100644 > --- a/mm/compaction.c > +++ b/mm/compaction.c > @@ -437,6 +437,24 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, > > if (!valid_page) > valid_page = page; > + > + /* > + * For compound pages such as THP and hugetlbfs, we can save > + * potentially a lot of iterations if we skip them at once. > + * The check is racy, but we can consider only valid values > + * and the only danger is skipping too much. > + */ > + if (PageCompound(page)) { > + unsigned int comp_order = compound_order(page); > + > + if (comp_order > 0 && comp_order < MAX_ORDER) { + if (comp_order < MAX_ORDER) { Might produce shorter/faster code. Dunno. Maybe. So much micro-optimisations. Applies to the previous patch as well. > + blockpfn += (1UL << comp_order) - 1; > + cursor += (1UL << comp_order) - 1; > + } > + > + goto isolate_fail; > + } > + > if (!PageBuddy(page)) > goto isolate_fail; > > @@ -496,6 +514,13 @@ isolate_fail: > > } > > + /* > + * There is a tiny chance that we have read bogus compound_order(), > + * so be careful to not go outside of the pageblock. > + */ > + if (unlikely(blockpfn > end_pfn)) > + blockpfn = end_pfn; > + > trace_mm_compaction_isolate_freepages(*start_pfn, blockpfn, > nr_scanned, total_isolated); > > -- > 2.1.4 > -- Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--
WARNING: multiple messages have this Message-ID (diff)
From: Michal Nazarewicz <mina86@mina86.com> To: Vlastimil Babka <vbabka@suse.cz>, Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Minchan Kim <minchan@kernel.org>, Mel Gorman <mgorman@suse.de>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>, Christoph Lameter <cl@linux.com>, Rik van Riel <riel@redhat.com>, David Rientjes <rientjes@google.com> Subject: Re: [PATCH 5/6] mm, compaction: skip compound pages by order in free scanner Date: Fri, 12 Jun 2015 12:18:07 +0200 [thread overview] Message-ID: <xa1tk2v9p6w0.fsf@mina86.com> (raw) In-Reply-To: <1433928754-966-6-git-send-email-vbabka@suse.cz> On Wed, Jun 10 2015, Vlastimil Babka wrote: > The compaction free scanner is looking for PageBuddy() pages and skipping all > others. For large compound pages such as THP or hugetlbfs, we can save a lot > of iterations if we skip them at once using their compound_order(). This is > generally unsafe and we can read a bogus value of order due to a race, but if > we are careful, the only danger is skipping too much. > > When tested with stress-highalloc from mmtests on 4GB system with 1GB hugetlbfs > pages, the vmstat compact_free_scanned count decreased by at least 15%. > > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> > Cc: Minchan Kim <minchan@kernel.org> > Cc: Mel Gorman <mgorman@suse.de> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Michal Nazarewicz <mina86@mina86.com> Acked-by: Michal Nazarewicz <mina86@mina86.com> > Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> > Cc: Christoph Lameter <cl@linux.com> > Cc: Rik van Riel <riel@redhat.com> > Cc: David Rientjes <rientjes@google.com> > --- > mm/compaction.c | 25 +++++++++++++++++++++++++ > 1 file changed, 25 insertions(+) > > diff --git a/mm/compaction.c b/mm/compaction.c > index e37d361..4a14084 100644 > --- a/mm/compaction.c > +++ b/mm/compaction.c > @@ -437,6 +437,24 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, > > if (!valid_page) > valid_page = page; > + > + /* > + * For compound pages such as THP and hugetlbfs, we can save > + * potentially a lot of iterations if we skip them at once. > + * The check is racy, but we can consider only valid values > + * and the only danger is skipping too much. > + */ > + if (PageCompound(page)) { > + unsigned int comp_order = compound_order(page); > + > + if (comp_order > 0 && comp_order < MAX_ORDER) { + if (comp_order < MAX_ORDER) { Might produce shorter/faster code. Dunno. Maybe. So much micro-optimisations. Applies to the previous patch as well. > + blockpfn += (1UL << comp_order) - 1; > + cursor += (1UL << comp_order) - 1; > + } > + > + goto isolate_fail; > + } > + > if (!PageBuddy(page)) > goto isolate_fail; > > @@ -496,6 +514,13 @@ isolate_fail: > > } > > + /* > + * There is a tiny chance that we have read bogus compound_order(), > + * so be careful to not go outside of the pageblock. > + */ > + if (unlikely(blockpfn > end_pfn)) > + blockpfn = end_pfn; > + > trace_mm_compaction_isolate_freepages(*start_pfn, blockpfn, > nr_scanned, total_isolated); > > -- > 2.1.4 > -- Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz (o o) ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-06-12 10:18 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-06-10 9:32 [PATCH 0/6] Assorted compaction cleanups and optimizations Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-10 9:32 ` [PATCH 1/6] mm, compaction: more robust check for scanners meeting Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-10 18:02 ` Rik van Riel 2015-06-10 18:02 ` Rik van Riel 2015-06-12 9:55 ` Michal Nazarewicz 2015-06-12 9:55 ` Michal Nazarewicz 2015-06-16 5:37 ` Joonsoo Kim 2015-06-16 5:37 ` Joonsoo Kim 2015-06-19 13:41 ` Mel Gorman 2015-06-19 13:41 ` Mel Gorman 2015-06-10 9:32 ` [PATCH 2/6] mm, compaction: simplify handling restart position in free pages scanner Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-16 5:38 ` Joonsoo Kim 2015-06-16 5:38 ` Joonsoo Kim 2015-06-19 13:48 ` Mel Gorman 2015-06-19 13:48 ` Mel Gorman 2015-06-10 9:32 ` [PATCH 3/6] mm, compaction: encapsulate resetting cached scanner positions Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-12 10:07 ` Michal Nazarewicz 2015-06-12 10:07 ` Michal Nazarewicz 2015-06-16 5:41 ` Joonsoo Kim 2015-06-16 5:41 ` Joonsoo Kim 2015-06-16 12:13 ` Vlastimil Babka 2015-06-16 12:13 ` Vlastimil Babka 2015-06-10 9:32 ` [PATCH 4/6] mm, compaction: always skip compound pages by order in migrate scanner Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-12 10:11 ` Michal Nazarewicz 2015-06-12 10:11 ` Michal Nazarewicz 2015-06-16 5:44 ` Joonsoo Kim 2015-06-16 5:44 ` Joonsoo Kim 2015-06-16 12:16 ` Vlastimil Babka 2015-06-16 12:16 ` Vlastimil Babka 2015-06-19 13:58 ` Mel Gorman 2015-06-19 13:58 ` Mel Gorman 2015-06-10 9:32 ` [PATCH 5/6] mm, compaction: skip compound pages by order in free scanner Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-12 10:18 ` Michal Nazarewicz [this message] 2015-06-12 10:18 ` Michal Nazarewicz 2015-06-16 5:45 ` Joonsoo Kim 2015-06-16 5:45 ` Joonsoo Kim 2015-06-10 9:32 ` [PATCH 6/6] mm, compaction: decouple updating pageblock_skip and cached pfn Vlastimil Babka 2015-06-10 9:32 ` Vlastimil Babka 2015-06-16 6:10 ` Joonsoo Kim 2015-06-16 6:10 ` Joonsoo Kim 2015-06-16 12:33 ` Vlastimil Babka 2015-06-16 12:33 ` Vlastimil Babka 2015-06-16 13:03 ` Joonsoo Kim 2015-06-16 13:03 ` Joonsoo Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=xa1tk2v9p6w0.fsf@mina86.com \ --to=mina86@mina86.com \ --cc=akpm@linux-foundation.org \ --cc=cl@linux.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@suse.de \ --cc=minchan@kernel.org \ --cc=n-horiguchi@ah.jp.nec.com \ --cc=riel@redhat.com \ --cc=rientjes@google.com \ --cc=vbabka@suse.cz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.