From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
To: zhong jiang <zhongjiang@huawei.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Vlastimil Babka <vbabka@suse.cz>, Aaron Lu <aaron.lu@intel.com>,
Mel Gorman <mgorman@suse.de>, Rik van Riel <riel@redhat.com>,
David Rientjes <rientjes@google.com>,
LKML <linux-kernel@vger.kernel.org>,
Linux Memory Management List <linux-mm@kvack.org>,
Xishi Qiu <qiuxishi@huawei.com>
Subject: Re: [PATCH v2 3/3] mm/compaction: speed up pageblock_pfn_to_page() when zone is contiguous
Date: Mon, 15 Feb 2016 11:42:20 +0900 [thread overview]
Message-ID: <20160215024220.GA30918@js1304-P5Q-DELUXE> (raw)
In-Reply-To: <56C0550F.8020402@huawei.com>
On Sun, Feb 14, 2016 at 06:21:03PM +0800, zhong jiang wrote:
> On 2016/2/6 0:11, Joonsoo Kim wrote:
> > 2016-02-05 9:49 GMT+09:00 Andrew Morton <akpm@linux-foundation.org>:
> >> On Thu, 4 Feb 2016 15:19:35 +0900 Joonsoo Kim <js1304@gmail.com> wrote:
> >>
> >>> There is a performance drop report due to hugepage allocation and in there
> >>> half of cpu time are spent on pageblock_pfn_to_page() in compaction [1].
> >>> In that workload, compaction is triggered to make hugepage but most of
> >>> pageblocks are un-available for compaction due to pageblock type and
> >>> skip bit so compaction usually fails. Most costly operations in this case
> >>> is to find valid pageblock while scanning whole zone range. To check
> >>> if pageblock is valid to compact, valid pfn within pageblock is required
> >>> and we can obtain it by calling pageblock_pfn_to_page(). This function
> >>> checks whether pageblock is in a single zone and return valid pfn
> >>> if possible. Problem is that we need to check it every time before
> >>> scanning pageblock even if we re-visit it and this turns out to
> >>> be very expensive in this workload.
> >>>
> >>> Although we have no way to skip this pageblock check in the system
> >>> where hole exists at arbitrary position, we can use cached value for
> >>> zone continuity and just do pfn_to_page() in the system where hole doesn't
> >>> exist. This optimization considerably speeds up in above workload.
> >>>
> >>> Before vs After
> >>> Max: 1096 MB/s vs 1325 MB/s
> >>> Min: 635 MB/s 1015 MB/s
> >>> Avg: 899 MB/s 1194 MB/s
> >>>
> >>> Avg is improved by roughly 30% [2].
> >>>
> >>> [1]: http://www.spinics.net/lists/linux-mm/msg97378.html
> >>> [2]: https://lkml.org/lkml/2015/12/9/23
> >>>
> >>> ...
> >>>
> >>> --- a/include/linux/memory_hotplug.h
> >>> +++ b/include/linux/memory_hotplug.h
> >>> @@ -196,6 +196,9 @@ void put_online_mems(void);
> >>> void mem_hotplug_begin(void);
> >>> void mem_hotplug_done(void);
> >>>
> >>> +extern void set_zone_contiguous(struct zone *zone);
> >>> +extern void clear_zone_contiguous(struct zone *zone);
> >>> +
> >>> #else /* ! CONFIG_MEMORY_HOTPLUG */
> >>> /*
> >>> * Stub functions for when hotplug is off
> >>
> >> Was it really intended that these declarations only exist if
> >> CONFIG_MEMORY_HOTPLUG? Seems unrelated.
> >
> > These are called for caching memory layout whether it is contiguous
> > or not. So, they are always called in memory initialization. Then,
> > hotplug could change memory layout so they should be called
> > there, too. So, they are defined in page_alloc.c and exported only
> > if CONFIG_MEMORY_HOTPLUG.
> >
> >> The i386 allnocofnig build fails in preditable ways so I fixed that up
> >> as below, but it seems wrong.
> >
> > Yeah, it seems wrong to me. :)
> > Here goes fix.
> >
> > ----------->8------------
> >>From ed6add18bc361e00a7ac6746de6eeb62109e6416 Mon Sep 17 00:00:00 2001
> > From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> > Date: Thu, 10 Dec 2015 17:03:54 +0900
> > Subject: [PATCH] mm/compaction: speed up pageblock_pfn_to_page() when zone is
> > contiguous
> >
> > There is a performance drop report due to hugepage allocation and in there
> > half of cpu time are spent on pageblock_pfn_to_page() in compaction [1].
> > In that workload, compaction is triggered to make hugepage but most of
> > pageblocks are un-available for compaction due to pageblock type and
> > skip bit so compaction usually fails. Most costly operations in this case
> > is to find valid pageblock while scanning whole zone range. To check
> > if pageblock is valid to compact, valid pfn within pageblock is required
> > and we can obtain it by calling pageblock_pfn_to_page(). This function
> > checks whether pageblock is in a single zone and return valid pfn
> > if possible. Problem is that we need to check it every time before
> > scanning pageblock even if we re-visit it and this turns out to
> > be very expensive in this workload.
> >
> > Although we have no way to skip this pageblock check in the system
> > where hole exists at arbitrary position, we can use cached value for
> > zone continuity and just do pfn_to_page() in the system where hole doesn't
> > exist. This optimization considerably speeds up in above workload.
> >
> > Before vs After
> > Max: 1096 MB/s vs 1325 MB/s
> > Min: 635 MB/s 1015 MB/s
> > Avg: 899 MB/s 1194 MB/s
> >
> > Avg is improved by roughly 30% [2].
> >
> > [1]: http://www.spinics.net/lists/linux-mm/msg97378.html
> > [2]: https://lkml.org/lkml/2015/12/9/23
> >
> > v3
> > o remove pfn_valid_within() check for all pages in the pageblock
> > because pageblock_pfn_to_page() is only called with pageblock aligned pfn.
>
> I have a question about the zone continuity. because hole exists at
> arbitrary position in a page block. Therefore, only pageblock_pf_to_page()
> is insufficiency, whether pageblock aligned pfn or not , the pfn_valid_within()
> is necessary.
>
> eh: 120M-122M is a range of page block, but the 120.5M-121.5M is holes, only by
> pageblock_pfn_to_page() to conclude in the result is inaccurate
contiguous may be misleading word. It doesn't represent there are no
hole. It only represents that all pageblocks within zone span belong to
corresponding zone and validity of all pageblock aligned pfn is
checked. So, if it is set, we can safely call pfn_to_page() for pageblock
aligned pfn in that zone without checking pfn_valid().
Thanks.
next prev parent reply other threads:[~2016-02-15 2:41 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-04 6:19 [PATCH v2 1/3] mm/compaction: fix invalid free_pfn and compact_cached_free_pfn Joonsoo Kim
2016-02-04 6:19 ` [PATCH v2 2/3] mm/compaction: pass only pageblock aligned range to pageblock_pfn_to_page Joonsoo Kim
2016-02-10 12:52 ` Vlastimil Babka
2016-02-04 6:19 ` [PATCH v2 3/3] mm/compaction: speed up pageblock_pfn_to_page() when zone is contiguous Joonsoo Kim
2016-02-05 0:49 ` Andrew Morton
2016-02-05 16:11 ` Joonsoo Kim
2016-02-09 17:58 ` Vlastimil Babka
2016-02-09 20:53 ` Andrew Morton
2016-02-10 13:42 ` Vlastimil Babka
2016-02-10 18:58 ` Andrew Morton
2016-02-11 1:58 ` Joonsoo Kim
2016-02-14 10:21 ` zhong jiang
2016-02-15 2:42 ` Joonsoo Kim [this message]
2016-02-15 10:06 ` Xishi Qiu
2016-02-15 14:24 ` Joonsoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160215024220.GA30918@js1304-P5Q-DELUXE \
--to=iamjoonsoo.kim@lge.com \
--cc=aaron.lu@intel.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=qiuxishi@huawei.com \
--cc=riel@redhat.com \
--cc=rientjes@google.com \
--cc=vbabka@suse.cz \
--cc=zhongjiang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).