From: Tejun Heo <tj@kernel.org> To: axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org, hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org, vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com, fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com, khlebnikov@yandex-team.ru Subject: [PATCHSET 2/3 v3 block/for-4.2/core] writeback: cgroup writeback backpressure propagation Date: Fri, 22 May 2015 18:23:17 -0400 [thread overview] Message-ID: <1432333416-6221-1-git-send-email-tj@kernel.org> (raw) Hello, Changes from the last take[L] are * Rebased on top of block/for-4.2/core. While the previous patchset[1] implemented cgroup writeback support, the IO back pressure propagation mechanism implemented in balance_dirty_pages() and its subroutines isn't yet aware of cgroup writeback. Processes belonging to a memcg may have access to only subset of total memory available in the system and not factoring this into dirty throttling rendered it completely ineffective for processes under memcg limits and memcg ended up building a separate ad-hoc degenerate mechanism directly into vmscan code to limit page dirtying. This patchset refactors the dirty throttling logic implemented in balance_dirty_pages() and its subroutines os that it can handle both global and memcg memory domains. Dirty throttling mechanism is applied against both the global and memcg constraints and the more restricted of the two is used for actual throttling. This makes the dirty throttling mechanism operational for memcg domains including writeback-bandwidth-proportional dirty page distribution inside them. This patchset contains the following 19 patches. 0001-memcg-make-mem_cgroup_read_-stat-event-iterate-possi.patch 0002-writeback-clean-up-wb_dirty_limit.patch 0003-writeback-reorganize-__-wb_update_bandwidth.patch 0004-writeback-implement-wb_domain.patch 0005-writeback-move-global_dirty_limit-into-wb_domain.patch 0006-writeback-consolidate-dirty-throttle-parameters-into.patch 0007-writeback-add-dirty_throttle_control-wb_bg_thresh.patch 0008-writeback-make-__wb_calc_thresh-take-dirty_throttle_.patch 0009-writeback-add-dirty_throttle_control-pos_ratio.patch 0010-writeback-add-dirty_throttle_control-wb_completions.patch 0011-writeback-add-dirty_throttle_control-dom.patch 0012-writeback-make-__wb_writeout_inc-and-hard_dirty_limi.patch 0013-writeback-separate-out-domain_dirty_limits.patch 0014-writeback-move-over_bground_thresh-to-mm-page-writeb.patch 0015-writeback-update-wb_over_bg_thresh-to-use-wb_domain-.patch 0016-writeback-implement-memcg-wb_domain.patch 0017-writeback-reset-wb_domain-dirty_limit-_tstmp-when-me.patch 0018-writeback-implement-memcg-writeback-domain-based-thr.patch 0019-mm-vmscan-disable-memcg-direct-reclaim-stalling-if-c.patch 0001-0003 are prep patches. 0004-0015 refactors dirty throttling logic so that it operates on wb_domain. 0016-0019 implement memcg wb_domain. This patchset is on top of block/for-4.2/core b04a5636a665 ("block: replace trylock with mutex_lock in blkdev_reread_part()") + [1] [PATCHSET 1/3 v4 block/for-4.2/core] writeback: cgroup writeback support and available in the following git branch. git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-cgroup-writeback-backpressure-20150522 diffstat follows. Thanks. fs/fs-writeback.c | 32 - include/linux/backing-dev-defs.h | 1 include/linux/memcontrol.h | 21 + include/linux/writeback.h | 84 +++- include/trace/events/writeback.h | 7 mm/backing-dev.c | 15 mm/memcontrol.c | 145 +++++-- mm/page-writeback.c | 744 +++++++++++++++++++++++++-------------- mm/vmscan.c | 51 ++ 9 files changed, 739 insertions(+), 361 deletions(-) -- tejun [L] http://lkml.kernel.org/g/1428350674-8303-1-git-send-email-tj@kernel.org [1] http://lkml.kernel.org/g/1432329245-5844-1-git-send-email-tj@kernel.org
WARNING: multiple messages have this Message-ID (diff)
From: Tejun Heo <tj@kernel.org> To: axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org, hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org, vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com, fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com, khlebnikov@yandex-team.ru Subject: [PATCHSET 2/3 v3 block/for-4.2/core] writeback: cgroup writeback backpressure propagation Date: Fri, 22 May 2015 18:23:17 -0400 [thread overview] Message-ID: <1432333416-6221-1-git-send-email-tj@kernel.org> (raw) Hello, Changes from the last take[L] are * Rebased on top of block/for-4.2/core. While the previous patchset[1] implemented cgroup writeback support, the IO back pressure propagation mechanism implemented in balance_dirty_pages() and its subroutines isn't yet aware of cgroup writeback. Processes belonging to a memcg may have access to only subset of total memory available in the system and not factoring this into dirty throttling rendered it completely ineffective for processes under memcg limits and memcg ended up building a separate ad-hoc degenerate mechanism directly into vmscan code to limit page dirtying. This patchset refactors the dirty throttling logic implemented in balance_dirty_pages() and its subroutines os that it can handle both global and memcg memory domains. Dirty throttling mechanism is applied against both the global and memcg constraints and the more restricted of the two is used for actual throttling. This makes the dirty throttling mechanism operational for memcg domains including writeback-bandwidth-proportional dirty page distribution inside them. This patchset contains the following 19 patches. 0001-memcg-make-mem_cgroup_read_-stat-event-iterate-possi.patch 0002-writeback-clean-up-wb_dirty_limit.patch 0003-writeback-reorganize-__-wb_update_bandwidth.patch 0004-writeback-implement-wb_domain.patch 0005-writeback-move-global_dirty_limit-into-wb_domain.patch 0006-writeback-consolidate-dirty-throttle-parameters-into.patch 0007-writeback-add-dirty_throttle_control-wb_bg_thresh.patch 0008-writeback-make-__wb_calc_thresh-take-dirty_throttle_.patch 0009-writeback-add-dirty_throttle_control-pos_ratio.patch 0010-writeback-add-dirty_throttle_control-wb_completions.patch 0011-writeback-add-dirty_throttle_control-dom.patch 0012-writeback-make-__wb_writeout_inc-and-hard_dirty_limi.patch 0013-writeback-separate-out-domain_dirty_limits.patch 0014-writeback-move-over_bground_thresh-to-mm-page-writeb.patch 0015-writeback-update-wb_over_bg_thresh-to-use-wb_domain-.patch 0016-writeback-implement-memcg-wb_domain.patch 0017-writeback-reset-wb_domain-dirty_limit-_tstmp-when-me.patch 0018-writeback-implement-memcg-writeback-domain-based-thr.patch 0019-mm-vmscan-disable-memcg-direct-reclaim-stalling-if-c.patch 0001-0003 are prep patches. 0004-0015 refactors dirty throttling logic so that it operates on wb_domain. 0016-0019 implement memcg wb_domain. This patchset is on top of block/for-4.2/core b04a5636a665 ("block: replace trylock with mutex_lock in blkdev_reread_part()") + [1] [PATCHSET 1/3 v4 block/for-4.2/core] writeback: cgroup writeback support and available in the following git branch. git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-cgroup-writeback-backpressure-20150522 diffstat follows. Thanks. fs/fs-writeback.c | 32 - include/linux/backing-dev-defs.h | 1 include/linux/memcontrol.h | 21 + include/linux/writeback.h | 84 +++- include/trace/events/writeback.h | 7 mm/backing-dev.c | 15 mm/memcontrol.c | 145 +++++-- mm/page-writeback.c | 744 +++++++++++++++++++++++++-------------- mm/vmscan.c | 51 ++ 9 files changed, 739 insertions(+), 361 deletions(-) -- tejun [L] http://lkml.kernel.org/g/1428350674-8303-1-git-send-email-tj@kernel.org [1] http://lkml.kernel.org/g/1432329245-5844-1-git-send-email-tj@kernel.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2015-05-22 22:23 UTC|newest] Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-05-22 22:23 Tejun Heo [this message] 2015-05-22 22:23 ` [PATCHSET 2/3 v3 block/for-4.2/core] writeback: cgroup writeback backpressure propagation Tejun Heo 2015-05-22 22:23 ` [PATCH 01/19] memcg: make mem_cgroup_read_{stat|event}() iterate possible cpus instead of online Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 23:12 ` Johannes Weiner 2015-05-22 23:12 ` Johannes Weiner 2015-05-22 23:12 ` Johannes Weiner 2015-06-17 14:41 ` Michal Hocko 2015-06-17 14:41 ` Michal Hocko 2015-05-22 22:23 ` [PATCH 02/19] writeback: clean up wb_dirty_limit() Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 03/19] writeback: reorganize [__]wb_update_bandwidth() Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 04/19] writeback: implement wb_domain Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 05/19] writeback: move global_dirty_limit into wb_domain Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 06/19] writeback: consolidate dirty throttle parameters into dirty_throttle_control Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 07/19] writeback: add dirty_throttle_control->wb_bg_thresh Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 08/19] writeback: make __wb_calc_thresh() take dirty_throttle_control Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 09/19] writeback: add dirty_throttle_control->pos_ratio Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 10/19] writeback: add dirty_throttle_control->wb_completions Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 11/19] writeback: add dirty_throttle_control->dom Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 12/19] writeback: make __wb_writeout_inc() and hard_dirty_limit() take wb_domaas a parameter Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 13/19] writeback: separate out domain_dirty_limits() Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 14/19] writeback: move over_bground_thresh() to mm/page-writeback.c Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 15/19] writeback: update wb_over_bg_thresh() to use wb_domain aware operations Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 16/19] writeback: implement memcg wb_domain Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 17/19] writeback: reset wb_domain->dirty_limit[_tstmp] when memcg domain size changes Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 18/19] writeback: implement memcg writeback domain based throttling Tejun Heo 2015-05-22 22:23 ` Tejun Heo 2015-05-22 22:23 ` [PATCH 19/19] mm: vmscan: disable memcg direct reclaim stalling if cgroup writeback support is in use Tejun Heo 2015-05-22 22:23 ` Tejun Heo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1432333416-6221-1-git-send-email-tj@kernel.org \ --to=tj@kernel.org \ --cc=axboe@kernel.dk \ --cc=cgroups@vger.kernel.org \ --cc=clm@fb.com \ --cc=david@fromorbit.com \ --cc=fengguang.wu@intel.com \ --cc=gthelen@google.com \ --cc=hannes@cmpxchg.org \ --cc=hch@infradead.org \ --cc=jack@suse.cz \ --cc=khlebnikov@yandex-team.ru \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lizefan@huawei.com \ --cc=mhocko@suse.cz \ --cc=vgoyal@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.