From: Kemeng Shi <shikemeng@huaweicloud.com>
To: akpm@linux-foundation.org, willy@infradead.org, jack@suse.cz,
bfoster@redhat.com, tj@kernel.org
Cc: dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
linux-fsdevel@vger.kernel.org
Subject: [PATCH v5 0/5] Improve visibility of writeback
Date: Tue, 23 Apr 2024 11:46:38 +0800 [thread overview]
Message-ID: <20240423034643.141219-1-shikemeng@huaweicloud.com> (raw)
v4->v5:
-Add new patch to fix build problem of "writeback: support retrieving
per group debug writeback stats of bdi". The new fix patch can be folded
into it's pre-patch if the pre-patch is not merge into tree or it could
be pick-up to apply.
v3->v4:
-Fix build warning that filepages, headroom and writeback in
cgwb_calc_thresh is used uninitialized when CONFIG_CGROUP_WRITEBACK
is not enabled.
v2->v3:
-Drop patches to protect non-exist race and to define GDTC_INIT_NO_WB to
null.
-Add wb_tryget to wb from which we collect stats to bdi stats.
-Create wb_stats when CONFIG_CGROUP_WRITEBACK is not enabled.
-Add a blank line between two wb stats in wb_stats.
v1->v2:
-Send cleanup to wq_monitor.py separately.
-Add patch to avoid use after free of bdi.
-Rename wb_calc_cg_thresh to cgwb_calc_thresh as Tejun suggested.
-Use rcu walk to avoid use after free.
-Add debug output to each related patches.
This series tries to improve visilibity of writeback. Patch 1 make
/sys/kernel/debug/bdi/xxx/stats show writeback info of whole bdi
instead of only writeback info in root cgroup. Patch 2 add a new
debug file /sys/kernel/debug/bdi/xxx/wb_stats to show per wb writeback
info. Patch 3 add wb_monitor.py to monitor basic writeback info
of running system, more info could be added on demand. Patch 4
is a random cleanup. More details can be found in respective
patches. Thanks!
Following domain hierarchy is tested:
global domain (320G)
/ \
cgroup domain1(10G) cgroup domain2(10G)
| |
bdi wb1 wb2
/* all writeback info of bdi is successfully collected */
cat stats
BdiWriteback: 4704 kB
BdiReclaimable: 1294496 kB
BdiDirtyThresh: 204208088 kB
DirtyThresh: 195259944 kB
BackgroundThresh: 32503588 kB
BdiDirtied: 48519296 kB
BdiWritten: 47225696 kB
BdiWriteBandwidth: 1173892 kBps
b_dirty: 1
b_io: 0
b_more_io: 1
b_dirty_time: 0
bdi_list: 1
state: 1
/* per wb writeback info of bdi is collected */
cat /sys/kernel/debug/bdi/252:16/wb_stats
WbCgIno: 1
WbWriteback: 0 kB
WbReclaimable: 0 kB
WbDirtyThresh: 0 kB
WbDirtied: 0 kB
WbWritten: 0 kB
WbWriteBandwidth: 102400 kBps
b_dirty: 0
b_io: 0
b_more_io: 0
b_dirty_time: 0
state: 1
WbCgIno: 4208
WbWriteback: 59808 kB
WbReclaimable: 676480 kB
WbDirtyThresh: 6004624 kB
WbDirtied: 23348192 kB
WbWritten: 22614592 kB
WbWriteBandwidth: 593204 kBps
b_dirty: 1
b_io: 1
b_more_io: 0
b_dirty_time: 0
state: 7
WbCgIno: 4249
WbWriteback: 144256 kB
WbReclaimable: 432096 kB
WbDirtyThresh: 6004344 kB
WbDirtied: 25727744 kB
WbWritten: 25154752 kB
WbWriteBandwidth: 577904 kBps
b_dirty: 0
b_io: 1
b_more_io: 0
b_dirty_time: 0
state: 7
The wb_monitor.py script output is as following:
./wb_monitor.py 252:16 -c
writeback reclaimable dirtied written avg_bw
252:16_1 0 0 0 0 102400
252:16_4284 672 820064 9230368 8410304 685612
252:16_4325 896 819840 10491264 9671648 652348
252:16 1568 1639904 19721632 18081952 1440360
writeback reclaimable dirtied written avg_bw
252:16_1 0 0 0 0 102400
252:16_4284 672 820064 9230368 8410304 685612
252:16_4325 896 819840 10491264 9671648 652348
252:16 1568 1639904 19721632 18081952 1440360
...
Kemeng Shi (5):
writeback: collect stats of all wb of bdi in bdi_debug_stats_show
writeback: support retrieving per group debug writeback stats of bdi
writeback: fix build problems of "writeback: support retrieving per
group debug writeback stats of bdi"
writeback: add wb_monitor.py script to monitor writeback info on bdi
writeback: rename nr_reclaimable to nr_dirty in balance_dirty_pages
include/linux/writeback.h | 1 +
mm/backing-dev.c | 177 +++++++++++++++++++++++++++++-----
mm/page-writeback.c | 27 +++++-
tools/writeback/wb_monitor.py | 172 +++++++++++++++++++++++++++++++++
4 files changed, 348 insertions(+), 29 deletions(-)
create mode 100644 tools/writeback/wb_monitor.py
--
2.30.0
next reply other threads:[~2024-04-23 3:46 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-23 3:46 Kemeng Shi [this message]
2024-04-23 3:46 ` [PATCH v5 1/5] writeback: collect stats of all wb of bdi in bdi_debug_stats_show Kemeng Shi
2024-04-23 3:46 ` [PATCH v5 2/5] writeback: support retrieving per group debug writeback stats of bdi Kemeng Shi
2024-04-23 3:46 ` [PATCH v5 3/5] writeback: fix build problems of "writeback: support retrieving per group debug writeback stats of bdi" Kemeng Shi
2024-04-23 3:53 ` Kemeng Shi
2024-04-24 13:27 ` Johannes Weiner
2024-04-25 1:22 ` Kemeng Shi
2024-04-23 3:46 ` [PATCH v5 4/5] writeback: add wb_monitor.py script to monitor writeback info on bdi Kemeng Shi
2024-04-23 3:46 ` [PATCH v5 5/5] writeback: rename nr_reclaimable to nr_dirty in balance_dirty_pages Kemeng Shi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240423034643.141219-1-shikemeng@huaweicloud.com \
--to=shikemeng@huaweicloud.com \
--cc=akpm@linux-foundation.org \
--cc=bfoster@redhat.com \
--cc=dhowells@redhat.com \
--cc=dsterba@suse.com \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mjguzik@gmail.com \
--cc=tj@kernel.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).