From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Ian Rogers <irogers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>,
Adrian Hunter <adrian.hunter@intel.com>,
Kan Liang <kan.liang@linux.intel.com>,
James Clark <james.clark@arm.com>,
Athira Rajeev <atrajeev@linux.vnet.ibm.com>,
Colin Ian King <colin.i.king@gmail.com>,
nabijaczleweli@nabijaczleweli.xyz, Leo Yan <leo.yan@linux.dev>,
Song Liu <song@kernel.org>,
Ilkka Koskinen <ilkka@os.amperecomputing.com>,
Ben Gainey <ben.gainey@arm.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
Yanteng Si <siyanteng@loongson.cn>,
Sun Haiyong <sunhaiyong@loongson.cn>,
Changbin Du <changbin.du@huawei.com>,
Andi Kleen <ak@linux.intel.com>,
Thomas Richter <tmricht@linux.ibm.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
Dima Kogan <dima@secretsauce.net>,
zhaimingbing <zhaimingbing@cmss.chinamobile.com>,
Paran Lee <p4ranlee@gmail.com>, Li Dong <lidong@vivo.com>,
Tiezhu Yang <yangtiezhu@loongson.cn>,
Yang Jihong <yangjihong1@huawei.com>,
Chengen Du <chengen.du@canonical.com>,
linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 0/7] dso/dsos memory savings and clean up
Date: Mon, 29 Apr 2024 16:16:59 -0300 [thread overview]
Message-ID: <Zi_yKzGMoDU0-L3K@x1> (raw)
In-Reply-To: <20240429184614.1224041-1-irogers@google.com>
On Mon, Apr 29, 2024 at 11:46:07AM -0700, Ian Rogers wrote:
> 7 more patches from:
> https://lore.kernel.org/lkml/20240202061532.1939474-1-irogers@google.com/
> a near half year old adventure in trying to lower perf's dynamic
> memory use. Bits like the memory overhead of opendir are on the
> sidelines for now, too much fighting over how
> distributions/C-libraries present getdents. These changes are more
> good old fashioned replace an rb-tree with a sorted array and add
> reference count tracking.
>
> The changes migrate dsos code, the collection of dso structs, more
> into the dsos.c/dsos.h files. As with maps and threads, this is done
> so the internals can be changed - replacing a linked list (for fast
> iteration) and an rb-tree (for fast finds) with a lazily sorted
> array. The complexity of operations remain roughly the same, although
> iterating an array is likely faster than iterating a linked list, the
Th> memory usage is at least reduced by half.
>
> As fixing the memory usage necessitates changing operations like find,
> modify these operations so that they increment the reference count to
> avoid races like a find in dsos and a remove. Similarly tighten up
> lock usage so that operations working on dsos state hold the
> appropriate lock. Note, since this series is partially applied in the
> perf-tools-next tree currently some memory leaks have been introduced.
>
> v5. Rebase, adding use of accessors to dso as necessary. Previous
> versions were all rebases or dropping merged patches.
So, on an Intel machine:
⬢[acme@toolbox perf-tools-next]$ git log --oneline -8
fb401385575211e6 (HEAD -> perf-tools-next) perf dso: Use container_of to avoid a pointer in 'struct dso_data'
0fe118d129ab1c77 perf dso: Reference counting related fixes
35e44adf6103a407 perf dso: Add reference count checking and accessor functions
35673675ebbbac5d perf dsos: Switch hand code to bsearch()
654d60f2f5c737cd perf dsos: Remove __dsos__findnew_link_by_longname_id()
94b0ba802e090b66 perf dsos: Remove __dsos__addnew()
47692286dd856469 perf dsos: Switch backing storage to array from rbtree/list
8c618b58c89ce4c2 (perf-tools-next.korg/perf-tools-next, acme.korg/perf-tools-next) perf test: Reintroduce -p/--parallel and make -S/--sequential the default
⬢[acme@toolbox perf-tools-next]$
When I'm at:
8c618b58c89ce4c2 (perf-tools-next.korg/perf-tools-next, acme.korg/perf-tools-next) perf test: Reintroduce -p/--parallel and make -S/--sequential the default
root@x1:~# perf -v
perf version 6.9.rc5.g8c618b58c89c
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : Ok
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : Ok
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : Ok
root@x1:~# time perf test "lock contention"
87: kernel lock contention analysis test : Ok
real 0m9.143s
user 0m5.201s
sys 0m4.812s
root@x1:~#
Moving to the first patch in this series:
⬢[acme@toolbox perf-tools-next]$ git log --oneline -2
47692286dd856469 (HEAD) perf dsos: Switch backing storage to array from rbtree/list
8c618b58c89ce4c2 (perf-tools-next.korg/perf-tools-next, acme.korg/perf-tools-next) perf test: Reintroduce -p/--parallel and make -S/--sequential the default
⬢[acme@toolbox perf-tools-next]$ alias m
alias m='rm -rf ~/libexec/perf-core/ ; make -k CORESIGHT=1 O=/tmp/build/$(basename $PWD)/ -C tools/perf install-bin && perf test pythond'
⬢[acme@toolbox perf-tools-next]$ rm -rf /tmp/build/$(basename $PWD)/ ; mkdir -p /tmp/build/$(basename $PWD)/ ; m
<SNIP>
root@x1:~# perf -v
perf version 6.9.rc5.g47692286dd85
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test -vvv "lock contention"
87: kernel lock contention analysis test:
--- start ---
test child forked, pid 2279518
Testing perf lock record and perf lock contention
[Fail] Recorded result count is not 1: 2
---- end(-1) ----
87: kernel lock contention analysis test : FAILED!
root@x1:~#
This breaks bisectability, but then lets see if at the end of the series
it works:
⬢[acme@toolbox perf-tools-next]$ git log --oneline -8
fb401385575211e6 (HEAD -> perf-tools-next) perf dso: Use container_of to avoid a pointer in 'struct dso_data'
0fe118d129ab1c77 perf dso: Reference counting related fixes
35e44adf6103a407 perf dso: Add reference count checking and accessor functions
35673675ebbbac5d perf dsos: Switch hand code to bsearch()
654d60f2f5c737cd perf dsos: Remove __dsos__findnew_link_by_longname_id()
94b0ba802e090b66 perf dsos: Remove __dsos__addnew()
47692286dd856469 perf dsos: Switch backing storage to array from rbtree/list
8c618b58c89ce4c2 (perf-tools-next.korg/perf-tools-next, acme.korg/perf-tools-next) perf test: Reintroduce -p/--parallel and make -S/--sequential the default
⬢[acme@toolbox perf-tools-next]$ rm -rf /tmp/build/$(basename $PWD)/ ; mkdir -p /tmp/build/$(basename $PWD)/ ; m
root@x1:~# perf -v
perf version 6.9.rc5.gfb4013855752
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test "lock contention"
87: kernel lock contention analysis test : FAILED!
root@x1:~# perf test -vvv "lock contention"
87: kernel lock contention analysis test:
--- start ---
test child forked, pid 2289060
Testing perf lock record and perf lock contention
Testing perf lock contention --use-bpf
Testing perf lock record and perf lock contention at the same time
[Fail] Recorded result count is not 1: 2
---- end(-1) ----
87: kernel lock contention analysis test : FAILED!
root@x1:~#
⬢[acme@toolbox perf-tools-next]$ gcc --version | head -1
gcc (GCC) 13.2.1 20240316 (Red Hat 13.2.1-7)
⬢[acme@toolbox perf-tools-next]$ rpm -q gcc
gcc-13.2.1-7.fc39.x86_64
⬢[acme@toolbox perf-tools-next]$
root@x1:~# grep -m1 'model name' /proc/cpuinfo
model name : 13th Gen Intel(R) Core(TM) i7-1365U
root@x1:~# free
total used free shared buff/cache available
Mem: 32507912 9081644 3531432 1987616 22554868 23426268
Swap: 8388604 314112 8074492
root@x1:~#
- Arnaldo
next prev parent reply other threads:[~2024-04-29 19:17 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-29 18:46 [PATCH v5 0/7] dso/dsos memory savings and clean up Ian Rogers
2024-04-29 18:46 ` [PATCH v5 1/7] perf dsos: Switch backing storage to array from rbtree/list Ian Rogers
2024-05-03 20:21 ` Arnaldo Carvalho de Melo
2024-05-04 18:14 ` Ian Rogers
2024-05-04 18:28 ` Arnaldo Carvalho de Melo
2024-04-29 18:46 ` [PATCH v5 2/7] perf dsos: Remove __dsos__addnew Ian Rogers
2024-04-29 18:46 ` [PATCH v5 3/7] perf dsos: Remove __dsos__findnew_link_by_longname_id Ian Rogers
2024-04-29 18:46 ` [PATCH v5 4/7] perf dsos: Switch hand code to bsearch Ian Rogers
2024-04-29 18:46 ` [PATCH v5 5/7] perf dso: Add reference count checking and accessor functions Ian Rogers
2024-04-29 18:46 ` [PATCH v5 6/7] perf dso: Reference counting related fixes Ian Rogers
2024-04-29 18:46 ` [PATCH v5 7/7] perf dso: Use container_of to avoid a pointer in dso_data Ian Rogers
2024-04-29 19:16 ` Arnaldo Carvalho de Melo [this message]
2024-04-29 19:50 ` [PATCH v5 0/7] dso/dsos memory savings and clean up Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zi_yKzGMoDU0-L3K@x1 \
--to=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=atrajeev@linux.vnet.ibm.com \
--cc=ben.gainey@arm.com \
--cc=changbin.du@huawei.com \
--cc=chengen.du@canonical.com \
--cc=colin.i.king@gmail.com \
--cc=dima@secretsauce.net \
--cc=ilkka@os.amperecomputing.com \
--cc=irogers@google.com \
--cc=james.clark@arm.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=kprateek.nayak@amd.com \
--cc=leo.yan@linux.dev \
--cc=lidong@vivo.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=nabijaczleweli@nabijaczleweli.xyz \
--cc=namhyung@kernel.org \
--cc=p4ranlee@gmail.com \
--cc=peterz@infradead.org \
--cc=siyanteng@loongson.cn \
--cc=song@kernel.org \
--cc=sunhaiyong@loongson.cn \
--cc=tmricht@linux.ibm.com \
--cc=yangjihong1@huawei.com \
--cc=yangtiezhu@loongson.cn \
--cc=zhaimingbing@cmss.chinamobile.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).