From: Julia Lawall <julia.lawall@inria.fr>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Linux Trace Devel <linux-trace-devel@vger.kernel.org>,
Yordan Karadzhov <y.karadz@gmail.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
Ross Zwisler <zwisler@google.com>
Subject: Re: [PATCH] libtracecmd: Use an rbtree for mapping of cache pages
Date: Tue, 17 Oct 2023 16:57:49 +0200 (CEST) [thread overview]
Message-ID: <cbfe1ac7-3cda-b58d-8c22-b5ffa83aed18@inria.fr> (raw)
In-Reply-To: <20231017103548.5b2f7af7@gandalf.local.home>
On Tue, 17 Oct 2023, Steven Rostedt wrote:
> On Tue, 17 Oct 2023 09:34:30 +0200 (CEST)
> Julia Lawall <julia.lawall@inria.fr> wrote:
>
> > > And exited out when it finished loading and the result was:
> > >
> > > real 6m14.772s
> > > user 6m0.649s
> > > sys 0m12.718s
> > >
> > > That's over 6 minutes to load the trace.dat file!!!
> > >
> > > I ran perf record on it and it showed 77% of the time was in free_zpage().
> > >
> > > I pulled out my old algorithms book and wrote up a rbtree for internal use
> > > of libtracecmd. Then I switched the cache into a binary rbtree to do the
> > > look ups. As the lookups used both where the memory of the compressed page
> > > is mapped as well as the offset depending on how the search was done, I
> > > found that it only used the memory allocation address in one location.
> > > Luckily, the memory allocation mapping lookup also had access to the
> > > offset of the file the memory represented. That allowed me to make all
> > > lookups use the file offset (Thanks to Masami Hiramatsu for helping me
> > > realize that).
> > >
> > > After converting the cache to an rbtree lookup, I ran kernelshark again on
> > > opening that file and exited out as soon as it finished loading and the
> > > timings was:
> > >
> > > real 1m22.356s
> > > user 1m10.532s
> > > sys 0m10.901s
> > >
> > > Still a bit long, but it dropped from over 6 minutes to under 1 1/2
> > > minutes. Also, free_zpages() was no longer in the perf record output.
> >
> > Does it impact trace-cmd report?
>
> Not as drastically as the above, but running this on the same trace.dat
> file without the patch:
>
> $ time trace-cmd report trace.dat > /dev/null
> real 9m20.390s
> user 9m16.391s
> sys 0m3.529s
>
> With the patch:
>
> $ time trace-cmd report trace.dat > /dev/null
> real 6m22.935s
> user 6m19.537s
> sys 0m3.139s
>
> So it does bring it down by a third.
Great!
julia
>
> I need to send a v2 as I found I left some debugging code in, as well as I
> found a small bug in the update of the color of the deleted node if it
> wasn't the node to be deleted.
>
> -- Steve
>
prev parent reply other threads:[~2023-10-17 14:57 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-17 3:00 [PATCH] libtracecmd: Use an rbtree for mapping of cache pages Steven Rostedt
2023-10-17 7:34 ` Julia Lawall
2023-10-17 14:35 ` Steven Rostedt
2023-10-17 14:57 ` Julia Lawall [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cbfe1ac7-3cda-b58d-8c22-b5ffa83aed18@inria.fr \
--to=julia.lawall@inria.fr \
--cc=linux-trace-devel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=rostedt@goodmis.org \
--cc=y.karadz@gmail.com \
--cc=zwisler@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).