From: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
Steven Rostedt <rostedt@goodmis.org>,
Florent Revest <revest@chromium.org>
Cc: linux-trace-kernel@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>,
Martin KaFai Lau <martin.lau@linux.dev>,
bpf <bpf@vger.kernel.org>, Sven Schnelle <svens@linux.ibm.com>,
Alexei Starovoitov <ast@kernel.org>, Jiri Olsa <jolsa@kernel.org>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Alan Maguire <alan.maguire@oracle.com>,
Mark Rutland <mark.rutland@arm.com>,
Peter Zijlstra <peterz@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>, Guo Ren <guoren@kernel.org>
Subject: [PATCH v10 00/36] tracing: fprobe: function_graph: Multi-function graph and fprobe on fgraph
Date: Tue, 7 May 2024 23:08:00 +0900 [thread overview]
Message-ID: <171509088006.162236.7227326999861366050.stgit@devnote2> (raw)
Hi,
Here is the 10th version of the series to re-implement the fprobe on
function-graph tracer. The previous version is;
https://lore.kernel.org/all/171318533841.254850.15841395205784342850.stgit@devnote2/
This version is ported on the latest kernel (v6.9-rc6 + probes/for-next)
and fixed some bugs + performance optimizations.
- [7/36] Fix terminology in comments and code. Use "offset" instead
of "index" for shadow stack. This also update macros.
- [18/36] Fix supported data size bug.
- [29/36] Define bpf_kprobe_multi_pt_regs only if it is used.
- [36/36] Add likely() to skip timestamp.
Overview
--------
This series does major 2 changes, enable multiple function-graphs on
the ftrace (e.g. allow function-graph on sub instances) and rewrite the
fprobe on this function-graph.
The former changes had been sent from Steven Rostedt 4 years ago (*),
which allows users to set different setting function-graph tracer (and
other tracers based on function-graph) in each trace-instances at the
same time.
(*) https://lore.kernel.org/all/20190525031633.811342628@goodmis.org/
The purpose of latter change are;
1) Remove dependency of the rethook from fprobe so that we can reduce
the return hook code and shadow stack.
2) Make 'ftrace_regs' the common trace interface for the function
boundary.
1) Currently we have 2(or 3) different function return hook codes,
the function-graph tracer and rethook (and legacy kretprobe).
But since this is redundant and needs double maintenance cost,
I would like to unify those. From the user's viewpoint, function-
graph tracer is very useful to grasp the execution path. For this
purpose, it is hard to use the rethook in the function-graph
tracer, but the opposite is possible. (Strictly speaking, kretprobe
can not use it because it requires 'pt_regs' for historical reasons.)
2) Now the fprobe provides the 'pt_regs' for its handler, but that is
wrong for the function entry and exit. Moreover, depending on the
architecture, there is no way to accurately reproduce 'pt_regs'
outside of interrupt or exception handlers. This means fprobe should
not use 'pt_regs' because it does not use such exceptions.
(Conversely, kprobe should use 'pt_regs' because it is an abstract
interface of the software breakpoint exception.)
This series changes fprobe to use function-graph tracer for tracing
function entry and exit, instead of mixture of ftrace and rethook.
Unlike the rethook which is a per-task list of system-wide allocated
nodes, the function graph's ret_stack is a per-task shadow stack.
Thus it does not need to set 'nr_maxactive' (which is the number of
pre-allocated nodes).
Also the handlers will get the 'ftrace_regs' instead of 'pt_regs'.
Since eBPF mulit_kprobe/multi_kretprobe events still use 'pt_regs' as
their register interface, this changes it to convert 'ftrace_regs' to
'pt_regs'. Of course this conversion makes an incomplete 'pt_regs',
so users must access only registers for function parameters or
return value.
Design
------
Instead of using ftrace's function entry hook directly, the new fprobe
is built on top of the function-graph's entry and return callbacks
with 'ftrace_regs'.
Since the fprobe requires access to 'ftrace_regs', the architecture
must support CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS and
CONFIG_HAVE_FTRACE_GRAPH_FUNC, which enables to call function-graph
entry callback with 'ftrace_regs', and also
CONFIG_HAVE_FUNCTION_GRAPH_FREGS, which passes the ftrace_regs to
return_to_handler.
All fprobes share a single function-graph ops (means shares a common
ftrace filter) similar to the kprobe-on-ftrace. This needs another
layer to find corresponding fprobe in the common function-graph
callbacks, but has much better scalability, since the number of
registered function-graph ops is limited.
In the entry callback, the fprobe runs its entry_handler and saves the
address of 'fprobe' on the function-graph's shadow stack as data. The
return callback decodes the data to get the 'fprobe' address, and runs
the exit_handler.
The fprobe introduces two hash-tables, one is for entry callback which
searches fprobes related to the given function address passed by entry
callback. The other is for a return callback which checks if the given
'fprobe' data structure pointer is still valid. Note that it is
possible to unregister fprobe before the return callback runs. Thus
the address validation must be done before using it in the return
callback.
This series can be applied against the probes/for-next branch, which
is based on v6.9-rc6.
This series can also be found below branch.
https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git/log/?h=topic/fprobe-on-fgraph
Thank you,
---
Masami Hiramatsu (Google) (21):
tracing: Add a comment about ftrace_regs definition
tracing: Rename ftrace_regs_return_value to ftrace_regs_get_return_value
x86: tracing: Add ftrace_regs definition in the header
function_graph: Use a simple LRU for fgraph_array index number
ftrace: Add multiple fgraph storage selftest
function_graph: Pass ftrace_regs to entryfunc
function_graph: Replace fgraph_ret_regs with ftrace_regs
function_graph: Pass ftrace_regs to retfunc
fprobe: Use ftrace_regs in fprobe entry handler
fprobe: Use ftrace_regs in fprobe exit handler
tracing: Add ftrace_partial_regs() for converting ftrace_regs to pt_regs
tracing: Add ftrace_fill_perf_regs() for perf event
tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS
bpf: Enable kprobe_multi feature if CONFIG_FPROBE is enabled
ftrace: Add CONFIG_HAVE_FTRACE_GRAPH_FUNC
fprobe: Rewrite fprobe on function-graph tracer
tracing/fprobe: Remove nr_maxactive from fprobe
selftests: ftrace: Remove obsolate maxactive syntax check
selftests/ftrace: Add a test case for repeating register/unregister fprobe
Documentation: probes: Update fprobe on function-graph tracer
fgraph: Skip recording calltime/rettime if it is not nneeded
Steven Rostedt (VMware) (15):
function_graph: Convert ret_stack to a series of longs
fgraph: Use BUILD_BUG_ON() to make sure we have structures divisible by long
function_graph: Add an array structure that will allow multiple callbacks
function_graph: Allow multiple users to attach to function graph
function_graph: Remove logic around ftrace_graph_entry and return
ftrace/function_graph: Pass fgraph_ops to function graph callbacks
ftrace: Allow function_graph tracer to be enabled in instances
ftrace: Allow ftrace startup flags exist without dynamic ftrace
function_graph: Have the instances use their own ftrace_ops for filtering
function_graph: Add "task variables" per task for fgraph_ops
function_graph: Move set_graph_function tests to shadow stack global var
function_graph: Move graph depth stored data to shadow stack global var
function_graph: Move graph notrace bit to shadow stack global var
function_graph: Implement fgraph_reserve_data() and fgraph_retrieve_data()
function_graph: Add selftest for passing local variables
Documentation/trace/fprobe.rst | 42 +
arch/arm64/Kconfig | 3
arch/arm64/include/asm/ftrace.h | 47 +
arch/arm64/kernel/asm-offsets.c | 12
arch/arm64/kernel/entry-ftrace.S | 32 -
arch/arm64/kernel/ftrace.c | 21
arch/loongarch/Kconfig | 4
arch/loongarch/include/asm/ftrace.h | 32 -
arch/loongarch/kernel/asm-offsets.c | 12
arch/loongarch/kernel/ftrace_dyn.c | 15
arch/loongarch/kernel/mcount.S | 17
arch/loongarch/kernel/mcount_dyn.S | 14
arch/powerpc/Kconfig | 1
arch/powerpc/include/asm/ftrace.h | 15
arch/powerpc/kernel/trace/ftrace.c | 3
arch/powerpc/kernel/trace/ftrace_64_pg.c | 10
arch/riscv/Kconfig | 3
arch/riscv/include/asm/ftrace.h | 21
arch/riscv/kernel/ftrace.c | 15
arch/riscv/kernel/mcount.S | 24
arch/s390/Kconfig | 3
arch/s390/include/asm/ftrace.h | 39 -
arch/s390/kernel/asm-offsets.c | 6
arch/s390/kernel/mcount.S | 9
arch/x86/Kconfig | 4
arch/x86/include/asm/ftrace.h | 43 -
arch/x86/kernel/ftrace.c | 51 +
arch/x86/kernel/ftrace_32.S | 15
arch/x86/kernel/ftrace_64.S | 17
include/linux/fprobe.h | 57 +
include/linux/ftrace.h | 172 +++
include/linux/sched.h | 2
include/linux/trace_recursion.h | 39 -
kernel/trace/Kconfig | 23
kernel/trace/bpf_trace.c | 19
kernel/trace/fgraph.c | 1024 ++++++++++++++++----
kernel/trace/fprobe.c | 637 +++++++++---
kernel/trace/ftrace.c | 13
kernel/trace/ftrace_internal.h | 2
kernel/trace/trace.h | 96 ++
kernel/trace/trace_fprobe.c | 147 +--
kernel/trace/trace_functions.c | 8
kernel/trace/trace_functions_graph.c | 98 +-
kernel/trace/trace_irqsoff.c | 12
kernel/trace/trace_probe_tmpl.h | 2
kernel/trace/trace_sched_wakeup.c | 12
kernel/trace/trace_selftest.c | 262 +++++
lib/test_fprobe.c | 51 -
samples/fprobe/fprobe_example.c | 4
.../test.d/dynevent/add_remove_fprobe_repeat.tc | 19
.../ftrace/test.d/dynevent/fprobe_syntax_errors.tc | 4
51 files changed, 2347 insertions(+), 886 deletions(-)
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_fprobe_repeat.tc
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
next reply other threads:[~2024-05-07 14:08 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 14:08 Masami Hiramatsu (Google) [this message]
2024-05-07 14:08 ` [PATCH v10 01/36] tracing: Add a comment about ftrace_regs definition Masami Hiramatsu (Google)
2024-05-07 14:08 ` [PATCH v10 02/36] tracing: Rename ftrace_regs_return_value to ftrace_regs_get_return_value Masami Hiramatsu (Google)
2024-05-07 14:08 ` [PATCH v10 03/36] x86: tracing: Add ftrace_regs definition in the header Masami Hiramatsu (Google)
2024-05-07 14:08 ` [PATCH v10 04/36] function_graph: Convert ret_stack to a series of longs Masami Hiramatsu (Google)
2024-05-07 14:08 ` [PATCH v10 05/36] fgraph: Use BUILD_BUG_ON() to make sure we have structures divisible by long Masami Hiramatsu (Google)
2024-05-07 14:09 ` [PATCH v10 06/36] function_graph: Add an array structure that will allow multiple callbacks Masami Hiramatsu (Google)
2024-05-07 14:09 ` [PATCH v10 07/36] function_graph: Allow multiple users to attach to function graph Masami Hiramatsu (Google)
2024-05-07 14:09 ` [PATCH v10 08/36] function_graph: Remove logic around ftrace_graph_entry and return Masami Hiramatsu (Google)
2024-05-07 14:09 ` [PATCH v10 09/36] ftrace/function_graph: Pass fgraph_ops to function graph callbacks Masami Hiramatsu (Google)
2024-05-07 14:09 ` [PATCH v10 10/36] ftrace: Allow function_graph tracer to be enabled in instances Masami Hiramatsu (Google)
2024-05-07 14:10 ` [PATCH v10 11/36] ftrace: Allow ftrace startup flags exist without dynamic ftrace Masami Hiramatsu (Google)
2024-05-07 14:10 ` [PATCH v10 12/36] function_graph: Have the instances use their own ftrace_ops for filtering Masami Hiramatsu (Google)
2024-05-07 14:10 ` [PATCH v10 13/36] function_graph: Use a simple LRU for fgraph_array index number Masami Hiramatsu (Google)
2024-05-07 14:10 ` [PATCH v10 14/36] function_graph: Add "task variables" per task for fgraph_ops Masami Hiramatsu (Google)
2024-05-07 14:10 ` [PATCH v10 15/36] function_graph: Move set_graph_function tests to shadow stack global var Masami Hiramatsu (Google)
2024-05-07 14:11 ` [PATCH v10 16/36] function_graph: Move graph depth stored data " Masami Hiramatsu (Google)
2024-05-07 14:11 ` [PATCH v10 17/36] function_graph: Move graph notrace bit " Masami Hiramatsu (Google)
2024-05-07 14:11 ` [PATCH v10 18/36] function_graph: Implement fgraph_reserve_data() and fgraph_retrieve_data() Masami Hiramatsu (Google)
2024-05-07 14:11 ` [PATCH v10 19/36] function_graph: Add selftest for passing local variables Masami Hiramatsu (Google)
2024-05-07 14:11 ` [PATCH v10 20/36] ftrace: Add multiple fgraph storage selftest Masami Hiramatsu (Google)
2024-05-07 14:12 ` [PATCH v10 21/36] function_graph: Pass ftrace_regs to entryfunc Masami Hiramatsu (Google)
2024-05-07 14:12 ` [PATCH v10 22/36] function_graph: Replace fgraph_ret_regs with ftrace_regs Masami Hiramatsu (Google)
2024-05-07 14:12 ` [PATCH v10 23/36] function_graph: Pass ftrace_regs to retfunc Masami Hiramatsu (Google)
2024-05-07 14:12 ` [PATCH v10 24/36] fprobe: Use ftrace_regs in fprobe entry handler Masami Hiramatsu (Google)
2024-05-07 14:12 ` [PATCH v10 25/36] fprobe: Use ftrace_regs in fprobe exit handler Masami Hiramatsu (Google)
2024-05-07 14:13 ` [PATCH v10 26/36] tracing: Add ftrace_partial_regs() for converting ftrace_regs to pt_regs Masami Hiramatsu (Google)
2024-05-07 14:13 ` [PATCH v10 27/36] tracing: Add ftrace_fill_perf_regs() for perf event Masami Hiramatsu (Google)
2024-05-07 14:13 ` [PATCH v10 28/36] tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS Masami Hiramatsu (Google)
2024-05-07 14:13 ` [PATCH v10 29/36] bpf: Enable kprobe_multi feature if CONFIG_FPROBE is enabled Masami Hiramatsu (Google)
2024-05-07 14:13 ` [PATCH v10 30/36] ftrace: Add CONFIG_HAVE_FTRACE_GRAPH_FUNC Masami Hiramatsu (Google)
2024-05-07 14:14 ` [PATCH v10 31/36] fprobe: Rewrite fprobe on function-graph tracer Masami Hiramatsu (Google)
2024-05-07 14:14 ` [PATCH v10 32/36] tracing/fprobe: Remove nr_maxactive from fprobe Masami Hiramatsu (Google)
2024-05-07 14:14 ` [PATCH v10 33/36] selftests: ftrace: Remove obsolate maxactive syntax check Masami Hiramatsu (Google)
2024-05-07 14:14 ` [PATCH v10 34/36] selftests/ftrace: Add a test case for repeating register/unregister fprobe Masami Hiramatsu (Google)
2024-05-07 14:14 ` [PATCH v10 35/36] Documentation: probes: Update fprobe on function-graph tracer Masami Hiramatsu (Google)
2024-05-07 14:15 ` [PATCH v10 36/36] fgraph: Skip recording calltime/rettime if it is not nneeded Masami Hiramatsu (Google)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=171509088006.162236.7227326999861366050.stgit@devnote2 \
--to=mhiramat@kernel.org \
--cc=acme@kernel.org \
--cc=alan.maguire@oracle.com \
--cc=alexei.starovoitov@gmail.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=guoren@kernel.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=martin.lau@linux.dev \
--cc=peterz@infradead.org \
--cc=revest@chromium.org \
--cc=rostedt@goodmis.org \
--cc=svens@linux.ibm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).