* [PATCHv6 0/4] perf: Add mmap2 build id support
@ 2021-01-11 21:38 Jiri Olsa
2021-01-11 21:38 ` [PATCH 1/4] bpf: Move stack_map_get_build_id into lib Jiri Olsa
` (4 more replies)
0 siblings, 5 replies; 8+ messages in thread
From: Jiri Olsa @ 2021-01-11 21:38 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: lkml, Peter Zijlstra, Ingo Molnar, Mark Rutland, Namhyung Kim,
Alexander Shishkin, Michael Petlan, Song Liu, Ian Rogers,
Stephane Eranian, Alexei Budankov, Andi Kleen, Adrian Hunter
hi,
adding the support to have buildid stored in mmap2 event,
so we can bypass the final perf record hunt on build ids.
This patchset allows perf to record build ID in mmap2 event,
and adds perf tooling to store/download binaries to .debug
cache based on these build IDs.
Note that the build id retrieval code is stolen from bpf
code, where it's been used (together with file offsets)
to replace IPs in user space stack traces. It's now added
under lib directory.
v6 changes:
- last 4 patches rebased Arnaldo's perf/core
v5 changes:
- rebased on latest perf/core
- several patches already pulled in
- fixed trace+probe_vfs_getname.sh output redirection
- fixed changelogs [Arnaldo]
- renamed BUILD_ID_SIZE to BUILD_ID_SIZE_MAX [Song]
v4 changes:
- fixed typo in changelog [Namhyung]
- removed force_download bool from struct dso_store_data,
because it's not used [Namhyung]
v3 changes:
- added acks
- removed forgotten debug code [Arnaldo]
- fixed readlink termination [Ian]
- fixed doc for --debuginfod=URLs [Ian]
- adopted kernel's memchr_inv function and used
it in build_id__is_defined function [Arnaldo]
On recording server:
- on the recording server we can run record with --buildid-mmap
option to store build ids in mmap2 events:
# perf record --buildid-mmap
^C[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.836 MB perf.data ]
- it stores nothing to ~/.debug cache:
# find ~/.debug
find: ‘/root/.debug’: No such file or directory
- and still reports properly:
# perf report --stdio
...
99.82% swapper [kernel.kallsyms] [k] native_safe_halt
0.03% swapper [kernel.kallsyms] [k] finish_task_switch
0.02% swapper [kernel.kallsyms] [k] __softirqentry_text_start
0.01% kcompactd0 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
0.01% ksoftirqd/6 [kernel.kallsyms] [k] slab_free_freelist_hook
0.01% kworker/17:1H-x [kernel.kallsyms] [k] slab_free_freelist_hook
- display used/hit build ids:
# perf buildid-list | head -5
5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore
589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g
94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so
559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so
40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so
- store build id binaries into build id cache:
# perf buildid-cache -a perf.data
OK 5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore
OK 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g
OK 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so
OK 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so
OK 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so
OK a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd
OK e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash
OK 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find
# find ~/.debug | head -5
/root/.debug
/root/.debug/proc
/root/.debug/proc/kcore
/root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7
/root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7/kallsyms
- run debuginfod daemon to provide binaries to another server (below)
(the initialization could take some time)
# debuginfod -F /
On another server:
- copy perf.data from 'record' server and run:
$ find ~/.debug/
find: ‘/home/jolsa/.debug/’: No such file or directory
$ perf buildid-list | head -5
No kallsyms or vmlinux with build-id 5dcec522abf136fcfd3128f47e131f2365834dd7 was found
5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms]
5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko
589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g
94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so
559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so
- report does not show anything (kernel build id does not match):
$ perf report --stdio
...
76.73% swapper [kernel.kallsyms] [k] 0xffffffff81aa8ebe
1.89% find [kernel.kallsyms] [k] 0xffffffff810f2167
0.93% sshd [kernel.kallsyms] [k] 0xffffffff8153380c
0.83% swapper [kernel.kallsyms] [k] 0xffffffff81104b0b
0.71% kworker/u40:2-e [kernel.kallsyms] [k] 0xffffffff810f3850
0.70% kworker/u40:0-e [kernel.kallsyms] [k] 0xffffffff810f3850
0.64% find [kernel.kallsyms] [k] 0xffffffff81a9ba0a
0.63% find [kernel.kallsyms] [k] 0xffffffff81aa93b0
- add build ids does not work, because existing binaries (on another server)
have different build ids:
$ perf buildid-cache -a perf.data
No kallsyms or vmlinux with build-id 5dcec522abf136fcfd3128f47e131f2365834dd7 was found
FAIL 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms]
FAIL 5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko
FAIL 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g
FAIL 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so
FAIL 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so
FAIL 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so
FAIL a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd
FAIL e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash
FAIL 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find
- add build ids with debuginfod setup pointing to record server:
$ perf buildid-cache -a perf.data --debuginfod http://192.168.122.174:8002
No kallsyms or vmlinux with build-id 5dcec522abf136fcfd3128f47e131f2365834dd7 was found
OK 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms]
OK 5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko
OK 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g
OK 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so
OK 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so
OK 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so
OK a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd
OK e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash
OK 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find
- and report works:
$ perf report --stdio
...
76.73% swapper [kernel.kallsyms] [k] native_safe_halt
1.91% find [kernel.kallsyms] [k] queue_work_on
0.93% sshd [kernel.kallsyms] [k] iowrite16
0.83% swapper [kernel.kallsyms] [k] finish_task_switch
0.72% kworker/u40:2-e [kernel.kallsyms] [k] process_one_work
0.70% kworker/u40:0-e [kernel.kallsyms] [k] process_one_work
0.64% find [kernel.kallsyms] [k] syscall_enter_from_user_mode
0.63% find [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
- because we have the data in build id cache:
$ find ~/.debug | head -10
.../.debug
.../.debug/home
.../.debug/home/jolsa
.../.debug/home/jolsa/.cache
.../.debug/home/jolsa/.cache/debuginfod_client
.../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7
.../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable
.../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7
.../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7/elf
.../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7/debug
Available also in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/build_id
thanks,
jirka
---
Jiri Olsa (4):
bpf: Move stack_map_get_build_id into lib
bpf: Add size arg to build_id_parse function
perf: Add build id data in mmap2 event
perf buildid-cache: Add support to add build ids from perf data
include/linux/buildid.h | 12 ++++++
include/uapi/linux/perf_event.h | 42 ++++++++++++++++++---
kernel/bpf/stackmap.c | 143 ++-------------------------------------------------------------------
kernel/events/core.c | 32 ++++++++++++++--
lib/Makefile | 3 +-
lib/buildid.c | 149 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
tools/perf/Documentation/perf-buildid-cache.txt | 12 +++++-
tools/perf/builtin-buildid-cache.c | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
tools/perf/util/probe-event.c | 6 +--
9 files changed, 457 insertions(+), 155 deletions(-)
create mode 100644 include/linux/buildid.h
create mode 100644 lib/buildid.c
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/4] bpf: Move stack_map_get_build_id into lib
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
@ 2021-01-11 21:38 ` Jiri Olsa
2021-01-11 21:38 ` [PATCH 2/4] bpf: Add size arg to build_id_parse function Jiri Olsa
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jiri Olsa @ 2021-01-11 21:38 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Alexei Starovoitov, Song Liu, lkml, Peter Zijlstra, Ingo Molnar,
Mark Rutland, Namhyung Kim, Alexander Shishkin, Michael Petlan,
Ian Rogers, Stephane Eranian, Alexei Budankov, Andi Kleen,
Adrian Hunter
Moving stack_map_get_build_id into lib with
declaration in linux/buildid.h header:
int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id);
This function returns build id for given struct vm_area_struct.
There is no functional change to stack_map_get_build_id function.
Cc: Alexei Starovoitov <ast@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
include/linux/buildid.h | 11 ++++
kernel/bpf/stackmap.c | 143 ++--------------------------------------
lib/Makefile | 3 +-
lib/buildid.c | 136 ++++++++++++++++++++++++++++++++++++++
4 files changed, 153 insertions(+), 140 deletions(-)
create mode 100644 include/linux/buildid.h
create mode 100644 lib/buildid.c
diff --git a/include/linux/buildid.h b/include/linux/buildid.h
new file mode 100644
index 000000000000..08028a212589
--- /dev/null
+++ b/include/linux/buildid.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_BUILDID_H
+#define _LINUX_BUILDID_H
+
+#include <linux/mm_types.h>
+
+#define BUILD_ID_SIZE_MAX 20
+
+int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id);
+
+#endif
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index aea96b638473..55d254a59f07 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -7,10 +7,9 @@
#include <linux/kernel.h>
#include <linux/stacktrace.h>
#include <linux/perf_event.h>
-#include <linux/elf.h>
-#include <linux/pagemap.h>
#include <linux/irq_work.h>
#include <linux/btf_ids.h>
+#include <linux/buildid.h>
#include "percpu_freelist.h"
#define STACK_CREATE_FLAG_MASK \
@@ -143,140 +142,6 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr)
return ERR_PTR(err);
}
-#define BPF_BUILD_ID 3
-/*
- * Parse build id from the note segment. This logic can be shared between
- * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are
- * identical.
- */
-static inline int stack_map_parse_build_id(void *page_addr,
- unsigned char *build_id,
- void *note_start,
- Elf32_Word note_size)
-{
- Elf32_Word note_offs = 0, new_offs;
-
- /* check for overflow */
- if (note_start < page_addr || note_start + note_size < note_start)
- return -EINVAL;
-
- /* only supports note that fits in the first page */
- if (note_start + note_size > page_addr + PAGE_SIZE)
- return -EINVAL;
-
- while (note_offs + sizeof(Elf32_Nhdr) < note_size) {
- Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_offs);
-
- if (nhdr->n_type == BPF_BUILD_ID &&
- nhdr->n_namesz == sizeof("GNU") &&
- nhdr->n_descsz > 0 &&
- nhdr->n_descsz <= BPF_BUILD_ID_SIZE) {
- memcpy(build_id,
- note_start + note_offs +
- ALIGN(sizeof("GNU"), 4) + sizeof(Elf32_Nhdr),
- nhdr->n_descsz);
- memset(build_id + nhdr->n_descsz, 0,
- BPF_BUILD_ID_SIZE - nhdr->n_descsz);
- return 0;
- }
- new_offs = note_offs + sizeof(Elf32_Nhdr) +
- ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4);
- if (new_offs <= note_offs) /* overflow */
- break;
- note_offs = new_offs;
- }
- return -EINVAL;
-}
-
-/* Parse build ID from 32-bit ELF */
-static int stack_map_get_build_id_32(void *page_addr,
- unsigned char *build_id)
-{
- Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr;
- Elf32_Phdr *phdr;
- int i;
-
- /* only supports phdr that fits in one page */
- if (ehdr->e_phnum >
- (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr))
- return -EINVAL;
-
- phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr));
-
- for (i = 0; i < ehdr->e_phnum; ++i) {
- if (phdr[i].p_type == PT_NOTE &&
- !stack_map_parse_build_id(page_addr, build_id,
- page_addr + phdr[i].p_offset,
- phdr[i].p_filesz))
- return 0;
- }
- return -EINVAL;
-}
-
-/* Parse build ID from 64-bit ELF */
-static int stack_map_get_build_id_64(void *page_addr,
- unsigned char *build_id)
-{
- Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr;
- Elf64_Phdr *phdr;
- int i;
-
- /* only supports phdr that fits in one page */
- if (ehdr->e_phnum >
- (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr))
- return -EINVAL;
-
- phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr));
-
- for (i = 0; i < ehdr->e_phnum; ++i) {
- if (phdr[i].p_type == PT_NOTE &&
- !stack_map_parse_build_id(page_addr, build_id,
- page_addr + phdr[i].p_offset,
- phdr[i].p_filesz))
- return 0;
- }
- return -EINVAL;
-}
-
-/* Parse build ID of ELF file mapped to vma */
-static int stack_map_get_build_id(struct vm_area_struct *vma,
- unsigned char *build_id)
-{
- Elf32_Ehdr *ehdr;
- struct page *page;
- void *page_addr;
- int ret;
-
- /* only works for page backed storage */
- if (!vma->vm_file)
- return -EINVAL;
-
- page = find_get_page(vma->vm_file->f_mapping, 0);
- if (!page)
- return -EFAULT; /* page not mapped */
-
- ret = -EINVAL;
- page_addr = kmap_atomic(page);
- ehdr = (Elf32_Ehdr *)page_addr;
-
- /* compare magic x7f "ELF" */
- if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG) != 0)
- goto out;
-
- /* only support executable file and shared object file */
- if (ehdr->e_type != ET_EXEC && ehdr->e_type != ET_DYN)
- goto out;
-
- if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
- ret = stack_map_get_build_id_32(page_addr, build_id);
- else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64)
- ret = stack_map_get_build_id_64(page_addr, build_id);
-out:
- kunmap_atomic(page_addr);
- put_page(page);
- return ret;
-}
-
static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
u64 *ips, u32 trace_nr, bool user)
{
@@ -317,18 +182,18 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
for (i = 0; i < trace_nr; i++) {
id_offs[i].status = BPF_STACK_BUILD_ID_IP;
id_offs[i].ip = ips[i];
- memset(id_offs[i].build_id, 0, BPF_BUILD_ID_SIZE);
+ memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX);
}
return;
}
for (i = 0; i < trace_nr; i++) {
vma = find_vma(current->mm, ips[i]);
- if (!vma || stack_map_get_build_id(vma, id_offs[i].build_id)) {
+ if (!vma || build_id_parse(vma, id_offs[i].build_id)) {
/* per entry fall back to ips */
id_offs[i].status = BPF_STACK_BUILD_ID_IP;
id_offs[i].ip = ips[i];
- memset(id_offs[i].build_id, 0, BPF_BUILD_ID_SIZE);
+ memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX);
continue;
}
id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ips[i]
diff --git a/lib/Makefile b/lib/Makefile
index afeff05fa8c5..a6b160c3a4fa 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -36,7 +36,8 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
flex_proportions.o ratelimit.o show_mem.o \
is_single_threaded.o plist.o decompress.o kobject_uevent.o \
earlycpio.o seq_buf.o siphash.o dec_and_lock.o \
- nmi_backtrace.o nodemask.o win_minmax.o memcat_p.o
+ nmi_backtrace.o nodemask.o win_minmax.o memcat_p.o \
+ buildid.o
lib-$(CONFIG_PRINTK) += dump_stack.o
lib-$(CONFIG_SMP) += cpumask.o
diff --git a/lib/buildid.c b/lib/buildid.c
new file mode 100644
index 000000000000..4a4f520c0e29
--- /dev/null
+++ b/lib/buildid.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/buildid.h>
+#include <linux/elf.h>
+#include <linux/pagemap.h>
+
+#define BUILD_ID 3
+/*
+ * Parse build id from the note segment. This logic can be shared between
+ * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are
+ * identical.
+ */
+static inline int parse_build_id(void *page_addr,
+ unsigned char *build_id,
+ void *note_start,
+ Elf32_Word note_size)
+{
+ Elf32_Word note_offs = 0, new_offs;
+
+ /* check for overflow */
+ if (note_start < page_addr || note_start + note_size < note_start)
+ return -EINVAL;
+
+ /* only supports note that fits in the first page */
+ if (note_start + note_size > page_addr + PAGE_SIZE)
+ return -EINVAL;
+
+ while (note_offs + sizeof(Elf32_Nhdr) < note_size) {
+ Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_offs);
+
+ if (nhdr->n_type == BUILD_ID &&
+ nhdr->n_namesz == sizeof("GNU") &&
+ nhdr->n_descsz > 0 &&
+ nhdr->n_descsz <= BUILD_ID_SIZE_MAX) {
+ memcpy(build_id,
+ note_start + note_offs +
+ ALIGN(sizeof("GNU"), 4) + sizeof(Elf32_Nhdr),
+ nhdr->n_descsz);
+ memset(build_id + nhdr->n_descsz, 0,
+ BUILD_ID_SIZE_MAX - nhdr->n_descsz);
+ return 0;
+ }
+ new_offs = note_offs + sizeof(Elf32_Nhdr) +
+ ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4);
+ if (new_offs <= note_offs) /* overflow */
+ break;
+ note_offs = new_offs;
+ }
+ return -EINVAL;
+}
+
+/* Parse build ID from 32-bit ELF */
+static int get_build_id_32(void *page_addr, unsigned char *build_id)
+{
+ Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr;
+ Elf32_Phdr *phdr;
+ int i;
+
+ /* only supports phdr that fits in one page */
+ if (ehdr->e_phnum >
+ (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr))
+ return -EINVAL;
+
+ phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr));
+
+ for (i = 0; i < ehdr->e_phnum; ++i) {
+ if (phdr[i].p_type == PT_NOTE &&
+ !parse_build_id(page_addr, build_id,
+ page_addr + phdr[i].p_offset,
+ phdr[i].p_filesz))
+ return 0;
+ }
+ return -EINVAL;
+}
+
+/* Parse build ID from 64-bit ELF */
+static int get_build_id_64(void *page_addr, unsigned char *build_id)
+{
+ Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr;
+ Elf64_Phdr *phdr;
+ int i;
+
+ /* only supports phdr that fits in one page */
+ if (ehdr->e_phnum >
+ (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr))
+ return -EINVAL;
+
+ phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr));
+
+ for (i = 0; i < ehdr->e_phnum; ++i) {
+ if (phdr[i].p_type == PT_NOTE &&
+ !parse_build_id(page_addr, build_id,
+ page_addr + phdr[i].p_offset,
+ phdr[i].p_filesz))
+ return 0;
+ }
+ return -EINVAL;
+}
+
+/* Parse build ID of ELF file mapped to vma */
+int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id)
+{
+ Elf32_Ehdr *ehdr;
+ struct page *page;
+ void *page_addr;
+ int ret;
+
+ /* only works for page backed storage */
+ if (!vma->vm_file)
+ return -EINVAL;
+
+ page = find_get_page(vma->vm_file->f_mapping, 0);
+ if (!page)
+ return -EFAULT; /* page not mapped */
+
+ ret = -EINVAL;
+ page_addr = kmap_atomic(page);
+ ehdr = (Elf32_Ehdr *)page_addr;
+
+ /* compare magic x7f "ELF" */
+ if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG) != 0)
+ goto out;
+
+ /* only support executable file and shared object file */
+ if (ehdr->e_type != ET_EXEC && ehdr->e_type != ET_DYN)
+ goto out;
+
+ if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
+ ret = get_build_id_32(page_addr, build_id);
+ else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64)
+ ret = get_build_id_64(page_addr, build_id);
+out:
+ kunmap_atomic(page_addr);
+ put_page(page);
+ return ret;
+}
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/4] bpf: Add size arg to build_id_parse function
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
2021-01-11 21:38 ` [PATCH 1/4] bpf: Move stack_map_get_build_id into lib Jiri Olsa
@ 2021-01-11 21:38 ` Jiri Olsa
2021-01-11 21:38 ` [PATCH 3/4] perf: Add build id data in mmap2 event Jiri Olsa
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jiri Olsa @ 2021-01-11 21:38 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Alexei Starovoitov, Song Liu, lkml, Peter Zijlstra, Ingo Molnar,
Mark Rutland, Namhyung Kim, Alexander Shishkin, Michael Petlan,
Ian Rogers, Stephane Eranian, Alexei Budankov, Andi Kleen,
Adrian Hunter
It's possible to have other build id types (other than default SHA1).
Currently there's also ld support for MD5 build id.
Adding size argument to build_id_parse function, that returns (if defined)
size of the parsed build id, so we can recognize the build id type.
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
include/linux/buildid.h | 3 ++-
kernel/bpf/stackmap.c | 2 +-
lib/buildid.c | 29 +++++++++++++++++++++--------
3 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/include/linux/buildid.h b/include/linux/buildid.h
index 08028a212589..40232f90db6e 100644
--- a/include/linux/buildid.h
+++ b/include/linux/buildid.h
@@ -6,6 +6,7 @@
#define BUILD_ID_SIZE_MAX 20
-int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id);
+int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id,
+ __u32 *size);
#endif
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index 55d254a59f07..cabaf7db8efc 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -189,7 +189,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
for (i = 0; i < trace_nr; i++) {
vma = find_vma(current->mm, ips[i]);
- if (!vma || build_id_parse(vma, id_offs[i].build_id)) {
+ if (!vma || build_id_parse(vma, id_offs[i].build_id, NULL)) {
/* per entry fall back to ips */
id_offs[i].status = BPF_STACK_BUILD_ID_IP;
id_offs[i].ip = ips[i];
diff --git a/lib/buildid.c b/lib/buildid.c
index 4a4f520c0e29..6156997c3895 100644
--- a/lib/buildid.c
+++ b/lib/buildid.c
@@ -12,6 +12,7 @@
*/
static inline int parse_build_id(void *page_addr,
unsigned char *build_id,
+ __u32 *size,
void *note_start,
Elf32_Word note_size)
{
@@ -38,6 +39,8 @@ static inline int parse_build_id(void *page_addr,
nhdr->n_descsz);
memset(build_id + nhdr->n_descsz, 0,
BUILD_ID_SIZE_MAX - nhdr->n_descsz);
+ if (size)
+ *size = nhdr->n_descsz;
return 0;
}
new_offs = note_offs + sizeof(Elf32_Nhdr) +
@@ -50,7 +53,8 @@ static inline int parse_build_id(void *page_addr,
}
/* Parse build ID from 32-bit ELF */
-static int get_build_id_32(void *page_addr, unsigned char *build_id)
+static int get_build_id_32(void *page_addr, unsigned char *build_id,
+ __u32 *size)
{
Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr;
Elf32_Phdr *phdr;
@@ -65,7 +69,7 @@ static int get_build_id_32(void *page_addr, unsigned char *build_id)
for (i = 0; i < ehdr->e_phnum; ++i) {
if (phdr[i].p_type == PT_NOTE &&
- !parse_build_id(page_addr, build_id,
+ !parse_build_id(page_addr, build_id, size,
page_addr + phdr[i].p_offset,
phdr[i].p_filesz))
return 0;
@@ -74,7 +78,8 @@ static int get_build_id_32(void *page_addr, unsigned char *build_id)
}
/* Parse build ID from 64-bit ELF */
-static int get_build_id_64(void *page_addr, unsigned char *build_id)
+static int get_build_id_64(void *page_addr, unsigned char *build_id,
+ __u32 *size)
{
Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr;
Elf64_Phdr *phdr;
@@ -89,7 +94,7 @@ static int get_build_id_64(void *page_addr, unsigned char *build_id)
for (i = 0; i < ehdr->e_phnum; ++i) {
if (phdr[i].p_type == PT_NOTE &&
- !parse_build_id(page_addr, build_id,
+ !parse_build_id(page_addr, build_id, size,
page_addr + phdr[i].p_offset,
phdr[i].p_filesz))
return 0;
@@ -97,8 +102,16 @@ static int get_build_id_64(void *page_addr, unsigned char *build_id)
return -EINVAL;
}
-/* Parse build ID of ELF file mapped to vma */
-int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id)
+/*
+ * Parse build ID of ELF file mapped to vma
+ * @vma: vma object
+ * @build_id: buffer to store build id, at least BUILD_ID_SIZE long
+ * @size: returns actual build id size in case of success
+ *
+ * Returns 0 on success, otherwise error (< 0).
+ */
+int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id,
+ __u32 *size)
{
Elf32_Ehdr *ehdr;
struct page *page;
@@ -126,9 +139,9 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id)
goto out;
if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
- ret = get_build_id_32(page_addr, build_id);
+ ret = get_build_id_32(page_addr, build_id, size);
else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64)
- ret = get_build_id_64(page_addr, build_id);
+ ret = get_build_id_64(page_addr, build_id, size);
out:
kunmap_atomic(page_addr);
put_page(page);
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/4] perf: Add build id data in mmap2 event
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
2021-01-11 21:38 ` [PATCH 1/4] bpf: Move stack_map_get_build_id into lib Jiri Olsa
2021-01-11 21:38 ` [PATCH 2/4] bpf: Add size arg to build_id_parse function Jiri Olsa
@ 2021-01-11 21:38 ` Jiri Olsa
2021-01-11 21:38 ` [PATCH 4/4] perf buildid-cache: Add support to add build ids from perf data Jiri Olsa
2021-01-12 2:49 ` [PATCHv6 0/4] perf: Add mmap2 build id support Alexei Starovoitov
4 siblings, 0 replies; 8+ messages in thread
From: Jiri Olsa @ 2021-01-11 21:38 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Peter Zijlstra, lkml, Peter Zijlstra, Ingo Molnar, Mark Rutland,
Namhyung Kim, Alexander Shishkin, Michael Petlan, Song Liu,
Ian Rogers, Stephane Eranian, Alexei Budankov, Andi Kleen,
Adrian Hunter
Adding support to carry build id data in mmap2 event.
The build id data replaces maj/min/ino/ino_generation
fields, which are also used to identify map's binary,
so it's ok to replace them with build id data:
union {
struct {
u32 maj;
u32 min;
u64 ino;
u64 ino_generation;
};
struct {
u8 build_id_size;
u8 __reserved_1;
u16 __reserved_2;
u8 build_id[20];
};
};
Replaced maj/min/ino/ino_generation fields give us size
of 24 bytes. We use 20 bytes for build id data, 1 byte
for size and rest is unused.
There's new misc bit for mmap2 to signal there's build
id data in it:
#define PERF_RECORD_MISC_MMAP_BUILD_ID (1 << 14)
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
include/uapi/linux/perf_event.h | 42 +++++++++++++++++++++++++++++----
kernel/events/core.c | 32 +++++++++++++++++++++----
2 files changed, 65 insertions(+), 9 deletions(-)
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index b15e3447cd9f..cb6f84103560 100644
--- a/include/uapi/linux/perf_event.h
+++ b/include/uapi/linux/perf_event.h
@@ -386,7 +386,8 @@ struct perf_event_attr {
aux_output : 1, /* generate AUX records instead of events */
cgroup : 1, /* include cgroup events */
text_poke : 1, /* include text poke events */
- __reserved_1 : 30;
+ build_id : 1, /* use build id in mmap2 events */
+ __reserved_1 : 29;
union {
__u32 wakeup_events; /* wakeup every n events */
@@ -659,6 +660,22 @@ struct perf_event_mmap_page {
__u64 aux_size;
};
+/*
+ * The current state of perf_event_header::misc bits usage:
+ * ('|' used bit, '-' unused bit)
+ *
+ * 012 CDEF
+ * |||---------||||
+ *
+ * Where:
+ * 0-2 CPUMODE_MASK
+ *
+ * C PROC_MAP_PARSE_TIMEOUT
+ * D MMAP_DATA / COMM_EXEC / FORK_EXEC / SWITCH_OUT
+ * E MMAP_BUILD_ID / EXACT_IP / SCHED_OUT_PREEMPT
+ * F (reserved)
+ */
+
#define PERF_RECORD_MISC_CPUMODE_MASK (7 << 0)
#define PERF_RECORD_MISC_CPUMODE_UNKNOWN (0 << 0)
#define PERF_RECORD_MISC_KERNEL (1 << 0)
@@ -690,6 +707,7 @@ struct perf_event_mmap_page {
*
* PERF_RECORD_MISC_EXACT_IP - PERF_RECORD_SAMPLE of precise events
* PERF_RECORD_MISC_SWITCH_OUT_PREEMPT - PERF_RECORD_SWITCH* events
+ * PERF_RECORD_MISC_MMAP_BUILD_ID - PERF_RECORD_MMAP2 event
*
*
* PERF_RECORD_MISC_EXACT_IP:
@@ -699,9 +717,13 @@ struct perf_event_mmap_page {
*
* PERF_RECORD_MISC_SWITCH_OUT_PREEMPT:
* Indicates that thread was preempted in TASK_RUNNING state.
+ *
+ * PERF_RECORD_MISC_MMAP_BUILD_ID:
+ * Indicates that mmap2 event carries build id data.
*/
#define PERF_RECORD_MISC_EXACT_IP (1 << 14)
#define PERF_RECORD_MISC_SWITCH_OUT_PREEMPT (1 << 14)
+#define PERF_RECORD_MISC_MMAP_BUILD_ID (1 << 14)
/*
* Reserve the last bit to indicate some extended misc field
*/
@@ -915,10 +937,20 @@ enum perf_event_type {
* u64 addr;
* u64 len;
* u64 pgoff;
- * u32 maj;
- * u32 min;
- * u64 ino;
- * u64 ino_generation;
+ * union {
+ * struct {
+ * u32 maj;
+ * u32 min;
+ * u64 ino;
+ * u64 ino_generation;
+ * };
+ * struct {
+ * u8 build_id_size;
+ * u8 __reserved_1;
+ * u16 __reserved_2;
+ * u8 build_id[20];
+ * };
+ * };
* u32 prot, flags;
* char filename[];
* struct sample_id sample_id;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 55d18791a72d..c37401e3e5f7 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -53,6 +53,7 @@
#include <linux/min_heap.h>
#include <linux/highmem.h>
#include <linux/pgtable.h>
+#include <linux/buildid.h>
#include "internal.h"
@@ -397,6 +398,7 @@ static atomic_t nr_ksymbol_events __read_mostly;
static atomic_t nr_bpf_events __read_mostly;
static atomic_t nr_cgroup_events __read_mostly;
static atomic_t nr_text_poke_events __read_mostly;
+static atomic_t nr_build_id_events __read_mostly;
static LIST_HEAD(pmus);
static DEFINE_MUTEX(pmus_lock);
@@ -4673,6 +4675,8 @@ static void unaccount_event(struct perf_event *event)
dec = true;
if (event->attr.mmap || event->attr.mmap_data)
atomic_dec(&nr_mmap_events);
+ if (event->attr.build_id)
+ atomic_dec(&nr_build_id_events);
if (event->attr.comm)
atomic_dec(&nr_comm_events);
if (event->attr.namespaces)
@@ -8046,6 +8050,8 @@ struct perf_mmap_event {
u64 ino;
u64 ino_generation;
u32 prot, flags;
+ u8 build_id[BUILD_ID_SIZE_MAX];
+ u32 build_id_size;
struct {
struct perf_event_header header;
@@ -8077,6 +8083,7 @@ static void perf_event_mmap_output(struct perf_event *event,
struct perf_sample_data sample;
int size = mmap_event->event_id.header.size;
u32 type = mmap_event->event_id.header.type;
+ bool use_build_id;
int ret;
if (!perf_event_mmap_match(event, data))
@@ -8101,13 +8108,25 @@ static void perf_event_mmap_output(struct perf_event *event,
mmap_event->event_id.pid = perf_event_pid(event, current);
mmap_event->event_id.tid = perf_event_tid(event, current);
+ use_build_id = event->attr.build_id && mmap_event->build_id_size;
+
+ if (event->attr.mmap2 && use_build_id)
+ mmap_event->event_id.header.misc |= PERF_RECORD_MISC_MMAP_BUILD_ID;
+
perf_output_put(&handle, mmap_event->event_id);
if (event->attr.mmap2) {
- perf_output_put(&handle, mmap_event->maj);
- perf_output_put(&handle, mmap_event->min);
- perf_output_put(&handle, mmap_event->ino);
- perf_output_put(&handle, mmap_event->ino_generation);
+ if (use_build_id) {
+ u8 size[4] = { (u8) mmap_event->build_id_size, 0, 0, 0 };
+
+ __output_copy(&handle, size, 4);
+ __output_copy(&handle, mmap_event->build_id, BUILD_ID_SIZE_MAX);
+ } else {
+ perf_output_put(&handle, mmap_event->maj);
+ perf_output_put(&handle, mmap_event->min);
+ perf_output_put(&handle, mmap_event->ino);
+ perf_output_put(&handle, mmap_event->ino_generation);
+ }
perf_output_put(&handle, mmap_event->prot);
perf_output_put(&handle, mmap_event->flags);
}
@@ -8236,6 +8255,9 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)
mmap_event->event_id.header.size = sizeof(mmap_event->event_id) + size;
+ if (atomic_read(&nr_build_id_events))
+ build_id_parse(vma, mmap_event->build_id, &mmap_event->build_id_size);
+
perf_iterate_sb(perf_event_mmap_output,
mmap_event,
NULL);
@@ -11172,6 +11194,8 @@ static void account_event(struct perf_event *event)
inc = true;
if (event->attr.mmap || event->attr.mmap_data)
atomic_inc(&nr_mmap_events);
+ if (event->attr.build_id)
+ atomic_inc(&nr_build_id_events);
if (event->attr.comm)
atomic_inc(&nr_comm_events);
if (event->attr.namespaces)
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 4/4] perf buildid-cache: Add support to add build ids from perf data
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
` (2 preceding siblings ...)
2021-01-11 21:38 ` [PATCH 3/4] perf: Add build id data in mmap2 event Jiri Olsa
@ 2021-01-11 21:38 ` Jiri Olsa
2021-01-12 2:49 ` [PATCHv6 0/4] perf: Add mmap2 build id support Alexei Starovoitov
4 siblings, 0 replies; 8+ messages in thread
From: Jiri Olsa @ 2021-01-11 21:38 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: lkml, Peter Zijlstra, Ingo Molnar, Mark Rutland, Namhyung Kim,
Alexander Shishkin, Michael Petlan, Song Liu, Ian Rogers,
Stephane Eranian, Alexei Budankov, Andi Kleen, Adrian Hunter
Adding support to specify perf data file as -a option file
argument.
Examples below assume debuginfod daemon is running on
192.168.122.174, like:
# debuginfod -F /
If the file is detected to be perf data file, it is processed
and all dso objects with sample hit are stored to the build
id cache.
$ DEBUGINFOD_URLS=http://192.168.122.174:8002 perf buildid-cache -a perf.data
OK 5dcec522abf136fcfd3128f47e131f2365834dd7 /home/jolsa/.debug/.build-id/5d/cec522abf136fcfd3128f47e131f2365834dd7/elf
OK 5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko
By default we store only dso with hits, but it's possible to
specify 'all' to store all dso objects, like:
-a perf.data,all
$ DEBUGINFOD_URLS=http://192.168.122.174:8002 perf buildid-cache -a perf.data,all
OK 5dcec522abf136fcfd3128f47e131f2365834dd7 /home/jolsa/.debug/.build-id/5d/cec522abf136fcfd3128f47e131f2365834dd7/elf
OK 6ce92dc7c31f12fe5b7775a2bb8b14a3546ce2cd /lib/modules/5.10.0-rc2speed+/kernel/drivers/firmware/qemu_fw_cfg.ko
OK bf3f6d32dccc159f841fc3658c241d0e74c61fbb /lib/modules/5.10.0-rc2speed+/kernel/drivers/block/virtio_blk.ko
OK e896b4329cf9f190f1a0fae933f425ff8f71b052 /lib/modules/5.10.0-rc2speed+/kernel/drivers/char/virtio_console.ko
OK 5bedc933cb59e053ecb472f327bd73c548364479 /lib/modules/5.10.0-rc2speed+/kernel/drivers/input/serio/serio_raw.ko
OK cecc506368a8b7a473a5f900d26f0d3d914a9c23 /lib/modules/5.10.0-rc2speed+/kernel/arch/x86/crypto/crc32c-intel.ko
OK 91076fb3646d061a0a42cf7bddb339a665ee4f80 /lib/modules/5.10.0-rc2speed+/kernel/arch/x86/crypto/ghash-clmulni-intel.ko
OK 4e2a304d788bb8e2e950bc82a5944e042afa0bf2 /lib/modules/5.10.0-rc2speed+/kernel/drivers/media/cec/core/cec.ko
OK 31ab0da5ad81e6803280177f507a95f3053d585e /lib/modules/5.10.0-rc2speed+/kernel/lib/libcrc32c.ko
OK f6154bca47c149f48c942fcc3d653041dd285c65 /lib/modules/5.10.0-rc2speed+/kernel/drivers/gpu/drm/ttm/ttm.ko
OK 723f5852de81590d54b23b38c160d3618b41951b /lib/modules/5.10.0-rc2speed+/kernel/arch/x86/crypto/crct10dif-pclmul.ko
OK 06b1eab7f141cbc3e5a5db47909c8ab5cb242e40 /lib/modules/5.10.0-rc2speed+/kernel/drivers/gpu/drm/drm_ttm_helper.ko
OK 38292b862cf3ff87489508fdb4895efa45780813 /lib/modules/5.10.0-rc2speed+/kernel/drivers/gpu/drm/qxl/qxl.ko
OK cdf51e58609bf2ce4837a7b195e0ccae0a930907 /lib/modules/5.10.0-rc2speed+/kernel/arch/x86/crypto/crc32-pclmul.ko
OK 5ca8958388f6688452ecc2cb83d6031394c659ad /lib/modules/5.10.0-rc2speed+/kernel/drivers/gpu/drm/drm.ko
OK 236bc4e4f38bf3559007566cb32b3dcc1bc28d2d /lib/modules/5.10.0-rc2speed+/kernel/drivers/gpu/drm/drm_kms_helper.ko
OK 5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko
OK 66db2be3efaa43bb5a5c481986e9554e1885cc69 /usr/lib/systemd/systemd
OK 7db607d9f2de89860d9639712da64c8bacd31e4b /usr/lib64/libm-2.30.so
OK 55b5f9652e1d17c1dd58f62628d5063428e5db91 /usr/lib64/libudev.so.1.6.15
OK 63b97070bf097130713bb6c89cf7100b5f3c9b17 /usr/lib64/libunistring.so.2.1.0
...
Once perf data is specified, no other file can be specified in
the option, otherwise it causes syntax error.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
.../perf/Documentation/perf-buildid-cache.txt | 12 +-
tools/perf/builtin-buildid-cache.c | 213 +++++++++++++++++-
tools/perf/util/probe-event.c | 6 +-
3 files changed, 225 insertions(+), 6 deletions(-)
diff --git a/tools/perf/Documentation/perf-buildid-cache.txt b/tools/perf/Documentation/perf-buildid-cache.txt
index bb167e32a1d7..b9987d1399ca 100644
--- a/tools/perf/Documentation/perf-buildid-cache.txt
+++ b/tools/perf/Documentation/perf-buildid-cache.txt
@@ -23,7 +23,17 @@ OPTIONS
-------
-a::
--add=::
- Add specified file to the cache.
+ Add specified file or perf.data binaries to the cache.
+
+ If the file is detected to be perf data file, it is processed
+ and all dso objects with sample hit are stored to the cache.
+
+ It's possible to specify 'all' to store all dso objects, like:
+ -a perf.data,all
+
+ Once perf data is specified, no other file can be specified in
+ the option, otherwise it causes syntax error.
+
-f::
--force::
Don't complain, do it.
diff --git a/tools/perf/builtin-buildid-cache.c b/tools/perf/builtin-buildid-cache.c
index ecd0d3cb6f5c..be6826284d26 100644
--- a/tools/perf/builtin-buildid-cache.c
+++ b/tools/perf/builtin-buildid-cache.c
@@ -30,6 +30,11 @@
#include "util/config.h"
#include <linux/string.h>
#include <linux/err.h>
+#include <linux/zalloc.h>
+#include <sys/stat.h>
+#ifdef HAVE_DEBUGINFOD_SUPPORT
+#include <elfutils/debuginfod.h>
+#endif
static int build_id_cache__kcore_buildid(const char *proc_dir, char *sbuildid)
{
@@ -359,6 +364,203 @@ static int perf_buildid_cache_config(const char *var, const char *value, void *c
return 0;
}
+#ifdef HAVE_DEBUGINFOD_SUPPORT
+static int call_debuginfod(const char *sbuild_id, char **path, bool debuginfo)
+{
+ debuginfod_client *c;
+ int fd;
+
+ c = debuginfod_begin();
+ if (c == NULL)
+ return -1;
+
+ pr_debug("trying debuginfod for executable <%s> ... ", sbuild_id);
+
+ if (debuginfo) {
+ fd = debuginfod_find_debuginfo(c, (const unsigned char *) sbuild_id,
+ 0, path);
+ } else {
+ fd = debuginfod_find_executable(c, (const unsigned char *) sbuild_id,
+ 0, path);
+ }
+ if (fd >= 0)
+ close(fd); /* retaining reference by realname */
+
+ debuginfod_end(c);
+ pr_debug("%s%s\n", *path ? "OK " : "FAILED", *path ? *path : "");
+ return *path ? 0 : -1;
+}
+#else
+static int call_debuginfod(const char *sbuild_id __maybe_unused,
+ char **path __maybe_unused,
+ bool debuginfo __maybe_unused)
+{
+ return -1;
+}
+#endif
+
+struct dso_store_data {
+ bool hits;
+};
+
+static int dso_store(struct dso *dso, struct machine *machine, void *priv)
+{
+ struct dso_store_data *data = priv;
+ char sbuild_id[SBUILD_ID_SIZE];
+ struct build_id bid;
+ char *path = NULL, *link = NULL;
+ bool is_kallsyms;
+ int err = -1;
+
+ /*
+ * There's no build id in dso, nothing to do..
+ */
+ if (!dso->has_build_id || !build_id__is_defined(&dso->bid))
+ return 0;
+
+ if (data->hits && !dso->hit)
+ return 0;
+
+ /*
+ * The storing process is:
+ * - get build id of the dso
+ * - check if it is already in cache
+ * - check if it matches provided build id from mmap2 event
+ * - if not, try debuginfod to download the binary
+ * - store binary to build id database
+ */
+ is_kallsyms = !strcmp(machine->mmap_name, dso->short_name);
+ build_id__sprintf(&dso->bid, sbuild_id);
+
+ link = build_id_cache__linkname(sbuild_id, NULL, 0);
+ if (!link)
+ return -ENOMEM;
+
+ if (!access(link, X_OK)) {
+ pr_debug("already in cache - %s <%s>\n", dso->long_name, sbuild_id);
+ err = 0;
+ goto out;
+ }
+
+ path = strdup(dso->long_name);
+ if (!path)
+ goto out;
+
+ if (is_kallsyms) {
+ /*
+ * Find out if we are on the same kernel as perf.data
+ * and store kallsyms in that case.
+ */
+ err = sysfs__read_build_id("/sys/kernel/notes", &bid);
+ if (err < 0)
+ goto out;
+ } else {
+ struct nscookie nsc;
+ struct stat st;
+
+ nsinfo__mountns_enter(dso->nsinfo, &nsc);
+
+ /*
+ * Does the file exists in the first place, if it does,
+ * resolve path and read the build id.
+ */
+ if (stat(dso->long_name, &st)) {
+ nsinfo__mountns_exit(&nsc);
+ zfree(&path);
+ goto try_download;
+ }
+
+ err = filename__read_build_id(dso->long_name, &bid);
+ nsinfo__mountns_exit(&nsc);
+
+ if (err <= 0)
+ goto out;
+ }
+
+ /*
+ * If we match, then what we want in mmap2 event
+ * is what we got in the binary,
+ */
+ if (bid.size != dso->bid.size || memcmp(&bid, &dso->bid, bid.size)) {
+ char sbid[SBUILD_ID_SIZE];
+
+ build_id__sprintf(&bid, sbid);
+ pr_debug("mmap build id <%s> does not match for %s <%s>\n",
+ sbuild_id, path, sbid);
+ zfree(&path);
+ }
+
+try_download:
+ /*
+ * We did not match build id or did not find the
+ * binary - try debuginfod as last resort.
+ */
+ if (!path) {
+ bool debuginfo;
+ char *tmp = NULL;
+
+ /*
+ * The debuginfo retrieval for standard binaries
+ * is handled within build_id_cache__add function.
+ *
+ * For kernel and kernel modules we have to ask
+ * for debuginfo directly, because debuginfod
+ * does not treat them as binaries.
+ */
+ debuginfo = is_kallsyms ||
+ is_kernel_module(dso->long_name, PERF_RECORD_MISC_CPUMODE_UNKNOWN);
+
+ if (call_debuginfod(sbuild_id, &tmp, debuginfo)) {
+ err = -1;
+ goto out;
+ }
+
+ path = tmp;
+
+ /*
+ * The kernel dso is now elf binary, so disable is_kallsyms
+ * so build_id_cache__add can prepare proper file names.
+ */
+ is_kallsyms = false;
+ }
+
+ pr_debug("linking %s %s <%s>\n", dso->short_name, path, sbuild_id);
+
+ err = build_id_cache__add(sbuild_id, path, path,
+ dso->nsinfo, is_kallsyms, false);
+out:
+ free(path);
+ fprintf(stderr, "%s %s %s\n", err ? "FAIL" : "OK ", sbuild_id, dso->long_name);
+ return 0;
+}
+
+static int
+build_id_cache__add_perf_data(const char *path, bool all)
+{
+ struct perf_session *session;
+ struct dso_store_data priv = {
+ .hits = !all,
+ };
+ struct perf_data data = {
+ .path = path,
+ .mode = PERF_DATA_MODE_READ,
+ };
+ int err;
+
+ session = perf_session__new(&data, false, &build_id__mark_dso_hit_ops);
+ if (IS_ERR(session))
+ return PTR_ERR(session);
+
+ err = perf_session__process_events(session);
+ if (err)
+ goto out;
+
+ err = __perf_session__cache_build_ids(session, dso_store, &priv);
+out:
+ perf_session__delete(session);
+ return err;
+}
+
int cmd_buildid_cache(int argc, const char **argv)
{
struct strlist *list;
@@ -462,7 +664,15 @@ int cmd_buildid_cache(int argc, const char **argv)
list = strlist__new(add_name_list_str, NULL);
if (list) {
strlist__for_each_entry(pos, list)
- if (build_id_cache__add_file(pos->s, nsi)) {
+ if (is_perf_data(pos->s)) {
+ struct str_node *all_pos = strlist__next(pos);
+ bool all = !strcmp("all", all_pos ? all_pos->s : "");
+
+ if (build_id_cache__add_perf_data(pos->s, all))
+ pr_warning("Couldn't add build ids from %s\n", pos->s);
+ if (all)
+ pos = all_pos;
+ } else if (build_id_cache__add_file(pos->s, nsi)) {
if (errno == EEXIST) {
pr_debug("%s already in the cache\n",
pos->s);
@@ -471,7 +681,6 @@ int cmd_buildid_cache(int argc, const char **argv)
pr_warning("Couldn't add %s: %s\n",
pos->s, str_error_r(errno, sbuf, sizeof(sbuf)));
}
-
strlist__delete(list);
}
}
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 8eae2afff71a..e821bb977c9b 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -1616,9 +1616,9 @@ static int parse_perf_probe_point(char *arg, struct perf_probe_event *pev)
return -EINVAL;
}
- pr_debug("symbol:%s file:%s line:%d offset:%lu return:%d lazy:%s\n",
- pp->function, pp->file, pp->line, pp->offset, pp->retprobe,
- pp->lazy_line);
+ pr_debug2("symbol:%s file:%s line:%d offset:%lu return:%d lazy:%s\n",
+ pp->function, pp->file, pp->line, pp->offset, pp->retprobe,
+ pp->lazy_line);
return 0;
}
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCHv6 0/4] perf: Add mmap2 build id support
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
` (3 preceding siblings ...)
2021-01-11 21:38 ` [PATCH 4/4] perf buildid-cache: Add support to add build ids from perf data Jiri Olsa
@ 2021-01-12 2:49 ` Alexei Starovoitov
2021-01-12 9:19 ` Jiri Olsa
4 siblings, 1 reply; 8+ messages in thread
From: Alexei Starovoitov @ 2021-01-12 2:49 UTC (permalink / raw)
To: Jiri Olsa
Cc: Arnaldo Carvalho de Melo, lkml, Peter Zijlstra, Ingo Molnar,
Mark Rutland, Namhyung Kim, Alexander Shishkin, Michael Petlan,
Song Liu, Ian Rogers, Stephane Eranian, Alexei Budankov,
Andi Kleen, Adrian Hunter
On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote:
> hi,
> adding the support to have buildid stored in mmap2 event,
> so we can bypass the final perf record hunt on build ids.
>
> This patchset allows perf to record build ID in mmap2 event,
> and adds perf tooling to store/download binaries to .debug
> cache based on these build IDs.
>
> Note that the build id retrieval code is stolen from bpf
> code, where it's been used (together with file offsets)
> to replace IPs in user space stack traces. It's now added
> under lib directory.
>
> v6 changes:
> - last 4 patches rebased Arnaldo's perf/core
There were no issues with v5 as far as I can remember.
This is just a resubmit to get it landed ?
Last time we couldn't quite figure out which tree to go through.
I think the recommend path was to go via bpf-next.
Is it still the case?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCHv6 0/4] perf: Add mmap2 build id support
2021-01-12 2:49 ` [PATCHv6 0/4] perf: Add mmap2 build id support Alexei Starovoitov
@ 2021-01-12 9:19 ` Jiri Olsa
2021-01-14 3:26 ` Alexei Starovoitov
0 siblings, 1 reply; 8+ messages in thread
From: Jiri Olsa @ 2021-01-12 9:19 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Jiri Olsa, Arnaldo Carvalho de Melo, lkml, Peter Zijlstra,
Ingo Molnar, Mark Rutland, Namhyung Kim, Alexander Shishkin,
Michael Petlan, Song Liu, Ian Rogers, Stephane Eranian,
Alexei Budankov, Andi Kleen, Adrian Hunter
On Mon, Jan 11, 2021 at 06:49:58PM -0800, Alexei Starovoitov wrote:
> On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote:
> > hi,
> > adding the support to have buildid stored in mmap2 event,
> > so we can bypass the final perf record hunt on build ids.
> >
> > This patchset allows perf to record build ID in mmap2 event,
> > and adds perf tooling to store/download binaries to .debug
> > cache based on these build IDs.
> >
> > Note that the build id retrieval code is stolen from bpf
> > code, where it's been used (together with file offsets)
> > to replace IPs in user space stack traces. It's now added
> > under lib directory.
> >
> > v6 changes:
> > - last 4 patches rebased Arnaldo's perf/core
>
> There were no issues with v5 as far as I can remember.
> This is just a resubmit to get it landed ?
yes, exactly
> Last time we couldn't quite figure out which tree to go through.
> I think the recommend path was to go via bpf-next.
> Is it still the case?
bpf-next would be best for kernel changes,
perf: Add build id data in mmap2 event
bpf: Add size arg to build_id_parse function
bpf: Move stack_map_get_build_id into lib
the 'perf buildid-cache' change needs to go through Arnaldo's tree,
because it depends on changes he already pulled in
thanks,
jirka
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCHv6 0/4] perf: Add mmap2 build id support
2021-01-12 9:19 ` Jiri Olsa
@ 2021-01-14 3:26 ` Alexei Starovoitov
0 siblings, 0 replies; 8+ messages in thread
From: Alexei Starovoitov @ 2021-01-14 3:26 UTC (permalink / raw)
To: Jiri Olsa
Cc: Jiri Olsa, Arnaldo Carvalho de Melo, lkml, Peter Zijlstra,
Ingo Molnar, Mark Rutland, Namhyung Kim, Alexander Shishkin,
Michael Petlan, Song Liu, Ian Rogers, Stephane Eranian,
Alexei Budankov, Andi Kleen, Adrian Hunter
On Tue, Jan 12, 2021 at 1:19 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Mon, Jan 11, 2021 at 06:49:58PM -0800, Alexei Starovoitov wrote:
> > On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote:
> > > hi,
> > > adding the support to have buildid stored in mmap2 event,
> > > so we can bypass the final perf record hunt on build ids.
> > >
> > > This patchset allows perf to record build ID in mmap2 event,
> > > and adds perf tooling to store/download binaries to .debug
> > > cache based on these build IDs.
> > >
> > > Note that the build id retrieval code is stolen from bpf
> > > code, where it's been used (together with file offsets)
> > > to replace IPs in user space stack traces. It's now added
> > > under lib directory.
> > >
> > > v6 changes:
> > > - last 4 patches rebased Arnaldo's perf/core
> >
> > There were no issues with v5 as far as I can remember.
> > This is just a resubmit to get it landed ?
>
> yes, exactly
>
> > Last time we couldn't quite figure out which tree to go through.
> > I think the recommend path was to go via bpf-next.
> > Is it still the case?
>
> bpf-next would be best for kernel changes,
> perf: Add build id data in mmap2 event
> bpf: Add size arg to build_id_parse function
> bpf: Move stack_map_get_build_id into lib
Then please cc them to bpf@vger and add [PATCH bpf-next]
otherwise it's all very confusing.
> the 'perf buildid-cache' change needs to go through Arnaldo's tree,
> because it depends on changes he already pulled in
Also don't include the 4th patch in the series if it isn't meant for bpf-next.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-01-14 3:27 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-11 21:38 [PATCHv6 0/4] perf: Add mmap2 build id support Jiri Olsa
2021-01-11 21:38 ` [PATCH 1/4] bpf: Move stack_map_get_build_id into lib Jiri Olsa
2021-01-11 21:38 ` [PATCH 2/4] bpf: Add size arg to build_id_parse function Jiri Olsa
2021-01-11 21:38 ` [PATCH 3/4] perf: Add build id data in mmap2 event Jiri Olsa
2021-01-11 21:38 ` [PATCH 4/4] perf buildid-cache: Add support to add build ids from perf data Jiri Olsa
2021-01-12 2:49 ` [PATCHv6 0/4] perf: Add mmap2 build id support Alexei Starovoitov
2021-01-12 9:19 ` Jiri Olsa
2021-01-14 3:26 ` Alexei Starovoitov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.