* [PATCH v2 0/6] Generate address range data for built-in modules
@ 2024-05-11 22:40 Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 1/6] kbuild: add modules.builtin.objs Kris Van Hees
` (6 more replies)
0 siblings, 7 replies; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
Especially for tracing applications, it is convenient to be able to
refer to a symbol using a <module name, symbol name> pair and to be able
to translate an address into a <nodule mname, symbol name> pair. But
that does not work if the module is built into the kernel because the
object files that comprise the built-in module implementation are simply
linked into the kernel image along with all other kernel object files.
This is especially visible when providing tracing scripts for support
purposes, where the developer of the script targets a particular kernel
version, but does not have control over whether the target system has
a particular module as loadable module or built-in module. When tracing
symbols within a module, referring them by <module name, symbol name>
pairs is both convenient and aids symbol lookup. But that naming will
not work if the module name information is lost if the module is built
into the kernel on the target system.
Earlier work addressing this loss of information for built-in modules
involved adding module name information to the kallsyms data, but that
required more invasive code in the kernel proper. This work never did
get merged into the kernel tree.
All that is really needed is knowing whether a given address belongs to
a particular module (or multiple modules if they share an object file).
Or in other words, whether that address falls within an address range
that is associated with one or more modules.
This patch series is baaed on Luis Chamberlain's patch to generate
modules.builtin.objs, associating built-in modules with their object
files. Using this data, vmlinux.o.map and vmlinux.map can be parsed in
a single pass to generate a modules.buitin.ranges file with offset range
information (relative to the base address of the associated section) for
built-in modules. The file gets installed along with the other
modules.builtin.* files.
The impact on the kernel build is minimal because everything is done
using a single-pass AWK script. The generated data size is minimal as
well, (depending on the exact kernel configuration) usually in the range
of 500-700 lines, with a file size of 20-40KB.
Changes since v1:
- Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
- Moved the config option to the tracers section
- 2nd arg to generate_builtin_ranges.awk should be vmlinux.map
Kris Van Hees (5):
trace: add CONFIG_BUILTIN_MODULE_RANGES option
kbuild: generate a linker map for vmlinux.o
module: script to generate offset ranges for builtin modules
kbuild: generate modules.builtin.ranges when linking the kernel
module: add install target for modules.builtin.ranges
Luis Chamberlain (1):
kbuild: add modules.builtin.objs
.gitignore | 2 +-
Documentation/dontdiff | 2 +-
Documentation/kbuild/kbuild.rst | 5 ++
Makefile | 8 +-
include/linux/module.h | 4 +-
kernel/trace/Kconfig | 17 ++++
scripts/Makefile.lib | 5 +-
scripts/Makefile.modinst | 11 ++-
scripts/Makefile.vmlinux | 17 ++++
scripts/Makefile.vmlinux_o | 18 ++++-
scripts/generate_builtin_ranges.awk | 149 ++++++++++++++++++++++++++++++++++++
11 files changed, 228 insertions(+), 10 deletions(-)
create mode 100755 scripts/generate_builtin_ranges.awk
base-commit: dd5a440a31fae6e459c0d6271dddd62825505361
--
2.42.0
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 1/6] kbuild: add modules.builtin.objs
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option Kris Van Hees
` (5 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Luis Chamberlain, Nick Alcock, Kris Van Hees, Steven Rostedt,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
From: Luis Chamberlain <mcgrof@kernel.org>
The file modules.builtin names all modules that are built into the
kernel; this is checked by modprobe to not fail when trying to load
something built-in. But for tools which want to see which object files
make up each module, we want to help them with such a mapping as it is
not easy to get this otherwise.
We do this by just extending scripts/Makefile.lib with a new variable
and define to capture all object files included in this module, store it
in a new objs= modinfo stanza, then extract it just before linking into
a new file modules.builtin.objs with a layout roughly modelled on a
makefile:
path/to/module.o: path/to/constituent.o path/to/other-constituent.o
Single-file built-in modules get a line reading
path/to/module.o:
Note that the .modinfo section is discarded at the link stage, so the
kernel is not bloated at all (see include/asm-generic/vmlinux.lds.h).
Orabug: 29891866
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>
---
Changes since v1:
- None
---
.gitignore | 2 +-
Documentation/dontdiff | 2 +-
Documentation/kbuild/kbuild.rst | 5 +++++
Makefile | 8 ++++++--
include/linux/module.h | 4 +++-
scripts/Makefile.lib | 5 ++++-
scripts/Makefile.modinst | 6 +++---
scripts/Makefile.vmlinux_o | 15 ++++++++++++++-
8 files changed, 37 insertions(+), 10 deletions(-)
diff --git a/.gitignore b/.gitignore
index c59dc60ba62ef..62ede8565a2aa 100644
--- a/.gitignore
+++ b/.gitignore
@@ -69,7 +69,7 @@ modules.order
/System.map
/Module.markers
/modules.builtin
-/modules.builtin.modinfo
+/modules.builtin.*
/modules.nsdeps
#
diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index 3c399f132e2db..75b9655e57914 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -179,7 +179,7 @@ mkutf8data
modpost
modules-only.symvers
modules.builtin
-modules.builtin.modinfo
+modules.builtin.*
modules.nsdeps
modules.order
modversions.h*
diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.rst
index 9c8d1d046ea56..79e104ffee715 100644
--- a/Documentation/kbuild/kbuild.rst
+++ b/Documentation/kbuild/kbuild.rst
@@ -17,6 +17,11 @@ modules.builtin
This file lists all modules that are built into the kernel. This is used
by modprobe to not fail when trying to load something builtin.
+modules.builtin.objs
+-----------------------
+This file contains object mapping of modules that are built into the kernel
+to their corresponding object files used to build the module.
+
modules.builtin.modinfo
-----------------------
This file contains modinfo from all modules that are built into the kernel.
diff --git a/Makefile b/Makefile
index d51d411d44a82..cc979f9874f5a 100644
--- a/Makefile
+++ b/Makefile
@@ -1140,7 +1140,11 @@ PHONY += vmlinux_o
vmlinux_o: vmlinux.a $(KBUILD_VMLINUX_LIBS)
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux_o
-vmlinux.o modules.builtin.modinfo modules.builtin: vmlinux_o
+MODULES_BUILTIN := modules.builtin.modinfo
+MODULES_BUILTIN += modules.builtin
+MODULES_BUILTIN += modules.builtin.objs
+
+vmlinux.o $(MODULES_BUILTIN): vmlinux_o
@:
PHONY += vmlinux
@@ -1476,7 +1480,7 @@ endif # CONFIG_MODULES
# Directories & files removed with 'make clean'
CLEAN_FILES += vmlinux.symvers modules-only.symvers \
- modules.builtin modules.builtin.modinfo modules.nsdeps \
+ modules.builtin modules.builtin.* modules.nsdeps \
compile_commands.json .thinlto-cache rust/test \
rust-project.json .vmlinux.objs .vmlinux.export.c
diff --git a/include/linux/module.h b/include/linux/module.h
index 1153b0d99a808..cbfff06e00cd6 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -180,7 +180,9 @@ extern void cleanup_module(void);
#ifdef MODULE
#define MODULE_FILE
#else
-#define MODULE_FILE MODULE_INFO(file, KBUILD_MODFILE);
+#define MODULE_FILE \
+ MODULE_INFO(file, KBUILD_MODFILE); \
+ MODULE_INFO(objs, KBUILD_MODOBJS);
#endif
/*
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 3179747cbd2cc..3b3baa78d4fbd 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -118,6 +118,8 @@ modname-multi = $(sort $(foreach m,$(multi-obj-ym),\
__modname = $(or $(modname-multi),$(basetarget))
modname = $(subst $(space),:,$(__modname))
+modname-objs = $($(modname)-objs) $($(modname)-y) $($(modname)-Y)
+modname-objs-prefixed = $(sort $(strip $(addprefix $(obj)/, $(modname-objs))))
modfile = $(addprefix $(obj)/,$(__modname))
# target with $(obj)/ and its suffix stripped
@@ -131,7 +133,8 @@ name-fix = $(call stringify,$(call name-fix-token,$1))
basename_flags = -DKBUILD_BASENAME=$(call name-fix,$(basetarget))
modname_flags = -DKBUILD_MODNAME=$(call name-fix,$(modname)) \
-D__KBUILD_MODNAME=kmod_$(call name-fix-token,$(modname))
-modfile_flags = -DKBUILD_MODFILE=$(call stringify,$(modfile))
+modfile_flags = -DKBUILD_MODFILE=$(call stringify,$(modfile)) \
+ -DKBUILD_MODOBJS=$(call stringify,$(modfile).o:$(subst $(space),|,$(modname-objs-prefixed)))
_c_flags = $(filter-out $(CFLAGS_REMOVE_$(target-stem).o), \
$(filter-out $(ccflags-remove-y), \
diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index 0afd75472679f..b45586aa1de49 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -30,10 +30,10 @@ $(MODLIB)/modules.order: modules.order FORCE
quiet_cmd_install_modorder = INSTALL $@
cmd_install_modorder = sed 's:^\(.*\)\.o$$:kernel/\1.ko:' $< > $@
-# Install modules.builtin(.modinfo) even when CONFIG_MODULES is disabled.
-install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo)
+# Install modules.builtin(.modinfo,.objs) even when CONFIG_MODULES is disabled.
+install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin.objs)
-$(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo): $(MODLIB)/%: % FORCE
+$(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin.objs): $(MODLIB)/%: % FORCE
$(call cmd,install)
endif
diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
index 6de297916ce68..508b3294e2cf1 100644
--- a/scripts/Makefile.vmlinux_o
+++ b/scripts/Makefile.vmlinux_o
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
PHONY := __default
-__default: vmlinux.o modules.builtin.modinfo modules.builtin
+__default: vmlinux.o modules.builtin.modinfo modules.builtin modules.builtin.objs
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
@@ -87,6 +87,19 @@ targets += modules.builtin
modules.builtin: modules.builtin.modinfo FORCE
$(call if_changed,modules_builtin)
+# module.builtin.objs
+# ---------------------------------------------------------------------------
+quiet_cmd_modules_builtin_objs = GEN $@
+ cmd_modules_builtin_objs = \
+ tr '\0' '\n' < $< | \
+ sed -n 's/^[[:alnum:]:_]*\.objs=//p' | \
+ tr ' ' '\n' | uniq | sed -e 's|:|: |' -e 's:|: :g' | \
+ tr -s ' ' > $@
+
+targets += modules.builtin.objs
+modules.builtin.objs: modules.builtin.modinfo FORCE
+ $(call if_changed,modules_builtin_objs)
+
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
# ---------------------------------------------------------------------------
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 1/6] kbuild: add modules.builtin.objs Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-14 23:20 ` kernel test robot
2024-05-11 22:40 ` [PATCH v2 3/6] kbuild: generate a linker map for vmlinux.o Kris Van Hees
` (4 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Nick Alcock, Alan Maguire, Steven Rostedt,
Luis Chamberlain, Masami Hiramatsu, Nick Desaulniers, Jiri Olsa,
Elena Zannoni
The CONFIG_BUILTIN_MODULE_RANGES option controls whether offset range data
is generated for kernel modules that are built into the kernel image.
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
---
Changes since v1:
- Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
- Moved the config option to the tracers section
---
kernel/trace/Kconfig | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 47345bf1d4a9f..839a56e971cc0 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -188,6 +188,23 @@ menuconfig FTRACE
if FTRACE
+config BUILTIN_MODULE_RANGES
+ bool "Generate address range information for builtin modules"
+ select VMLINUX_MAP
+ help
+ When modules are built into the kernel, there will be no module name
+ associated with its symbols in /proc/kallsyms. Tracers may want to
+ identify symbols by module name and symbol name regardless of whether
+ the module is configured as loadable or not.
+
+ This option generates modules.builtin.ranges in the build tree with
+ offset ranges (per ELF section) for the module(s) they belong to.
+ It also records an anchor symbol to determine the load address of the
+ section.
+
+ It is fully compatible with CONFIG_RANDOMIZE_BASE and similar late-
+ address-modification options.
+
config BOOTTIME_TRACING
bool "Boot-time Tracing support"
depends on TRACING
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 3/6] kbuild: generate a linker map for vmlinux.o
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 1/6] kbuild: add modules.builtin.objs Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 4/6] module: script to generate offset ranges for builtin modules Kris Van Hees
` (3 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Nick Alcock, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
When CONFIG_BUILTIN_MODULE_RANGES is set, a linker map for vmlinux.o needs
to be generated. The generation of offset range data for builtin modules
depends on that linker map to know what offsets in an ELF section belong
to an object file for a particular builtin module.
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
---
Changes since v1:
- Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
---
scripts/Makefile.vmlinux_o | 3 +++
1 file changed, 3 insertions(+)
diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o
index 508b3294e2cf1..e4a92838780d5 100644
--- a/scripts/Makefile.vmlinux_o
+++ b/scripts/Makefile.vmlinux_o
@@ -45,9 +45,12 @@ objtool-args = $(vmlinux-objtool-args-y) --link
# Link of vmlinux.o used for section mismatch analysis
# ---------------------------------------------------------------------------
+vmlinux-o-ld-args-$(CONFIG_BUILTIN_MODULE_RANGES) += -Map=$@.map
+
quiet_cmd_ld_vmlinux.o = LD $@
cmd_ld_vmlinux.o = \
$(LD) ${KBUILD_LDFLAGS} -r -o $@ \
+ $(vmlinux-o-ld-args-y) \
$(addprefix -T , $(initcalls-lds)) \
--whole-archive vmlinux.a --no-whole-archive \
--start-group $(KBUILD_VMLINUX_LIBS) --end-group \
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 4/6] module: script to generate offset ranges for builtin modules
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
` (2 preceding siblings ...)
2024-05-11 22:40 ` [PATCH v2 3/6] kbuild: generate a linker map for vmlinux.o Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel Kris Van Hees
` (2 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Nick Alcock, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
The offset range data for builtin modules is generated using:
- modules.builtin.objs: associates object files with module names
- vmlinux.map: provides load order of sections and offset of first member
per section
- vmlinux.o.map: provides offset of object file content per section
The generated data will look like:
.text 00000000-00000000 = _text
.text 0000baf0-0000cb10 amd_uncore
.text 0009bd10-0009c8e0 iosf_mbi
...
.text 008e6660-008e9630 snd_soc_wcd_mbhc
.text 008e9630-008ea610 snd_soc_wcd9335 snd_soc_wcd934x snd_soc_wcd938x
.text 008ea610-008ea780 snd_soc_wcd9335
...
.data 00000000-00000000 = _sdata
.data 0000f020-0000f680 amd_uncore
For each ELF section, it lists the offset of the first symbol. This can
be used to determine the base address of the section at runtime.
Next, it lists (in strict ascending order) offset ranges in that section
that cover the symbols of one or more builtin modules. Multiple ranges
can apply to a single module, and ranges can be shared between modules.
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
---
Changes since v1:
- Updated commit msg (vmlinux.o -> vmlinux.map)
---
scripts/generate_builtin_ranges.awk | 150 ++++++++++++++++++++++++++++
1 file changed, 150 insertions(+)
create mode 100755 scripts/generate_builtin_ranges.awk
diff --git a/scripts/generate_builtin_ranges.awk b/scripts/generate_builtin_ranges.awk
new file mode 100755
index 0000000000000..d5d668c97bd7f
--- /dev/null
+++ b/scripts/generate_builtin_ranges.awk
@@ -0,0 +1,150 @@
+#!/usr/bin/gawk -f
+# SPDX-License-Identifier: GPL-2.0
+
+FNR == 1 {
+ FC++;
+}
+
+# (1) Build a mapping to associate object files with built-in module names.
+#
+# The first file argument is used as input (modules.builtin.objs).
+#
+FC == 1 {
+ sub(/:/, "");
+ mod = $1;
+ sub(/([^/]*\/)+/, "", mod);
+ sub(/\.o$/, "", mod);
+ gsub(/-/, "_", mod);
+
+ if (NF > 1) {
+ for (i = 2; i <= NF; i++) {
+ if ($i in mods)
+ mods[$i] = mods[$i] " " mod;
+ else
+ mods[$i] = mod;
+ }
+ } else
+ mods[$1] = mod;
+
+ next;
+}
+
+# (2) Determine the load address for each section.
+#
+# The second file argument is used as input (vmlinux.map).
+# Since some AWK implementations cannot handle large integers, we strip of the
+# first 4 hex digits from the address. This is safe because the kernel space
+# is not large enough for addresses to extend into those digits.
+#
+FC == 2 && /^\./ && NF > 2 {
+ if (type)
+ delete sect_addend[type];
+
+ if ($1 ~ /percpu/)
+ next;
+
+ raw_addr = $2;
+ addr_prefix = "^" substr($2, 1, 6);
+ sub(addr_prefix, "0x", $2);
+ base = strtonum($2);
+ type = $1;
+ anchor = 0;
+ sect_base[type] = base;
+
+ next;
+}
+
+!type {
+ next;
+}
+
+# (3) We need to determine the base address of the section so that ranges can
+# be expressed based on offsets from the base address. This accommodates the
+# kernel sections getting loaded at different addresses than what is recorded
+# in vmlinux.map.
+#
+# At runtime, we will need to determine the base address of each section we are
+# interested in. We do that by recording the offset of the first symbol in the
+# section. Once we know the address of this symbol in the running kernel, we
+# can calculate the base address of the section.
+#
+# If possible, we use an explicit anchor symbol (sym = .) listed at the base
+# address (offset 0).
+#
+# If there is no such symbol, we record the first symbol in the section along
+# with its offset.
+#
+# We also determine the offset of the first member in the section in case the
+# final linking inserts some content between the start of the section and the
+# first member. I.e. in that case, vmlinux.map will list the first member at
+# a non-zero offset whereas vmlinux.o.map will list it at offset 0. We record
+# the addend so we can apply it when processing vmlinux.o.map (next).
+#
+FC == 2 && !anchor && raw_addr == $1 && $3 == "=" && $4 == "." {
+ anchor = sprintf("%s %08x-%08x = %s", type, 0, 0, $2);
+ sect_anchor[type] = anchor;
+
+ next;
+}
+
+FC == 2 && !anchor && $1 ~ /^0x/ && $2 !~ /^0x/ && NF <= 4 {
+ sub(addr_prefix, "0x", $1);
+ addr = strtonum($1) - base;
+ anchor = sprintf("%s %08x-%08x = %s", type, addr, addr, $2);
+ sect_anchor[type] = anchor;
+
+ next;
+}
+
+FC == 2 && base && /^ \./ && $1 == type && NF == 4 {
+ sub(addr_prefix, "0x", $2);
+ addr = strtonum($2);
+ sect_addend[type] = addr - base;
+
+ if (anchor) {
+ base = 0;
+ type = 0;
+ }
+
+ next;
+}
+
+# (4) Collect offset ranges (relative to the section base address) for built-in
+# modules.
+#
+FC == 3 && /^ \./ && NF == 4 && $3 != "0x0" {
+ type = $1;
+ if (!(type in sect_addend))
+ next;
+
+ sub(addr_prefix, "0x", $2);
+ addr = strtonum($2) + sect_addend[type];
+
+ if ($4 in mods)
+ mod = mods[$4];
+ else
+ mod = "";
+
+ if (mod == mod_name)
+ next;
+
+ if (mod_name) {
+ idx = mod_start + sect_base[type] + sect_addend[type];
+ entries[idx] = sprintf("%s %08x-%08x %s", type, mod_start, addr, mod_name);
+ count[type]++;
+ }
+
+ mod_name = mod;
+ mod_start = addr;
+}
+
+END {
+ for (type in count) {
+ if (type in sect_anchor)
+ entries[sect_base[type]] = sect_anchor[type];
+ }
+
+ n = asorti(entries, indices);
+ for (i = 1; i <= n; i++)
+ print entries[indices[i]];
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
` (3 preceding siblings ...)
2024-05-11 22:40 ` [PATCH v2 4/6] module: script to generate offset ranges for builtin modules Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-13 5:19 ` Masahiro Yamada
2024-05-11 22:40 ` [PATCH v2 6/6] module: add install target for modules.builtin.ranges Kris Van Hees
2024-05-13 4:43 ` [PATCH v2 0/6] Generate address range data for built-in modules Masahiro Yamada
6 siblings, 1 reply; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Nick Alcock, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
---
Changes since v1:
- Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
---
scripts/Makefile.vmlinux | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index c9f3e03124d7f..54095d72f7fd7 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -36,6 +36,23 @@ targets += vmlinux
vmlinux: scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE
+$(call if_changed_dep,link_vmlinux)
+# module.builtin.ranges
+# ---------------------------------------------------------------------------
+ifdef CONFIG_BUILTIN_MODULE_RANGES
+__default: modules.builtin.ranges
+
+quiet_cmd_modules_builtin_ranges = GEN $@
+ cmd_modules_builtin_ranges = \
+ $(srctree)/scripts/generate_builtin_ranges.awk \
+ $(filter-out FORCE,$+) > $@
+
+vmlinux.map: vmlinux
+
+targets += modules.builtin.ranges
+modules.builtin.ranges: modules.builtin.objs vmlinux.map vmlinux.o.map FORCE
+ $(call if_changed,modules_builtin_ranges)
+endif
+
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
# ---------------------------------------------------------------------------
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 6/6] module: add install target for modules.builtin.ranges
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
` (4 preceding siblings ...)
2024-05-11 22:40 ` [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel Kris Van Hees
@ 2024-05-11 22:40 ` Kris Van Hees
2024-05-13 5:22 ` Masahiro Yamada
2024-05-13 4:43 ` [PATCH v2 0/6] Generate address range data for built-in modules Masahiro Yamada
6 siblings, 1 reply; 14+ messages in thread
From: Kris Van Hees @ 2024-05-11 22:40 UTC (permalink / raw
To: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel
Cc: Kris Van Hees, Nick Alcock, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
When CONFIG_BUILTIN_MODULE_RANGES is enabled, the modules.builtin.ranges
file should be installed in the module install location.
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
---
Changes since v1:
- Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
---
scripts/Makefile.modinst | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index b45586aa1de49..e185dacae892a 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -36,6 +36,11 @@ install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo mod
$(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin.objs): $(MODLIB)/%: % FORCE
$(call cmd,install)
+install-$(CONFIG_BUILTIN_MODULE_RANGES) += $(MODLIB)/modules.builtin.ranges
+
+$(MODLIB)/modules.builtin.ranges: modules.builtin.ranges FORCE
+ $(call cmd,install)
+
endif
modules := $(call read-file, $(MODORDER))
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v2 0/6] Generate address range data for built-in modules
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
` (5 preceding siblings ...)
2024-05-11 22:40 ` [PATCH v2 6/6] module: add install target for modules.builtin.ranges Kris Van Hees
@ 2024-05-13 4:43 ` Masahiro Yamada
2024-05-15 16:50 ` Kris Van Hees
6 siblings, 1 reply; 14+ messages in thread
From: Masahiro Yamada @ 2024-05-13 4:43 UTC (permalink / raw
To: Kris Van Hees
Cc: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel,
Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Sun, May 12, 2024 at 7:42 AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
>
> Especially for tracing applications, it is convenient to be able to
> refer to a symbol using a <module name, symbol name> pair and to be able
> to translate an address into a <nodule mname, symbol name> pair. But
> that does not work if the module is built into the kernel because the
> object files that comprise the built-in module implementation are simply
> linked into the kernel image along with all other kernel object files.
>
> This is especially visible when providing tracing scripts for support
> purposes, where the developer of the script targets a particular kernel
> version, but does not have control over whether the target system has
> a particular module as loadable module or built-in module. When tracing
> symbols within a module, referring them by <module name, symbol name>
> pairs is both convenient and aids symbol lookup. But that naming will
> not work if the module name information is lost if the module is built
> into the kernel on the target system.
>
> Earlier work addressing this loss of information for built-in modules
> involved adding module name information to the kallsyms data, but that
> required more invasive code in the kernel proper. This work never did
> get merged into the kernel tree.
>
> All that is really needed is knowing whether a given address belongs to
> a particular module (or multiple modules if they share an object file).
> Or in other words, whether that address falls within an address range
> that is associated with one or more modules.
>
> This patch series is baaed on Luis Chamberlain's patch to generate
> modules.builtin.objs, associating built-in modules with their object
> files. Using this data, vmlinux.o.map and vmlinux.map can be parsed in
> a single pass to generate a modules.buitin.ranges file with offset range
> information (relative to the base address of the associated section) for
> built-in modules. The file gets installed along with the other
> modules.builtin.* files.
I still do not want to see modules.builtin.objs.
During the vmlinux.o.map parse, every time an object path
is encountered, you can open the corresponding .cmd file.
Let's say, you have the following in vmlinux.o.map:
.text 0x00000000007d4fe0 0x46c8 drivers/i2c/i2c-core-base.o
You can check drivers/i2c/.i2c-core-base.o.cmd
$ cat drivers/i2c/.i2c-core-base.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
-DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
Now you know this object is part of drivers/i2c/i2c-core
(that is, its modname is "i2c-core")
Next, you will get the following:
.text 0x00000000007dc550 0x13c4 drivers/i2c/i2c-core-acpi.o
$ cat drivers/i2c/.i2c-core-acpi.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
-DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
This one is also a part of drivers/i2c/i2c-core
You will get the address range of "i2c-core" without changing Makefiles.
You still need to modify scripts/Makefile.vmlinux(_o)
but you can implement everything else in your script,
although I did not fully understand the gawk script.
Now, you can use Python if you like:
https://lore.kernel.org/lkml/20240512-python-version-v2-1-382870a1fa1d@linaro.org/
Presumably, python code will be more readable for many people.
GNU awk is not documented in Documentation/process/changes.rst
If you insist on using gawk, you need to add it to the doc.
Having said that, I often hope to filter traced functions
by an object path instead of a modname because modname
filtering is only useful tristate code.
For example, filter by "path:drivers/i2c/" or "path:drivers/i2c/i2c-core*"
rather than "mod:i2c-core"
<object path, symbol name> reference will be useful for always-builtin code.
>
> The impact on the kernel build is minimal because everything is done
> using a single-pass AWK script. The generated data size is minimal as
> well, (depending on the exact kernel configuration) usually in the range
> of 500-700 lines, with a file size of 20-40KB.
>
> Changes since v1:
> - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> - Moved the config option to the tracers section
> - 2nd arg to generate_builtin_ranges.awk should be vmlinux.map
>
> Kris Van Hees (5):
> trace: add CONFIG_BUILTIN_MODULE_RANGES option
> kbuild: generate a linker map for vmlinux.o
> module: script to generate offset ranges for builtin modules
> kbuild: generate modules.builtin.ranges when linking the kernel
> module: add install target for modules.builtin.ranges
>
> Luis Chamberlain (1):
> kbuild: add modules.builtin.objs
>
> .gitignore | 2 +-
> Documentation/dontdiff | 2 +-
> Documentation/kbuild/kbuild.rst | 5 ++
> Makefile | 8 +-
> include/linux/module.h | 4 +-
> kernel/trace/Kconfig | 17 ++++
> scripts/Makefile.lib | 5 +-
> scripts/Makefile.modinst | 11 ++-
> scripts/Makefile.vmlinux | 17 ++++
> scripts/Makefile.vmlinux_o | 18 ++++-
> scripts/generate_builtin_ranges.awk | 149 ++++++++++++++++++++++++++++++++++++
> 11 files changed, 228 insertions(+), 10 deletions(-)
> create mode 100755 scripts/generate_builtin_ranges.awk
>
>
> base-commit: dd5a440a31fae6e459c0d6271dddd62825505361
> --
> 2.42.0
>
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel
2024-05-11 22:40 ` [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel Kris Van Hees
@ 2024-05-13 5:19 ` Masahiro Yamada
0 siblings, 0 replies; 14+ messages in thread
From: Masahiro Yamada @ 2024-05-13 5:19 UTC (permalink / raw
To: Kris Van Hees
Cc: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel,
Nick Alcock, Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Sun, May 12, 2024 at 7:44 AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
>
> Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> ---
> Changes since v1:
> - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> ---
> scripts/Makefile.vmlinux | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
>
> diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
> index c9f3e03124d7f..54095d72f7fd7 100644
> --- a/scripts/Makefile.vmlinux
> +++ b/scripts/Makefile.vmlinux
> @@ -36,6 +36,23 @@ targets += vmlinux
> vmlinux: scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE
> +$(call if_changed_dep,link_vmlinux)
>
> +# module.builtin.ranges
> +# ---------------------------------------------------------------------------
> +ifdef CONFIG_BUILTIN_MODULE_RANGES
> +__default: modules.builtin.ranges
> +
> +quiet_cmd_modules_builtin_ranges = GEN $@
> + cmd_modules_builtin_ranges = \
> + $(srctree)/scripts/generate_builtin_ranges.awk \
> + $(filter-out FORCE,$+) > $@
$(filter-out FORCE,$+)
->
$(real-prereqs)
> +
> +vmlinux.map: vmlinux
> +
> +targets += modules.builtin.ranges
> +modules.builtin.ranges: modules.builtin.objs vmlinux.map vmlinux.o.map FORCE
> + $(call if_changed,modules_builtin_ranges)
> +endif
> +
> # Add FORCE to the prequisites of a target to force it to be always rebuilt.
> # ---------------------------------------------------------------------------
>
> --
> 2.43.0
>
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 6/6] module: add install target for modules.builtin.ranges
2024-05-11 22:40 ` [PATCH v2 6/6] module: add install target for modules.builtin.ranges Kris Van Hees
@ 2024-05-13 5:22 ` Masahiro Yamada
2024-05-13 11:38 ` Masahiro Yamada
0 siblings, 1 reply; 14+ messages in thread
From: Masahiro Yamada @ 2024-05-13 5:22 UTC (permalink / raw
To: Kris Van Hees
Cc: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel,
Nick Alcock, Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Sun, May 12, 2024 at 7:59 AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
>
> When CONFIG_BUILTIN_MODULE_RANGES is enabled, the modules.builtin.ranges
> file should be installed in the module install location.
>
> Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> ---
> Changes since v1:
> - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> ---
> scripts/Makefile.modinst | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index b45586aa1de49..e185dacae892a 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -36,6 +36,11 @@ install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo mod
> $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin.objs): $(MODLIB)/%: % FORCE
> $(call cmd,install)
>
> +install-$(CONFIG_BUILTIN_MODULE_RANGES) += $(MODLIB)/modules.builtin.ranges
> +
> +$(MODLIB)/modules.builtin.ranges: modules.builtin.ranges FORCE
> + $(call cmd,install)
> +
Why add this to $(addprefix ...) part?
> endif
>
> modules := $(call read-file, $(MODORDER))
> --
> 2.43.0
>
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 6/6] module: add install target for modules.builtin.ranges
2024-05-13 5:22 ` Masahiro Yamada
@ 2024-05-13 11:38 ` Masahiro Yamada
0 siblings, 0 replies; 14+ messages in thread
From: Masahiro Yamada @ 2024-05-13 11:38 UTC (permalink / raw
To: Kris Van Hees
Cc: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel,
Nick Alcock, Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Mon, May 13, 2024 at 2:22 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sun, May 12, 2024 at 7:59 AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
> >
> > When CONFIG_BUILTIN_MODULE_RANGES is enabled, the modules.builtin.ranges
> > file should be installed in the module install location.
> >
> > Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
> > Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
> > ---
> > Changes since v1:
> > - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> > ---
> > scripts/Makefile.modinst | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> > index b45586aa1de49..e185dacae892a 100644
> > --- a/scripts/Makefile.modinst
> > +++ b/scripts/Makefile.modinst
> > @@ -36,6 +36,11 @@ install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo mod
> > $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin.objs): $(MODLIB)/%: % FORCE
> > $(call cmd,install)
> >
> > +install-$(CONFIG_BUILTIN_MODULE_RANGES) += $(MODLIB)/modules.builtin.ranges
> > +
> > +$(MODLIB)/modules.builtin.ranges: modules.builtin.ranges FORCE
> > + $(call cmd,install)
> > +
>
> Why add this to $(addprefix ...) part?
I meant, why don't you add this to $(addprefix ...)
Like this:
$(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo
modules.builtin.ranges): $(MODLIB)/%: % FORCE
$(call cmd,install)
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option
2024-05-11 22:40 ` [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option Kris Van Hees
@ 2024-05-14 23:20 ` kernel test robot
0 siblings, 0 replies; 14+ messages in thread
From: kernel test robot @ 2024-05-14 23:20 UTC (permalink / raw
To: Kris Van Hees, linux-kernel, linux-kbuild, linux-modules,
linux-trace-kernel
Cc: Paul Gazzillo, Necip Fazil Yildiran, oe-kbuild-all, Kris Van Hees,
Nick Alcock, Alan Maguire, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
Hi Kris,
kernel test robot noticed the following build warnings:
[auto build test WARNING on dd5a440a31fae6e459c0d6271dddd62825505361]
url: https://github.com/intel-lab-lkp/linux/commits/Kris-Van-Hees/kbuild-add-modules-builtin-objs/20240512-065954
base: dd5a440a31fae6e459c0d6271dddd62825505361
patch link: https://lore.kernel.org/r/20240511224035.27775-3-kris.van.hees%40oracle.com
patch subject: [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option
config: arc-kismet-CONFIG_VMLINUX_MAP-CONFIG_BUILTIN_MODULE_RANGES-0-0 (https://download.01.org/0day-ci/archive/20240515/202405150623.lmS5sVhM-lkp@intel.com/config)
reproduce: (https://download.01.org/0day-ci/archive/20240515/202405150623.lmS5sVhM-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202405150623.lmS5sVhM-lkp@intel.com/
kismet warnings: (new ones prefixed by >>)
>> kismet: WARNING: unmet direct dependencies detected for VMLINUX_MAP when selected by BUILTIN_MODULE_RANGES
WARNING: unmet direct dependencies detected for VMLINUX_MAP
Depends on [n]: EXPERT [=n]
Selected by [y]:
- BUILTIN_MODULE_RANGES [=y] && FTRACE [=y]
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 0/6] Generate address range data for built-in modules
2024-05-13 4:43 ` [PATCH v2 0/6] Generate address range data for built-in modules Masahiro Yamada
@ 2024-05-15 16:50 ` Kris Van Hees
2024-05-16 2:56 ` Masahiro Yamada
0 siblings, 1 reply; 14+ messages in thread
From: Kris Van Hees @ 2024-05-15 16:50 UTC (permalink / raw
To: Masahiro Yamada
Cc: Kris Van Hees, linux-kernel, linux-kbuild, linux-modules,
linux-trace-kernel, Steven Rostedt, Luis Chamberlain,
Masami Hiramatsu, Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Mon, May 13, 2024 at 01:43:15PM +0900, Masahiro Yamada wrote:
> On Sun, May 12, 2024 at 7:42???AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
> >
> > Especially for tracing applications, it is convenient to be able to
> > refer to a symbol using a <module name, symbol name> pair and to be able
> > to translate an address into a <nodule mname, symbol name> pair. But
> > that does not work if the module is built into the kernel because the
> > object files that comprise the built-in module implementation are simply
> > linked into the kernel image along with all other kernel object files.
> >
> > This is especially visible when providing tracing scripts for support
> > purposes, where the developer of the script targets a particular kernel
> > version, but does not have control over whether the target system has
> > a particular module as loadable module or built-in module. When tracing
> > symbols within a module, referring them by <module name, symbol name>
> > pairs is both convenient and aids symbol lookup. But that naming will
> > not work if the module name information is lost if the module is built
> > into the kernel on the target system.
> >
> > Earlier work addressing this loss of information for built-in modules
> > involved adding module name information to the kallsyms data, but that
> > required more invasive code in the kernel proper. This work never did
> > get merged into the kernel tree.
> >
> > All that is really needed is knowing whether a given address belongs to
> > a particular module (or multiple modules if they share an object file).
> > Or in other words, whether that address falls within an address range
> > that is associated with one or more modules.
> >
> > This patch series is baaed on Luis Chamberlain's patch to generate
> > modules.builtin.objs, associating built-in modules with their object
> > files. Using this data, vmlinux.o.map and vmlinux.map can be parsed in
> > a single pass to generate a modules.buitin.ranges file with offset range
> > information (relative to the base address of the associated section) for
> > built-in modules. The file gets installed along with the other
> > modules.builtin.* files.
>
>
>
> I still do not want to see modules.builtin.objs.
>
>
> During the vmlinux.o.map parse, every time an object path
> is encountered, you can open the corresponding .cmd file.
>
>
>
> Let's say, you have the following in vmlinux.o.map:
>
> .text 0x00000000007d4fe0 0x46c8 drivers/i2c/i2c-core-base.o
>
>
>
> You can check drivers/i2c/.i2c-core-base.o.cmd
>
>
> $ cat drivers/i2c/.i2c-core-base.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
>
>
> Now you know this object is part of drivers/i2c/i2c-core
> (that is, its modname is "i2c-core")
>
>
>
>
> Next, you will get the following:
>
> .text 0x00000000007dc550 0x13c4 drivers/i2c/i2c-core-acpi.o
>
>
> $ cat drivers/i2c/.i2c-core-acpi.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
>
>
> This one is also a part of drivers/i2c/i2c-core
>
>
> You will get the address range of "i2c-core" without changing Makefiles.
Thank you for this suggestion. I have this approach now implemented, making
use of both KBUILD_MODFILE and KBUILD_MODNAME (both are needed to conclusively
determine that an object belongs to a module).
However, this is not catching objects that are compiled from assembler source,
because modfile_flags and modname_flags are not added to the assembler flags,
and thus KBUILD_MODFILE and KBUILD_MODNAME are not present in the .cmd file
for those objects.
It would seem that it is harmless to add those flags to assembler flags, so
would that be an acceptable solution? It definitely would provide consistency
with non-asm objects. And we already pass modfile and modname flags to the
non-asm builds for objects that most certainly do not belong in modules amnyway,
e.g.
$ cat arch/x86/boot/.cmdline.o.cmd| tr ' ' '\n' | grep -- -DKBUILD_MOD
-DKBUILD_MODFILE='"arch/x86/boot/cmdline"'
-DKBUILD_MODNAME='"cmdline"'
> You still need to modify scripts/Makefile.vmlinux(_o)
> but you can implement everything else in your script,
> although I did not fully understand the gawk script.
>
>
> Now, you can use Python if you like:
>
> https://lore.kernel.org/lkml/20240512-python-version-v2-1-382870a1fa1d@linaro.org/
>
> Presumably, python code will be more readable for many people.
>
>
> GNU awk is not documented in Documentation/process/changes.rst
> If you insist on using gawk, you need to add it to the doc.
>
>
>
>
>
> Having said that, I often hope to filter traced functions
> by an object path instead of a modname because modname
> filtering is only useful tristate code.
> For example, filter by "path:drivers/i2c/" or "path:drivers/i2c/i2c-core*"
> rather than "mod:i2c-core"
>
> <object path, symbol name> reference will be useful for always-builtin code.
>
>
>
>
> >
> > The impact on the kernel build is minimal because everything is done
> > using a single-pass AWK script. The generated data size is minimal as
> > well, (depending on the exact kernel configuration) usually in the range
> > of 500-700 lines, with a file size of 20-40KB.
> >
> > Changes since v1:
> > - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> > - Moved the config option to the tracers section
> > - 2nd arg to generate_builtin_ranges.awk should be vmlinux.map
> >
> > Kris Van Hees (5):
> > trace: add CONFIG_BUILTIN_MODULE_RANGES option
> > kbuild: generate a linker map for vmlinux.o
> > module: script to generate offset ranges for builtin modules
> > kbuild: generate modules.builtin.ranges when linking the kernel
> > module: add install target for modules.builtin.ranges
> >
> > Luis Chamberlain (1):
> > kbuild: add modules.builtin.objs
> >
> > .gitignore | 2 +-
> > Documentation/dontdiff | 2 +-
> > Documentation/kbuild/kbuild.rst | 5 ++
> > Makefile | 8 +-
> > include/linux/module.h | 4 +-
> > kernel/trace/Kconfig | 17 ++++
> > scripts/Makefile.lib | 5 +-
> > scripts/Makefile.modinst | 11 ++-
> > scripts/Makefile.vmlinux | 17 ++++
> > scripts/Makefile.vmlinux_o | 18 ++++-
> > scripts/generate_builtin_ranges.awk | 149 ++++++++++++++++++++++++++++++++++++
> > 11 files changed, 228 insertions(+), 10 deletions(-)
> > create mode 100755 scripts/generate_builtin_ranges.awk
> >
> >
> > base-commit: dd5a440a31fae6e459c0d6271dddd62825505361
> > --
> > 2.42.0
> >
> >
>
>
> --
> Best Regards
> Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 0/6] Generate address range data for built-in modules
2024-05-15 16:50 ` Kris Van Hees
@ 2024-05-16 2:56 ` Masahiro Yamada
0 siblings, 0 replies; 14+ messages in thread
From: Masahiro Yamada @ 2024-05-16 2:56 UTC (permalink / raw
To: Kris Van Hees
Cc: linux-kernel, linux-kbuild, linux-modules, linux-trace-kernel,
Steven Rostedt, Luis Chamberlain, Masami Hiramatsu,
Nick Desaulniers, Jiri Olsa, Elena Zannoni
On Thu, May 16, 2024 at 1:50 AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
>
> On Mon, May 13, 2024 at 01:43:15PM +0900, Masahiro Yamada wrote:
> > On Sun, May 12, 2024 at 7:42???AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
> > >
> > > Especially for tracing applications, it is convenient to be able to
> > > refer to a symbol using a <module name, symbol name> pair and to be able
> > > to translate an address into a <nodule mname, symbol name> pair. But
> > > that does not work if the module is built into the kernel because the
> > > object files that comprise the built-in module implementation are simply
> > > linked into the kernel image along with all other kernel object files.
> > >
> > > This is especially visible when providing tracing scripts for support
> > > purposes, where the developer of the script targets a particular kernel
> > > version, but does not have control over whether the target system has
> > > a particular module as loadable module or built-in module. When tracing
> > > symbols within a module, referring them by <module name, symbol name>
> > > pairs is both convenient and aids symbol lookup. But that naming will
> > > not work if the module name information is lost if the module is built
> > > into the kernel on the target system.
> > >
> > > Earlier work addressing this loss of information for built-in modules
> > > involved adding module name information to the kallsyms data, but that
> > > required more invasive code in the kernel proper. This work never did
> > > get merged into the kernel tree.
> > >
> > > All that is really needed is knowing whether a given address belongs to
> > > a particular module (or multiple modules if they share an object file).
> > > Or in other words, whether that address falls within an address range
> > > that is associated with one or more modules.
> > >
> > > This patch series is baaed on Luis Chamberlain's patch to generate
> > > modules.builtin.objs, associating built-in modules with their object
> > > files. Using this data, vmlinux.o.map and vmlinux.map can be parsed in
> > > a single pass to generate a modules.buitin.ranges file with offset range
> > > information (relative to the base address of the associated section) for
> > > built-in modules. The file gets installed along with the other
> > > modules.builtin.* files.
> >
> >
> >
> > I still do not want to see modules.builtin.objs.
> >
> >
> > During the vmlinux.o.map parse, every time an object path
> > is encountered, you can open the corresponding .cmd file.
> >
> >
> >
> > Let's say, you have the following in vmlinux.o.map:
> >
> > .text 0x00000000007d4fe0 0x46c8 drivers/i2c/i2c-core-base.o
> >
> >
> >
> > You can check drivers/i2c/.i2c-core-base.o.cmd
> >
> >
> > $ cat drivers/i2c/.i2c-core-base.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> > -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
> >
> >
> > Now you know this object is part of drivers/i2c/i2c-core
> > (that is, its modname is "i2c-core")
> >
> >
> >
> >
> > Next, you will get the following:
> >
> > .text 0x00000000007dc550 0x13c4 drivers/i2c/i2c-core-acpi.o
> >
> >
> > $ cat drivers/i2c/.i2c-core-acpi.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> > -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
> >
> >
> > This one is also a part of drivers/i2c/i2c-core
> >
> >
> > You will get the address range of "i2c-core" without changing Makefiles.
>
> Thank you for this suggestion. I have this approach now implemented, making
> use of both KBUILD_MODFILE and KBUILD_MODNAME (both are needed to conclusively
> determine that an object belongs to a module).
>
> However, this is not catching objects that are compiled from assembler source,
> because modfile_flags and modname_flags are not added to the assembler flags,
> and thus KBUILD_MODFILE and KBUILD_MODNAME are not present in the .cmd file
> for those objects.
>
> It would seem that it is harmless to add those flags to assembler flags, so
> would that be an acceptable solution? It definitely would provide consistency
> with non-asm objects. And we already pass modfile and modname flags to the
> non-asm builds for objects that most certainly do not belong in modules amnyway,
> e.g.
>
> $ cat arch/x86/boot/.cmdline.o.cmd| tr ' ' '\n' | grep -- -DKBUILD_MOD
> -DKBUILD_MODFILE='"arch/x86/boot/cmdline"'
> -DKBUILD_MODNAME='"cmdline"'
I am fine with passing these to *.S files,
as the -D is a preprocessor option.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-05-16 2:57 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-11 22:40 [PATCH v2 0/6] Generate address range data for built-in modules Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 1/6] kbuild: add modules.builtin.objs Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 2/6] trace: add CONFIG_BUILTIN_MODULE_RANGES option Kris Van Hees
2024-05-14 23:20 ` kernel test robot
2024-05-11 22:40 ` [PATCH v2 3/6] kbuild: generate a linker map for vmlinux.o Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 4/6] module: script to generate offset ranges for builtin modules Kris Van Hees
2024-05-11 22:40 ` [PATCH v2 5/6] kbuild: generate modules.builtin.ranges when linking the kernel Kris Van Hees
2024-05-13 5:19 ` Masahiro Yamada
2024-05-11 22:40 ` [PATCH v2 6/6] module: add install target for modules.builtin.ranges Kris Van Hees
2024-05-13 5:22 ` Masahiro Yamada
2024-05-13 11:38 ` Masahiro Yamada
2024-05-13 4:43 ` [PATCH v2 0/6] Generate address range data for built-in modules Masahiro Yamada
2024-05-15 16:50 ` Kris Van Hees
2024-05-16 2:56 ` Masahiro Yamada
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).