* [PATCH v6 00/21] Split sphinx call logic from docs Makefile
@ 2025-09-16 10:22 Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 01/21] scripts/jobserver-exec: move the code to a class Mauro Carvalho Chehab
` (22 more replies)
0 siblings, 23 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Linux Doc Mailing List, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Hi Jon,
This series should probably be called:
"Move the trick-or-treat build hacks accumulated over time
into a single place and document them."
as this reflects its main goal. As such:
- it places the jobserver logic on a library;
- it removes sphinx/parallel-wrapper.sh;
- the code now properly implements a jobserver-aware logic
to do the parallelism when called via GNU make, failing back to
"-j" when there's no jobserver;
- converts check-variable-fonts.sh to Python and uses it via
function call;
- drops an extra script to generate man pages, adding a makefile
target for it;
- ensures that return code is 0 when PDF successfully builds;
- about half of the script is comments and documentation.
I tried to do my best to document all tricks that are inside the
script. This way, the docs build steps is now documented.
It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.
Such script was written in a way that it can be called either
directly or via a Makefile. Running outside Makefile is
interesting specially when debug is needed. The command line
interface replaces the need of having lots of env vars before
calling sphinx-build:
$ ./tools/docs/sphinx-build-wrapper --help
usage: sphinx-build-wrapper [-h]
[--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
[--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
[-j JOBS] [-i] [-V [VENV]]
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Kernel documentation builder
positional arguments:
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Documentation target to build
options:
-h, --help show this help message and exit
--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
Specific directories to build
--conf CONF Sphinx configuration file
--builddir BUILDDIR Sphinx configuration file
--theme THEME Sphinx theme to use
--css CSS Custom CSS file for HTML/EPUB
--paper {,a4,letter} Paper size for LaTeX/PDF output
-v, --verbose place build in verbose mode
-j, --jobs JOBS Sets number of jobs to use with sphinx-build
-i, --interactive Change latex default to run in interactive mode
-V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)
the only mandatory argument is the target, which is identical with
"make" targets.
The call inside Makefile doesn't use the last four arguments. They're
there to help identifying problems at the build:
-v makes the output verbose;
-j helps to test parallelism;
-i runs latexmk in interactive mode, allowing to debug PDF
build issues;
-V is useful when testing it with different venvs.
When used with GNU make (or some other make which implements jobserver),
a call like:
make -j <targets> htmldocs
will make the wrapper to automatically use POSIX jobserver to claim
the number of available job slots, calling sphinx-build with a
"-j" parameter reflecting it. ON such case, the default can be
overriden via SPHINXDIRS argument.
Visiable changes when compared with the old behavior:
When V=0, the only visible difference is that:
- pdfdocs target now returns 0 on success, 1 on failures.
This addresses an issue over the current process where we
it always return success even on failures;
- it will now print the name of PDF files that failed to build,
if any.
In verbose mode, sphinx-build-wrapper and sphinx-build command lines
are now displayed.
---
v6:
- On success, PDF output is identical as before when V=0;
- when V=1 is used, PDF output will print a build summary,
as on v5;
- solved a problem when multiple PDF files have the same
basename but are located on different directories;
- merged a patch series converting check-variable-fonts.sh
to Python. Its logic is now called directly without running
a subprocess.
- venv patch moved to the end.
v5:
- merged comments with the script;
- placed n_jobs on a separate function;
- nitpick: dropped a for loop used instead of list append.
v4:
- updated references for sphinx-pre-install after its rename;
- added some extra patches to add more options to python_version,
allowing it to bail out and suggest alternatives;
- added a patch at the end to explicitly break doc builds when
python3 points to python3.6 or older.
v3:
- rebased on the top of docs-next;
- added two patches to build man files that were on a separate
patch series.
v2:
- there's no generic exception handler anymore;
- it moves sphinx-pre-install to tools/docs;
- the logic which ensures a minimal Python version got moved
to a library, which is now used by both pre-install and wrapper;
- The first wrapper (05/13) doesn't contain comments (except for
shebang and SPDX). The goal is to help showing the size increase
when moving from Makefile to Python. Some file increase is
unavoidable, as Makefile is more compact: no includes, multple
statements per line, no argparse, etc;
- The second patch adds docstrings and comments. It has almost
the same size of the code itself;
- I moved the venv logic to a third wrapper patch;
- I fixed an issue at the paraller build logic;
- There are no generic except blocks anymore.
Mauro Carvalho Chehab (21):
scripts/jobserver-exec: move the code to a class
scripts/jobserver-exec: move its class to the lib directory
scripts/jobserver-exec: add a help message
scripts: check-variable-fonts.sh: convert to Python
tools/docs: check-variable-fonts.py: split into a lib and an exec file
scripts: sphinx-pre-install: move it to tools/docs
tools/docs: python_version: move version check from sphinx-pre-install
tools/docs: python_version: drop a debug print
tools/docs: python_version: allow check for alternatives and bail out
tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
docs: parallel-wrapper.sh: remove script
docs: Makefile: document latex/PDF PAPER= parameter
tools/docs: sphinx-build-wrapper: add an argument for LaTeX
interactive mode
tools/docs,scripts: sphinx-*: prevent sphinx-build crashes
tools/docs: sphinx-build-wrapper: allow building PDF files in parallel
tools/docs: sphinx-build-wrapper: Fix output for duplicated names
docs: add support to build manpages from kerneldoc output
tools: kernel-doc: add a see also section at man pages
scripts: kdoc_parser.py: warn about Python version only once
tools/docs: sphinx-* break documentation bulds on openSUSE
tools/docs: sphinx-build-wrapper: add support to run inside venv
Documentation/Makefile | 136 +--
Documentation/doc-guide/kernel-doc.rst | 29 +-
Documentation/doc-guide/sphinx.rst | 4 +-
Documentation/sphinx/kerneldoc-preamble.sty | 2 +-
Documentation/sphinx/parallel-wrapper.sh | 33 -
.../translations/it_IT/doc-guide/sphinx.rst | 4 +-
.../translations/zh_CN/doc-guide/sphinx.rst | 4 +-
Documentation/translations/zh_CN/how-to.rst | 2 +-
MAINTAINERS | 4 +-
Makefile | 2 +-
scripts/check-variable-fonts.sh | 115 ---
scripts/jobserver-exec | 88 +-
scripts/lib/jobserver.py | 149 ++++
scripts/lib/kdoc/kdoc_files.py | 5 +-
scripts/lib/kdoc/kdoc_output.py | 84 +-
scripts/lib/kdoc/kdoc_parser.py | 7 +-
scripts/split-man.pl | 28 -
tools/docs/check-variable-fonts.py | 23 +
tools/docs/lib/latex_fonts.py | 162 ++++
tools/docs/lib/python_version.py | 178 ++++
tools/docs/sphinx-build-wrapper | 791 ++++++++++++++++++
{scripts => tools/docs}/sphinx-pre-install | 135 +--
22 files changed, 1502 insertions(+), 483 deletions(-)
delete mode 100644 Documentation/sphinx/parallel-wrapper.sh
delete mode 100755 scripts/check-variable-fonts.sh
create mode 100755 scripts/lib/jobserver.py
delete mode 100755 scripts/split-man.pl
create mode 100755 tools/docs/check-variable-fonts.py
create mode 100755 tools/docs/lib/latex_fonts.py
create mode 100644 tools/docs/lib/python_version.py
create mode 100755 tools/docs/sphinx-build-wrapper
rename {scripts => tools/docs}/sphinx-pre-install (93%)
--
2.51.0
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v6 01/21] scripts/jobserver-exec: move the code to a class
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 02/21] scripts/jobserver-exec: move its class to the lib directory Mauro Carvalho Chehab
` (21 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Convert the code inside jobserver-exec to a class and
properly document it.
Using a class allows reusing the jobserver logic on other
scripts.
While the main code remains unchanged, being compatible with
Python 2.6 and 3.0+, its coding style now follows a more
modern standard, having tabs replaced by a 4-spaces
indent, passing autopep8, black and pylint.
The code now allows allows using a pythonic way to
enter/exit a python code, e.g. it now supports:
with JobserverExec() as jobserver:
jobserver.run(sys.argv[1:])
With the new code, the __exit__() function should ensure
that the jobserver slot will be closed at the end, even if
something bad happens somewhere.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/jobserver-exec | 218 ++++++++++++++++++++++++++++-------------
1 file changed, 151 insertions(+), 67 deletions(-)
diff --git a/scripts/jobserver-exec b/scripts/jobserver-exec
index 7eca035472d3..b386b1a845de 100755
--- a/scripts/jobserver-exec
+++ b/scripts/jobserver-exec
@@ -1,77 +1,161 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0+
#
+# pylint: disable=C0103,C0209
+#
# This determines how many parallel tasks "make" is expecting, as it is
# not exposed via an special variables, reserves them all, runs a subprocess
# with PARALLELISM environment variable set, and releases the jobs back again.
#
# https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
-from __future__ import print_function
-import os, sys, errno
+
+"""
+Interacts with the POSIX jobserver during the Kernel build time.
+
+A "normal" jobserver task, like the one initiated by a make subrocess would do:
+
+ - open read/write file descriptors to communicate with the job server;
+ - ask for one slot by calling:
+ claim = os.read(reader, 1)
+ - when the job finshes, call:
+ os.write(writer, b"+") # os.write(writer, claim)
+
+Here, the goal is different: This script aims to get the remaining number
+of slots available, using all of them to run a command which handle tasks in
+parallel. To to that, it has a loop that ends only after there are no
+slots left. It then increments the number by one, in order to allow a
+call equivalent to make -j$((claim+1)), e.g. having a parent make creating
+$claim child to do the actual work.
+
+The end goal here is to keep the total number of build tasks under the
+limit established by the initial make -j$n_proc call.
+"""
+
+import errno
+import os
import subprocess
+import sys
-# Extract and prepare jobserver file descriptors from environment.
-claim = 0
-jobs = b""
-try:
- # Fetch the make environment options.
- flags = os.environ['MAKEFLAGS']
-
- # Look for "--jobserver=R,W"
- # Note that GNU Make has used --jobserver-fds and --jobserver-auth
- # so this handles all of them.
- opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
-
- # Parse out R,W file descriptor numbers and set them nonblocking.
- # If the MAKEFLAGS variable contains multiple instances of the
- # --jobserver-auth= option, the last one is relevant.
- fds = opts[-1].split("=", 1)[1]
-
- # Starting with GNU Make 4.4, named pipes are used for reader and writer.
- # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
- _, _, path = fds.partition('fifo:')
-
- if path:
- reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
- writer = os.open(path, os.O_WRONLY)
- else:
- reader, writer = [int(x) for x in fds.split(",", 1)]
- # Open a private copy of reader to avoid setting nonblocking
- # on an unexpecting process with the same reader fd.
- reader = os.open("/proc/self/fd/%d" % (reader),
- os.O_RDONLY | os.O_NONBLOCK)
-
- # Read out as many jobserver slots as possible.
- while True:
- try:
- slot = os.read(reader, 8)
- jobs += slot
- except (OSError, IOError) as e:
- if e.errno == errno.EWOULDBLOCK:
- # Stop at the end of the jobserver queue.
- break
- # If something went wrong, give back the jobs.
- if len(jobs):
- os.write(writer, jobs)
- raise e
- # Add a bump for our caller's reserveration, since we're just going
- # to sit here blocked on our child.
- claim = len(jobs) + 1
-except (KeyError, IndexError, ValueError, OSError, IOError) as e:
- # Any missing environment strings or bad fds should result in just
- # not being parallel.
- pass
-
-# We can only claim parallelism if there was a jobserver (i.e. a top-level
-# "-jN" argument) and there were no other failures. Otherwise leave out the
-# environment variable and let the child figure out what is best.
-if claim > 0:
- os.environ['PARALLELISM'] = '%d' % (claim)
-
-rc = subprocess.call(sys.argv[1:])
-
-# Return all the reserved slots.
-if len(jobs):
- os.write(writer, jobs)
-
-sys.exit(rc)
+
+class JobserverExec:
+ """
+ Claim all slots from make using POSIX Jobserver.
+
+ The main methods here are:
+ - open(): reserves all slots;
+ - close(): method returns all used slots back to make;
+ - run(): executes a command setting PARALLELISM=<available slots jobs + 1>
+ """
+
+ def __init__(self):
+ """Initialize internal vars"""
+ self.claim = 0
+ self.jobs = b""
+ self.reader = None
+ self.writer = None
+ self.is_open = False
+
+ def open(self):
+ """Reserve all available slots to be claimed later on"""
+
+ if self.is_open:
+ return
+
+ try:
+ # Fetch the make environment options.
+ flags = os.environ["MAKEFLAGS"]
+ # Look for "--jobserver=R,W"
+ # Note that GNU Make has used --jobserver-fds and --jobserver-auth
+ # so this handles all of them.
+ opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
+
+ # Parse out R,W file descriptor numbers and set them nonblocking.
+ # If the MAKEFLAGS variable contains multiple instances of the
+ # --jobserver-auth= option, the last one is relevant.
+ fds = opts[-1].split("=", 1)[1]
+
+ # Starting with GNU Make 4.4, named pipes are used for reader
+ # and writer.
+ # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
+ _, _, path = fds.partition("fifo:")
+
+ if path:
+ self.reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
+ self.writer = os.open(path, os.O_WRONLY)
+ else:
+ self.reader, self.writer = [int(x) for x in fds.split(",", 1)]
+ # Open a private copy of reader to avoid setting nonblocking
+ # on an unexpecting process with the same reader fd.
+ self.reader = os.open("/proc/self/fd/%d" % (self.reader),
+ os.O_RDONLY | os.O_NONBLOCK)
+
+ # Read out as many jobserver slots as possible
+ while True:
+ try:
+ slot = os.read(self.reader, 8)
+ self.jobs += slot
+ except (OSError, IOError) as e:
+ if e.errno == errno.EWOULDBLOCK:
+ # Stop at the end of the jobserver queue.
+ break
+ # If something went wrong, give back the jobs.
+ if self.jobs:
+ os.write(self.writer, self.jobs)
+ raise e
+
+ # Add a bump for our caller's reserveration, since we're just going
+ # to sit here blocked on our child.
+ self.claim = len(self.jobs) + 1
+
+ except (KeyError, IndexError, ValueError, OSError, IOError):
+ # Any missing environment strings or bad fds should result in just
+ # not being parallel.
+ self.claim = None
+
+ self.is_open = True
+
+ def close(self):
+ """Return all reserved slots to Jobserver"""
+
+ if not self.is_open:
+ return
+
+ # Return all the reserved slots.
+ if len(self.jobs):
+ os.write(self.writer, self.jobs)
+
+ self.is_open = False
+
+ def __enter__(self):
+ self.open()
+ return self
+
+ def __exit__(self, exc_type, exc_value, exc_traceback):
+ self.close()
+
+ def run(self, cmd):
+ """
+ Run a command setting PARALLELISM env variable to the number of
+ available job slots (claim) + 1, e.g. it will reserve claim slots
+ to do the actual build work, plus one to monitor its childs.
+ """
+ self.open() # Ensure that self.claim is set
+
+ # We can only claim parallelism if there was a jobserver (i.e. a
+ # top-level "-jN" argument) and there were no other failures. Otherwise
+ # leave out the environment variable and let the child figure out what
+ # is best.
+ if self.claim:
+ os.environ["PARALLELISM"] = str(self.claim)
+
+ return subprocess.call(cmd)
+
+
+def main():
+ """Main program"""
+ with JobserverExec() as jobserver:
+ jobserver.run(sys.argv[1:])
+
+
+if __name__ == "__main__":
+ main()
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 02/21] scripts/jobserver-exec: move its class to the lib directory
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 01/21] scripts/jobserver-exec: move the code to a class Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 03/21] scripts/jobserver-exec: add a help message Mauro Carvalho Chehab
` (20 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
To make it easier to be re-used, move the JobserverExec class
to the library directory.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/jobserver-exec | 152 +++------------------------------------
scripts/lib/jobserver.py | 149 ++++++++++++++++++++++++++++++++++++++
2 files changed, 160 insertions(+), 141 deletions(-)
create mode 100755 scripts/lib/jobserver.py
diff --git a/scripts/jobserver-exec b/scripts/jobserver-exec
index b386b1a845de..40a0f0058733 100755
--- a/scripts/jobserver-exec
+++ b/scripts/jobserver-exec
@@ -1,155 +1,25 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0+
-#
-# pylint: disable=C0103,C0209
-#
-# This determines how many parallel tasks "make" is expecting, as it is
-# not exposed via an special variables, reserves them all, runs a subprocess
-# with PARALLELISM environment variable set, and releases the jobs back again.
-#
-# https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
-"""
-Interacts with the POSIX jobserver during the Kernel build time.
-
-A "normal" jobserver task, like the one initiated by a make subrocess would do:
-
- - open read/write file descriptors to communicate with the job server;
- - ask for one slot by calling:
- claim = os.read(reader, 1)
- - when the job finshes, call:
- os.write(writer, b"+") # os.write(writer, claim)
-
-Here, the goal is different: This script aims to get the remaining number
-of slots available, using all of them to run a command which handle tasks in
-parallel. To to that, it has a loop that ends only after there are no
-slots left. It then increments the number by one, in order to allow a
-call equivalent to make -j$((claim+1)), e.g. having a parent make creating
-$claim child to do the actual work.
-
-The end goal here is to keep the total number of build tasks under the
-limit established by the initial make -j$n_proc call.
-"""
-
-import errno
import os
-import subprocess
import sys
+LIB_DIR = "lib"
+SRC_DIR = os.path.dirname(os.path.realpath(__file__))
-class JobserverExec:
- """
- Claim all slots from make using POSIX Jobserver.
+sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
- The main methods here are:
- - open(): reserves all slots;
- - close(): method returns all used slots back to make;
- - run(): executes a command setting PARALLELISM=<available slots jobs + 1>
- """
+from jobserver import JobserverExec # pylint: disable=C0415
- def __init__(self):
- """Initialize internal vars"""
- self.claim = 0
- self.jobs = b""
- self.reader = None
- self.writer = None
- self.is_open = False
- def open(self):
- """Reserve all available slots to be claimed later on"""
-
- if self.is_open:
- return
-
- try:
- # Fetch the make environment options.
- flags = os.environ["MAKEFLAGS"]
- # Look for "--jobserver=R,W"
- # Note that GNU Make has used --jobserver-fds and --jobserver-auth
- # so this handles all of them.
- opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
-
- # Parse out R,W file descriptor numbers and set them nonblocking.
- # If the MAKEFLAGS variable contains multiple instances of the
- # --jobserver-auth= option, the last one is relevant.
- fds = opts[-1].split("=", 1)[1]
-
- # Starting with GNU Make 4.4, named pipes are used for reader
- # and writer.
- # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
- _, _, path = fds.partition("fifo:")
-
- if path:
- self.reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
- self.writer = os.open(path, os.O_WRONLY)
- else:
- self.reader, self.writer = [int(x) for x in fds.split(",", 1)]
- # Open a private copy of reader to avoid setting nonblocking
- # on an unexpecting process with the same reader fd.
- self.reader = os.open("/proc/self/fd/%d" % (self.reader),
- os.O_RDONLY | os.O_NONBLOCK)
-
- # Read out as many jobserver slots as possible
- while True:
- try:
- slot = os.read(self.reader, 8)
- self.jobs += slot
- except (OSError, IOError) as e:
- if e.errno == errno.EWOULDBLOCK:
- # Stop at the end of the jobserver queue.
- break
- # If something went wrong, give back the jobs.
- if self.jobs:
- os.write(self.writer, self.jobs)
- raise e
-
- # Add a bump for our caller's reserveration, since we're just going
- # to sit here blocked on our child.
- self.claim = len(self.jobs) + 1
-
- except (KeyError, IndexError, ValueError, OSError, IOError):
- # Any missing environment strings or bad fds should result in just
- # not being parallel.
- self.claim = None
-
- self.is_open = True
-
- def close(self):
- """Return all reserved slots to Jobserver"""
-
- if not self.is_open:
- return
-
- # Return all the reserved slots.
- if len(self.jobs):
- os.write(self.writer, self.jobs)
-
- self.is_open = False
-
- def __enter__(self):
- self.open()
- return self
-
- def __exit__(self, exc_type, exc_value, exc_traceback):
- self.close()
-
- def run(self, cmd):
- """
- Run a command setting PARALLELISM env variable to the number of
- available job slots (claim) + 1, e.g. it will reserve claim slots
- to do the actual build work, plus one to monitor its childs.
- """
- self.open() # Ensure that self.claim is set
-
- # We can only claim parallelism if there was a jobserver (i.e. a
- # top-level "-jN" argument) and there were no other failures. Otherwise
- # leave out the environment variable and let the child figure out what
- # is best.
- if self.claim:
- os.environ["PARALLELISM"] = str(self.claim)
-
- return subprocess.call(cmd)
+"""
+Determines how many parallel tasks "make" is expecting, as it is
+not exposed via an special variables, reserves them all, runs a subprocess
+with PARALLELISM environment variable set, and releases the jobs back again.
+See:
+ https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
+"""
def main():
"""Main program"""
diff --git a/scripts/lib/jobserver.py b/scripts/lib/jobserver.py
new file mode 100755
index 000000000000..98d8b0ff0c89
--- /dev/null
+++ b/scripts/lib/jobserver.py
@@ -0,0 +1,149 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0+
+#
+# pylint: disable=C0103,C0209
+#
+#
+
+"""
+Interacts with the POSIX jobserver during the Kernel build time.
+
+A "normal" jobserver task, like the one initiated by a make subrocess would do:
+
+ - open read/write file descriptors to communicate with the job server;
+ - ask for one slot by calling:
+ claim = os.read(reader, 1)
+ - when the job finshes, call:
+ os.write(writer, b"+") # os.write(writer, claim)
+
+Here, the goal is different: This script aims to get the remaining number
+of slots available, using all of them to run a command which handle tasks in
+parallel. To to that, it has a loop that ends only after there are no
+slots left. It then increments the number by one, in order to allow a
+call equivalent to make -j$((claim+1)), e.g. having a parent make creating
+$claim child to do the actual work.
+
+The end goal here is to keep the total number of build tasks under the
+limit established by the initial make -j$n_proc call.
+
+See:
+ https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
+"""
+
+import errno
+import os
+import subprocess
+import sys
+
+class JobserverExec:
+ """
+ Claim all slots from make using POSIX Jobserver.
+
+ The main methods here are:
+ - open(): reserves all slots;
+ - close(): method returns all used slots back to make;
+ - run(): executes a command setting PARALLELISM=<available slots jobs + 1>
+ """
+
+ def __init__(self):
+ """Initialize internal vars"""
+ self.claim = 0
+ self.jobs = b""
+ self.reader = None
+ self.writer = None
+ self.is_open = False
+
+ def open(self):
+ """Reserve all available slots to be claimed later on"""
+
+ if self.is_open:
+ return
+
+ try:
+ # Fetch the make environment options.
+ flags = os.environ["MAKEFLAGS"]
+ # Look for "--jobserver=R,W"
+ # Note that GNU Make has used --jobserver-fds and --jobserver-auth
+ # so this handles all of them.
+ opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
+
+ # Parse out R,W file descriptor numbers and set them nonblocking.
+ # If the MAKEFLAGS variable contains multiple instances of the
+ # --jobserver-auth= option, the last one is relevant.
+ fds = opts[-1].split("=", 1)[1]
+
+ # Starting with GNU Make 4.4, named pipes are used for reader
+ # and writer.
+ # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134
+ _, _, path = fds.partition("fifo:")
+
+ if path:
+ self.reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
+ self.writer = os.open(path, os.O_WRONLY)
+ else:
+ self.reader, self.writer = [int(x) for x in fds.split(",", 1)]
+ # Open a private copy of reader to avoid setting nonblocking
+ # on an unexpecting process with the same reader fd.
+ self.reader = os.open("/proc/self/fd/%d" % (self.reader),
+ os.O_RDONLY | os.O_NONBLOCK)
+
+ # Read out as many jobserver slots as possible
+ while True:
+ try:
+ slot = os.read(self.reader, 8)
+ self.jobs += slot
+ except (OSError, IOError) as e:
+ if e.errno == errno.EWOULDBLOCK:
+ # Stop at the end of the jobserver queue.
+ break
+ # If something went wrong, give back the jobs.
+ if self.jobs:
+ os.write(self.writer, self.jobs)
+ raise e
+
+ # Add a bump for our caller's reserveration, since we're just going
+ # to sit here blocked on our child.
+ self.claim = len(self.jobs) + 1
+
+ except (KeyError, IndexError, ValueError, OSError, IOError):
+ # Any missing environment strings or bad fds should result in just
+ # not being parallel.
+ self.claim = None
+
+ self.is_open = True
+
+ def close(self):
+ """Return all reserved slots to Jobserver"""
+
+ if not self.is_open:
+ return
+
+ # Return all the reserved slots.
+ if len(self.jobs):
+ os.write(self.writer, self.jobs)
+
+ self.is_open = False
+
+ def __enter__(self):
+ self.open()
+ return self
+
+ def __exit__(self, exc_type, exc_value, exc_traceback):
+ self.close()
+
+ def run(self, cmd, *args, **pwargs):
+ """
+ Run a command setting PARALLELISM env variable to the number of
+ available job slots (claim) + 1, e.g. it will reserve claim slots
+ to do the actual build work, plus one to monitor its childs.
+ """
+ self.open() # Ensure that self.claim is set
+
+ # We can only claim parallelism if there was a jobserver (i.e. a
+ # top-level "-jN" argument) and there were no other failures. Otherwise
+ # leave out the environment variable and let the child figure out what
+ # is best.
+ if self.claim:
+ os.environ["PARALLELISM"] = str(self.claim)
+
+ return subprocess.call(cmd, *args, **pwargs)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 03/21] scripts/jobserver-exec: add a help message
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 01/21] scripts/jobserver-exec: move the code to a class Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 02/21] scripts/jobserver-exec: move its class to the lib directory Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python Mauro Carvalho Chehab
` (19 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Currently, calling it without an argument shows an ugly error
message. Instead, print a message using pythondoc as description.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/jobserver-exec | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/scripts/jobserver-exec b/scripts/jobserver-exec
index 40a0f0058733..ae23afd344ec 100755
--- a/scripts/jobserver-exec
+++ b/scripts/jobserver-exec
@@ -1,6 +1,15 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0+
+"""
+Determines how many parallel tasks "make" is expecting, as it is
+not exposed via any special variables, reserves them all, runs a subprocess
+with PARALLELISM environment variable set, and releases the jobs back again.
+
+See:
+ https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
+"""
+
import os
import sys
@@ -12,17 +21,12 @@ sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
from jobserver import JobserverExec # pylint: disable=C0415
-"""
-Determines how many parallel tasks "make" is expecting, as it is
-not exposed via an special variables, reserves them all, runs a subprocess
-with PARALLELISM environment variable set, and releases the jobs back again.
-
-See:
- https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
-"""
-
def main():
"""Main program"""
+ if len(sys.argv) < 2:
+ name = os.path.basename(__file__)
+ sys.exit("usage: " + name +" command [args ...]\n" + __doc__)
+
with JobserverExec() as jobserver:
jobserver.run(sys.argv[1:])
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (2 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 03/21] scripts/jobserver-exec: add a help message Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-17 1:09 ` Akira Yokosawa
2025-09-16 10:22 ` [PATCH v6 05/21] tools/docs: check-variable-fonts.py: split into a lib and an exec file Mauro Carvalho Chehab
` (18 subsequent siblings)
22 siblings, 1 reply; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
This script handle errors when trying to build translations
with make pdfdocs.
As part of our cleanup work to remove hacks from docs Makefile,
convert this to python, preparing it to be part of a library
to be called by sphinx-build-wrapper.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
MAINTAINERS | 2 +-
...iable-fonts.sh => check-variable-fonts.py} | 104 +++++++++++++-----
2 files changed, 78 insertions(+), 28 deletions(-)
rename scripts/{check-variable-fonts.sh => check-variable-fonts.py} (61%)
diff --git a/MAINTAINERS b/MAINTAINERS
index ef87548b8f88..88d8f7435e6d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7301,7 +7301,7 @@ S: Maintained
P: Documentation/doc-guide/maintainer-profile.rst
T: git git://git.lwn.net/linux.git docs-next
F: Documentation/
-F: scripts/check-variable-fonts.sh
+F: scripts/check-variable-fonts.py
F: scripts/checktransupdate.py
F: scripts/documentation-file-ref-check
F: scripts/get_abi.py
diff --git a/scripts/check-variable-fonts.sh b/scripts/check-variable-fonts.py
similarity index 61%
rename from scripts/check-variable-fonts.sh
rename to scripts/check-variable-fonts.py
index ce63f0acea5f..71b88b680a73 100755
--- a/scripts/check-variable-fonts.sh
+++ b/scripts/check-variable-fonts.py
@@ -1,7 +1,9 @@
-#!/bin/sh
+#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (C) Akira Yokosawa, 2024
#
+# Ported to Python by (c) Mauro Carvalho Chehab, 2025
+#
# For "make pdfdocs", reports of build errors of translations.pdf started
# arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE
# tumbleweed have started deploying variable-font [3] format of "Noto CJK"
@@ -87,29 +89,77 @@
# Denylisting should be less invasive, as it is effective only while
# XeLaTeX runs in "make pdfdocs".
-# Default per-user fontconfig path (overridden by env variable)
-: ${FONTS_CONF_DENY_VF:=$HOME/deny-vf}
-
-export XDG_CONFIG_HOME=${FONTS_CONF_DENY_VF}
-
-notocjkvffonts=`fc-list : file family variable | \
- grep 'variable=True' | \
- grep -E -e 'Noto (Sans|Sans Mono|Serif) CJK' | \
- sed -e 's/^/ /' -e 's/: Noto S.*$//' | sort | uniq`
-
-if [ "x$notocjkvffonts" != "x" ] ; then
- echo '============================================================================='
- echo 'XeTeX is confused by "variable font" files listed below:'
- echo "$notocjkvffonts"
- echo
- echo 'For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.'
- echo 'Or, CJK pages can be skipped by uninstalling texlive-xecjk.'
- echo
- echo 'For more info on denylisting, other options, and variable font, see header'
- echo 'comments of scripts/check-variable-fonts.sh.'
- echo '============================================================================='
-fi
-
-# As this script is invoked from Makefile's error path, always error exit
-# regardless of whether any variable font is discovered or not.
-exit 1
+import os
+import re
+import subprocess
+import sys
+import textwrap
+
+class LatexFontChecker:
+ """
+ Detect problems with CJK variable fonts that affect PDF builds for
+ translations.
+ """
+
+ def __init__(self):
+ deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf")
+
+ self.environ = os.environ.copy()
+ self.environ['XDG_CONFIG_HOMEF'] = os.path.expanduser(deny_vf)
+
+ self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK")
+
+ def get_noto_cjk_vf_fonts(self):
+ """Get Noto CJK fonts"""
+
+ cjk_fonts = set()
+ cmd = ["fc-list", ":", "file", "family", "variable"]
+ try:
+ result = subprocess.run(cmd,stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ universal_newlines=True,
+ env=self.environ,
+ check=True)
+
+ except subprocess.CalledProcessError as exc:
+ sys.exit(f"Error running fc-list: {repr(exc)}")
+
+ for line in result.stdout.splitlines():
+ if 'variable=True' not in line:
+ continue
+
+ match = self.re_cjk.search(line)
+ if match:
+ cjk_fonts.add(match.group(1))
+
+ return sorted(cjk_fonts)
+
+ def check(self):
+ """Check for problems with CJK fonts"""
+
+ fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), " ")
+ if not fonts:
+ return None
+
+ rel_file = os.path.relpath(__file__, os.getcwd())
+
+ msg = "=" * 77 + "\n"
+ msg += 'XeTeX is confused by "variable font" files listed below:\n'
+ msg += fonts + "\n"
+ msg += textwrap.dedent(f"""
+ For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
+ Or, CJK pages can be skipped by uninstalling texlive-xecjk.
+
+ For more info on denylisting, other options, and variable font, see header
+ comments of {rel_file}.
+ """)
+ msg += "=" * 77
+
+ return msg
+
+if __name__ == "__main__":
+ msg = LatexFontChecker().check()
+ if msg:
+ print(msg)
+
+ sys.exit(1)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 05/21] tools/docs: check-variable-fonts.py: split into a lib and an exec file
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (3 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 06/21] scripts: sphinx-pre-install: move it to tools/docs Mauro Carvalho Chehab
` (17 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
As we'll be using the actual code inside sphinx-build-wrapper,
split the library from the executable, placing the exec at
the new place we've been using:
tools/docs
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 2 +-
MAINTAINERS | 1 -
scripts/check-variable-fonts.py | 165 -----------------------------
tools/docs/check-variable-fonts.py | 23 ++++
tools/docs/lib/latex_fonts.py | 162 ++++++++++++++++++++++++++++
5 files changed, 186 insertions(+), 167 deletions(-)
delete mode 100755 scripts/check-variable-fonts.py
create mode 100755 tools/docs/check-variable-fonts.py
create mode 100755 tools/docs/lib/latex_fonts.py
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 5c20c68be89a..b630d489b113 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -146,7 +146,7 @@ pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
pdfdocs: latexdocs
@$(srctree)/scripts/sphinx-pre-install --version-check
$(foreach var,$(SPHINXDIRS), \
- $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
+ $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/tools/docs/check-variable-fonts.py || exit; \
mkdir -p $(BUILDDIR)/$(var)/pdf; \
mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \
)
diff --git a/MAINTAINERS b/MAINTAINERS
index 88d8f7435e6d..76dd823bfcc4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7301,7 +7301,6 @@ S: Maintained
P: Documentation/doc-guide/maintainer-profile.rst
T: git git://git.lwn.net/linux.git docs-next
F: Documentation/
-F: scripts/check-variable-fonts.py
F: scripts/checktransupdate.py
F: scripts/documentation-file-ref-check
F: scripts/get_abi.py
diff --git a/scripts/check-variable-fonts.py b/scripts/check-variable-fonts.py
deleted file mode 100755
index 71b88b680a73..000000000000
--- a/scripts/check-variable-fonts.py
+++ /dev/null
@@ -1,165 +0,0 @@
-#!/usr/bin/env python3
-# SPDX-License-Identifier: GPL-2.0-only
-# Copyright (C) Akira Yokosawa, 2024
-#
-# Ported to Python by (c) Mauro Carvalho Chehab, 2025
-#
-# For "make pdfdocs", reports of build errors of translations.pdf started
-# arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE
-# tumbleweed have started deploying variable-font [3] format of "Noto CJK"
-# fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK
-# (Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which
-# does not (and likely never will) understand variable fonts for historical
-# reasons.
-#
-# The build error happens even when both of variable- and non-variable-format
-# fonts are found on the build system. To make matters worse, Fedora enlists
-# variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN,
-# -zh_TW, etc. Hence developers who have interest in CJK pages are more
-# likely to encounter the build errors.
-#
-# This script is invoked from the error path of "make pdfdocs" and emits
-# suggestions if variable-font files of "Noto CJK" fonts are in the list of
-# fonts accessible from XeTeX.
-#
-# References:
-# [1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
-# [2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
-# [3]: https://en.wikipedia.org/wiki/Variable_font
-# [4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
-# [5]: https://build.opensuse.org/request/show/1157217
-#
-#===========================================================================
-# Workarounds for building translations.pdf
-#===========================================================================
-#
-# * Denylist "variable font" Noto CJK fonts.
-# - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with
-# tweaks if necessary. Remove leading "# ".
-# - Path of fontconfig/fonts.conf can be overridden by setting an env
-# variable FONTS_CONF_DENY_VF.
-#
-# * Template:
-# -----------------------------------------------------------------
-# <?xml version="1.0"?>
-# <!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
-# <fontconfig>
-# <!--
-# Ignore variable-font glob (not to break xetex)
-# -->
-# <selectfont>
-# <rejectfont>
-# <!--
-# for Fedora
-# -->
-# <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
-# <!--
-# for openSUSE tumbleweed
-# -->
-# <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
-# </rejectfont>
-# </selectfont>
-# </fontconfig>
-# -----------------------------------------------------------------
-#
-# The denylisting is activated for "make pdfdocs".
-#
-# * For skipping CJK pages in PDF
-# - Uninstall texlive-xecjk.
-# Denylisting is not needed in this case.
-#
-# * For printing CJK pages in PDF
-# - Need non-variable "Noto CJK" fonts.
-# * Fedora
-# - google-noto-sans-cjk-fonts
-# - google-noto-serif-cjk-fonts
-# * openSUSE tumbleweed
-# - Non-variable "Noto CJK" fonts are not available as distro packages
-# as of April, 2024. Fetch a set of font files from upstream Noto
-# CJK Font released at:
-# https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc
-# and at:
-# https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc
-# , then uncompress and deploy them.
-# - Remember to update fontconfig cache by running fc-cache.
-#
-# !!! Caution !!!
-# Uninstalling "variable font" packages can be dangerous.
-# They might be depended upon by other packages important for your work.
-# Denylisting should be less invasive, as it is effective only while
-# XeLaTeX runs in "make pdfdocs".
-
-import os
-import re
-import subprocess
-import sys
-import textwrap
-
-class LatexFontChecker:
- """
- Detect problems with CJK variable fonts that affect PDF builds for
- translations.
- """
-
- def __init__(self):
- deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf")
-
- self.environ = os.environ.copy()
- self.environ['XDG_CONFIG_HOMEF'] = os.path.expanduser(deny_vf)
-
- self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK")
-
- def get_noto_cjk_vf_fonts(self):
- """Get Noto CJK fonts"""
-
- cjk_fonts = set()
- cmd = ["fc-list", ":", "file", "family", "variable"]
- try:
- result = subprocess.run(cmd,stdout=subprocess.PIPE,
- stderr=subprocess.PIPE,
- universal_newlines=True,
- env=self.environ,
- check=True)
-
- except subprocess.CalledProcessError as exc:
- sys.exit(f"Error running fc-list: {repr(exc)}")
-
- for line in result.stdout.splitlines():
- if 'variable=True' not in line:
- continue
-
- match = self.re_cjk.search(line)
- if match:
- cjk_fonts.add(match.group(1))
-
- return sorted(cjk_fonts)
-
- def check(self):
- """Check for problems with CJK fonts"""
-
- fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), " ")
- if not fonts:
- return None
-
- rel_file = os.path.relpath(__file__, os.getcwd())
-
- msg = "=" * 77 + "\n"
- msg += 'XeTeX is confused by "variable font" files listed below:\n'
- msg += fonts + "\n"
- msg += textwrap.dedent(f"""
- For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
- Or, CJK pages can be skipped by uninstalling texlive-xecjk.
-
- For more info on denylisting, other options, and variable font, see header
- comments of {rel_file}.
- """)
- msg += "=" * 77
-
- return msg
-
-if __name__ == "__main__":
- msg = LatexFontChecker().check()
- if msg:
- print(msg)
-
- sys.exit(1)
diff --git a/tools/docs/check-variable-fonts.py b/tools/docs/check-variable-fonts.py
new file mode 100755
index 000000000000..79b28f0f7d85
--- /dev/null
+++ b/tools/docs/check-variable-fonts.py
@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) Akira Yokosawa, 2024
+#
+# Ported to Python by (c) Mauro Carvalho Chehab, 2025
+#
+# pylint: disable=C0103
+
+"""
+Detect problematic Noto CJK variable fonts.
+
+or more details, see lib/latex_fonts.py.
+"""
+
+import sys
+
+from lib.latex_fonts import LatexFontChecker
+
+msg = LatexFontChecker().check()
+if msg:
+ print(msg)
+
+sys.exit(1)
diff --git a/tools/docs/lib/latex_fonts.py b/tools/docs/lib/latex_fonts.py
new file mode 100755
index 000000000000..e03412e3947e
--- /dev/null
+++ b/tools/docs/lib/latex_fonts.py
@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) Akira Yokosawa, 2024
+#
+# Ported to Python by (c) Mauro Carvalho Chehab, 2025
+
+"""
+Detect problematic Noto CJK variable fonts.
+
+For "make pdfdocs", reports of build errors of translations.pdf started
+arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE
+tumbleweed have started deploying variable-font [3] format of "Noto CJK"
+fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK
+(Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which
+does not (and likely never will) understand variable fonts for historical
+reasons.
+
+The build error happens even when both of variable- and non-variable-format
+fonts are found on the build system. To make matters worse, Fedora enlists
+variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN,
+-zh_TW, etc. Hence developers who have interest in CJK pages are more
+likely to encounter the build errors.
+
+This script is invoked from the error path of "make pdfdocs" and emits
+suggestions if variable-font files of "Noto CJK" fonts are in the list of
+fonts accessible from XeTeX.
+
+References:
+[1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
+[2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
+[3]: https://en.wikipedia.org/wiki/Variable_font
+[4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
+[5]: https://build.opensuse.org/request/show/1157217
+
+#===========================================================================
+Workarounds for building translations.pdf
+#===========================================================================
+
+* Denylist "variable font" Noto CJK fonts.
+ - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with
+ tweaks if necessary. Remove leading "".
+ - Path of fontconfig/fonts.conf can be overridden by setting an env
+ variable FONTS_CONF_DENY_VF.
+
+ * Template:
+-----------------------------------------------------------------
+<?xml version="1.0"?>
+<!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
+<fontconfig>
+<!--
+ Ignore variable-font glob (not to break xetex)
+-->
+ <selectfont>
+ <rejectfont>
+ <!--
+ for Fedora
+ -->
+ <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
+ <!--
+ for openSUSE tumbleweed
+ -->
+ <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
+ </rejectfont>
+ </selectfont>
+</fontconfig>
+-----------------------------------------------------------------
+
+ The denylisting is activated for "make pdfdocs".
+
+* For skipping CJK pages in PDF
+ - Uninstall texlive-xecjk.
+ Denylisting is not needed in this case.
+
+* For printing CJK pages in PDF
+ - Need non-variable "Noto CJK" fonts.
+ * Fedora
+ - google-noto-sans-cjk-fonts
+ - google-noto-serif-cjk-fonts
+ * openSUSE tumbleweed
+ - Non-variable "Noto CJK" fonts are not available as distro packages
+ as of April, 2024. Fetch a set of font files from upstream Noto
+ CJK Font released at:
+ https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc
+ and at:
+ https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc
+ , then uncompress and deploy them.
+ - Remember to update fontconfig cache by running fc-cache.
+
+!!! Caution !!!
+ Uninstalling "variable font" packages can be dangerous.
+ They might be depended upon by other packages important for your work.
+ Denylisting should be less invasive, as it is effective only while
+ XeLaTeX runs in "make pdfdocs".
+"""
+
+import os
+import re
+import subprocess
+import textwrap
+import sys
+
+class LatexFontChecker:
+ """
+ Detect problems with CJK variable fonts that affect PDF builds for
+ translations.
+ """
+
+ def __init__(self):
+ deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf")
+
+ self.environ = os.environ.copy()
+ self.environ['XDG_CONFIG_HOMEF'] = os.path.expanduser(deny_vf)
+
+ self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK")
+
+ def get_noto_cjk_vf_fonts(self):
+ """Get Noto CJK fonts"""
+
+ cjk_fonts = set()
+ cmd = ["fc-list", ":", "file", "family", "variable"]
+ try:
+ result = subprocess.run(cmd,stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ universal_newlines=True,
+ env=self.environ,
+ check=True)
+
+ except subprocess.CalledProcessError as exc:
+ sys.exit(f"Error running fc-list: {repr(exc)}")
+
+ for line in result.stdout.splitlines():
+ if 'variable=True' not in line:
+ continue
+
+ match = self.re_cjk.search(line)
+ if match:
+ cjk_fonts.add(match.group(1))
+
+ return sorted(cjk_fonts)
+
+ def check(self):
+ """Check for problems with CJK fonts"""
+
+ fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), " ")
+ if not fonts:
+ return None
+
+ rel_file = os.path.relpath(__file__, os.getcwd())
+
+ msg = "=" * 77 + "\n"
+ msg += 'XeTeX is confused by "variable font" files listed below:\n'
+ msg += fonts + "\n"
+ msg += textwrap.dedent(f"""
+ For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
+ Or, CJK pages can be skipped by uninstalling texlive-xecjk.
+
+ For more info on denylisting, other options, and variable font, see header
+ comments of {rel_file}.
+ """)
+ msg += "=" * 77
+
+ return msg
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 06/21] scripts: sphinx-pre-install: move it to tools/docs
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (4 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 05/21] tools/docs: check-variable-fonts.py: split into a lib and an exec file Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 07/21] tools/docs: python_version: move version check from sphinx-pre-install Mauro Carvalho Chehab
` (16 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Alex Shi, Dongliang Mu, Federico Vaga,
Mauro Carvalho Chehab, Randy Dunlap, Yanteng Si, linux-kernel
As we're reorganizing the place where doc scripts are located,
move this one to tools/docs.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 14 +++++++-------
Documentation/doc-guide/sphinx.rst | 4 ++--
Documentation/sphinx/kerneldoc-preamble.sty | 2 +-
.../translations/it_IT/doc-guide/sphinx.rst | 4 ++--
.../translations/zh_CN/doc-guide/sphinx.rst | 4 ++--
Documentation/translations/zh_CN/how-to.rst | 2 +-
MAINTAINERS | 3 +--
{scripts => tools/docs}/sphinx-pre-install | 0
8 files changed, 16 insertions(+), 17 deletions(-)
rename {scripts => tools/docs}/sphinx-pre-install (100%)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index b630d489b113..7570d4cf3b13 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -46,7 +46,7 @@ ifeq ($(HAVE_SPHINX),0)
.DEFAULT:
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
@echo
- @$(srctree)/scripts/sphinx-pre-install
+ @$(srctree)/tools/docs/sphinx-pre-install
@echo " SKIP Sphinx $@ target."
else # HAVE_SPHINX
@@ -105,7 +105,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
fi
htmldocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
# If Rust support is available and .config exists, add rustdoc generated contents.
@@ -119,7 +119,7 @@ endif
endif
texinfodocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
# Note: the 'info' Make target is generated by sphinx itself when
@@ -131,7 +131,7 @@ linkcheckdocs:
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
latexdocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
ifeq ($(HAVE_PDFLATEX),0)
@@ -144,7 +144,7 @@ else # HAVE_PDFLATEX
pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
pdfdocs: latexdocs
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
$(foreach var,$(SPHINXDIRS), \
$(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/tools/docs/check-variable-fonts.py || exit; \
mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -154,11 +154,11 @@ pdfdocs: latexdocs
endif # HAVE_PDFLATEX
epubdocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
xmldocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/tools/docs/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
endif # HAVE_SPHINX
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 607589592bfb..932f68c53075 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -106,7 +106,7 @@ There's a script that automatically checks for Sphinx dependencies. If it can
recognize your distribution, it will also give a hint about the install
command line options for your distro::
- $ ./scripts/sphinx-pre-install
+ $ ./tools/docs/sphinx-pre-install
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
Warning: better to also install "texlive-luatex85".
You should run:
@@ -116,7 +116,7 @@ command line options for your distro::
. sphinx_2.4.4/bin/activate
pip install -r Documentation/sphinx/requirements.txt
- Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+ Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
By default, it checks all the requirements for both html and PDF, including
the requirements for images, math expressions and LaTeX build, and assumes
diff --git a/Documentation/sphinx/kerneldoc-preamble.sty b/Documentation/sphinx/kerneldoc-preamble.sty
index 5d68395539fe..16d9ff46fdf6 100644
--- a/Documentation/sphinx/kerneldoc-preamble.sty
+++ b/Documentation/sphinx/kerneldoc-preamble.sty
@@ -220,7 +220,7 @@
If you want them, please install non-variable ``Noto Sans CJK''
font families along with the texlive-xecjk package by following
instructions from
- \sphinxcode{./scripts/sphinx-pre-install}.
+ \sphinxcode{./tools/docs/sphinx-pre-install}.
Having optional non-variable ``Noto Serif CJK'' font families will
improve the looks of those translations.
\end{sphinxadmonition}}
diff --git a/Documentation/translations/it_IT/doc-guide/sphinx.rst b/Documentation/translations/it_IT/doc-guide/sphinx.rst
index 1f513bc33618..a5c5d935febf 100644
--- a/Documentation/translations/it_IT/doc-guide/sphinx.rst
+++ b/Documentation/translations/it_IT/doc-guide/sphinx.rst
@@ -109,7 +109,7 @@ Sphinx. Se lo script riesce a riconoscere la vostra distribuzione, allora
sar� in grado di darvi dei suggerimenti su come procedere per completare
l'installazione::
- $ ./scripts/sphinx-pre-install
+ $ ./tools/docs/sphinx-pre-install
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
Warning: better to also install "texlive-luatex85".
You should run:
@@ -119,7 +119,7 @@ l'installazione::
. sphinx_2.4.4/bin/activate
pip install -r Documentation/sphinx/requirements.txt
- Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+ Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
L'impostazione predefinita prevede il controllo dei requisiti per la generazione
di documenti html e PDF, includendo anche il supporto per le immagini, le
diff --git a/Documentation/translations/zh_CN/doc-guide/sphinx.rst b/Documentation/translations/zh_CN/doc-guide/sphinx.rst
index 23eac67fbc30..3375c6f3a811 100644
--- a/Documentation/translations/zh_CN/doc-guide/sphinx.rst
+++ b/Documentation/translations/zh_CN/doc-guide/sphinx.rst
@@ -84,7 +84,7 @@ PDF\u548cLaTeX\u6784\u5efa
\u8fd9\u6709\u4e00\u4e2a\u811a\u672c\u53ef\u4ee5\u81ea\u52a8\u68c0\u67e5Sphinx\u4f9d\u8d56\u9879\u3002\u5982\u679c\u5b83\u8ba4\u5f97\u60a8\u7684\u53d1\u884c\u7248\uff0c\u8fd8\u4f1a\u63d0\u793a\u60a8\u6240\u7528\u53d1\u884c
\u7248\u7684\u5b89\u88c5\u547d\u4ee4::
- $ ./scripts/sphinx-pre-install
+ $ ./tools/docs/sphinx-pre-install
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
Warning: better to also install "texlive-luatex85".
You should run:
@@ -94,7 +94,7 @@ PDF\u548cLaTeX\u6784\u5efa
. sphinx_2.4.4/bin/activate
pip install -r Documentation/sphinx/requirements.txt
- Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+ Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468.
\u9ed8\u8ba4\u60c5\u51b5\u4e0b\uff0c\u5b83\u4f1a\u68c0\u67e5html\u548cPDF\u7684\u6240\u6709\u4f9d\u8d56\u9879\uff0c\u5305\u62ec\u56fe\u50cf\u3001\u6570\u5b66\u8868\u8fbe\u5f0f\u548cLaTeX\u6784\u5efa\u7684
\u9700\u6c42\uff0c\u5e76\u5047\u8bbe\u5c06\u4f7f\u7528\u865a\u62dfPython\u73af\u5883\u3002html\u6784\u5efa\u6240\u9700\u7684\u4f9d\u8d56\u9879\u88ab\u8ba4\u4e3a\u662f\u5fc5\u9700\u7684\uff0c\u5176\u4ed6\u4f9d
diff --git a/Documentation/translations/zh_CN/how-to.rst b/Documentation/translations/zh_CN/how-to.rst
index ddd99c0f9b4d..714664fec308 100644
--- a/Documentation/translations/zh_CN/how-to.rst
+++ b/Documentation/translations/zh_CN/how-to.rst
@@ -64,7 +64,7 @@ Linux \u53d1\u884c\u7248\u548c\u7b80\u5355\u5730\u4f7f\u7528 Linux \u547d\u4ee4\u884c\uff0c\u90a3\u4e48\u53ef\u4ee5\u8fc5\u901f\u5f00\u59cb\u4e86
::
cd linux
- ./scripts/sphinx-pre-install
+ ./tools/docs/sphinx-pre-install
\u4ee5 Fedora \u4e3a\u4f8b\uff0c\u5b83\u7684\u8f93\u51fa\u662f\u8fd9\u6837\u7684::
diff --git a/MAINTAINERS b/MAINTAINERS
index 76dd823bfcc4..16a5d6ab627d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7309,7 +7309,6 @@ F: scripts/lib/abi/*
F: scripts/lib/kdoc/*
F: tools/docs/*
F: tools/net/ynl/pyynl/lib/doc_generator.py
-F: scripts/sphinx-pre-install
X: Documentation/ABI/
X: Documentation/admin-guide/media/
X: Documentation/devicetree/
@@ -7344,7 +7343,7 @@ L: linux-doc@vger.kernel.org
S: Maintained
F: Documentation/sphinx/parse-headers.pl
F: scripts/documentation-file-ref-check
-F: scripts/sphinx-pre-install
+F: tools/docs/sphinx-pre-install
DOCUMENTATION/ITALIAN
M: Federico Vaga <federico.vaga@vaga.pv.it>
diff --git a/scripts/sphinx-pre-install b/tools/docs/sphinx-pre-install
similarity index 100%
rename from scripts/sphinx-pre-install
rename to tools/docs/sphinx-pre-install
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 07/21] tools/docs: python_version: move version check from sphinx-pre-install
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (5 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 06/21] scripts: sphinx-pre-install: move it to tools/docs Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 08/21] tools/docs: python_version: drop a debug print Mauro Carvalho Chehab
` (15 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
The sphinx-pre-install code has some logic to deal with Python
version, which ensures that a minimal version will be enforced
for documentation build logic.
Move it to a separate library to allow re-using its code.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/lib/python_version.py | 133 +++++++++++++++++++++++++++++++
tools/docs/sphinx-pre-install | 120 +++-------------------------
2 files changed, 146 insertions(+), 107 deletions(-)
create mode 100644 tools/docs/lib/python_version.py
diff --git a/tools/docs/lib/python_version.py b/tools/docs/lib/python_version.py
new file mode 100644
index 000000000000..0519d524e547
--- /dev/null
+++ b/tools/docs/lib/python_version.py
@@ -0,0 +1,133 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2017-2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
+
+"""
+Handle Python version check logic.
+
+Not all Python versions are supported by scripts. Yet, on some cases,
+like during documentation build, a newer version of python could be
+available.
+
+This class allows checking if the minimal requirements are followed.
+
+Better than that, PythonVersion.check_python() not only checks the minimal
+requirements, but it automatically switches to a the newest available
+Python version if present.
+
+"""
+
+import os
+import re
+import subprocess
+import sys
+
+from glob import glob
+
+class PythonVersion:
+ """
+ Ancillary methods that checks for missing dependencies for different
+ types of types, like binaries, python modules, rpm deps, etc.
+ """
+
+ def __init__(self, version):
+ """�nitialize self.version tuple from a version string"""
+ self.version = self.parse_version(version)
+
+ @staticmethod
+ def parse_version(version):
+ """Convert a major.minor.patch version into a tuple"""
+ return tuple(int(x) for x in version.split("."))
+
+ @staticmethod
+ def ver_str(version):
+ """Returns a version tuple as major.minor.patch"""
+ return ".".join([str(x) for x in version])
+
+ def __str__(self):
+ """Returns a version tuple as major.minor.patch from self.version"""
+ return self.ver_str(self.version)
+
+ @staticmethod
+ def get_python_version(cmd):
+ """
+ Get python version from a Python binary. As we need to detect if
+ are out there newer python binaries, we can't rely on sys.release here.
+ """
+
+ kwargs = {}
+ if sys.version_info < (3, 7):
+ kwargs['universal_newlines'] = True
+ else:
+ kwargs['text'] = True
+
+ result = subprocess.run([cmd, "--version"],
+ stdout = subprocess.PIPE,
+ stderr = subprocess.PIPE,
+ **kwargs, check=False)
+
+ version = result.stdout.strip()
+
+ match = re.search(r"(\d+\.\d+\.\d+)", version)
+ if match:
+ return PythonVersion.parse_version(match.group(1))
+
+ print(f"Can't parse version {version}")
+ return (0, 0, 0)
+
+ @staticmethod
+ def find_python(min_version):
+ """
+ Detect if are out there any python 3.xy version newer than the
+ current one.
+
+ Note: this routine is limited to up to 2 digits for python3. We
+ may need to update it one day, hopefully on a distant future.
+ """
+ patterns = [
+ "python3.[0-9]",
+ "python3.[0-9][0-9]",
+ ]
+
+ # Seek for a python binary newer than min_version
+ for path in os.getenv("PATH", "").split(":"):
+ for pattern in patterns:
+ for cmd in glob(os.path.join(path, pattern)):
+ if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
+ version = PythonVersion.get_python_version(cmd)
+ if version >= min_version:
+ return cmd
+
+ return None
+
+ @staticmethod
+ def check_python(min_version):
+ """
+ Check if the current python binary satisfies our minimal requirement
+ for Sphinx build. If not, re-run with a newer version if found.
+ """
+ cur_ver = sys.version_info[:3]
+ if cur_ver >= min_version:
+ ver = PythonVersion.ver_str(cur_ver)
+ print(f"Python version: {ver}")
+
+ return
+
+ python_ver = PythonVersion.ver_str(cur_ver)
+
+ new_python_cmd = PythonVersion.find_python(min_version)
+ if not new_python_cmd:
+ print(f"ERROR: Python version {python_ver} is not spported anymore\n")
+ print(" Can't find a new version. This script may fail")
+ return
+
+ # Restart script using the newer version
+ script_path = os.path.abspath(sys.argv[0])
+ args = [new_python_cmd, script_path] + sys.argv[1:]
+
+ print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
+
+ try:
+ os.execv(new_python_cmd, args)
+ except OSError as e:
+ sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
diff --git a/tools/docs/sphinx-pre-install b/tools/docs/sphinx-pre-install
index 954ed3dc0645..d6d673b7945c 100755
--- a/tools/docs/sphinx-pre-install
+++ b/tools/docs/sphinx-pre-install
@@ -32,20 +32,10 @@ import subprocess
import sys
from glob import glob
+from lib.python_version import PythonVersion
-def parse_version(version):
- """Convert a major.minor.patch version into a tuple"""
- return tuple(int(x) for x in version.split("."))
-
-
-def ver_str(version):
- """Returns a version tuple as major.minor.patch"""
-
- return ".".join([str(x) for x in version])
-
-
-RECOMMENDED_VERSION = parse_version("3.4.3")
-MIN_PYTHON_VERSION = parse_version("3.7")
+RECOMMENDED_VERSION = PythonVersion("3.4.3").version
+MIN_PYTHON_VERSION = PythonVersion("3.7").version
class DepManager:
@@ -235,95 +225,11 @@ class AncillaryMethods:
return None
- @staticmethod
- def get_python_version(cmd):
- """
- Get python version from a Python binary. As we need to detect if
- are out there newer python binaries, we can't rely on sys.release here.
- """
-
- result = SphinxDependencyChecker.run([cmd, "--version"],
- capture_output=True, text=True)
- version = result.stdout.strip()
-
- match = re.search(r"(\d+\.\d+\.\d+)", version)
- if match:
- return parse_version(match.group(1))
-
- print(f"Can't parse version {version}")
- return (0, 0, 0)
-
- @staticmethod
- def find_python():
- """
- Detect if are out there any python 3.xy version newer than the
- current one.
-
- Note: this routine is limited to up to 2 digits for python3. We
- may need to update it one day, hopefully on a distant future.
- """
- patterns = [
- "python3.[0-9]",
- "python3.[0-9][0-9]",
- ]
-
- # Seek for a python binary newer than MIN_PYTHON_VERSION
- for path in os.getenv("PATH", "").split(":"):
- for pattern in patterns:
- for cmd in glob(os.path.join(path, pattern)):
- if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
- version = SphinxDependencyChecker.get_python_version(cmd)
- if version >= MIN_PYTHON_VERSION:
- return cmd
-
- @staticmethod
- def check_python():
- """
- Check if the current python binary satisfies our minimal requirement
- for Sphinx build. If not, re-run with a newer version if found.
- """
- cur_ver = sys.version_info[:3]
- if cur_ver >= MIN_PYTHON_VERSION:
- ver = ver_str(cur_ver)
- print(f"Python version: {ver}")
-
- # This could be useful for debugging purposes
- if SphinxDependencyChecker.which("docutils"):
- result = SphinxDependencyChecker.run(["docutils", "--version"],
- capture_output=True, text=True)
- ver = result.stdout.strip()
- match = re.search(r"(\d+\.\d+\.\d+)", ver)
- if match:
- ver = match.group(1)
-
- print(f"Docutils version: {ver}")
-
- return
-
- python_ver = ver_str(cur_ver)
-
- new_python_cmd = SphinxDependencyChecker.find_python()
- if not new_python_cmd:
- print(f"ERROR: Python version {python_ver} is not spported anymore\n")
- print(" Can't find a new version. This script may fail")
- return
-
- # Restart script using the newer version
- script_path = os.path.abspath(sys.argv[0])
- args = [new_python_cmd, script_path] + sys.argv[1:]
-
- print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
-
- try:
- os.execv(new_python_cmd, args)
- except OSError as e:
- sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
-
@staticmethod
def run(*args, **kwargs):
"""
Excecute a command, hiding its output by default.
- Preserve comatibility with older Python versions.
+ Preserve compatibility with older Python versions.
"""
capture_output = kwargs.pop('capture_output', False)
@@ -527,11 +433,11 @@ class MissingCheckers(AncillaryMethods):
for line in result.stdout.split("\n"):
match = re.match(r"^sphinx-build\s+([\d\.]+)(?:\+(?:/[\da-f]+)|b\d+)?\s*$", line)
if match:
- return parse_version(match.group(1))
+ return PythonVersion.parse_version(match.group(1))
match = re.match(r"^Sphinx.*\s+([\d\.]+)\s*$", line)
if match:
- return parse_version(match.group(1))
+ return PythonVersion.parse_version(match.group(1))
def check_sphinx(self, conf):
"""
@@ -542,7 +448,7 @@ class MissingCheckers(AncillaryMethods):
for line in f:
match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line)
if match:
- self.min_version = parse_version(match.group(1))
+ self.min_version = PythonVersion.parse_version(match.group(1))
break
except IOError:
sys.exit(f"Can't open {conf}")
@@ -562,8 +468,8 @@ class MissingCheckers(AncillaryMethods):
sys.exit(f"{sphinx} didn't return its version")
if self.cur_version < self.min_version:
- curver = ver_str(self.cur_version)
- minver = ver_str(self.min_version)
+ curver = PythonVersion.ver_str(self.cur_version)
+ minver = PythonVersion.ver_str(self.min_version)
print(f"ERROR: Sphinx version is {curver}. It should be >= {minver}")
self.need_sphinx = 1
@@ -1304,7 +1210,7 @@ class SphinxDependencyChecker(MissingCheckers):
else:
if self.need_sphinx and ver >= self.min_version:
return (f, ver)
- elif parse_version(ver) > self.cur_version:
+ elif PythonVersion.parse_version(ver) > self.cur_version:
return (f, ver)
return ("", ver)
@@ -1411,7 +1317,7 @@ class SphinxDependencyChecker(MissingCheckers):
return
if self.latest_avail_ver:
- latest_avail_ver = ver_str(self.latest_avail_ver)
+ latest_avail_ver = PythonVersion.ver_str(self.latest_avail_ver)
if not self.need_sphinx:
# sphinx-build is present and its version is >= $min_version
@@ -1507,7 +1413,7 @@ class SphinxDependencyChecker(MissingCheckers):
else:
print("Unknown OS")
if self.cur_version != (0, 0, 0):
- ver = ver_str(self.cur_version)
+ ver = PythonVersion.ver_str(self.cur_version)
print(f"Sphinx version: {ver}\n")
# Check the type of virtual env, depending on Python version
@@ -1613,7 +1519,7 @@ def main():
checker = SphinxDependencyChecker(args)
- checker.check_python()
+ PythonVersion.check_python(MIN_PYTHON_VERSION)
checker.check_needs()
# Call main if not used as module
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 08/21] tools/docs: python_version: drop a debug print
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (6 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 07/21] tools/docs: python_version: move version check from sphinx-pre-install Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 09/21] tools/docs: python_version: allow check for alternatives and bail out Mauro Carvalho Chehab
` (14 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
The version print at the lib was added for debugging purposes.
Get rid of it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/lib/python_version.py | 2 --
1 file changed, 2 deletions(-)
diff --git a/tools/docs/lib/python_version.py b/tools/docs/lib/python_version.py
index 0519d524e547..660bfe7d23fa 100644
--- a/tools/docs/lib/python_version.py
+++ b/tools/docs/lib/python_version.py
@@ -109,8 +109,6 @@ class PythonVersion:
cur_ver = sys.version_info[:3]
if cur_ver >= min_version:
ver = PythonVersion.ver_str(cur_ver)
- print(f"Python version: {ver}")
-
return
python_ver = PythonVersion.ver_str(cur_ver)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 09/21] tools/docs: python_version: allow check for alternatives and bail out
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (7 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 08/21] tools/docs: python_version: drop a debug print Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build Mauro Carvalho Chehab
` (13 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
The caller script may not want an automatic execution of the new
version. Add two parameters to allow showing alternatives and to
bail out if version is incompatible.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/lib/python_version.py | 43 ++++++++++++++++++++++++++------
1 file changed, 35 insertions(+), 8 deletions(-)
diff --git a/tools/docs/lib/python_version.py b/tools/docs/lib/python_version.py
index 660bfe7d23fa..a9fda2470a26 100644
--- a/tools/docs/lib/python_version.py
+++ b/tools/docs/lib/python_version.py
@@ -85,10 +85,12 @@ class PythonVersion:
may need to update it one day, hopefully on a distant future.
"""
patterns = [
- "python3.[0-9]",
"python3.[0-9][0-9]",
+ "python3.[0-9]",
]
+ python_cmd = []
+
# Seek for a python binary newer than min_version
for path in os.getenv("PATH", "").split(":"):
for pattern in patterns:
@@ -96,12 +98,13 @@ class PythonVersion:
if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
version = PythonVersion.get_python_version(cmd)
if version >= min_version:
- return cmd
+ python_cmd.append((version, cmd))
- return None
+ return sorted(python_cmd, reverse=True)
@staticmethod
- def check_python(min_version):
+ def check_python(min_version, show_alternatives=False, bail_out=False,
+ success_on_error=False):
"""
Check if the current python binary satisfies our minimal requirement
for Sphinx build. If not, re-run with a newer version if found.
@@ -113,18 +116,42 @@ class PythonVersion:
python_ver = PythonVersion.ver_str(cur_ver)
- new_python_cmd = PythonVersion.find_python(min_version)
- if not new_python_cmd:
+ available_versions = PythonVersion.find_python(min_version)
+ if not available_versions:
print(f"ERROR: Python version {python_ver} is not spported anymore\n")
print(" Can't find a new version. This script may fail")
return
- # Restart script using the newer version
script_path = os.path.abspath(sys.argv[0])
- args = [new_python_cmd, script_path] + sys.argv[1:]
+
+ # Check possible alternatives
+ if available_versions:
+ new_python_cmd = available_versions[0][1]
+ else:
+ new_python_cmd = None
+
+ if show_alternatives:
+ print("You could run, instead:")
+ for _, cmd in available_versions:
+ args = [cmd, script_path] + sys.argv[1:]
+
+ cmd_str = " ".join(args)
+ print(f" {cmd_str}")
+ print()
+
+ if bail_out:
+ msg = f"Python {python_ver} not supported. Bailing out"
+ if success_on_error:
+ print(msg, file=sys.stderr)
+ sys.exit(0)
+ else:
+ sys.exit(msg)
print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
+ # Restart script using the newer version
+ args = [new_python_cmd, script_path] + sys.argv[1:]
+
try:
os.execv(new_python_cmd, args)
except OSError as e:
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (8 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 09/21] tools/docs: python_version: allow check for alternatives and bail out Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-17 8:35 ` Akira Yokosawa
2025-09-16 10:22 ` [PATCH v6 11/21] docs: parallel-wrapper.sh: remove script Mauro Carvalho Chehab
` (12 subsequent siblings)
22 siblings, 1 reply; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Björn Roy Baron, Alex Gaynor,
Alice Ryhl, Andreas Hindborg, Benno Lossin, Boqun Feng,
Danilo Krummrich, Gary Guo, Mauro Carvalho Chehab, Miguel Ojeda,
Trevor Gross, linux-kernel, rust-for-linux
There are too much magic inside docs Makefile to properly run
sphinx-build. Create an ancillary script that contains all
kernel-related sphinx-build call logic currently at Makefile.
Such script is designed to work both as an standalone command
and as part of a Makefile. As such, it properly handles POSIX
jobserver used by GNU make.
On a side note, there was a line number increase due to the
conversion (ignoring comments) is:
Documentation/Makefile | 131 +++----------
tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++
2 files changed, 323 insertions(+), 101 deletions(-)
Comments and descriptions adds:
tools/docs/sphinx-build-wrapper | 261 +++++++++++++++++++++++++++++++-
So, about half of the script are comments/descriptions.
This is because some things are more verbosed on Python and because
it requires reading env vars from Makefile. Besides it, this script
has some extra features that don't exist at the Makefile:
- It can be called directly from command line;
- It properly return PDF build errors.
When running the script alone, it will only take handle sphinx-build
targets. On other words, it won't runn make rustdoc after building
htmlfiles, nor it will run the extra check scripts.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 131 ++-----
tools/docs/sphinx-build-wrapper | 581 ++++++++++++++++++++++++++++++++
2 files changed, 611 insertions(+), 101 deletions(-)
create mode 100755 tools/docs/sphinx-build-wrapper
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 7570d4cf3b13..4736f02b6c9e 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -23,21 +23,22 @@ SPHINXOPTS =
SPHINXDIRS = .
DOCS_THEME =
DOCS_CSS =
-_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
SPHINX_CONF = conf.py
PAPER =
BUILDDIR = $(obj)/output
PDFLATEX = xelatex
LATEXOPTS = -interaction=batchmode -no-shell-escape
+PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__)
+
+# Wrapper for sphinx-build
+
+BUILD_WRAPPER = $(srctree)/tools/docs/sphinx-build-wrapper
+
# For denylisting "variable font" files
# Can be overridden by setting as an env variable
FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf
-ifeq ($(findstring 1, $(KBUILD_VERBOSE)),)
-SPHINXOPTS += "-q"
-endif
-
# User-friendly check for sphinx-build
HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi)
@@ -51,63 +52,31 @@ ifeq ($(HAVE_SPHINX),0)
else # HAVE_SPHINX
-# User-friendly check for pdflatex and latexmk
-HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
-HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
+# Common documentation targets
+infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
+ $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
+ +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
+ --sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \
+ --builddir="$(BUILDDIR)" \
+ --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
-ifeq ($(HAVE_LATEXMK),1)
- PDFLATEX := latexmk -$(PDFLATEX)
-endif #HAVE_LATEXMK
-
-# Internal variables.
-PAPEROPT_a4 = -D latex_elements.papersize=a4paper
-PAPEROPT_letter = -D latex_elements.papersize=letterpaper
-ALLSPHINXOPTS = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
-ALLSPHINXOPTS += $(PAPEROPT_$(PAPER)) $(SPHINXOPTS)
-ifneq ($(wildcard $(srctree)/.config),)
-ifeq ($(CONFIG_RUST),y)
- # Let Sphinx know we will include rustdoc
- ALLSPHINXOPTS += -t rustdoc
-endif
+# Special handling for pdfdocs
+ifeq ($(shell which $(PDFLATEX) >/dev/null 2>&1; echo $$?),0)
+pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
+else
+pdfdocs:
+ $(warning The '$(PDFLATEX)' command was not found. Make sure you have it installed and in PATH to produce PDF output.)
+ @echo " SKIP Sphinx $@ target."
endif
-# the i18n builder cannot share the environment and doctrees with the others
-I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
-
-# commands; the 'cmd' from scripts/Kbuild.include is not *loopable*
-loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
-
-# $2 sphinx builder e.g. "html"
-# $3 name of the build subfolder / e.g. "userspace-api/media", used as:
-# * dest folder relative to $(BUILDDIR) and
-# * cache folder relative to $(BUILDDIR)/.doctrees
-# $4 dest subfolder e.g. "man" for man pages at userspace-api/media/man
-# $5 reST source folder relative to $(src),
-# e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media
-
-PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__)
-
-quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
- cmd_sphinx = \
- PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \
- BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(src)/$5/$(SPHINX_CONF)) \
- $(PYTHON3) $(srctree)/scripts/jobserver-exec \
- $(CONFIG_SHELL) $(srctree)/Documentation/sphinx/parallel-wrapper.sh \
- $(SPHINXBUILD) \
- -b $2 \
- -c $(abspath $(src)) \
- -d $(abspath $(BUILDDIR)/.doctrees/$3) \
- -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) \
- $(ALLSPHINXOPTS) \
- $(abspath $(src)/$5) \
- $(abspath $(BUILDDIR)/$3/$4) && \
- if [ "x$(DOCS_CSS)" != "x" ]; then \
- cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_static/; \
- fi
+# HTML main logic is identical to other targets. However, if rust is enabled,
+# an extra step at the end is required to generate rustdoc.
htmldocs:
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
-
+ $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
+ +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
+ --sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \
+ --builddir="$(BUILDDIR)" \
+ --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
# If Rust support is available and .config exists, add rustdoc generated contents.
# If there are any, the errors from this make rustdoc will be displayed but
# won't stop the execution of htmldocs
@@ -118,49 +87,6 @@ ifeq ($(CONFIG_RUST),y)
endif
endif
-texinfodocs:
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
-
-# Note: the 'info' Make target is generated by sphinx itself when
-# running the texinfodocs target define above.
-infodocs: texinfodocs
- $(MAKE) -C $(BUILDDIR)/texinfo info
-
-linkcheckdocs:
- @$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
-
-latexdocs:
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
-
-ifeq ($(HAVE_PDFLATEX),0)
-
-pdfdocs:
- $(warning The '$(PDFLATEX)' command was not found. Make sure you have it installed and in PATH to produce PDF output.)
- @echo " SKIP Sphinx $@ target."
-
-else # HAVE_PDFLATEX
-
-pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
-pdfdocs: latexdocs
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- $(foreach var,$(SPHINXDIRS), \
- $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/tools/docs/check-variable-fonts.py || exit; \
- mkdir -p $(BUILDDIR)/$(var)/pdf; \
- mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \
- )
-
-endif # HAVE_PDFLATEX
-
-epubdocs:
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
-
-xmldocs:
- @$(srctree)/tools/docs/sphinx-pre-install --version-check
- @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
-
endif # HAVE_SPHINX
# The following targets are independent of HAVE_SPHINX, and the rules should
@@ -172,6 +98,9 @@ refcheckdocs:
cleandocs:
$(Q)rm -rf $(BUILDDIR)
+# Used only on help
+_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
+
dochelp:
@echo ' Linux kernel internal documentation in different formats from ReST:'
@echo ' htmldocs - HTML'
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
new file mode 100755
index 000000000000..a65a2297eb40
--- /dev/null
+++ b/tools/docs/sphinx-build-wrapper
@@ -0,0 +1,581 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
+#
+# pylint: disable=R0902, R0912, R0913, R0914, R0915, R0917, C0103
+#
+# Converted from docs Makefile and parallel-wrapper.sh, both under
+# GPLv2, copyrighted since 2008 by the following authors:
+#
+# Akira Yokosawa <akiyks@gmail.com>
+# Arnd Bergmann <arnd@arndb.de>
+# Breno Leitao <leitao@debian.org>
+# Carlos Bilbao <carlos.bilbao@amd.com>
+# Dave Young <dyoung@redhat.com>
+# Donald Hunter <donald.hunter@gmail.com>
+# Geert Uytterhoeven <geert+renesas@glider.be>
+# Jani Nikula <jani.nikula@intel.com>
+# Jan Stancek <jstancek@redhat.com>
+# Jonathan Corbet <corbet@lwn.net>
+# Joshua Clayton <stillcompiling@gmail.com>
+# Kees Cook <keescook@chromium.org>
+# Linus Torvalds <torvalds@linux-foundation.org>
+# Magnus Damm <damm+renesas@opensource.se>
+# Masahiro Yamada <masahiroy@kernel.org>
+# Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
+# Maxim Cournoyer <maxim.cournoyer@gmail.com>
+# Peter Foley <pefoley2@pefoley.com>
+# Randy Dunlap <rdunlap@infradead.org>
+# Rob Herring <robh@kernel.org>
+# Shuah Khan <shuahkh@osg.samsung.com>
+# Thorsten Blum <thorsten.blum@toblux.com>
+# Tomas Winkler <tomas.winkler@intel.com>
+
+
+"""
+Sphinx build wrapper that handles Kernel-specific business rules:
+
+- it gets the Kernel build environment vars;
+- it determines what's the best parallelism;
+- it handles SPHINXDIRS
+
+This tool ensures that MIN_PYTHON_VERSION is satisfied. If version is
+below that, it seeks for a new Python version. If found, it re-runs using
+the newer version.
+"""
+
+import argparse
+import os
+import shlex
+import shutil
+import subprocess
+import sys
+
+from lib.python_version import PythonVersion
+from lib.latex_fonts import LatexFontChecker
+
+LIB_DIR = "../../scripts/lib"
+SRC_DIR = os.path.dirname(os.path.realpath(__file__))
+
+sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
+
+from jobserver import JobserverExec # pylint: disable=C0413,C0411,E0401
+
+#
+# Some constants
+#
+MIN_PYTHON_VERSION = PythonVersion("3.7").version
+PAPER = ["", "a4", "letter"]
+
+TARGETS = {
+ "cleandocs": { "builder": "clean" },
+ "linkcheckdocs": { "builder": "linkcheck" },
+ "htmldocs": { "builder": "html" },
+ "epubdocs": { "builder": "epub", "out_dir": "epub" },
+ "texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" },
+ "infodocs": { "builder": "texinfo", "out_dir": "texinfo" },
+ "latexdocs": { "builder": "latex", "out_dir": "latex" },
+ "pdfdocs": { "builder": "latex", "out_dir": "latex" },
+ "xmldocs": { "builder": "xml", "out_dir": "xml" },
+}
+
+
+#
+# SphinxBuilder class
+#
+
+class SphinxBuilder:
+ """
+ Handles a sphinx-build target, adding needed arguments to build
+ with the Kernel.
+ """
+
+ def is_rust_enabled(self):
+ """Check if rust is enabled at .config"""
+ config_path = os.path.join(self.srctree, ".config")
+ if os.path.isfile(config_path):
+ with open(config_path, "r", encoding="utf-8") as f:
+ return "CONFIG_RUST=y" in f.read()
+ return False
+
+ def get_path(self, path, use_cwd=False, abs_path=False):
+ """
+ Ancillary routine to handle patches the right way, as shell does.
+
+ It first expands "~" and "~user". Then, if patch is not absolute,
+ join self.srctree. Finally, if requested, convert to abspath.
+ """
+
+ path = os.path.expanduser(path)
+ if not path.startswith("/"):
+ if use_cwd:
+ base = os.getcwd()
+ else:
+ base = self.srctree
+
+ path = os.path.join(base, path)
+
+ if abs_path:
+ return os.path.abspath(path)
+
+ return path
+
+ def get_sphinx_extra_opts(self, n_jobs):
+ """
+ Get the number of jobs to be used for docs build passed via command
+ line and desired sphinx verbosity.
+
+ The number of jobs can be on different places:
+
+ 1) It can be passed via "-j" argument;
+ 2) The SPHINXOPTS="-j8" env var may have "-j";
+ 3) if called via GNU make, -j specifies the desired number of jobs.
+ with GNU makefile, this number is available via POSIX jobserver;
+ 4) if none of the above is available, it should default to "-jauto",
+ and let sphinx decide the best value.
+ """
+
+ #
+ # SPHINXOPTS env var, if used, contains extra arguments to be used
+ # by sphinx-build time. Among them, it may contain sphinx verbosity
+ # and desired number of parallel jobs.
+ #
+ parser = argparse.ArgumentParser()
+ parser.add_argument('-j', '--jobs', type=int)
+ parser.add_argument('-q', '--quiet', type=int)
+
+ #
+ # Other sphinx-build arguments go as-is, so place them
+ # at self.sphinxopts, using shell parser
+ #
+ sphinxopts = shlex.split(os.environ.get("SPHINXOPTS", ""))
+
+ #
+ # Build a list of sphinx args, honoring verbosity here if specified
+ #
+
+ verbose = self.verbose
+ sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts)
+ if sphinx_args.quiet is True:
+ verbose = False
+
+ #
+ # If the user explicitly sets "-j" at command line, use it.
+ # Otherwise, pick it from SPHINXOPTS args
+ #
+ if n_jobs:
+ self.n_jobs = n_jobs
+ elif sphinx_args.jobs:
+ self.n_jobs = sphinx_args.jobs
+ else:
+ self.n_jobs = None
+
+ if not verbose:
+ self.sphinxopts += ["-q"]
+
+ def __init__(self, builddir, verbose=False, n_jobs=None):
+ """Initialize internal variables"""
+ self.verbose = None
+
+ #
+ # Normal variables passed from Kernel's makefile
+ #
+ self.kernelversion = os.environ.get("KERNELVERSION", "unknown")
+ self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown")
+ self.pdflatex = os.environ.get("PDFLATEX", "xelatex")
+ self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape")
+
+ if not verbose:
+ verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "")
+
+ if verbose is not None:
+ self.verbose = verbose
+
+ #
+ # Source tree directory. This needs to be at os.environ, as
+ # Sphinx extensions use it
+ #
+ self.srctree = os.environ.get("srctree")
+ if not self.srctree:
+ self.srctree = "."
+ os.environ["srctree"] = self.srctree
+
+ #
+ # Now that we can expand srctree, get other directories as well
+ #
+ self.sphinxbuild = os.environ.get("SPHINXBUILD", "sphinx-build")
+ self.kerneldoc = self.get_path(os.environ.get("KERNELDOC",
+ "scripts/kernel-doc.py"))
+ self.builddir = self.get_path(builddir, use_cwd=True, abs_path=True)
+
+ self.config_rust = self.is_rust_enabled()
+
+ #
+ # Get directory locations for LaTeX build toolchain
+ #
+ self.pdflatex_cmd = shutil.which(self.pdflatex)
+ self.latexmk_cmd = shutil.which("latexmk")
+
+ self.env = os.environ.copy()
+
+ self.get_sphinx_extra_opts(n_jobs)
+
+ def run_sphinx(self, sphinx_build, build_args, *args, **pwargs):
+ """
+ Executes sphinx-build using current python3 command.
+
+ When calling via GNU make, POSIX jobserver is used to tell how
+ many jobs are still available from a job pool. claim all remaining
+ jobs, as we don't want sphinx-build to run in parallel with other
+ jobs.
+
+ Despite that, the user may actually force a different value than
+ the number of available jobs via command line.
+
+ The "with" logic here is used to ensure that the claimed jobs will
+ be freed once subprocess finishes
+ """
+
+ with JobserverExec() as jobserver:
+ if jobserver.claim:
+ #
+ # when GNU make is used, claim available jobs from jobserver
+ #
+ n_jobs = str(jobserver.claim)
+ else:
+ #
+ # Otherwise, let sphinx decide by default
+ #
+ n_jobs = "auto"
+
+ #
+ # If explicitly requested via command line, override default
+ #
+ if self.n_jobs:
+ n_jobs = str(self.n_jobs)
+
+ cmd = [sys.executable, sphinx_build]
+ cmd += [f"-j{n_jobs}"]
+ cmd += self.sphinxopts
+ cmd += build_args
+
+ if self.verbose:
+ print(" ".join(cmd))
+
+ return subprocess.call(cmd, *args, **pwargs)
+
+ def handle_html(self, css, output_dir):
+ """
+ Extra steps for HTML and epub output.
+
+ For such targets, we need to ensure that CSS will be properly
+ copied to the output _static directory
+ """
+
+ if not css:
+ return
+
+ css = os.path.expanduser(css)
+ if not css.startswith("/"):
+ css = os.path.join(self.srctree, css)
+
+ static_dir = os.path.join(output_dir, "_static")
+ os.makedirs(static_dir, exist_ok=True)
+
+ try:
+ shutil.copy2(css, static_dir)
+ except (OSError, IOError) as e:
+ print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr)
+
+ def handle_pdf(self, output_dirs):
+ """
+ Extra steps for PDF output.
+
+ As PDF is handled via a LaTeX output, after building the .tex file,
+ a new build is needed to create the PDF output from the latex
+ directory.
+ """
+ builds = {}
+ max_len = 0
+
+ for from_dir in output_dirs:
+ pdf_dir = os.path.join(from_dir, "../pdf")
+ os.makedirs(pdf_dir, exist_ok=True)
+
+ if self.latexmk_cmd:
+ latex_cmd = [self.latexmk_cmd, f"-{self.pdflatex}"]
+ else:
+ latex_cmd = [self.pdflatex]
+
+ latex_cmd.extend(shlex.split(self.latexopts))
+
+ tex_suffix = ".tex"
+
+ #
+ # Process each .tex file
+ #
+
+ has_tex = False
+ build_failed = False
+ with os.scandir(from_dir) as it:
+ for entry in it:
+ if not entry.name.endswith(tex_suffix):
+ continue
+
+ name = entry.name[:-len(tex_suffix)]
+ has_tex = True
+
+ #
+ # LaTeX PDF error code is almost useless for us:
+ # any warning makes it non-zero. For kernel doc builds it
+ # always return non-zero even when build succeeds.
+ # So, let's do the best next thing: check if all PDF
+ # files were built. If they're, print a summary and
+ # return 0 at the end of this function
+ #
+ try:
+ subprocess.run(latex_cmd + [entry.path],
+ cwd=from_dir, check=True)
+ except subprocess.CalledProcessError:
+ pass
+
+ pdf_name = name + ".pdf"
+ pdf_from = os.path.join(from_dir, pdf_name)
+ pdf_to = os.path.join(pdf_dir, pdf_name)
+
+ if os.path.exists(pdf_from):
+ os.rename(pdf_from, pdf_to)
+ builds[name] = os.path.relpath(pdf_to, self.builddir)
+ else:
+ builds[name] = "FAILED"
+ build_failed = True
+
+ name = entry.name.removesuffix(".tex")
+ max_len = max(max_len, len(name))
+
+ if not has_tex:
+ name = os.path.basename(from_dir)
+ max_len = max(max_len, len(name))
+ builds[name] = "FAILED (no .tex)"
+ build_failed = True
+
+ msg = "Summary"
+ msg += "\n" + "=" * len(msg)
+ print()
+ print(msg)
+
+ for pdf_name, pdf_file in builds.items():
+ print(f"{pdf_name:<{max_len}}: {pdf_file}")
+
+ print()
+
+ if build_failed:
+ msg = LatexFontChecker().check()
+ if msg:
+ print(msg)
+
+ sys.exit("PDF build failed: not all PDF files were created.")
+ else:
+ print("All PDF files were built.")
+
+ def handle_info(self, output_dirs):
+ """
+ Extra steps for Info output.
+
+ For texinfo generation, an additional make is needed from the
+ texinfo directory.
+ """
+
+ for output_dir in output_dirs:
+ try:
+ subprocess.run(["make", "info"], cwd=output_dir, check=True)
+ except subprocess.CalledProcessError as e:
+ sys.exit(f"Error generating info docs: {e}")
+
+ def cleandocs(self, builder): # pylint: disable=W0613
+ """Remove documentation output directory"""
+ shutil.rmtree(self.builddir, ignore_errors=True)
+
+ def build(self, target, sphinxdirs=None, conf="conf.py",
+ theme=None, css=None, paper=None):
+ """
+ Build documentation using Sphinx. This is the core function of this
+ module. It prepares all arguments required by sphinx-build.
+ """
+
+ builder = TARGETS[target]["builder"]
+ out_dir = TARGETS[target].get("out_dir", "")
+
+ #
+ # Cleandocs doesn't require sphinx-build
+ #
+ if target == "cleandocs":
+ self.cleandocs(builder)
+ return
+
+ if theme:
+ os.environ["DOCS_THEME"] = theme
+
+ #
+ # Other targets require sphinx-build, so check if it exists
+ #
+ sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"])
+ if not sphinxbuild:
+ sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n")
+
+ if builder == "latex":
+ if not self.pdflatex_cmd and not self.latexmk_cmd:
+ sys.exit("Error: pdflatex or latexmk required for PDF generation")
+
+ docs_dir = os.path.abspath(os.path.join(self.srctree, "Documentation"))
+
+ #
+ # Fill in base arguments for Sphinx build
+ #
+ kerneldoc = self.kerneldoc
+ if kerneldoc.startswith(self.srctree):
+ kerneldoc = os.path.relpath(kerneldoc, self.srctree)
+
+ args = [ "-b", builder, "-c", docs_dir ]
+
+ if builder == "latex":
+ if not paper:
+ paper = PAPER[1]
+
+ args.extend(["-D", f"latex_elements.papersize={paper}paper"])
+
+ if self.config_rust:
+ args.extend(["-t", "rustdoc"])
+
+ if conf:
+ self.env["SPHINX_CONF"] = self.get_path(conf, abs_path=True)
+
+ if not sphinxdirs:
+ sphinxdirs = os.environ.get("SPHINXDIRS", ".")
+
+ #
+ # sphinxdirs can be a list or a whitespace-separated string
+ #
+ sphinxdirs_list = []
+ for sphinxdir in sphinxdirs:
+ if isinstance(sphinxdir, list):
+ sphinxdirs_list += sphinxdir
+ else:
+ sphinxdirs_list += sphinxdir.split()
+
+ #
+ # Step 1: Build each directory in separate.
+ #
+ # This is not the best way of handling it, as cross-references between
+ # them will be broken, but this is what we've been doing since
+ # the beginning.
+ #
+ output_dirs = []
+ for sphinxdir in sphinxdirs_list:
+ src_dir = os.path.join(docs_dir, sphinxdir)
+ doctree_dir = os.path.join(self.builddir, ".doctrees")
+ output_dir = os.path.join(self.builddir, sphinxdir, out_dir)
+
+ #
+ # Make directory names canonical
+ #
+ src_dir = os.path.normpath(src_dir)
+ doctree_dir = os.path.normpath(doctree_dir)
+ output_dir = os.path.normpath(output_dir)
+
+ os.makedirs(doctree_dir, exist_ok=True)
+ os.makedirs(output_dir, exist_ok=True)
+
+ output_dirs.append(output_dir)
+
+ build_args = args + [
+ "-d", doctree_dir,
+ "-D", f"kerneldoc_bin={kerneldoc}",
+ "-D", f"version={self.kernelversion}",
+ "-D", f"release={self.kernelrelease}",
+ "-D", f"kerneldoc_srctree={self.srctree}",
+ src_dir,
+ output_dir,
+ ]
+
+ try:
+ self.run_sphinx(sphinxbuild, build_args, env=self.env)
+ except (OSError, ValueError, subprocess.SubprocessError) as e:
+ sys.exit(f"Build failed: {repr(e)}")
+
+ #
+ # Ensure that each html/epub output will have needed static files
+ #
+ if target in ["htmldocs", "epubdocs"]:
+ self.handle_html(css, output_dir)
+
+ #
+ # Step 2: Some targets (PDF and info) require an extra step once
+ # sphinx-build finishes
+ #
+ if target == "pdfdocs":
+ self.handle_pdf(output_dirs)
+ elif target == "infodocs":
+ self.handle_info(output_dirs)
+
+def jobs_type(value):
+ """
+ Handle valid values for -j. Accepts Sphinx "-jauto", plus a number
+ equal or bigger than one.
+ """
+ if value is None:
+ return None
+
+ if value.lower() == 'auto':
+ return value.lower()
+
+ try:
+ if int(value) >= 1:
+ return value
+
+ raise argparse.ArgumentTypeError(f"Minimum jobs is 1, got {value}")
+ except ValueError:
+ raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}") # pylint: disable=W0707
+
+def main():
+ """
+ Main function. The only mandatory argument is the target. If not
+ specified, the other arguments will use default values if not
+ specified at os.environ.
+ """
+ parser = argparse.ArgumentParser(description="Kernel documentation builder")
+
+ parser.add_argument("target", choices=list(TARGETS.keys()),
+ help="Documentation target to build")
+ parser.add_argument("--sphinxdirs", nargs="+",
+ help="Specific directories to build")
+ parser.add_argument("--conf", default="conf.py",
+ help="Sphinx configuration file")
+ parser.add_argument("--builddir", default="output",
+ help="Sphinx configuration file")
+
+ parser.add_argument("--theme", help="Sphinx theme to use")
+
+ parser.add_argument("--css", help="Custom CSS file for HTML/EPUB")
+
+ parser.add_argument("--paper", choices=PAPER, default=PAPER[0],
+ help="Paper size for LaTeX/PDF output")
+
+ parser.add_argument("-v", "--verbose", action='store_true',
+ help="place build in verbose mode")
+
+ parser.add_argument('-j', '--jobs', type=jobs_type,
+ help="Sets number of jobs to use with sphinx-build")
+
+ args = parser.parse_args()
+
+ PythonVersion.check_python(MIN_PYTHON_VERSION)
+
+ builder = SphinxBuilder(builddir=args.builddir,
+ verbose=args.verbose, n_jobs=args.jobs)
+
+ builder.build(args.target, sphinxdirs=args.sphinxdirs, conf=args.conf,
+ theme=args.theme, css=args.css, paper=args.paper)
+
+if __name__ == "__main__":
+ main()
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 11/21] docs: parallel-wrapper.sh: remove script
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (9 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 12/21] docs: Makefile: document latex/PDF PAPER= parameter Mauro Carvalho Chehab
` (11 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
The only usage of this script was docs Makefile. Now that
it is using the new sphinx-build-wrapper, which has inside
the code from parallel-wrapper.sh, we can drop this script.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/parallel-wrapper.sh | 33 ------------------------
1 file changed, 33 deletions(-)
delete mode 100644 Documentation/sphinx/parallel-wrapper.sh
diff --git a/Documentation/sphinx/parallel-wrapper.sh b/Documentation/sphinx/parallel-wrapper.sh
deleted file mode 100644
index e54c44ce117d..000000000000
--- a/Documentation/sphinx/parallel-wrapper.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0+
-#
-# Figure out if we should follow a specific parallelism from the make
-# environment (as exported by scripts/jobserver-exec), or fall back to
-# the "auto" parallelism when "-jN" is not specified at the top-level
-# "make" invocation.
-
-sphinx="$1"
-shift || true
-
-parallel="$PARALLELISM"
-if [ -z "$parallel" ] ; then
- # If no parallelism is specified at the top-level make, then
- # fall back to the expected "-jauto" mode that the "htmldocs"
- # target has had.
- auto=$(perl -e 'open IN,"'"$sphinx"' --version 2>&1 |";
- while (<IN>) {
- if (m/([\d\.]+)/) {
- print "auto" if ($1 >= "1.7")
- }
- }
- close IN')
- if [ -n "$auto" ] ; then
- parallel="$auto"
- fi
-fi
-# Only if some parallelism has been determined do we add the -jN option.
-if [ -n "$parallel" ] ; then
- parallel="-j$parallel"
-fi
-
-exec "$sphinx" $parallel "$@"
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 12/21] docs: Makefile: document latex/PDF PAPER= parameter
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (10 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 11/21] docs: parallel-wrapper.sh: remove script Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 13/21] tools/docs: sphinx-build-wrapper: add an argument for LaTeX interactive mode Mauro Carvalho Chehab
` (10 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
While the build system supports this for a long time, this was
never documented. Add a documentation for it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 4736f02b6c9e..0e1d8657a5cc 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -126,4 +126,6 @@ dochelp:
@echo
@echo ' make DOCS_CSS={a .css file} adds a DOCS_CSS override file for html/epub output.'
@echo
+ @echo ' make PAPER={a4|letter} Specifies the paper size used for LaTeX/PDF output.'
+ @echo
@echo ' Default location for the generated documents is Documentation/output'
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 13/21] tools/docs: sphinx-build-wrapper: add an argument for LaTeX interactive mode
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (11 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 12/21] docs: Makefile: document latex/PDF PAPER= parameter Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 14/21] tools/docs,scripts: sphinx-*: prevent sphinx-build crashes Mauro Carvalho Chehab
` (9 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
By default, we use LaTeX batch mode to build docs. This way, when
an error happens, the build fails. This is good for normal builds,
but when debugging problems with pdf generation, the best is to
use interactive mode.
We already support it via LATEXOPTS, but having a command line
argument makes it easier:
Interactive mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v -i
...
Running 'xelatex --no-pdf -no-pdf -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Default batch mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v
...
Running 'xelatex --no-pdf -no-pdf -interaction=batchmode -no-shell-escape -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/sphinx-build-wrapper | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index a65a2297eb40..413f51575de8 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -173,7 +173,7 @@ class SphinxBuilder:
if not verbose:
self.sphinxopts += ["-q"]
- def __init__(self, builddir, verbose=False, n_jobs=None):
+ def __init__(self, builddir, verbose=False, n_jobs=None, interactive=None):
"""Initialize internal variables"""
self.verbose = None
@@ -183,7 +183,11 @@ class SphinxBuilder:
self.kernelversion = os.environ.get("KERNELVERSION", "unknown")
self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown")
self.pdflatex = os.environ.get("PDFLATEX", "xelatex")
- self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape")
+
+ if not interactive:
+ self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape")
+ else:
+ self.latexopts = os.environ.get("LATEXOPTS", "")
if not verbose:
verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "")
@@ -567,12 +571,16 @@ def main():
parser.add_argument('-j', '--jobs', type=jobs_type,
help="Sets number of jobs to use with sphinx-build")
+ parser.add_argument('-i', '--interactive', action='store_true',
+ help="Change latex default to run in interactive mode")
+
args = parser.parse_args()
PythonVersion.check_python(MIN_PYTHON_VERSION)
builder = SphinxBuilder(builddir=args.builddir,
- verbose=args.verbose, n_jobs=args.jobs)
+ verbose=args.verbose, n_jobs=args.jobs,
+ interactive=args.interactive)
builder.build(args.target, sphinxdirs=args.sphinxdirs, conf=args.conf,
theme=args.theme, css=args.css, paper=args.paper)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 14/21] tools/docs,scripts: sphinx-*: prevent sphinx-build crashes
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (12 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 13/21] tools/docs: sphinx-build-wrapper: add an argument for LaTeX interactive mode Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 15/21] tools/docs: sphinx-build-wrapper: allow building PDF files in parallel Mauro Carvalho Chehab
` (8 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
On a properly set system, LANG and LC_ALL is always defined.
However, some distros like Debian, Gentoo and their variants
start with those undefioned.
When Sphinx tries to set a locale with:
locale.setlocale(locale.LC_ALL, '')
It raises an exception, making Sphinx fail. This is more likely
to happen with test containers.
Add a logic to detect and workaround such issue by setting
locale to C.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/sphinx-build-wrapper | 11 +++++++++++
tools/docs/sphinx-pre-install | 14 +++++++++++++-
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 413f51575de8..629abb99b9f3 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -45,6 +45,7 @@ the newer version.
"""
import argparse
+import locale
import os
import shlex
import shutil
@@ -457,6 +458,16 @@ class SphinxBuilder:
if not sphinxdirs:
sphinxdirs = os.environ.get("SPHINXDIRS", ".")
+ #
+ # The sphinx-build tool has a bug: internally, it tries to set
+ # locale with locale.setlocale(locale.LC_ALL, ''). This causes a
+ # crash if language is not set. Detect and fix it.
+ #
+ try:
+ locale.setlocale(locale.LC_ALL, '')
+ except locale.Error:
+ self.env["LC_ALL"] = "C"
+
#
# sphinxdirs can be a list or a whitespace-separated string
#
diff --git a/tools/docs/sphinx-pre-install b/tools/docs/sphinx-pre-install
index d6d673b7945c..663d4e2a3f57 100755
--- a/tools/docs/sphinx-pre-install
+++ b/tools/docs/sphinx-pre-install
@@ -26,6 +26,7 @@ system pacage install is recommended.
"""
import argparse
+import locale
import os
import re
import subprocess
@@ -422,8 +423,19 @@ class MissingCheckers(AncillaryMethods):
"""
Gets sphinx-build version.
"""
+ env = os.environ.copy()
+
+ # The sphinx-build tool has a bug: internally, it tries to set
+ # locale with locale.setlocale(locale.LC_ALL, ''). This causes a
+ # crash if language is not set. Detect and fix it.
try:
- result = self.run([cmd, "--version"],
+ locale.setlocale(locale.LC_ALL, '')
+ except Exception:
+ env["LC_ALL"] = "C"
+ env["LANG"] = "C"
+
+ try:
+ result = self.run([cmd, "--version"], env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True, check=True)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 15/21] tools/docs: sphinx-build-wrapper: allow building PDF files in parallel
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (13 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 14/21] tools/docs,scripts: sphinx-*: prevent sphinx-build crashes Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 16/21] tools/docs: sphinx-build-wrapper: Fix output for duplicated names Mauro Carvalho Chehab
` (7 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Use POSIX jobserver when available or -j<number> to run PDF
builds in parallel, restoring pdf build performance. Yet,
running it when debugging troubles is a bad idea, so, when
calling directly via command line, except if "-j" is splicitly
requested, it will serialize the build.
With such change, a PDF doc builds now takes around 5 minutes
on a Ryzen 9 machine with 32 cpu threads:
# Explicitly paralelize both Sphinx and LaTeX pdf builds
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs -j 33
real 5m17.901s
user 15m1.499s
sys 2m31.482s
# Use POSIX jobserver to paralelize both sphinx-build and LaTeX
$ make cleandocs; time make pdfdocs
real 5m22.369s
user 15m9.076s
sys 2m31.419s
# Serializes PDF build, while keeping Sphinx parallelized.
# it is equivalent of passing -jauto via command line
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs
real 11m20.901s
user 13m2.910s
sys 1m44.553s
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/sphinx-build-wrapper | 218 +++++++++++++++++++++++---------
1 file changed, 156 insertions(+), 62 deletions(-)
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 629abb99b9f3..0e2cb087cdac 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -52,6 +52,8 @@ import shutil
import subprocess
import sys
+from concurrent import futures
+
from lib.python_version import PythonVersion
from lib.latex_fonts import LatexFontChecker
@@ -292,6 +294,87 @@ class SphinxBuilder:
except (OSError, IOError) as e:
print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr)
+ def build_pdf_file(self, latex_cmd, from_dir, path):
+ """Builds a single pdf file using latex_cmd"""
+ try:
+ subprocess.run(latex_cmd + [path],
+ cwd=from_dir, check=True)
+
+ return True
+ except subprocess.CalledProcessError:
+ return False
+
+ def pdf_parallel_build(self, tex_suffix, latex_cmd, tex_files, n_jobs):
+ """Build PDF files in parallel if possible"""
+ builds = {}
+ build_failed = False
+ max_len = 0
+ has_tex = False
+
+ #
+ # LaTeX PDF error code is almost useless for us:
+ # any warning makes it non-zero. For kernel doc builds it always return
+ # non-zero even when build succeeds. So, let's do the best next thing:
+ # Ignore build errors. At the end, check if all PDF files were built,
+ # printing a summary with the built ones and returning 0 if all of
+ # them were actually built.
+ #
+ with futures.ThreadPoolExecutor(max_workers=n_jobs) as executor:
+ jobs = {}
+
+ for from_dir, pdf_dir, entry in tex_files:
+ name = entry.name
+
+ if not name.endswith(tex_suffix):
+ continue
+
+ name = name[:-len(tex_suffix)]
+
+ max_len = max(max_len, len(name))
+
+ has_tex = True
+
+ future = executor.submit(self.build_pdf_file, latex_cmd,
+ from_dir, entry.path)
+ jobs[future] = (from_dir, pdf_dir, name)
+
+ for future in futures.as_completed(jobs):
+ from_dir, pdf_dir, name = jobs[future]
+
+ pdf_name = name + ".pdf"
+ pdf_from = os.path.join(from_dir, pdf_name)
+
+ try:
+ success = future.result()
+
+ if success and os.path.exists(pdf_from):
+ pdf_to = os.path.join(pdf_dir, pdf_name)
+
+ os.rename(pdf_from, pdf_to)
+
+ #
+ # if verbose, get the name of built PDF file
+ #
+ if self.verbose:
+ builds[name] = os.path.relpath(pdf_to, self.builddir)
+ else:
+ builds[name] = "FAILED"
+ build_failed = True
+ except futures.Error as e:
+ builds[name] = f"FAILED ({repr(e)})"
+ build_failed = True
+
+ #
+ # Handle case where no .tex files were found
+ #
+ if not has_tex:
+ name = "Sphinx LaTeX builder"
+ max_len = max(max_len, len(name))
+ builds[name] = "FAILED (no .tex file was generated)"
+ build_failed = True
+
+ return builds, build_failed, max_len
+
def handle_pdf(self, output_dirs):
"""
Extra steps for PDF output.
@@ -302,7 +385,9 @@ class SphinxBuilder:
"""
builds = {}
max_len = 0
+ tex_suffix = ".tex"
+ tex_files = []
for from_dir in output_dirs:
pdf_dir = os.path.join(from_dir, "../pdf")
os.makedirs(pdf_dir, exist_ok=True)
@@ -314,74 +399,83 @@ class SphinxBuilder:
latex_cmd.extend(shlex.split(self.latexopts))
- tex_suffix = ".tex"
-
- #
- # Process each .tex file
- #
-
- has_tex = False
- build_failed = False
+ # Get a list of tex files to process
with os.scandir(from_dir) as it:
for entry in it:
- if not entry.name.endswith(tex_suffix):
- continue
-
- name = entry.name[:-len(tex_suffix)]
- has_tex = True
-
- #
- # LaTeX PDF error code is almost useless for us:
- # any warning makes it non-zero. For kernel doc builds it
- # always return non-zero even when build succeeds.
- # So, let's do the best next thing: check if all PDF
- # files were built. If they're, print a summary and
- # return 0 at the end of this function
- #
- try:
- subprocess.run(latex_cmd + [entry.path],
- cwd=from_dir, check=True)
- except subprocess.CalledProcessError:
- pass
-
- pdf_name = name + ".pdf"
- pdf_from = os.path.join(from_dir, pdf_name)
- pdf_to = os.path.join(pdf_dir, pdf_name)
-
- if os.path.exists(pdf_from):
- os.rename(pdf_from, pdf_to)
- builds[name] = os.path.relpath(pdf_to, self.builddir)
- else:
- builds[name] = "FAILED"
- build_failed = True
-
- name = entry.name.removesuffix(".tex")
- max_len = max(max_len, len(name))
-
- if not has_tex:
- name = os.path.basename(from_dir)
- max_len = max(max_len, len(name))
- builds[name] = "FAILED (no .tex)"
- build_failed = True
-
- msg = "Summary"
- msg += "\n" + "=" * len(msg)
- print()
- print(msg)
-
- for pdf_name, pdf_file in builds.items():
- print(f"{pdf_name:<{max_len}}: {pdf_file}")
-
- print()
-
- if build_failed:
+ if entry.name.endswith(tex_suffix):
+ tex_files.append((from_dir, pdf_dir, entry))
+
+ #
+ # When using make, this won't be used, as the number of jobs comes
+ # from POSIX jobserver. So, this covers the case where build comes
+ # from command line. On such case, serialize by default, except if
+ # the user explicitly sets the number of jobs.
+ #
+ n_jobs = 1
+
+ # n_jobs is either an integer or "auto". Only use it if it is a number
+ if self.n_jobs:
+ try:
+ n_jobs = int(self.n_jobs)
+ except ValueError:
+ pass
+
+ #
+ # When using make, jobserver.claim is the number of jobs that were
+ # used with "-j" and that aren't used by other make targets
+ #
+ with JobserverExec() as jobserver:
+ n_jobs = 1
+
+ #
+ # Handle the case when a parameter is passed via command line,
+ # using it as default, if jobserver doesn't claim anything
+ #
+ if self.n_jobs:
+ try:
+ n_jobs = int(self.n_jobs)
+ except ValueError:
+ pass
+
+ if jobserver.claim:
+ n_jobs = jobserver.claim
+
+ builds, build_failed, max_len = self.pdf_parallel_build(tex_suffix,
+ latex_cmd,
+ tex_files,
+ n_jobs)
+
+ #
+ # In verbose mode, print a summary with the build results per file.
+ # Otherwise, print a single line with all failures, if any.
+ # On both cases, return code 1 indicates build failures,
+ #
+ if self.verbose:
+ msg = "Summary"
+ msg += "\n" + "=" * len(msg)
+ print()
+ print(msg)
+
+ for pdf_name, pdf_file in builds.items():
+ print(f"{pdf_name:<{max_len}}: {pdf_file}")
+
+ print()
+ if build_failed:
+ msg = LatexFontChecker().check()
+ if msg:
+ print(msg)
+
+ sys.exit("Error: not all PDF files were created.")
+
+ elif build_failed:
+ n_failures = len(builds)
+ failures = ", ".join(builds.keys())
+
msg = LatexFontChecker().check()
if msg:
print(msg)
- sys.exit("PDF build failed: not all PDF files were created.")
- else:
- print("All PDF files were built.")
+ sys.exit(f"Error: Can't build {n_failures} PDF file(s): {failures}")
def handle_info(self, output_dirs):
"""
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 16/21] tools/docs: sphinx-build-wrapper: Fix output for duplicated names
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (14 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 15/21] tools/docs: sphinx-build-wrapper: allow building PDF files in parallel Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 17/21] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab
` (6 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
When SPHINXDIRS is used, basename may be identical for different
files. If this happens, the summary and error detection won't be
accurate.
Fix it by using relative names from builddir.
While here, don't duplicate names. Report, instead:
- SUCCESS
output PDF file was built
- FAILED
latexmk/xelatex didn't build any PDF output
- FAILED: no .tex files were generated
Sphinx didn't build any tex file for SPHINXDIRS directories
- FAILED ({python exception})
When a concurrent.futures is catched. Usually indicates an
internal error at the build logic.
With that, building multiple dirs with the same name is reported
properly:
$ make V=1 SPHINXDIRS="admin-guide/media driver-api/media userspace-api/media" pdfdocs
Summary
=======
admin-guide/media/pdf/media.pdf : SUCCESS
driver-api/media/pdf/media.pdf : SUCCESS
userspace-api/media/pdf/media.pdf: SUCCESS
And if at least one of them fails, return code will be 1.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/sphinx-build-wrapper | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 0e2cb087cdac..cd7c450832d0 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -329,9 +329,6 @@ class SphinxBuilder:
continue
name = name[:-len(tex_suffix)]
-
- max_len = max(max_len, len(name))
-
has_tex = True
future = executor.submit(self.build_pdf_file, latex_cmd,
@@ -343,34 +340,35 @@ class SphinxBuilder:
pdf_name = name + ".pdf"
pdf_from = os.path.join(from_dir, pdf_name)
+ pdf_to = os.path.join(pdf_dir, pdf_name)
+ out_name = os.path.relpath(pdf_to, self.builddir)
+ max_len = max(max_len, len(out_name))
try:
success = future.result()
if success and os.path.exists(pdf_from):
- pdf_to = os.path.join(pdf_dir, pdf_name)
-
os.rename(pdf_from, pdf_to)
#
# if verbose, get the name of built PDF file
#
if self.verbose:
- builds[name] = os.path.relpath(pdf_to, self.builddir)
+ builds[out_name] = "SUCCESS"
else:
- builds[name] = "FAILED"
+ builds[out_name] = "FAILED"
build_failed = True
except futures.Error as e:
- builds[name] = f"FAILED ({repr(e)})"
+ builds[out_name] = f"FAILED ({repr(e)})"
build_failed = True
#
# Handle case where no .tex files were found
#
if not has_tex:
- name = "Sphinx LaTeX builder"
- max_len = max(max_len, len(name))
- builds[name] = "FAILED (no .tex file was generated)"
+ out_name = "LaTeX files"
+ max_len = max(max_len, len(out_name))
+ builds[out_name] = "FAILED: no .tex files were generated"
build_failed = True
return builds, build_failed, max_len
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 17/21] docs: add support to build manpages from kerneldoc output
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (15 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 16/21] tools/docs: sphinx-build-wrapper: Fix output for duplicated names Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 18/21] tools: kernel-doc: add a see also section at man pages Mauro Carvalho Chehab
` (5 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Thomas Weißschuh, Alice Ryhl,
Masahiro Yamada, Mauro Carvalho Chehab, Miguel Ojeda,
Nathan Chancellor, Nicolas Schier, Randy Dunlap, Tamir Duberstein,
linux-kbuild, linux-kernel
Generating man files currently requires running a separate
script. The target also doesn't appear at the docs Makefile.
Add support for mandocs at the Makefile, adding the build
logic inside sphinx-build-wrapper, updating documentation
and dropping the ancillary script.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 3 +-
Documentation/doc-guide/kernel-doc.rst | 29 ++++-----
Makefile | 2 +-
scripts/split-man.pl | 28 ---------
tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++--
5 files changed, 95 insertions(+), 48 deletions(-)
delete mode 100755 scripts/split-man.pl
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 0e1d8657a5cc..f9b6e9386a58 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0)
else # HAVE_SPHINX
# Common documentation targets
-infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
+mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
$(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
+$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
--sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \
@@ -106,6 +106,7 @@ dochelp:
@echo ' htmldocs - HTML'
@echo ' texinfodocs - Texinfo'
@echo ' infodocs - Info'
+ @echo ' mandocs - Man pages'
@echo ' latexdocs - LaTeX'
@echo ' pdfdocs - PDF'
@echo ' epubdocs - EPUB'
diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index af9697e60165..4370cc8fbcf5 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -579,20 +579,23 @@ source.
How to use kernel-doc to generate man pages
-------------------------------------------
-If you just want to use kernel-doc to generate man pages you can do this
-from the kernel git tree::
+To generate man pages for all files that contain kernel-doc markups, run::
- $ scripts/kernel-doc -man \
- $(git grep -l '/\*\*' -- :^Documentation :^tools) \
- | scripts/split-man.pl /tmp/man
+ $ make mandocs
-Some older versions of git do not support some of the variants of syntax for
-path exclusion. One of the following commands may work for those versions::
+Or calling ``script-build-wrapper`` directly::
- $ scripts/kernel-doc -man \
- $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \
- | scripts/split-man.pl /tmp/man
+ $ ./tools/docs/sphinx-build-wrapper mandocs
- $ scripts/kernel-doc -man \
- $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \
- | scripts/split-man.pl /tmp/man
+The output will be at ``/man`` directory inside the output directory
+(by default: ``Documentation/output``).
+
+Optionally, it is possible to generate a partial set of man pages by
+using SPHINXDIRS:
+
+ $ make SPHINXDIRS=driver-api/media mandocs
+
+.. note::
+
+ When SPHINXDIRS={subdir} is used, it will only generate man pages for
+ the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file.
diff --git a/Makefile b/Makefile
index 6bfe776bf3c5..9bd44afeda26 100644
--- a/Makefile
+++ b/Makefile
@@ -1800,7 +1800,7 @@ $(help-board-dirs): help-%:
# Documentation targets
# ---------------------------------------------------------------------------
DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
- linkcheckdocs dochelp refcheckdocs texinfodocs infodocs
+ linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs
PHONY += $(DOC_TARGETS)
$(DOC_TARGETS):
$(Q)$(MAKE) $(build)=Documentation $@
diff --git a/scripts/split-man.pl b/scripts/split-man.pl
deleted file mode 100755
index 96bd99dc977a..000000000000
--- a/scripts/split-man.pl
+++ /dev/null
@@ -1,28 +0,0 @@
-#!/usr/bin/env perl
-# SPDX-License-Identifier: GPL-2.0
-#
-# Author: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
-#
-# Produce manpages from kernel-doc.
-# See Documentation/doc-guide/kernel-doc.rst for instructions
-
-if ($#ARGV < 0) {
- die "where do I put the results?\n";
-}
-
-mkdir $ARGV[0],0777;
-$state = 0;
-while (<STDIN>) {
- if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) {
- if ($state == 1) { close OUT }
- $state = 1;
- $fn = "$ARGV[0]/$1.9";
- print STDERR "Creating $fn\n";
- open OUT, ">$fn" or die "can't open $fn: $!\n";
- print OUT $_;
- } elsif ($state != 0) {
- print OUT $_;
- }
-}
-
-close OUT;
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index cd7c450832d0..eff6b41b7d88 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -47,12 +47,14 @@ the newer version.
import argparse
import locale
import os
+import re
import shlex
import shutil
import subprocess
import sys
from concurrent import futures
+from glob import glob
from lib.python_version import PythonVersion
from lib.latex_fonts import LatexFontChecker
@@ -77,6 +79,7 @@ TARGETS = {
"epubdocs": { "builder": "epub", "out_dir": "epub" },
"texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" },
"infodocs": { "builder": "texinfo", "out_dir": "texinfo" },
+ "mandocs": { "builder": "man", "out_dir": "man" },
"latexdocs": { "builder": "latex", "out_dir": "latex" },
"pdfdocs": { "builder": "latex", "out_dir": "latex" },
"xmldocs": { "builder": "xml", "out_dir": "xml" },
@@ -489,6 +492,71 @@ class SphinxBuilder:
except subprocess.CalledProcessError as e:
sys.exit(f"Error generating info docs: {e}")
+ def handle_man(self, kerneldoc, docs_dir, src_dir, output_dir):
+ """
+ Create man pages from kernel-doc output
+ """
+
+ re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)")
+ re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"')
+
+ if docs_dir == src_dir:
+ #
+ # Pick the entire set of kernel-doc markups from the entire tree
+ #
+ kdoc_files = set([self.srctree])
+ else:
+ kdoc_files = set()
+
+ for fname in glob(os.path.join(src_dir, "**"), recursive=True):
+ if os.path.isfile(fname) and fname.endswith(".rst"):
+ with open(fname, "r", encoding="utf-8") as in_fp:
+ data = in_fp.read()
+
+ for line in data.split("\n"):
+ match = re_kernel_doc.match(line)
+ if match:
+ if os.path.isfile(match.group(1)):
+ kdoc_files.add(match.group(1))
+
+ if not kdoc_files:
+ sys.exit(f"Directory {src_dir} doesn't contain kernel-doc tags")
+
+ cmd = [ kerneldoc, "-m" ] + sorted(kdoc_files)
+ try:
+ if self.verbose:
+ print(" ".join(cmd))
+
+ result = subprocess.run(cmd, stdout=subprocess.PIPE, text= True)
+
+ if result.returncode:
+ print(f"Warning: kernel-doc returned {result.returncode} warnings")
+
+ except (OSError, ValueError, subprocess.SubprocessError) as e:
+ sys.exit(f"Failed to create man pages for {src_dir}: {repr(e)}")
+
+ fp = None
+ try:
+ for line in result.stdout.split("\n"):
+ match = re_man.match(line)
+ if not match:
+ if fp:
+ fp.write(line + '\n')
+ continue
+
+ if fp:
+ fp.close()
+
+ fname = f"{output_dir}/{match.group(2)}.{match.group(1)}"
+
+ if self.verbose:
+ print(f"Creating {fname}")
+ fp = open(fname, "w", encoding="utf-8")
+ fp.write(line + '\n')
+ finally:
+ if fp:
+ fp.close()
+
def cleandocs(self, builder): # pylint: disable=W0613
"""Remove documentation output directory"""
shutil.rmtree(self.builddir, ignore_errors=True)
@@ -517,7 +585,7 @@ class SphinxBuilder:
# Other targets require sphinx-build, so check if it exists
#
sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"])
- if not sphinxbuild:
+ if not sphinxbuild and target != "mandocs":
sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n")
if builder == "latex":
@@ -605,10 +673,13 @@ class SphinxBuilder:
output_dir,
]
- try:
- self.run_sphinx(sphinxbuild, build_args, env=self.env)
- except (OSError, ValueError, subprocess.SubprocessError) as e:
- sys.exit(f"Build failed: {repr(e)}")
+ if target == "mandocs":
+ self.handle_man(kerneldoc, docs_dir, src_dir, output_dir)
+ else:
+ try:
+ self.run_sphinx(sphinxbuild, build_args, env=self.env)
+ except (OSError, ValueError, subprocess.SubprocessError) as e:
+ sys.exit(f"Build failed: {repr(e)}")
#
# Ensure that each html/epub output will have needed static files
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 18/21] tools: kernel-doc: add a see also section at man pages
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (16 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 17/21] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 19/21] scripts: kdoc_parser.py: warn about Python version only once Mauro Carvalho Chehab
` (4 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
While cross-references are complex, as related ones can be on
different files, we can at least correlate the ones that belong
to the same file, adding a SEE ALSO section for them.
The result is not bad. See for instance:
$ tools/docs/sphinx-build-wrapper --sphinxdirs driver-api/media -- mandocs
$ man Documentation/output/driver-api/man/edac_pci_add_device.9
edac_pci_add_device(9) Kernel Hacker's Manual edac_pci_add_device(9)
NAME
edac_pci_add_device - Insert the 'edac_dev' structure into the
edac_pci global list and create sysfs entries associated with
edac_pci structure.
SYNOPSIS
int edac_pci_add_device (struct edac_pci_ctl_info *pci , int
edac_idx );
ARGUMENTS
pci pointer to the edac_device structure to be added to
the list
edac_idx A unique numeric identifier to be assigned to the
RETURN
0 on Success, or an error code on failure
SEE ALSO
edac_pci_alloc_ctl_info(9), edac_pci_free_ctl_info(9),
edac_pci_alloc_index(9), edac_pci_del_device(9), edac_pci_cre\u2010
ate_generic_ctl(9), edac_pci_release_generic_ctl(9),
edac_pci_create_sysfs(9), edac_pci_remove_sysfs(9)
August 2025 edac_pci_add_device edac_pci_add_device(9)
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/kdoc/kdoc_files.py | 5 +-
scripts/lib/kdoc/kdoc_output.py | 84 +++++++++++++++++++++++++++++++--
2 files changed, 83 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py
index 9e09b45b02fa..061c033f32da 100644
--- a/scripts/lib/kdoc/kdoc_files.py
+++ b/scripts/lib/kdoc/kdoc_files.py
@@ -275,7 +275,10 @@ class KernelFiles():
self.config.log.warning("No kernel-doc for file %s", fname)
continue
- for arg in self.results[fname]:
+ symbols = self.results[fname]
+ self.out_style.set_symbols(symbols)
+
+ for arg in symbols:
m = self.out_msg(fname, arg.name, arg)
if m is None:
diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output.py
index ea8914537ba0..1eca9a918558 100644
--- a/scripts/lib/kdoc/kdoc_output.py
+++ b/scripts/lib/kdoc/kdoc_output.py
@@ -215,6 +215,9 @@ class OutputFormat:
# Virtual methods to be overridden by inherited classes
# At the base class, those do nothing.
+ def set_symbols(self, symbols):
+ """Get a list of all symbols from kernel_doc"""
+
def out_doc(self, fname, name, args):
"""Outputs a DOC block"""
@@ -577,6 +580,7 @@ class ManFormat(OutputFormat):
super().__init__()
self.modulename = modulename
+ self.symbols = []
dt = None
tstamp = os.environ.get("KBUILD_BUILD_TIMESTAMP")
@@ -593,6 +597,68 @@ class ManFormat(OutputFormat):
self.man_date = dt.strftime("%B %Y")
+ def arg_name(self, args, name):
+ """
+ Return the name that will be used for the man page.
+
+ As we may have the same name on different namespaces,
+ prepend the data type for all types except functions and typedefs.
+
+ The doc section is special: it uses the modulename.
+ """
+
+ dtype = args.type
+
+ if dtype == "doc":
+ return self.modulename
+
+ if dtype in ["function", "typedef"]:
+ return name
+
+ return f"{dtype} {name}"
+
+ def set_symbols(self, symbols):
+ """
+ Get a list of all symbols from kernel_doc.
+
+ Man pages will uses it to add a SEE ALSO section with other
+ symbols at the same file.
+ """
+ self.symbols = symbols
+
+ def out_tail(self, fname, name, args):
+ """Adds a tail for all man pages"""
+
+ # SEE ALSO section
+ if len(self.symbols) >= 2:
+ cur_name = self.arg_name(args, name)
+
+ self.data += f'.SH "SEE ALSO"' + "\n.PP\n"
+ related = []
+ for arg in self.symbols:
+ out_name = self.arg_name(arg, arg.name)
+
+ if cur_name == out_name:
+ continue
+
+ related.append(f"\\fB{out_name}\\fR(9)")
+
+ self.data += ",\n".join(related) + "\n"
+
+ # TODO: does it make sense to add other sections? Maybe
+ # REPORTING ISSUES? LICENSE?
+
+ def msg(self, fname, name, args):
+ """
+ Handles a single entry from kernel-doc parser.
+
+ Add a tail at the end of man pages output.
+ """
+ super().msg(fname, name, args)
+ self.out_tail(fname, name, args)
+
+ return self.data
+
def output_highlight(self, block):
"""
Outputs a C symbol that may require being highlighted with
@@ -618,7 +684,9 @@ class ManFormat(OutputFormat):
if not self.check_doc(name, args):
return
- self.data += f'.TH "{self.modulename}" 9 "{self.modulename}" "{self.man_date}" "API Manual" LINUX' + "\n"
+ out_name = self.arg_name(args, name)
+
+ self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
for section, text in args.sections.items():
self.data += f'.SH "{section}"' + "\n"
@@ -627,7 +695,9 @@ class ManFormat(OutputFormat):
def out_function(self, fname, name, args):
"""output function in man"""
- self.data += f'.TH "{name}" 9 "{name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
+ out_name = self.arg_name(args, name)
+
+ self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
self.data += ".SH NAME\n"
self.data += f"{name} \\- {args['purpose']}\n"
@@ -671,7 +741,9 @@ class ManFormat(OutputFormat):
self.output_highlight(text)
def out_enum(self, fname, name, args):
- self.data += f'.TH "{self.modulename}" 9 "enum {name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+ out_name = self.arg_name(args, name)
+
+ self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
self.data += ".SH NAME\n"
self.data += f"enum {name} \\- {args['purpose']}\n"
@@ -703,8 +775,9 @@ class ManFormat(OutputFormat):
def out_typedef(self, fname, name, args):
module = self.modulename
purpose = args.get('purpose')
+ out_name = self.arg_name(args, name)
- self.data += f'.TH "{module}" 9 "{name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+ self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
self.data += ".SH NAME\n"
self.data += f"typedef {name} \\- {purpose}\n"
@@ -717,8 +790,9 @@ class ManFormat(OutputFormat):
module = self.modulename
purpose = args.get('purpose')
definition = args.get('definition')
+ out_name = self.arg_name(args, name)
- self.data += f'.TH "{module}" 9 "{args.type} {name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+ self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
self.data += ".SH NAME\n"
self.data += f"{args.type} {name} \\- {purpose}\n"
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 19/21] scripts: kdoc_parser.py: warn about Python version only once
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (17 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 18/21] tools: kernel-doc: add a see also section at man pages Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 20/21] tools/docs: sphinx-* break documentation bulds on openSUSE Mauro Carvalho Chehab
` (3 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
When running kernel-doc over multiple documents, it emits
one error message per file with is not what we want:
$ python3.6 scripts/kernel-doc.py . --none
...
Warning: ./include/trace/events/swiotlb.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/iommu.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/sock.h:0 Python 3.7 or later is required for correct results
...
Change the logic to warn it only once at the library:
$ python3.6 scripts/kernel-doc.py . --none
Warning: Python 3.7 or later is required for correct results
Warning: ./include/cxl/features.h:0 Python 3.7 or later is required for correct results
When running from command line, it warns twice, but that sounds
ok.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/kdoc/kdoc_parser.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index a560546c1867..574972e1f741 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -314,6 +314,7 @@ class KernelEntry:
self.section = SECTION_DEFAULT
self._contents = []
+python_warning = False
class KernelDoc:
"""
@@ -347,9 +348,13 @@ class KernelDoc:
# We need Python 3.7 for its "dicts remember the insertion
# order" guarantee
#
- if sys.version_info.major == 3 and sys.version_info.minor < 7:
+ global python_warning
+ if (not python_warning and
+ sys.version_info.major == 3 and sys.version_info.minor < 7):
+
self.emit_msg(0,
'Python 3.7 or later is required for correct results')
+ python_warning = True
def emit_msg(self, ln, msg, warning=True):
"""Emit a message"""
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 20/21] tools/docs: sphinx-* break documentation bulds on openSUSE
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (18 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 19/21] scripts: kdoc_parser.py: warn about Python version only once Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 21/21] tools/docs: sphinx-build-wrapper: add support to run inside venv Mauro Carvalho Chehab
` (2 subsequent siblings)
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Björn Roy Baron, Alex Gaynor,
Alice Ryhl, Andreas Hindborg, Benno Lossin, Boqun Feng,
Danilo Krummrich, Gary Guo, Mauro Carvalho Chehab, Miguel Ojeda,
Trevor Gross, linux-kernel, rust-for-linux
Before this patch, building htmldocs on opensuseLEAP works
fine:
# make htmldocs
Available Python versions:
/usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Using alabaster theme
Using Python kernel-doc
...
As the logic detects that Python 3.6 is too old and recommends
intalling python311-Sphinx. If installed, documentation builds
work like a charm.
Yet, some develpers complained that running python3.11 instead
of python3 should not happen. So, let's break the build to make
them happier:
$ make htmldocs
Python 3.6.15 not supported. Bailing out
You could run, instead:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs \
--sphinxdirs=. --conf=conf.py --builddir=Documentation/output --theme= --css= \
--paper=
Python 3.6.15 not supported. Bailing out
make[2]: *** [Documentation/Makefile:76: htmldocs] Error 1
make[1]: *** [Makefile:1806: htmldocs] Error 2
make: *** [Makefile:248: __sub-make] Error 2
It should be noticed that:
1. after this change, sphinx-pre-install needs to be called
by hand:
$ /usr/bin/python3.11 tools/docs/sphinx-pre-install
Detected OS: openSUSE Leap 15.6.
Sphinx version: 7.2.6
All optional dependencies are met.
Needed package dependencies are met.
2. sphinx-build-wrapper will auto-detect python3.11 and
suggest a way to build the docs using the parameters passed
via make variables. In this specific example:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs --sphinxdirs=. --conf=conf.py --theme= --css= --paper=
3. As this needs to be executed outside docs Makefile, it won't run
the validation check scripts nor build Rust documentation if
enabled, as the extra scripts are part of the docs Makefile.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/lib/python_version.py | 28 ++++++++++++++++++++++++----
tools/docs/sphinx-build-wrapper | 3 ++-
tools/docs/sphinx-pre-install | 3 ++-
3 files changed, 28 insertions(+), 6 deletions(-)
diff --git a/tools/docs/lib/python_version.py b/tools/docs/lib/python_version.py
index a9fda2470a26..4fde1b882164 100644
--- a/tools/docs/lib/python_version.py
+++ b/tools/docs/lib/python_version.py
@@ -20,9 +20,11 @@ Python version if present.
import os
import re
import subprocess
+import shlex
import sys
from glob import glob
+from textwrap import indent
class PythonVersion:
"""
@@ -44,6 +46,25 @@ class PythonVersion:
"""Returns a version tuple as major.minor.patch"""
return ".".join([str(x) for x in version])
+ @staticmethod
+ def cmd_print(cmd, max_len=80):
+ cmd_line = []
+
+ for w in cmd:
+ w = shlex.quote(w)
+
+ if cmd_line:
+ if not max_len or len(cmd_line[-1]) + len(w) < max_len:
+ cmd_line[-1] += " " + w
+ continue
+ else:
+ cmd_line[-1] += " \\"
+ cmd_line.append(w)
+ else:
+ cmd_line.append(w)
+
+ return "\n ".join(cmd_line)
+
def __str__(self):
"""Returns a version tuple as major.minor.patch from self.version"""
return self.ver_str(self.version)
@@ -130,14 +151,13 @@ class PythonVersion:
else:
new_python_cmd = None
- if show_alternatives:
+ if show_alternatives and available_versions:
print("You could run, instead:")
for _, cmd in available_versions:
args = [cmd, script_path] + sys.argv[1:]
- cmd_str = " ".join(args)
- print(f" {cmd_str}")
- print()
+ cmd_str = indent(PythonVersion.cmd_print(args), " ")
+ print(f"{cmd_str}\n")
if bail_out:
msg = f"Python {python_ver} not supported. Bailing out"
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index eff6b41b7d88..90ad0b22b472 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -750,7 +750,8 @@ def main():
args = parser.parse_args()
- PythonVersion.check_python(MIN_PYTHON_VERSION)
+ PythonVersion.check_python(MIN_PYTHON_VERSION, show_alternatives=True,
+ bail_out=True)
builder = SphinxBuilder(builddir=args.builddir,
verbose=args.verbose, n_jobs=args.jobs,
diff --git a/tools/docs/sphinx-pre-install b/tools/docs/sphinx-pre-install
index 663d4e2a3f57..698989584b6a 100755
--- a/tools/docs/sphinx-pre-install
+++ b/tools/docs/sphinx-pre-install
@@ -1531,7 +1531,8 @@ def main():
checker = SphinxDependencyChecker(args)
- PythonVersion.check_python(MIN_PYTHON_VERSION)
+ PythonVersion.check_python(MIN_PYTHON_VERSION,
+ bail_out=True, success_on_error=True)
checker.check_needs()
# Call main if not used as module
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 21/21] tools/docs: sphinx-build-wrapper: add support to run inside venv
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (19 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 20/21] tools/docs: sphinx-* break documentation bulds on openSUSE Mauro Carvalho Chehab
@ 2025-09-16 10:22 ` Mauro Carvalho Chehab
2025-09-16 10:31 ` [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
2025-09-16 15:47 ` Jonathan Corbet
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:22 UTC (permalink / raw)
To: Jonathan Corbet, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Sometimes, it is desired to run Sphinx from a virtual environment.
Add a command line parameter to automatically build Sphinx from
such environment.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
tools/docs/sphinx-build-wrapper | 33 ++++++++++++++++++++++++++++++---
1 file changed, 30 insertions(+), 3 deletions(-)
diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 90ad0b22b472..b611dbe250f9 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -69,6 +69,7 @@ from jobserver import JobserverExec # pylint: disable=C0413,C0411,E0401
#
# Some constants
#
+VENV_DEFAULT = "sphinx_latest"
MIN_PYTHON_VERSION = PythonVersion("3.7").version
PAPER = ["", "a4", "letter"]
@@ -179,8 +180,10 @@ class SphinxBuilder:
if not verbose:
self.sphinxopts += ["-q"]
- def __init__(self, builddir, verbose=False, n_jobs=None, interactive=None):
+ def __init__(self, builddir, venv=None, verbose=False, n_jobs=None,
+ interactive=None):
"""Initialize internal variables"""
+ self.venv = venv
self.verbose = None
#
@@ -230,6 +233,21 @@ class SphinxBuilder:
self.get_sphinx_extra_opts(n_jobs)
+ #
+ # If venv command line argument is specified, run Sphinx from venv
+ #
+ if venv:
+ bin_dir = os.path.join(venv, "bin")
+ if not os.path.isfile(os.path.join(bin_dir, "activate")):
+ sys.exit(f"Venv {venv} not found.")
+
+ # "activate" virtual env
+ self.env["PATH"] = bin_dir + ":" + self.env["PATH"]
+ self.env["VIRTUAL_ENV"] = venv
+ if "PYTHONHOME" in self.env:
+ del self.env["PYTHONHOME"]
+ print(f"Setting venv to {venv}")
+
def run_sphinx(self, sphinx_build, build_args, *args, **pwargs):
"""
Executes sphinx-build using current python3 command.
@@ -264,7 +282,12 @@ class SphinxBuilder:
if self.n_jobs:
n_jobs = str(self.n_jobs)
- cmd = [sys.executable, sphinx_build]
+ if self.venv:
+ cmd = ["python"]
+ else:
+ cmd = [sys.executable,]
+
+ cmd += [sphinx_build]
cmd += [f"-j{n_jobs}"]
cmd += self.sphinxopts
cmd += build_args
@@ -748,12 +771,16 @@ def main():
parser.add_argument('-i', '--interactive', action='store_true',
help="Change latex default to run in interactive mode")
+ parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}',
+ default=None,
+ help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})')
+
args = parser.parse_args()
PythonVersion.check_python(MIN_PYTHON_VERSION, show_alternatives=True,
bail_out=True)
- builder = SphinxBuilder(builddir=args.builddir,
+ builder = SphinxBuilder(builddir=args.builddir, venv=args.venv,
verbose=args.verbose, n_jobs=args.jobs,
interactive=args.interactive)
--
2.51.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [PATCH v6 00/21] Split sphinx call logic from docs Makefile
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (20 preceding siblings ...)
2025-09-16 10:22 ` [PATCH v6 21/21] tools/docs: sphinx-build-wrapper: add support to run inside venv Mauro Carvalho Chehab
@ 2025-09-16 10:31 ` Mauro Carvalho Chehab
2025-09-16 15:47 ` Jonathan Corbet
22 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-16 10:31 UTC (permalink / raw)
To: Linux Doc Mailing List, Jonathan Corbet; +Cc: linux-kernel
Em Tue, 16 Sep 2025 12:22:36 +0200
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:
> Hi Jon,
>
> This series should probably be called:
>
> "Move the trick-or-treat build hacks accumulated over time
> into a single place and document them."
>
> as this reflects its main goal. As such:
>
> - it places the jobserver logic on a library;
> - it removes sphinx/parallel-wrapper.sh;
> - the code now properly implements a jobserver-aware logic
> to do the parallelism when called via GNU make, failing back to
> "-j" when there's no jobserver;
> - converts check-variable-fonts.sh to Python and uses it via
> function call;
> - drops an extra script to generate man pages, adding a makefile
> target for it;
> - ensures that return code is 0 when PDF successfully builds;
> - about half of the script is comments and documentation.
>
> I tried to do my best to document all tricks that are inside the
> script. This way, the docs build steps is now documented.
>
> It should be noticed that it is out of the scope of this series
> to change the implementation. Surely the process can be improved,
> but first let's consolidate and document everything on a single
> place.
>
> Such script was written in a way that it can be called either
> directly or via a Makefile. Running outside Makefile is
> interesting specially when debug is needed. The command line
> interface replaces the need of having lots of env vars before
> calling sphinx-build:
>
> $ ./tools/docs/sphinx-build-wrapper --help
> usage: sphinx-build-wrapper [-h]
> [--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
> [--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
> [-j JOBS] [-i] [-V [VENV]]
> {cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
>
> Kernel documentation builder
>
> positional arguments:
> {cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
> Documentation target to build
>
> options:
> -h, --help show this help message and exit
> --sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
> Specific directories to build
> --conf CONF Sphinx configuration file
> --builddir BUILDDIR Sphinx configuration file
> --theme THEME Sphinx theme to use
> --css CSS Custom CSS file for HTML/EPUB
> --paper {,a4,letter} Paper size for LaTeX/PDF output
> -v, --verbose place build in verbose mode
> -j, --jobs JOBS Sets number of jobs to use with sphinx-build
> -i, --interactive Change latex default to run in interactive mode
> -V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)
>
> the only mandatory argument is the target, which is identical with
> "make" targets.
>
> The call inside Makefile doesn't use the last four arguments. They're
> there to help identifying problems at the build:
>
> -v makes the output verbose;
> -j helps to test parallelism;
> -i runs latexmk in interactive mode, allowing to debug PDF
> build issues;
> -V is useful when testing it with different venvs.
>
> When used with GNU make (or some other make which implements jobserver),
> a call like:
>
> make -j <targets> htmldocs
>
> will make the wrapper to automatically use POSIX jobserver to claim
> the number of available job slots, calling sphinx-build with a
> "-j" parameter reflecting it. ON such case, the default can be
> overriden via SPHINXDIRS argument.
>
> Visiable changes when compared with the old behavior:
Visiable -> Visible
> When V=0, the only visible difference is that:
> - pdfdocs target now returns 0 on success, 1 on failures.
> This addresses an issue over the current process where we
> it always return success even on failures;
> - it will now print the name of PDF files that failed to build,
> if any.
>
> In verbose mode, sphinx-build-wrapper and sphinx-build command lines
> are now displayed.
It will also print a summary of all PDF built files, for pdfdocs
target in verbose mode.
>
> ---
>
> v6:
> - On success, PDF output is identical as before when V=0;
> - when V=1 is used, PDF output will print a build summary,
> as on v5;
> - solved a problem when multiple PDF files have the same
> basename but are located on different directories;
> - merged a patch series converting check-variable-fonts.sh
> to Python. Its logic is now called directly without running
> a subprocess.
> - venv patch moved to the end.
>
> v5:
> - merged comments with the script;
> - placed n_jobs on a separate function;
> - nitpick: dropped a for loop used instead of list append.
>
> v4:
> - updated references for sphinx-pre-install after its rename;
> - added some extra patches to add more options to python_version,
> allowing it to bail out and suggest alternatives;
> - added a patch at the end to explicitly break doc builds when
> python3 points to python3.6 or older.
>
> v3:
> - rebased on the top of docs-next;
> - added two patches to build man files that were on a separate
> patch series.
>
> v2:
> - there's no generic exception handler anymore;
> - it moves sphinx-pre-install to tools/docs;
> - the logic which ensures a minimal Python version got moved
> to a library, which is now used by both pre-install and wrapper;
> - The first wrapper (05/13) doesn't contain comments (except for
> shebang and SPDX). The goal is to help showing the size increase
> when moving from Makefile to Python. Some file increase is
> unavoidable, as Makefile is more compact: no includes, multple
> statements per line, no argparse, etc;
> - The second patch adds docstrings and comments. It has almost
> the same size of the code itself;
> - I moved the venv logic to a third wrapper patch;
> - I fixed an issue at the paraller build logic;
> - There are no generic except blocks anymore.
>
>
> Mauro Carvalho Chehab (21):
> scripts/jobserver-exec: move the code to a class
> scripts/jobserver-exec: move its class to the lib directory
> scripts/jobserver-exec: add a help message
> scripts: check-variable-fonts.sh: convert to Python
> tools/docs: check-variable-fonts.py: split into a lib and an exec file
> scripts: sphinx-pre-install: move it to tools/docs
> tools/docs: python_version: move version check from sphinx-pre-install
> tools/docs: python_version: drop a debug print
> tools/docs: python_version: allow check for alternatives and bail out
> tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
> docs: parallel-wrapper.sh: remove script
> docs: Makefile: document latex/PDF PAPER= parameter
> tools/docs: sphinx-build-wrapper: add an argument for LaTeX
> interactive mode
> tools/docs,scripts: sphinx-*: prevent sphinx-build crashes
> tools/docs: sphinx-build-wrapper: allow building PDF files in parallel
> tools/docs: sphinx-build-wrapper: Fix output for duplicated names
> docs: add support to build manpages from kerneldoc output
> tools: kernel-doc: add a see also section at man pages
> scripts: kdoc_parser.py: warn about Python version only once
> tools/docs: sphinx-* break documentation bulds on openSUSE
> tools/docs: sphinx-build-wrapper: add support to run inside venv
>
> Documentation/Makefile | 136 +--
> Documentation/doc-guide/kernel-doc.rst | 29 +-
> Documentation/doc-guide/sphinx.rst | 4 +-
> Documentation/sphinx/kerneldoc-preamble.sty | 2 +-
> Documentation/sphinx/parallel-wrapper.sh | 33 -
> .../translations/it_IT/doc-guide/sphinx.rst | 4 +-
> .../translations/zh_CN/doc-guide/sphinx.rst | 4 +-
> Documentation/translations/zh_CN/how-to.rst | 2 +-
> MAINTAINERS | 4 +-
> Makefile | 2 +-
> scripts/check-variable-fonts.sh | 115 ---
> scripts/jobserver-exec | 88 +-
> scripts/lib/jobserver.py | 149 ++++
> scripts/lib/kdoc/kdoc_files.py | 5 +-
> scripts/lib/kdoc/kdoc_output.py | 84 +-
> scripts/lib/kdoc/kdoc_parser.py | 7 +-
> scripts/split-man.pl | 28 -
> tools/docs/check-variable-fonts.py | 23 +
> tools/docs/lib/latex_fonts.py | 162 ++++
> tools/docs/lib/python_version.py | 178 ++++
> tools/docs/sphinx-build-wrapper | 791 ++++++++++++++++++
> {scripts => tools/docs}/sphinx-pre-install | 135 +--
> 22 files changed, 1502 insertions(+), 483 deletions(-)
> delete mode 100644 Documentation/sphinx/parallel-wrapper.sh
> delete mode 100755 scripts/check-variable-fonts.sh
> create mode 100755 scripts/lib/jobserver.py
> delete mode 100755 scripts/split-man.pl
> create mode 100755 tools/docs/check-variable-fonts.py
> create mode 100755 tools/docs/lib/latex_fonts.py
> create mode 100644 tools/docs/lib/python_version.py
> create mode 100755 tools/docs/sphinx-build-wrapper
> rename {scripts => tools/docs}/sphinx-pre-install (93%)
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 00/21] Split sphinx call logic from docs Makefile
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
` (21 preceding siblings ...)
2025-09-16 10:31 ` [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
@ 2025-09-16 15:47 ` Jonathan Corbet
22 siblings, 0 replies; 34+ messages in thread
From: Jonathan Corbet @ 2025-09-16 15:47 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> Hi Jon,
>
> This series should probably be called:
>
> "Move the trick-or-treat build hacks accumulated over time
> into a single place and document them."
This one doesn't apply - it looks like the same encoding problems with
the translation changes...?
jon
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python
2025-09-16 10:22 ` [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python Mauro Carvalho Chehab
@ 2025-09-17 1:09 ` Akira Yokosawa
2025-09-17 8:48 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 34+ messages in thread
From: Akira Yokosawa @ 2025-09-17 1:09 UTC (permalink / raw)
To: mchehab+huawei; +Cc: corbet, linux-doc, linux-kernel, Akira Yokosawa
On Tue, 16 Sep 2025 12:22:40 +0200, Mauro Carvalho Chehab wrote:
> This script handle errors when trying to build translations
> with make pdfdocs.
>
> As part of our cleanup work to remove hacks from docs Makefile,
> convert this to python, preparing it to be part of a library
> to be called by sphinx-build-wrapper.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
I could apply up to 05/21 of v6 and did some quick tests under
Fedora (where Noto CJK VF fonts are installed).
At 3/21, "./scripts/check-variable-fonts.sh" doesn't say a word.
At 4/21, "./scripts/check-variable-fonts.py" complains:
=============================================================================
XeTeX is confused by "variable font" files listed below:
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc
/usr/share/fonts/google-noto-sans-mono-cjk-vf-fonts/NotoSansMonoCJK-VF.ttc
/usr/share/fonts/google-noto-serif-cjk-vf-fonts/NotoSerifCJK-VF.ttc
For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
Or, CJK pages can be skipped by uninstalling texlive-xecjk.
For more info on denylisting, other options, and variable font, see header
comments of scripts/check-variable-fonts.py.
=============================================================================
Of course, I have followed the suggestions in the header comments.
So I have to NAK on 4/21.
Regards,
Akira
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-16 10:22 ` [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build Mauro Carvalho Chehab
@ 2025-09-17 8:35 ` Akira Yokosawa
2025-09-17 8:45 ` Akira Yokosawa
` (2 more replies)
0 siblings, 3 replies; 34+ messages in thread
From: Akira Yokosawa @ 2025-09-17 8:35 UTC (permalink / raw)
To: mchehab+huawei
Cc: corbet, linux-doc, linux-kernel, Jani Nikula, Akira Yokosawa
[+CC: Jani, -CC: rust people and list]
On Tue, 16 Sep 2025 12:22:46 +0200, Mauro Carvalho Chehab wrote:
> There are too much magic inside docs Makefile to properly run
> sphinx-build. Create an ancillary script that contains all
> kernel-related sphinx-build call logic currently at Makefile.
>
> Such script is designed to work both as an standalone command
> and as part of a Makefile. As such, it properly handles POSIX
> jobserver used by GNU make.
>
> On a side note, there was a line number increase due to the
> conversion (ignoring comments) is:
>
> Documentation/Makefile | 131 +++----------
> tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++
> 2 files changed, 323 insertions(+), 101 deletions(-)
>
> Comments and descriptions adds:
> tools/docs/sphinx-build-wrapper | 261 +++++++++++++++++++++++++++++++-
>
> So, about half of the script are comments/descriptions.
>
> This is because some things are more verbosed on Python and because
> it requires reading env vars from Makefile. Besides it, this script
> has some extra features that don't exist at the Makefile:
>
> - It can be called directly from command line;
> - It properly return PDF build errors.
>
> When running the script alone, it will only take handle sphinx-build
> targets. On other words, it won't runn make rustdoc after building
> htmlfiles, nor it will run the extra check scripts.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> Documentation/Makefile | 131 ++-----
> tools/docs/sphinx-build-wrapper | 581 ++++++++++++++++++++++++++++++++
> 2 files changed, 611 insertions(+), 101 deletions(-)
> create mode 100755 tools/docs/sphinx-build-wrapper
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 7570d4cf3b13..4736f02b6c9e 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -23,21 +23,22 @@ SPHINXOPTS =
> SPHINXDIRS = .
> DOCS_THEME =
> DOCS_CSS =
> -_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
Wait! In the cover-letter, you said:
It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.
Removing current restriction on SPHINXDIRS does look inconsistent with
your own words to me.
So, I guess I have to NAK 06/21 as well.
Regards,
Akira
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-17 8:35 ` Akira Yokosawa
@ 2025-09-17 8:45 ` Akira Yokosawa
2025-09-17 15:11 ` Jonathan Corbet
2025-09-17 15:18 ` Mauro Carvalho Chehab
2 siblings, 0 replies; 34+ messages in thread
From: Akira Yokosawa @ 2025-09-17 8:45 UTC (permalink / raw)
To: mchehab+huawei
Cc: corbet, linux-doc, linux-kernel, Jani Nikula, Akira Yokosawa
On Wed, 17 Sep 2025 17:35:31 +0900, Akira Yokosawa wrote:
> So, I guess I have to NAK 06/21 as well.
I meant 10/21.
Sorry for the noise.
>
> Regards,
> Akira
>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python
2025-09-17 1:09 ` Akira Yokosawa
@ 2025-09-17 8:48 ` Mauro Carvalho Chehab
2025-09-17 23:22 ` Akira Yokosawa
0 siblings, 1 reply; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-17 8:48 UTC (permalink / raw)
To: Akira Yokosawa; +Cc: corbet, linux-doc, linux-kernel
Em Wed, 17 Sep 2025 10:09:05 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:
> On Tue, 16 Sep 2025 12:22:40 +0200, Mauro Carvalho Chehab wrote:
> > This script handle errors when trying to build translations
> > with make pdfdocs.
> >
> > As part of our cleanup work to remove hacks from docs Makefile,
> > convert this to python, preparing it to be part of a library
> > to be called by sphinx-build-wrapper.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>
> I could apply up to 05/21 of v6 and did some quick tests under
> Fedora (where Noto CJK VF fonts are installed).
>
> At 3/21, "./scripts/check-variable-fonts.sh" doesn't say a word.
>
> At 4/21, "./scripts/check-variable-fonts.py" complains:
I got a little bit confused with the above. I guess you picked the
wrong patch numbers, but yeah, there is a bisect issue, caused by
the part reorder I did moving this change to happen before adding
the script. Basically, I updated docs Makefile the wrong way.
Thanks for pointing it!
For v7 I'll ensure that all patches will properly print the suggestions
from the script.
> =============================================================================
> XeTeX is confused by "variable font" files listed below:
> /usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc
> /usr/share/fonts/google-noto-sans-mono-cjk-vf-fonts/NotoSansMonoCJK-VF.ttc
> /usr/share/fonts/google-noto-serif-cjk-vf-fonts/NotoSerifCJK-VF.ttc
>
> For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
> Or, CJK pages can be skipped by uninstalling texlive-xecjk.
>
> For more info on denylisting, other options, and variable font, see header
> comments of scripts/check-variable-fonts.py.
> =============================================================================
>
> Of course, I have followed the suggestions in the header comments.
I didn't try to follow the suggestions to solve the issue on Fedora yet.
It is on my todo list to test it.
The new script has an exact copy of the instructions of the previous one.
So, up to patch 09/21 from this series, there won't be any change at
doc build, except for the script conversion and some code cleanups
and reordering.
Patch 09/21 moves the env logic of FONTS_CONF_DENY_VF to the wrapper.
So, in thesis, fixing it before-after the series shouldn't have any
impact (I didn't test yet. Will do on my next respin). Btw, we should
probably document it at make help.
If the instructions from the header is wrong, we need to update it
on a separate patch series.
> So I have to NAK on 4/21.
>
> Regards,
> Akira
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-17 8:35 ` Akira Yokosawa
2025-09-17 8:45 ` Akira Yokosawa
@ 2025-09-17 15:11 ` Jonathan Corbet
2025-09-17 23:43 ` Akira Yokosawa
2025-09-17 15:18 ` Mauro Carvalho Chehab
2 siblings, 1 reply; 34+ messages in thread
From: Jonathan Corbet @ 2025-09-17 15:11 UTC (permalink / raw)
To: Akira Yokosawa, mchehab+huawei
Cc: linux-doc, linux-kernel, Jani Nikula, Akira Yokosawa
Akira Yokosawa <akiyks@gmail.com> writes:
> Wait! In the cover-letter, you said:
>
> It should be noticed that it is out of the scope of this series
> to change the implementation. Surely the process can be improved,
> but first let's consolidate and document everything on a single
> place.
>
> Removing current restriction on SPHINXDIRS does look inconsistent with
> your own words to me.
>
> So, I guess I have to NAK 06/21 as well.
Is there an actual problem with this change that we need to know about?
I am not quite understanding the objection here.
Thanks,
jon
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-17 8:35 ` Akira Yokosawa
2025-09-17 8:45 ` Akira Yokosawa
2025-09-17 15:11 ` Jonathan Corbet
@ 2025-09-17 15:18 ` Mauro Carvalho Chehab
2 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-17 15:18 UTC (permalink / raw)
To: Akira Yokosawa
Cc: mchehab+huawei, corbet, linux-doc, linux-kernel, Jani Nikula
On Wed, Sep 17, 2025 at 05:35:31PM +0900, Akira Yokosawa wrote:
> [+CC: Jani, -CC: rust people and list]
>
> On Tue, 16 Sep 2025 12:22:46 +0200, Mauro Carvalho Chehab wrote:
> > There are too much magic inside docs Makefile to properly run
> > sphinx-build. Create an ancillary script that contains all
> > kernel-related sphinx-build call logic currently at Makefile.
> >
> > Such script is designed to work both as an standalone command
> > and as part of a Makefile. As such, it properly handles POSIX
> > jobserver used by GNU make.
> >
> > On a side note, there was a line number increase due to the
> > conversion (ignoring comments) is:
> >
> > Documentation/Makefile | 131 +++----------
> > tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++
> > 2 files changed, 323 insertions(+), 101 deletions(-)
> >
> > Comments and descriptions adds:
> > tools/docs/sphinx-build-wrapper | 261 +++++++++++++++++++++++++++++++-
> >
> > So, about half of the script are comments/descriptions.
> >
> > This is because some things are more verbosed on Python and because
> > it requires reading env vars from Makefile. Besides it, this script
> > has some extra features that don't exist at the Makefile:
> >
> > - It can be called directly from command line;
> > - It properly return PDF build errors.
> >
> > When running the script alone, it will only take handle sphinx-build
> > targets. On other words, it won't runn make rustdoc after building
> > htmlfiles, nor it will run the extra check scripts.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> > Documentation/Makefile | 131 ++-----
> > tools/docs/sphinx-build-wrapper | 581 ++++++++++++++++++++++++++++++++
> > 2 files changed, 611 insertions(+), 101 deletions(-)
> > create mode 100755 tools/docs/sphinx-build-wrapper
> >
> > diff --git a/Documentation/Makefile b/Documentation/Makefile
> > index 7570d4cf3b13..4736f02b6c9e 100644
> > --- a/Documentation/Makefile
> > +++ b/Documentation/Makefile
> > @@ -23,21 +23,22 @@ SPHINXOPTS =
> > SPHINXDIRS = .
> > DOCS_THEME =
> > DOCS_CSS =
> > -_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
>
> Wait! In the cover-letter, you said:
>
> It should be noticed that it is out of the scope of this series
> to change the implementation. Surely the process can be improved,
> but first let's consolidate and document everything on a single
> place.
>
> Removing current restriction on SPHINXDIRS does look inconsistent with
> your own words to me.
You misread the patch. Look better some hunks below:
+# Used only on help
+_SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst)))
+
Patch doesn't remove _SPHINXDIRS. It just moves it to be closer to the
help and document that this is used only there.
>
> So, I guess I have to NAK 06/21 as well.
>
> Regards,
> Akira
>
--
Thanks,
Mauro
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python
2025-09-17 8:48 ` Mauro Carvalho Chehab
@ 2025-09-17 23:22 ` Akira Yokosawa
2025-09-18 8:51 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 34+ messages in thread
From: Akira Yokosawa @ 2025-09-17 23:22 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: corbet, linux-doc, linux-kernel, Akira Yokosawa
Hi,
On Wed, 17 Sep 2025 10:48:18 +0200, Mauro Carvalho Chehab wrote:
> Em Wed, 17 Sep 2025 10:09:05 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
>
>> On Tue, 16 Sep 2025 12:22:40 +0200, Mauro Carvalho Chehab wrote:
>>> This script handle errors when trying to build translations
>>> with make pdfdocs.
>>>
>>> As part of our cleanup work to remove hacks from docs Makefile,
>>> convert this to python, preparing it to be part of a library
>>> to be called by sphinx-build-wrapper.
>>>
>>> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>>
>> I could apply up to 05/21 of v6 and did some quick tests under
>> Fedora (where Noto CJK VF fonts are installed).
>>
>> At 3/21, "./scripts/check-variable-fonts.sh" doesn't say a word.
>>
>> At 4/21, "./scripts/check-variable-fonts.py" complains:
>
> I got a little bit confused with the above. I guess you picked the
> wrong patch numbers, but yeah, there is a bisect issue, caused by
> the part reorder I did moving this change to happen before adding
> the script. Basically, I updated docs Makefile the wrong way.
>
> Thanks for pointing it!
>
> For v7 I'll ensure that all patches will properly print the suggestions
> from the script.
>
>> =============================================================================
>> XeTeX is confused by "variable font" files listed below:
>> /usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc
>> /usr/share/fonts/google-noto-sans-mono-cjk-vf-fonts/NotoSansMonoCJK-VF.ttc
>> /usr/share/fonts/google-noto-serif-cjk-vf-fonts/NotoSerifCJK-VF.ttc
>>
>> For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
>> Or, CJK pages can be skipped by uninstalling texlive-xecjk.
>>
>> For more info on denylisting, other options, and variable font, see header
>> comments of scripts/check-variable-fonts.py.
>> =============================================================================
>>
>> Of course, I have followed the suggestions in the header comments.
>
> I didn't try to follow the suggestions to solve the issue on Fedora yet.
> It is on my todo list to test it.
>
> The new script has an exact copy of the instructions of the previous one.
>
> So, up to patch 09/21 from this series, there won't be any change at
> doc build, except for the script conversion and some code cleanups
> and reordering.
>
> Patch 09/21 moves the env logic of FONTS_CONF_DENY_VF to the wrapper.
> So, in thesis, fixing it before-after the series shouldn't have any
> impact (I didn't test yet. Will do on my next respin). Btw, we should
> probably document it at make help.
>
> If the instructions from the header is wrong, we need to update it
> on a separate patch series.
>
I have tested v7.
With v7 fully applied, it is now possible to build translations.pdf on
Fedora. Nice!
HOWEVER, running
./tools/docs/check-variable-fonts.py
still complains. I'm not sure but there might be some minor issue (typo?)
in the translation from .sh into .py ???
So I have to keep the NAK on v7's 7/24 ("scripts: check-variable-fonts.sh:
convert to Python") for the moment.
Please run the script under a terminal session and see what happens ...
Regards,
Akira
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-17 15:11 ` Jonathan Corbet
@ 2025-09-17 23:43 ` Akira Yokosawa
2025-09-18 8:24 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 34+ messages in thread
From: Akira Yokosawa @ 2025-09-17 23:43 UTC (permalink / raw)
To: Jonathan Corbet, mchehab+huawei; +Cc: linux-doc, linux-kernel, Jani Nikula
Hi Jon,
Jonathan Corbet wrote:
> Akira Yokosawa <akiyks@gmail.com> writes:
>
>> Wait! In the cover-letter, you said:
>>
>> It should be noticed that it is out of the scope of this series
>> to change the implementation. Surely the process can be improved,
>> but first let's consolidate and document everything on a single
>> place.
>>
>> Removing current restriction on SPHINXDIRS does look inconsistent with
>> your own words to me.
>>
>> So, I guess I have to NAK 06/21 as well.
>
> Is there an actual problem with this change that we need to know about?
> I am not quite understanding the objection here.
As Mauro has pointed out, and as I could not apply v6 series, I failed
to look at the whole patch.
My knee jerk reaction came from the fact that, for example,
make SPHINXDIRS=translations/zh_CN pdfdocs
won't build. This is because I didn't know such a sub-directory is
allowed (despite what "make dochelp" says) in SPHINXDIRS.
At the time I made "improvements in CJK font configs", I embedded
hacky ".. raw:: latex \kerneldocCJKoff" and others in:
Documentations/index.rst
/*/index.rst
, assuming all of those latex macros would appear in translations.tex
in the right order.
I admit it was not ideal, but I could not, and still can not, come up
with a more robust approach.
Hopefully, this explains enough for you.
Regards,
Akira
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build
2025-09-17 23:43 ` Akira Yokosawa
@ 2025-09-18 8:24 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-18 8:24 UTC (permalink / raw)
To: Akira Yokosawa; +Cc: Jonathan Corbet, linux-doc, linux-kernel, Jani Nikula
Em Thu, 18 Sep 2025 08:43:19 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:
> Hi Jon,
>
> Jonathan Corbet wrote:
> > Akira Yokosawa <akiyks@gmail.com> writes:
> >
> >> Wait! In the cover-letter, you said:
> >>
> >> It should be noticed that it is out of the scope of this series
> >> to change the implementation. Surely the process can be improved,
> >> but first let's consolidate and document everything on a single
> >> place.
> >>
> >> Removing current restriction on SPHINXDIRS does look inconsistent with
> >> your own words to me.
> >>
> >> So, I guess I have to NAK 06/21 as well.
> >
> > Is there an actual problem with this change that we need to know about?
> > I am not quite understanding the objection here.
>
> As Mauro has pointed out, and as I could not apply v6 series, I failed
> to look at the whole patch.
>
> My knee jerk reaction came from the fact that, for example,
>
> make SPHINXDIRS=translations/zh_CN pdfdocs
>
> won't build. This is because I didn't know such a sub-directory is
> allowed (despite what "make dochelp" says) in SPHINXDIRS.
The build system does support it, provided that the directory has
an index.rst file (not all subdirs have)...
>
> At the time I made "improvements in CJK font configs", I embedded
> hacky ".. raw:: latex \kerneldocCJKoff" and others in:
>
> Documentations/index.rst
> /*/index.rst
... and that ".. raw:: " entries don't depend on previous .rst files.
> , assuming all of those latex macros would appear in translations.tex
> in the right order.
>
> I admit it was not ideal, but I could not, and still can not, come up
> with a more robust approach.
For LaTeX build, ".. raw:: " entries can be unavoidable, but you could
place it either:
- at conf.py if they're global;
- on each rst file (that's what we do on media);
- in the case of translations, for the languages that require CJK.
Grepping it:
$ git grep kerneldocCJK Documentation/translations/
Documentation/translations/index.rst: \kerneldocCJKoff
Documentation/translations/it_IT/index.rst: \kerneldocCJKoff
Documentation/translations/ja_JP/index.rst: \kerneldocCJKon
Documentation/translations/ko_KR/index.rst: \kerneldocCJKon
Documentation/translations/ko_KR/process/howto.rst: \kerneldocCJKoff
Documentation/translations/ko_KR/process/howto.rst: \kerneldocCJKon
Documentation/translations/sp_SP/index.rst: \kerneldocCJKoff
Documentation/translations/zh_CN/index.rst: \kerneldocCJKon
Documentation/translations/zh_TW/index.rst: \kerneldocCJKon
Indeed it assumes that translations/index.rst will be the last one,
as it is needed to disable \kerneldocCJKoff.
What I would do is move \kerneldocCJK to each book, e.g.:
zh_CN/index: will have a \kerneldocCJK{on/off} pair;
zh_TW/index: will have a \kerneldocCJK{on/off} pair;
it_IT/index: won't use it, as it doesn't need CJK fonts;
ko_KR/index: will have a \kerneldocCJK{on/off} pair;
ja_JP/index: will have a \kerneldocCJK{on/off} pair;
sp_SP/index: will have a \kerneldocCJK{on/off} pair;
process/index: won't use it, as everything there is in English;
This would likely allow creating each translation on separate books
like:
make SPHINXDIRS="translations/zh_CN translations/ko_KR ..." pdfdocs
Heh, the audience for each language is completely different, so
merging them altogether is actually weird. This doesn't matter
much for html output, but for all other outputs, ideally each
translation should be a separate book.
With the current Makefile-hacky-based-approach, supporting separate
build books would be very complex, but with a wrapper containing the
entire building logic, it doesn't sound hard to add support in the
future to build translations as separate entities.=
Heh, when we added Sphinx support, we have a single Documentation
directory, but now we have multiple ones:
$ find . -name Documentation
./tools/bpf/bpftool/Documentation
./tools/perf/Documentation
./tools/memory-model/Documentation
./tools/lib/perf/Documentation
./tools/objtool/Documentation
./tools/build/Documentation
./Documentation
./drivers/staging/most/Documentation
./drivers/staging/greybus/Documentation
./drivers/staging/iio/Documentation
Considering that, and considering the some of the above books can
be in ReST format, it doesn't sound too complex to add a --docdir
parameter at sphinx-build-wrapper and do things like:
./tools/docs/sphinx-build-wrapper --docdir Documentation/translations/zh_CN htmldocs
./tools/docs/sphinx-build-wrapper --docdir Documentation/translations/zh_TW epubdocs
./tools/docs/sphinx-build-wrapper --docdir ./tools/bpf/bpftool/Documentation mandocs
On such scenario, we likely need intersphinx, as translation books contain lots of
references pointing to the English one.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python
2025-09-17 23:22 ` Akira Yokosawa
@ 2025-09-18 8:51 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 34+ messages in thread
From: Mauro Carvalho Chehab @ 2025-09-18 8:51 UTC (permalink / raw)
To: Akira Yokosawa; +Cc: corbet, linux-doc, linux-kernel
Em Thu, 18 Sep 2025 08:22:44 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:
> Hi,
>
> On Wed, 17 Sep 2025 10:48:18 +0200, Mauro Carvalho Chehab wrote:
> > Em Wed, 17 Sep 2025 10:09:05 +0900
> > Akira Yokosawa <akiyks@gmail.com> escreveu:
> >
> >> On Tue, 16 Sep 2025 12:22:40 +0200, Mauro Carvalho Chehab wrote:
> >>> This script handle errors when trying to build translations
> >>> with make pdfdocs.
> >>>
> >>> As part of our cleanup work to remove hacks from docs Makefile,
> >>> convert this to python, preparing it to be part of a library
> >>> to be called by sphinx-build-wrapper.
> >>>
> >>> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> >>
> >> I could apply up to 05/21 of v6 and did some quick tests under
> >> Fedora (where Noto CJK VF fonts are installed).
> >>
> >> At 3/21, "./scripts/check-variable-fonts.sh" doesn't say a word.
> >>
> >> At 4/21, "./scripts/check-variable-fonts.py" complains:
> >
> > I got a little bit confused with the above. I guess you picked the
> > wrong patch numbers, but yeah, there is a bisect issue, caused by
> > the part reorder I did moving this change to happen before adding
> > the script. Basically, I updated docs Makefile the wrong way.
> >
> > Thanks for pointing it!
> >
> > For v7 I'll ensure that all patches will properly print the suggestions
> > from the script.
> >
> >> =============================================================================
> >> XeTeX is confused by "variable font" files listed below:
> >> /usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc
> >> /usr/share/fonts/google-noto-sans-mono-cjk-vf-fonts/NotoSansMonoCJK-VF.ttc
> >> /usr/share/fonts/google-noto-serif-cjk-vf-fonts/NotoSerifCJK-VF.ttc
> >>
> >> For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
> >> Or, CJK pages can be skipped by uninstalling texlive-xecjk.
> >>
> >> For more info on denylisting, other options, and variable font, see header
> >> comments of scripts/check-variable-fonts.py.
> >> =============================================================================
> >>
> >> Of course, I have followed the suggestions in the header comments.
> >
> > I didn't try to follow the suggestions to solve the issue on Fedora yet.
> > It is on my todo list to test it.
> >
> > The new script has an exact copy of the instructions of the previous one.
> >
> > So, up to patch 09/21 from this series, there won't be any change at
> > doc build, except for the script conversion and some code cleanups
> > and reordering.
> >
> > Patch 09/21 moves the env logic of FONTS_CONF_DENY_VF to the wrapper.
> > So, in thesis, fixing it before-after the series shouldn't have any
> > impact (I didn't test yet. Will do on my next respin). Btw, we should
> > probably document it at make help.
> >
> > If the instructions from the header is wrong, we need to update it
> > on a separate patch series.
> >
>
> I have tested v7.
>
> With v7 fully applied, it is now possible to build translations.pdf on
> Fedora. Nice!
>
> HOWEVER, running
>
> ./tools/docs/check-variable-fonts.py
>
> still complains. I'm not sure but there might be some minor issue (typo?)
> in the translation from .sh into .py ???
>
> So I have to keep the NAK on v7's 7/24 ("scripts: check-variable-fonts.sh:
> convert to Python") for the moment.
>
> Please run the script under a terminal session and see what happens ...
Yeah, there was a typo there. I fixed it for the next respin.
To better allow running it manually, I'm adding to the tool a new command
line argument:
$ tools/docs/check-variable-fonts.py -h
...
options:
-h, --help show this help message and exit
--deny-vf DENY_VF XDG_CONFIG_HOME dir containing fontconfig/fonts.conf file
And changed the class __init__ logic to optionally use it:
def __init__(self, deny_vf=None):
if not deny_vf:
deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf")
This way, it will keep using FONTS_CONF_DENY_VF (defaulting to ~/deny-vf),
yet allowing it to be overriden via command line:
$ tools/docs/check-variable-fonts.py
<no output>
$ tools/docs/check-variable-fonts.py --deny-vf ~/deny-vf/
<no output>
$ tools/docs/check-variable-fonts.py --deny-vf ~/dont-deny-vf/
=============================================================================
XeTeX is confused by "variable font" files listed below:
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc
/usr/share/fonts/google-noto-sans-mono-cjk-vf-fonts/NotoSansMonoCJK-VF.ttc
/usr/share/fonts/google-noto-serif-cjk-vf-fonts/NotoSerifCJK-VF.ttc
For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.
Or, CJK pages can be skipped by uninstalling texlive-xecjk.
For more info on denylisting, other options, and variable font, run:
tools/docs/check-variable-fonts.py -h
=============================================================================
Thanks,
Mauro
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2025-09-18 8:51 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-16 10:22 [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 01/21] scripts/jobserver-exec: move the code to a class Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 02/21] scripts/jobserver-exec: move its class to the lib directory Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 03/21] scripts/jobserver-exec: add a help message Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 04/21] scripts: check-variable-fonts.sh: convert to Python Mauro Carvalho Chehab
2025-09-17 1:09 ` Akira Yokosawa
2025-09-17 8:48 ` Mauro Carvalho Chehab
2025-09-17 23:22 ` Akira Yokosawa
2025-09-18 8:51 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 05/21] tools/docs: check-variable-fonts.py: split into a lib and an exec file Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 06/21] scripts: sphinx-pre-install: move it to tools/docs Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 07/21] tools/docs: python_version: move version check from sphinx-pre-install Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 08/21] tools/docs: python_version: drop a debug print Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 09/21] tools/docs: python_version: allow check for alternatives and bail out Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 10/21] tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build Mauro Carvalho Chehab
2025-09-17 8:35 ` Akira Yokosawa
2025-09-17 8:45 ` Akira Yokosawa
2025-09-17 15:11 ` Jonathan Corbet
2025-09-17 23:43 ` Akira Yokosawa
2025-09-18 8:24 ` Mauro Carvalho Chehab
2025-09-17 15:18 ` Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 11/21] docs: parallel-wrapper.sh: remove script Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 12/21] docs: Makefile: document latex/PDF PAPER= parameter Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 13/21] tools/docs: sphinx-build-wrapper: add an argument for LaTeX interactive mode Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 14/21] tools/docs,scripts: sphinx-*: prevent sphinx-build crashes Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 15/21] tools/docs: sphinx-build-wrapper: allow building PDF files in parallel Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 16/21] tools/docs: sphinx-build-wrapper: Fix output for duplicated names Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 17/21] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 18/21] tools: kernel-doc: add a see also section at man pages Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 19/21] scripts: kdoc_parser.py: warn about Python version only once Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 20/21] tools/docs: sphinx-* break documentation bulds on openSUSE Mauro Carvalho Chehab
2025-09-16 10:22 ` [PATCH v6 21/21] tools/docs: sphinx-build-wrapper: add support to run inside venv Mauro Carvalho Chehab
2025-09-16 10:31 ` [PATCH v6 00/21] Split sphinx call logic from docs Makefile Mauro Carvalho Chehab
2025-09-16 15:47 ` Jonathan Corbet
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).