From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
To: Linux Doc Mailing List <linux-doc@vger.kernel.org>,
Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>,
"Akira Yokosawa" <akiyks@gmail.com>,
"Breno Leitao" <leitao@debian.org>,
"David S. Miller" <davem@davemloft.net>,
"Donald Hunter" <donald.hunter@gmail.com>,
"Eric Dumazet" <edumazet@google.com>,
"Ignacio Encinas Rubio" <ignacio@iencinas.com>,
"Jan Stancek" <jstancek@redhat.com>,
"Marco Elver" <elver@google.com>,
"Mauro Carvalho Chehab" <mchehab+huawei@kernel.org>,
"Paolo Abeni" <pabeni@redhat.com>,
"Randy Dunlap" <rdunlap@infradead.org>,
"Ruben Wauters" <rubenru09@aol.com>,
"Shuah Khan" <skhan@linuxfoundation.org>,
joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev,
linux-kernel@vger.kernel.org, lkmm@lists.linux.dev,
netdev@vger.kernel.org, peterz@infradead.org,
stern@rowland.harvard.edu
Subject: [PATCH v9 14/13] sphinx: parser_yaml.py: fix line numbers information
Date: Fri, 11 Jul 2025 10:36:23 +0200 [thread overview]
Message-ID: <4d1e0f5283ae1c6874cef272c5760035eb51278a.1752222934.git.mchehab+huawei@kernel.org> (raw)
In-Reply-To: <cover.1752076293.git.mchehab+huawei@kernel.org>
As reported by Donald, this code:
rst_parser = RSTParser()
rst_parser.parse('\n'.join(result), document)
breaks line parsing. As an alternative, I tested a variant of it:
rst_parser.parse(result, document)
but still line number was not preserved. As Donald noted,
standard Parser classes don't have a direct mechanism to preserve
line numbers from ViewList().
So, instead, let's use a mechanism similar to what we do already at
kerneldoc.py: call the statemachine mechanism directly there.
I double-checked when states and statemachine were introduced:
both were back in 2002. I also tested doc build with docutils 0.16
and 0.21.2. It worked with both, so it seems to be stable enough
for our needs.
Reported-by: Donald Hunter <donald.hunter@gmail.com>
Closes: https://lore.kernel.org/linux-doc/m24ivk78ng.fsf@gmail.com/T/#u
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
PS.: I'm opting to send this as 14/13 to avoid respanning the entire
series again just due to this extra change.
Documentation/sphinx/parser_yaml.py | 21 ++++++++++++++-------
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
index 1602b31f448e..634d84a202fc 100755
--- a/Documentation/sphinx/parser_yaml.py
+++ b/Documentation/sphinx/parser_yaml.py
@@ -11,7 +11,9 @@ import sys
from pprint import pformat
+from docutils import statemachine
from docutils.parsers.rst import Parser as RSTParser
+from docutils.parsers.rst import states
from docutils.statemachine import ViewList
from sphinx.util import logging
@@ -56,6 +58,8 @@ class YamlParser(Parser):
re_lineno = re.compile(r"\.\. LINENO ([0-9]+)$")
+ tab_width = 8
+
def rst_parse(self, inputstring, document, msg):
"""
Receives a ReST content that was previously converted by the
@@ -66,10 +70,18 @@ class YamlParser(Parser):
result = ViewList()
+ self.statemachine = states.RSTStateMachine(state_classes=states.state_classes,
+ initial_state='Body',
+ debug=document.reporter.debug_flag)
+
try:
# Parse message with RSTParser
lineoffset = 0;
- for line in msg.split('\n'):
+
+ lines = statemachine.string2lines(msg, self.tab_width,
+ convert_whitespace=True)
+
+ for line in lines:
match = self.re_lineno.match(line)
if match:
lineoffset = int(match.group(1))
@@ -77,12 +89,7 @@ class YamlParser(Parser):
result.append(line, document.current_source, lineoffset)
- # Fix backward compatibility with docutils < 0.17.1
- if "tab_width" not in vars(document.settings):
- document.settings.tab_width = 8
-
- rst_parser = RSTParser()
- rst_parser.parse('\n'.join(result), document)
+ self.statemachine.run(result, document)
except Exception as e:
document.reporter.error("YAML parsing error: %s" % pformat(e))
--
2.50.0
prev parent reply other threads:[~2025-07-11 8:37 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-09 15:58 [PATCH v9 00/13] Don't generate netlink .rst files inside $(srctree) Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 01/13] docs: netlink: netlink-raw.rst: use :ref: instead of :doc: Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 02/13] tools: ynl_gen_rst.py: Split library from command line tool Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 03/13] docs: netlink: index.rst: add a netlink index file Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 04/13] tools: ynl_gen_rst.py: cleanup coding style Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 05/13] docs: sphinx: add a parser for yaml files for Netlink specs Mauro Carvalho Chehab
2025-07-10 8:27 ` Donald Hunter
2025-07-09 15:58 ` [PATCH v9 06/13] docs: use parser_yaml extension to handle " Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 07/13] docs: uapi: netlink: update netlink specs link Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 08/13] tools: ynl_gen_rst.py: drop support for generating index files Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 09/13] docs: netlink: remove obsolete .gitignore from unused directory Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 10/13] MAINTAINERS: add netlink_yml_parser.py to linux-doc Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 11/13] tools: netlink_yml_parser.py: add line numbers to parsed data Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 12/13] docs: parser_yaml.py: add support for line numbers from the parser Mauro Carvalho Chehab
2025-07-10 11:25 ` Donald Hunter
2025-07-10 14:25 ` Donald Hunter
2025-07-10 17:57 ` Mauro Carvalho Chehab
2025-07-11 9:51 ` Donald Hunter
2025-07-11 17:40 ` Mauro Carvalho Chehab
2025-07-09 15:58 ` [PATCH v9 13/13] docs: parser_yaml.py: fix backward compatibility with old docutils Mauro Carvalho Chehab
2025-07-11 8:36 ` Mauro Carvalho Chehab [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4d1e0f5283ae1c6874cef272c5760035eb51278a.1752222934.git.mchehab+huawei@kernel.org \
--to=mchehab+huawei@kernel.org \
--cc=akiyks@gmail.com \
--cc=corbet@lwn.net \
--cc=davem@davemloft.net \
--cc=donald.hunter@gmail.com \
--cc=edumazet@google.com \
--cc=elver@google.com \
--cc=ignacio@iencinas.com \
--cc=joel@joelfernandes.org \
--cc=jstancek@redhat.com \
--cc=leitao@debian.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel-mentees@lists.linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=lkmm@lists.linux.dev \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rubenru09@aol.com \
--cc=skhan@linuxfoundation.org \
--cc=stern@rowland.harvard.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).