Re: [Buildroot] [PATCH 1/1] package/fwts: disable parallel build

From: Julien Olivain <ju.o@free.fr>
To: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: buildroot@buildroot.org
Subject: Re: [Buildroot] [PATCH 1/1] package/fwts: disable parallel build
Date: Thu, 02 May 2024 12:02:36 +0200	[thread overview]
Message-ID: <df3e5518e98e13c79a52bf3e9fdccb38@free.fr> (raw)
In-Reply-To: <f5ecc25e82957dfe5938eb52af816eaf@free.fr>

Hi again Thomas,

On 01/05/2024 22:27, Julien Olivain wrote:
> Hi Thomas,
> 
> On 01/05/2024 21:55, Thomas Petazzoni wrote:
>> On Sat, 20 Apr 2024 15:22:51 +0200
>> Julien Olivain <ju.o@free.fr> wrote:
>> 
>>> fwts randomly fail to build in parallel with error:
>>> 
>>>     mv: cannot stat 'dtcompilerparser.tab.c': No such file or 
>>> directory
>>>     mv: cannot stat 'prparser.tab.c': No such file or directory
>>> 
>>> This is a known upstream issue. See:
>>> https://github.com/fwts/fwts/issues/7
>>> 
>>> This commit workaround this issue by disabling parallel builds.
>>> 
>>> Signed-off-by: Julien Olivain <ju.o@free.fr>
>>> ---
>>>  package/fwts/fwts.mk | 3 +++
>>>  1 file changed, 3 insertions(+)
>> 
>> Hm, I don't see any failures in the autobuilders. Did you encounter
>> this issue personally? Why don't we see it in the autobuilders?
> 
> On my side, I see those failures on systems with with 20 cores and more 
> on an
> idle system doing only the Buildroot build. Maybe the autobuilders have 
> a
> different load, or less cores?

After testing a bit more, it appears the build always fail with
GNU make 4.4.1. Testing in the Buildroot docker reference image with
the same conditions (20+ cores, idle system), and which includes GNU
make 4.3 does not fail.

Looking at the host load, it seems GNU make 4.3 serializes some part of
the build, whereas GNU make 4.4.1 parallelize most of the build (and 
fail).

FWTS uses the .NOTPARALLEL special target with prerequisites:
https://github.com/fwts/fwts/blob/V24.03.00/src/libfwtsiasl/Makefile.am#L67

It seems the two versions of GNU make are not honoring this the same 
way.
It also seems the .NOTPARALLEL _with_ prerequisites is a feature added
in GNU make 4.4. See:
https://git.savannah.gnu.org/cgit/make.git/commit/?id=f6ea899d83bf00fe9201fde0ca9cf7af8e443677
https://lists.gnu.org/archive/html/help-make/2022-10/msg00020.html

Finally, the FWTS Makefile rules might also be incorrect, at:
https://github.com/fwts/fwts/blob/V24.03.00/src/libfwtsiasl/Makefile.am#L67

Quoting the Make manual:
https://www.gnu.org/software/make/manual/html_node/Parallel-Disable.html
"""
If the .NOTPARALLEL special target has prerequisites, then each of those
prerequisites will be considered a target and all prerequisites of these
targets will be run serially.
"""

Note the serialization will happen on the prerequisites of the target
set as prerequisites of .NOTPARALLEL.

My current understanding is that GNU Make 4.3 just ignores the
prerequisites of the .NOTPARALLEL and interprets it as if there
was none, which makes it serialize a bigger part of the build.

With this analysis, I may have an idea of a patch in the FWTS
Makefile.am. I'll do some more tests and propose a new patch.

Best regards,

Julien.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot