* [Buildroot] [PATCH 1/1] package/fwts: disable parallel build
@ 2024-04-20 13:22 Julien Olivain
2024-05-01 19:55 ` Thomas Petazzoni via buildroot
0 siblings, 1 reply; 4+ messages in thread
From: Julien Olivain @ 2024-04-20 13:22 UTC (permalink / raw)
To: buildroot; +Cc: Julien Olivain
fwts randomly fail to build in parallel with error:
mv: cannot stat 'dtcompilerparser.tab.c': No such file or directory
mv: cannot stat 'prparser.tab.c': No such file or directory
This is a known upstream issue. See:
https://github.com/fwts/fwts/issues/7
This commit workaround this issue by disabling parallel builds.
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
package/fwts/fwts.mk | 3 +++
1 file changed, 3 insertions(+)
diff --git a/package/fwts/fwts.mk b/package/fwts/fwts.mk
index f06609777db..cf3243c437a 100644
--- a/package/fwts/fwts.mk
+++ b/package/fwts/fwts.mk
@@ -14,6 +14,9 @@ FWTS_AUTORECONF = YES
FWTS_DEPENDENCIES = host-bison host-flex host-pkgconf libglib2 libbsd \
$(if $(BR2_PACKAGE_BASH_COMPLETION),bash-completion) \
$(if $(BR2_PACKAGE_DTC),dtc)
+# fwts randomly fail to build in parallel
+# see: https://github.com/fwts/fwts/issues/7
+FWTS_MAKE = $(MAKE1)
ifeq ($(BR2_OPTIMIZE_0),y)
FWTS_CONF_ENV = CFLAGS="$(TARGET_CFLAGS) -O1"
--
2.44.0
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/fwts: disable parallel build
2024-04-20 13:22 [Buildroot] [PATCH 1/1] package/fwts: disable parallel build Julien Olivain
@ 2024-05-01 19:55 ` Thomas Petazzoni via buildroot
2024-05-01 20:27 ` Julien Olivain
0 siblings, 1 reply; 4+ messages in thread
From: Thomas Petazzoni via buildroot @ 2024-05-01 19:55 UTC (permalink / raw)
To: Julien Olivain; +Cc: buildroot
On Sat, 20 Apr 2024 15:22:51 +0200
Julien Olivain <ju.o@free.fr> wrote:
> fwts randomly fail to build in parallel with error:
>
> mv: cannot stat 'dtcompilerparser.tab.c': No such file or directory
> mv: cannot stat 'prparser.tab.c': No such file or directory
>
> This is a known upstream issue. See:
> https://github.com/fwts/fwts/issues/7
>
> This commit workaround this issue by disabling parallel builds.
>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
> package/fwts/fwts.mk | 3 +++
> 1 file changed, 3 insertions(+)
Hm, I don't see any failures in the autobuilders. Did you encounter
this issue personally? Why don't we see it in the autobuilders?
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/fwts: disable parallel build
2024-05-01 19:55 ` Thomas Petazzoni via buildroot
@ 2024-05-01 20:27 ` Julien Olivain
2024-05-02 10:02 ` Julien Olivain
0 siblings, 1 reply; 4+ messages in thread
From: Julien Olivain @ 2024-05-01 20:27 UTC (permalink / raw)
To: Thomas Petazzoni; +Cc: buildroot
Hi Thomas,
On 01/05/2024 21:55, Thomas Petazzoni wrote:
> On Sat, 20 Apr 2024 15:22:51 +0200
> Julien Olivain <ju.o@free.fr> wrote:
>
>> fwts randomly fail to build in parallel with error:
>>
>> mv: cannot stat 'dtcompilerparser.tab.c': No such file or
>> directory
>> mv: cannot stat 'prparser.tab.c': No such file or directory
>>
>> This is a known upstream issue. See:
>> https://github.com/fwts/fwts/issues/7
>>
>> This commit workaround this issue by disabling parallel builds.
>>
>> Signed-off-by: Julien Olivain <ju.o@free.fr>
>> ---
>> package/fwts/fwts.mk | 3 +++
>> 1 file changed, 3 insertions(+)
>
> Hm, I don't see any failures in the autobuilders. Did you encounter
> this issue personally? Why don't we see it in the autobuilders?
On my side, I see those failures on systems with with 20 cores and more
on an
idle system doing only the Buildroot build. Maybe the autobuilders have
a
different load, or less cores?
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
Best regards,
Julien.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Buildroot] [PATCH 1/1] package/fwts: disable parallel build
2024-05-01 20:27 ` Julien Olivain
@ 2024-05-02 10:02 ` Julien Olivain
0 siblings, 0 replies; 4+ messages in thread
From: Julien Olivain @ 2024-05-02 10:02 UTC (permalink / raw)
To: Thomas Petazzoni; +Cc: buildroot
Hi again Thomas,
On 01/05/2024 22:27, Julien Olivain wrote:
> Hi Thomas,
>
> On 01/05/2024 21:55, Thomas Petazzoni wrote:
>> On Sat, 20 Apr 2024 15:22:51 +0200
>> Julien Olivain <ju.o@free.fr> wrote:
>>
>>> fwts randomly fail to build in parallel with error:
>>>
>>> mv: cannot stat 'dtcompilerparser.tab.c': No such file or
>>> directory
>>> mv: cannot stat 'prparser.tab.c': No such file or directory
>>>
>>> This is a known upstream issue. See:
>>> https://github.com/fwts/fwts/issues/7
>>>
>>> This commit workaround this issue by disabling parallel builds.
>>>
>>> Signed-off-by: Julien Olivain <ju.o@free.fr>
>>> ---
>>> package/fwts/fwts.mk | 3 +++
>>> 1 file changed, 3 insertions(+)
>>
>> Hm, I don't see any failures in the autobuilders. Did you encounter
>> this issue personally? Why don't we see it in the autobuilders?
>
> On my side, I see those failures on systems with with 20 cores and more
> on an
> idle system doing only the Buildroot build. Maybe the autobuilders have
> a
> different load, or less cores?
After testing a bit more, it appears the build always fail with
GNU make 4.4.1. Testing in the Buildroot docker reference image with
the same conditions (20+ cores, idle system), and which includes GNU
make 4.3 does not fail.
Looking at the host load, it seems GNU make 4.3 serializes some part of
the build, whereas GNU make 4.4.1 parallelize most of the build (and
fail).
FWTS uses the .NOTPARALLEL special target with prerequisites:
https://github.com/fwts/fwts/blob/V24.03.00/src/libfwtsiasl/Makefile.am#L67
It seems the two versions of GNU make are not honoring this the same
way.
It also seems the .NOTPARALLEL _with_ prerequisites is a feature added
in GNU make 4.4. See:
https://git.savannah.gnu.org/cgit/make.git/commit/?id=f6ea899d83bf00fe9201fde0ca9cf7af8e443677
https://lists.gnu.org/archive/html/help-make/2022-10/msg00020.html
Finally, the FWTS Makefile rules might also be incorrect, at:
https://github.com/fwts/fwts/blob/V24.03.00/src/libfwtsiasl/Makefile.am#L67
Quoting the Make manual:
https://www.gnu.org/software/make/manual/html_node/Parallel-Disable.html
"""
If the .NOTPARALLEL special target has prerequisites, then each of those
prerequisites will be considered a target and all prerequisites of these
targets will be run serially.
"""
Note the serialization will happen on the prerequisites of the target
set as prerequisites of .NOTPARALLEL.
My current understanding is that GNU Make 4.3 just ignores the
prerequisites of the .NOTPARALLEL and interprets it as if there
was none, which makes it serialize a bigger part of the build.
With this analysis, I may have an idea of a patch in the FWTS
Makefile.am. I'll do some more tests and propose a new patch.
Best regards,
Julien.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-05-02 10:02 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-20 13:22 [Buildroot] [PATCH 1/1] package/fwts: disable parallel build Julien Olivain
2024-05-01 19:55 ` Thomas Petazzoni via buildroot
2024-05-01 20:27 ` Julien Olivain
2024-05-02 10:02 ` Julien Olivain
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.