From: Petri Latvala <petri.latvala@intel.com>
To: Daniel Stone <daniels@collabora.com>
Cc: igt-dev@lists.freedesktop.org
Subject: Re: [igt-dev] [PATCH] ci: Use FDO_CI_CONCURRENT to limit build/test concurrency
Date: Thu, 29 Jul 2021 17:46:33 +0300 [thread overview]
Message-ID: <YQK/SeM3HliYlOna@platvala-desk.ger.corp.intel.com> (raw)
In-Reply-To: <20210729143755.349932-1-daniels@collabora.com>
On Thu, Jul 29, 2021 at 03:37:55PM +0100, Daniel Stone wrote:
> ${FDO_CI_CONCURRENT} is set to an appropriate concurrency on our shared
> runners. Using all available CPUs will just DoS the world and would
> explain a lot about the timeouts.
>
> Signed-off-by: Daniel Stone <daniels@collabora.com>
> ---
> .gitlab-ci.yml | 38 +++++++++++++++++++-------------------
> 1 file changed, 19 insertions(+), 19 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 3b0e98ae..b5edfee8 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -64,9 +64,9 @@ build:tests-fedora:
> stage: build
> script:
> - meson --prefix=/opt/igt $MESON_OPTIONS build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> - DESTDIR="$PWD/installdir" ninja -C build install
> - - ninja -C build igt-gpu-tools-doc
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} igt-gpu-tools-doc
> artifacts:
> paths:
> - build
> @@ -77,7 +77,7 @@ build:tests-fedora-no-libunwind:
> script:
> - dnf remove -y libunwind-devel
> - meson -Dlibunwind=disabled build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
>
> # documentation building is different with meson<0.47
> build:tests-fedora-oldest-meson:
> @@ -86,15 +86,15 @@ build:tests-fedora-oldest-meson:
> - dnf remove -y meson
> - pip install /usr/src/meson-*.tar.gz
> - meson $MESON_OPTIONS build
> - - ninja -C build || ninja -C build -j 1
> - - ninja -C build igt-gpu-tools-doc
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} igt-gpu-tools-doc
>
> # ensure igt_nouveau stubs build when nouveau support for libdrm is disabled
> build:tests-fedora-no-libdrm-nouveau:
> stage: build
> script:
> - meson -Dlibdrm_drivers= build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
>
> build:tests-fedora-clang:
> stage: build
> @@ -102,7 +102,7 @@ build:tests-fedora-clang:
> CC: clang
> script:
> - meson $MESON_OPTIONS build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> artifacts:
> paths:
> - build
> @@ -112,7 +112,7 @@ build:tests-debian-meson:
> stage: build
> script:
> - meson $MESON_OPTIONS build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> - cp build/tests/test-list.txt meson-test-list.txt
> artifacts:
> paths:
> @@ -123,7 +123,7 @@ build:tests-debian-minimal:
> stage: build
> script:
> - meson -Dtests=disabled -Dlibdrm_drivers="" build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> artifacts:
> paths:
> - build
> @@ -134,7 +134,7 @@ build:tests-debian-meson-armhf:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig/
> - meson --cross-file meson-cross-armhf.txt build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> artifacts:
> paths:
> - build
> @@ -145,7 +145,7 @@ build:tests-debian-meson-arm64:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig/
> - meson --cross-file meson-cross-arm64.txt build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> artifacts:
> paths:
> - build
> @@ -156,7 +156,7 @@ build:tests-debian-meson-mips:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/mips-linux-gnu/pkgconfig/
> - meson --cross-file meson-cross-mips.txt build
> - - ninja -C build || ninja -C build -j 1
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> artifacts:
> paths:
> - build
> @@ -168,7 +168,7 @@ test:ninja-test:
> dependencies:
> - build:tests-fedora
> stage: test
> - script: ninja -C build test
> + script: meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
usage: meson [-h] {setup,configure,dist,install,introspect,init,test,wrap,subprojects,help,rewrite,compile} ...
meson: error: unrecognized arguments: -C --num-processes 4
Doesn't ninja -C build -j $... test work for this?
--
Petri Latvala
>
> test:ninja-test-clang:
> retry: 2
> @@ -177,7 +177,7 @@ test:ninja-test-clang:
> variables:
> CC: clang
> stage: test
> - script: ninja -C build test
> + script: meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
>
> test:ninja-test-minimal:
> retry: 2
> @@ -185,7 +185,7 @@ test:ninja-test-minimal:
> dependencies:
> - build:tests-debian-minimal
> stage: test
> - script: ninja -C build test
> + script: meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
>
> test:ninja-test-arm64:
> retry: 2
> @@ -196,7 +196,7 @@ test:ninja-test-arm64:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig/
> - env > build/envdump.txt
> - - ninja -C build test
> + - meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
> artifacts:
> paths:
> - build
> @@ -211,7 +211,7 @@ test:ninja-test-armhf:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig/
> - env > build/envdump.txt
> - - ninja -C build test
> + - meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
> artifacts:
> paths:
> - build
> @@ -226,7 +226,7 @@ test:ninja-test-mips:
> script:
> - export PKG_CONFIG_PATH=/usr/lib/mips-linux-gnu/pkgconfig/
> - env > build/envdump.txt
> - - ninja -C build test
> + - meson -C build test --num-processes ${FDO_CI_CONCURRENT:-4}
> artifacts:
> paths:
> - build
> @@ -255,7 +255,7 @@ pages:
> - build:tests-fedora
> stage: deploy
> script:
> - - ninja -C build igt-gpu-tools-doc
> + - ninja -C build -j${FDO_CI_CONCURRENT:-4} igt-gpu-tools-doc
> - cp -r build/docs/reference/igt-gpu-tools/html public
> artifacts:
> paths:
> --
> 2.31.1
>
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
next prev parent reply other threads:[~2021-07-29 14:44 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-29 14:37 [igt-dev] [PATCH] ci: Use FDO_CI_CONCURRENT to limit build/test concurrency Daniel Stone
2021-07-29 14:46 ` Petri Latvala [this message]
2021-07-29 14:51 ` Petri Latvala
2021-07-29 14:55 ` Daniel Stone
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YQK/SeM3HliYlOna@platvala-desk.ger.corp.intel.com \
--to=petri.latvala@intel.com \
--cc=daniels@collabora.com \
--cc=igt-dev@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.