* [PATCH] oeqa/ptest: print a warning if ptests failed
@ 2020-12-19 17:29 Alexander Kanavin
2020-12-19 23:56 ` [OE-core] " Richard Purdie
0 siblings, 1 reply; 3+ messages in thread
From: Alexander Kanavin @ 2020-12-19 17:29 UTC (permalink / raw
To: openembedded-core; +Cc: Alexander Kanavin
This allows spotting ptest regressions without having hard ptest failures
(for that full ptest stability should be achieved).
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
meta/lib/oeqa/runtime/cases/ptest.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
index a9572c81f0..0800f3c27f 100644
--- a/meta/lib/oeqa/runtime/cases/ptest.py
+++ b/meta/lib/oeqa/runtime/cases/ptest.py
@@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase):
failmsg = failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests)
if failmsg:
+ self.logger.warning("There were failing ptests.")
self.fail(failmsg)
--
2.29.2
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [OE-core] [PATCH] oeqa/ptest: print a warning if ptests failed
2020-12-19 17:29 [PATCH] oeqa/ptest: print a warning if ptests failed Alexander Kanavin
@ 2020-12-19 23:56 ` Richard Purdie
2020-12-20 9:34 ` Alexander Kanavin
0 siblings, 1 reply; 3+ messages in thread
From: Richard Purdie @ 2020-12-19 23:56 UTC (permalink / raw
To: Alexander Kanavin, openembedded-core; +Cc: Jon Mason, Ross Burton
On Sat, 2020-12-19 at 18:29 +0100, Alexander Kanavin wrote:
> This allows spotting ptest regressions without having hard ptest failures
> (for that full ptest stability should be achieved).
>
> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> ---
> meta/lib/oeqa/runtime/cases/ptest.py | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
> index a9572c81f0..0800f3c27f 100644
> --- a/meta/lib/oeqa/runtime/cases/ptest.py
> +++ b/meta/lib/oeqa/runtime/cases/ptest.py
> @@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase):
> failmsg = failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests)
>
> if failmsg:
> + self.logger.warning("There were failing ptests.")
> self.fail(failmsg)
Have you looked at this on the arm worker? That seems to have four
failing tests over three recipes:
{'libinput': ['libinput-test-suite'],
'strace': ['qual_fault-syscall.test', 'qual_fault.test'],
'valgrind': ['drd/tests/std_list']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1355
Not sure if these always fail or are intermittent?
Cheers,
Richard
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [OE-core] [PATCH] oeqa/ptest: print a warning if ptests failed
2020-12-19 23:56 ` [OE-core] " Richard Purdie
@ 2020-12-20 9:34 ` Alexander Kanavin
0 siblings, 0 replies; 3+ messages in thread
From: Alexander Kanavin @ 2020-12-20 9:34 UTC (permalink / raw
To: Richard Purdie; +Cc: OE-core, Jon Mason, Ross Burton
[-- Attachment #1: Type: text/plain, Size: 1777 bytes --]
On Sun, 20 Dec 2020 at 00:56, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:
> On Sat, 2020-12-19 at 18:29 +0100, Alexander Kanavin wrote:
> > This allows spotting ptest regressions without having hard ptest failures
> > (for that full ptest stability should be achieved).
> >
> > Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> > ---
> > meta/lib/oeqa/runtime/cases/ptest.py | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/lib/oeqa/runtime/cases/ptest.py
> b/meta/lib/oeqa/runtime/cases/ptest.py
> > index a9572c81f0..0800f3c27f 100644
> > --- a/meta/lib/oeqa/runtime/cases/ptest.py
> > +++ b/meta/lib/oeqa/runtime/cases/ptest.py
> > @@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase):
> > failmsg = failmsg + "Failed ptests:\n%s" %
> pprint.pformat(failed_tests)
> >
> > if failmsg:
> > + self.logger.warning("There were failing ptests.")
> > self.fail(failmsg)
>
> Have you looked at this on the arm worker? That seems to have four
> failing tests over three recipes:
>
> {'libinput': ['libinput-test-suite'],
> 'strace': ['qual_fault-syscall.test', 'qual_fault.test'],
> 'valgrind': ['drd/tests/std_list']}
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1355
>
> Not sure if these always fail or are intermittent?
>
libinput failure should be addressed by Bruce's patch to enable UINPUT
kernel option (I think I saw that in master-next, but now I don't).
The rest are intermittent, as latest master build only shows one different
intermittent failure in glib (which I'm trying to reproduce with additional
diagnostics):
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1356
Alex
[-- Attachment #2: Type: text/html, Size: 2620 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-12-20 9:34 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-12-19 17:29 [PATCH] oeqa/ptest: print a warning if ptests failed Alexander Kanavin
2020-12-19 23:56 ` [OE-core] " Richard Purdie
2020-12-20 9:34 ` Alexander Kanavin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.