* [PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default
@ 2021-07-09 20:56 sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files sakib.sajal
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: sakib.sajal @ 2021-07-09 20:56 UTC (permalink / raw
To: yocto
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
---
config.json | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/config.json b/config.json
index f54081b..99593c8 100644
--- a/config.json
+++ b/config.json
@@ -61,7 +61,9 @@
"RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'",
"BB_HEARTBEAT_EVENT = '60'",
"BB_LOG_HOST_STAT_ON_INTERVAL = '1'",
- "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'"
+ "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 15'",
+ "BB_LOG_HOST_STAT_ON_FAILURE = '1'",
+ "BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l'"
]
},
"templates" : {
--
2.25.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files
2021-07-09 20:56 [PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default sakib.sajal
@ 2021-07-09 20:56 ` sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 3/4] summarize_top_output.py: summarize data logged on failure sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 4/4] scripts/generate-testresult-index.py: update index to list host data sakib.sajal
2 siblings, 0 replies; 4+ messages in thread
From: sakib.sajal @ 2021-07-09 20:56 UTC (permalink / raw
To: yocto
data collection on failure is enabled by default.
Collect all host stats files created.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
---
scripts/collect-results | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/scripts/collect-results b/scripts/collect-results
index 7178380..93834d6 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -18,21 +18,27 @@ if [ -e $WORKDIR/buildhistory ]; then
fi
fi
-HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
+HSFILE=$WORKDIR/tmp/buildstats/*/host_stats*
d="intermittent_failure_host_data"
mkdir -p $DEST/$target/$d
-step=0
+step_i=1
+step_f=1
for f in $HSFILE; do
if [ -e $f ]; then
cp $f $DEST/$target/$d
+ if [[ "$f" == *"failure"* ]] ; then
+ mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_f}.txt
+ step_f=$((step_f+1))
+ continue
+ fi
grep -m 1 "^top -" $f
if [ $? -eq 0 ]; then
- mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step}_top.txt
+ mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_i}_top.txt
else
- mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step}.txt
+ mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_i}.txt
fi
- step=$((step+1))
+ step_i=$((step_i+1))
fi
done
--
2.25.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH yocto-autobuilder-helper 3/4] summarize_top_output.py: summarize data logged on failure
2021-07-09 20:56 [PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files sakib.sajal
@ 2021-07-09 20:56 ` sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 4/4] scripts/generate-testresult-index.py: update index to list host data sakib.sajal
2 siblings, 0 replies; 4+ messages in thread
From: sakib.sajal @ 2021-07-09 20:56 UTC (permalink / raw
To: yocto
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
---
scripts/summarize_top_output.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
index 50c9b0a..4e96ce9 100755
--- a/scripts/summarize_top_output.py
+++ b/scripts/summarize_top_output.py
@@ -168,9 +168,11 @@ def main():
target = sys.argv[2]
host_data_dir = "intermittent_failure_host_data"
directory = os.path.join(dest, target, host_data_dir)
- for f in glob.glob(directory + "/*_top.txt"):
- outputs = list_top_outputs(f)
- short_summary, summary, kernel_summary, zombie_summary, other_build = summarize_top(outputs, target)
- write_summary(short_summary, summary, kernel_summary, zombie_summary, other_build, target, f)
+ regs = (directory + "/*_top.txt", directory + "/*_failure_*.txt")
+ for exts in regs:
+ for f in glob.glob(exts):
+ outputs = list_top_outputs(f)
+ short_summary, summary, kernel_summary, zombie_summary, other_build = summarize_top(outputs, target)
+ write_summary(short_summary, summary, kernel_summary, zombie_summary, other_build, target, f)
main()
--
2.25.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH yocto-autobuilder-helper 4/4] scripts/generate-testresult-index.py: update index to list host data
2021-07-09 20:56 [PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 3/4] summarize_top_output.py: summarize data logged on failure sakib.sajal
@ 2021-07-09 20:56 ` sakib.sajal
2 siblings, 0 replies; 4+ messages in thread
From: sakib.sajal @ 2021-07-09 20:56 UTC (permalink / raw
To: yocto
Host data is collected on failure by default. List the collected files
in the index.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
---
scripts/generate-testresult-index.py | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index d85d606..5c33778 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -152,13 +152,15 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
buildhistory.append((reldir + "testresults/qemuarm/buildhistory.txt", "qemuarm"))
hd = []
- counter = 0
- # do we really need the loop?
- for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
+ for p in glob.glob(buildpath + "/*/*/host_stats*summary.txt"):
n_split = p.split(build)
res = reldir[0:-1] + n_split[1]
- hd.append((res, str(counter)))
- counter += 1
+ n = os.path.basename(p).split("host_stats_")[-1]
+ if "failure" in n:
+ n = n.split("_summary.txt")[0]
+ elif "top" in n:
+ n = n.split("_top_summary.txt")[0]
+ hd.append((res, n))
branch = get_build_branch(buildpath)
--
2.25.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-07-09 20:56 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-07-09 20:56 [PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 3/4] summarize_top_output.py: summarize data logged on failure sakib.sajal
2021-07-09 20:56 ` [PATCH yocto-autobuilder-helper 4/4] scripts/generate-testresult-index.py: update index to list host data sakib.sajal
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.