tf-a-builder.yaml: resubmit LAVA jobs in case failure
Currently, at the TF LAVA lab there are two issues, not related to
user changes, impacting negatively the Open CI: Juno
infrastucture (corrupted SD cards, broken board, etc.)
[2] and FVP TFTF intermittent errors (TFTF execution hangs at some
particular test) [3]. The former (JUNO jobs) barely occurs and in most
cases the second submitted job passes correctly while the latter (FVP
TFTF jobs) issue is happening at least once in all top-level jobs,
causing false negatives reviews.
To cover both cases and logic does not get too complex, the approach
proposed consists of submitting the LAVA job, JUNO or FVP, in case of
failure at most tree times, reducing considerably the negative votes
given by the CI into user's gerrit review. The downside of this
approach is that the test phase may take longer (3 instead of 1 job
execution) but patch [4] proposes shorter and more realistic timeout
values, allowing LAVA to fail quicker due to [4] and re-trigger the
LAVA job again.
Finally, all this logic is a workaround while [3] is further investigated.
[1] https://tf.validation.linaro.org/
[2] https://linaro.atlassian.net/browse/STG-2489
[3] https://linaro.atlassian.net/browse/TFC-176
[4] https://review.trustedfirmware.org/c/ci/tf-a-ci-scripts/+/11759
Signed-off-by: Leonardo Sandoval <leonardo.sandoval@linaro.org>
Change-Id: Ib707100c605677c981ac1cd1cc08e6da146e8334
diff --git a/tf-a-builder.yaml b/tf-a-builder.yaml
index 1f02af5..fbf14d4 100644
--- a/tf-a-builder.yaml
+++ b/tf-a-builder.yaml
@@ -136,16 +136,21 @@
#!/bin/bash
squad='tf-a-job-configs/tf-a-builder/squad.sh'
+ # FIXME: Juno and FVP jobs may fail due to non-related users changes,
+ # so CI needs to resubmit the job, at most three times:
+ # Juno jobs may fail due to LAVA lab infrastructure issues (see
+ # https://projects.linaro.org/browse/LSS-2128)
+ # FVP jobs may hang at some particular TFTF test (see
+ # https://linaro.atlassian.net/browse/TFC-176)
if ! $squad; then
- # Juno jobs may fail due to LAVA lab infrastructure issues (see
- # https://projects.linaro.org/browse/LSS-2128) so we need to
- # resubmit it if first job failed
- if grep -q 'device_type: juno' artefacts/debug/job.yaml; then
+ if ! $squad; then
if ! $squad; then
echo "LAVA JOB RESULT: 1"
+ else
+ echo "LAVA JOB RESULT: 0"
fi
else
- echo "LAVA JOB RESULT: 1"
+ echo "LAVA JOB RESULT: 0"
fi
else
echo "LAVA JOB RESULT: 0"