fix: work around PIP cache concurrency issues
Our issue with PIP appears to stem from its cache locking mechanism,
which doesn't seem capable of handling the number of concurrent
accesses to the cache as the CI generates.
Instead, we will create a cache for each concurrent executor so that
each executor has its own unique cache. It's not ideal, especially with
a lot of executors, but there's little else we can do short of not
cache at all.
Change-Id: Id46becb489328f9d169b69c56bc85d96cfab30e3
Signed-off-by: Chris Kay <chris.kay@arm.com>
diff --git a/script/install_python_deps_tf.sh b/script/install_python_deps_tf.sh
index 49963a6..62fd795 100644
--- a/script/install_python_deps_tf.sh
+++ b/script/install_python_deps_tf.sh
@@ -6,7 +6,15 @@
# SPDX-License-Identifier: BSD-3-Clause
#
-python3 -m venv .venv && \
- source .venv/bin/activate && \
- python3 -m pip install -r "${tf_root}/docs/requirements.txt" \
- --cache-dir "${project_filer}/pip-cache" --retries 30 --verbose
+python3 -m venv .venv
+
+source .venv/bin/activate
+
+(
+ export PIP_CACHE_DIR=${project_filer}/pip-cache
+
+ python3 -m pip install --upgrade pip
+ python3 -m pip install -r "${tf_root}/docs/requirements.txt" ||
+ python3 -m pip install -r "${tf_root}/docs/requirements.txt" \
+ --no-cache-dir # Avoid cache concurrency issues
+)