Update TF-M CI scripts overview
Add instructions on TF-M build helper and LAVA helper.
Signed-off-by: Xinyu Zhang <xinyu.zhang@arm.com>
Change-Id: I117cace35bf122086bab1b9d7e9269a0f3913496
diff --git a/docs/images/TFM-images/TFM-filter-group.png b/docs/images/TFM-images/TFM-filter-group.png
new file mode 100644
index 0000000..d43a055
--- /dev/null
+++ b/docs/images/TFM-images/TFM-filter-group.png
Binary files differ
diff --git a/docs/index.md b/docs/index.md
index 8bb5c87..ee0e3d4 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -776,7 +776,145 @@
## TF-M CI scripts overview
- **TODO - This section is missing**
+TF-M Open CI has two repos:
+* [tf-m-job-configs](https://git.trustedfirmware.org/ci/tf-m-job-configs.git): Job configs in [Jenkins Job Builder](#the-tf-jenkins-job-builder-jjb-configs) format.
+* [tf-m-ci-scripts](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git): Scripts that run in Jenkins to build and test TF-M.
+
+CI scripts use 2 tools to generate commands:
+ * Build helper: Generate commands for building TF-M images.
+ * LAVA helper: Generate config files for [LAVA instance](#tf-lava-instance) to run TF-M images and monitor the test results.
+
+### Build helper
+
+Build helper is a tool to generate build commands for different TF-M configs.
+
+#### Config group
+
+Build helper categorizes all the TF-M configs that are built in CI into different [groups](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_configs.py?h=refs/heads/release/1.7.x#n937), depending on CI jobs. The config groups for each CI job are set as the default value of [Filter Group](https://git.trustedfirmware.org/ci/tf-m-job-configs.git/tree/tf-m-nightly.yaml?h=refs/heads/release/1.7.x#n24), which is configured by [tf-m-job-configs](https://git.trustedfirmware.org/ci/tf-m-job-configs.git).
+
+<div style="text-align: center"><img src="images/TFM-images/TFM-filter-group.png" alt="alt text" title="Filter Group" width=70%/><div style="text-align: left">
+
+#### Generate CMake build command
+
+In each config group, values of following 9 TF-M build config parameters in *seed_params* are combined to generate all configs covered in this group.
+
+| Parameter | TF-M CMake Build Config | Value |
+| ---------------- | ------------------------- | ------------------------------------------------ |
+| tfm_platform | TFM_PLATFORM | arm/mps2/an521, etc. |
+| compiler | TFM_TOOLCHAIN_FILE | ARMCLANG_6_13 (ARMClang v6.13), etc. |
+| isolation_level | TFM_ISOLATION_LEVEL | 1 / 2 / 3 |
+| test_regression | TEST_BL2, TEST_S, TEST_NS | True / False |
+| test_psa_api | TEST_PSA_API | OFF / IPC / CRYPTO/INITIAL_ATTESTATION / STORAGE |
+| cmake_build_type | CMAKE_BUILD_TYPE | Debug / Release / Minsizerel |
+| with_bl2 | BL2 | True / False |
+| profile | TFM_PROFILE | profile_small / profile_medium / profile_medium_arotless / profile_large,<br>empty string for base profile by default |
+| extra_params | Other customized configs | Build config abbreviation in [mapExtraParams](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_config_maps.py?h=refs/heads/release/1.7.x#n56) |
+
+For example, in config group [config_profile_m_arotless](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_configs.py?h=refs/heads/release/1.7.x#n376), 24 configs are generated by combining values in each seed params.
+
+``` python3
+config_profile_m_arotless = {"seed_params": {
+ "tfm_platform": ["arm/musca_b1"],
+ "compiler": ["GCC_10_3", "ARMCLANG_6_13"],
+ "isolation_level": ["1"],
+ "test_regression": [True, False],
+ "test_psa_api": ["OFF"],
+ "cmake_build_type": ["Debug", "Release", "Minsizerel"],
+ "with_bl2": [True],
+ "profile": ["profile_medium_arotless"],
+ "extra_params": ["", "PSOFF"]
+ },
+ "common_params": _common_tfm_builder_cfg,
+ "invalid": _common_tfm_invalid_configs + []
+ }
+```
+
+The combination of these 9 *seed_params* is filled into the [build config template](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_configs.py?h=refs/heads/release/1.7.x#n48) by Build Helper and finally generates the complete TF-M build command for this config.
+
+To add some specific TF-M build parameters into the build command, a map of *"abbreviation: build command"* is added to [mapExtraParams](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_config_maps.py?h=refs/heads/release/1.7.x#n56) in build_helper_config_maps.py, and the abbreviation is added to *extra_params*. For example, *"PSOFF"* in config group *config_profile_m_arotless* is the abbreviation of build config *"-DTFM_PARTITION_PROTECTED_STORAGE=OFF "*.
+
+#### Filter out invalid configs
+
+To filter out specific configs from the config group, a config tuple with all the 9 seed params in sequence is added to the *invalid* list. Character "\*" stands for all values of the corresponding config parameter. For example, in config group *config_profile_s*, all configs for AN519 built with GCC v10.3 in Minsizerel type is not contained in this group.
+
+``` python3
+config_profile_s = {"seed_params": {
+ "tfm_platform": ["arm/mps2/an519", "arm/mps2/an521"],
+ "compiler": ["GCC_10_3", "ARMCLANG_6_13"],
+ "isolation_level": ["1"],
+ "test_regression": [True, False],
+ "test_psa_api": ["OFF"],
+ "cmake_build_type": ["Debug", "Release", "Minsizerel"],
+ "with_bl2": [True],
+ "profile": ["profile_small"],
+ "extra_params": ["PSOFF"]
+ },
+ "common_params": _common_tfm_builder_cfg,
+ "invalid": _common_tfm_invalid_configs + [
+ ("arm/mps2/an519", "GCC_10_3", "*", "*",
+ "*", "Minsizerel", "*", "*", "*")
+ ]
+ }
+```
+
+#### Extra commands after building TF-M
+
+If any extra command is needed by some platforms after building TF-M, the command is added to [build_cmds](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_configs.py?h=refs/heads/release/1.7.x). For example, Musca B1 needs to run *srec_cat* command to generate the final image, the command is added [here](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_configs.py?h=refs/heads/release/1.7.x#n78).
+
+#### Config name format
+
+To keep the config name short and clear, config tuple is converted to string in a brief format, by using abbreviations to map the long config parameters into a shorter format in [build_helper_config_maps.py](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/build_helper/build_helper_config_maps.py?h=refs/heads/release/1.7.x). The detailed conversion logic is [here](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/tfm_ci_pylib/tfm_build_manager.py?h=refs/heads/release/1.7.x#n573).
+
+#### How to add new configs
+
+To support new TF-M features, it is recommanded to add a new config group for the new feature. Following steps are needed:
+1. Add new config group in CI scripts. Please take this patch for reference: https://review.trustedfirmware.org/c/ci/tf-m-ci-scripts/+/17665
+
+2. Run the following commands locally to verify whether the new config group works as expected.
+
+ To generate all configs covered in the new config group, run command:
+ ``` bash
+ python3 <path of tf-m-ci-scripts>/configs.py -g <group_name>
+ ```
+
+ To get build command for a specific config generated by Build Helper, run command:
+ ``` bash
+ python3 <path of tf-m-ci-scripts>/configs.py -b <config_name>
+ ```
+
+ To get the detailed values of each build parameters of a specifig config, run command:
+ ``` bash
+ python3 <path of tf-m-ci-scripts>/configs.py <config_name>
+ ```
+
+3. Enable the new config group by adding it as the default group in job configs. Please take this patch for reference: https://review.trustedfirmware.org/c/ci/tf-m-job-configs/+/17841
+
+4. Verify the changes on Staging CI. For details please refer to [Staging Trusted Firmware System](#staging-trusted-firmware-system) section. If any permission is needed, please contact tf-openci@lists.trustedfirmware.org for help.
+
+### LAVA helper
+
+LAVA helper is a tool to generate config files for [LAVA instance](#tf-lava-instance). The config files are submitted to LAVA server by Jenkins to run TF-M images on LAVA boards and monitor the test results.
+
+All Open CI supported boards (including FVPs) are listed in [lava_gen_config_map](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n296), mapped with their configurations. The values of each config are filled by LAVA helper into the corresponding jinja2 template of the platform in [lava_helper/jinja2_templates](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/jinja2_templates?h=refs/heads/release/1.7.x).
+
+For exmple, in [MPS2 FVP template file](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/jinja2_templates/fvp_mps2.jinja2?h=refs/heads/release/1.7.x), the path of FVP image and FVP command are set in this file. LAVA helper fills all the config values in [fvp_mps2_an521_bl2](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n151) into this template file, and then submit this config file to LAVA server to start the test on MPS2 FVP.
+
+To decide whether the test run as expected, LAVA server uses __*monitors*__ to monitor the target log strings. Once the specified string is matched by LAVA monitor before timeout, it means that the test works as expected. Otherwise the tests fail.
+
+There are five kinds of LAVA monitors defined in TF-M Open CI:
+ * [monitors_no_reg_tests](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n33): Applied when regression tests are disabled.
+ * [monitors_mcuboot_tests](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n44): Applied when BL2 tests are enabled.
+ * [monitors_s_reg_tests](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n52): Applied when secure regression tests are enabled.
+ * [monitors_ns_reg_tests](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n60): Applied when non-secure regression tests are enabled.
+ * [monitors_arch_tests](https://git.trustedfirmware.org/ci/tf-m-ci-scripts.git/tree/lava_helper/lava_helper_configs.py?h=refs/heads/release/1.7.x#n68): Applied when PSA Arch tests are enabled.
+
+#### How to add new platform
+
+Platforms of [trustedfirmware.org](https://www.trustedfirmware.org/) members can be added to Open CI.
+
+To add physical boards, please contact tf-openci@lists.trustedfirmware.org for help.
+
+To add FVP, please follow the instructions in [FVP Doker Images](#fvp-docker-images) to upload the new FVP image. If any permission or resources are needed, please contact tf-openci@lists.trustedfirmware.org for help.
# TF LAVA Instance
