| Gilles Peskine | 508caf5 | 2019-09-16 16:29:15 +0200 | [diff] [blame] | 1 | # Mbed TLS test framework |
| 2 | |
| 3 | This document is an overview of the Mbed TLS test framework and test tools. |
| 4 | |
| 5 | This document is incomplete. You can help by expanding it. |
| 6 | |
| 7 | ## Running tests |
| 8 | |
| 9 | ### Outcome file |
| 10 | |
| 11 | #### Generating an outcome file |
| 12 | |
| 13 | Unit tests and `ssl-opt.sh` record the outcome of each test case in a **test outcome file**. This feature is enabled if the environment variable `MBEDTLS_TEST_OUTCOME_FILE` is set. Set it to the path of the desired file. |
| 14 | |
| 15 | If you run `all.sh --outcome-file test-outcome.csv`, this collects the outcome of all the test cases in `test-outcome.csv`. |
| 16 | |
| 17 | #### Outcome file format |
| 18 | |
| 19 | The outcome file is in a CSV format using `;` (semicolon) as the delimiter and no quoting. This means that fields may not contain newlines or semicolons. There is no title line. |
| 20 | |
| 21 | The outcome file has 6 fields: |
| 22 | |
| 23 | * **Platform**: a description of the platform, e.g. `Linux-x86_64` or `Linux-x86_64-gcc7-msan`. |
| 24 | * **Configuration**: a unique description of the configuration (`config.h`). |
| 25 | * **Test suite**: `test_suite_xxx` or `ssl-opt`. |
| 26 | * **Test case**: the description of the test case. |
| 27 | * **Result**: one of `PASS`, `SKIP` or `FAIL`. |
| 28 | * **Cause**: more information explaining the result. |