Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 1 | ####################################### |
| 2 | TF-Fuzz (Trusted-Firmware Fuzzer) guide |
| 3 | ####################################### |
| 4 | |
| 5 | ************ |
| 6 | Introduction |
| 7 | ************ |
| 8 | |
| 9 | TF-Fuzz is a TF-M fuzzing tool, at the PSA-call level. At the time of writing |
| 10 | this at least, presentations available at: |
| 11 | |
| 12 | - https://www.trustedfirmware.org/docs/TF-M_Fuzzing_Tool_TFOrg.pdf |
| 13 | - https://zoom.us/rec/share/1dxZcZit111IadadyFqFU7IoP5X5aaa8gXUdr_UInxmMbyLzEqEmXQdx79-IWQ9p |
| 14 | |
| 15 | (These presentation materials may not all be viewable by all parties.) |
| 16 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 17 | A suite generator tool is also provided to make tests output by TF-Fuzz |
| 18 | runnable as a test suite in the regression tester. |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 19 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 20 | |
| 21 | ******************************* |
| 22 | Building and Installing TF-Fuzz |
| 23 | ******************************* |
| 24 | |
| 25 | .. Note:: |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 26 | |
| 27 | These instructions assume the use of Ubuntu Linux. |
| 28 | |
| 29 | |
| 30 | The following dependencies are required to build TF-Fuzz: |
| 31 | |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 32 | .. code-block:: bash |
| 33 | |
| 34 | sudo apt-get update |
| 35 | sudo apt-get install build-essential bison flex |
| 36 | |
| 37 | |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 38 | To build TF-Fuzz, simply type ``make`` in this directory. The executable, |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 39 | called ``tfz``, is placed in the ``bin/`` directory. |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 40 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 41 | ====================================== |
| 42 | Installing the TF-Fuzz suite generator |
| 43 | ====================================== |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 44 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 45 | **Requirements:** Python 3.8 or later; a built ``tfz`` binary. |
| 46 | |
| 47 | |
| 48 | The suite generator is installable as a Python package through ``pip``: |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 49 | |
| 50 | .. code-block:: bash |
| 51 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 52 | cd <path/to/tf-tools>/tf_fuzz |
| 53 | pip3 install tfz-suitegen |
| 54 | |
| 55 | Once installed, ``tfz-suitegen`` can be ran by typing ``python3 -m tfz-suitegen``. |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 56 | |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 57 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 58 | ****************************************** |
| 59 | Generating and running tests using TF-Fuzz |
| 60 | ****************************************** |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 61 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 62 | **Full usage information can be found by running** ``./bin/tfz`` **and** ``python3 -m tfz-suitegen --help`` **.** |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 63 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 64 | The ``demo`` folder contains some example test specifications. The below steps |
| 65 | describe how to build and run these with the TF-M regression tester. |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 66 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 67 | #. Turn the test specifications into a test suite: |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 68 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 69 | .. code-block:: bash |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 70 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 71 | python3 -m tfz-suitegen <path/to/tf_fuzz> <path/to/tf_fuzz>/demo build_suite |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 72 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 73 | This creates an :external:ref:`out-of-tree test suite |
| 74 | <tfm_test_suites_addition:out-of-tree regression test suites>` containing |
| 75 | all the tests in the ``demo`` folder. |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 76 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 77 | .. note:: |
| 78 | Only files with the ``.test`` extension are included in the test suite. |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 79 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 80 | #. Build the regression tests as normal, adding the following CMake flag to the SPE build: |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 81 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 82 | .. code-block:: bash |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 83 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 84 | -DEXTRA_NS_TEST_SUITE_PATH=<absolute_path_to>/build_suite |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 85 | |
Nik Dewally | 3a98fe3 | 2024-07-09 16:30:17 +0100 | [diff] [blame] | 86 | For full instructions on how to build and run tests see |
| 87 | :external:doc:`building/tests_build_instruction` and |
| 88 | :external:ref:`building/run_tfm_examples_on_arm_platforms:run tf-m tests and |
| 89 | applications on arm platforms`. |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 90 | |
Nik Dewally | 3db11f7 | 2024-07-19 10:23:30 +0100 | [diff] [blame] | 91 | .. warning:: |
| 92 | |
| 93 | Some of the provided demos are expected to fail. |
| 94 | |
| 95 | |
Nik Dewally | fc87b06 | 2024-07-16 14:05:17 +0100 | [diff] [blame] | 96 | ************************************ |
| 97 | Running the TF-Fuzz regression tests |
| 98 | ************************************ |
| 99 | |
| 100 | To run the regression test suite: |
| 101 | |
| 102 | .. code-block:: bash |
| 103 | |
| 104 | cd <path/to/tf-tools>/tf_fuzz/tfz-cpp |
| 105 | make |
| 106 | cd regression |
| 107 | bash regress |
| 108 | |
| 109 | |
| 110 | For more details, see :doc:`source_structure/regression_dir`. |
| 111 | |
| 112 | |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 113 | .. toctree:: |
| 114 | :caption: Table of Contents |
| 115 | :maxdepth: 1 |
| 116 | |
| 117 | Source Structure <source_structure/index> |
| 118 | |
Karl Zhang | 3de5ab1 | 2021-05-31 11:45:48 +0800 | [diff] [blame] | 119 | -------------- |
| 120 | |
Nik Dewally | 6fd2f99 | 2024-07-01 13:53:23 +0100 | [diff] [blame] | 121 | *Copyright (c) 2020-2024, Arm Limited. All rights reserved.* |