User guide
==========

This page describes how to get started with compiling and running unit tests on
the host machine.

Host machine requirements
-------------------------

The system has been successfully tested on the following platforms:

- Ubuntu 19.04
- Ubuntu 18.04
- Arch Linux
- MSYS2 MinGW64

Tools
-----

The following applications are expected to be installed in the build machine:

- CMake >=3.11

- Python >=3.4

- c-picker

  - pyyaml

  - clang (pip package if not included in libclang)

  - libclang

- git

- Toolchain

- Native build system

Ubuntu 19.04
^^^^^^^^^^^^

On Ubuntu 19.04 use the following commands to install the required tools:

::

  sudo apt-get install cmake git python3 python3-pip libclang-dev build-essential
  sudo pip3 install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker

Ubuntu 18.04
^^^^^^^^^^^^

The official Ubuntu 18.04 package repository only contains CMake 3.10 which not
satisfies the requirements for building unit tests. Fortunately there's an
official CMake APT repository provided by Kitware: https://apt.kitware.com/ By
adding this server to the repository list an up-to-date version of CMake can be
installed on Ubuntu 18.04.

::

  wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc \
    2>/dev/null | sudo apt-key add -
  sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

  sudo apt-get install cmake git python3 python3-pip libclang-dev build-essential
  sudo pip3 install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker

Arch Linux
^^^^^^^^^^

On Arch Linux use the following commands to install the required tools:

::

  sudo pacman -Sy cmake git python python-pip python-yaml make gcc clang
  sudo pip install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker

MSYS2 MinGW64
^^^^^^^^^^^^^

::

  pacman -Sy mingw-w64-x86_64-cmake git mingw-w64-x86_64-python3 \
    mingw-w64-x86_64-python3-pip make mingw-w64-x86_64-gcc mingw-w64-x86_64-clang
  pip install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker


Getting unit test source code
-----------------------------

Download the unit test source code using the following command:

::

  git clone https://gerrit.oss.arm.com/trusted-firmware/tf-a-unit-tests.git


Getting Trusted Firmware-A source code
--------------------------------------

Download the TF-A source code using the following command:

::

  git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git


Building unit tests
-------------------

Building unit tests start with running CMake which will check all the
prerequisites and generate the native build system's input files. This example
uses Unix Makefiles. Unit tests exercise the code directly by compiling it into
the same binary as the test code. In this case the tests need to know where to
find the TF-A source files and this is why the :cmake:variable:`TF_A_PATH` is
specified.

::

  cd tf-a-unit-tests
  mkdir build
  cd build
  cmake -DTF_A_PATH=../../trusted-firmware-a -G"Unix Makefiles" ..

After running the previous steps the makefiles are generated into the build
directory so make can build unit tests. During unit test development if only the
source files have changed it's not necessary to re-run cmake it's only needed to
run make as shown below.

::

  make -j

For building single unit tests suites the test's name can be used as a makefile
target. Let's assume there's a test suite called bl1_fwu.

::

  make bl1_fwu


Running unit tests
------------------

CMake provides a built-in tool called ctest for running all the tests using a
single command. It is also able to filter tests or run them in parallel for
speeding up the tests process. Run all the tests using the following command:

::

  ctest

Each unit test suite has its own executable. The easiest way of running single
test suite is running it as a simple executable.

::

  ./bl1_fwu


Debugging unit tests
--------------------

As it was mentioned in the previous section test suites are basically separate
executables so they can be debugged as any other native applications on the host
machine. In a Linux environment gdb or IDE's built-in debugger can be utilized
for debugging.

::

  gdb ./bl1_fwu


Measuring code coverage
-----------------------

Inspecting code coverage is a useful method for detecting parts of the code
which is not exercised by tests. The build system includes an option for
generating code coverage report of the unit tests. The coverage is processed by
``lcov`` which needs to be installed for this feature. Also the coverage
measurement in only available when GCC is used as a compiler.

First of all the :cmake:variable:`COVERAGE` option has to be enabled by using
the following command line when invoking CMake.

::

  cmake -DTF_A_PATH=../../trusted-firmware-a -DCOVERAGE=ON -G"Unix Makefiles" ..

This makes CMake to build the binaries with coverage information included. The
rest of the build process works the same way as before.

Before collecting coverage info and generating reports the tests must be run as
the coverage is a runtime measurement. See section `Running unit tests`_ for
more information about running unit tests.

The :cmake:variable:`COVERAGE` option adds two new build targets called
``coverage`` and ``coverage_report``. They can be used simply by running the
following commands if ``make`` is used as a build system.

::

  make coverage
  make coverage_report

The ``coverage`` target generates lcov info files for further processing. If
there are coverage files available from different sources (i.e. coverages of
other tests) they can be merged with the unit test coverage file and evaluated
together. Currently two coverage info files are generated during the build. One
of them contains the coverage of code under test (i.e. Trusted Firmware-A) and
the other one has the coverage of the unit tests themselves.

The ``coverage_report`` target generates a HTML report from the coverage info
files. The coverage reports can be found in the build directory's subdirectories
having ``-coverage`` suffix in their names. The report shows the directory
structure of the code and each file can be inspected individually. Line,
function and branch coverage is included.


--------------

*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*