Initial version of unit testing documentation
This commit includes the 'User guide' and 'Implementing tests' sections
for helping people building, running and writing unit tests. A more
detailed version of documentation is available under the 'Component
user manuals section'.
Change-Id: I67e93ac805d1f4e7727964f3d95a70436ae34733
Signed-off-by: Imre Kis <imre.kis@arm.com>
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 86c55b5..71bb31c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -4,6 +4,84 @@
# SPDX-License-Identifier: BSD-3-Clause
#
+#[===[.rst:
+Processing flow
+---------------
+
+The following steps show how the CMake based build system works. The first
+couple steps are responsible for detecting and checking the environment. The
+following steps are the fetching and building phase of CppUTest. Finally it
+collects the test cases.
+
+1. Checking if :cmake:variable:`TF_A_PATH` is specified
+
+2. Checking git
+
+3. Fetching CppUTest
+
+4. Building CppUTest
+
+5. Configuring coverage targets if they are enabled
+
+6. Collecting unit test suites
+
+Note that the included modules can also execute further checks and commands on
+init.
+
+
+Variables
+---------
+
+The following cache entries can be set with the ``-D`` command line option of
+CMake when the configuration phase is first run. This allows applying
+environment specific configuration to the build. The current values of these
+variables can be checked by opening ``CMakeCache.txt`` in the build directory or
+by using ``cmake-gui``.
+
+.. cmake:variable:: TF_A_PATH
+
+Path of the Trusted Firmware-A source directory. **This needs to be specified
+by the developer** to point to a suitable working copy of TF-A.
+
+.. cmake:variable:: TF_A_UNIT_TESTS_PATH
+
+Root directory of unit test repository. It can be used to reference source
+files from the unit test repository relative to its root directory.
+
+.. cmake:variable:: CPPUTEST_URL
+
+URL of the CppUTest git repository. By default it points to the official Github
+repository of CppUTest. It can be used to specify a different CppUTest mirror.
+
+.. cmake:variable:: CPPUTEST_REFSPEC
+
+CppUTest git refspec. The default value selects the latest release.
+
+.. cmake:variable:: CPPUTEST_INSTALL_PATH
+
+Temporary directory used during CppUTest build
+
+.. cmake:variable:: CPPUTEST_PACKAGE_PATH
+
+Path of the CppUTest CMake package directory
+
+.. cmake:variable:: CPICKER_CACHE_PATH
+
+Directory of c-picker generated files. Subdirectories are added according to
+the path of the original source file's path.
+
+.. cmake:variable:: CLANG_LIBRARY_PATH
+
+c-picker uses libclang to parse the source files. If defined this variable
+specifies the path of the library.
+
+.. cmake:variable:: COVERAGE
+
+Adds compiler flags for coverage measurement and enables ``coverage`` and
+``coverage_report`` targets.
+
+#]===]
+
cmake_minimum_required(VERSION 3.11...3.15) # TODO: test with ubuntu
project(tf-a-unit-tests)
diff --git a/cmake/Coverage.cmake b/cmake/Coverage.cmake
index dd1e6ce..e5fae00 100644
--- a/cmake/Coverage.cmake
+++ b/cmake/Coverage.cmake
@@ -4,6 +4,52 @@
# SPDX-License-Identifier: BSD-3-Clause
#
+#[===[.rst:
+Coverage CMake module
+---------------------
+
+Control flow
+^^^^^^^^^^^^
+
+Using the code coverage feature of the system starts with including
+``Coverage`` module. This will implicitly check if all the requirements for
+generating coverage are fulfilled. This includes checking the following
+conditions.
+
+- Compiler is GCC
+- lcov executables exist
+- ``c-picker-coverage-mapper`` is available
+
+As the next step it sets the compiler flags to make GCC to generate binaries
+with coverage information.
+
+
+Variables
+^^^^^^^^^
+
+The module sets the following variables while it's checking its prerequisites.
+
+.. cmake:variable:: LCOV_COMMAND
+
+Path of lcov executable
+
+.. cmake:variable:: GENHTML_COMMAND
+
+Path of genhtml executable which is part of the lcov package.
+
+.. cmake:variable:: CPICKER_COVERAGE_MAPPER_COMMAND
+
+Path of ``c-picker-coverage-mapper`` executable which is provided by c-picker
+pip package.
+
+
+Functions
+^^^^^^^^^
+
+The module also contains functions for setting up the coverage feature.
+
+#]===]
+
include_guard(DIRECTORY)
# Checking GCC
@@ -56,8 +102,51 @@
add_dependencies(${TARGET} ${TARGET}_target_${CUSTOM_TARGET_SUFFIX})
endfunction()
-# Generates LCOV coverage info file by processing the .gcda and .gcno files.
-# The function also maps coverage of the c-picker generated files to the original source lines
+#[===[.rst:
+.. cmake:command:: coverage_generate
+
+ .. code-block:: cmake
+
+ coverage_generate(
+ NAME test_name
+ SOURCE_DIR source_directory
+ BINARY_DIR binary_directory
+ CPICKER_MAPPING_PATH c_picker_mapping_path
+ OUTPUT_FILE output_file
+ )
+
+ The function outputs an lcov info file for further processing. It also handles
+ the remapping of the coverage of the c-picker generated files.
+
+ Control flow:
+
+ 1. Running the ``lcov`` command for collecting the coverage data from the
+ available ``.gcda`` and ``.gcno`` files in the ``BINARY_DIR``.
+
+ 2. The output of previous step is processed by ``c-picker-coverage-mapper``.
+ This will remap the coverage of files in ``CPICKER_MAPPING_PATH`` to the
+ original source files.
+
+ 3. Adds the output file to the ``coverage`` target's dependency list.
+
+ Inputs:
+
+ ``NAME``
+ Test name included in lcov info file
+
+ ``SOURCE_DIR``
+ Directory of source files
+
+ ``BINARY_DIR``
+ Directory of the ``.gcda`` and ``.gcno`` files
+
+ ``CPICKER_MAPPING_PATH``
+ Path of c-picker generated files
+
+ ``OUTPUT_FILE``
+ Output lcov coverage info file
+
+#]===]
function(coverage_generate)
set(_OPTIONS_ARGS)
set(_ONE_VALUE_ARGS NAME SOURCE_DIR BINARY_DIR CPICKER_MAPPING_PATH OUTPUT_FILE)
@@ -93,7 +182,33 @@
)
endfunction()
-# Filters coverage info of files from the matching directory
+#[===[.rst:
+.. cmake:command:: coverage_filter
+
+ .. code-block:: cmake
+
+ coverage_filter(
+ INPUT_FILE input_file
+ OUTPUT_FILE output_file
+ INCLUDE_DIRECTORY include_directory
+ )
+
+ The function filters the coverage data by including only the coverage of the
+ files of ``INCLUDE_DIRECTORY`` or its subdirectories. It adds the filtered
+ output file to the ``coverage`` target's dependency list.
+
+ Inputs:
+
+ ``INPUT_FILE``
+ Input lcov coverage info file
+
+ ``OUTPUT_FILE``
+ Output lcov coverage info file
+
+ ``INCLUDE_DIRECTORY``
+ Root directory of included files
+
+#]===]
function(coverage_filter)
set(_OPTIONS_ARGS)
set(_ONE_VALUE_ARGS INPUT_FILE OUTPUT_FILE INCLUDE_DIRECTORY)
@@ -122,7 +237,30 @@
)
endfunction()
-# Generated an HTML report from the LCOV info file
+#[===[.rst:
+.. cmake:command:: coverage_generate_report
+
+ .. code-block:: cmake
+
+ coverage_generate_report(
+ INPUT_FILE input_file
+ OUTPUT_DIRECTORY output_directory
+ )
+
+ The function generates a HTML coverage report from the lcov info file into
+ the ``OUTPUT_DIRECTORY``. It adds the output directory to the
+ ``coverage_report`` target's dependency list.
+
+ Inputs:
+
+ ``INPUT_FILE``
+ Input lcov coverage info file
+
+ ``OUTPUT_DIRECTORY``
+ Output directory of the coverage report where the ``index.html`` is the
+ root file of the report.
+
+#]===]
function(coverage_generate_report)
set(_OPTIONS_ARGS)
set(_ONE_VALUE_ARGS INPUT_FILE OUTPUT_DIRECTORY)
diff --git a/cmake/FindLibClang.cmake b/cmake/FindLibClang.cmake
index c0eefc5..2b8c741 100644
--- a/cmake/FindLibClang.cmake
+++ b/cmake/FindLibClang.cmake
@@ -1,14 +1,22 @@
#
-# Copyright (c) 2019, Arm Limited. All rights reserved.
+# Copyright (c) 2019-2020, Arm Limited. All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
#
#[=======================================================================[.rst:
FindLibClang
--------
+------------
-Finds the LibClang library.
+CMake module for finding the LibClang library.
+
+Control flow
+^^^^^^^^^^^^
+
+1. Running ``llvm-config`` if exists
+2. Searching for library at common places
+3. Searching in Windows registry if available
+
Imported Targets
^^^^^^^^^^^^^^^^
@@ -18,6 +26,7 @@
``LibClang``
The Clang library
+
Result Variables
^^^^^^^^^^^^^^^^
diff --git a/cmake/UnitTest.cmake b/cmake/UnitTest.cmake
index 782dad9..d557eb0 100644
--- a/cmake/UnitTest.cmake
+++ b/cmake/UnitTest.cmake
@@ -4,14 +4,44 @@
# SPDX-License-Identifier: BSD-3-Clause
#
-include_guard(DIRECTORY)
+#[===[.rst:
+UnitTest CMake module
+---------------------
-# Trying to set CLANG_LIBRARY_PATH using the following methods
-# 1. Using cache or command line definition
-# Show warning if environment variable is also set but has different value
-# 2. Copying the value of CLANG_LIBRARY_PATH environment variable if set
-# 3. find_package (llvm-config, common paths or Windows registry)
-# If none of the above steps succeeded CMake emits a fatal error and stops
+Control flow
+^^^^^^^^^^^^
+
+1. Setting :cmake:variable:`CLANG_LIBRARY_PATH`
+
+ 1. Using :cmake:variable:`CLANG_LIBRARY_PATH` CMake variable
+
+ 2. Using ``CLANG_LIBRARY_PATH`` environment variable
+
+ 3. Trying to find by ``find_package`` function which calls :cmake:module:`FindLibClang`.
+
+2. Checking if ``c-picker`` command is available
+
+
+Variables
+^^^^^^^^^
+
+The module sets the following variables while it's checking its prerequisites.
+
+.. cmake:variable:: CLANG_LIBRARY_PATH
+
+libclang directory for c-picker
+
+.. cmake:variable:: CPICKER_COMMAND
+
+Path of c-picker executable which is part of the c-picker pip package.
+
+
+Functions
+^^^^^^^^^
+
+#]===]
+
+include_guard(DIRECTORY)
set(CLANG_LIBRARY_PATH_HELP "libclang directory for c-picker")
@@ -42,15 +72,71 @@
message(FATAL_ERROR "Please install c-picker using pip")
endif()
-# Global dependencies:
-# Variables
-# CPICKER_COMMAND: command of the c-picker
-# CPICKER_CACHE_PATH: root directory of the c-picker generate files
-# UNIT_TEST_COMMON_SOURCES: common source files for every test build
-# CLANG_LIBRARY_PATH: libclang directory for c-picker
-# Modules
-# CTest module should be included in the root CMakeLists.txt before calling this function
+#[===[.rst:
+.. cmake:command:: unit_test_add_suite
+ .. code-block:: cmake
+
+ unit_test_add_suite(
+ NAME test_name
+ SOURCES source_files
+ INCLUDE_DIRECTORIES include_directories
+ COMPILE_DEFINITIONS defines
+ DEPENDS dependencies
+ )
+
+ The ``unit_test_add_suite`` CMake function provides a convenient interface for
+ defining unit test suites. Basically its input is the test source files, include
+ paths and macro definitions and it internally does all the necessary steps to
+ have the test binary registered in CTest as a result.
+
+ Control flow:
+
+ 1. Adding new executable named ``NAME``
+
+ 2. Iterating throught ``SOURCES``
+
+ 1. If it's a normal source file add to the executable's source list
+ 2. If it's a YAML file add as a c-picker custom command and add the generated
+ file to the executable's source list
+
+ 3. Setting include directories
+
+ 4. Setting defines
+
+ 5. Adding extra dependencies of the test build
+
+ 6. Adding executable to the system as a test
+
+ Inputs:
+
+ ``NAME``
+ Unique name of the test suite
+
+ ``SOURCES`` (multi, optional)
+ Source files
+
+ ``INCLUDE_DIRECTORIES`` (multi, optional)
+ Include directories
+
+ ``COMPILE_DEFINITIONS`` (multi, optional)
+ Defines
+
+ ``DEPENDS`` (multi, optional)
+ Extra targets as dependencies of the test build
+
+ Global dependencies:
+
+ ``CPICKER_CACHE_PATH``
+ Root directory of the c-picker generated files
+
+ ``UNIT_TEST_COMMON_SOURCES``
+ Common source files for every test build
+
+ ``CTest``
+ Built-in testing module of CMake
+
+#]===]
function(unit_test_add_suite)
set(_OPTIONS_ARGS)
set(_ONE_VALUE_ARGS NAME)
diff --git a/docs/build_instructions.rst b/docs/build_instructions.rst
deleted file mode 100644
index 5bc1149..0000000
--- a/docs/build_instructions.rst
+++ /dev/null
@@ -1,12 +0,0 @@
-Build Instructions
-==================
-
-TODO: add info here about:
- * how to build the unit tests
- * how to build documentation
-
---------------
-
-*Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
-
-SPDX-License-Identifier: BSD-3-Clause
diff --git a/docs/components/build_system.rst b/docs/components/build_system.rst
new file mode 100644
index 0000000..672681f
--- /dev/null
+++ b/docs/components/build_system.rst
@@ -0,0 +1,25 @@
+Build system
+============
+
+.. cmake-module:: ../../CMakeLists.txt
+
+.. cmake-module:: ../../cmake/UnitTest.cmake
+
+.. cmake-module:: ../../cmake/FindLibClang.cmake
+
+Generating documentation
+------------------------
+
+``sphinx-builder`` is used for building the documentation which is required to
+be installed on the machine. The documentation can be built by executing the
+following commands.
+
+::
+
+ cd docs
+ make html
+
+
+--------------
+
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
diff --git a/docs/components/c_picker.rst b/docs/components/c_picker.rst
new file mode 100644
index 0000000..673d2ce
--- /dev/null
+++ b/docs/components/c_picker.rst
@@ -0,0 +1,63 @@
+c-picker
+========
+
+c-picker uses ``libclang``'s Python interface for parsing source files.
+
+Command line options
+--------------------
+
+- ``-h, --help`` - Showing help message
+- ``--root ROOT`` - Root source directory
+- ``--config CONFIG`` - Configuration file (``.json|.yml``)
+- ``--output OUTPUT`` - Output file
+- ``--print-dependencies`` - Prints the dependencies
+- ``--version`` - Shows the program's version number and exit
+- ``--args ...`` - clang arguments
+
+
+Configuration file format
+-------------------------
+
+c-picker configuration can be described in ``JSON`` or ``YAML`` format using the
+same object structure.
+
+- ``elements`` - List of elements to pick from the original code
+
+ - ``name`` - Name of the element
+ - ``type`` - Type of the element: ``include``, ``function``, ``variable``
+ - ``args`` - Arguments for clang used during picking the element
+ - ``options`` - Currenly the only options is ``remove_static`` which removes
+ the ``static`` qualifier from the element's definition.
+
+- ``options`` - Global options for all elements
+- ``args`` - Global clang arguments for all elements
+
+YAML example
+------------
+
+YAML format is preferred because it can contain metadata like comments and
+licence information.
+
+.. code-block:: YAML
+
+ elements:
+ - file: bl1/bl1_fwu.c
+ type: variable
+ name: bl1_fwu_loaded_ids
+ options: [remove_static]
+ - file: bl1/bl1_fwu.c
+ type: function
+ name: bl1_fwu_add_loaded_id
+ options: [remove_static]
+ - file: bl1/bl1_fwu.c
+ type: function
+ name: bl1_fwu_remove_loaded_id
+ options: [remove_static]
+ args:
+ - -DFWU_MAX_SIMULTANEOUS_IMAGES=10
+ - -DINVALID_IMAGE_ID=0xffffffff
+
+
+--------------
+
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
diff --git a/docs/components/code_coverage.rst b/docs/components/code_coverage.rst
new file mode 100644
index 0000000..6ac6d9f
--- /dev/null
+++ b/docs/components/code_coverage.rst
@@ -0,0 +1,31 @@
+Code coverage
+=============
+
+Coverage processing flow
+------------------------
+
+1. Prerequisites
+
+ 1. Having :cmake:variable:`COVERAGE`` CMake variable set to ``ON``
+
+ 2. Building all or selected test binaries with coverage flags
+
+ 3. Running all or selected test binaries
+
+2. Collecting coverage data from ``.gcda`` and ``.gcno`` file into ``lcov``
+ coverage info file
+
+3. Mapping c-picker generated files' coverage to the original source lines
+
+4. Filtering coverage data for separating the coverage of the code under tests
+ and the coverage of the test code
+
+5. Generating HTML coverage report from the filtered lcov info files
+
+
+.. cmake-module:: ../../cmake/Coverage.cmake
+
+
+--------------
+
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
diff --git a/docs/components/cppumock.rst b/docs/components/cppumock.rst
new file mode 100644
index 0000000..145c4ad
--- /dev/null
+++ b/docs/components/cppumock.rst
@@ -0,0 +1,212 @@
+CppUMock
+========
+
+CppUMock is the built-in mocking system of CppUTest. This chapter describes the
+basic features of the system. For details please refer the `official CppUMock
+manual`_.
+
+System functions
+----------------
+
+Checking expectations
+^^^^^^^^^^^^^^^^^^^^^
+
+After defining expected calls an invoking actual call the test should check if
+all the expected calls have happened. This can be done by calling
+``mock().checkExpectations()``. The common place to put this function call is
+the ``TEST_TEARDOWN`` function of the test group.
+
+
+Resetting mocking system
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+After the last interaction with the mocking system the developer should reset
+the state of the system by calling ``mock().clear()``. The common place to put
+this function call is the ``TEST_TEARDOWN`` function of the test group after
+calling ``mock().checkExpectations()``.
+
+
+Namespaces
+^^^^^^^^^^
+
+All interactions with the mocking system start by calling ``mock()``. This
+function has an option ``name`` string parameter which can be used for limiting
+the scope of the mocking operation.
+
+.. code-block:: C++
+
+ mock("bl1").expectOneCall("bl1_main");
+
+
+Enable/disable
+^^^^^^^^^^^^^^
+
+The mocking system can be enabled/disabled runtime using the functions below.
+This causes call expected and actual call to be ignored. Default settings are
+restored by ``mock().clear()``.
+
+- ``enable()``
+- ``disable()``
+
+.. code-block:: C++
+
+ mock().disable();
+ // All CppUMock calls are ignored after this point
+ mock().enable();
+ // CppUMock calls are working again
+
+ mock().disable();
+ // All CppUMock calls are ignored after this point
+ [...]
+
+ mock().clear();
+ // Default settings are restored
+
+
+String order
+^^^^^^^^^^^^
+
+After defining multiple expected function call the mocking system always
+The mocking system always uses the next matching function when an actual call
+happens but by default it doesn't check if different function calls happen in
+the order of definition. This behaviour can be turned on using the
+``mock().strictOrder()`` function. This option is also set to default by the
+``mock().clear()`` function.
+
+.. code-block:: C++
+
+ mock().expectOneCall("A");
+ mock().expectOneCall("B");
+
+ mock().actualCall("B");
+ mock().actualCall("A");
+ // No error
+
+ mock().strictOrder();
+ mock().expectOneCall("A");
+ mock().expectOneCall("B");
+
+ mock().actualCall("B"); // Error generated here
+ mock().actualCall("A");
+
+
+Ignoring unspecified calls
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If there are addition actual calls happening in the test case which are
+unrelated to the test case (i.e. log or other messages) the unspecified calls
+can be ignored by adding ``mock().ignoreOtherCalls()`` to the test case. This
+function should be used carefully because it can hide unexpected call which are
+indicating errors in the code. The affect of this call ends by calling
+``mock().clear()``.
+
+
+Specifying object
+-----------------
+
+In object oriented environment member function should validate if the function
+call happened on the suitable object. This is done by adding the following
+function to the mock specification.
+
+- ``onObject(objectPtr)``
+
+
+Validating parameters
+---------------------
+
+Each supported parameter type has a corresponding function. These are the same
+in the expected and actual calls.
+
+- ``withBoolParameter(name, bool value)``
+- ``withIntParameter(name, int value)``
+- ``withUnsignedIntParameter(name, unsigned int value)``
+- ``withLongIntParameter(name, long int value)``
+- ``withUnsignedLongIntParameter(name, unsigned long int value)``
+- ``withDoubleParameter(name, double value)``
+- ``withStringParameter(name, const char* value)``
+- ``withPointerParameter(name, void* value)``
+- ``withFunctionPointerParameter(name, void (*value)())``
+- ``withConstPointerParameter(name, const void* value)``
+- ``withMemoryBufferParameter(name, const unsigned char* value, size_t size)``
+
+If custum types are defined and copier/comparator objects were installed the
+following function can handle these parameters.
+
+- ``withParameterOfType(typeName, name, value)``
+
+There's an option to copying data from the test environment into the mock
+function. When setting expectations the following function can be used to set
+the pointer and the address of the data. **The mocking system will not create a
+copy of this data** so the original data should be kept intact until the actual
+call happens.
+
+- ``withOutputParameterReturning(name, value, size)``
+- ``withOutputParameterOfTypeReturning(typeName, name, value)``
+
+In the actual call the pair of these function are shown below.
+
+- ``withOutputParameter(name, output)``
+- ``withOutputParameterOfType(typeName, name, output)``
+
+
+Ignoring parameters
+^^^^^^^^^^^^^^^^^^^
+
+There are cases when the developer doesn't want to specify all parameters. The
+following function can set this behaviour in the expected call.
+
+- ``ignoreOtherParameters()``
+
+
+Specifying return values
+------------------------
+
+Using function name overloading the return values are specified by calling
+``andReturnValue`` and the parameter type will determine the exact function.
+
+- ``andReturnValue(bool value)``
+- ``andReturnValue(int value)``
+- ``andReturnValue(unsigned int value)``
+- ``andReturnValue(long int value)``
+- ``andReturnValue(unsigned long int value)``
+- ``andReturnValue(double value)``
+- ``andReturnValue(const char* value)``
+- ``andReturnValue(void* value)``
+- ``andReturnValue(const void* value)``
+- ``andReturnValue(void (*value)())``
+
+
+Returning value in actual calls
+-------------------------------
+
+All of these function have version with ``OrDefault(type default_value)``
+suffix. These version return a default value if the return value was not
+specified in the expected call.
+
+- ``bool returnBoolValue()``
+- ``int returnIntValue()``
+- ``unsigned int returnUnsignedIntValue()``
+- ``long int returnLongIntValue()``
+- ``unsigned long int returnUnsignedLongIntValue()``
+- ``double returnDoubleValue()``
+- ``const char * returnStringValue()``
+- ``void * returnPointerValue()``
+- ``const void * returnConstPointerValue()``
+- ``void (*returnFunctionPointerValue())()``
+
+
+Debugging CppUMock errors
+-------------------------
+
+Debugging CppUMock errors can be hard unlike assertion errors because a mocking
+failure can happen multiple layers of function calls under the test case. The
+mocking system has a very similar feature to CppUTest's ``UT_CRASH()`` which is
+``mock().crashOnFailure()``. By enabling this feature the code will crash on
+mocking errors and the developer could easily catch it with the debugger.
+
+
+--------------
+
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
+
+.. _`official CppUMock manual`: https://cpputest.github.io/mocking_manual.html
\ No newline at end of file
diff --git a/docs/components/cpputest.rst b/docs/components/cpputest.rst
new file mode 100644
index 0000000..4963b76
--- /dev/null
+++ b/docs/components/cpputest.rst
@@ -0,0 +1,291 @@
+CppUTest
+========
+
+This document is based on CppUTest v3.8. CppUtest is a unit testing framework
+for testing C and C++ code. This document introduces the basic features of the
+framework. For further information check the `official manual of CppUTest`_.
+
+
+Why CppUTest?
+-------------
+
+First of all it was not our goal to develop a new unit testing framework while
+plenty of open source solutions are already available. There were no special
+requirements agains the unit testing framework that would rule out all existing
+frameworks so we only had to choose a suitable one for our current and possible
+future needs.
+
+We ended up selecting CppUTest because of its small footprint and easy
+portability. It also goes along with the standard xUnit frameworks' principles
+and provides a standard interface to the outside world. Some details are listed
+below.
+
+- C/C++ support
+- Small footprint (compared to Google Test)
+- Easy to use on embedded systems
+- Built-in mocking system (CppUMock)
+- Implements four-phase testing pattern (setup, exercise, verify, teardown)
+- Selective run of test cases
+- Standard output format (JUnit, TeamCity)
+
+
+Assertions
+----------
+
+Generally is a good practice to use more specific assertions because it can
+output more informative error messages in case of a failure. The generic form or
+assertions is ``assert(expected, actual)``. Each assert type has a _TEXT variant
+for user defined error messages as last parameter.
+
+- Boolean
+
+ - ``CHECK(condition)`` - Same as ``CHECK_TRUE``
+ - ``CHECK_TRUE(condition)``
+ - ``CHECK_FALSE(condition)``
+
+- C string
+
+ - ``STRCMP_EQUAL(expected, actual)``
+ - ``STRNCMP_EQUAL(expected, actual, length)``
+ - ``STRCMP_NOCASE_EQUAL(expected, actual)``
+ - ``STRCMP_CONTAINS(expected, actual)``
+ - ``STRCMP_NOCASE_CONTAINS(expected, actual)``
+
+- Integer
+
+ - ``LONGS_EQUAL(expected, actual)``
+ - ``UNSIGNED_LONGS_EQUAL(expected, actual)``
+ - ``LONGLONGS_EQUAL(expected, actual)``
+ - ``UNSIGNED_LONGLONGS_EQUAL(expected, actual)``
+ - ``BYTES_EQUAL(expected, actual)``
+ - ``SIGNED_BYTES_EQUAL(expected, actual)``
+ - ``POINTERS_EQUAL(expected, actual)``
+ - ``FUNCTIONPOINTERS_EQUAL(expected, actual)``
+
+- Enums
+
+ - ``ENUMS_EQUAL_INT(expected, actual)``
+ - ``ENUMS_EQUAL_TYPE(underlying_type, expected, actual)``
+
+- Other assertions
+
+ - ``CHECK_EQUAL(expected, actual)`` - Requires ``operator=`` and
+ ``StringFrom(type)`` to be implemented
+ - ``CHECK_COMPARE(first, relop, second)`` - Same as ``CHECK_EQUAL`` but with
+ any type of compare
+ - ``DOUBLES_EQUAL(expected, actual, threshold)``
+ - ``MEMCMP_EQUAL(expected, actual, size)``
+ - ``BITS_EQUAL(expected, actual, mask)``
+ - ``FAIL()`` or ``FAIL_TEST()`` - Test case fails if called
+ - ``CHECK_THROWS(expected, expression)`` - Catching C++ exceptions
+
+- Miscellaneous macros
+
+ - ``IGNORE_TEST`` - Same as ``TEST`` but it’s not called
+ - ``TEST_EXIT`` - Exists test
+ - ``UT_CRASH()`` - Crashes the test which is easy to catch with debugger
+ - ``UT_PRINT(text)`` - Generic print function
+
+
+Test runner
+-----------
+
+Test cases are collected automatically. Under the hood the ``TEST`` macros are
+creating global instances of classes and their constructor registers the test
+cases into the test registry. This happens before entering the ``main``
+function. In the ``main`` function only the ``RUN_ALL_TESTS`` macro needs to be
+placed with the command line arguments passed to it. On executing the binary the
+command line arguments will control the behaviour of the test process.
+
+.. code-block:: C++
+
+ #include "CppUTest/CommandLineTestRunner.h"
+
+ int main(int argc, char* argv[]) {
+ return RUN_ALL_TESTS(argc, argv);
+ }
+
+The default ``main`` implementation is added to all unit test suites by the
+build system.
+
+
+Command line options
+--------------------
+
+Command line options are available mainly for configuring the output format of
+the test binary and for filtering test groups or cases.
+
+- Output
+
+ - ``-v`` - Prints each test name before running them
+ - ``-c`` - Colorized output
+ - ``-o{normal, junit, teamcity}`` - Output format, junit can be processed by
+ most CIs
+ - ``-k packageName`` - Package name for junit output
+ - ``-lg`` - List test groups
+ - ``-ln`` - List test cases
+
+- Other
+
+ - ``-p`` - Runs each test case in separate processes
+ - ``-ri`` - Runs ignored test cases
+ - ``-r#`` - Repeats testing ``#`` times
+ - ``-s seed`` - Shuffles tests
+
+- Filtering test cases
+
+ - ``"TEST(groupName, testName)"`` - Running single test
+ - ``"IGNORE_TEST(groupName, testName)"`` -- Running single ignored test
+ - ``-g text`` - Runing groups containing text
+ - ``-n text`` - Runing tests containing text
+ - ``-sg text`` - Runing groups matching text
+ - ``-sn text`` - Runing tests matching text
+ - ``-xg text`` - Excluding groups containing text
+ - ``-xn text`` - Excluding tests containing text
+ - ``-xsg text`` - Excluding groups matching text
+ - ``-xsn text`` - Excluding tests matching text
+
+
+Troubleshooting
+---------------
+
+Output messages
+^^^^^^^^^^^^^^^
+
+When one of tests fails the first step is to run it separately and check its
+output message. Usually it shows the exact line of the file where the error
+happened.
+
+::
+
+ test_memcmp.cpp:17: error: Failure in TEST(memcmp, empty)
+ expected <1 0x1>
+ but was <0 0x0>
+
+The executed tests can be followed by adding ``-v`` command line option.
+
+::
+
+ ./memcmp -v
+ TEST(memcmp, different) - 0 ms
+ TEST(memcmp, same) - 0 ms
+ TEST(memcmp, empty) - 0 ms
+
+ OK (3 tests, 3 ran, 1 checks, 0 ignored, 0 filtered out, 0 ms)
+
+
+Catching failure with debugger
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If a failure happens in a helper function or in a loop where the assertion
+is called multiple times it is harder to get the exact environment of a failure.
+In this case it's a good practice to put a ``UT_CRASH()`` call into a
+conditional block which hits when the failure happens. This way the debugger can
+stop on failure because the code emits a signal.
+
+.. code-block:: C++
+
+ TEST(magic, iterate) {
+ int result;
+
+ for(int i = 0; i < 1000; i++) {
+ result = magic_function(i);
+
+ // Debug code
+ if (result) {
+ UT_CRASH();
+ }
+
+ LONGS_EQUAL(0, result);
+ }
+ }
+
+
+Using ``FAIL`` macro
+^^^^^^^^^^^^^^^^^^^^
+
+It's recommended to use ``FAIL`` macro in conditions that should never occur in
+tests. For example if a test case loads test data from an external file but the
+file could not be opened the ``FAIL`` macro should be used with an informative
+message.
+
+.. code-block:: C++
+
+ fd = open("test.bin", O_RDONLY);
+ if (fd < 0) {
+ FAIL("test.bin open failed");
+ }
+
+
+Interference between test cases
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Test cases can interfere if there's a global resource which was not restored to
+its original state after leaving a test case. This can be hard to find but at
+least the it's easy to make sure that this is root case of an error. Let's
+assume there's a global variable which is set during the test case but it
+original value is not restore at the end. CppUTest has an command line option
+for running each test case in separate process. This makes the global variable
+to have its original value at the beginning of the test cases. Basically if the
+test works by passing argument ``-p`` when running but fails without it, there's
+a good chance for having an interference between test cases.
+
+.. code-block:: C++
+
+ int x = 0;
+
+ TEST_GROUP(crosstalk) {
+ };
+
+ TEST(crosstalk, a) {
+ LONGS_EQUAL(0, x);
+ x = 1;
+ }
+
+ TEST(crosstalk, b) {
+ LONGS_EQUAL(0, x);
+ x = 1;
+ }
+
+ TEST(crosstalk, c) {
+ LONGS_EQUAL(0, x);
+ x = 1;
+ }
+
+By running the test executable with different command line arguments it produces
+a different result.
+
+.. code-block::
+
+ ./crosstalk -v
+
+ TEST(crosstalk, c) - 0 ms
+ TEST(crosstalk, b)
+ test_crosstalk.cpp:37: error:
+ Failure in TEST(crosstalk, b)
+ expected <0 0x0>
+ but was <1 0x1>
+
+ - 0 ms
+ TEST(crosstalk, a)
+ test_crosstalk.cpp:32: error: Failure in TEST(crosstalk, a)
+ expected <0 0x0>
+ but was <1 0x1>
+
+ - 0 ms
+
+ Errors (2 failures, 3 tests, 3 ran, 3 checks, 0 ignored, 0 filtered out, 0 ms)
+
+ ./crosstalk -v -p
+ TEST(crosstalk, c) - 1 ms
+ TEST(crosstalk, b) - 0 ms
+ TEST(crosstalk, a) - 0 ms
+
+ OK (3 tests, 0 ran, 0 checks, 0 ignored, 0 filtered out, 2 ms)
+
+
+--------------
+
+*Copyright (c) 2019, Arm Limited. All rights reserved.*
+
+.. _`official manual of CppUTest`: https://cpputest.github.io/
\ No newline at end of file
diff --git a/docs/components/index.rst b/docs/components/index.rst
new file mode 100644
index 0000000..3e8a68b
--- /dev/null
+++ b/docs/components/index.rst
@@ -0,0 +1,14 @@
+Component user manuals
+======================
+
+.. toctree::
+ cpputest
+ cppumock
+ c_picker
+ build_system
+ code_coverage
+
+
+--------------
+
+*Copyright (c) 2019, Arm Limited. All rights reserved.*
diff --git a/docs/conf.py b/docs/conf.py
index 669fc46..efe0d10 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,8 +1,17 @@
# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2019-2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# See the options documentation at http://www.sphinx-doc.org/en/master/config
# -- Metadata about this file ------------------------------------------------
__date__ = "08 October 2019"
-__copyright__ = "Copyright (c) 2019 ARM Limited; SPDX-License-Identifier: BSD-3-Clause"
+__copyright__ = "Copyright (c) 2019-2020 Arm Limited; SPDX-License-Identifier: BSD-3-Clause"
# Configuration file for the Sphinx documentation builder.
@@ -18,8 +27,8 @@
# -- Project information -----------------------------------------------------
project = 'TF-A Unit Tests'
-copyright = 'Copyright (c) 2019 ARM Limited'
-author = 'Imre Kis'
+copyright = 'Copyright (c) 2019-2020 Arm Limited'
+author = 'Arm Limited'
# The full version, including alpha/beta/rc tags
release = '0.0.1'
@@ -30,7 +39,7 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['sphinx.ext.autosectionlabel', 'sphinxcontrib.plantuml']
+extensions = ['sphinx.ext.autosectionlabel', 'sphinxcontrib.plantuml', 'sphinxcontrib.moderncmakedomain']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
diff --git a/docs/contributing.rst b/docs/contributing.rst
index 9993d84..4422352 100644
--- a/docs/contributing.rst
+++ b/docs/contributing.rst
@@ -1,5 +1,5 @@
-Contributing
-============
+Contributor's guide
+===================
TODO: add info here about:
* DCO and commit message format
@@ -10,6 +10,4 @@
--------------
-*Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
-
-SPDX-License-Identifier: BSD-3-Clause
+*Copyright (c) 2019, Arm Limited. All rights reserved.*
diff --git a/docs/global_substitutions.txt b/docs/global_substitutions.txt
index 18cc48a..29afbc7 100644
--- a/docs/global_substitutions.txt
+++ b/docs/global_substitutions.txt
@@ -1,63 +1,12 @@
-.. |AArch32| replace:: :term:`AArch32`
-.. |AArch64| replace:: :term:`AArch64`
-.. |API| replace:: :term:`API`
-.. |CoT| replace:: :term:`CoT`
-.. |COT| replace:: :term:`COT`
-.. |CSS| replace:: :term:`CSS`
-.. |CVE| replace:: :term:`CVE`
-.. |DS-5| replace:: :term:`DS-5`
-.. |DT| replace:: :term:`DT`
-.. |EL| replace:: :term:`EL`
-.. |EHF| replace:: :term:`EHF`
-.. |FDT| replace:: :term:`FDT`
-.. |FIP| replace:: :term:`FIP`
-.. |FVP| replace:: :term:`FVP`
-.. |FWU| replace:: :term:`FWU`
-.. |GIC| replace:: :term:`GIC`
-.. |ISA| replace:: :term:`ISA`
-.. |Linaro| replace:: :term:`Linaro`
-.. |MMU| replace:: :term:`MMU`
-.. |MPAM| replace:: :term:`MPAM`
-.. |MPIDR| replace:: :term:`MPIDR`
-.. |OEN| replace:: :term:`OEN`
-.. |OP-TEE| replace:: :term:`OP-TEE`
-.. |OTE| replace:: :term:`OTE`
-.. |PDD| replace:: :term:`PDD`
-.. |PMF| replace:: :term:`PMF`
-.. |PSCI| replace:: :term:`PSCI`
-.. |RAS| replace:: :term:`RAS`
-.. |ROT| replace:: :term:`ROT`
-.. |SCMI| replace:: :term:`SCMI`
-.. |SCP| replace:: :term:`SCP`
-.. |SDEI| replace:: :term:`SDEI`
-.. |SDS| replace:: :term:`SDS`
-.. |SEA| replace:: :term:`SEA`
-.. |SiP| replace:: :term:`SiP`
-.. |SIP| replace:: :term:`SIP`
-.. |SMC| replace:: :term:`SMC`
-.. |SMCCC| replace:: :term:`SMCCC`
-.. |SoC| replace:: :term:`SoC`
-.. |SP| replace:: :term:`SP`
-.. |SPCI| replace:: :term:`SPCI`
-.. |SPD| replace:: :term:`SPD`
-.. |SPM| replace:: :term:`SPM`
-.. |SVE| replace:: :term:`SVE`
-.. |TBB| replace:: :term:`TBB`
-.. |TBBR| replace:: :term:`TBBR`
-.. |TEE| replace:: :term:`TEE`
+.. |CppUMock| replace:: :term:`CppUMock`
+.. |CppUTest| replace:: :term:`CppUTest`
+.. |Test case| replace:: :term:`Test case`
+.. |Test group| replace:: :term:`Test group`
+.. |Test suite| replace:: :term:`Test suite`
.. |TF-A| replace:: :term:`TF-A`
-.. |TF-M| replace:: :term:`TF-M`
-.. |TLB| replace:: :term:`TLB`
-.. |TLK| replace:: :term:`TLK`
-.. |TSP| replace:: :term:`TSP`
-.. |TZC| replace:: :term:`TZC`
-.. |UEFI| replace:: :term:`UEFI`
-.. |WDOG| replace:: :term:`WDOG`
-.. |XLAT| replace:: :term:`XLAT`
..
--------------
- *Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
+ *Copyright (c) 2019, Arm Limited. All rights reserved.*
- SPDX-License-Identifier: BSD-3-Clause
\ No newline at end of file
diff --git a/docs/glossary.rst b/docs/glossary.rst
new file mode 100644
index 0000000..591b76c
--- /dev/null
+++ b/docs/glossary.rst
@@ -0,0 +1,36 @@
+Glossary
+========
+
+This glossary provides definitions for terms and abbreviations used in the TF-A
+documentation.
+
+You can find additional definitions in the `Arm Glossary`_.
+
+.. glossary::
+ :sorted:
+
+ CppUMock
+ Built-in mocking system of CppUTest.
+
+ CppUTest
+ Open source C/C++ unit testing framework.
+
+ Test case
+ Single use case tested. Defined by ``TEST`` macro of CppUTest.
+
+ Test group
+ Multiple test cases with common setup/teardown steps and helper functions
+ and variables. Defined by ``TEST_GROUP`` macro of CppUTest.
+
+ Test suite
+ Test binary which contains one or more test groups. Defined by
+ :cmake:command:`unit_test_add_suite` CMake function.
+
+ TF-A
+ Trusted Firmware-A
+
+--------------
+
+*Copyright (c) 2020, Arm Limited. All rights reserved.*
+
+.. _`Arm Glossary`: https://developer.arm.com/support/arm-glossary
diff --git a/docs/implementing_tests.rst b/docs/implementing_tests.rst
new file mode 100644
index 0000000..cc107d2
--- /dev/null
+++ b/docs/implementing_tests.rst
@@ -0,0 +1,366 @@
+Implementing tests
+==================
+
+Concept of unit testing
+-----------------------
+
+First of all unit tests exercise the C code on a function level. The tests
+should call functions directly from the code under tests and verify if their
+return values are matching the expected ones and the functions are behaving
+according to the specification.
+
+Because of the function level testing the dependencies of the tested functions
+should be detached. This is done by mocking the underlying layer. This provides
+an additional advantage of controlling and verifying all the call to the lower
+layer.
+
+
+Adding new unit test suite
+--------------------------
+
+The first step is to define a new unit test suite. If a completely new module is
+being test the test suite definition should be created in a separate ``.cmake``
+file which is placed in the test files' directory. Otherwise the test
+definition can be added to an existing ``.cmake`` file. These files should be
+included in the root ``CMakeLists.txt``.
+
+The ``UnitTest`` CMake module defines the ``unit_test_add_suite`` function so
+before using this function the module must be included in the ``.cmake`` file.
+The function first requires a unique test name which will be test binary's name.
+The test sources, include directories and macro definition are passed to the
+function in the matching arguments. CMake variables can be used to reference
+files relative to common directories:
+
+- ``CMAKE_CURRENT_LIST_DIR`` - Relative to the ``.cmake`` file
+- :cmake:variable:`TF_A_PATH` - Relative to the Trusted Firmware-A root directory
+- :cmake:variable:`TF_A_UNIT_TESTS_PATH` - Relative to the unit test root directory
+
+.. code-block:: cmake
+
+ # tests/new_module/new_test_suite.cmake
+ include(UnitTest)
+
+ unit_test_add_suite(
+ NAME [unique test name]
+ SOURCES
+ [source files]
+ INCLUDE_DIRECTORIES
+ [include directories]
+ COMPILE_DEFINITIONS
+ [defines]
+ )
+
+.. code-block:: cmake
+
+ # Root CMakeLists.txt
+ include(tests/new_module/new_test_suite.cmake)
+
+Example test definition
+^^^^^^^^^^^^^^^^^^^^^^^
+
+.. code-block:: cmake
+
+ unit_test_add_suite(
+ NAME memcmp
+ SOURCES
+ ${CMAKE_CURRENT_LIST_DIR}/test_memcmp.cpp
+ ${CMAKE_CURRENT_LIST_DIR}/memcmp.yml
+ INCLUDE_DIRECTORIES
+ ${TF_A_PATH}/include
+ ${TF_A_PATH}/include/lib/libc/aarch64/
+ )
+
+
+Using c-picker
+--------------
+
+c-picker is a simple tool used for detaching dependencies of the code under
+test. It can copy elements (i.e. functions, variables, etc.) from the original
+source code into generated files. This way the developer can pick functions from
+compilation units and surround them with a mocked environment.
+
+If a ``.yml`` file listed among source files the build system invokes c-picker
+and the generated ``.c`` file is implicitly added to the source file list.
+
+Example .yml file
+^^^^^^^^^^^^^^^^^
+
+In this simple example c-picker is instructed to copy the include directives and
+the ``memcmp`` function from the ``lib/libc/memcmp.c`` file. The root directory
+of the source files referenced by c-picker is the Trusted Firmware-A root
+directory.
+
+.. code-block:: yaml
+
+ elements:
+ - file: lib/libc/memcmp.c
+ type: include
+ - file: lib/libc/memcmp.c
+ type: function
+ name: memcmp
+
+
+Writing unit tests
+------------------
+
+Unit test code should be placed in ``.cpp`` files.
+
+Four-phase test pattern
+^^^^^^^^^^^^^^^^^^^^^^^
+
+All tests cases should follow the four-phase test pattern. This consists of four
+simple steps that altogether ensure the isolation between test cases. These
+steps follows below.
+
+- Setup
+- Exercise
+- Verify
+- Teardown
+
+After the teardown step all global states should be the same as they were at the
+beginning of the setup step.
+
+CppUTest
+^^^^^^^^
+
+CppUTest is an open source unit testing framework for C/C++. It is written in
+C++ so all the useful features of the language is available while testing. It
+automatically collects and runs the defined ``TEST_GROUPS`` and provides an
+interface for implementing the four-phase test pattern. Furthermore the
+framework has assertion macros for many variable types and test scenarios.
+
+Include
+'''''''
+
+The unit test source files should include the CppUTest header after all other
+headers to avoid conflicts.
+
+.. code-block:: C++
+
+ // Other headers
+ // [...]
+
+ #include "CppUTest/TestHarness.h"
+
+Test group
+''''''''''
+
+The next step is to define a test group. When multiple tests cases are written
+around testing the same function or couple related functions these tests cases
+should be part of the same test group. Basically test cases in a test group
+share have same setup/teardown sequence. In CppUTest the ``TEST_GROUP`` macro
+defines a new class which can contain member variables and functions. Special
+setup/teardown function are defined using ``TEST_SETUP`` and ``TEST_TEARDOWN``
+macros. These functions are called before/after running each test case of the
+group so all the common initilization and cleanup code should go into these
+functions.
+
+.. code-block:: C++
+
+ TEST_GROUP(List) {
+ TEST_SETUP() {
+ list = list_alloc();
+ }
+
+ TEST_TEARDOWN() {
+ list_cleanup(list);
+ }
+
+ bool has_element(int value) {
+ for (int i = 0; i < list_count (list); i++) {
+ if (list_get(i) == value) { return true; }
+ }
+ return false;
+ }
+
+ List* list;
+ };
+
+
+Test case
+'''''''''
+
+Test cases are defined by the ``TEST`` macro. This macro defines a child class
+of the test group's class so it can access the member functions and variables of
+the test group. The test case's block itself is the body of the function of the
+child class.
+
+.. code-block:: C++
+
+ TEST(List, add_one) {
+ // Exercise
+ const int test_value = 5;
+ list_add(list, test_value);
+
+ // Verify using CHECK_TRUE assertion and TEST_GROUP member function
+ CHECK_TRUE(has_element(test_value));
+ }
+
+ TEST(List, add_two) {
+ // Exercise
+ const int test_value1 = 5;
+ const int test_value2 = 6;
+ list_add(list, test_value1);
+ list_add(list, test_value2);
+
+ // Verify
+ CHECK_TRUE(has_element(test_value1));
+ CHECK_TRUE(has_element(test_value2));
+ }
+
+CppUMock
+^^^^^^^^
+
+During unit testing the dependencies of the tested functions should be replaced
+by stubs or mocks. When using mocks the developer would usually like to check if
+the function was called with corrent parameters and would like to return
+controlled values from the function. When a mocked function is called multiple
+times from the tested function maybe it should check or return different values
+on each call. This is where CppUMock comes handy.
+
+All CppUMock interactions start with calling the ``mock()`` function. This
+returs a reference to the mocking system. At this point the developer either
+wants to define expected or actual calls. This is achiveable by calling
+``expectOneCall(functionName)`` or ``expectNCalls(amount, functionName)`` or
+``actualCall(functionName)`` member functions of ``mock()`` call's return value.
+Registering expected calls are done in the test case before exercising the code
+and actual calls happen from the mocked functions.
+
+After this point the following functions can be chained:
+
+- ``onObject(object)`` - In C++ it is usually the ``this`` pointer but it can be
+ useful in C too.
+- ``with[type]Parameter(name, value)`` - Specifying and checking call parameters
+- ``andReturnValue(result)`` - Specifying return value when defining expected
+ call
+- ``return[type]Value()`` - Returning value from function
+
+The mocking system has two important functions. ``mock().checkExpectation()``
+checks if all the expected calls have been fulfilled and and the
+``mock().clear()`` removes all the expected calls from CppUMock's registry.
+These two functions are usually called from the ``TEST_TEARDOWN`` function
+because there should not be any crosstalk between test cases through the mocking
+system.
+
+CppUMock's typical use-case is shown below by a simple example of the
+``print_to_eeprom`` function.
+
+.. code-block:: C++
+
+ int eeprom_write(const char* str); /* From eeprom.h */
+
+ int printf_to_eeprom(const char* format, ...) {
+ char buffer[256];
+ int length, written_bytes = 0, eeprom_write_result;
+ va_list args;
+
+ va_start(args, format);
+ length = vsnprintf(buffer, sizeof(buffer), format, args);
+ va_end(args);
+
+ if (length < 0) {
+ return length;
+ }
+
+ while(written_bytes < length) {
+ eeprom_write_result = eeprom_write(&buffer[written_bytes]);
+ if (eeprom_write_result < 0) {
+ return eeprom_write_result;
+ }
+ written_bytes += eeprom_write_result;
+ }
+
+ return written_bytes;
+ }
+
+Having the code snipped above a real life usage of the function would look like
+something shown in the following sequence diagram.
+
+.. uml:: resources/sequence_print_to_eeprom.puml
+
+It would be really hard to test unit this whole system so all the lower layers
+should be separated and mock on the first possible level. In the following
+example the ``print_to_eeprom`` function is being tested and the
+``eeprom_write`` function is mocked. In test cases where ``eeprom_write``
+function is expected to be called the test case should first call the
+``expect_write`` function. This registers an expected call to CppUMocks internal
+database and when the call actually happens it matches the call parameters with
+the entry in the database. It also returns the previously specified value.
+
+.. code-block:: C++
+
+ TEST_GROUP(printf_to_eeprom) {
+ TEST_TEARDOWN() {
+ mock().checkExpectations();
+ mock().clear();
+ }
+
+ void expect_write(const char* str, int result) {
+ mock().expectOneCall("eeprom_write").withStringParameter("str", str).
+ andReturnValue(result);
+ }
+ };
+
+ /* Mocked function */
+ int eeprom_write(const char* str) {
+ return mock().actualCall("eeprom_write").withStringParameter("str", str).
+ returnIntValue();
+ }
+
+ TEST(printf_to_eeprom, empty) {
+ LONGS_EQUAL(0, printf_to_eeprom(""))
+ }
+
+ TEST(printf_to_eeprom, two_writes) {
+ expect_write("hello1hello2", 6);
+ expect_write("hello2", 6);
+ LONGS_EQUAL(12, printf_to_eeprom("hello%dhello%d", 1, 2))
+ }
+
+ TEST(printf_to_eeprom, error) {
+ expect_write("hello", -1);
+ LONGS_EQUAL(-1, printf_to_eeprom("hello"))
+ }
+
+This how the ``printf_to_eeprom/two_writes`` test case's sequence diagram looks
+like after mocking ``eeprom_write``. The test case became able to check the
+parameters of multiple calls and it could return controlled values.
+
+.. uml:: resources/sequence_print_to_eeprom_mock.puml
+
+
+Analyzing code coverage
+-----------------------
+
+The code coverage reports can be easily used for finding untested parts of the
+code. The two main parts of the coverage report are the line coverage and the
+branch coverage. Line coverage shows that how many times the tests ran the given
+line of the source code. It is beneficial to increase the line coverage however
+100% line coverage is still not enough to consider the code fully tested.
+
+Let's have a look on the following example.
+
+.. code-block:: C++
+
+ void set_pointer_value(unsigned int id, unsigned int value) {
+ unsigned int *pointer;
+
+ if (id < MAX_ID) {
+ pointer = get_pointer(id);
+ }
+
+ *pointer = value;
+ }
+
+The 100% line coverage is achievable by testing the function with an ``id``
+value smaller than ``MAX_ID``. However if an ``id`` larger than or equal to
+``MAX_ID`` is used as a parameter of this function it will try to write to a
+memory address pointed by an uninitialized variable. To catch untested
+conditions like this the branch coverage comes handy. It will show that only one
+branch of the ``if`` statement has been tested as the condition was always true
+in the tests.
+
+
+--------------
+
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
diff --git a/docs/index.rst b/docs/index.rst
index ff41bc7..76f8f3f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,17 +1,55 @@
-Welcome to TF-A Unit Tests's documentation!
-===========================================
+Trusted Firmware-A Unit Test Documentation
+==========================================
.. toctree::
- :maxdepth: 2
- :caption: Contents:
+ :maxdepth: 1
+ :hidden:
+ :numbered:
+
+ Home<self>
+ user_guide
+ implementing_tests
+ components/index
contributing
- build_instructions
- software_requirements
- user_guide
+ glossary
+ license
+
+This repository contains the unit testing system and the unit tests of the
+`Trusted Firmware-A`_ (TF-A) project. These tests are meant to run on host
+machine and used to cover platform independent code on the unit test level. In
+this case a unit is considered to be a C function or couple related functions.
+Each unit test suite compiles into a binary which can be run and debugged as any
+ordinary executable.
+
+The system uses CppUTest as unit testing framework. The tests are written in C++
+in order to be able to use CppUTests' all useful features like the automatic
+collection of test cases and the CppUMock mocking system.
+
+Separating dependencies apart from the code under test is a crutial step in unit
+testing systems. In many cases this can be easily done by linking mocked
+functions to the tested code but sometimes it's difficult like when the code
+under test and its dependencies are in the same compilation unit. For separating
+the code under test from its dependencies a tool called c-picker can be used. It
+can pick pieces of code (functions, variables, etc.) based on descriptor files.
+
+The build system is based on CMake. The repository contains CMake modules for
+defining unit test suites. It also invokes c-picker if a descriptor file is
+listed among the test sources. CMake has a built in test driver system called
+ctest. It runs all the test binaries and produces an well structured output.
+Test filtering and parallel test run is also available.
+
+For measuring unit test coverage lcov is utilized. The coverage of c-picker
+generated sources is mapped to the original sources files. Coverage currently
+only works with GCC.
+
+As a next step start with reading the :ref:`User guide` and the
+:ref:`Implementing tests` section of this manual. For detailed description of
+the components check the :ref:`Component user manuals` section.
+
--------------
-*Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
+*Copyright (c) 2019, Arm Limited. All rights reserved.*
-SPDX-License-Identifier: BSD-3-Clause
+.. _`Trusted Firmware-A`: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/about
\ No newline at end of file
diff --git a/docs/license.rst b/docs/license.rst
new file mode 100644
index 0000000..56e0873
--- /dev/null
+++ b/docs/license.rst
@@ -0,0 +1,48 @@
+License
+=======
+
+The software is provided under a BSD-3-Clause license (below). Contributions to
+this project are accepted under the same license with developer sign-off as
+described in the :ref:`Contributor's guide`.
+
+::
+
+ Copyright (c) [XXXX-]YYYY, <OWNER>. All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without modification,
+ are permitted provided that the following conditions are met:
+
+ - Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+ - Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+ - Neither the name of Arm nor the names of its contributors may be used to
+ endorse or promote products derived from this software without specific
+ prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+SPDX Identifiers
+----------------
+
+Individual files contain the following tag instead of the full license text.
+
+::
+
+ SPDX-License-Identifier: BSD-3-Clause
+
+This enables machine processing of license information based on the SPDX
+License Identifiers that are here available: http://spdx.org/licenses/
+
diff --git a/docs/resources/TrustedFirmware-Logo_standard-white.png b/docs/resources/TrustedFirmware-Logo_standard-white.png
new file mode 100644
index 0000000..e7bff71
--- /dev/null
+++ b/docs/resources/TrustedFirmware-Logo_standard-white.png
Binary files differ
diff --git a/docs/resources/sequence_print_to_eeprom.puml b/docs/resources/sequence_print_to_eeprom.puml
new file mode 100644
index 0000000..3f8c108
--- /dev/null
+++ b/docs/resources/sequence_print_to_eeprom.puml
@@ -0,0 +1,39 @@
+@startuml
+participant Application as APP
+
+activate APP
+APP -> Print: print_to_eeprom(fmt, ...)
+activate Print
+
+ Print -> EEPROM: eeprom_write(buffer)
+ activate EEPROM
+
+ EEPROM -> I2C: i2c_write(address, buffer)
+ activate I2C
+
+ I2C -> Hardware: I2C transaction
+ activate Hardware
+ Hardware --> I2C: ack/nack
+ deactivate Hardware
+
+ I2C --> EEPROM: result
+ deactivate I2C
+
+ EEPROM -> I2C: i2c_write(address, buffer+ result)
+ activate I2C
+
+ I2C -> Hardware: I2C transaction
+ activate Hardware
+ Hardware --> I2C: ack/nack
+ deactivate Hardware
+
+ I2C --> EEPROM: result
+ deactivate I2C
+
+ EEPROM --> Print: result
+ deactivate EEPROM
+
+Print -> APP: result
+deactivate Print
+
+@enduml
\ No newline at end of file
diff --git a/docs/resources/sequence_print_to_eeprom_mock.puml b/docs/resources/sequence_print_to_eeprom_mock.puml
new file mode 100644
index 0000000..67a4325
--- /dev/null
+++ b/docs/resources/sequence_print_to_eeprom_mock.puml
@@ -0,0 +1,41 @@
+@startuml
+participant "Test case" as TC
+participant "Code under test" as CUT
+
+group Setting expectations
+ activate TC
+ TC -> Mock: expect_write("hello1hello2", 6)
+ activate Mock
+ deactivate Mock
+ TC -> Mock: expect_write("hello2", 6)
+ activate Mock
+ deactivate Mock
+end
+
+TC -> CUT: print_to_eeprom("hello%dhello%d", 1, 2)
+activate CUT
+
+ group Using mocked interface
+ CUT -> Mock: eeprom_write("hello1hello2")
+ activate Mock
+ Mock --> CUT: 6
+ deactivate Mock
+
+ CUT -> Mock: eeprom_write("hello2")
+ activate Mock
+ Mock --> CUT: 6
+ deactivate Mock
+ end
+
+CUT --> TC: 6 + 6 = 12
+deactivate CUT
+
+TC -> Mock: mock().checkExpectation()
+activate Mock
+deactivate Mock
+
+TC -> Mock: mock().clear()
+activate Mock
+deactivate Mock
+
+@enduml
\ No newline at end of file
diff --git a/docs/software_requirements.rst b/docs/software_requirements.rst
deleted file mode 100644
index 2cd19d1..0000000
--- a/docs/software_requirements.rst
+++ /dev/null
@@ -1,13 +0,0 @@
-Software Requirements
-=====================
-
-TODO: add info here about:
- * what is needed to build and run unit tests
- * list of supported dev env's
- * setup instructions for windows and linux
-
---------------
-
-*Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
-
-SPDX-License-Identifier: BSD-3-Clause
diff --git a/docs/user_guide.rst b/docs/user_guide.rst
index a3199d7..aec9b1d 100644
--- a/docs/user_guide.rst
+++ b/docs/user_guide.rst
@@ -1,10 +1,222 @@
-User Guide
+User guide
==========
-TODO: add text here
+This page describes how to get started with compiling and running unit tests on
+the host machine.
+
+Host machine requirements
+-------------------------
+
+The system has been successfully tested on the following platforms:
+
+- Ubuntu 19.04
+- Ubuntu 18.04
+- Arch Linux
+- MSYS2 MinGW64
+
+Tools
+-----
+
+The following applications are expected to be installed in the build machine:
+
+- CMake >=3.11
+
+- Python >=3.4
+
+- c-picker
+
+ - pyyaml
+
+ - clang (pip package if not included in libclang)
+
+ - libclang
+
+- git
+
+- Toolchain
+
+- Native build system
+
+Ubuntu 19.04
+^^^^^^^^^^^^
+
+On Ubuntu 19.04 use the following commands to install the required tools:
+
+::
+
+ sudo apt-get install cmake git python3 python3-pip libclang-dev build-essential
+ sudo pip3 install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker
+
+Ubuntu 18.04
+^^^^^^^^^^^^
+
+The official Ubuntu 18.04 package repository only contains CMake 3.10 which not
+satisfies the requirements for building unit tests. Fortunately there's an
+official CMake APT repository provided by Kitware: https://apt.kitware.com/ By
+adding this server to the repository list an up-to-date version of CMake can be
+installed on Ubuntu 18.04.
+
+::
+
+ wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc \
+ 2>/dev/null | sudo apt-key add -
+ sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
+
+ sudo apt-get install cmake git python3 python3-pip libclang-dev build-essential
+ sudo pip3 install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker
+
+Arch Linux
+^^^^^^^^^^
+
+On Arch Linux use the following commands to install the required tools:
+
+::
+
+ sudo pacman -Sy cmake git python python-pip python-yaml make gcc clang
+ sudo pip install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker
+
+MSYS2 MinGW64
+^^^^^^^^^^^^^
+
+::
+
+ pacman -Sy mingw-w64-x86_64-cmake git mingw-w64-x86_64-python3 \
+ mingw-w64-x86_64-python3-pip make mingw-w64-x86_64-gcc mingw-w64-x86_64-clang
+ pip install git+https://gerrit.oss.arm.com/iot-sw/shared/c-picker
+
+
+Getting unit test source code
+-----------------------------
+
+Download the unit test source code using the following command:
+
+::
+
+ git clone https://gerrit.oss.arm.com/trusted-firmware/tf-a-unit-tests.git
+
+
+Getting Trusted Firmware-A source code
+--------------------------------------
+
+Download the TF-A source code using the following command:
+
+::
+
+ git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
+
+
+Building unit tests
+-------------------
+
+Building unit tests start with running CMake which will check all the
+prerequisites and generate the native build system's input files. This example
+uses Unix Makefiles. Unit tests exercise the code directly by compiling it into
+the same binary as the test code. In this case the tests need to know where to
+find the TF-A source files and this is why the :cmake:variable:`TF_A_PATH` is
+specified.
+
+::
+
+ cd tf-a-unit-tests
+ mkdir build
+ cd build
+ cmake -DTF_A_PATH=../../trusted-firmware-a -G"Unix Makefiles" ..
+
+After running the previous steps the makefiles are generated into the build
+directory so make can build unit tests. During unit test development if only the
+source files have changed it's not necessary to re-run cmake it's only needed to
+run make as shown below.
+
+::
+
+ make -j
+
+For building single unit tests suites the test's name can be used as a makefile
+target. Let's assume there's a test suite called bl1_fwu.
+
+::
+
+ make bl1_fwu
+
+
+Running unit tests
+------------------
+
+CMake provides a built-in tool called ctest for running all the tests using a
+single command. It is also able to filter tests or run them in parallel for
+speeding up the tests process. Run all the tests using the following command:
+
+::
+
+ ctest
+
+Each unit test suite has its own executable. The easiest way of running single
+test suite is running it as a simple executable.
+
+::
+
+ ./bl1_fwu
+
+
+Debugging unit tests
+--------------------
+
+As it was mentioned in the previous section test suites are basically separate
+executables so they can be debugged as any other native applications on the host
+machine. In a Linux environment gdb or IDE's built-in debugger can be utilized
+for debugging.
+
+::
+
+ gdb ./bl1_fwu
+
+
+Measuring code coverage
+-----------------------
+
+Inspecting code coverage is a useful method for detecting parts of the code
+which is not exercised by tests. The build system includes an option for
+generating code coverage report of the unit tests. The coverage is processed by
+``lcov`` which needs to be installed for this feature. Also the coverage
+measurement in only available when GCC is used as a compiler.
+
+First of all the :cmake:variable:`COVERAGE` option has to be enabled by using
+the following command line when invoking CMake.
+
+::
+
+ cmake -DTF_A_PATH=../../trusted-firmware-a -DCOVERAGE=ON -G"Unix Makefiles" ..
+
+This makes CMake to build the binaries with coverage information included. The
+rest of the build process works the same way as before.
+
+Before collecting coverage info and generating reports the tests must be run as
+the coverage is a runtime measurement. See section `Running unit tests`_ for
+more information about running unit tests.
+
+The :cmake:variable:`COVERAGE` option adds two new build targets called
+``coverage`` and ``coverage_report``. They can be used simply by running the
+following commands if ``make`` is used as a build system.
+
+::
+
+ make coverage
+ make coverage_report
+
+The ``coverage`` target generates lcov info files for further processing. If
+there are coverage files available from different sources (i.e. coverages of
+other tests) they can be merged with the unit test coverage file and evaluated
+together. Currently two coverage info files are generated during the build. One
+of them contains the coverage of code under test (i.e. Trusted Firmware-A) and
+the other one has the coverage of the unit tests themselves.
+
+The ``coverage_report`` target generates a HTML report from the coverage info
+files. The coverage reports can be found in the build directory's subdirectories
+having ``-coverage`` suffix in their names. The report shows the directory
+structure of the code and each file can be inspected individually. Line,
+function and branch coverage is included.
+
--------------
-*Copyright (c) 2019, Arm Limited and Contributors. All rights reserved.*
-
-SPDX-License-Identifier: BSD-3-Clause
+*Copyright (c) 2019-2020, Arm Limited. All rights reserved.*
