Merge pull request #10446 from gilles-peskine-arm/migration-guide-4.0-consolidate

Consolidate migration guide
tree: 6ee0ba4582db17ea9216f5819f3cd3fc0520ddc2
  1. .github/
  2. 3rdparty/
  3. ChangeLog.d/
  4. cmake/
  5. configs/
  6. docs/
  7. doxygen/
  8. include/
  9. library/
  10. pkgconfig/
  11. programs/
  12. scripts/
  13. tests/
  14. .gitattributes
  15. .gitignore
  16. .gitmodules
  17. .globalrc
  18. .mypy.ini
  19. .pylintrc
  20. .readthedocs.yaml
  21. .travis.yml
  22. .uncrustify.cfg
  23. BRANCHES.md
  24. BUGS.md
  25. ChangeLog
  26. CMakeLists.txt
  27. CONTRIBUTING.md
  28. DartConfiguration.tcl
  29. dco.txt
  30. LICENSE
  31. README.md
  32. SECURITY.md
  33. SUPPORT.md
README.md

README for Mbed TLS

Mbed TLS is a C library that implements X.509 certificate manipulation and the TLS and DTLS protocols. Its small code footprint makes it suitable for embedded systems. Mbed TLS includes the TF-PSA-Crypto repository that provides an implementation of the PSA Cryptography API.

Configuration

Configuration options related to X.509 and TLS are available in include/mbedtls/mbedtls_config.h, while cryptography and platform options are located in the TF-PSA-Crypto configuration file tf-psa-crypto/include/psa/crypto_config.h.

With the default platform options, Mbed TLS should build out of the box on most systems.

These configuration files can be edited manually, or programmatically using the Python script scripts/config.py (run with --help for usage instructions).

We provide some non-standard configurations focused on specific use cases in the configs/ directory. You can read more about those in configs/README.txt.

Documentation

The main Mbed TLS documentation is available via ReadTheDocs.

To generate a local copy of the library documentation in HTML format, tailored to your compile-time configuration:

  1. Make sure that Doxygen is installed.
  2. Run cmake -B /path/to/build_dir /path/to/mbedtls/source
  3. Run cmake --build /path/to/build_dir --target mbedtls-apidoc
  4. Open one of the main generated HTML files:
    • apidoc/index.html
    • apidoc/modules.html or apidoc/topics.html

For other sources of documentation, see the SUPPORT document.

Compiling

We use CMake to configure and drive our build process. Three libraries are built: libtfpsacrypto, libmbedx509, and libmbedtls. Note that libmbedtls depends on libmbedx509 and libtfpsacrypto, and libmbedx509 depends on libtfpsacrypto. As a result, some linkers will expect flags to be in a specific order, for example the GNU linker wants -lmbedtls -lmbedx509 -ltfpsacrypto. The cryptographic library libtfpsacrypto is also provided under its legacy name, libmbedcrypto.

Tool versions

You need the following tools to build the library from the main branch with the provided CMake files. Mbed TLS minimum tool version requirements are set based on the versions shipped in the latest or penultimate (depending on the release cadence) long-term support releases of major Linux distributions, namely at time of writing: Ubuntu 22.04, RHEL 9, and SLES 15 SP4.

  • CMake 3.20.2 or later.
  • A build system like Make or Ninja for which CMake can generate build files.
  • A C99 toolchain (compiler, linker, archiver). We actively test with GCC 5.4, Clang 3.8, Arm Compiler 6, and Visual Studio 2017 Compiler. More recent versions should work. Slightly older versions may work.
  • Python 3.8 or later to generate the test code. Python is also needed to build the development branch (see next section).
  • Perl to run the tests, and to generate some source files in the development branch.
  • Doxygen 1.8.14 or later (if building the documentation; slightly older versions should work).

Git usage

The supported branches (see BRANCHES.md) use Git submodules. They contain two submodules: the framework submodule and the tf-psa-crypto submodule, except for the 3.6 LTS branch, which contains only the framework submodule. Release tags also use Git submodules.

After cloning or checking out a branch or tag, run: git submodule update --init --recursive to initialize and update the submodules before building.

However, the official source release tarballs (e.g. mbedtls-4.0.0-beta.tar.bz2) include the contents of the submodules.

Generated source files in the development branch

The source code of Mbed TLS includes some files that are automatically generated by scripts and whose content depends only on the Mbed TLS source, not on the platform or on the library configuration. These files are not included in the development branch of Mbed TLS, but the generated files are included in official releases. This section explains how to generate the missing files in the development branch.

The following tools are required:

  • Perl, for some library source files.
  • Python 3 and some Python packages, for some library source files, sample programs and test data. To install the necessary packages, run:
    python3 -m pip install --user -r scripts/basic.requirements.txt
    
    Depending on your Python installation, you may need to invoke python instead of python3. To install the packages system-wide or in a virtual environment, omit the --user option.
  • A C compiler for the host platform, for some test data.

The scripts that generate the configuration-independent files will look for a host C compiler in the following places (in order of preference):

  1. The HOSTCC environment variable. This can be used if CC is pointing to a cross-compiler.
  2. The CC environment variable.
  3. An executable called cc in the current path.

Note: If you have multiple toolchains installed, it is recommended to set CC or HOSTCC to the intended host compiler before generating the files.

Any of the following methods are available to generate the configuration-independent files:

  • On non-Windows systems, when not cross-compiling, CMake generates the required files automatically.
  • Run framework/scripts/make_generated_files.py to generate all the configuration-independent files.

CMake

In order to build the libraries using CMake in a separate directory (recommended), just enter at the command line:

mkdir /path/to/build_dir && cd /path/to/build_dir
cmake /path/to/mbedtls_source
cmake --build .

In order to run the tests, enter:

ctest

The test suites need Python to be built. If you don't have Python installed, you'll want to disable the test suites with:

cmake -DENABLE_TESTING=Off /path/to/mbedtls_source

To configure CMake for building shared libraries, use:

cmake -DUSE_SHARED_MBEDTLS_LIBRARY=On /path/to/mbedtls_source

There are many different build types available with CMake. Most of them are available for gcc and clang, though some are compiler-specific:

  • Release. This generates the default code without any unnecessary information in the binary files.
  • Debug. This generates debug information and disables optimization of the code.
  • Coverage. This generates code coverage information in addition to debug information.
  • ASan. This instruments the code with AddressSanitizer to check for memory errors. (This includes LeakSanitizer, with recent version of gcc and clang.) (With recent version of clang, this mode also instruments the code with UndefinedSanitizer to check for undefined behaviour.)
  • ASanDbg. Same as ASan but slower, with debug information and better stack traces.
  • MemSan. This instruments the code with MemorySanitizer to check for uninitialised memory reads.
  • MemSanDbg. Same as MemSan but slower, with debug information, better stack traces and origin tracking.
  • Check. This activates the compiler warnings that depend on optimization and treats all warnings as errors.
  • TSan. This instruments the code with ThreadSanitizer to detect data races and other threading-related concurrency issues at runtime.
  • TSanDbg. Same as TSan but slower, with debug information, better stack traces and origin tracking.

Switching build types in CMake is simple. For debug mode, enter at the command line:

cmake -D CMAKE_BUILD_TYPE=Debug /path/to/mbedtls_source

To list other available CMake options, use:

cmake -LH

Note that, with CMake, you can't adjust the compiler or its flags after the initial invocation of cmake. This means that CC=your_cc make and make CC=your_cc will not work (similarly with CFLAGS and other variables). These variables need to be adjusted when invoking cmake for the first time, for example:

CC=your_cc cmake /path/to/mbedtls_source

If you already invoked cmake and want to change those settings, you need to invoke the configuration phase of CMake again with the new settings.

Note that it is possible to build in-place; this will however overwrite the legacy Makefiles still used for testing purposes (see scripts/tmp_ignore_makefiles.sh if you want to prevent git status from showing them as modified). In order to do so, from the Mbed TLS source directory, use:

cmake .
cmake --build .

If you want to change CC or CFLAGS afterwards, you will need to remove the CMake cache. This can be done with the following command using GNU find:

find . -iname '*cmake*' -not -name CMakeLists.txt -exec rm -rf {} +

You can now make the desired change:

CC=your_cc cmake .
cmake --build .

Regarding variables, also note that if you set CFLAGS when invoking cmake, your value of CFLAGS doesn't override the content provided by CMake (depending on the build mode as seen above), it's merely prepended to it.

Consuming Mbed TLS

Mbed TLS provides a CMake package configuration file for consumption as a dependency in other CMake projects. You can load its CMake targets with:

find_package(MbedTLS REQUIRED)

You can help CMake find the package:

  • By setting the variable MbedTLS_DIR to ${YOUR_MBEDTLS_BUILD_DIR}/cmake, as shown in programs/test/cmake_package/CMakeLists.txt, or
  • By adding the Mbed TLS installation prefix to CMAKE_PREFIX_PATH, as shown in programs/test/cmake_package_install/CMakeLists.txt.

After a successful find_package(MbedTLS), the following imported targets are available:

  • MbedTLS::tfpsacrypto, the crypto library
  • MbedTLS::mbedtls, the TLS library
  • MbedTLS::mbedx509, the X.509 library

You can then use these directly through target_link_libraries():

add_executable(xyz)

target_link_libraries(xyz
    PUBLIC MbedTLS::mbedtls
           MbedTLS::tfpsacrypto
           MbedTLS::mbedx509)

This will link the Mbed TLS libraries to your library or application, and add its include directories to your target (transitively, in the case of PUBLIC or INTERFACE link libraries).

Mbed TLS as a subproject

Mbed TLS supports being built as a CMake subproject. One can use add_subdirectory() from a parent CMake project to include Mbed TLS as a subproject.

Example programs

We've included example programs for a lot of different features and uses in programs/. Please note that the goal of these sample programs is to demonstrate specific features of the library, and the code may need to be adapted to build a real-world application.

Tests

Mbed TLS includes an elaborate test suite in tests/ that initially requires Python to generate the tests files (e.g. test_suite_ssl.c). These files are generated from a function file (e.g. suites/test_suite_ssl.function) and a data file (e.g. suites/test_suite_ssl.data). The function file contains the test functions. The data file contains the test cases, specified as parameters that will be passed to the test function.

For machines with a Unix shell and OpenSSL (and optionally GnuTLS) installed, additional test scripts are available:

  • tests/ssl-opt.sh runs integration tests for various TLS options (renegotiation, resumption, etc.) and tests interoperability of these options with other implementations.
  • tests/compat.sh tests interoperability of every ciphersuite with other implementations.
  • tests/scripts/depends.py tests builds in configurations with a single curve, key exchange, hash, cipher, or pkalg on.
  • tests/scripts/all.sh runs a combination of the above tests, plus some more, with various build options (such as ASan, full mbedtls_config.h, etc).

Instead of manually installing the required versions of all tools required for testing, it is possible to use the Docker images from our CI systems, as explained in our testing infrastructure repository.

Porting Mbed TLS

Mbed TLS can be ported to many different architectures, OS's and platforms. Before starting a port, you may find the following Knowledge Base articles useful:

Mbed TLS is mostly written in portable C99; however, it has a few platform requirements that go beyond the standard, but are met by most modern architectures:

  • Bytes must be 8 bits.
  • All-bits-zero must be a valid representation of a null pointer.
  • Signed integers must be represented using two's complement.
  • int and size_t must be at least 32 bits wide.
  • The types uint8_t, uint16_t, uint32_t and their signed equivalents must be available.
  • Mixed-endian platforms are not supported.
  • SIZE_MAX must be at least as big as INT_MAX and UINT_MAX.

License

Unless specifically indicated otherwise in a file, Mbed TLS files are provided under a dual Apache-2.0 OR GPL-2.0-or-later license. See the LICENSE file for the full text of these licenses, and the 'License and Copyright' section in the contributing guidelines for more information.

Contributing

We gratefully accept bug reports and contributions from the community. Please see the contributing guidelines for details on how to do this.

Contact