blob: 8a7a981234ed7a851abc6c41ac0d6c5a75e6d52f [file] [log] [blame]
Olivier Deprez8c4cb2d2023-10-27 16:07:11 +02001Foreword
2========
3
4- This document describes the FF-A implementation from `[1]`_ for the
5 configuration where the SPMC resides at S-EL2 on platforms implementing the
6 FEAT_SEL2 architecture extension.
7- It is not an architecture specification and it might provide assumptions on
8 sections mandated as implementation-defined in the specification.
9- It covers the implications of TF-A used as a bootloader, and Hafnium used as a
10 reference code base for an SPMC.
11
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020012Terminology
J-Alvesf7490db2023-10-19 17:57:22 +010013===========
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020014
15- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines
16 (or partitions) in the normal world.
17- The term SPMC refers to the S-EL2 component managing secure partitions in
18 the secure world when the FEAT_SEL2 architecture extension is implemented.
19- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure
20 partition and implementing the FF-A ABI on platforms not implementing the
21 FEAT_SEL2 architecture extension.
22- The term VM refers to a normal world Virtual Machine managed by an Hypervisor.
23- The term SP refers to a secure world "Virtual Machine" managed by an SPMC.
24
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020025Sample reference stack
26======================
27
28The following diagram illustrates a possible configuration when the
J-Alves5eafd222023-10-26 14:19:21 +010029FEAT_SEL2 architecture extension is implemented, showing the |SPMD|
30and |SPMC|, one or multiple secure partitions, with an optional
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020031Hypervisor:
32
J-Alvesc1693772023-10-26 12:41:53 +010033.. image:: ../resources/diagrams/Hafnium_overview_SPMD.png
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020034
J-Alves5eafd222023-10-26 14:19:21 +010035Integration with TF-A (Bootloader and SPMD)
36===========================================
37
38The `TF-A project`_ provides the reference implementation for the secure monitor
39for Arm A class devices, executing at EL3. It includes the implementation of the
40|SPMD|, which manages the world-switch, to relay the FF-A calls to the |SPMC|.
41
42TF-A also serves as the system bootlader, and it was used in the reference
J-Alvesd547d6d2024-05-14 14:59:54 +010043implementation for the SPMC and SPs.
J-Alves5eafd222023-10-26 14:19:21 +010044SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.).
45Thus they are supplied as distinct signed entities within the FIP flash
46image. The FIP image itself is not signed hence this provides the ability
47to upgrade SPs in the field.
48
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020049TF-A build options
J-Alves5eafd222023-10-26 14:19:21 +010050------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020051
J-Alvesd8094162023-10-26 12:44:33 +010052This section explains the TF-A build options for an FF-A based SPM, in which SPMD
53is located at EL3.
54
55This is a step needed for integrating Hafnium as the S-EL2 SPMC and
56the TF-A as SPMD, together making the SPM component.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020057
58- **SPD=spmd**: this option selects the SPMD component to relay the FF-A
59 protocol from NWd to SWd back and forth. It is not possible to
60 enable another Secure Payload Dispatcher when this option is chosen.
61- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception
62 level to being at S-EL2. It defaults to enabled (value 1) when
J-Alvesd8094162023-10-26 12:44:33 +010063 SPD=spmd is chosen.The context save/restore routine and exhaustive list
64 of registers is visible at `[4]`_. When set the reference software stack
65 assumes enablement of FEAT_PAuth, FEAT_BTI and FEAT_MTE architecture
66 extensions.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020067- **SP_LAYOUT_FILE**: this option specifies a text description file
68 providing paths to SP binary images and manifests in DTS format
J-Alves5eafd222023-10-26 14:19:21 +010069 (see `Secure Partitions Layout File`_). It is required when ``SPMD_SPM_AT_SEL2``
J-Alvesd8094162023-10-26 12:44:33 +010070 is enabled, i.e. when multiple secure partitions are to be loaded by BL2 on
71 behalf of the SPMC.
72- **BL32** option is re-purposed to specify the SPMC image. It can specify either
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020073 the Hafnium binary path (built for the secure world) or the path to a TEE
74 binary implementing FF-A interfaces.
J-Alvesd8094162023-10-26 12:44:33 +010075- **BL33** option to specify normal world loader such as U-Boot or the UEFI
76 framework payload, which would use FF-A calls during runtime to interact with
77 Hafnium as the SPMC.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020078
J-Alvesd8094162023-10-26 12:44:33 +010079As a result of configuring ``SPD=spmd`` and ``SPMD_SPM_AT_SEL2`` TF-A provides
80context save/restore operations when entering/exiting an EL2 execution context.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020081
J-Alvesd8094162023-10-26 12:44:33 +010082There are other build options that relate support other valid FF-A
83system configurations where the SPMC is implemented at S-EL1 and EL3.
84Note that they conflict with those needed to integrate with Hafnium as the SPMC.
85For more details refer to |TF-A| build options `[10]`_.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020086
87Sample TF-A build command line when FEAT_SEL2 architecture extension is
J-Alvesd8094162023-10-26 12:44:33 +010088implemented and the SPMC is located at S-EL2, for Arm's FVP platform:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020089
90.. code:: shell
91
92 make \
93 CROSS_COMPILE=aarch64-none-elf- \
94 PLAT=fvp \
95 SPD=spmd \
96 ARM_ARCH_MINOR=5 \
97 BRANCH_PROTECTION=1 \
J-Alves874737a2024-03-20 17:30:24 +000098 ENABLE_FEAT_MTE2=1 \
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020099 BL32=<path-to-hafnium-binary> \
100 BL33=<path-to-bl33-binary> \
101 SP_LAYOUT_FILE=sp_layout.json \
102 all fip
103
104Sample TF-A build command line when FEAT_SEL2 architecture extension is
105implemented, the SPMC is located at S-EL2, and enabling secure boot:
106
107.. code:: shell
108
109 make \
110 CROSS_COMPILE=aarch64-none-elf- \
111 PLAT=fvp \
112 SPD=spmd \
113 ARM_ARCH_MINOR=5 \
114 BRANCH_PROTECTION=1 \
J-Alves874737a2024-03-20 17:30:24 +0000115 ENABLE_FEAT_MTE2=1 \
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200116 BL32=<path-to-hafnium-binary> \
117 BL33=<path-to-bl33-binary> \
118 SP_LAYOUT_FILE=sp_layout.json \
119 MBEDTLS_DIR=<path-to-mbedtls-lib> \
120 TRUSTED_BOARD_BOOT=1 \
121 COT=dualroot \
122 ARM_ROTPK_LOCATION=devel_rsa \
123 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \
124 GENERATE_COT=1 \
125 all fip
126
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200127FVP model invocation
J-Alves5eafd222023-10-26 14:19:21 +0100128--------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200129
130The FVP command line needs the following options to exercise the S-EL2 SPMC:
131
132+---------------------------------------------------+------------------------------------+
133| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, |
134| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. |
135+---------------------------------------------------+------------------------------------+
136| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the |
137| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. |
138| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | |
139| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | |
140| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | |
141| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | |
142| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | |
143| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | |
144+---------------------------------------------------+------------------------------------+
145| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. |
146| - cluster1.has_branch_target_exception=1 | |
147+---------------------------------------------------+------------------------------------+
148| - cluster0.has_pointer_authentication=2 | Implements FEAT_PAuth |
149| - cluster1.has_pointer_authentication=2 | |
150+---------------------------------------------------+------------------------------------+
151| - cluster0.memory_tagging_support_level=2 | Implements FEAT_MTE2 |
152| - cluster1.memory_tagging_support_level=2 | |
153| - bp.dram_metadata.is_enabled=1 | |
154+---------------------------------------------------+------------------------------------+
155
156Sample FVP command line invocation:
157
158.. code:: shell
159
160 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \
161 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \
162 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \
163 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \
164 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \
165 -C bp.pl011_uart2.out_file=fvp-uart2.log \
166 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 \
167 -C cluster0.has_pointer_authentication=2 -C cluster1.has_pointer_authentication=2 \
168 -C cluster0.has_branch_target_exception=1 -C cluster1.has_branch_target_exception=1 \
169 -C cluster0.memory_tagging_support_level=2 -C cluster1.memory_tagging_support_level=2 \
170 -C bp.dram_metadata.is_enabled=1 \
171 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B \
172 -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 \
173 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 \
174 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0
175
J-Alves5eafd222023-10-26 14:19:21 +0100176SPMC Configuration
177==================
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200178
J-Alves5eafd222023-10-26 14:19:21 +0100179This section details the configuration files required to deploy Hafnium as the SPMC,
180along with those required to configure each secure partion.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200181
J-Alves5eafd222023-10-26 14:19:21 +0100182SPMC Manifest
183-------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200184
J-Alves5eafd222023-10-26 14:19:21 +0100185This manifest contains the SPMC *attribute* node consumed by the SPMD at boot
186time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves
187two different cases:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200188
J-Alves5eafd222023-10-26 14:19:21 +0100189The SPMC manifest is used by the SPMD to setup the environment required by the
190SPMC to run at S-EL2. SPs run at S-EL1 or S-EL0.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200191
J-Alves5eafd222023-10-26 14:19:21 +0100192.. code:: shell
193
194 attribute {
195 spmc_id = <0x8000>;
196 maj_ver = <0x1>;
197 min_ver = <0x1>;
198 exec_state = <0x0>;
199 load_address = <0x0 0x6000000>;
200 entrypoint = <0x0 0x6000000>;
201 binary_size = <0x60000>;
202 };
203
J-Alvesc28ee3e2024-05-14 18:29:26 +0100204* *spmc_id* defines the endpoint ID value that SPMC can query through
J-Alves5eafd222023-10-26 14:19:21 +0100205 ``FFA_ID_GET``.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100206* *maj_ver/min_ver*. SPMD checks provided FF-A version versus its internal
J-Alves5eafd222023-10-26 14:19:21 +0100207 version and aborts if not matching.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100208* *exec_state* defines the SPMC execution state (AArch64 or AArch32).
J-Alves5eafd222023-10-26 14:19:21 +0100209 Notice Hafnium used as a SPMC only supports AArch64.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100210* *load_address* and *binary_size* are mostly used to verify secondary
J-Alves5eafd222023-10-26 14:19:21 +0100211 entry points fit into the loaded binary image.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100212* *entrypoint* defines the cold boot primary core entry point used by
J-Alves5eafd222023-10-26 14:19:21 +0100213 SPMD (currently matches ``BL32_BASE``) to enter the SPMC.
214
215Other nodes in the manifest are consumed by Hafnium in the secure world.
216A sample can be found at `[7]`_:
217
J-Alvesc28ee3e2024-05-14 18:29:26 +0100218* The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute
219 indicates a |FF-A| compliant SP. The *load_address* field specifies the load
J-Alves5eafd222023-10-26 14:19:21 +0100220 address at which BL2 loaded the SP package.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100221* The *cpus* node provides the platform topology and allows MPIDR to VMPIDR mapping.
J-Alves5eafd222023-10-26 14:19:21 +0100222 Note the primary core is declared first, then secondary cores are declared
223 in reverse order.
J-Alvesc28ee3e2024-05-14 18:29:26 +0100224* The *memory* nodes provide platform information on the ranges of memory
J-Alves5eafd222023-10-26 14:19:21 +0100225 available for use by SPs at runtime. These ranges relate to either
J-Alvesc28ee3e2024-05-14 18:29:26 +0100226 normal or device and secure or non-secure memory, depending on the *device_type*
227 field. The system integrator must exclude the memory used by other components
228 that are not SPs, such as the monitor, or the SPMC itself, the OS Kernel/Hypervisor,
229 NWd VMs, or peripherals that shall not be used by any of the SPs. The following are
230 the supported *device_type* fields:
231
232 * "memory": normal secure memory.
233 * "ns-memory": normal non-secure memory.
234 * "device-memory": device secure memory.
235 * "ns-device-memory": device non-secure memory.
236
237 The SPMC limits the SP's address space such that they can only refer to memory
238 inside of those ranges, either by defining memory region or device region nodes in
239 their manifest as well as memory starting at the load address until the limit
240 defined by the memory size. The SPMC also checks for overlaps between the regions.
241 Thus, the SPMC prevents rogue SPs from tampering with memory from other
J-Alves5eafd222023-10-26 14:19:21 +0100242 components.
243
J-Alvesc143a342023-11-07 12:17:44 +0000244.. code:: shell
245
246 memory@0 {
247 device_type = "memory";
248 reg = <0x0 0x6000000 0x2000000 0x0 0xff000000 0x1000000>;
249 };
250
251 memory@1 {
252 device_type = "ns-memory";
253 reg = <0x0 0x90010000 0x70000000>;
254 };
255
J-Alvesc28ee3e2024-05-14 18:29:26 +0100256 memory@2 {
257 device_type = "device-memory";
258 reg = <0x0 0x1c090000 0x0 0x40000>, /* UART */
259 <0x0 0x2bfe0000 0x0 0x20000>, /* SMMUv3TestEngine */
260 <0x0 0x2a490000 0x0 0x20000>, /* SP805 Trusted Watchdog */
261 <0x0 0x1c130000 0x0 0x10000>; /* Virtio block device */
262 };
263
264 memory@3 {
265 device_type = "ns-device-memory";
266 reg = <0x0 0x1C1F0000 0x0 0x10000>; /* LCD */
267 };
268
J-Alvesc143a342023-11-07 12:17:44 +0000269Above find an example representation of the referred memory description. The
270ranges are described in a list of unsigned 32-bit values, in which the first
271two addresses relate to the based physical address, followed by the respective
272page size. The first secure range defined in the node below has base address
273`0x0 0x6000000` and size `0x2000000`; following there is another range with
274base address `0x0 0xff000000` and size `0x1000000`.
275
J-Alves5eafd222023-10-26 14:19:21 +0100276Secure Partitions Configuration
277-------------------------------
278
279SP Manifests
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200280~~~~~~~~~~~~
281
282An SP manifest describes SP attributes as defined in `[1]`_
283(partition manifest at virtual FF-A instance) in DTS format. It is
284represented as a single file associated with the SP. A sample is
285provided by `[5]`_. A binding document is provided by `[6]`_.
286
J-Alves5eafd222023-10-26 14:19:21 +0100287Platform topology
288~~~~~~~~~~~~~~~~~
289
290The *execution-ctx-count* SP manifest field can take the value of one or the
291total number of PEs. The FF-A specification `[1]`_ recommends the
292following SP types:
293
294- Pinned MP SPs: an execution context matches a physical PE. MP SPs must
295 implement the same number of ECs as the number of PEs in the platform.
296- Migratable UP SPs: a single execution context can run and be migrated on any
297 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can
298 receive a direct message request originating from any physical core targeting
299 the single execution context.
300
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200301Secure Partition packages
302~~~~~~~~~~~~~~~~~~~~~~~~~
303
304Secure partitions are bundled as independent package files consisting
305of:
306
307- a header
308- a DTB
309- an image payload
310
311The header starts with a magic value and offset values to SP DTB and
312image payload. Each SP package is loaded independently by BL2 loader
313and verified for authenticity and integrity.
314
315The SP package identified by its UUID (matching FF-A uuid property) is
316inserted as a single entry into the FIP at end of the TF-A build flow
317as shown:
318
319.. code:: shell
320
321 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw"
322 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw"
323 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw"
324 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw"
325 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config"
326 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config"
327 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config"
328 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config"
329 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config"
330 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob"
331 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob"
332
333.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml
334
J-Alves5eafd222023-10-26 14:19:21 +0100335Secure Partitions Layout File
336~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200337
338A json-formatted description file is passed to the build flow specifying paths
339to the SP binary image and associated DTS partition manifest file. The latter
340is processed by the dtc compiler to generate a DTB fed into the SP package.
Karl Meakin82593ce2023-08-30 16:38:28 +0100341Each partition can be configured with the following fields:
342
343:code:`image`
344 - Specifies the filename and offset of the image within the SP package.
345 - Can be written as :code:`"image": { "file": "path", "offset": 0x1234 }` to
346 give both :code:`image.file` and :code:`image.offset` values explicitly, or
347 can be written as :code:`"image": "path"` to give :code:`image.file` and value
348 and leave :code:`image.offset` absent.
349
350 :code:`image.file`
351 - Specifies the filename of the image.
352
353 :code:`image.offset`
354 - Specifies the offset of the image within the SP package.
355 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC.
356 - Optional. Defaults to :code:`0x4000`.
357
358:code:`pm`
359 - Specifies the filename and offset of the partition manifest within the SP package.
360 - Can be written as :code:`"pm": { "file": "path", "offset": 0x1234 }` to
361 give both :code:`pm.file` and :code:`pm.offset` values explicitly, or
362 can be written as :code:`"pm": "path"` to give :code:`pm.file` and value
363 and leave :code:`pm.offset` absent.
364
365 :code:`pm.file`
366 - Specifies the filename of the partition manifest.
367
368 :code:`pm.offset`
369 - Specifies the offset of the partition manifest within the SP package.
370 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC.
371 - Optional. Defaults to :code:`0x1000`.
372
373:code:`image.offset` and :code:`pm.offset` can be leveraged to support SPs with
374S1 translation granules that differ from 4KB, and to configure the regions
375allocated within the SP package, as well as to comply with the requirements for
376the implementation of the boot information protocol (see `Passing boot data to
377the SP`_ for more details).
378
379:code:`owner`
380 - Specifies the SP owner, identifying the signing domain in case of dual root CoT.
381 - Possible values are :code:`SiP` (silicon owner) or :code:`Plat` (platform owner).
382 - Optional. Defaults to :code:`SiP`.
383
384:code:`uuid`
385 - Specifies the UUID of the partition.
386 - Optional. Defaults to the value of the :code:`uuid` field from the DTS partition manifest.
387
388:code:`physical-load-address`
389 - Specifies the :code:`load_address` field of the generated DTS fragment.
390 - Optional. Defaults to the value of the :code:`load-address` from the DTS partition manifest.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200391
392.. code:: shell
393
394 {
395 "tee1" : {
396 "image": "tee1.bin",
397 "pm": "tee1.dts",
398 "owner": "SiP",
399 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f"
400 },
401
402 "tee2" : {
403 "image": "tee2.bin",
404 "pm": "tee2.dts",
405 "owner": "Plat"
406 },
407
408 "tee3" : {
409 "image": {
410 "file": "tee3.bin",
411 "offset":"0x2000"
412 },
413 "pm": {
414 "file": "tee3.dts",
415 "offset":"0x6000"
416 },
417 "owner": "Plat"
418 },
419 }
420
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200421SPMC boot
J-Alves5eafd222023-10-26 14:19:21 +0100422=========
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200423
424The SPMC is loaded by BL2 as the BL32 image.
425
426The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_.
427
428BL2 passes the SPMC manifest address to BL31 through a register.
429
430At boot time, the SPMD in BL31 runs from the primary core, initializes the core
431contexts and launches the SPMC (BL32) passing the following information through
432registers:
433
434- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob).
435- X1 holds the ``HW_CONFIG`` physical address.
436- X4 holds the currently running core linear id.
437
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200438Secure boot
J-Alves5eafd222023-10-26 14:19:21 +0100439-----------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200440
441The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC,
442SPMC manifest, secure partitions and verifies them for authenticity and integrity.
443Refer to TBBR specification `[3]`_.
444
445The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows
446the use of two root keys namely S-ROTPK and NS-ROTPK:
447
448- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK.
449- BL33 may be signed by the OEM using NS-ROTPK.
450- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK).
451- A maximum of 4 partitions can be signed with the S-ROTPK key and 4 partitions
452 signed with the NS-ROTPK key.
453
J-Alves5eafd222023-10-26 14:19:21 +0100454Also refer to `Secure Partitions Configuration`_ and `TF-A build options`_ sections.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200455
456Boot phases
457-----------
458
459Primary core boot-up
460~~~~~~~~~~~~~~~~~~~~
461
462Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical
463core. The SPMC performs its platform initializations and registers the SPMC
464secondary physical core entry point physical address by the use of the
465`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD
466at secure physical FF-A instance).
467
J-Alvesc143a342023-11-07 12:17:44 +0000468The SPMC then creates secure partitions base on SP packages and manifests. Each
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200469secure partition is launched in sequence (`SP Boot order`_) on their "primary"
470execution context. If the primary boot physical core linear id is N, an MP SP is
471started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a
472UP SP, it is started using its unique EC0 on PE[N].
473
474The SP primary EC (or the EC used when the partition is booted as described
475above):
476
477- Performs the overall SP boot time initialization, and in case of a MP SP,
478 prepares the SP environment for other execution contexts.
479- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure
480 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA
481 entry point for other execution contexts.
482- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or
483 ``FFA_ERROR`` in case of failure.
484
485Secondary cores boot-up
486~~~~~~~~~~~~~~~~~~~~~~~
487
488Once the system is started and NWd brought up, a secondary physical core is
489woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism
490calls into the SPMD on the newly woken up physical core. Then the SPMC is
491entered at the secondary physical core entry point.
492
493In the current implementation, the first SP is resumed on the coresponding EC
494(the virtual CPU which matches the physical core). The implication is that the
495first SP must be a MP SP.
496
497In a linux based system, once secure and normal worlds are booted but prior to
498a NWd FF-A driver has been loaded:
499
500- The first SP has initialized all its ECs in response to primary core boot up
501 (at system initialization) and secondary core boot up (as a result of linux
502 invoking PSCI_CPU_ON for all secondary cores).
503- Other SPs have their first execution context initialized as a result of secure
504 world initialization on the primary boot core. Other ECs for those SPs have to
505 be run first through ffa_run to complete their initialization (which results
506 in the EC completing with FFA_MSG_WAIT).
507
508Refer to `Power management`_ for further details.
509
J-Alves5eafd222023-10-26 14:19:21 +0100510Loading of SPs
511--------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200512
J-Alves5eafd222023-10-26 14:19:21 +0100513At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted
514below:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200515
J-Alves5eafd222023-10-26 14:19:21 +0100516.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200517
J-Alves5eafd222023-10-26 14:19:21 +0100518Note this boot flow is an implementation sample on Arm's FVP platform.
519Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a
520different boot flow. The flow restricts to a maximum of 8 secure partitions.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200521
J-Alves5eafd222023-10-26 14:19:21 +0100522SP Boot order
523~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200524
J-Alves5eafd222023-10-26 14:19:21 +0100525SP manifests provide an optional boot order attribute meant to resolve
526dependencies such as an SP providing a service required to properly boot
527another SP. SPMC boots the SPs in accordance to the boot order attribute,
528lowest to the highest value. If the boot order attribute is absent from the FF-A
529manifest, the SP is treated as if it had the highest boot order value
530(i.e. lowest booting priority). The FF-A specification mandates this field
531is unique to each SP.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200532
J-Alves5eafd222023-10-26 14:19:21 +0100533It is possible for an SP to call into another SP through a direct request
534provided the latter SP has already been booted.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200535
J-Alves5eafd222023-10-26 14:19:21 +0100536Passing boot data to the SP
537~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200538
J-Alves5eafd222023-10-26 14:19:21 +0100539In `[1]`_ , the section "Boot information protocol" defines a method for passing
540data to the SPs at boot time. It specifies the format for the boot information
541descriptor and boot information header structures, which describe the data to be
542exchanged between SPMC and SP.
543The specification also defines the types of data that can be passed.
544The aggregate of both the boot info structures and the data itself is designated
545the boot information blob, and is passed to a Partition as a contiguous memory
546region.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200547
J-Alves5eafd222023-10-26 14:19:21 +0100548Currently, the SPM implementation supports the FDT type which is used to pass the
549partition's DTB manifest.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200550
J-Alves5eafd222023-10-26 14:19:21 +0100551The region for the boot information blob is allocated through the SP package.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200552
J-Alves5eafd222023-10-26 14:19:21 +0100553.. image:: ../resources/diagrams/partition-package.png
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200554
J-Alves5eafd222023-10-26 14:19:21 +0100555To adjust the space allocated for the boot information blob, the json description
556of the SP (see section `Secure Partitions Layout File`_) shall be updated to contain
557the manifest offset. If no offset is provided the manifest offset defaults to 0x1000,
558which is the page size in the Hafnium SPMC.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200559
J-Alves5eafd222023-10-26 14:19:21 +0100560The configuration of the boot protocol is done in the SPs manifest. As defined by
561the specification, the manifest field 'gp-register-num' configures the GP register
562which shall be used to pass the address to the partitions boot information blob when
563booting the partition.
564In addition, the Hafnium SPMC implementation requires the boot information arguments
565to be listed in a designated DT node:
566
567.. code:: shell
568
569 boot-info {
570 compatible = "arm,ffa-manifest-boot-info";
571 ffa_manifest;
572 };
573
574The whole secure partition package image (see `Secure Partition packages`_) is
575mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can
576retrieve the address for the boot information blob in the designated GP register,
577process the boot information header and descriptors, access its own manifest
578DTB blob and extract its partition manifest properties.
579
580SPMC Runtime
581============
582
583Parsing SP partition manifests
584------------------------------
585
586Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_.
587Note the current implementation may not implement all optional fields.
588
589The SP manifest may contain memory and device regions nodes:
590
591- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at
592 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can
593 specify RX/TX buffer regions in which case it is not necessary for an SP
594 to explicitly invoke the ``FFA_RXTX_MAP`` interface. The memory referred
595 shall be contained within the memory ranges defined in SPMC manifest. The
596 NS bit in the attributes field should be consistent with the security
597 state of the range that it relates to. I.e. non-secure memory shall be
598 part of a non-secure memory range, and secure memory shall be contained
599 in a secure memory range of a given platform.
600- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or
601 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate
602 additional resources (e.g. interrupts).
603
604For the SPMC, base addresses for memory and device region nodes are IPAs provided
605the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation regime.
606
Olivier Deprezb8bd7d72023-10-27 16:14:13 +0200607ote: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the
J-Alves5eafd222023-10-26 14:19:21 +0100608same set of page tables. It is still open whether two sets of page tables shall
609be provided per SP. The memory region node as defined in the specification
610provides a memory security attribute hinting to map either to the secure or
611non-secure EL1&0 Stage-2 table if it exists.
612
613Secure partitions scheduling
614----------------------------
615
Olivier Deprez8c4cb2d2023-10-27 16:07:11 +0200616The FF-A specification `[1]`_ provides two ways to allocate CPU cycles to
J-Alves5eafd222023-10-26 14:19:21 +0100617secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of:
618
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500619- the FFA_MSG_SEND_DIRECT_REQ (or FFA_MSG_SEND_DIRECT_REQ2) interface.
J-Alves5eafd222023-10-26 14:19:21 +0100620- the FFA_RUN interface.
621
622Additionally a secure interrupt can pre-empt the normal world execution and give
623CPU cycles by transitioning to EL3 and S-EL2.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200624
625Mandatory interfaces
626--------------------
627
628The following interfaces are exposed to SPs:
629
630- ``FFA_VERSION``
631- ``FFA_FEATURES``
632- ``FFA_RX_RELEASE``
633- ``FFA_RXTX_MAP``
634- ``FFA_RXTX_UNMAP``
635- ``FFA_PARTITION_INFO_GET``
636- ``FFA_ID_GET``
637- ``FFA_MSG_WAIT``
638- ``FFA_MSG_SEND_DIRECT_REQ``
639- ``FFA_MSG_SEND_DIRECT_RESP``
640- ``FFA_MEM_DONATE``
641- ``FFA_MEM_LEND``
642- ``FFA_MEM_SHARE``
643- ``FFA_MEM_RETRIEVE_REQ``
644- ``FFA_MEM_RETRIEVE_RESP``
645- ``FFA_MEM_RELINQUISH``
646- ``FFA_MEM_FRAG_RX``
647- ``FFA_MEM_FRAG_TX``
648- ``FFA_MEM_RECLAIM``
649- ``FFA_RUN``
650
651As part of the FF-A v1.1 support, the following interfaces were added:
652
653 - ``FFA_NOTIFICATION_BITMAP_CREATE``
654 - ``FFA_NOTIFICATION_BITMAP_DESTROY``
655 - ``FFA_NOTIFICATION_BIND``
656 - ``FFA_NOTIFICATION_UNBIND``
657 - ``FFA_NOTIFICATION_SET``
658 - ``FFA_NOTIFICATION_GET``
659 - ``FFA_NOTIFICATION_INFO_GET``
660 - ``FFA_SPM_ID_GET``
661 - ``FFA_SECONDARY_EP_REGISTER``
662 - ``FFA_MEM_PERM_GET``
663 - ``FFA_MEM_PERM_SET``
664 - ``FFA_MSG_SEND2``
665 - ``FFA_RX_ACQUIRE``
666
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700667As part of the FF-A v1.2 support, the following interfaces were added:
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500668
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700669- ``FFA_PARTITION_INFO_GET_REGS``
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500670- ``FFA_MSG_SEND_DIRECT_REQ2``
671- ``FFA_MSG_SEND_DIRECT_RESP2``
Karl Meakind40979f2024-05-13 10:21:56 +0100672- ``FFA_CONSOLE_LOG``
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700673
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200674FFA_VERSION
675~~~~~~~~~~~
676
677``FFA_VERSION`` requires a *requested_version* parameter from the caller.
678The returned value depends on the caller:
679
680- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version
681 specified in the SPMC manifest.
682- SP: the SPMC returns its own implemented version.
683- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version.
684
685FFA_FEATURES
686~~~~~~~~~~~~
687
688FF-A features supported by the SPMC may be discovered by secure partitions at
689boot (that is prior to NWd is booted) or run-time.
690
691The SPMC calling FFA_FEATURES at secure physical FF-A instance always get
692FFA_SUCCESS from the SPMD.
693
Karl Meakin963a5d72024-05-13 10:32:29 +0100694S-EL1 partitions calling FFA_FEATURES at virtual FF-A instance with NPI and MEI
695interrupt feature IDs get FFA_SUCCESS.
696
697S-EL0 partitions are not supported for NPI: ``FFA_NOT_SUPPORTED`` will be
698returned.
699
700Physical FF-A instances are not supported for NPI and MEI: ``FFA_NOT_SUPPORTED``
701will be returned.
702
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200703The request made by an Hypervisor or OS kernel is forwarded to the SPMC and
704the response relayed back to the NWd.
705
706FFA_RXTX_MAP/FFA_RXTX_UNMAP
707~~~~~~~~~~~~~~~~~~~~~~~~~~~
708
709When invoked from a secure partition FFA_RXTX_MAP maps the provided send and
710receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation
711regime as secure buffers in the MMU descriptors.
712
713When invoked from the Hypervisor or OS kernel, the buffers are mapped into the
714SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU
715descriptors. The provided addresses may be owned by a VM in the normal world,
716which is expected to receive messages from the secure world. The SPMC will in
717this case allocate internal state structures to facilitate RX buffer access
718synchronization (through FFA_RX_ACQUIRE interface), and to permit SPs to send
Karl Meakinb1dbca92024-01-24 16:51:22 +0000719messages. The addresses used must be contained in the SPMC manifest NS memory
720node (see `SPMC manifest`_).
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200721
722The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the
723caller, either it being the Hypervisor or OS kernel, as well as a secure
Karl Meakinb1dbca92024-01-24 16:51:22 +0000724partition, and restores them in the VM's translation regime so that they can be
725used for memory sharing operations from the normal world again.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200726
Karl Meakin963a5d72024-05-13 10:32:29 +0100727The minimum and maximum buffer sizes supported by the FF-A instance can be
728queried by calling ``FFA_FEATURES`` with the ``FFA_RXTX_MAP`` function ID.
729
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200730FFA_PARTITION_INFO_GET
731~~~~~~~~~~~~~~~~~~~~~~
732
733Partition info get call can originate:
734
735- from SP to SPMC
736- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD.
737
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700738FFA_PARTITION_INFO_GET_REGS
739~~~~~~~~~~~~~~~~~~~~~~~~~~~
740
741This call can originate:
742
743- from SP to SPMC
744- from SPMC to SPMD
745- from Hypervsior or OS kernel to SPMC. The request is relayed by the SPMD.
746
747The primary use of this ABI is to return partition information via registers
748as opposed to via RX/TX buffers and is useful in cases where sharing memory is
749difficult.
750
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200751FFA_ID_GET
752~~~~~~~~~~
753
754The FF-A id space is split into a non-secure space and secure space:
755
756- FF-A ID with bit 15 clear relates to VMs.
757- FF-A ID with bit 15 set related to SPs.
758- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD
759 and SPMC.
760
761The SPMD returns:
762
763- The default zero value on invocation from the Hypervisor.
764- The ``spmc_id`` value specified in the SPMC manifest on invocation from
765 the SPMC (see `SPMC manifest`_)
766
767This convention helps the SPMC to determine the origin and destination worlds in
768an FF-A ABI invocation. In particular the SPMC shall filter unauthorized
769transactions in its world switch routine. It must not be permitted for a VM to
770use a secure FF-A ID as origin world by spoofing:
771
772- A VM-to-SP direct request/response shall set the origin world to be non-secure
773 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15
774 set).
775- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15
776 for both origin and destination IDs.
777
778An incoming direct message request arriving at SPMD from NWd is forwarded to
779SPMC without a specific check. The SPMC is resumed through eret and "knows" the
780message is coming from normal world in this specific code path. Thus the origin
781endpoint ID must be checked by SPMC for being a normal world ID.
782
783An SP sending a direct message request must have bit 15 set in its origin
784endpoint ID and this can be checked by the SPMC when the SP invokes the ABI.
785
786The SPMC shall reject the direct message if the claimed world in origin endpoint
787ID is not consistent:
788
789- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal
790 world ID",
791- or initiated by an SP and thus origin endpoint ID must be a "secure world ID".
792
793
794FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP
795~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
796
797This is a mandatory interface for secure partitions consisting in direct request
798and responses with the following rules:
799
800- An SP can send a direct request to another SP.
801- An SP can receive a direct request from another SP.
802- An SP can send a direct response to another SP.
803- An SP cannot send a direct request to an Hypervisor or OS kernel.
804- An Hypervisor or OS kernel can send a direct request to an SP.
805- An SP can send a direct response to an Hypervisor or OS kernel.
806
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500807FFA_MSG_SEND_DIRECT_REQ2/FFA_MSG_SEND_DIRECT_RESP2
808~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
809
810The primary usage of these ABIs is to send a direct request to a specified
811UUID within an SP that has multiple UUIDs declared in its manifest.
812
813Secondarily, it can be used to send a direct request with an extended
814set of message payload arguments.
815
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200816FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY
817~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
818
819The secure partitions notifications bitmap are statically allocated by the SPMC.
820Hence, this interface is not to be issued by secure partitions.
821
822At initialization, the SPMC is not aware of VMs/partitions deployed in the
823normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC
824to be prepared to handle notifications for the provided VM ID.
825
826FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND
827~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
828
829Pair of interfaces to manage permissions to signal notifications. Prior to
830handling notifications, an FF-A endpoint must allow a given sender to signal a
831bitmap of notifications.
832
833If the receiver doesn't have notification support enabled in its FF-A manifest,
834it won't be able to bind notifications, hence forbidding it to receive any
835notifications.
836
837FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET
838~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
839
840FFA_NOTIFICATION_GET retrieves all pending global notifications and
841per-vCPU notifications targeted to the current vCPU.
842
843Hafnium maintains a global count of pending notifications which gets incremented
844and decremented when handling FFA_NOTIFICATION_SET and FFA_NOTIFICATION_GET
845respectively. A delayed SRI is triggered if the counter is non-zero when the
846SPMC returns to normal world.
847
848FFA_NOTIFICATION_INFO_GET
849~~~~~~~~~~~~~~~~~~~~~~~~~
850
851Hafnium maintains a global count of pending notifications whose information
852has been retrieved by this interface. The count is incremented and decremented
853when handling FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET respectively.
854It also tracks notifications whose information has been retrieved individually,
855such that it avoids duplicating returned information for subsequent calls to
856FFA_NOTIFICATION_INFO_GET. For each notification, this state information is
857reset when receiver called FFA_NOTIFICATION_GET to retrieve them.
858
859FFA_SPM_ID_GET
860~~~~~~~~~~~~~~
861
862Returns the FF-A ID allocated to an SPM component which can be one of SPMD
863or SPMC.
864
865At initialization, the SPMC queries the SPMD for the SPMC ID, using the
866FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using
867the FFA_SPM_ID_GET interface at the secure physical FF-A instance.
868
869Secure partitions call this interface at the virtual FF-A instance, to which
870the SPMC returns the priorly retrieved SPMC ID.
871
872The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the
873SPMD, which returns the SPMC ID.
874
875FFA_SECONDARY_EP_REGISTER
876~~~~~~~~~~~~~~~~~~~~~~~~~
877
878When the SPMC boots, all secure partitions are initialized on their primary
879Execution Context.
880
881The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition
882from its first execution context, to provide the entry point address for
883secondary execution contexts.
884
885A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from
886the NWd or by invocation of FFA_RUN.
887
888FFA_RX_ACQUIRE/FFA_RX_RELEASE
889~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
890
891The RX buffers can be used to pass information to an FF-A endpoint in the
892following scenarios:
893
894 - When it was targetted by a FFA_MSG_SEND2 invokation from another endpoint.
895 - Return the result of calling ``FFA_PARTITION_INFO_GET``.
896 - In a memory share operation, as part of the ``FFA_MEM_RETRIEVE_RESP``,
897 with the memory descriptor of the shared memory.
898
899If a normal world VM is expected to exchange messages with secure world,
900its RX/TX buffer addresses are forwarded to the SPMC via FFA_RXTX_MAP ABI,
901and are from this moment owned by the SPMC.
902The hypervisor must call the FFA_RX_ACQUIRE interface before attempting
903to use the RX buffer, in any of the aforementioned scenarios. A successful
904call to FFA_RX_ACQUIRE transfers ownership of RX buffer to hypervisor, such
905that it can be safely used.
906
907The FFA_RX_RELEASE interface is used after the FF-A endpoint is done with
908processing the data received in its RX buffer. If the RX buffer has been
909acquired by the hypervisor, the FFA_RX_RELEASE call must be forwarded to
910the SPMC to reestablish SPMC's RX ownership.
911
912An attempt from an SP to send a message to a normal world VM whose RX buffer
913was acquired by the hypervisor fails with error code FFA_BUSY, to preserve
914the RX buffer integrity.
915The operation could then be conducted after FFA_RX_RELEASE.
916
917FFA_MSG_SEND2
918~~~~~~~~~~~~~
919
920Hafnium copies a message from the sender TX buffer into receiver's RX buffer.
921For messages from SPs to VMs, operation is only possible if the SPMC owns
922the receiver's RX buffer.
923
924Both receiver and sender need to enable support for indirect messaging,
925in their respective partition manifest. The discovery of support
926of such feature can be done via FFA_PARTITION_INFO_GET.
927
928On a successful message send, Hafnium pends an RX buffer full framework
929notification for the receiver, to inform it about a message in the RX buffer.
930
931The handling of framework notifications is similar to that of
932global notifications. Binding of these is not necessary, as these are
933reserved to be used by the hypervisor or SPMC.
934
Karl Meakind40979f2024-05-13 10:21:56 +0100935FFA_CONSOLE_LOG
936~~~~~~~~~~~~~~~
937
938``FFA_CONSOLE_LOG`` allows debug logging to the UART console.
939Characters are packed into registers:
940 - `w2-w7` (|SMCCC| 32-bit)
941 - `x2-x7` (|SMCCC| 64-bit, before v1.2)
942 - `x2-x17` (|SMCCC| 64-bit, v1.2 or later)
943
Madhukar Pappireddy0b2304b2023-08-15 18:05:21 -0500944Paravirtualized interfaces
945--------------------------
946
947Hafnium SPMC implements the following implementation-defined interface(s):
948
949HF_INTERRUPT_ENABLE
950~~~~~~~~~~~~~~~~~~~
951
952Enables or disables the given virtual interrupt for the calling execution
953context. Returns 0 on success, or -1 if the interrupt id is invalid.
954
955HF_INTERRUPT_GET
956~~~~~~~~~~~~~~~~
957
958Returns the ID of the next pending virtual interrupt for the calling execution
959context, and acknowledges it (i.e. marks it as no longer pending). Returns
960HF_INVALID_INTID if there are no pending interrupts.
961
962HF_INTERRUPT_DEACTIVATE
963~~~~~~~~~~~~~~~~~~~~~~~
964
965Drops the current interrupt priority and deactivates the given virtual and
966physical interrupt ID for the calling execution context. Returns 0 on success,
967or -1 otherwise.
968
969HF_INTERRUPT_RECONFIGURE
970~~~~~~~~~~~~~~~~~~~~~~~~
971
972An SP specifies the list of interrupts it owns through its partition manifest.
973This paravirtualized interface allows an SP to reconfigure a physical interrupt
974in runtime. It accepts three arguments, namely, interrupt ID, command and value.
975The command & value pair signify what change is being requested by the current
976Secure Partition for the given interrupt.
977
978SPMC returns 0 to indicate that the command was processed successfully or -1 if
979it failed to do so. At present, this interface only supports the following
980commands:
981
982 - ``INT_RECONFIGURE_TARGET_PE``
983 - Change the target CPU of the interrupt.
984 - Value represents linear CPU index in the range 0 to (MAX_CPUS - 1).
985
986 - ``INT_RECONFIGURE_SEC_STATE``
987 - Change the security state of the interrupt.
988 - Value must be either 0 (Non-secure) or 1 (Secure).
989
990 - ``INT_RECONFIGURE_ENABLE``
991 - Enable or disable the physical interrupt.
992 - Value must be either 0 (Disable) or 1 (Enable).
993
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200994SPMC-SPMD direct requests/responses
995-----------------------------------
996
997Implementation-defined FF-A IDs are allocated to the SPMC and SPMD.
998Using those IDs in source/destination fields of a direct request/response
999permits SPMD to SPMC communication and either way.
1000
1001- SPMC to SPMD direct request/response uses SMC conduit.
1002- SPMD to SPMC direct request/response uses ERET conduit.
1003
1004This is used in particular to convey power management messages.
1005
J-Alves5eafd222023-10-26 14:19:21 +01001006Notifications
1007-------------
1008
1009The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous
1010communication mechanism with non-blocking semantics. It allows for one FF-A
1011endpoint to signal another for service provision, without hindering its current
1012progress.
1013
1014Hafnium currently supports 64 notifications. The IDs of each notification define
1015a position in a 64-bit bitmap.
1016
1017The signaling of notifications can interchangeably happen between NWd and SWd
1018FF-A endpoints.
1019
1020The SPMC is in charge of managing notifications from SPs to SPs, from SPs to
1021VMs, and from VMs to SPs. An hypervisor component would only manage
1022notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints
1023deployed in NWd, the Hypervisor or OS kernel must invoke the interface
1024FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A
1025endpoint in the NWd that supports it.
1026
1027A sender can signal notifications once the receiver has provided it with
1028permissions. Permissions are provided by invoking the interface
1029FFA_NOTIFICATION_BIND.
1030
1031Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth
1032they are considered to be in a pending sate. The receiver can retrieve its
1033pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment,
1034are considered to be handled.
1035
1036Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler
1037that is in charge of donating CPU cycles for notifications handling. The
1038FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about
1039which FF-A endpoints have pending notifications. The receiver scheduler is
1040called and informed by the FF-A driver, and it should allocate CPU cycles to the
1041receiver.
1042
1043There are two types of notifications supported:
1044
Olivier Deprezb8bd7d72023-10-27 16:14:13 +02001045- Global, which are targeted to an FF-A endpoint and can be handled within any
1046 of its execution contexts, as determined by the scheduler of the system.
J-Alves5eafd222023-10-26 14:19:21 +01001047- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a
1048 a specific execution context, as determined by the sender.
1049
1050The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give
1051permissions to the sender.
1052
1053Notification signaling resorts to two interrupts:
1054
1055- Schedule Receiver Interrupt: non-secure physical interrupt to be handled by
1056 the FF-A driver within the receiver scheduler. At initialization the SPMC
1057 donates an SGI ID chosen from the secure SGI IDs range and configures it as
1058 non-secure. The SPMC triggers this SGI on the currently running core when
1059 there are pending notifications, and the respective receivers need CPU cycles
1060 to handle them.
1061- Notifications Pending Interrupt: virtual interrupt to be handled by the
1062 receiver of the notification. Set when there are pending notifications for the
1063 given secure partition. The NPI is pended when the NWd relinquishes CPU cycles
1064 to an SP.
1065
1066The notifications receipt support is enabled in the partition FF-A manifest.
1067
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001068Memory Sharing
1069--------------
1070
J-Alvesd547d6d2024-05-14 14:59:54 +01001071The Hafnium implementation aligns with FF-A v1.2 ALP0 specification,
1072'FF-A Memory Management Protocol' supplement `[11]`_. Hafnium supports
1073the following ABIs:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001074
1075 - ``FFA_MEM_SHARE`` - for shared access between lender and borrower.
1076 - ``FFA_MEM_LEND`` - borrower to obtain exclusive access, though lender
1077 retains ownership of the memory.
1078 - ``FFA_MEM_DONATE`` - lender permanently relinquishes ownership of memory
1079 to the borrower.
1080
1081The ``FFA_MEM_RETRIEVE_REQ`` interface is for the borrower to request the
1082memory to be mapped into its address space: for S-EL1 partitions the SPM updates
1083their stage 2 translation regime; for S-EL0 partitions the SPM updates their
1084stage 1 translation regime. On a successful call, the SPMC responds back with
1085``FFA_MEM_RETRIEVE_RESP``.
1086
1087The ``FFA_MEM_RELINQUISH`` interface is for when the borrower is done with using
1088a memory region.
1089
1090The ``FFA_MEM_RECLAIM`` interface is for the owner of the memory to reestablish
1091its ownership and exclusive access to the memory shared.
1092
1093The memory transaction descriptors are transmitted via RX/TX buffers. In
1094situations where the size of the memory transaction descriptor exceeds the
1095size of the RX/TX buffers, Hafnium provides support for fragmented transmission
1096of the full transaction descriptor. The ``FFA_MEM_FRAG_RX`` and ``FFA_MEM_FRAG_TX``
1097interfaces are for receiving and transmitting the next fragment, respectively.
1098
1099If lender and borrower(s) are SPs, all memory sharing operations are supported.
1100
1101Hafnium also supports memory sharing operations between the normal world and the
1102secure world. If there is an SP involved, the SPMC allocates data to track the
1103state of the operation.
1104
J-Alvesda82a1a2023-10-17 11:45:49 +01001105An SP can not share, lend or donate memory to the NWd.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001106
J-Alvesd547d6d2024-05-14 14:59:54 +01001107The SPMC is also the designated allocator for the memory handle, when borrowers
1108include at least an SP. The SPMC doesn't support the hypervisor to be allocator
1109to the memory handle.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001110
1111Hafnium also supports memory lend and share targetting multiple borrowers.
1112This is the case for a lender SP to multiple SPs, and for a lender VM to
1113multiple endpoints (from both secure world and normal world). If there is
1114at least one borrower VM, the hypervisor is in charge of managing its
J-Alvesd547d6d2024-05-14 14:59:54 +01001115stage 2 translation on a successful memory retrieve. However, the hypervisor could
1116rely on the SPMC to keep track of the state of the operation, namely:
1117if all fragments to the memory descriptors have been sent, and if the retrievers
1118are still using the memory at any given moment. In this case, the hypervisor might
1119need to request the SPMC to obtain a description of the used memory regions.
1120For example, when handling an ``FFA_MEM_RECLAIM`` the hypervisor retrieve request
1121can be used to obtain that state information, do the necessary validations,
1122and update stage-2 memory translation of the lender.
1123Hafnium currently only supports one borrower from the NWd, in a multiple borrower
1124scenario as described. If there is only a single borrower VM, the SPMC will
1125return error to the lender on call to either share, lend or donate ABIs.
1126
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001127The semantics of ``FFA_MEM_DONATE`` implies ownership transmission,
1128which should target only one partition.
1129
1130The memory share interfaces are backwards compatible with memory transaction
Daniel Boulbyd5041122024-01-31 14:24:54 +00001131descriptors from FF-A v1.0. Starting from FF-A v1.1, with the introduction
1132of the `Endpoint memory access descriptor size` and
1133`Endpoint memory access descriptor access offset` fields (from Table 11.20 of the
1134FF-A v1.2 ALP0 specification), memory transaction descriptors are forward
1135compatible, so can be used internally by Hafnium as they are sent.
1136These fields must be valid for a memory access descriptor defined for a compatible
1137FF-A version to the SPMC FF-A version. For a transaction from an FF-A v1.0 endpoint
1138the memory transaction descriptor will be translated to an FF-A v1.1 descriptor for
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001139Hafnium's internal processing of the operation. If the FF-A version of a
1140borrower is v1.0, Hafnium provides FF-A v1.0 compliant memory transaction
1141descriptors on memory retrieve response.
1142
J-Alvesffc82062023-11-07 14:19:00 +00001143In the section :ref:`SPMC Configuration` there is a mention of non-secure memory
1144range, that limit the memory region nodes the SP can define. Whatever is left of
1145the memory region node carve-outs, the SPMC utilizes the memory to create a set of
1146page tables it associates with the NWd. The memory sharing operations incoming from
1147the NWd should refer to addresses belonging to these page tables. The intent
1148is for SPs not to be able to get access to regions they are not intended to access.
1149This requires special care from the system integrator to configure the memory ranges
1150correctly, such that any SP can't be given access and interfere with execution of
1151other components. More information in the :ref:`Threat Model`.
1152
Daniel Boulbydfc312e2024-05-14 17:10:01 +01001153Hafnium SPMC supports memory management transactions for device memory regions.
1154Currently this is limited to only the ``FFA_MEM_LEND`` interface and
1155to a single borrower. The device memory region used in the transaction must have
1156been decalared in the SPMC manifest as described above. Memory defined in a device
1157region node is given the attributes Device-nGnRnE, since this is the most restrictive
1158memory type the memory must be lent with these attrbutes as well.
1159
J-Alvesd547d6d2024-05-14 14:59:54 +01001160In |RME| enabled platforms, there is the ability to change the |PAS|
1161of a given memory region `[12]`_. The SPMC can leverage this feature to fulfill the
1162semantics of the ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE`` from the NWd into the SWd.
1163Currently, there is the implementation for the FVP platform to issue a
1164platform-specific SMC call to the EL3 monitor to change the PAS of the regions being
1165lent/donated. This shall guarantee the NWd can't tamper with the memory whilst
1166the SWd software expects exclusive access. For any other platform, the API under
1167the 'src/memory_protect' module can be redefined to leverage an equivalent platform
1168specific mechanism. For reference, check the `SPMC FVP build configuration`_.
1169
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001170PE MMU configuration
1171--------------------
1172
1173With secure virtualization enabled (``HCR_EL2.VM = 1``) and for S-EL1
1174partitions, two IPA spaces (secure and non-secure) are output from the
1175secure EL1&0 Stage-1 translation.
1176The EL1&0 Stage-2 translation hardware is fed by:
1177
1178- A secure IPA when the SP EL1&0 Stage-1 MMU is disabled.
1179- One of secure or non-secure IPA when the secure EL1&0 Stage-1 MMU is enabled.
1180
1181``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the
1182NS/S IPA translations. The following controls are set up:
1183``VSTCR_EL2.SW = 0`` , ``VSTCR_EL2.SA = 0``, ``VTCR_EL2.NSW = 0``,
1184``VTCR_EL2.NSA = 1``:
1185
1186- Stage-2 translations for the NS IPA space access the NS PA space.
1187- Stage-2 translation table walks for the NS IPA space are to the secure PA space.
1188
1189Secure and non-secure IPA regions (rooted to by ``VTTBR_EL2`` and ``VSTTBR_EL2``)
1190use the same set of Stage-2 page tables within a SP.
1191
1192The ``VTCR_EL2/VSTCR_EL2/VTTBR_EL2/VSTTBR_EL2`` virtual address space
1193configuration is made part of a vCPU context.
1194
1195For S-EL0 partitions with VHE enabled, a single secure EL2&0 Stage-1 translation
1196regime is used for both Hafnium and the partition.
1197
1198Schedule modes and SP Call chains
1199---------------------------------
1200
1201An SP execution context is said to be in SPMC scheduled mode if CPU cycles are
1202allocated to it by SPMC. Correspondingly, an SP execution context is said to be
1203in Normal world scheduled mode if CPU cycles are allocated by the normal world.
1204
1205A call chain represents all SPs in a sequence of invocations of a direct message
1206request. When execution on a PE is in the secure state, only a single call chain
1207that runs in the Normal World scheduled mode can exist. FF-A v1.1 spec allows
1208any number of call chains to run in the SPMC scheduled mode but the Hafnium
1209SPMC restricts the number of call chains in SPMC scheduled mode to only one for
1210keeping the implementation simple.
1211
1212Partition runtime models
1213------------------------
1214
1215The runtime model of an endpoint describes the transitions permitted for an
1216execution context between various states. These are the four partition runtime
1217models supported (refer to `[1]`_ section 7):
1218
1219 - RTM_FFA_RUN: runtime model presented to an execution context that is
1220 allocated CPU cycles through FFA_RUN interface.
1221 - RTM_FFA_DIR_REQ: runtime model presented to an execution context that is
Kathleen Capella6e3abcf2024-02-05 16:17:35 -05001222 allocated CPU cycles through FFA_MSG_SEND_DIRECT_REQ or FFA_MSG_SEND_DIRECT_REQ2
1223 interface.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001224 - RTM_SEC_INTERRUPT: runtime model presented to an execution context that is
1225 allocated CPU cycles by SPMC to handle a secure interrupt.
1226 - RTM_SP_INIT: runtime model presented to an execution context that is
1227 allocated CPU cycles by SPMC to initialize its state.
1228
1229If an endpoint execution context attempts to make an invalid transition or a
1230valid transition that could lead to a loop in the call chain, SPMC denies the
1231transition with the help of above runtime models.
1232
1233Interrupt management
1234--------------------
1235
1236GIC ownership
1237~~~~~~~~~~~~~
1238
1239The SPMC owns the GIC configuration. Secure and non-secure interrupts are
1240trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt
1241IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects
1242virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP.
1243
1244Abbreviations:
1245
1246 - NS-Int: A non-secure physical interrupt. It requires a switch to the normal
1247 world to be handled if it triggers while execution is in secure world.
1248 - Other S-Int: A secure physical interrupt targeted to an SP different from
1249 the one that is currently running.
1250 - Self S-Int: A secure physical interrupt targeted to the SP that is currently
1251 running.
1252
1253Non-secure interrupt handling
1254~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1255
1256This section documents the actions supported in SPMC in response to a non-secure
1257interrupt as per the guidance provided by FF-A v1.1 EAC0 specification.
1258An SP specifies one of the following actions in its partition manifest:
1259
1260 - Non-secure interrupt is signaled.
1261 - Non-secure interrupt is signaled after a managed exit.
1262 - Non-secure interrupt is queued.
1263
1264An SP execution context in a call chain could specify a less permissive action
1265than subsequent SP execution contexts in the same call chain. The less
1266permissive action takes precedence over the more permissive actions specified
1267by the subsequent execution contexts. Please refer to FF-A v1.1 EAC0 section
12688.3.1 for further explanation.
1269
1270Secure interrupt handling
1271~~~~~~~~~~~~~~~~~~~~~~~~~
1272
1273This section documents the support implemented for secure interrupt handling in
1274SPMC as per the guidance provided by FF-A v1.1 EAC0 specification.
1275The following assumptions are made about the system configuration:
1276
1277 - In the current implementation, S-EL1 SPs are expected to use the para
1278 virtualized ABIs for interrupt management rather than accessing the virtual
1279 GIC interface.
1280 - Unless explicitly stated otherwise, this support is applicable only for
1281 S-EL1 SPs managed by SPMC.
1282 - Secure interrupts are configured as G1S or G0 interrupts.
1283 - All physical interrupts are routed to SPMC when running a secure partition
1284 execution context.
1285 - All endpoints with multiple execution contexts have their contexts pinned
1286 to corresponding CPUs. Hence, a secure virtual interrupt cannot be signaled
1287 to a target vCPU that is currently running or blocked on a different
1288 physical CPU.
1289
1290A physical secure interrupt could trigger while CPU is executing in normal world
1291or secure world.
1292The action of SPMC for a secure interrupt depends on: the state of the target
1293execution context of the SP that is responsible for handling the interrupt;
1294whether the interrupt triggered while execution was in normal world or secure
1295world.
1296
1297Secure interrupt signaling mechanisms
1298~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1299
1300Signaling refers to the mechanisms used by SPMC to indicate to the SP execution
1301context that it has a pending virtual interrupt and to further run the SP
1302execution context, such that it can handle the virtual interrupt. SPMC uses
1303either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling
1304to S-EL1 SPs. When normal world execution is preempted by a secure interrupt,
1305the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC
1306running in S-EL2.
1307
1308+-----------+---------+---------------+---------------------------------------+
1309| SP State | Conduit | Interface and | Description |
1310| | | parameters | |
1311+-----------+---------+---------------+---------------------------------------+
1312| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending |
1313| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and |
1314| | | | resumes execution context of SP |
1315| | | | through ERET. |
1316+-----------+---------+---------------+---------------------------------------+
1317| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt |
1318| | vIRQ | | is pending. It pends vIRQ signal and |
1319| | | | resumes execution context of SP |
1320| | | | through ERET. |
1321+-----------+---------+---------------+---------------------------------------+
1322| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does |
1323| | | | not resume execution context of SP. |
1324+-----------+---------+---------------+---------------------------------------+
1325| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes|
1326| | vIRQ | | execution context of SP through ERET. |
1327+-----------+---------+---------------+---------------------------------------+
1328
1329Secure interrupt completion mechanisms
1330~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1331
1332A SP signals secure interrupt handling completion to the SPMC through the
1333following mechanisms:
1334
1335 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state.
1336 - ``FFA_RUN`` ABI if its was in BLOCKED state.
1337
1338This is a remnant of SPMC implementation based on the FF-A v1.0 specification.
1339In the current implementation, S-EL1 SPs use the para-virtualized HVC interface
1340implemented by SPMC to perform priority drop and interrupt deactivation (SPMC
1341configures EOImode = 0, i.e. priority drop and deactivation are done together).
1342The SPMC performs checks to deny the state transition upon invocation of
1343either FFA_MSG_WAIT or FFA_RUN interface if the SP didn't perform the
1344deactivation of the secure virtual interrupt.
1345
1346If the current SP execution context was preempted by a secure interrupt to be
1347handled by execution context of target SP, SPMC resumes current SP after signal
1348completion by target SP execution context.
1349
1350Actions for a secure interrupt triggered while execution is in normal world
1351~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1352
1353+-------------------+----------+-----------------------------------------------+
1354| State of target | Action | Description |
1355| execution context | | |
1356+-------------------+----------+-----------------------------------------------+
1357| WAITING | Signaled | This starts a new call chain in SPMC scheduled|
1358| | | mode. |
1359+-------------------+----------+-----------------------------------------------+
1360| PREEMPTED | Queued | The target execution must have been preempted |
1361| | | by a non-secure interrupt. SPMC queues the |
1362| | | secure virtual interrupt now. It is signaled |
1363| | | when the target execution context next enters |
1364| | | the RUNNING state. |
1365+-------------------+----------+-----------------------------------------------+
1366| BLOCKED, RUNNING | NA | The target execution context is blocked or |
1367| | | running on a different CPU. This is not |
1368| | | supported by current SPMC implementation and |
1369| | | execution hits panic. |
1370+-------------------+----------+-----------------------------------------------+
1371
1372If normal world execution was preempted by a secure interrupt, SPMC uses
1373FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling
1374and further returns execution to normal world.
1375
1376The following figure describes interrupt handling flow when a secure interrupt
1377triggers while execution is in normal world:
1378
1379.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png
1380
1381A brief description of the events:
1382
1383 - 1) Secure interrupt triggers while normal world is running.
1384 - 2) FIQ gets trapped to EL3.
1385 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI.
1386 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends
1387 vIRQ).
1388 - 5) Assuming SP1 vCPU is in WAITING state, SPMC signals virtual interrupt
1389 using FFA_INTERRUPT with interrupt id as an argument and resumes the SP1
1390 vCPU using ERET in SPMC scheduled mode.
1391 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual
1392 interrupt is not masked i.e., PSTATE.I = 0
1393 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized
1394 HVC call. SPMC clears the pending virtual interrupt state management
1395 and returns the pending virtual interrupt id.
1396 - 8) SP1 services the virtual interrupt and invokes the paravirtualized
1397 de-activation HVC call. SPMC de-activates the physical interrupt,
1398 clears the fields tracking the secure interrupt and resumes SP1 vCPU.
1399 - 9) SP1 performs secure interrupt completion through FFA_MSG_WAIT ABI.
1400 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME.
1401 - 11) EL3 resumes normal world execution.
1402
1403Actions for a secure interrupt triggered while execution is in secure world
1404~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1405
1406+-------------------+----------+------------------------------------------------+
1407| State of target | Action | Description |
1408| execution context | | |
1409+-------------------+----------+------------------------------------------------+
1410| WAITING | Signaled | This starts a new call chain in SPMC scheduled |
1411| | | mode. |
1412+-------------------+----------+------------------------------------------------+
1413| PREEMPTED by Self | Signaled | The target execution context reenters the |
1414| S-Int | | RUNNING state to handle the secure virtual |
1415| | | interrupt. |
1416+-------------------+----------+------------------------------------------------+
1417| PREEMPTED by | Queued | SPMC queues the secure virtual interrupt now. |
1418| NS-Int | | It is signaled when the target execution |
1419| | | context next enters the RUNNING state. |
1420+-------------------+----------+------------------------------------------------+
1421| BLOCKED | Signaled | Both preempted and target execution contexts |
1422| | | must have been part of the Normal world |
1423| | | scheduled call chain. Refer scenario 1 of |
1424| | | Table 8.4 in the FF-A v1.1 EAC0 spec. |
1425+-------------------+----------+------------------------------------------------+
1426| RUNNING | NA | The target execution context is running on a |
1427| | | different CPU. This scenario is not supported |
1428| | | by current SPMC implementation and execution |
1429| | | hits panic. |
1430+-------------------+----------+------------------------------------------------+
1431
1432The following figure describes interrupt handling flow when a secure interrupt
1433triggers while execution is in secure world. We assume OS kernel sends a direct
1434request message to SP1. Further, SP1 sends a direct request message to SP2. SP1
1435enters BLOCKED state and SPMC resumes SP2.
1436
1437.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png
1438
1439A brief description of the events:
1440
1441 - 1) Secure interrupt triggers while SP2 is running.
1442 - 2) SP2 gets preempted and execution traps to SPMC as IRQ.
1443 - 3) SPMC finds the target vCPU of secure partition responsible for handling
1444 this secure interrupt. In this scenario, it is SP1.
1445 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface.
1446 SPMC further resumes SP1 through ERET conduit. Note that SP1 remains in
1447 Normal world schedule mode.
1448 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual
1449 interrupt is not masked i.e., PSTATE.I = 0
1450 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized
1451 HVC call. SPMC clears the pending virtual interrupt state management
1452 and returns the pending virtual interrupt id.
1453 - 8) SP1 services the virtual interrupt and invokes the paravirtualized
1454 de-activation HVC call. SPMC de-activates the physical interrupt and
1455 clears the fields tracking the secure interrupt and resumes SP1 vCPU.
1456 - 9) Since SP1 direct request completed with FFA_INTERRUPT, it resumes the
1457 direct request to SP2 by invoking FFA_RUN.
1458 - 9) SPMC resumes the pre-empted vCPU of SP2.
1459
1460EL3 interrupt handling
1461~~~~~~~~~~~~~~~~~~~~~~
1462
1463In GICv3 based systems, EL3 interrupts are configured as Group0 secure
1464interrupts. Execution traps to SPMC when a Group0 interrupt triggers while an
1465SP is running. Further, SPMC running at S-EL2 uses FFA_EL3_INTR_HANDLE ABI to
1466request EL3 platform firmware to handle a pending Group0 interrupt.
1467Similarly, SPMD registers a handler with interrupt management framework to
1468delegate handling of Group0 interrupt to the platform if the interrupt triggers
1469in normal world.
1470
1471 - Platform hook
1472
1473 - plat_spmd_handle_group0_interrupt
1474
1475 SPMD provides platform hook to handle Group0 secure interrupts. In the
1476 current design, SPMD expects the platform not to delegate handling to the
1477 NWd (such as through SDEI) while processing Group0 interrupts.
1478
1479Power management
1480----------------
1481
1482In platforms with or without secure virtualization:
1483
1484- The NWd owns the platform PM policy.
1485- The Hypervisor or OS kernel is the component initiating PSCI service calls.
1486- The EL3 PSCI library is in charge of the PM coordination and control
1487 (eventually writing to platform registers).
1488- While coordinating PM events, the PSCI library calls backs into the Secure
1489 Payload Dispatcher for events the latter has statically registered to.
1490
1491When using the SPMD as a Secure Payload Dispatcher:
1492
1493- A power management event is relayed through the SPD hook to the SPMC.
1494- In the current implementation only cpu on (svc_on_finish) and cpu off
1495 (svc_off) hooks are registered.
1496- The behavior for the cpu on event is described in `Secondary cores boot-up`_.
1497 The SPMC is entered through its secondary physical core entry point.
1498- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The PM event is
1499 signaled to the SPMC through a power management framework message.
1500 It consists in a SPMD-to-SPMC direct request/response (`SPMC-SPMD direct
1501 requests/responses`_) conveying the event details and SPMC response.
1502 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and
1503 updates its internal state to reflect the physical core is being turned off.
1504 In the current implementation no SP is resumed as a consequence. This behavior
1505 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux
1506 userspace.
1507
1508Arm architecture extensions for security hardening
J-Alves5eafd222023-10-26 14:19:21 +01001509--------------------------------------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001510
1511Hafnium supports the following architecture extensions for security hardening:
1512
1513- Pointer authentication (FEAT_PAuth): the extension permits detection of forged
1514 pointers used by ROP type of attacks through the signing of the pointer
1515 value. Hafnium is built with the compiler branch protection option to permit
1516 generation of a pointer authentication code for return addresses (pointer
1517 authentication for instructions). The APIA key is used while Hafnium runs.
1518 A random key is generated at boot time and restored upon entry into Hafnium
1519 at run-time. APIA and other keys (APIB, APDA, APDB, APGA) are saved/restored
1520 in vCPU contexts permitting to enable pointer authentication in VMs/SPs.
1521- Branch Target Identification (FEAT_BTI): the extension permits detection of
1522 unexpected indirect branches used by JOP type of attacks. Hafnium is built
1523 with the compiler branch protection option, inserting land pads at function
1524 prologues that are reached by indirect branch instructions (BR/BLR).
1525 Hafnium code pages are marked as guarded in the EL2 Stage-1 MMU descriptors
1526 such that an indirect branch must always target a landpad. A fault is
1527 triggered otherwise. VMs/SPs can (independently) mark their code pages as
1528 guarded in the EL1&0 Stage-1 translation regime.
1529- Memory Tagging Extension (FEAT_MTE): the option permits detection of out of
1530 bound memory array accesses or re-use of an already freed memory region.
1531 Hafnium enables the compiler option permitting to leverage MTE stack tagging
1532 applied to core stacks. Core stacks are marked as normal tagged memory in the
1533 EL2 Stage-1 translation regime. A synchronous data abort is generated upon tag
1534 check failure on load/stores. A random seed is generated at boot time and
1535 restored upon entry into Hafnium. MTE system registers are saved/restored in
1536 vCPU contexts permitting MTE usage from VMs/SPs.
J-Alvesd547d6d2024-05-14 14:59:54 +01001537- Realm Management Extension (FEAT_RME): can be deployed in platforms that leverage
1538 RME for physical address isolation. The SPMC is capable of recovering from a
1539 Granule Protection Fault, if inadvertently accessing a region with the wrong security
1540 state setting. Also, the ability to change dynamically the physical address space of
1541 a region, can be used to enhance the handling of ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE``.
1542 More details in the section about `Memory Sharing`_.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001543
1544SMMUv3 support in Hafnium
J-Alves5eafd222023-10-26 14:19:21 +01001545-------------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001546
1547An SMMU is analogous to an MMU in a CPU. It performs address translations for
1548Direct Memory Access (DMA) requests from system I/O devices.
1549The responsibilities of an SMMU include:
1550
1551- Translation: Incoming DMA requests are translated from bus address space to
1552 system physical address space using translation tables compliant to
1553 Armv8/Armv7 VMSA descriptor format.
1554- Protection: An I/O device can be prohibited from read, write access to a
1555 memory region or allowed.
1556- Isolation: Traffic from each individial device can be independently managed.
1557 The devices are differentiated from each other using unique translation
1558 tables.
1559
1560The following diagram illustrates a typical SMMU IP integrated in a SoC with
1561several I/O devices along with Interconnect and Memory system.
1562
1563.. image:: ../resources/diagrams/MMU-600.png
1564
1565SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides
1566support for SMMUv3 driver in both normal and secure world. A brief introduction
1567of SMMUv3 functionality and the corresponding software support in Hafnium is
1568provided here.
1569
1570SMMUv3 features
J-Alves5eafd222023-10-26 14:19:21 +01001571~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001572
1573- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2)
1574 translation support. It can either bypass or abort incoming translations as
1575 well.
1576- Traffic (memory transactions) from each upstream I/O peripheral device,
1577 referred to as Stream, can be independently managed using a combination of
1578 several memory based configuration structures. This allows the SMMUv3 to
1579 support a large number of streams with each stream assigned to a unique
1580 translation context.
1581- Support for Armv8.1 VMSA where the SMMU shares the translation tables with
1582 a Processing Element. AArch32(LPAE) and AArch64 translation table format
1583 are supported by SMMUv3.
1584- SMMUv3 offers non-secure stream support with secure stream support being
1585 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU
1586 instance for secure and non-secure stream support.
1587- It also supports sub-streams to differentiate traffic from a virtualized
1588 peripheral associated with a VM/SP.
1589- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A
1590 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2
1591 for providing Secure Stage2 translation support to upstream peripheral
1592 devices.
1593
1594SMMUv3 Programming Interfaces
J-Alves5eafd222023-10-26 14:19:21 +01001595~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001596
1597SMMUv3 has three software interfaces that are used by the Hafnium driver to
1598configure the behaviour of SMMUv3 and manage the streams.
1599
1600- Memory based data strutures that provide unique translation context for
1601 each stream.
1602- Memory based circular buffers for command queue and event queue.
1603- A large number of SMMU configuration registers that are memory mapped during
1604 boot time by Hafnium driver. Except a few registers, all configuration
1605 registers have independent secure and non-secure versions to configure the
1606 behaviour of SMMUv3 for translation of secure and non-secure streams
1607 respectively.
1608
1609Peripheral device manifest
J-Alves5eafd222023-10-26 14:19:21 +01001610~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001611
1612Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices.
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001613These DMA devices are dependent on PE endpoint to initiate and receive memory
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001614management transactions on their behalf. The acccess to the MMIO regions of
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001615any such device is assigned to the endpoint during boot.
1616The :ref:`device node<device_region_node>` of the corresponding partition
1617manifest must specify these additional properties for each peripheral device in
1618the system:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001619
1620- smmu-id: This field helps to identify the SMMU instance that this device is
1621 upstream of.
1622- stream-ids: List of stream IDs assigned to this device.
1623
1624.. code:: shell
1625
1626 smmuv3-testengine {
1627 base-address = <0x00000000 0x2bfe0000>;
1628 pages-count = <32>;
1629 attributes = <0x3>;
1630 smmu-id = <0>;
1631 stream-ids = <0x0 0x1>;
1632 interrupts = <0x2 0x3>, <0x4 0x5>;
1633 exclusive-access;
1634 };
1635
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001636DMA isolation
1637-------------
1638
1639Hafnium, with help of SMMUv3 driver, enables the support for static DMA
1640isolation. The DMA device is explicitly granted access to a specific
1641memory region only if the partition requests it by declaring the following
1642properties of the DMA device in the :ref:`memory region node<memory_region_node>`
1643of the partition manifest:
1644
1645- smmu-id
1646- stream-ids
1647- stream-ids-access-permissions
1648
1649SMMUv3 driver uses a unqiue set of stage 2 translations for the DMA device
1650rather than those used on behalf of the PE endpoint. This ensures that the DMA
1651device has a limited visibility of the physical address space.
1652
1653.. code:: shell
1654
1655 smmuv3-memcpy-src {
1656 description = "smmuv3-memcpy-source";
1657 pages-count = <4>;
1658 base-address = <0x00000000 0x7400000>;
1659 attributes = <0x3>; /* read-write */
1660 smmu-id = <0>;
1661 stream-ids = <0x0 0x1>;
1662 stream-ids-access-permissions = <0x3 0x3>;
1663 };
1664
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001665SMMUv3 driver limitations
J-Alves5eafd222023-10-26 14:19:21 +01001666~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001667
1668The primary design goal for the Hafnium SMMU driver is to support secure
1669streams.
1670
1671- Currently, the driver only supports Stage2 translations. No support for
1672 Stage1 or nested translations.
1673- Supports only AArch64 translation format.
1674- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS,
1675 Fault handling, Performance Monitor Extensions, Event Handling, MPAM.
1676- No support for independent peripheral devices.
1677
1678S-EL0 Partition support
J-Alves5eafd222023-10-26 14:19:21 +01001679-----------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001680The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using
1681FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world
1682with ARMv8.4 and FEAT_SEL2).
1683
1684S-EL0 partitions are useful for simple partitions that don't require full
1685Trusted OS functionality. It is also useful to reduce jitter and cycle
1686stealing from normal world since they are more lightweight than VMs.
1687
1688S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by
1689the SPMC. They are differentiated primarily by the 'exception-level' property
1690and the 'execution-ctx-count' property in the SP manifest. They are host apps
1691under the single EL2&0 Stage-1 translation regime controlled by the SPMC and
1692call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions
1693can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions
1694for memory regions.
1695
1696S-EL0 partitions are required by the FF-A specification to be UP endpoints,
1697capable of migrating, and the SPMC enforces this requirement. The SPMC allows
1698a S-EL0 partition to accept a direct message from secure world and normal world,
1699and generate direct responses to them.
1700All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not supported.
1701
Olivier Deprezb8bd7d72023-10-27 16:14:13 +02001702Interrupt handling, Memory sharing, indirect messaging, and notifications features
1703in context of S-EL0 partitions are supported.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001704
1705References
1706==========
1707
J-Alves5eafd222023-10-26 14:19:21 +01001708.. _TF-A project: https://trustedfirmware-a.readthedocs.io/en/latest/
1709
J-Alvesd547d6d2024-05-14 14:59:54 +01001710.. _SPMC FVP build configuration: https://github.com/TF-Hafnium/hafnium-project-reference/blob/main/BUILD.gn#L143
1711
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001712.. _[1]:
1713
1714[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__
1715
1716.. _[2]:
1717
1718[2] `Secure Partition Manager using MM interface <https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partition-manager-mm.html>`__
1719
1720.. _[3]:
1721
1722[3] `Trusted Boot Board Requirements
1723Client <https://developer.arm.com/documentation/den0006/d/>`__
1724
1725.. _[4]:
1726
1727[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45
1728
1729.. _[5]:
1730
1731[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts
1732
1733.. _[6]:
1734
1735[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html
1736
1737.. _[7]:
1738
1739[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts
1740
1741.. _[8]:
1742
1743[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/
1744
1745.. _[9]:
1746
1747[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot
1748
J-Alvesd8094162023-10-26 12:44:33 +01001749.. _[10]:
1750
1751[10] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/build-options.html#
1752
J-Alvesd547d6d2024-05-14 14:59:54 +01001753 .. _[11]:
1754
1755[11] https://developer.arm.com/documentation/den0140/a
1756
1757 .. _[12]:
1758
1759[12] https://developer.arm.com/documentation/den0129/latest/
1760
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001761--------------
1762
1763*Copyright (c) 2020-2023, Arm Limited and Contributors. All rights reserved.*