blob: d9b86ec0b37d50d6c4a2cdbeabda93fe401054bb [file] [log] [blame]
Olivier Deprez8c4cb2d2023-10-27 16:07:11 +02001Foreword
2========
3
4- This document describes the FF-A implementation from `[1]`_ for the
5 configuration where the SPMC resides at S-EL2 on platforms implementing the
6 FEAT_SEL2 architecture extension.
7- It is not an architecture specification and it might provide assumptions on
8 sections mandated as implementation-defined in the specification.
9- It covers the implications of TF-A used as a bootloader, and Hafnium used as a
10 reference code base for an SPMC.
11
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020012Terminology
J-Alvesf7490db2023-10-19 17:57:22 +010013===========
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020014
15- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines
16 (or partitions) in the normal world.
17- The term SPMC refers to the S-EL2 component managing secure partitions in
18 the secure world when the FEAT_SEL2 architecture extension is implemented.
19- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure
20 partition and implementing the FF-A ABI on platforms not implementing the
21 FEAT_SEL2 architecture extension.
22- The term VM refers to a normal world Virtual Machine managed by an Hypervisor.
23- The term SP refers to a secure world "Virtual Machine" managed by an SPMC.
24
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020025Sample reference stack
26======================
27
28The following diagram illustrates a possible configuration when the
J-Alves5eafd222023-10-26 14:19:21 +010029FEAT_SEL2 architecture extension is implemented, showing the |SPMD|
30and |SPMC|, one or multiple secure partitions, with an optional
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020031Hypervisor:
32
J-Alvesc1693772023-10-26 12:41:53 +010033.. image:: ../resources/diagrams/Hafnium_overview_SPMD.png
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020034
J-Alves5eafd222023-10-26 14:19:21 +010035Integration with TF-A (Bootloader and SPMD)
36===========================================
37
38The `TF-A project`_ provides the reference implementation for the secure monitor
39for Arm A class devices, executing at EL3. It includes the implementation of the
40|SPMD|, which manages the world-switch, to relay the FF-A calls to the |SPMC|.
41
42TF-A also serves as the system bootlader, and it was used in the reference
J-Alvesd547d6d2024-05-14 14:59:54 +010043implementation for the SPMC and SPs.
J-Alves5eafd222023-10-26 14:19:21 +010044SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.).
45Thus they are supplied as distinct signed entities within the FIP flash
46image. The FIP image itself is not signed hence this provides the ability
47to upgrade SPs in the field.
48
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020049TF-A build options
J-Alves5eafd222023-10-26 14:19:21 +010050------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020051
J-Alvesd8094162023-10-26 12:44:33 +010052This section explains the TF-A build options for an FF-A based SPM, in which SPMD
53is located at EL3.
54
55This is a step needed for integrating Hafnium as the S-EL2 SPMC and
56the TF-A as SPMD, together making the SPM component.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020057
58- **SPD=spmd**: this option selects the SPMD component to relay the FF-A
59 protocol from NWd to SWd back and forth. It is not possible to
60 enable another Secure Payload Dispatcher when this option is chosen.
61- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception
62 level to being at S-EL2. It defaults to enabled (value 1) when
J-Alvesd8094162023-10-26 12:44:33 +010063 SPD=spmd is chosen.The context save/restore routine and exhaustive list
64 of registers is visible at `[4]`_. When set the reference software stack
65 assumes enablement of FEAT_PAuth, FEAT_BTI and FEAT_MTE architecture
66 extensions.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020067- **SP_LAYOUT_FILE**: this option specifies a text description file
68 providing paths to SP binary images and manifests in DTS format
J-Alves5eafd222023-10-26 14:19:21 +010069 (see `Secure Partitions Layout File`_). It is required when ``SPMD_SPM_AT_SEL2``
J-Alvesd8094162023-10-26 12:44:33 +010070 is enabled, i.e. when multiple secure partitions are to be loaded by BL2 on
71 behalf of the SPMC.
72- **BL32** option is re-purposed to specify the SPMC image. It can specify either
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020073 the Hafnium binary path (built for the secure world) or the path to a TEE
74 binary implementing FF-A interfaces.
J-Alvesd8094162023-10-26 12:44:33 +010075- **BL33** option to specify normal world loader such as U-Boot or the UEFI
76 framework payload, which would use FF-A calls during runtime to interact with
77 Hafnium as the SPMC.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020078
J-Alvesd8094162023-10-26 12:44:33 +010079As a result of configuring ``SPD=spmd`` and ``SPMD_SPM_AT_SEL2`` TF-A provides
80context save/restore operations when entering/exiting an EL2 execution context.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020081
J-Alvesd8094162023-10-26 12:44:33 +010082There are other build options that relate support other valid FF-A
83system configurations where the SPMC is implemented at S-EL1 and EL3.
84Note that they conflict with those needed to integrate with Hafnium as the SPMC.
85For more details refer to |TF-A| build options `[10]`_.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020086
87Sample TF-A build command line when FEAT_SEL2 architecture extension is
J-Alvesd8094162023-10-26 12:44:33 +010088implemented and the SPMC is located at S-EL2, for Arm's FVP platform:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020089
90.. code:: shell
91
92 make \
93 CROSS_COMPILE=aarch64-none-elf- \
94 PLAT=fvp \
95 SPD=spmd \
96 ARM_ARCH_MINOR=5 \
97 BRANCH_PROTECTION=1 \
J-Alves874737a2024-03-20 17:30:24 +000098 ENABLE_FEAT_MTE2=1 \
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +020099 BL32=<path-to-hafnium-binary> \
100 BL33=<path-to-bl33-binary> \
101 SP_LAYOUT_FILE=sp_layout.json \
102 all fip
103
104Sample TF-A build command line when FEAT_SEL2 architecture extension is
105implemented, the SPMC is located at S-EL2, and enabling secure boot:
106
107.. code:: shell
108
109 make \
110 CROSS_COMPILE=aarch64-none-elf- \
111 PLAT=fvp \
112 SPD=spmd \
113 ARM_ARCH_MINOR=5 \
114 BRANCH_PROTECTION=1 \
J-Alves874737a2024-03-20 17:30:24 +0000115 ENABLE_FEAT_MTE2=1 \
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200116 BL32=<path-to-hafnium-binary> \
117 BL33=<path-to-bl33-binary> \
118 SP_LAYOUT_FILE=sp_layout.json \
119 MBEDTLS_DIR=<path-to-mbedtls-lib> \
120 TRUSTED_BOARD_BOOT=1 \
121 COT=dualroot \
122 ARM_ROTPK_LOCATION=devel_rsa \
123 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \
124 GENERATE_COT=1 \
125 all fip
126
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200127FVP model invocation
J-Alves5eafd222023-10-26 14:19:21 +0100128--------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200129
130The FVP command line needs the following options to exercise the S-EL2 SPMC:
131
132+---------------------------------------------------+------------------------------------+
133| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, |
134| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. |
135+---------------------------------------------------+------------------------------------+
136| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the |
137| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. |
138| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | |
139| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | |
140| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | |
141| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | |
142| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | |
143| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | |
144+---------------------------------------------------+------------------------------------+
145| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. |
146| - cluster1.has_branch_target_exception=1 | |
147+---------------------------------------------------+------------------------------------+
148| - cluster0.has_pointer_authentication=2 | Implements FEAT_PAuth |
149| - cluster1.has_pointer_authentication=2 | |
150+---------------------------------------------------+------------------------------------+
151| - cluster0.memory_tagging_support_level=2 | Implements FEAT_MTE2 |
152| - cluster1.memory_tagging_support_level=2 | |
153| - bp.dram_metadata.is_enabled=1 | |
154+---------------------------------------------------+------------------------------------+
155
156Sample FVP command line invocation:
157
158.. code:: shell
159
160 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \
161 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \
162 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \
163 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \
164 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \
165 -C bp.pl011_uart2.out_file=fvp-uart2.log \
166 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 \
167 -C cluster0.has_pointer_authentication=2 -C cluster1.has_pointer_authentication=2 \
168 -C cluster0.has_branch_target_exception=1 -C cluster1.has_branch_target_exception=1 \
169 -C cluster0.memory_tagging_support_level=2 -C cluster1.memory_tagging_support_level=2 \
170 -C bp.dram_metadata.is_enabled=1 \
171 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B \
172 -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 \
173 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 \
174 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0
175
J-Alves5eafd222023-10-26 14:19:21 +0100176SPMC Configuration
177==================
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200178
J-Alves5eafd222023-10-26 14:19:21 +0100179This section details the configuration files required to deploy Hafnium as the SPMC,
180along with those required to configure each secure partion.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200181
J-Alves5eafd222023-10-26 14:19:21 +0100182SPMC Manifest
183-------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200184
J-Alves5eafd222023-10-26 14:19:21 +0100185This manifest contains the SPMC *attribute* node consumed by the SPMD at boot
186time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves
187two different cases:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200188
J-Alves5eafd222023-10-26 14:19:21 +0100189The SPMC manifest is used by the SPMD to setup the environment required by the
190SPMC to run at S-EL2. SPs run at S-EL1 or S-EL0.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200191
J-Alves5eafd222023-10-26 14:19:21 +0100192.. code:: shell
193
194 attribute {
195 spmc_id = <0x8000>;
196 maj_ver = <0x1>;
197 min_ver = <0x1>;
198 exec_state = <0x0>;
199 load_address = <0x0 0x6000000>;
200 entrypoint = <0x0 0x6000000>;
201 binary_size = <0x60000>;
202 };
203
204- *spmc_id* defines the endpoint ID value that SPMC can query through
205 ``FFA_ID_GET``.
206- *maj_ver/min_ver*. SPMD checks provided FF-A version versus its internal
207 version and aborts if not matching.
208- *exec_state* defines the SPMC execution state (AArch64 or AArch32).
209 Notice Hafnium used as a SPMC only supports AArch64.
210- *load_address* and *binary_size* are mostly used to verify secondary
211 entry points fit into the loaded binary image.
212- *entrypoint* defines the cold boot primary core entry point used by
213 SPMD (currently matches ``BL32_BASE``) to enter the SPMC.
214
215Other nodes in the manifest are consumed by Hafnium in the secure world.
216A sample can be found at `[7]`_:
217
218- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute
219 indicates a FF-A compliant SP. The *load_address* field specifies the load
220 address at which BL2 loaded the SP package.
221- The *cpus* node provides the platform topology and allows MPIDR to VMPIDR mapping.
222 Note the primary core is declared first, then secondary cores are declared
223 in reverse order.
224- The *memory* nodes provide platform information on the ranges of memory
225 available for use by SPs at runtime. These ranges relate to either
226 secure or non-secure memory, depending on the *device_type* field.
227 If the field specifies "memory" the range is secure, else if it specifies
228 "ns-memory" the memory is non-secure. The system integrator must exclude
229 the memory used by other components that are not SPs, such as the monitor,
230 or the SPMC itself, the OS Kernel/Hypervisor, or other NWd VMs.
231 The SPMC limits the SP's address space such that they can only refer to memory
232 inside of those ranges, either by defining memory region nodes in their manifest
233 as well as memory starting at the load address until the limit defined by the memory
234 size. Thus, the SPMC prevents rogue SPs from tampering with memory from other
235 components.
236
J-Alvesc143a342023-11-07 12:17:44 +0000237.. code:: shell
238
239 memory@0 {
240 device_type = "memory";
241 reg = <0x0 0x6000000 0x2000000 0x0 0xff000000 0x1000000>;
242 };
243
244 memory@1 {
245 device_type = "ns-memory";
246 reg = <0x0 0x90010000 0x70000000>;
247 };
248
249Above find an example representation of the referred memory description. The
250ranges are described in a list of unsigned 32-bit values, in which the first
251two addresses relate to the based physical address, followed by the respective
252page size. The first secure range defined in the node below has base address
253`0x0 0x6000000` and size `0x2000000`; following there is another range with
254base address `0x0 0xff000000` and size `0x1000000`.
255
J-Alves5eafd222023-10-26 14:19:21 +0100256Secure Partitions Configuration
257-------------------------------
258
259SP Manifests
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200260~~~~~~~~~~~~
261
262An SP manifest describes SP attributes as defined in `[1]`_
263(partition manifest at virtual FF-A instance) in DTS format. It is
264represented as a single file associated with the SP. A sample is
265provided by `[5]`_. A binding document is provided by `[6]`_.
266
J-Alves5eafd222023-10-26 14:19:21 +0100267Platform topology
268~~~~~~~~~~~~~~~~~
269
270The *execution-ctx-count* SP manifest field can take the value of one or the
271total number of PEs. The FF-A specification `[1]`_ recommends the
272following SP types:
273
274- Pinned MP SPs: an execution context matches a physical PE. MP SPs must
275 implement the same number of ECs as the number of PEs in the platform.
276- Migratable UP SPs: a single execution context can run and be migrated on any
277 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can
278 receive a direct message request originating from any physical core targeting
279 the single execution context.
280
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200281Secure Partition packages
282~~~~~~~~~~~~~~~~~~~~~~~~~
283
284Secure partitions are bundled as independent package files consisting
285of:
286
287- a header
288- a DTB
289- an image payload
290
291The header starts with a magic value and offset values to SP DTB and
292image payload. Each SP package is loaded independently by BL2 loader
293and verified for authenticity and integrity.
294
295The SP package identified by its UUID (matching FF-A uuid property) is
296inserted as a single entry into the FIP at end of the TF-A build flow
297as shown:
298
299.. code:: shell
300
301 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw"
302 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw"
303 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw"
304 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw"
305 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config"
306 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config"
307 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config"
308 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config"
309 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config"
310 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob"
311 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob"
312
313.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml
314
J-Alves5eafd222023-10-26 14:19:21 +0100315Secure Partitions Layout File
316~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200317
318A json-formatted description file is passed to the build flow specifying paths
319to the SP binary image and associated DTS partition manifest file. The latter
320is processed by the dtc compiler to generate a DTB fed into the SP package.
Karl Meakin82593ce2023-08-30 16:38:28 +0100321Each partition can be configured with the following fields:
322
323:code:`image`
324 - Specifies the filename and offset of the image within the SP package.
325 - Can be written as :code:`"image": { "file": "path", "offset": 0x1234 }` to
326 give both :code:`image.file` and :code:`image.offset` values explicitly, or
327 can be written as :code:`"image": "path"` to give :code:`image.file` and value
328 and leave :code:`image.offset` absent.
329
330 :code:`image.file`
331 - Specifies the filename of the image.
332
333 :code:`image.offset`
334 - Specifies the offset of the image within the SP package.
335 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC.
336 - Optional. Defaults to :code:`0x4000`.
337
338:code:`pm`
339 - Specifies the filename and offset of the partition manifest within the SP package.
340 - Can be written as :code:`"pm": { "file": "path", "offset": 0x1234 }` to
341 give both :code:`pm.file` and :code:`pm.offset` values explicitly, or
342 can be written as :code:`"pm": "path"` to give :code:`pm.file` and value
343 and leave :code:`pm.offset` absent.
344
345 :code:`pm.file`
346 - Specifies the filename of the partition manifest.
347
348 :code:`pm.offset`
349 - Specifies the offset of the partition manifest within the SP package.
350 - Must be 4KB aligned, because that is the translation granule supported by Hafnium SPMC.
351 - Optional. Defaults to :code:`0x1000`.
352
353:code:`image.offset` and :code:`pm.offset` can be leveraged to support SPs with
354S1 translation granules that differ from 4KB, and to configure the regions
355allocated within the SP package, as well as to comply with the requirements for
356the implementation of the boot information protocol (see `Passing boot data to
357the SP`_ for more details).
358
359:code:`owner`
360 - Specifies the SP owner, identifying the signing domain in case of dual root CoT.
361 - Possible values are :code:`SiP` (silicon owner) or :code:`Plat` (platform owner).
362 - Optional. Defaults to :code:`SiP`.
363
364:code:`uuid`
365 - Specifies the UUID of the partition.
366 - Optional. Defaults to the value of the :code:`uuid` field from the DTS partition manifest.
367
368:code:`physical-load-address`
369 - Specifies the :code:`load_address` field of the generated DTS fragment.
370 - Optional. Defaults to the value of the :code:`load-address` from the DTS partition manifest.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200371
372.. code:: shell
373
374 {
375 "tee1" : {
376 "image": "tee1.bin",
377 "pm": "tee1.dts",
378 "owner": "SiP",
379 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f"
380 },
381
382 "tee2" : {
383 "image": "tee2.bin",
384 "pm": "tee2.dts",
385 "owner": "Plat"
386 },
387
388 "tee3" : {
389 "image": {
390 "file": "tee3.bin",
391 "offset":"0x2000"
392 },
393 "pm": {
394 "file": "tee3.dts",
395 "offset":"0x6000"
396 },
397 "owner": "Plat"
398 },
399 }
400
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200401SPMC boot
J-Alves5eafd222023-10-26 14:19:21 +0100402=========
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200403
404The SPMC is loaded by BL2 as the BL32 image.
405
406The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_.
407
408BL2 passes the SPMC manifest address to BL31 through a register.
409
410At boot time, the SPMD in BL31 runs from the primary core, initializes the core
411contexts and launches the SPMC (BL32) passing the following information through
412registers:
413
414- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob).
415- X1 holds the ``HW_CONFIG`` physical address.
416- X4 holds the currently running core linear id.
417
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200418Secure boot
J-Alves5eafd222023-10-26 14:19:21 +0100419-----------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200420
421The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC,
422SPMC manifest, secure partitions and verifies them for authenticity and integrity.
423Refer to TBBR specification `[3]`_.
424
425The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows
426the use of two root keys namely S-ROTPK and NS-ROTPK:
427
428- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK.
429- BL33 may be signed by the OEM using NS-ROTPK.
430- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK).
431- A maximum of 4 partitions can be signed with the S-ROTPK key and 4 partitions
432 signed with the NS-ROTPK key.
433
J-Alves5eafd222023-10-26 14:19:21 +0100434Also refer to `Secure Partitions Configuration`_ and `TF-A build options`_ sections.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200435
436Boot phases
437-----------
438
439Primary core boot-up
440~~~~~~~~~~~~~~~~~~~~
441
442Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical
443core. The SPMC performs its platform initializations and registers the SPMC
444secondary physical core entry point physical address by the use of the
445`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD
446at secure physical FF-A instance).
447
J-Alvesc143a342023-11-07 12:17:44 +0000448The SPMC then creates secure partitions base on SP packages and manifests. Each
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200449secure partition is launched in sequence (`SP Boot order`_) on their "primary"
450execution context. If the primary boot physical core linear id is N, an MP SP is
451started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a
452UP SP, it is started using its unique EC0 on PE[N].
453
454The SP primary EC (or the EC used when the partition is booted as described
455above):
456
457- Performs the overall SP boot time initialization, and in case of a MP SP,
458 prepares the SP environment for other execution contexts.
459- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure
460 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA
461 entry point for other execution contexts.
462- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or
463 ``FFA_ERROR`` in case of failure.
464
465Secondary cores boot-up
466~~~~~~~~~~~~~~~~~~~~~~~
467
468Once the system is started and NWd brought up, a secondary physical core is
469woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism
470calls into the SPMD on the newly woken up physical core. Then the SPMC is
471entered at the secondary physical core entry point.
472
473In the current implementation, the first SP is resumed on the coresponding EC
474(the virtual CPU which matches the physical core). The implication is that the
475first SP must be a MP SP.
476
477In a linux based system, once secure and normal worlds are booted but prior to
478a NWd FF-A driver has been loaded:
479
480- The first SP has initialized all its ECs in response to primary core boot up
481 (at system initialization) and secondary core boot up (as a result of linux
482 invoking PSCI_CPU_ON for all secondary cores).
483- Other SPs have their first execution context initialized as a result of secure
484 world initialization on the primary boot core. Other ECs for those SPs have to
485 be run first through ffa_run to complete their initialization (which results
486 in the EC completing with FFA_MSG_WAIT).
487
488Refer to `Power management`_ for further details.
489
J-Alves5eafd222023-10-26 14:19:21 +0100490Loading of SPs
491--------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200492
J-Alves5eafd222023-10-26 14:19:21 +0100493At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted
494below:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200495
J-Alves5eafd222023-10-26 14:19:21 +0100496.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200497
J-Alves5eafd222023-10-26 14:19:21 +0100498Note this boot flow is an implementation sample on Arm's FVP platform.
499Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a
500different boot flow. The flow restricts to a maximum of 8 secure partitions.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200501
J-Alves5eafd222023-10-26 14:19:21 +0100502SP Boot order
503~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200504
J-Alves5eafd222023-10-26 14:19:21 +0100505SP manifests provide an optional boot order attribute meant to resolve
506dependencies such as an SP providing a service required to properly boot
507another SP. SPMC boots the SPs in accordance to the boot order attribute,
508lowest to the highest value. If the boot order attribute is absent from the FF-A
509manifest, the SP is treated as if it had the highest boot order value
510(i.e. lowest booting priority). The FF-A specification mandates this field
511is unique to each SP.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200512
J-Alves5eafd222023-10-26 14:19:21 +0100513It is possible for an SP to call into another SP through a direct request
514provided the latter SP has already been booted.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200515
J-Alves5eafd222023-10-26 14:19:21 +0100516Passing boot data to the SP
517~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200518
J-Alves5eafd222023-10-26 14:19:21 +0100519In `[1]`_ , the section "Boot information protocol" defines a method for passing
520data to the SPs at boot time. It specifies the format for the boot information
521descriptor and boot information header structures, which describe the data to be
522exchanged between SPMC and SP.
523The specification also defines the types of data that can be passed.
524The aggregate of both the boot info structures and the data itself is designated
525the boot information blob, and is passed to a Partition as a contiguous memory
526region.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200527
J-Alves5eafd222023-10-26 14:19:21 +0100528Currently, the SPM implementation supports the FDT type which is used to pass the
529partition's DTB manifest.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200530
J-Alves5eafd222023-10-26 14:19:21 +0100531The region for the boot information blob is allocated through the SP package.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200532
J-Alves5eafd222023-10-26 14:19:21 +0100533.. image:: ../resources/diagrams/partition-package.png
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200534
J-Alves5eafd222023-10-26 14:19:21 +0100535To adjust the space allocated for the boot information blob, the json description
536of the SP (see section `Secure Partitions Layout File`_) shall be updated to contain
537the manifest offset. If no offset is provided the manifest offset defaults to 0x1000,
538which is the page size in the Hafnium SPMC.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200539
J-Alves5eafd222023-10-26 14:19:21 +0100540The configuration of the boot protocol is done in the SPs manifest. As defined by
541the specification, the manifest field 'gp-register-num' configures the GP register
542which shall be used to pass the address to the partitions boot information blob when
543booting the partition.
544In addition, the Hafnium SPMC implementation requires the boot information arguments
545to be listed in a designated DT node:
546
547.. code:: shell
548
549 boot-info {
550 compatible = "arm,ffa-manifest-boot-info";
551 ffa_manifest;
552 };
553
554The whole secure partition package image (see `Secure Partition packages`_) is
555mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can
556retrieve the address for the boot information blob in the designated GP register,
557process the boot information header and descriptors, access its own manifest
558DTB blob and extract its partition manifest properties.
559
560SPMC Runtime
561============
562
563Parsing SP partition manifests
564------------------------------
565
566Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_.
567Note the current implementation may not implement all optional fields.
568
569The SP manifest may contain memory and device regions nodes:
570
571- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at
572 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can
573 specify RX/TX buffer regions in which case it is not necessary for an SP
574 to explicitly invoke the ``FFA_RXTX_MAP`` interface. The memory referred
575 shall be contained within the memory ranges defined in SPMC manifest. The
576 NS bit in the attributes field should be consistent with the security
577 state of the range that it relates to. I.e. non-secure memory shall be
578 part of a non-secure memory range, and secure memory shall be contained
579 in a secure memory range of a given platform.
580- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or
581 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate
582 additional resources (e.g. interrupts).
583
584For the SPMC, base addresses for memory and device region nodes are IPAs provided
585the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation regime.
586
Olivier Deprezb8bd7d72023-10-27 16:14:13 +0200587ote: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the
J-Alves5eafd222023-10-26 14:19:21 +0100588same set of page tables. It is still open whether two sets of page tables shall
589be provided per SP. The memory region node as defined in the specification
590provides a memory security attribute hinting to map either to the secure or
591non-secure EL1&0 Stage-2 table if it exists.
592
593Secure partitions scheduling
594----------------------------
595
Olivier Deprez8c4cb2d2023-10-27 16:07:11 +0200596The FF-A specification `[1]`_ provides two ways to allocate CPU cycles to
J-Alves5eafd222023-10-26 14:19:21 +0100597secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of:
598
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500599- the FFA_MSG_SEND_DIRECT_REQ (or FFA_MSG_SEND_DIRECT_REQ2) interface.
J-Alves5eafd222023-10-26 14:19:21 +0100600- the FFA_RUN interface.
601
602Additionally a secure interrupt can pre-empt the normal world execution and give
603CPU cycles by transitioning to EL3 and S-EL2.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200604
605Mandatory interfaces
606--------------------
607
608The following interfaces are exposed to SPs:
609
610- ``FFA_VERSION``
611- ``FFA_FEATURES``
612- ``FFA_RX_RELEASE``
613- ``FFA_RXTX_MAP``
614- ``FFA_RXTX_UNMAP``
615- ``FFA_PARTITION_INFO_GET``
616- ``FFA_ID_GET``
617- ``FFA_MSG_WAIT``
618- ``FFA_MSG_SEND_DIRECT_REQ``
619- ``FFA_MSG_SEND_DIRECT_RESP``
620- ``FFA_MEM_DONATE``
621- ``FFA_MEM_LEND``
622- ``FFA_MEM_SHARE``
623- ``FFA_MEM_RETRIEVE_REQ``
624- ``FFA_MEM_RETRIEVE_RESP``
625- ``FFA_MEM_RELINQUISH``
626- ``FFA_MEM_FRAG_RX``
627- ``FFA_MEM_FRAG_TX``
628- ``FFA_MEM_RECLAIM``
629- ``FFA_RUN``
630
631As part of the FF-A v1.1 support, the following interfaces were added:
632
633 - ``FFA_NOTIFICATION_BITMAP_CREATE``
634 - ``FFA_NOTIFICATION_BITMAP_DESTROY``
635 - ``FFA_NOTIFICATION_BIND``
636 - ``FFA_NOTIFICATION_UNBIND``
637 - ``FFA_NOTIFICATION_SET``
638 - ``FFA_NOTIFICATION_GET``
639 - ``FFA_NOTIFICATION_INFO_GET``
640 - ``FFA_SPM_ID_GET``
641 - ``FFA_SECONDARY_EP_REGISTER``
642 - ``FFA_MEM_PERM_GET``
643 - ``FFA_MEM_PERM_SET``
644 - ``FFA_MSG_SEND2``
645 - ``FFA_RX_ACQUIRE``
646
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700647As part of the FF-A v1.2 support, the following interfaces were added:
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500648
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700649- ``FFA_PARTITION_INFO_GET_REGS``
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500650- ``FFA_MSG_SEND_DIRECT_REQ2``
651- ``FFA_MSG_SEND_DIRECT_RESP2``
Karl Meakind40979f2024-05-13 10:21:56 +0100652- ``FFA_CONSOLE_LOG``
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700653
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200654FFA_VERSION
655~~~~~~~~~~~
656
657``FFA_VERSION`` requires a *requested_version* parameter from the caller.
658The returned value depends on the caller:
659
660- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version
661 specified in the SPMC manifest.
662- SP: the SPMC returns its own implemented version.
663- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version.
664
665FFA_FEATURES
666~~~~~~~~~~~~
667
668FF-A features supported by the SPMC may be discovered by secure partitions at
669boot (that is prior to NWd is booted) or run-time.
670
671The SPMC calling FFA_FEATURES at secure physical FF-A instance always get
672FFA_SUCCESS from the SPMD.
673
Karl Meakin963a5d72024-05-13 10:32:29 +0100674S-EL1 partitions calling FFA_FEATURES at virtual FF-A instance with NPI and MEI
675interrupt feature IDs get FFA_SUCCESS.
676
677S-EL0 partitions are not supported for NPI: ``FFA_NOT_SUPPORTED`` will be
678returned.
679
680Physical FF-A instances are not supported for NPI and MEI: ``FFA_NOT_SUPPORTED``
681will be returned.
682
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200683The request made by an Hypervisor or OS kernel is forwarded to the SPMC and
684the response relayed back to the NWd.
685
686FFA_RXTX_MAP/FFA_RXTX_UNMAP
687~~~~~~~~~~~~~~~~~~~~~~~~~~~
688
689When invoked from a secure partition FFA_RXTX_MAP maps the provided send and
690receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation
691regime as secure buffers in the MMU descriptors.
692
693When invoked from the Hypervisor or OS kernel, the buffers are mapped into the
694SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU
695descriptors. The provided addresses may be owned by a VM in the normal world,
696which is expected to receive messages from the secure world. The SPMC will in
697this case allocate internal state structures to facilitate RX buffer access
698synchronization (through FFA_RX_ACQUIRE interface), and to permit SPs to send
Karl Meakinb1dbca92024-01-24 16:51:22 +0000699messages. The addresses used must be contained in the SPMC manifest NS memory
700node (see `SPMC manifest`_).
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200701
702The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the
703caller, either it being the Hypervisor or OS kernel, as well as a secure
Karl Meakinb1dbca92024-01-24 16:51:22 +0000704partition, and restores them in the VM's translation regime so that they can be
705used for memory sharing operations from the normal world again.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200706
Karl Meakin963a5d72024-05-13 10:32:29 +0100707The minimum and maximum buffer sizes supported by the FF-A instance can be
708queried by calling ``FFA_FEATURES`` with the ``FFA_RXTX_MAP`` function ID.
709
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200710FFA_PARTITION_INFO_GET
711~~~~~~~~~~~~~~~~~~~~~~
712
713Partition info get call can originate:
714
715- from SP to SPMC
716- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD.
717
Raghu Krishnamurthy4a793e92023-08-09 10:10:23 -0700718FFA_PARTITION_INFO_GET_REGS
719~~~~~~~~~~~~~~~~~~~~~~~~~~~
720
721This call can originate:
722
723- from SP to SPMC
724- from SPMC to SPMD
725- from Hypervsior or OS kernel to SPMC. The request is relayed by the SPMD.
726
727The primary use of this ABI is to return partition information via registers
728as opposed to via RX/TX buffers and is useful in cases where sharing memory is
729difficult.
730
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200731FFA_ID_GET
732~~~~~~~~~~
733
734The FF-A id space is split into a non-secure space and secure space:
735
736- FF-A ID with bit 15 clear relates to VMs.
737- FF-A ID with bit 15 set related to SPs.
738- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD
739 and SPMC.
740
741The SPMD returns:
742
743- The default zero value on invocation from the Hypervisor.
744- The ``spmc_id`` value specified in the SPMC manifest on invocation from
745 the SPMC (see `SPMC manifest`_)
746
747This convention helps the SPMC to determine the origin and destination worlds in
748an FF-A ABI invocation. In particular the SPMC shall filter unauthorized
749transactions in its world switch routine. It must not be permitted for a VM to
750use a secure FF-A ID as origin world by spoofing:
751
752- A VM-to-SP direct request/response shall set the origin world to be non-secure
753 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15
754 set).
755- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15
756 for both origin and destination IDs.
757
758An incoming direct message request arriving at SPMD from NWd is forwarded to
759SPMC without a specific check. The SPMC is resumed through eret and "knows" the
760message is coming from normal world in this specific code path. Thus the origin
761endpoint ID must be checked by SPMC for being a normal world ID.
762
763An SP sending a direct message request must have bit 15 set in its origin
764endpoint ID and this can be checked by the SPMC when the SP invokes the ABI.
765
766The SPMC shall reject the direct message if the claimed world in origin endpoint
767ID is not consistent:
768
769- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal
770 world ID",
771- or initiated by an SP and thus origin endpoint ID must be a "secure world ID".
772
773
774FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP
775~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
776
777This is a mandatory interface for secure partitions consisting in direct request
778and responses with the following rules:
779
780- An SP can send a direct request to another SP.
781- An SP can receive a direct request from another SP.
782- An SP can send a direct response to another SP.
783- An SP cannot send a direct request to an Hypervisor or OS kernel.
784- An Hypervisor or OS kernel can send a direct request to an SP.
785- An SP can send a direct response to an Hypervisor or OS kernel.
786
Kathleen Capella6e3abcf2024-02-05 16:17:35 -0500787FFA_MSG_SEND_DIRECT_REQ2/FFA_MSG_SEND_DIRECT_RESP2
788~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
789
790The primary usage of these ABIs is to send a direct request to a specified
791UUID within an SP that has multiple UUIDs declared in its manifest.
792
793Secondarily, it can be used to send a direct request with an extended
794set of message payload arguments.
795
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200796FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY
797~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
798
799The secure partitions notifications bitmap are statically allocated by the SPMC.
800Hence, this interface is not to be issued by secure partitions.
801
802At initialization, the SPMC is not aware of VMs/partitions deployed in the
803normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC
804to be prepared to handle notifications for the provided VM ID.
805
806FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND
807~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
808
809Pair of interfaces to manage permissions to signal notifications. Prior to
810handling notifications, an FF-A endpoint must allow a given sender to signal a
811bitmap of notifications.
812
813If the receiver doesn't have notification support enabled in its FF-A manifest,
814it won't be able to bind notifications, hence forbidding it to receive any
815notifications.
816
817FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET
818~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
819
820FFA_NOTIFICATION_GET retrieves all pending global notifications and
821per-vCPU notifications targeted to the current vCPU.
822
823Hafnium maintains a global count of pending notifications which gets incremented
824and decremented when handling FFA_NOTIFICATION_SET and FFA_NOTIFICATION_GET
825respectively. A delayed SRI is triggered if the counter is non-zero when the
826SPMC returns to normal world.
827
828FFA_NOTIFICATION_INFO_GET
829~~~~~~~~~~~~~~~~~~~~~~~~~
830
831Hafnium maintains a global count of pending notifications whose information
832has been retrieved by this interface. The count is incremented and decremented
833when handling FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET respectively.
834It also tracks notifications whose information has been retrieved individually,
835such that it avoids duplicating returned information for subsequent calls to
836FFA_NOTIFICATION_INFO_GET. For each notification, this state information is
837reset when receiver called FFA_NOTIFICATION_GET to retrieve them.
838
839FFA_SPM_ID_GET
840~~~~~~~~~~~~~~
841
842Returns the FF-A ID allocated to an SPM component which can be one of SPMD
843or SPMC.
844
845At initialization, the SPMC queries the SPMD for the SPMC ID, using the
846FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using
847the FFA_SPM_ID_GET interface at the secure physical FF-A instance.
848
849Secure partitions call this interface at the virtual FF-A instance, to which
850the SPMC returns the priorly retrieved SPMC ID.
851
852The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the
853SPMD, which returns the SPMC ID.
854
855FFA_SECONDARY_EP_REGISTER
856~~~~~~~~~~~~~~~~~~~~~~~~~
857
858When the SPMC boots, all secure partitions are initialized on their primary
859Execution Context.
860
861The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition
862from its first execution context, to provide the entry point address for
863secondary execution contexts.
864
865A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from
866the NWd or by invocation of FFA_RUN.
867
868FFA_RX_ACQUIRE/FFA_RX_RELEASE
869~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
870
871The RX buffers can be used to pass information to an FF-A endpoint in the
872following scenarios:
873
874 - When it was targetted by a FFA_MSG_SEND2 invokation from another endpoint.
875 - Return the result of calling ``FFA_PARTITION_INFO_GET``.
876 - In a memory share operation, as part of the ``FFA_MEM_RETRIEVE_RESP``,
877 with the memory descriptor of the shared memory.
878
879If a normal world VM is expected to exchange messages with secure world,
880its RX/TX buffer addresses are forwarded to the SPMC via FFA_RXTX_MAP ABI,
881and are from this moment owned by the SPMC.
882The hypervisor must call the FFA_RX_ACQUIRE interface before attempting
883to use the RX buffer, in any of the aforementioned scenarios. A successful
884call to FFA_RX_ACQUIRE transfers ownership of RX buffer to hypervisor, such
885that it can be safely used.
886
887The FFA_RX_RELEASE interface is used after the FF-A endpoint is done with
888processing the data received in its RX buffer. If the RX buffer has been
889acquired by the hypervisor, the FFA_RX_RELEASE call must be forwarded to
890the SPMC to reestablish SPMC's RX ownership.
891
892An attempt from an SP to send a message to a normal world VM whose RX buffer
893was acquired by the hypervisor fails with error code FFA_BUSY, to preserve
894the RX buffer integrity.
895The operation could then be conducted after FFA_RX_RELEASE.
896
897FFA_MSG_SEND2
898~~~~~~~~~~~~~
899
900Hafnium copies a message from the sender TX buffer into receiver's RX buffer.
901For messages from SPs to VMs, operation is only possible if the SPMC owns
902the receiver's RX buffer.
903
904Both receiver and sender need to enable support for indirect messaging,
905in their respective partition manifest. The discovery of support
906of such feature can be done via FFA_PARTITION_INFO_GET.
907
908On a successful message send, Hafnium pends an RX buffer full framework
909notification for the receiver, to inform it about a message in the RX buffer.
910
911The handling of framework notifications is similar to that of
912global notifications. Binding of these is not necessary, as these are
913reserved to be used by the hypervisor or SPMC.
914
Karl Meakind40979f2024-05-13 10:21:56 +0100915FFA_CONSOLE_LOG
916~~~~~~~~~~~~~~~
917
918``FFA_CONSOLE_LOG`` allows debug logging to the UART console.
919Characters are packed into registers:
920 - `w2-w7` (|SMCCC| 32-bit)
921 - `x2-x7` (|SMCCC| 64-bit, before v1.2)
922 - `x2-x17` (|SMCCC| 64-bit, v1.2 or later)
923
Madhukar Pappireddy0b2304b2023-08-15 18:05:21 -0500924Paravirtualized interfaces
925--------------------------
926
927Hafnium SPMC implements the following implementation-defined interface(s):
928
929HF_INTERRUPT_ENABLE
930~~~~~~~~~~~~~~~~~~~
931
932Enables or disables the given virtual interrupt for the calling execution
933context. Returns 0 on success, or -1 if the interrupt id is invalid.
934
935HF_INTERRUPT_GET
936~~~~~~~~~~~~~~~~
937
938Returns the ID of the next pending virtual interrupt for the calling execution
939context, and acknowledges it (i.e. marks it as no longer pending). Returns
940HF_INVALID_INTID if there are no pending interrupts.
941
942HF_INTERRUPT_DEACTIVATE
943~~~~~~~~~~~~~~~~~~~~~~~
944
945Drops the current interrupt priority and deactivates the given virtual and
946physical interrupt ID for the calling execution context. Returns 0 on success,
947or -1 otherwise.
948
949HF_INTERRUPT_RECONFIGURE
950~~~~~~~~~~~~~~~~~~~~~~~~
951
952An SP specifies the list of interrupts it owns through its partition manifest.
953This paravirtualized interface allows an SP to reconfigure a physical interrupt
954in runtime. It accepts three arguments, namely, interrupt ID, command and value.
955The command & value pair signify what change is being requested by the current
956Secure Partition for the given interrupt.
957
958SPMC returns 0 to indicate that the command was processed successfully or -1 if
959it failed to do so. At present, this interface only supports the following
960commands:
961
962 - ``INT_RECONFIGURE_TARGET_PE``
963 - Change the target CPU of the interrupt.
964 - Value represents linear CPU index in the range 0 to (MAX_CPUS - 1).
965
966 - ``INT_RECONFIGURE_SEC_STATE``
967 - Change the security state of the interrupt.
968 - Value must be either 0 (Non-secure) or 1 (Secure).
969
970 - ``INT_RECONFIGURE_ENABLE``
971 - Enable or disable the physical interrupt.
972 - Value must be either 0 (Disable) or 1 (Enable).
973
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +0200974SPMC-SPMD direct requests/responses
975-----------------------------------
976
977Implementation-defined FF-A IDs are allocated to the SPMC and SPMD.
978Using those IDs in source/destination fields of a direct request/response
979permits SPMD to SPMC communication and either way.
980
981- SPMC to SPMD direct request/response uses SMC conduit.
982- SPMD to SPMC direct request/response uses ERET conduit.
983
984This is used in particular to convey power management messages.
985
J-Alves5eafd222023-10-26 14:19:21 +0100986Notifications
987-------------
988
989The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous
990communication mechanism with non-blocking semantics. It allows for one FF-A
991endpoint to signal another for service provision, without hindering its current
992progress.
993
994Hafnium currently supports 64 notifications. The IDs of each notification define
995a position in a 64-bit bitmap.
996
997The signaling of notifications can interchangeably happen between NWd and SWd
998FF-A endpoints.
999
1000The SPMC is in charge of managing notifications from SPs to SPs, from SPs to
1001VMs, and from VMs to SPs. An hypervisor component would only manage
1002notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints
1003deployed in NWd, the Hypervisor or OS kernel must invoke the interface
1004FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A
1005endpoint in the NWd that supports it.
1006
1007A sender can signal notifications once the receiver has provided it with
1008permissions. Permissions are provided by invoking the interface
1009FFA_NOTIFICATION_BIND.
1010
1011Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth
1012they are considered to be in a pending sate. The receiver can retrieve its
1013pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment,
1014are considered to be handled.
1015
1016Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler
1017that is in charge of donating CPU cycles for notifications handling. The
1018FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about
1019which FF-A endpoints have pending notifications. The receiver scheduler is
1020called and informed by the FF-A driver, and it should allocate CPU cycles to the
1021receiver.
1022
1023There are two types of notifications supported:
1024
Olivier Deprezb8bd7d72023-10-27 16:14:13 +02001025- Global, which are targeted to an FF-A endpoint and can be handled within any
1026 of its execution contexts, as determined by the scheduler of the system.
J-Alves5eafd222023-10-26 14:19:21 +01001027- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a
1028 a specific execution context, as determined by the sender.
1029
1030The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give
1031permissions to the sender.
1032
1033Notification signaling resorts to two interrupts:
1034
1035- Schedule Receiver Interrupt: non-secure physical interrupt to be handled by
1036 the FF-A driver within the receiver scheduler. At initialization the SPMC
1037 donates an SGI ID chosen from the secure SGI IDs range and configures it as
1038 non-secure. The SPMC triggers this SGI on the currently running core when
1039 there are pending notifications, and the respective receivers need CPU cycles
1040 to handle them.
1041- Notifications Pending Interrupt: virtual interrupt to be handled by the
1042 receiver of the notification. Set when there are pending notifications for the
1043 given secure partition. The NPI is pended when the NWd relinquishes CPU cycles
1044 to an SP.
1045
1046The notifications receipt support is enabled in the partition FF-A manifest.
1047
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001048Memory Sharing
1049--------------
1050
J-Alvesd547d6d2024-05-14 14:59:54 +01001051The Hafnium implementation aligns with FF-A v1.2 ALP0 specification,
1052'FF-A Memory Management Protocol' supplement `[11]`_. Hafnium supports
1053the following ABIs:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001054
1055 - ``FFA_MEM_SHARE`` - for shared access between lender and borrower.
1056 - ``FFA_MEM_LEND`` - borrower to obtain exclusive access, though lender
1057 retains ownership of the memory.
1058 - ``FFA_MEM_DONATE`` - lender permanently relinquishes ownership of memory
1059 to the borrower.
1060
1061The ``FFA_MEM_RETRIEVE_REQ`` interface is for the borrower to request the
1062memory to be mapped into its address space: for S-EL1 partitions the SPM updates
1063their stage 2 translation regime; for S-EL0 partitions the SPM updates their
1064stage 1 translation regime. On a successful call, the SPMC responds back with
1065``FFA_MEM_RETRIEVE_RESP``.
1066
1067The ``FFA_MEM_RELINQUISH`` interface is for when the borrower is done with using
1068a memory region.
1069
1070The ``FFA_MEM_RECLAIM`` interface is for the owner of the memory to reestablish
1071its ownership and exclusive access to the memory shared.
1072
1073The memory transaction descriptors are transmitted via RX/TX buffers. In
1074situations where the size of the memory transaction descriptor exceeds the
1075size of the RX/TX buffers, Hafnium provides support for fragmented transmission
1076of the full transaction descriptor. The ``FFA_MEM_FRAG_RX`` and ``FFA_MEM_FRAG_TX``
1077interfaces are for receiving and transmitting the next fragment, respectively.
1078
1079If lender and borrower(s) are SPs, all memory sharing operations are supported.
1080
1081Hafnium also supports memory sharing operations between the normal world and the
1082secure world. If there is an SP involved, the SPMC allocates data to track the
1083state of the operation.
1084
J-Alvesda82a1a2023-10-17 11:45:49 +01001085An SP can not share, lend or donate memory to the NWd.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001086
J-Alvesd547d6d2024-05-14 14:59:54 +01001087The SPMC is also the designated allocator for the memory handle, when borrowers
1088include at least an SP. The SPMC doesn't support the hypervisor to be allocator
1089to the memory handle.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001090
1091Hafnium also supports memory lend and share targetting multiple borrowers.
1092This is the case for a lender SP to multiple SPs, and for a lender VM to
1093multiple endpoints (from both secure world and normal world). If there is
1094at least one borrower VM, the hypervisor is in charge of managing its
J-Alvesd547d6d2024-05-14 14:59:54 +01001095stage 2 translation on a successful memory retrieve. However, the hypervisor could
1096rely on the SPMC to keep track of the state of the operation, namely:
1097if all fragments to the memory descriptors have been sent, and if the retrievers
1098are still using the memory at any given moment. In this case, the hypervisor might
1099need to request the SPMC to obtain a description of the used memory regions.
1100For example, when handling an ``FFA_MEM_RECLAIM`` the hypervisor retrieve request
1101can be used to obtain that state information, do the necessary validations,
1102and update stage-2 memory translation of the lender.
1103Hafnium currently only supports one borrower from the NWd, in a multiple borrower
1104scenario as described. If there is only a single borrower VM, the SPMC will
1105return error to the lender on call to either share, lend or donate ABIs.
1106
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001107The semantics of ``FFA_MEM_DONATE`` implies ownership transmission,
1108which should target only one partition.
1109
1110The memory share interfaces are backwards compatible with memory transaction
Daniel Boulbyd5041122024-01-31 14:24:54 +00001111descriptors from FF-A v1.0. Starting from FF-A v1.1, with the introduction
1112of the `Endpoint memory access descriptor size` and
1113`Endpoint memory access descriptor access offset` fields (from Table 11.20 of the
1114FF-A v1.2 ALP0 specification), memory transaction descriptors are forward
1115compatible, so can be used internally by Hafnium as they are sent.
1116These fields must be valid for a memory access descriptor defined for a compatible
1117FF-A version to the SPMC FF-A version. For a transaction from an FF-A v1.0 endpoint
1118the memory transaction descriptor will be translated to an FF-A v1.1 descriptor for
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001119Hafnium's internal processing of the operation. If the FF-A version of a
1120borrower is v1.0, Hafnium provides FF-A v1.0 compliant memory transaction
1121descriptors on memory retrieve response.
1122
J-Alvesffc82062023-11-07 14:19:00 +00001123In the section :ref:`SPMC Configuration` there is a mention of non-secure memory
1124range, that limit the memory region nodes the SP can define. Whatever is left of
1125the memory region node carve-outs, the SPMC utilizes the memory to create a set of
1126page tables it associates with the NWd. The memory sharing operations incoming from
1127the NWd should refer to addresses belonging to these page tables. The intent
1128is for SPs not to be able to get access to regions they are not intended to access.
1129This requires special care from the system integrator to configure the memory ranges
1130correctly, such that any SP can't be given access and interfere with execution of
1131other components. More information in the :ref:`Threat Model`.
1132
Daniel Boulbydfc312e2024-05-14 17:10:01 +01001133Hafnium SPMC supports memory management transactions for device memory regions.
1134Currently this is limited to only the ``FFA_MEM_LEND`` interface and
1135to a single borrower. The device memory region used in the transaction must have
1136been decalared in the SPMC manifest as described above. Memory defined in a device
1137region node is given the attributes Device-nGnRnE, since this is the most restrictive
1138memory type the memory must be lent with these attrbutes as well.
1139
J-Alvesd547d6d2024-05-14 14:59:54 +01001140In |RME| enabled platforms, there is the ability to change the |PAS|
1141of a given memory region `[12]`_. The SPMC can leverage this feature to fulfill the
1142semantics of the ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE`` from the NWd into the SWd.
1143Currently, there is the implementation for the FVP platform to issue a
1144platform-specific SMC call to the EL3 monitor to change the PAS of the regions being
1145lent/donated. This shall guarantee the NWd can't tamper with the memory whilst
1146the SWd software expects exclusive access. For any other platform, the API under
1147the 'src/memory_protect' module can be redefined to leverage an equivalent platform
1148specific mechanism. For reference, check the `SPMC FVP build configuration`_.
1149
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001150PE MMU configuration
1151--------------------
1152
1153With secure virtualization enabled (``HCR_EL2.VM = 1``) and for S-EL1
1154partitions, two IPA spaces (secure and non-secure) are output from the
1155secure EL1&0 Stage-1 translation.
1156The EL1&0 Stage-2 translation hardware is fed by:
1157
1158- A secure IPA when the SP EL1&0 Stage-1 MMU is disabled.
1159- One of secure or non-secure IPA when the secure EL1&0 Stage-1 MMU is enabled.
1160
1161``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the
1162NS/S IPA translations. The following controls are set up:
1163``VSTCR_EL2.SW = 0`` , ``VSTCR_EL2.SA = 0``, ``VTCR_EL2.NSW = 0``,
1164``VTCR_EL2.NSA = 1``:
1165
1166- Stage-2 translations for the NS IPA space access the NS PA space.
1167- Stage-2 translation table walks for the NS IPA space are to the secure PA space.
1168
1169Secure and non-secure IPA regions (rooted to by ``VTTBR_EL2`` and ``VSTTBR_EL2``)
1170use the same set of Stage-2 page tables within a SP.
1171
1172The ``VTCR_EL2/VSTCR_EL2/VTTBR_EL2/VSTTBR_EL2`` virtual address space
1173configuration is made part of a vCPU context.
1174
1175For S-EL0 partitions with VHE enabled, a single secure EL2&0 Stage-1 translation
1176regime is used for both Hafnium and the partition.
1177
1178Schedule modes and SP Call chains
1179---------------------------------
1180
1181An SP execution context is said to be in SPMC scheduled mode if CPU cycles are
1182allocated to it by SPMC. Correspondingly, an SP execution context is said to be
1183in Normal world scheduled mode if CPU cycles are allocated by the normal world.
1184
1185A call chain represents all SPs in a sequence of invocations of a direct message
1186request. When execution on a PE is in the secure state, only a single call chain
1187that runs in the Normal World scheduled mode can exist. FF-A v1.1 spec allows
1188any number of call chains to run in the SPMC scheduled mode but the Hafnium
1189SPMC restricts the number of call chains in SPMC scheduled mode to only one for
1190keeping the implementation simple.
1191
1192Partition runtime models
1193------------------------
1194
1195The runtime model of an endpoint describes the transitions permitted for an
1196execution context between various states. These are the four partition runtime
1197models supported (refer to `[1]`_ section 7):
1198
1199 - RTM_FFA_RUN: runtime model presented to an execution context that is
1200 allocated CPU cycles through FFA_RUN interface.
1201 - RTM_FFA_DIR_REQ: runtime model presented to an execution context that is
Kathleen Capella6e3abcf2024-02-05 16:17:35 -05001202 allocated CPU cycles through FFA_MSG_SEND_DIRECT_REQ or FFA_MSG_SEND_DIRECT_REQ2
1203 interface.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001204 - RTM_SEC_INTERRUPT: runtime model presented to an execution context that is
1205 allocated CPU cycles by SPMC to handle a secure interrupt.
1206 - RTM_SP_INIT: runtime model presented to an execution context that is
1207 allocated CPU cycles by SPMC to initialize its state.
1208
1209If an endpoint execution context attempts to make an invalid transition or a
1210valid transition that could lead to a loop in the call chain, SPMC denies the
1211transition with the help of above runtime models.
1212
1213Interrupt management
1214--------------------
1215
1216GIC ownership
1217~~~~~~~~~~~~~
1218
1219The SPMC owns the GIC configuration. Secure and non-secure interrupts are
1220trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt
1221IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects
1222virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP.
1223
1224Abbreviations:
1225
1226 - NS-Int: A non-secure physical interrupt. It requires a switch to the normal
1227 world to be handled if it triggers while execution is in secure world.
1228 - Other S-Int: A secure physical interrupt targeted to an SP different from
1229 the one that is currently running.
1230 - Self S-Int: A secure physical interrupt targeted to the SP that is currently
1231 running.
1232
1233Non-secure interrupt handling
1234~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1235
1236This section documents the actions supported in SPMC in response to a non-secure
1237interrupt as per the guidance provided by FF-A v1.1 EAC0 specification.
1238An SP specifies one of the following actions in its partition manifest:
1239
1240 - Non-secure interrupt is signaled.
1241 - Non-secure interrupt is signaled after a managed exit.
1242 - Non-secure interrupt is queued.
1243
1244An SP execution context in a call chain could specify a less permissive action
1245than subsequent SP execution contexts in the same call chain. The less
1246permissive action takes precedence over the more permissive actions specified
1247by the subsequent execution contexts. Please refer to FF-A v1.1 EAC0 section
12488.3.1 for further explanation.
1249
1250Secure interrupt handling
1251~~~~~~~~~~~~~~~~~~~~~~~~~
1252
1253This section documents the support implemented for secure interrupt handling in
1254SPMC as per the guidance provided by FF-A v1.1 EAC0 specification.
1255The following assumptions are made about the system configuration:
1256
1257 - In the current implementation, S-EL1 SPs are expected to use the para
1258 virtualized ABIs for interrupt management rather than accessing the virtual
1259 GIC interface.
1260 - Unless explicitly stated otherwise, this support is applicable only for
1261 S-EL1 SPs managed by SPMC.
1262 - Secure interrupts are configured as G1S or G0 interrupts.
1263 - All physical interrupts are routed to SPMC when running a secure partition
1264 execution context.
1265 - All endpoints with multiple execution contexts have their contexts pinned
1266 to corresponding CPUs. Hence, a secure virtual interrupt cannot be signaled
1267 to a target vCPU that is currently running or blocked on a different
1268 physical CPU.
1269
1270A physical secure interrupt could trigger while CPU is executing in normal world
1271or secure world.
1272The action of SPMC for a secure interrupt depends on: the state of the target
1273execution context of the SP that is responsible for handling the interrupt;
1274whether the interrupt triggered while execution was in normal world or secure
1275world.
1276
1277Secure interrupt signaling mechanisms
1278~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1279
1280Signaling refers to the mechanisms used by SPMC to indicate to the SP execution
1281context that it has a pending virtual interrupt and to further run the SP
1282execution context, such that it can handle the virtual interrupt. SPMC uses
1283either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling
1284to S-EL1 SPs. When normal world execution is preempted by a secure interrupt,
1285the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC
1286running in S-EL2.
1287
1288+-----------+---------+---------------+---------------------------------------+
1289| SP State | Conduit | Interface and | Description |
1290| | | parameters | |
1291+-----------+---------+---------------+---------------------------------------+
1292| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending |
1293| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and |
1294| | | | resumes execution context of SP |
1295| | | | through ERET. |
1296+-----------+---------+---------------+---------------------------------------+
1297| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt |
1298| | vIRQ | | is pending. It pends vIRQ signal and |
1299| | | | resumes execution context of SP |
1300| | | | through ERET. |
1301+-----------+---------+---------------+---------------------------------------+
1302| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does |
1303| | | | not resume execution context of SP. |
1304+-----------+---------+---------------+---------------------------------------+
1305| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes|
1306| | vIRQ | | execution context of SP through ERET. |
1307+-----------+---------+---------------+---------------------------------------+
1308
1309Secure interrupt completion mechanisms
1310~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1311
1312A SP signals secure interrupt handling completion to the SPMC through the
1313following mechanisms:
1314
1315 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state.
1316 - ``FFA_RUN`` ABI if its was in BLOCKED state.
1317
1318This is a remnant of SPMC implementation based on the FF-A v1.0 specification.
1319In the current implementation, S-EL1 SPs use the para-virtualized HVC interface
1320implemented by SPMC to perform priority drop and interrupt deactivation (SPMC
1321configures EOImode = 0, i.e. priority drop and deactivation are done together).
1322The SPMC performs checks to deny the state transition upon invocation of
1323either FFA_MSG_WAIT or FFA_RUN interface if the SP didn't perform the
1324deactivation of the secure virtual interrupt.
1325
1326If the current SP execution context was preempted by a secure interrupt to be
1327handled by execution context of target SP, SPMC resumes current SP after signal
1328completion by target SP execution context.
1329
1330Actions for a secure interrupt triggered while execution is in normal world
1331~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1332
1333+-------------------+----------+-----------------------------------------------+
1334| State of target | Action | Description |
1335| execution context | | |
1336+-------------------+----------+-----------------------------------------------+
1337| WAITING | Signaled | This starts a new call chain in SPMC scheduled|
1338| | | mode. |
1339+-------------------+----------+-----------------------------------------------+
1340| PREEMPTED | Queued | The target execution must have been preempted |
1341| | | by a non-secure interrupt. SPMC queues the |
1342| | | secure virtual interrupt now. It is signaled |
1343| | | when the target execution context next enters |
1344| | | the RUNNING state. |
1345+-------------------+----------+-----------------------------------------------+
1346| BLOCKED, RUNNING | NA | The target execution context is blocked or |
1347| | | running on a different CPU. This is not |
1348| | | supported by current SPMC implementation and |
1349| | | execution hits panic. |
1350+-------------------+----------+-----------------------------------------------+
1351
1352If normal world execution was preempted by a secure interrupt, SPMC uses
1353FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling
1354and further returns execution to normal world.
1355
1356The following figure describes interrupt handling flow when a secure interrupt
1357triggers while execution is in normal world:
1358
1359.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png
1360
1361A brief description of the events:
1362
1363 - 1) Secure interrupt triggers while normal world is running.
1364 - 2) FIQ gets trapped to EL3.
1365 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI.
1366 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends
1367 vIRQ).
1368 - 5) Assuming SP1 vCPU is in WAITING state, SPMC signals virtual interrupt
1369 using FFA_INTERRUPT with interrupt id as an argument and resumes the SP1
1370 vCPU using ERET in SPMC scheduled mode.
1371 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual
1372 interrupt is not masked i.e., PSTATE.I = 0
1373 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized
1374 HVC call. SPMC clears the pending virtual interrupt state management
1375 and returns the pending virtual interrupt id.
1376 - 8) SP1 services the virtual interrupt and invokes the paravirtualized
1377 de-activation HVC call. SPMC de-activates the physical interrupt,
1378 clears the fields tracking the secure interrupt and resumes SP1 vCPU.
1379 - 9) SP1 performs secure interrupt completion through FFA_MSG_WAIT ABI.
1380 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME.
1381 - 11) EL3 resumes normal world execution.
1382
1383Actions for a secure interrupt triggered while execution is in secure world
1384~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1385
1386+-------------------+----------+------------------------------------------------+
1387| State of target | Action | Description |
1388| execution context | | |
1389+-------------------+----------+------------------------------------------------+
1390| WAITING | Signaled | This starts a new call chain in SPMC scheduled |
1391| | | mode. |
1392+-------------------+----------+------------------------------------------------+
1393| PREEMPTED by Self | Signaled | The target execution context reenters the |
1394| S-Int | | RUNNING state to handle the secure virtual |
1395| | | interrupt. |
1396+-------------------+----------+------------------------------------------------+
1397| PREEMPTED by | Queued | SPMC queues the secure virtual interrupt now. |
1398| NS-Int | | It is signaled when the target execution |
1399| | | context next enters the RUNNING state. |
1400+-------------------+----------+------------------------------------------------+
1401| BLOCKED | Signaled | Both preempted and target execution contexts |
1402| | | must have been part of the Normal world |
1403| | | scheduled call chain. Refer scenario 1 of |
1404| | | Table 8.4 in the FF-A v1.1 EAC0 spec. |
1405+-------------------+----------+------------------------------------------------+
1406| RUNNING | NA | The target execution context is running on a |
1407| | | different CPU. This scenario is not supported |
1408| | | by current SPMC implementation and execution |
1409| | | hits panic. |
1410+-------------------+----------+------------------------------------------------+
1411
1412The following figure describes interrupt handling flow when a secure interrupt
1413triggers while execution is in secure world. We assume OS kernel sends a direct
1414request message to SP1. Further, SP1 sends a direct request message to SP2. SP1
1415enters BLOCKED state and SPMC resumes SP2.
1416
1417.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png
1418
1419A brief description of the events:
1420
1421 - 1) Secure interrupt triggers while SP2 is running.
1422 - 2) SP2 gets preempted and execution traps to SPMC as IRQ.
1423 - 3) SPMC finds the target vCPU of secure partition responsible for handling
1424 this secure interrupt. In this scenario, it is SP1.
1425 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface.
1426 SPMC further resumes SP1 through ERET conduit. Note that SP1 remains in
1427 Normal world schedule mode.
1428 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual
1429 interrupt is not masked i.e., PSTATE.I = 0
1430 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized
1431 HVC call. SPMC clears the pending virtual interrupt state management
1432 and returns the pending virtual interrupt id.
1433 - 8) SP1 services the virtual interrupt and invokes the paravirtualized
1434 de-activation HVC call. SPMC de-activates the physical interrupt and
1435 clears the fields tracking the secure interrupt and resumes SP1 vCPU.
1436 - 9) Since SP1 direct request completed with FFA_INTERRUPT, it resumes the
1437 direct request to SP2 by invoking FFA_RUN.
1438 - 9) SPMC resumes the pre-empted vCPU of SP2.
1439
1440EL3 interrupt handling
1441~~~~~~~~~~~~~~~~~~~~~~
1442
1443In GICv3 based systems, EL3 interrupts are configured as Group0 secure
1444interrupts. Execution traps to SPMC when a Group0 interrupt triggers while an
1445SP is running. Further, SPMC running at S-EL2 uses FFA_EL3_INTR_HANDLE ABI to
1446request EL3 platform firmware to handle a pending Group0 interrupt.
1447Similarly, SPMD registers a handler with interrupt management framework to
1448delegate handling of Group0 interrupt to the platform if the interrupt triggers
1449in normal world.
1450
1451 - Platform hook
1452
1453 - plat_spmd_handle_group0_interrupt
1454
1455 SPMD provides platform hook to handle Group0 secure interrupts. In the
1456 current design, SPMD expects the platform not to delegate handling to the
1457 NWd (such as through SDEI) while processing Group0 interrupts.
1458
1459Power management
1460----------------
1461
1462In platforms with or without secure virtualization:
1463
1464- The NWd owns the platform PM policy.
1465- The Hypervisor or OS kernel is the component initiating PSCI service calls.
1466- The EL3 PSCI library is in charge of the PM coordination and control
1467 (eventually writing to platform registers).
1468- While coordinating PM events, the PSCI library calls backs into the Secure
1469 Payload Dispatcher for events the latter has statically registered to.
1470
1471When using the SPMD as a Secure Payload Dispatcher:
1472
1473- A power management event is relayed through the SPD hook to the SPMC.
1474- In the current implementation only cpu on (svc_on_finish) and cpu off
1475 (svc_off) hooks are registered.
1476- The behavior for the cpu on event is described in `Secondary cores boot-up`_.
1477 The SPMC is entered through its secondary physical core entry point.
1478- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The PM event is
1479 signaled to the SPMC through a power management framework message.
1480 It consists in a SPMD-to-SPMC direct request/response (`SPMC-SPMD direct
1481 requests/responses`_) conveying the event details and SPMC response.
1482 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and
1483 updates its internal state to reflect the physical core is being turned off.
1484 In the current implementation no SP is resumed as a consequence. This behavior
1485 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux
1486 userspace.
1487
1488Arm architecture extensions for security hardening
J-Alves5eafd222023-10-26 14:19:21 +01001489--------------------------------------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001490
1491Hafnium supports the following architecture extensions for security hardening:
1492
1493- Pointer authentication (FEAT_PAuth): the extension permits detection of forged
1494 pointers used by ROP type of attacks through the signing of the pointer
1495 value. Hafnium is built with the compiler branch protection option to permit
1496 generation of a pointer authentication code for return addresses (pointer
1497 authentication for instructions). The APIA key is used while Hafnium runs.
1498 A random key is generated at boot time and restored upon entry into Hafnium
1499 at run-time. APIA and other keys (APIB, APDA, APDB, APGA) are saved/restored
1500 in vCPU contexts permitting to enable pointer authentication in VMs/SPs.
1501- Branch Target Identification (FEAT_BTI): the extension permits detection of
1502 unexpected indirect branches used by JOP type of attacks. Hafnium is built
1503 with the compiler branch protection option, inserting land pads at function
1504 prologues that are reached by indirect branch instructions (BR/BLR).
1505 Hafnium code pages are marked as guarded in the EL2 Stage-1 MMU descriptors
1506 such that an indirect branch must always target a landpad. A fault is
1507 triggered otherwise. VMs/SPs can (independently) mark their code pages as
1508 guarded in the EL1&0 Stage-1 translation regime.
1509- Memory Tagging Extension (FEAT_MTE): the option permits detection of out of
1510 bound memory array accesses or re-use of an already freed memory region.
1511 Hafnium enables the compiler option permitting to leverage MTE stack tagging
1512 applied to core stacks. Core stacks are marked as normal tagged memory in the
1513 EL2 Stage-1 translation regime. A synchronous data abort is generated upon tag
1514 check failure on load/stores. A random seed is generated at boot time and
1515 restored upon entry into Hafnium. MTE system registers are saved/restored in
1516 vCPU contexts permitting MTE usage from VMs/SPs.
J-Alvesd547d6d2024-05-14 14:59:54 +01001517- Realm Management Extension (FEAT_RME): can be deployed in platforms that leverage
1518 RME for physical address isolation. The SPMC is capable of recovering from a
1519 Granule Protection Fault, if inadvertently accessing a region with the wrong security
1520 state setting. Also, the ability to change dynamically the physical address space of
1521 a region, can be used to enhance the handling of ``FFA_MEM_LEND`` and ``FFA_MEM_DONATE``.
1522 More details in the section about `Memory Sharing`_.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001523
1524SMMUv3 support in Hafnium
J-Alves5eafd222023-10-26 14:19:21 +01001525-------------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001526
1527An SMMU is analogous to an MMU in a CPU. It performs address translations for
1528Direct Memory Access (DMA) requests from system I/O devices.
1529The responsibilities of an SMMU include:
1530
1531- Translation: Incoming DMA requests are translated from bus address space to
1532 system physical address space using translation tables compliant to
1533 Armv8/Armv7 VMSA descriptor format.
1534- Protection: An I/O device can be prohibited from read, write access to a
1535 memory region or allowed.
1536- Isolation: Traffic from each individial device can be independently managed.
1537 The devices are differentiated from each other using unique translation
1538 tables.
1539
1540The following diagram illustrates a typical SMMU IP integrated in a SoC with
1541several I/O devices along with Interconnect and Memory system.
1542
1543.. image:: ../resources/diagrams/MMU-600.png
1544
1545SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides
1546support for SMMUv3 driver in both normal and secure world. A brief introduction
1547of SMMUv3 functionality and the corresponding software support in Hafnium is
1548provided here.
1549
1550SMMUv3 features
J-Alves5eafd222023-10-26 14:19:21 +01001551~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001552
1553- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2)
1554 translation support. It can either bypass or abort incoming translations as
1555 well.
1556- Traffic (memory transactions) from each upstream I/O peripheral device,
1557 referred to as Stream, can be independently managed using a combination of
1558 several memory based configuration structures. This allows the SMMUv3 to
1559 support a large number of streams with each stream assigned to a unique
1560 translation context.
1561- Support for Armv8.1 VMSA where the SMMU shares the translation tables with
1562 a Processing Element. AArch32(LPAE) and AArch64 translation table format
1563 are supported by SMMUv3.
1564- SMMUv3 offers non-secure stream support with secure stream support being
1565 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU
1566 instance for secure and non-secure stream support.
1567- It also supports sub-streams to differentiate traffic from a virtualized
1568 peripheral associated with a VM/SP.
1569- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A
1570 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2
1571 for providing Secure Stage2 translation support to upstream peripheral
1572 devices.
1573
1574SMMUv3 Programming Interfaces
J-Alves5eafd222023-10-26 14:19:21 +01001575~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001576
1577SMMUv3 has three software interfaces that are used by the Hafnium driver to
1578configure the behaviour of SMMUv3 and manage the streams.
1579
1580- Memory based data strutures that provide unique translation context for
1581 each stream.
1582- Memory based circular buffers for command queue and event queue.
1583- A large number of SMMU configuration registers that are memory mapped during
1584 boot time by Hafnium driver. Except a few registers, all configuration
1585 registers have independent secure and non-secure versions to configure the
1586 behaviour of SMMUv3 for translation of secure and non-secure streams
1587 respectively.
1588
1589Peripheral device manifest
J-Alves5eafd222023-10-26 14:19:21 +01001590~~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001591
1592Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices.
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001593These DMA devices are dependent on PE endpoint to initiate and receive memory
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001594management transactions on their behalf. The acccess to the MMIO regions of
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001595any such device is assigned to the endpoint during boot.
1596The :ref:`device node<device_region_node>` of the corresponding partition
1597manifest must specify these additional properties for each peripheral device in
1598the system:
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001599
1600- smmu-id: This field helps to identify the SMMU instance that this device is
1601 upstream of.
1602- stream-ids: List of stream IDs assigned to this device.
1603
1604.. code:: shell
1605
1606 smmuv3-testengine {
1607 base-address = <0x00000000 0x2bfe0000>;
1608 pages-count = <32>;
1609 attributes = <0x3>;
1610 smmu-id = <0>;
1611 stream-ids = <0x0 0x1>;
1612 interrupts = <0x2 0x3>, <0x4 0x5>;
1613 exclusive-access;
1614 };
1615
Madhukar Pappireddy555f8882023-10-16 13:45:29 -05001616DMA isolation
1617-------------
1618
1619Hafnium, with help of SMMUv3 driver, enables the support for static DMA
1620isolation. The DMA device is explicitly granted access to a specific
1621memory region only if the partition requests it by declaring the following
1622properties of the DMA device in the :ref:`memory region node<memory_region_node>`
1623of the partition manifest:
1624
1625- smmu-id
1626- stream-ids
1627- stream-ids-access-permissions
1628
1629SMMUv3 driver uses a unqiue set of stage 2 translations for the DMA device
1630rather than those used on behalf of the PE endpoint. This ensures that the DMA
1631device has a limited visibility of the physical address space.
1632
1633.. code:: shell
1634
1635 smmuv3-memcpy-src {
1636 description = "smmuv3-memcpy-source";
1637 pages-count = <4>;
1638 base-address = <0x00000000 0x7400000>;
1639 attributes = <0x3>; /* read-write */
1640 smmu-id = <0>;
1641 stream-ids = <0x0 0x1>;
1642 stream-ids-access-permissions = <0x3 0x3>;
1643 };
1644
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001645SMMUv3 driver limitations
J-Alves5eafd222023-10-26 14:19:21 +01001646~~~~~~~~~~~~~~~~~~~~~~~~~
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001647
1648The primary design goal for the Hafnium SMMU driver is to support secure
1649streams.
1650
1651- Currently, the driver only supports Stage2 translations. No support for
1652 Stage1 or nested translations.
1653- Supports only AArch64 translation format.
1654- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS,
1655 Fault handling, Performance Monitor Extensions, Event Handling, MPAM.
1656- No support for independent peripheral devices.
1657
1658S-EL0 Partition support
J-Alves5eafd222023-10-26 14:19:21 +01001659-----------------------
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001660The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using
1661FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world
1662with ARMv8.4 and FEAT_SEL2).
1663
1664S-EL0 partitions are useful for simple partitions that don't require full
1665Trusted OS functionality. It is also useful to reduce jitter and cycle
1666stealing from normal world since they are more lightweight than VMs.
1667
1668S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by
1669the SPMC. They are differentiated primarily by the 'exception-level' property
1670and the 'execution-ctx-count' property in the SP manifest. They are host apps
1671under the single EL2&0 Stage-1 translation regime controlled by the SPMC and
1672call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions
1673can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions
1674for memory regions.
1675
1676S-EL0 partitions are required by the FF-A specification to be UP endpoints,
1677capable of migrating, and the SPMC enforces this requirement. The SPMC allows
1678a S-EL0 partition to accept a direct message from secure world and normal world,
1679and generate direct responses to them.
1680All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not supported.
1681
Olivier Deprezb8bd7d72023-10-27 16:14:13 +02001682Interrupt handling, Memory sharing, indirect messaging, and notifications features
1683in context of S-EL0 partitions are supported.
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001684
1685References
1686==========
1687
J-Alves5eafd222023-10-26 14:19:21 +01001688.. _TF-A project: https://trustedfirmware-a.readthedocs.io/en/latest/
1689
J-Alvesd547d6d2024-05-14 14:59:54 +01001690.. _SPMC FVP build configuration: https://github.com/TF-Hafnium/hafnium-project-reference/blob/main/BUILD.gn#L143
1691
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001692.. _[1]:
1693
1694[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__
1695
1696.. _[2]:
1697
1698[2] `Secure Partition Manager using MM interface <https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partition-manager-mm.html>`__
1699
1700.. _[3]:
1701
1702[3] `Trusted Boot Board Requirements
1703Client <https://developer.arm.com/documentation/den0006/d/>`__
1704
1705.. _[4]:
1706
1707[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45
1708
1709.. _[5]:
1710
1711[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts
1712
1713.. _[6]:
1714
1715[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html
1716
1717.. _[7]:
1718
1719[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts
1720
1721.. _[8]:
1722
1723[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/
1724
1725.. _[9]:
1726
1727[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot
1728
J-Alvesd8094162023-10-26 12:44:33 +01001729.. _[10]:
1730
1731[10] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/build-options.html#
1732
J-Alvesd547d6d2024-05-14 14:59:54 +01001733 .. _[11]:
1734
1735[11] https://developer.arm.com/documentation/den0140/a
1736
1737 .. _[12]:
1738
1739[12] https://developer.arm.com/documentation/den0129/latest/
1740
Olivier Deprezcbf7d5b2023-05-22 12:12:24 +02001741--------------
1742
1743*Copyright (c) 2020-2023, Arm Limited and Contributors. All rights reserved.*