blob: 4fa823a7d570f6783b55fd028b4caf7ac4cbd394 [file] [log] [blame]
Paul Beesley5c928952019-10-24 11:57:00 +00001Change Log & Release Notes
2==========================
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02003
4Please note that the Trusted Firmware-A Tests version follows the Trusted
5Firmware-A version for simplicity. At any point in time, TF-A Tests version
6`x.y` aims at testing TF-A version `x.y`. Different versions of TF-A and TF-A
7Tests are not guaranteed to be compatible. This also means that a version
8upgrade on the TF-A-Tests side might not necessarily introduce any new feature.
9
Juan Pablo Conde0a0f6742023-05-13 00:20:15 -050010Version 2.9
11-----------
12
13New features
14^^^^^^^^^^^^
15
16- More tests are made available in this release to help validate the
17 functionalities in the following areas:
18
19 - FF-A Features
20 - Realm Management Extension
21 - New Architecture Specific features related to v8.8
22 - New platform ports
23
24TFTF
25~~~~
26
27- FF-A testing:
28
29 - Reordered logs in the memory sharing tests.
30 - Memory share bumped to v1.1 EAC0.
31 - Updated tests for FFA_FEATURES(FFA_MEM_RETRIEVE_REQ).
32 - Fixed issues with RXTX buffer unmapping and dependencies on tests.
33 - Added check for execution state property of partitions.
34
35- New tests:
36
37 - Tests for Errata management firmware interface.
38 - Ensure FPU state registers context is preserved in RL/SE/NS.
39 - Modified FEAT_HCX test to also check for HCRX_EL2 init value.
40 - Added basic SME2 tests.
41 - PSCI tests for OS-initiated mode.
42 - Added "nop" test to be used in conjunction with TFX.
43 - Introduced capability to generate Sync External Aborts (EA) in TFTF.
44 - New test to generate an SError.
45 - Tests to check whether the PMU is functional and if the state is
46 preserved when switching worlds. PMEVTYPER.evtCount width extended.
47 - Added support for more SPE versions.
48
49- Platforms:
50
51 - RD-N2-Cfg3:
52
53 - Added TFTF support.
54
55- Miscellaneous:
56
57 - SIMD/FPU save/restore routine moved to common lib.
58 - Updated toolchain requirements documentation.
59 - Update SME/Mortlach tests.
60 - Unified Firmware First handling of lower EL EA.
61 - Moved trusted wdog API to spm_common.
62 - Added the ability to skip tests for AArch32.
63 - Added config file to allow doc defaults be changed.
64 - Modified tests for FEAT_RNG_TRAP.
65 - Moved 'Stress test timer framework' to a new test suite
66 'tests-timer-stress'.
67 - Support for new binutils versions.
68 - Removed deprecated SPM libs and test code.
69
70
71Realm Management Extension (RME)
72~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
73
74 - Added helper macro for RME tests.
75 - Test Secure interrupt can preempt Realm EL1.
76 - Added PMU Realm tests.
77 - Added BP_OPTION to REALM_CFLAGS to allow build realm payload with
78 BTI/Pauth support.
79 - Fixed build issues introduced by the TFTF Realm extension
80 enhancement tests.
81 - Test case return codes updated according to RMM Bet0 specification.
82 - Fixed build problem related to rmi_rec_enter verbose log.
83 - Added randomization of SMC RMI commands parameters and checking of
84 X4-X7 return values as per SMCCC v1.2.
85
86Cactus (Secure-EL1 test partition)
87~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
88
89 - Use of FFA_CONSOLE_LOG for debug logs.
90 - Test for consecutive same memory donation to other partitions.
91 - Now validating NWd can't share forbidden addresses.
92 - Support for registering irq handlers.
93 - Fixed attributes for NS memory region.
94 - Removal of memory regions not page-aligned.
95 - Added check for core linear id matching id passed by SPMC.
96
97Issues resolved since last release
98^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
99
100 - Build issue for older toolchains and other failures resolved.
101 - Dropped invalid access test from CI.
102 - Now checking that the PMU is supported before using any of it.
103 - Use of write instead of read to generate an SError to avoid sync
104 exceptions instead.
105 - Fixed broken link to TRNG_FW documentation.
106 - INIT_TFTF_MAILBOX() is called first for the invalid access test.
107
Juan Pablo Conde47e64d42022-11-16 18:33:40 -0500108Version 2.8
109-----------
110
111New features
112^^^^^^^^^^^^
113- More tests are made available in this release to help validate the
114 functionalities in the following areas:
115
116 - FF-A Features
117 - Realm Management Extension
118 - New Architecture Specific features related to v8.8
119 - New platform ports
120
121TFTF
122~~~~
123
124- FF-A testing:
125
126 - UUID included in partition information descriptors.
127 - Checks for size of partition information descriptors.
128 - Renamed FFA_MSG_RUN ABI function to FFA_RUN and allowed it to return from
129 Waiting state.
130 - Made ffa_tests available for Ivy.
131 - Updated verbose message log structure.
132 - Prevented generate_json.sh from being called more than once by requiring
133 a list of partitions to be supplied.
134 - Added a temporary workaround for unexpected affinity info state to prevent
135 a system panic.
136 - Added test to exercise FFA_CONSOLE_LOG ABI.
137
138 - FF-A v1.1 Secure interrupts
139
140 - Added managed exit to first and second SP in call chain.
141 - Added test to exercise managed exit by two SPs in a call chain.
142 - Added tests to exercise NS interrupt being queued and signaled to SP.
143
144- New tests:
145
146 - Tests for SVE operations in Normal World and discover SVE vector length.
147 - Added cleanup TRNG service tests.
148 - Added test for SMCCC_ARCH_WORKAROUND_3.
149 - Updated PAuth helpers to support QARMA3 algorithm.
150 - Added tests for RNG_TRAP.
151
152- Platforms:
153
154 - SGI:
155
156 - Introduced platform variant build option.
157 - Re-organized header files.
158 - Migrated to secure uart port for routing tftf logs.
159
160 - N1SDP:
161
162 - Added TFTF support for N1SDP.
163
164 - RD-N2:
165
166 - Added TFTF support for RD-N2.
167
168 - RD-N2-Cfg1:
169
170 - Added TFTF support for RD-N2-Cfg1.
171
172 - RD-V1:
173
174 - Added TFTF support for RD-V1.
175
176- Miscellaneous:
177
178 - Added a missing ISB instruction in SME test.
179 - Refactor to make some helper functions re-usable.
180 - Updated build command to clean EL3 payload image.
181 - Move renaming of the primary dts file for ivy partitions.
182 - Added check that verifies if a platform supports el3_payload before
183 building it.
184 - Updated memory share test to meet Hafnium specification.
185 - Updated toolchain requirements documentation.
186
187
188Realm Management Extension (RME)
189~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
190
191 - Added Realm payload management capabilities to TFTF to act as a NS Host.
192 - Added test to verify that RMM and SPM can co-exist and work properly.
193 - Added function to reset delegated buffers to non-delegated state.
194 - Re-used existing wait_for_non_lead_cpus() function helper.
195 - Refactored RMI FID macros to simplify usage.
196 - Added userguide for realm payload testing.
197
198Cactus (Secure-EL1 test partition)
199~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
200
201 - Corrected some tests message types from ERROR to VERBOSE.
202 - Increased the cactus number of xlat to allow the use of 48b PA size for
203 memory sharing between SPs.
204 - Introduced a new direct request message command to resume after managed
205 exit.
206 - Skip enabling virtual maintenance interrupts explicitly.
207 - Allowed sender to resume interrupted target vCPU.
208 - Added support for handling managed exit through vIRQ.
209 - Added support for discovering interrupt IDs of managed exit signals.
210 - Specified action in response to NS interrupt in manifest.
211
212Ivy (Secure-EL0 test partition)
213~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
214
215 - Allowed testing using VHE.
216 - Allowed Ivy partitions to use ffa_helpers functions.
217 - Requirement of common name for Ivy partitions for consistency.
218 - Specified action in response to NS interrupt in manifest.
219
220Issues resolved since last release
221^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
222
223 - Fixed SME header guard name.
224 - Fixed response for incorrect direct message request for FF-A.
225
Jayanth Dodderi Chidanand090491b2022-05-03 15:05:11 +0100226Version 2.7
227-----------
228
229New features
230^^^^^^^^^^^^
231- More tests are made available in this release to help validate the
232 functionalities in the following areas:
233
234 - FF-A Features
235 - New Architecture Specific features related to v8.7
236 - New platform port
237
238TFTF
239~~~~
240
241- FF-A testing:
242
243 - FF-A partition information structure is updated to include UUIDs.
244 - Memory Management helper functions are refactored to fetch the details
245 of smc call failures in tftf and cactus.
246 - Added test to validate memory sharing operations from SP to NS-endpoint
247 are denied by SPMC.
248 - Added test to ensure an endpoint that sets its version to v1.0 receives
249 v1.0 partition information descriptors as defined in v1.0 FF-A
250 specification.
251 - Added test to validate that memory is cleared on memory sharing operations
252 between normal world and secure world.
253
254 - FF-A v1.1 Secure interrupts
255
256 - Added support to enhance the secure interrupt handling test.
257 - Support for registering and unregistering custom handler that is
258 invoked by SP at the tail end of the virtual interrupt processing.
259 - Added support for querying the ID of the last serviced virtual interrupt.
260
261- New tests:
262
263 - Added test to validate that realm region access is being prevented from
264 normal world.
265 - Added test to validate that secure region access is being prevented from
266 normal world.
267 - Added test to validate that secure region access is being prevented from
268 realm world.
269 - Added test to validate that root region access is being prevented from
270 realm world.
271 - Added a test for v8.7 Advanced floating-point behavior (FEAT_AFP).
272 - Added a SPE test that reads static profiling system registers
273 of available SPE version i.e. FEAT_SPE/FEAT_SPEv1p1/FEAT_SPEv1p2.
274 - Added a test to validate functionality of WFET and WFIT instructions
275 introduced by v8.7 FEAT_WFxT.
276 - Added basic SME tests to ensure feature enablement by EL3 is proper for
277 its usage at lower non-secure ELs.
278 - Added test to check Data Independent timing (DIT) field of PSTATE is
279 retained on exception.
280 - Added test to ensure that EL3 has properly enabled access to FEAT_BRBE
281 from non-secure ELs.
282
283- Platforms:
284
285 - Add initial platform support for corstone1000.
286
287 - TC:
288
289 - Support for notification in tertiary SP manifest.
290
291 - FVP:
292
293 - Support to provide test memory addresses to validate the invalid
294 memory access test from tftf(ns-el2).
295
296- Miscellaneous:
297
298 - Added support to configure the physical/virtual address space for FVP.
299 - Added common header file for defining macros with size to support all the
300 platforms.
301 - Introduced handler for synchronous exceptions (AArch64).
302 - Added macros to extract the ISS portion of an ELx ESR exception syndrome
303 register.
304 - Support to dynamically map/unmap test region to validate invalid memory
305 access tests.
306 - Added support to receive boot information through secure partitions,
307 according to the FF-A v1.1 EAC0 specification.
308 - Added an helper API function from SPM test suite to initialize FFA-mailbox
309 and enable FF-A based message with SP.
310 - Updated the build string to display the rc-tagged version.
311
312Cactus (Secure-EL1 test partition)
313~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314
315 - Added test for nonsecure memory sharing between Secure Partitions(SPs).
316 - Added test to validate that a realm region cannot be accessed from secure
317 world.
318 - Added test to permit checking a root region cannot be accessed from secure
319 world.
320 - Extended the test command CACTUS_MEM_SEND_CMD to add support for memory
321 sharing flags.
322 - Added support to save the state of general purpose registers x0-x4 at the
323 entry to cold boot and restore them before jumping to entrypoint of cactus.
324
325Issues resolved since last release
326^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
327
328 - Fixed a bug to align RMI FIDs with SMCCC.
329 - Fixed encoding of vCPU and receiver IDs in the FFA_NOTIFICATION_GET
330 interface to comply with the FF-A v1.1 beta0 specification.
331 - Fixed memory retrieve request attributes by enforcing them to be inner
332 shareable rather than outer.
333 - Fixed static memory mapping of EL3 in EL2.
334 - Fixed a spurious error log message with memory share test.
335 - Aligning RMI FIDs with SMCCC.
336 - Fixed PSCI system suspend test suite execution in a four world system.
337 - Configured the build system to use DWARF 4 standard for debug builds with
338 ArmDS.
339 - Introduced macro IRQ_TWDOG_INTID for the Tegra210, Tegra186 and Tegra194
340 platforms to fix the compilation failures.
341
Madhukar Pappireddyda69f822021-11-09 16:05:54 -0600342Version 2.6
343-----------
344
345New features
346^^^^^^^^^^^^
347- More tests are made available in this release to help validate the
348 functionalities in the following areas:
349
350 - Firmware Framework for Arm A-profile(FF-A)
351 - Realm Management Extensions(RME)
352 - Embedded Trace Extension and Trace Buffer Extension (ETE and TRBE)
353
354TFTF
355~~~~
356
357- FF-A testing:
358
359 - Update FF-A version to v1.1
360 - Added helpers for SPM tests to check partition info of SPs from normal
361 world.
362 - Added tests to check for ffa_features supported.
363 - Added test for FFA_RXTX_UNMAP ABI.
364 - Added test for FFA_SPM_ID_GET.
365 - FF-A v1.1 Notifications
366
367 - Added test for notifications bitmap create and destroy ABIs.
368 - Added test for notifications set and get ABIs.
369 - Added test for notification INFO_GET ABI.
370 - Added test to check notifications pending interrupt is injected into
371 and handled by the expected vCPU in a MP setup.
372 - Added test for signaling from MP SP to UP SP.
373 - Added test to check notifications interrupt IDs retrieved with
374 FFA_FEATURES ABI.
375 - Added test to check functionality of notifications scheduled receiver
376 interrupt.
377
378 - FF-A v1.1 Secure interrupts
379
380 - Added support for handling secure interrupts in Cactus SP.
381 - Added several tests to exercise secure interrupt handling while SP
382 is in WAITING/RUNNING/BLOCKED state.
383
384- New tests:
385
386 - Enabled SVE tests
387 - Added test for trace system registers access.
388 - Added test for trace filter control registers access.
389 - Added test for trace buffer control registers access.
390 - Added test to check PSTATE in SDEI handler.
391 - Added test to check if HCRX_EL2 is accessible.
392
393- Platforms:
394
395 - TC0:
396
397 - Support for direct messaging with managed exit.
398 - Support for building S-EL0 Ivy partition.
399
400 - FVP:
401
402 - Update Cactus secure partitions to indicate Managed exit support.
403
404- Miscellaneous
405
406 - Added random seed generation capability and ability to specify build
407 parameters for SMC Fuzzer tool.
408
409Cactus (Secure-EL1 test partition)
410~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
411
412 - Added helper for Cactus SP sleep.
413 - Added test commands to request use of notifications interfaces.
414 - Added several commands that generate direct message requests to assist in
415 testing secure interrupt handling and notifications features in FF-A v1.1
416 - Added support for SP805 Trusted Watchdog module.
417
418Ivy (Secure-EL1 test partition)
419~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
420
421 - Add shim layer to Ivy partition and enable PIE.
422 - Define Ivy partition manifest and use FF-A for message handling.
423 - Prepare S-EL1/0 enviroment for enabling S-EL0 application.
424
425Realm Management Extension(RME)
426~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
427
428 - Added tests to run RMI and SPM on multiple CPUs concurrently.
429 - Added tests for multi CPU delegation and fail conditions.
430 - Added tests to query RMI version on multiple CPUs.
431
432Issues resolved since last release
433^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
434
435 - Fixed Ivy partition start address for TC0.
436 - Fixed SP manifests to use little endian format UUID.
437 - Fixed a bug in memory sharing test for Cactus SP.
438 - Invalidate data cache for NS_BL1U and NS_BL2U images.
439 - Fixed attributes to Read-Write only for memory regions described in partition
440 manifests.
441
Bipin Ravi7afea032021-05-06 14:32:52 -0500442Version 2.5
443-----------
444
445New features
446^^^^^^^^^^^^
447- More tests are made available in this release to help validate the
448 functionalities in the following areas:
449 - True Random Number Generator (TRNG) test scenarios.
450 - Multicore / Power State Controller Interface (PSCI) tests.
451 - v8.6 Activity Monitors Unit (AMU) enhancements test scenarios.
452 - Secure Partition Manager (SPM) / Firmware Framework (FF-A) v1.0 testing.
453 - Interrupt Handling between Non-secure and Secure world.
454 - Direct messages and memory sharing between Secure Partitions(SP).
455 - Many tests to exercise FF-A v1.0 ABIs.
456 - SPM saving/restoring the NS SIMD context enabling a normal world FF-A
457 endpoint (TFTF) and a secure partition to use SIMD vectors and
458 instructions independently.
459
460TFTF
461~~~~
462
463- SPM / FF-A v1.0 testing.
464 - Refactor FF-A memory sharing tests
465 - Created helper functions to initialize ffa_memory_region and to send
466 the respective memory region to the SP, making it possible to reuse
467 the logic in SP-to-SP memory share tests.
468 - Added comments to document relevant aspects about memory sharing.
469
470 - Trigger direct messaging between SPs.
471 - Use cactus command 'CACTUS_REQ_ECHO_SEND_CMD' to make cactus SPs
472 communicate with each other using direct message interfaces.
473
474 - Added helpers for SPM tests.
475 - Checking SPMC has expected FFA_VERSION.
476 - Checking that expected FF-A endpoints are deployed in the system.
477 - Getting global TFTF mailbox.
478
479- Replace '.inst' AArch64 machine directives with CPU Memory Tagging Extension
480 instructions in 'test_mte_instructions' function.
481
482- Add build option for Arm Feature Modifiers.
483 - This patch adds a new ARM_ARCH_FEATURE build option to add support
484 for compiler's feature modifiers.
485
486- Enable 8 cores support for Theodul DSU(DynamIQ Shared Unit) for the
487 Total Compute (TC0) platform.
488
489- New tests:
490
491 - Remove redundant code and add better tests for TRNG SMCs.
492 - Tests that the Version, Features, and RND calls conform to the spec.
493
494 - New tests for v8.6 AMU enhancements (FEAT_AMUv1p1)
495 - Make sure AMU offsets are being saved and restored properly.
496
497 - Tests to request SP-to-SP memory share.
498
499 - SP-to-SP direct messaging deadlock test.
500 - TFTF sends CACTUS_REQ_DEADLOCK_CMD to cactus SP.
501
502Cactus(Secure-EL1 test partition)
503~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
504
505- Enable managed exit for primary cactus secure partition.
506
507- Helper commands needed for interrupt testing.
508
509- Add handler from managed exit FIQ interrupt.
510
511- Make ffa_id global.
512
513- Implement HF_INTERRUPT_ENABLE Hafnium hypervisor call wrapper. With this
514 service, a secure partition calls into the SPMC to enable/disable a
515 particular virtual interrupt.
516
517- Invalidate the data cache for the cactus image.
518
519- Helper commands needed for interrupt testing.
520 - CACTUS_SLEEP_CMD & CACTUS_INTERRUPT_CMD added.
521
522- Decouple exception handling from tftf framework.
523 - With new interrupt related tests coming up in Cactus, added separate
524 exception handler code for irq/fiq in Cactus.
525
526- Hypervisor calls moved to a separate module.
527
528- Add secondary entry point register function.
529
530- Declare third SP instance as UP SP.
531
532- Provision a cold boot path for secondary cores (or secondary pinned
533 execution contexts).
534
535- Tidy message loop, commands definitions, direct messaging API definitions.
536
537- Helpers for error logging after FF-A calls.
538
539- Properly placing Cactus test files.
540
541- Tidying FF-A Memory Sharing tests.
542
543- Use CACTUS_ECHO_CMD in direct message tests.
544
545- Refactor handling of commands.
546 - Added helper macros to define a command handler, build a command table
547 in which each element is a pair of the handler and respective command
548 ID. Available tests have been moved to their own command handler.
549
550- Extend arguments in commands responses.
551 - In the test commands framework, added template to extend number of
552 values to include in a command response.
553
554- Check FF-A return is a valid direct response.
555 - Added a helper function to check if return of FFA_MSG_SEND_DIRECT_REQ
556 is FFA_MSG_SEND_DIRECT_RESP.
557
558- FFA_MSG_DIRECT_RESP call extended to use 5 registers.
559
560- Added accessors for arguments from FF-A calls.
561 - Some accessors for arguments from FF-A calls, namely for func id, error
562 code, and direct message destination/source.
563
564- Use virtual counter for sp_sleep.
565 - Changes sp_sleep() to use virtual counter instead of physical counter.
566
567- Checks if SIMD vectors are preserved in the normal world while transitioning
568 from normal world to secure world and back to normal world.
569
570- Tidying common code to tftf and cactus.
571
572- Refactor cactus_test_cmds.h to incorporate static inline functions instead
573 of macros to enforce type checking.
574
575- Removed reference to Hafnium in name from helper function and macro to
576 make them generic.
577
578- For consistency added the cmd id 'CACTUS_MEM_SEND_CMD'.
579
580- Add command to request memory sharing between SPs.
581
582- Add & handle commands 'CACTUS_REQ_ECHO_CMD' and 'CACTUS_ECHO_CMD'.
583
584- Update README with list of sample partitions.
585
586- Remove reference to PSA from xml test file.
587
588- Reduce tests verbosity in release mode.
589 - Update few NOTICE messages to VERBOSE/INFO.
590
591- Fix conversion issues on cactus responses.
592
593- Create RXTX map/configure helper macros and use them.
594
595- Update OP-TEE version used for testing to 3.10.
596 - SPMC as S-EL1 tests using OP-TEE depend on a static binary stored as
597 a CI file. This binary corresponds to a build of OP-TEE v3.10.
598
599- Add uart2 to device-regions node.
600 - First SP no longer has an open access to the full system peripheral
601 range and devices must be explicitly declared in the SP manifest.
602
603- New tests:
604
605 - Test for exercising SMMUv3 driver to perform stage2 translation.
606
607 - Test handling of non-secure interrupt while running SP.
608
609 - Add secondary cores direct messaging test for SPM.
610
611 - Testing deadlock by FF-A direct message.
612 - Added command CACTUS_DEADLOCK_CMD to file cactus_test_cmds.h to create
613 a deadlock scenario using FF-A direct message interfaces.
614
615 - Test SP-to-SP memory share operations
616 - Handle 'CACTUS_REQ_MEM_SEND_CMD' by sending memory to the receiver SP.
617
618 - Implemented test to validate FFA_RXTX_MAP ABI.
619
Manish V Badarkhe86600a82020-10-04 18:33:44 +0100620Version 2.4
621-----------
622
623New features
624^^^^^^^^^^^^
625- More tests are made available in this release to help validate the
626 functionalities in the following areas:
627 - SMCCC.
628 - New architecture specific features.
629 - FF-A features.
630 - New platform ports.
631
632- Various improvements to test framework and test suite such as documentation,
633 removing un-necessary dependencies, etc.
634
635TFTF
636~~~~
637
638- Remove dependencies from FVP to generic code by converting some FVP platform
639 specific macros to the common macros.
640
641- Remove make as a package dependency to compile TF-A test code.
642
643- Move defaults values and macro defs in a separate folder from Makefile.
644
645- Allow alternate stdout to be used apart from pl011 UART.
646
647- Get FVP platform's topology from build options to make FVP platform
648 configuration more flexible and eliminate test errors when the platform
649 is configured with number of CPUs less than default values in the makefile.
650
651- Update the FIP corrupt address which is used to corrupt BL2 image that helps
652 to trigger firmware update process.
653
654- Add explicit barrier before sev() in tftf_send_event_common API to avoid
655 core hang.
656
657- Align output properly on issuing make help_tests by removing dashes
658 and sort tests.
659
660- Moved a few FVP and Juno specific defined from common header files to platform
661 specific header files.
662
663- Replace SPCI with PSA FF-A in code as SPCI is now called as FF-A.
664
665- Add owner field to sp_layout generation to differentiate owner of SP which
666 could either be Silicon Provider or Platform provider.
667
668- Add v8.5 Branch Target Identifier(BTI) support in TFTF.
669
670- Remove dependency on SYS_CNT_BASE1 to read the memory mapped timers.
671
672- Enables SError aborts for all CPUs, during their power on sequence.
673
674- Documentation:
675
676 - Use conditional assignment on sphinx variables so that they can be
677 overwritten by environment and/or command line.
678
679 - Add support for documentation build as a target in Makefile.
680
681 - Update list of maintainers.
682
683 - Update documentation to explain how to locally build the documentation.
684
685 - Add .editorconfig from TF-A to define the coding style.
686
687 - Fix documentation to include 'path/to' prefix when specifying tftf.bin on
688 make fip cmd.
689
690 - Use docker to build documentation.
691
692 - Replace SPCI with PSA FF-A in documentation as SPCI is now called
693 as FF-A.
694
695- NVIDIA Tegra194:
696
697 - Skip CPU suspend tests requiring SGI as wake source as Tegra194 platforms
698 do not support CPU suspend power down and cannot be woken up with an SGI.
699
700 - Disable some system suspend test cases.
701
702 - Create dummy SMMU context for system resume to allow the System Resume
703 Firmware to complete without any errors or warnings.
704
705 - Increase RTC step value to 5ms as RTC consumes 250us for each register
706 read/write. Increase the step value to 5ms to cover all the register
707 read/write in program_timer().
708
709 - Skip some timer framework validation tests as CPUs on Tegra194 platforms
710 cannot be woken up with the RTC timer interrupt after power off.
711
712 - Introduce per-CPU Hypervisor Timer Interrupt ID.
713
714 - Skip PSCI STAT tests requiring PSTATE_TYPE_POWERDOWN as Tegra194 platforms
715 do not support CPU suspend with state type as PSTATE_TYPE_POWERDOWN.
716
717 - Disable boot requirement tests as Tegra194 platforms do not support memory
718 mapped timers.
719
720 - Skips the test "Create all power states and validate EL3 power state parsing"
721 from the "EL3 power state parser validation" test suite as it is not in
722 sync with this expectation.
723
724 - Moved reset, timers. wake, watchdog drivers from Tegra194 specific folder to
725 common driver folder so that these drivers can be used for other NVIDIA platforms.
726
727- New tests:
728
729 - Add test for SDEI RM_ANY routing mode.
730
731 - Add initial platform support for TC0.
732
733 - Add SMC fuzzing module test.
734
735 - Add test case for SMCCC_ARCH_SOC_ID feature.
736
737 - Add test that supports ARMv8.6-FGT in TF-A.
738
739 - Add test that supports ARMv8.6-ECV in TF-A.
740
741 - Add test for FFA_VERSION interface.
742
743 - Add test for FFA_FEATURES interface.
744
745 - Add console driver for the TI UART 16550.
746
747 - Add tests for FF-A memory sharing interfaces between tftf
748 and cactus secure partitions.
749
750 - NVIDIA Tegra194:
751
752 - Introduce platform port for Tegra194 to to initialize the tftf
753 framework and execute tests on the CPUs.
754
755 - Introduce power management support.
756
757 - Introduce support for RTC as wake source.
758
759 - Introduce system reset functionality test.
760
761 - Introduce watchdog timer test.
762
763 - Introduce support for NVIDIA Denver CPUs.
764
765 - Introduce RAS uncorrectable error injection test.
766
767 - Introduce tests to verify the Video Memory resize interface.
768
769 - Introduce test to inject RAS corrected errors for all supported
770 nodes from all CPUs.
771
772 - Introduce a test to get return value from SMC SiP function
773 TEGRA_SIP_GET_SMMU_PER.
774
775 - NVIDIA Tegra196:
776
777 - Introduce initial support for Tegra186 platforms.
778
779 - NVIDIA Tegra210:
780
781 - Introduce initial support for Tegra210 platforms.
782
783Secure partition - Cactus
784~~~~~~~~~~~~~~~~~~~~~~~~~
785
786- TFTF doesn't need to boot Secondary Cactus as Hafnium now boots all
787 partitions according to "boot-order" field value in the partition
788 manifests.
789
790- Remove test files related to deprecated SPCI Alpha specification and
791 SPRT interface.
792
793- Select different stdout device at runtime as primary VM can access
794 to UART while secondary VM's use hypervisor call to SPM for debug
795 logging.
796
797- An SP maps its RX/TX buffers in its EL1&0 Stage-1 translation regime.
798 The same RX/TX buffers are mapped by the SPMC in the SP's EL1&0
799 Stage-2 translation regime during boot time.
800
801- Update memory/device region nodes in manifest. Memory region has 3
802 entries such as RX buffer, TX buffer and dummy. These memory region
803 entries are mapped with attributes as "RX buffer: read-only",
804 "TX buffer: read-write" and "dummy: read-write-execute".
805 Device region mapped with read-write attribute.
806
807- Create tertiary partition without RX_TX region specified to test the
808 RXTX_MAP API.
809
810- Add third partition to ffa_partition_info_get test to test that a
811 partition can successfully get information about the third cactus
812 partition.
813
814- Map RXTX region to third partition to point the mailbox to this RXTX
815 region.
816
817- Adjust the number of EC context to max number of PEs as per the FF-A
818 specification mandating that a SP must either "Implement as many ECs
819 as the number of PEs (in case of a "multi-processor" SP with pinned
820 contexts)" or "Implement a single EC (in case of a migratable
821 "uni-processor" SP).
822
823- Updated cactus test payload and TFTF ids as it is decided to have
824 secure partition FF-A ids in the range from 0x8001 to 0xfffe, 0x8000
825 and 0xffff FF-A ids are reserved for the SPMC and the SPMD respectively
826 and in the non-secure worlds, FF-A id 0 is reserved for the hypervisor
827 and 1 to 0x7fff FF-A ids are reserved for VMs.
828
829- Break the message loop on bad message request instead of replying
830 with the FF-A error ABI to the SPMC.
831
832- Remove deprecated hypervisor calls spm_vm_get_count and spm_vcpu_get_count.
833 Instead use FFA_PARTITION_INFO_GET discovery ABI.
834
835- Implement hvc call 'SPM_INTERRUPT_GET' to get interrupt id.
836
837- Re-structure platform dependent files by moving platform dependent files
838 and macros to platform specific folder.
839
840- Adjust partition info get properties to support receipt of direct
841 message request.
842
843- New tests:
844
845 - Add FFA Version Test.
846
847 - Add FFA_FEATURES test.
848
849 - Add FFA_MEM_SHARE test
850
851 - Add FFA_MEM_LEND test.
852
853 - Add FFA_MEM_DONATE test.
854
855 - Add FFA_PARTITION_INFO_GET test.
856
857 - Add exception/interrupt framework.
858
859 - Add cactus support for TC0 platform.
860
861Issues resolved since last release
862^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
863
864- Update link to SMCCC specification.
865
866- Trim down the top-level readme file to give brief overview of the project
867 and also fix/update a number of broken/out-dated links in it.
868
869- Bug fix in Multicore IRQ spurious test.
870
871- Fix memory regions mapping with no NS bit set.
872
873- Reenable PSCI NODE_HW_STATE test which was disabled earlier due to
874 outdated SCP firmware.
875
876- Fix Aarch32 zeromem() function by avoiding infinite loop in 'zeromem'
877 function and optimizing 'memcpy4' function.
878
879- Add missing help_tests info on help target in the top-level Makefile.
880
881- Trim down the readme file as it does not need to provide detailed
882 information, instead it can simply be a landing page providing a brief
883 overview of the project and redirecting the reader to RTD for further
884 information.
885
886- Fix maximum number of CPUs in DSU cluster by setting maximum number of CPUs
887 in DSU cluster to 8.
888
Madhukar Pappireddybced2f32020-04-08 15:44:38 -0500889Version 2.3
890-----------
891
892New features
893^^^^^^^^^^^^
894
895- More tests are made available in this release to help validate
896 the functionality of TF-A.
897
898- CI upgraded to use GCC 9.2-2019.12 toolchain for tf-a-tests.
899
900- Various improvements to test framework and test suite.
901
902TFTF
903~~~~
904
905- Support for extended register usage as per SMCCC v1.2 specification.
906
907- Support for FVP platforms with SMT capabilities.
908
909- Improved support for documentation through addition of basic Sphinx
910 configuration and Makefile similar to TF-A repository.
911
912- Enhancement to libc library synchronous to TF-A code base.
913
914- ARMv8.3-PAuth enabled for all FWU tests in TFTF.
915
916- TFTF made RFC 4122 compliant by converting UUIDs to network order format.
917
918- Build improvement by deprecating custom AARCH64/AARCH32 macros in favor of
919 __arch64__ macro provided by compiler.
920
921- Support for HVC as a SMCCC conduit in TFTF.
922
923- New tests:
924
925 - AArch32 tests for checking if PMU counters leak in secure world.
926
927 - Add new debug filesystem (debugfs) test.
928
929 - Add a SPCI direct messaging test targeting bare-metal cactus SP.
930
931
932Secure partitions
933~~~~~~~~~~~~~~~~~
934
935Cactus
936~~~~~~
937
938- Several build improvements and symbol relocation fixup to make it position
939 independent executable.
940
941- Update of sample manifest to SPCI Beta1 format.
942
943- Support for generating JSON file as required by TF-A.
944
945Issues resolved since last release
946^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
947
948- Makefile bug fix for performing parallel builds.
949
950- Add missing D-cache invalidation of RW memory in tftf_entrypoint to safeguard
951 against possible corruption.
952
953- Fixes in GIC drivers to support base addresses beyond 4G range.
954
955- Fix build with XML::LibXML 2.0202 Perl module
956
957Known issues and limitations
958^^^^^^^^^^^^^^^^^^^^^^^^^^^^
959
960The sections below list the known issues and limitations of each test image
961provided in this repository. Unless and otherwise stated, issues and limitations
962stated in previous release continue to exist in this release.
963
964TFTF
965~~~~
966- NODE_HW_STATE test has been temporarily disabled for sgi575 platform due to a
967 dependency on SCP binaries version 2.5
968
Paul Beesley5c928952019-10-24 11:57:00 +0000969Version 2.2
970-----------
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500971
972New features
Paul Beesley5c928952019-10-24 11:57:00 +0000973^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500974
975- A wide range of tests are made available in this release to help validate
976 the functionality of TF-A.
977
978- Various improvements to test framework and test suite.
979
980TFTF
Paul Beesley5c928952019-10-24 11:57:00 +0000981~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500982
983- Enhancement to xlat table library synchronous to TF-A code base.
984
985- Enabled strict alignment checks (SCTLR.A & SCTLR.SA) in all images.
986
987- Support for a simple console driver. Currently it serves as a placeholder
988 with empty functions.
989
990- A topology helper API is added in the framework to get parent node info.
991
992- Support for FVP with clusters having upto 8 CPUs.
993
994- Enhanced linker script to separate code and RO data sections.
995
996- Relax SMC calls tests. The SMCCC specification recommends Trusted OSes to
997 mitigate the risk of leaking information by either preserving the register
998 state over the call, or returning a constant value, such as zero, in each
999 register. Tests only allowed the former behaviour and have been extended to
1000 allow the latter as well.
1001
1002- Pointer Authentication enabled on warm boot path with individual APIAKey
1003 generation for each CPU.
1004
1005- New tests:
1006
1007 - Basic unit tests for xlat table library v2.
1008
1009 - Tests for validating SVE support in TF-A.
1010
1011 - Stress tests for dynamic xlat table library.
1012
1013 - PSCI test to measure latencies when turning ON a cluster.
1014
1015 - Series of AArch64 tests that stress the secure world to leak sensitive
1016 counter values.
1017
1018 - Test to validate PSCI SYSTEM_RESET call.
1019
1020 - Basic tests to validate Memory Tagging Extensions are being enabled and
1021 ensuring no undesired leak of sensitive data occurs.
1022
1023- Enhanced tests:
1024
1025 - Improved tests for Pointer Authentication support. Checks are performed
1026 to see if pointer authentication keys are accessible as well as validate
1027 if secure keys are being leaked after a PSCI version call or TSP call.
1028
1029 - Improved AMU test to remove unexecuted code iterating over Group1 counters
1030 and fix the conditional check of AMU Group0 counter value.
1031
1032Secure partitions
Paul Beesley5c928952019-10-24 11:57:00 +00001033~~~~~~~~~~~~~~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001034
1035A new Secure Partition Quark is introduced in this release.
1036
1037Quark
Paul Beesley5c928952019-10-24 11:57:00 +00001038~~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001039
1040The Quark test secure partition provided is a simple service which returns a
1041magic number. Further, a simple test is added to test if Quark is functional.
1042
1043Issues resolved since last release
Paul Beesley5c928952019-10-24 11:57:00 +00001044^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001045
1046- Bug fix in libc memchr implementation.
1047
1048- Bug fix in calculation of number of CPUs.
1049
1050- Streamlined SMC WORKAROUND_2 test and fixed a false fail on Cortex-A76 CPU.
1051
1052- Pointer Authentication support is now available for secondary CPUs and the
1053 corresponding tests are stable in this release.
1054
1055Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001056^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001057
1058The sections below list the known issues and limitations of each test image
1059provided in this repository. Unless and otherwise stated, issues and limitations
1060stated in previous release continue to exist in this release.
1061
1062TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001063~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001064- Multicore spurious interrupt test is observed to have unstable behavior. As a
1065 temporary solution, this test is skipped for AArch64 Juno configurations.
1066
1067- Generating SVE instructions requires `O3` compilation optimization. Since the
1068 current build structure does not allow compilation flag modification for
1069 specific files, the function which tests support for SVE has been pre-compiled
1070 and added as an assembly file.
1071
1072
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001073
Paul Beesley5c928952019-10-24 11:57:00 +00001074Version 2.1
1075-----------
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001076
1077New features
Paul Beesley5c928952019-10-24 11:57:00 +00001078^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001079
1080- Add initial support for testing Secure Partition Client Interface (SPCI)
1081 and Secure Partition Run-Time (SPRT) standards.
1082
1083 Exercise the full communication flow throughout the software stack, involving:
1084
1085 - A Secure-EL0 test partition as the Trusted World agent.
1086
1087 - TFTF as the Normal World agent.
1088
1089 - The Secure Partition Manager (SPM) in TF-A.
1090
1091- Various stability improvements, code refactoring and clean ups.
1092
1093TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001094~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001095
1096- Reorganize tests build infrastructure to allow the selection of a subset of
1097 tests.
1098
1099- Reorganize the platform layer for improved clarity and simplicity.
1100
1101- Sanitise inclusion of drivers header files.
1102
1103- Enhance the test report format for improved clarity and conciseness.
1104
1105- Dump CPU registers when hitting an unexpected exception. Previously, this
1106 would silently loop forever.
1107
1108- Import libc from TF-A to better align the two code bases.
1109
1110- New tests:
1111
1112 - SPM tests for exercising communication through either the MM or SPCI/SPRT
1113 interfaces.
1114
1115 - SMC calling convention tests.
1116
1117 - Initial tests for Armv8.3 Pointer Authentication support (experimental).
1118
1119- New platform ports:
1120
1121 - `Arm SGI-575`_ FVP.
1122
1123 - Hikey960 board (experimental).
1124
1125 - `Arm Neoverse Reference Design N1 Edge (RD-N1-Edge)`_ FVP (experimental).
1126
1127Secure partitions
Paul Beesley5c928952019-10-24 11:57:00 +00001128~~~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001129
1130We now have 3 Secure Partitions to test the SPM implementation in TF-A.
1131
1132Cactus-MM
1133'''''''''
1134
1135The Cactus test secure partition provided in version 2.0 has been renamed into
1136"*Cactus-MM*". It is still responsible for testing the SPM implementation based
1137on the Arm Management Mode Interface.
1138
1139Cactus
1140''''''
1141
1142This is a new test secure partition (as the former "*Cactus*" has been renamed
1143into "*Cactus-MM*", see above).
1144
1145Unlike *Cactus-MM*, this image tests the SPM implementation based on the SPCI
1146and SPRT draft specifications.
1147
1148It runs in Secure-EL0 and performs the following tasks:
1149
1150- Test that TF-A has correctly setup the secure partition environment (access
1151 to cache maintenance operations, to floating point registers, etc.)
1152
1153- Test that TF-A accepts to change data access permissions and instruction
1154 permissions on behalf of Cactus for memory regions the latter owns.
1155
1156- Test communication with SPM through SPCI/SPRT interfaces.
1157
1158Ivy
1159'''
1160
1161This is also a new test secure partition. It is provided in order to test
1162multiple partitions support in TF-A. It is derived from Cactus and essentially
1163provides the same services but with different identifiers at the moment.
1164
1165EL3 payload
Paul Beesley5c928952019-10-24 11:57:00 +00001166~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001167
1168- New platform ports:
1169
1170 - `Arm SGI-575`_ FVP.
1171
1172 - `Arm Neoverse Reference Design N1 Edge (RD-N1-Edge)`_ FVP (experimental).
1173
1174Issues resolved since last release
Paul Beesley5c928952019-10-24 11:57:00 +00001175^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001176
1177- The GICv2 spurious IRQ test is no longer Juno-specific. It is now only
1178 GICv2-specific.
1179
1180- The manual tests in AArch32 state now work properly. After investigation,
1181 we identified that this issue was not AArch32 specific but concerned any
1182 test relying on state information persisting across reboots. It was due to
1183 an incorrect build configuration.
1184
1185- Cactus-MM now successfully links with GNU toolchain 7.3.1.
1186
1187Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001188^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001189
1190The sections below lists the known issues and limitations of each test image
1191provided in this repository.
1192
1193TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001194~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001195
1196The TFTF test image might be conceptually sub-divided further in 2 parts: the
1197tests themselves, and the test framework they are based upon.
1198
1199Test framework
Paul Beesley5c928952019-10-24 11:57:00 +00001200~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001201
1202- Some stability issues.
1203
1204- No mechanism to abort tests when they time out (e.g. this could be
1205 implemented using a watchdog).
1206
1207- No convenient way to include or exclude tests on a per-platform basis.
1208
1209- Power domains and affinity levels are considered equivalent but they may
1210 not necessarily be.
1211
1212- Need to provide better support to alleviate duplication of test code. There
1213 are some recurrent test patterns for which helper functions should be
1214 provided. For example, bringing up all CPUs on the platform and executing the
1215 same function on all of them, or programming an interrupt and waiting for it
1216 to trigger.
1217
1218- Every CPU that participates in a test must return from the test function. If
1219 it does not - e.g. because it powered itself off for testing purposes - then
1220 the test framework will wait forever for this CPU. This limitation is too
1221 restrictive for some tests.
1222
1223- No protection against interrupted flash operations. If the target is reset
1224 while some data is written to flash, the test framework might behave
1225 incorrectly on reset.
1226
1227- When compiling the code, if the generation of the ``tests_list.c`` and/or
1228 ``tests_list.h`` files fails, the build process is not aborted immediately
1229 and will only fail later on.
1230
1231- The directory layout requires further improvements. Most of the test
1232 framework code has been moved under the ``tftf/`` directory to better isolate
1233 it but this effort is not complete. As a result, there are still some TFTF
1234 files scattered around.
1235
1236- Pointer Authentication testing is experimental and incomplete at this stage.
1237 It is only enabled on the primary CPU on the cold boot.
1238
1239Tests
Paul Beesley5c928952019-10-24 11:57:00 +00001240~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001241
1242- Some tests are implemented for AArch64 only and are skipped on AArch32.
1243
1244- Some tests are not robust enough:
1245
1246 - Some tests might hang in some circumstances. For example, they might wait
1247 forever for a condition to become true.
1248
1249 - Some tests rely on arbitrary time delays instead of proper synchronization
1250 when executing order-sensitive steps.
1251
1252 - Some tests have been implemented in a practical manner: they seem to work
1253 on actual hardware but they make assumptions that are not guaranteed by
1254 the Arm architecture. Therefore, they might fail on some other platforms.
1255
1256- PSCI stress tests are very unreliable and will often hang. The root cause is
1257 not known for sure but this might be due to bad synchronization between CPUs.
1258
1259- The GICv2 spurious IRQ test sometimes fails with the following error message:
1260
1261 ``SMC @ lead CPU returned 0xFFFFFFFF 0x8 0xC``
1262
1263 The root cause is unknown.
1264
1265- The FWU tests take a long time to complete. This is because they wait for the
1266 watchdog to reset the system. On FVP, TF-A configures the watchdog period to
1267 about 4 min. This limit is excessive for an automated testing context and
1268 leaves the user without feedback and unable to determine if the tests are
1269 proceeding properly.
1270
1271- The test "Target timer to a power down cpu" sometimes fails with the
1272 following error message:
1273
1274 ``Expected timer switch: 4 Actual: 3``
1275
1276 The root cause is unknown.
1277
1278FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001279~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001280
1281- The FWU tests do not work on the revC of the Base AEM FVP. They only work on
1282 the revB.
1283
1284- NS-BL1U and NS-BL2U images reuse TFTF-specific code for legacy reasons. This
1285 is not a clean design and may cause confusion.
1286
1287Test secure partitions (Cactus, Cactus-MM, Ivy)
Paul Beesley5c928952019-10-24 11:57:00 +00001288~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001289
1290- This is experimental code. It's likely to change a lot as the secure
1291 partition software architecture evolves.
1292
1293- Supported on AArch64 FVP platform only.
1294
1295All test images
Paul Beesley5c928952019-10-24 11:57:00 +00001296~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001297
1298- TF-A Tests are derived from a fork of TF-A so:
1299
1300 - they've got some code in common but lag behind on some features.
1301
1302 - there might still be some irrelevant references to TF-A.
1303
1304- Some design issues.
1305 E.g. TF-A Tests inherited from the I/O layer of TF-A, which still needs a
1306 major rework.
1307
1308- Cannot build TF-A Tests with Clang. Only GCC is supported.
1309
1310- The build system does not cope well with parallel building. The user should
1311 not attempt to run multiple jobs in parallel with the ``-j`` option of `GNU
1312 make`.
1313
1314- The build system does not properly track build options. A clean build must be
1315 performed every time a build option changes.
1316
1317- UUIDs are not compliant to RFC 4122.
1318
1319- No floating point support. The code is compiled with GCC flag
1320 ``-mgeneral-regs-only``, which prevents the compiler from generating code
1321 that accesses floating point registers. This might limit some test scenarios.
1322
1323- The documentation is too lightweight.
1324
1325- Missing instruction barriers in some places before reading the system counter
1326 value. As a result, the CPU could speculatively read it and any delay loop
1327 calculations might be off (because based on stale values). We need to examine
1328 all such direct reads of the ``CNTPCT_EL0`` register and replace them with a
1329 call to ``syscounter_read()`` where appropriate.
1330
Paul Beesley5c928952019-10-24 11:57:00 +00001331Version 2.0
1332-----------
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001333
1334New features
Paul Beesley5c928952019-10-24 11:57:00 +00001335^^^^^^^^^^^^
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001336
1337This is the first public release of the Trusted Firmware-A Tests source code.
1338
1339TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001340~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001341
1342- Provides a baremetal test framework to exercise TF-A features through its
1343 ``SMC`` interface.
1344
1345- Integrates easily with TF-A: the TFTF binary is packaged in the FIP image
1346 as a ``BL33`` component.
1347
1348- Standalone binary that runs on the target without human intervention (except
1349 for some specific tests that require a manual target reset).
1350
1351- Designed for multi-core testing. The various sub-frameworks allow maximum
1352 parallelism in order to stress the firmware.
1353
1354- Displays test results on the UART output. This may then be parsed by an
1355 external tool and integrated in a continuous integration system.
1356
1357- Supports running in AArch64 (NS-EL2 or NS-EL1) and AArch32 states.
1358
1359- Supports parsing a tests manifest (XML file) listing the tests to include in
1360 the binary.
1361
1362- Detects most platform features at run time (e.g. topology, GIC version, ...).
1363
1364- Provides a topology enumeration framework. Allows tests to easily go through
1365 affinity levels and power domain nodes.
1366
1367- Provides an event framework to synchronize CPU operations in a multi-core
1368 context.
1369
1370- Provides a timer framework. Relies on a single global timer to generate
1371 interrupts for all CPUs in the system. This allows tests to easily program
1372 interrupts on demand to use as a wake-up event source to come out of CPU
1373 suspend state for example.
1374
1375- Provides a power-state enumeration framework. Abstracts the valid power
1376 states supported on the platform.
1377
1378- Provides helper functions for power management operations (CPU hotplug,
1379 CPU suspend, system suspend, ...) with proper saving of the hardware state.
1380
1381- Supports rebooting the platform at the end of each test for greater
1382 independence between tests.
1383
1384- Supports interrupting and resuming a test session. This relies on storing
1385 test results in non-volatile memory (e.g. flash).
1386
1387FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001388~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001389
1390- Provides example code to exercise the Firmware Update feature of TF-A.
1391
1392- Tests the robustness of the FWU state machine implemented in the TF-A by
1393 sending valid and invalid authentication, copy and image execution requests
1394 to the TF-A BL1 image.
1395
1396EL3 test payload
Paul Beesley5c928952019-10-24 11:57:00 +00001397~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001398
1399- Tests the ability of TF-A to load an EL3 payload.
1400
1401Cactus test secure partition
Paul Beesley5c928952019-10-24 11:57:00 +00001402~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001403
1404- Tests that TF-A has correctly setup the secure partition environment: it
1405 should be allowed to perform cache maintenance operations, access floating
1406 point registers, etc.
1407
1408- Tests the ability of a secure partition to request changing data access
1409 permissions and instruction permissions of memory regions it owns.
1410
1411- Tests the ability of a secure partition to handle StandaloneMM requests.
1412
1413Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001414^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001415
1416The sections below lists the known issues and limitations of each test image
1417provided in this repository.
1418
1419TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001420~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001421
1422The TFTF test image might be conceptually sub-divided further in 2 parts: the
1423tests themselves, and the test framework they are based upon.
1424
1425Test framework
Paul Beesley5c928952019-10-24 11:57:00 +00001426~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001427
1428- Some stability issues.
1429
1430- No mechanism to abort tests when they time out (e.g. this could be
1431 implemented using a watchdog).
1432
1433- No convenient way to include or exclude tests on a per-platform basis.
1434
1435- Power domains and affinity levels are considered equivalent but they may
1436 not necessarily be.
1437
1438- Need to provide better support to alleviate duplication of test code. There
1439 are some recurrent test patterns for which helper functions should be
1440 provided. For example, bringing up all CPUs on the platform and executing the
1441 same function on all of them, or programming an interrupt and waiting for it
1442 to trigger.
1443
1444- Every CPU that participates in a test must return from the test function. If
1445 it does not - e.g. because it powered itself off for testing purposes - then
1446 the test framework will wait forever for this CPU. This limitation is too
1447 restrictive for some tests.
1448
1449- No protection against interrupted flash operations. If the target is reset
1450 while some data is written to flash, the test framework might behave
1451 incorrectly on reset.
1452
1453- When compiling the code, if the generation of the tests_list.c and/or
1454 tests_list.h files fails, the build process is not aborted immediately and
1455 will only fail later on.
1456
1457- The directory layout is confusing. Most of the test framework code has been
1458 moved under the ``tftf/`` directory to better isolate it but this effort is
1459 not complete. As a result, there are still some TFTF files scattered around.
1460
1461Tests
Paul Beesley5c928952019-10-24 11:57:00 +00001462~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001463
1464- Some tests are implemented for AArch64 only and are skipped on AArch32.
1465
1466- Some tests are not robust enough:
1467
1468 - Some tests might hang in some circumstances. For example, they might wait
1469 forever for a condition to become true.
1470
1471 - Some tests rely on arbitrary time delays instead of proper synchronization
1472 when executing order-sensitive steps.
1473
1474 - Some tests have been implemented in a practical manner: they seem to work
1475 on actual hardware but they make assumptions that are not guaranteed by
1476 the Arm architecture. Therefore, they might fail on some other platforms.
1477
1478- PSCI stress tests are very unreliable and will often hang. The root cause is
1479 not known for sure but this might be due to bad synchronization between CPUs.
1480
1481- The GICv2 spurious IRQ test is Juno-specific. In reality, it should only be
1482 GICv2-specific. It should be reworked to remove any platform-specific
1483 assumption.
1484
1485- The GICv2 spurious IRQ test sometimes fails with the following error message:
1486
1487 ``SMC @ lead CPU returned 0xFFFFFFFF 0x8 0xC``
1488
1489 The root cause is unknown.
1490
1491- The manual tests in AArch32 mode do not work properly. They save some state
1492 information into non-volatile memory in order to detect the reset reason but
1493 this state does not appear to be retained. As a result, these tests keep
1494 resetting infinitely.
1495
1496- The FWU tests take a long time to complete. This is because they wait for the
1497 watchdog to reset the system. On FVP, TF-A configures the watchdog period to
1498 about 4 min. This is way too long in an automated testing context. Besides,
1499 the user gets not feedback, which may let them think that the tests are not
1500 working properly.
1501
1502- The test "Target timer to a power down cpu" sometimes fails with the
1503 following error message:
1504
1505 ``Expected timer switch: 4 Actual: 3``
1506
1507 The root cause is unknown.
1508
1509FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001510~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001511
Sandrine Bailleux7af6c6d2018-10-22 17:09:07 +02001512- The FWU tests do not work on the revC of the Base AEM FVP. They only work on
1513 the revB.
1514
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001515- NS-BL1U and NS-BL2U images reuse TFTF-specific code for legacy reasons. This
1516 is not a clean design and may cause confusion.
1517
1518Cactus test secure partition
Paul Beesley5c928952019-10-24 11:57:00 +00001519~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001520
1521- Cactus is experimental code. It's likely to change a lot as the secure
1522 partition software architecture evolves.
1523
1524- Fails to link with GNU toolchain 7.3.1.
1525
1526- Cactus is supported on AArch64 FVP platform only.
1527
1528All test images
Paul Beesley5c928952019-10-24 11:57:00 +00001529~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001530
1531- TF-A Tests are derived from a fork of TF-A so:
1532
1533 - they've got some code in common but lag behind on some features.
1534
1535 - there might still be some irrelevant references to TF-A.
1536
1537- Some design issues.
1538 E.g. TF-A Tests inherited from the I/O layer of TF-A, which still needs a
1539 major rework.
1540
1541- Cannot build TF-A Tests with Clang. Only GCC is supported.
1542
1543- The build system does not cope well with parallel building. The user should
1544 not attempt to run multiple jobs in parallel with the ``-j`` option of `GNU
1545 make`.
1546
1547- The build system does not properly track build options. A clean build must be
1548 performed every time a build option changes.
1549
1550- SMCCC v2 is not properly supported.
1551
1552- UUIDs are not compliant to RFC 4122.
1553
1554- No floating point support. The code is compiled with GCC flag
1555 ``-mgeneral-regs-only``, which prevents the compiler from generating code
1556 that accesses floating point registers. This might limit some test scenarios.
1557
1558- The documentation is too lightweight.
1559
1560--------------
1561
Jayanth Dodderi Chidanand090491b2022-05-03 15:05:11 +01001562*Copyright (c) 2018-2022, Arm Limited. All rights reserved.*
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001563
1564.. _Arm Neoverse Reference Design N1 Edge (RD-N1-Edge): https://developer.arm.com/products/system-design/reference-design/neoverse-reference-design
1565.. _Arm SGI-575: https://developer.arm.com/products/system-design/fixed-virtual-platforms