blob: 07cfbb66fafe12d51b19cd217a06a979e216ead4 [file] [log] [blame]
Paul Beesley5c928952019-10-24 11:57:00 +00001Change Log & Release Notes
2==========================
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02003
4Please note that the Trusted Firmware-A Tests version follows the Trusted
5Firmware-A version for simplicity. At any point in time, TF-A Tests version
6`x.y` aims at testing TF-A version `x.y`. Different versions of TF-A and TF-A
7Tests are not guaranteed to be compatible. This also means that a version
8upgrade on the TF-A-Tests side might not necessarily introduce any new feature.
9
Juan Pablo Conde0a0f6742023-05-13 00:20:15 -050010Version 2.9
11-----------
12
13New features
14^^^^^^^^^^^^
15
16- More tests are made available in this release to help validate the
17 functionalities in the following areas:
18
19 - FF-A Features
20 - Realm Management Extension
21 - New Architecture Specific features related to v8.8
22 - New platform ports
23
24TFTF
25~~~~
26
27- FF-A testing:
28
29 - Reordered logs in the memory sharing tests.
30 - Memory share bumped to v1.1 EAC0.
31 - Updated tests for FFA_FEATURES(FFA_MEM_RETRIEVE_REQ).
32 - Fixed issues with RXTX buffer unmapping and dependencies on tests.
33 - Added check for execution state property of partitions.
34
35- New tests:
36
37 - Tests for Errata management firmware interface.
38 - Ensure FPU state registers context is preserved in RL/SE/NS.
39 - Modified FEAT_HCX test to also check for HCRX_EL2 init value.
40 - Added basic SME2 tests.
41 - PSCI tests for OS-initiated mode.
42 - Added "nop" test to be used in conjunction with TFX.
43 - Introduced capability to generate Sync External Aborts (EA) in TFTF.
44 - New test to generate an SError.
45 - Tests to check whether the PMU is functional and if the state is
46 preserved when switching worlds. PMEVTYPER.evtCount width extended.
47 - Added support for more SPE versions.
48
49- Platforms:
50
51 - RD-N2-Cfg3:
52
53 - Added TFTF support.
54
55- Miscellaneous:
56
57 - SIMD/FPU save/restore routine moved to common lib.
58 - Updated toolchain requirements documentation.
59 - Update SME/Mortlach tests.
60 - Unified Firmware First handling of lower EL EA.
61 - Moved trusted wdog API to spm_common.
62 - Added the ability to skip tests for AArch32.
63 - Added config file to allow doc defaults be changed.
64 - Modified tests for FEAT_RNG_TRAP.
65 - Moved 'Stress test timer framework' to a new test suite
66 'tests-timer-stress'.
67 - Support for new binutils versions.
68 - Removed deprecated SPM libs and test code.
69
70
71Realm Management Extension (RME)
72~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
73
74 - Added helper macro for RME tests.
75 - Test Secure interrupt can preempt Realm EL1.
76 - Added PMU Realm tests.
77 - Added BP_OPTION to REALM_CFLAGS to allow build realm payload with
78 BTI/Pauth support.
79 - Fixed build issues introduced by the TFTF Realm extension
80 enhancement tests.
81 - Test case return codes updated according to RMM Bet0 specification.
82 - Fixed build problem related to rmi_rec_enter verbose log.
83 - Added randomization of SMC RMI commands parameters and checking of
84 X4-X7 return values as per SMCCC v1.2.
85
86Cactus (Secure-EL1 test partition)
87~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
88
89 - Use of FFA_CONSOLE_LOG for debug logs.
90 - Test for consecutive same memory donation to other partitions.
91 - Now validating NWd can't share forbidden addresses.
92 - Support for registering irq handlers.
93 - Fixed attributes for NS memory region.
94 - Removal of memory regions not page-aligned.
95 - Added check for core linear id matching id passed by SPMC.
96
97Issues resolved since last release
98^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
99
100 - Build issue for older toolchains and other failures resolved.
101 - Dropped invalid access test from CI.
102 - Now checking that the PMU is supported before using any of it.
103 - Use of write instead of read to generate an SError to avoid sync
104 exceptions instead.
105 - Fixed broken link to TRNG_FW documentation.
106 - INIT_TFTF_MAILBOX() is called first for the invalid access test.
107
Juan Pablo Conde47e64d42022-11-16 18:33:40 -0500108Version 2.8
109-----------
110
111New features
112^^^^^^^^^^^^
113- More tests are made available in this release to help validate the
114 functionalities in the following areas:
115
116 - FF-A Features
117 - Realm Management Extension
118 - New Architecture Specific features related to v8.8
119 - New platform ports
120
121TFTF
122~~~~
123
124- FF-A testing:
125
126 - UUID included in partition information descriptors.
127 - Checks for size of partition information descriptors.
128 - Renamed FFA_MSG_RUN ABI function to FFA_RUN and allowed it to return from
129 Waiting state.
130 - Made ffa_tests available for Ivy.
131 - Updated verbose message log structure.
132 - Prevented generate_json.sh from being called more than once by requiring
133 a list of partitions to be supplied.
134 - Added a temporary workaround for unexpected affinity info state to prevent
135 a system panic.
136 - Added test to exercise FFA_CONSOLE_LOG ABI.
137
138 - FF-A v1.1 Secure interrupts
139
140 - Added managed exit to first and second SP in call chain.
141 - Added test to exercise managed exit by two SPs in a call chain.
142 - Added tests to exercise NS interrupt being queued and signaled to SP.
143
144- New tests:
145
146 - Tests for SVE operations in Normal World and discover SVE vector length.
147 - Added cleanup TRNG service tests.
148 - Added test for SMCCC_ARCH_WORKAROUND_3.
149 - Updated PAuth helpers to support QARMA3 algorithm.
150 - Added tests for RNG_TRAP.
151
152- Platforms:
153
154 - SGI:
155
156 - Introduced platform variant build option.
157 - Re-organized header files.
158 - Migrated to secure uart port for routing tftf logs.
159
160 - N1SDP:
161
162 - Added TFTF support for N1SDP.
163
164 - RD-N2:
165
166 - Added TFTF support for RD-N2.
167
168 - RD-N2-Cfg1:
169
170 - Added TFTF support for RD-N2-Cfg1.
171
172 - RD-V1:
173
174 - Added TFTF support for RD-V1.
175
176- Miscellaneous:
177
178 - Added a missing ISB instruction in SME test.
179 - Refactor to make some helper functions re-usable.
180 - Updated build command to clean EL3 payload image.
181 - Move renaming of the primary dts file for ivy partitions.
182 - Added check that verifies if a platform supports el3_payload before
183 building it.
184 - Updated memory share test to meet Hafnium specification.
185 - Updated toolchain requirements documentation.
186
187
188Realm Management Extension (RME)
189~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
190
191 - Added Realm payload management capabilities to TFTF to act as a NS Host.
192 - Added test to verify that RMM and SPM can co-exist and work properly.
193 - Added function to reset delegated buffers to non-delegated state.
194 - Re-used existing wait_for_non_lead_cpus() function helper.
195 - Refactored RMI FID macros to simplify usage.
196 - Added userguide for realm payload testing.
197
198Cactus (Secure-EL1 test partition)
199~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
200
201 - Corrected some tests message types from ERROR to VERBOSE.
202 - Increased the cactus number of xlat to allow the use of 48b PA size for
203 memory sharing between SPs.
204 - Introduced a new direct request message command to resume after managed
205 exit.
206 - Skip enabling virtual maintenance interrupts explicitly.
207 - Allowed sender to resume interrupted target vCPU.
208 - Added support for handling managed exit through vIRQ.
209 - Added support for discovering interrupt IDs of managed exit signals.
210 - Specified action in response to NS interrupt in manifest.
211
212Ivy (Secure-EL0 test partition)
213~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
214
215 - Allowed testing using VHE.
216 - Allowed Ivy partitions to use ffa_helpers functions.
217 - Requirement of common name for Ivy partitions for consistency.
218 - Specified action in response to NS interrupt in manifest.
219
220Issues resolved since last release
221^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
222
223 - Fixed SME header guard name.
224 - Fixed response for incorrect direct message request for FF-A.
225
Jayanth Dodderi Chidanand090491b2022-05-03 15:05:11 +0100226Version 2.7
227-----------
228
229New features
230^^^^^^^^^^^^
231- More tests are made available in this release to help validate the
232 functionalities in the following areas:
233
234 - FF-A Features
235 - New Architecture Specific features related to v8.7
236 - New platform port
237
238TFTF
239~~~~
240
241- FF-A testing:
242
243 - FF-A partition information structure is updated to include UUIDs.
244 - Memory Management helper functions are refactored to fetch the details
245 of smc call failures in tftf and cactus.
246 - Added test to validate memory sharing operations from SP to NS-endpoint
247 are denied by SPMC.
248 - Added test to ensure an endpoint that sets its version to v1.0 receives
249 v1.0 partition information descriptors as defined in v1.0 FF-A
250 specification.
251 - Added test to validate that memory is cleared on memory sharing operations
252 between normal world and secure world.
253
254 - FF-A v1.1 Secure interrupts
255
256 - Added support to enhance the secure interrupt handling test.
257 - Support for registering and unregistering custom handler that is
258 invoked by SP at the tail end of the virtual interrupt processing.
259 - Added support for querying the ID of the last serviced virtual interrupt.
260
261- New tests:
262
263 - Added test to validate that realm region access is being prevented from
264 normal world.
265 - Added test to validate that secure region access is being prevented from
266 normal world.
267 - Added test to validate that secure region access is being prevented from
268 realm world.
269 - Added test to validate that root region access is being prevented from
270 realm world.
271 - Added a test for v8.7 Advanced floating-point behavior (FEAT_AFP).
272 - Added a SPE test that reads static profiling system registers
273 of available SPE version i.e. FEAT_SPE/FEAT_SPEv1p1/FEAT_SPEv1p2.
274 - Added a test to validate functionality of WFET and WFIT instructions
275 introduced by v8.7 FEAT_WFxT.
276 - Added basic SME tests to ensure feature enablement by EL3 is proper for
277 its usage at lower non-secure ELs.
278 - Added test to check Data Independent timing (DIT) field of PSTATE is
279 retained on exception.
280 - Added test to ensure that EL3 has properly enabled access to FEAT_BRBE
281 from non-secure ELs.
282
283- Platforms:
284
285 - Add initial platform support for corstone1000.
286
287 - TC:
288
289 - Support for notification in tertiary SP manifest.
290
291 - FVP:
292
293 - Support to provide test memory addresses to validate the invalid
294 memory access test from tftf(ns-el2).
295
296- Miscellaneous:
297
298 - Added support to configure the physical/virtual address space for FVP.
299 - Added common header file for defining macros with size to support all the
300 platforms.
301 - Introduced handler for synchronous exceptions (AArch64).
302 - Added macros to extract the ISS portion of an ELx ESR exception syndrome
303 register.
304 - Support to dynamically map/unmap test region to validate invalid memory
305 access tests.
306 - Added support to receive boot information through secure partitions,
307 according to the FF-A v1.1 EAC0 specification.
308 - Added an helper API function from SPM test suite to initialize FFA-mailbox
309 and enable FF-A based message with SP.
310 - Updated the build string to display the rc-tagged version.
311
312Cactus (Secure-EL1 test partition)
313~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314
315 - Added test for nonsecure memory sharing between Secure Partitions(SPs).
316 - Added test to validate that a realm region cannot be accessed from secure
317 world.
318 - Added test to permit checking a root region cannot be accessed from secure
319 world.
320 - Extended the test command CACTUS_MEM_SEND_CMD to add support for memory
321 sharing flags.
322 - Added support to save the state of general purpose registers x0-x4 at the
323 entry to cold boot and restore them before jumping to entrypoint of cactus.
324
325Issues resolved since last release
326^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
327
328 - Fixed a bug to align RMI FIDs with SMCCC.
329 - Fixed encoding of vCPU and receiver IDs in the FFA_NOTIFICATION_GET
330 interface to comply with the FF-A v1.1 beta0 specification.
331 - Fixed memory retrieve request attributes by enforcing them to be inner
332 shareable rather than outer.
333 - Fixed static memory mapping of EL3 in EL2.
334 - Fixed a spurious error log message with memory share test.
335 - Aligning RMI FIDs with SMCCC.
336 - Fixed PSCI system suspend test suite execution in a four world system.
337 - Configured the build system to use DWARF 4 standard for debug builds with
338 ArmDS.
339 - Introduced macro IRQ_TWDOG_INTID for the Tegra210, Tegra186 and Tegra194
340 platforms to fix the compilation failures.
341
Madhukar Pappireddyda69f822021-11-09 16:05:54 -0600342Version 2.6
343-----------
344
345New features
346^^^^^^^^^^^^
347- More tests are made available in this release to help validate the
348 functionalities in the following areas:
349
350 - Firmware Framework for Arm A-profile(FF-A)
351 - Realm Management Extensions(RME)
352 - Embedded Trace Extension and Trace Buffer Extension (ETE and TRBE)
353
354TFTF
355~~~~
356
357- FF-A testing:
358
359 - Update FF-A version to v1.1
360 - Added helpers for SPM tests to check partition info of SPs from normal
361 world.
362 - Added tests to check for ffa_features supported.
363 - Added test for FFA_RXTX_UNMAP ABI.
364 - Added test for FFA_SPM_ID_GET.
365 - FF-A v1.1 Notifications
366
367 - Added test for notifications bitmap create and destroy ABIs.
368 - Added test for notifications set and get ABIs.
369 - Added test for notification INFO_GET ABI.
370 - Added test to check notifications pending interrupt is injected into
371 and handled by the expected vCPU in a MP setup.
372 - Added test for signaling from MP SP to UP SP.
373 - Added test to check notifications interrupt IDs retrieved with
374 FFA_FEATURES ABI.
375 - Added test to check functionality of notifications scheduled receiver
376 interrupt.
377
378 - FF-A v1.1 Secure interrupts
379
380 - Added support for handling secure interrupts in Cactus SP.
381 - Added several tests to exercise secure interrupt handling while SP
382 is in WAITING/RUNNING/BLOCKED state.
383
384- New tests:
385
386 - Enabled SVE tests
387 - Added test for trace system registers access.
388 - Added test for trace filter control registers access.
389 - Added test for trace buffer control registers access.
390 - Added test to check PSTATE in SDEI handler.
391 - Added test to check if HCRX_EL2 is accessible.
392
393- Platforms:
394
395 - TC0:
396
397 - Support for direct messaging with managed exit.
398 - Support for building S-EL0 Ivy partition.
399
400 - FVP:
401
402 - Update Cactus secure partitions to indicate Managed exit support.
403
404- Miscellaneous
405
406 - Added random seed generation capability and ability to specify build
407 parameters for SMC Fuzzer tool.
408
409Cactus (Secure-EL1 test partition)
410~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
411
412 - Added helper for Cactus SP sleep.
413 - Added test commands to request use of notifications interfaces.
414 - Added several commands that generate direct message requests to assist in
415 testing secure interrupt handling and notifications features in FF-A v1.1
416 - Added support for SP805 Trusted Watchdog module.
417
418Ivy (Secure-EL1 test partition)
419~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
420
421 - Add shim layer to Ivy partition and enable PIE.
422 - Define Ivy partition manifest and use FF-A for message handling.
423 - Prepare S-EL1/0 enviroment for enabling S-EL0 application.
424
425Realm Management Extension(RME)
426~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
427
428 - Added tests to run RMI and SPM on multiple CPUs concurrently.
429 - Added tests for multi CPU delegation and fail conditions.
430 - Added tests to query RMI version on multiple CPUs.
431
432Issues resolved since last release
433^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
434
435 - Fixed Ivy partition start address for TC0.
436 - Fixed SP manifests to use little endian format UUID.
437 - Fixed a bug in memory sharing test for Cactus SP.
438 - Invalidate data cache for NS_BL1U and NS_BL2U images.
439 - Fixed attributes to Read-Write only for memory regions described in partition
440 manifests.
441
Bipin Ravi7afea032021-05-06 14:32:52 -0500442Version 2.5
443-----------
444
445New features
446^^^^^^^^^^^^
447- More tests are made available in this release to help validate the
448 functionalities in the following areas:
Madhukar Pappireddyb1193f92023-05-18 16:50:41 -0500449
Bipin Ravi7afea032021-05-06 14:32:52 -0500450 - True Random Number Generator (TRNG) test scenarios.
451 - Multicore / Power State Controller Interface (PSCI) tests.
452 - v8.6 Activity Monitors Unit (AMU) enhancements test scenarios.
453 - Secure Partition Manager (SPM) / Firmware Framework (FF-A) v1.0 testing.
454 - Interrupt Handling between Non-secure and Secure world.
455 - Direct messages and memory sharing between Secure Partitions(SP).
456 - Many tests to exercise FF-A v1.0 ABIs.
457 - SPM saving/restoring the NS SIMD context enabling a normal world FF-A
458 endpoint (TFTF) and a secure partition to use SIMD vectors and
459 instructions independently.
460
461TFTF
462~~~~
463
464- SPM / FF-A v1.0 testing.
465 - Refactor FF-A memory sharing tests
466 - Created helper functions to initialize ffa_memory_region and to send
467 the respective memory region to the SP, making it possible to reuse
468 the logic in SP-to-SP memory share tests.
469 - Added comments to document relevant aspects about memory sharing.
470
471 - Trigger direct messaging between SPs.
472 - Use cactus command 'CACTUS_REQ_ECHO_SEND_CMD' to make cactus SPs
473 communicate with each other using direct message interfaces.
474
475 - Added helpers for SPM tests.
476 - Checking SPMC has expected FFA_VERSION.
477 - Checking that expected FF-A endpoints are deployed in the system.
478 - Getting global TFTF mailbox.
479
480- Replace '.inst' AArch64 machine directives with CPU Memory Tagging Extension
481 instructions in 'test_mte_instructions' function.
482
483- Add build option for Arm Feature Modifiers.
484 - This patch adds a new ARM_ARCH_FEATURE build option to add support
485 for compiler's feature modifiers.
486
487- Enable 8 cores support for Theodul DSU(DynamIQ Shared Unit) for the
488 Total Compute (TC0) platform.
489
490- New tests:
491
492 - Remove redundant code and add better tests for TRNG SMCs.
493 - Tests that the Version, Features, and RND calls conform to the spec.
494
495 - New tests for v8.6 AMU enhancements (FEAT_AMUv1p1)
496 - Make sure AMU offsets are being saved and restored properly.
497
498 - Tests to request SP-to-SP memory share.
499
500 - SP-to-SP direct messaging deadlock test.
501 - TFTF sends CACTUS_REQ_DEADLOCK_CMD to cactus SP.
502
503Cactus(Secure-EL1 test partition)
504~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
505
506- Enable managed exit for primary cactus secure partition.
507
508- Helper commands needed for interrupt testing.
509
510- Add handler from managed exit FIQ interrupt.
511
512- Make ffa_id global.
513
514- Implement HF_INTERRUPT_ENABLE Hafnium hypervisor call wrapper. With this
515 service, a secure partition calls into the SPMC to enable/disable a
516 particular virtual interrupt.
517
518- Invalidate the data cache for the cactus image.
519
520- Helper commands needed for interrupt testing.
521 - CACTUS_SLEEP_CMD & CACTUS_INTERRUPT_CMD added.
522
523- Decouple exception handling from tftf framework.
524 - With new interrupt related tests coming up in Cactus, added separate
525 exception handler code for irq/fiq in Cactus.
526
527- Hypervisor calls moved to a separate module.
528
529- Add secondary entry point register function.
530
531- Declare third SP instance as UP SP.
532
533- Provision a cold boot path for secondary cores (or secondary pinned
534 execution contexts).
535
536- Tidy message loop, commands definitions, direct messaging API definitions.
537
538- Helpers for error logging after FF-A calls.
539
540- Properly placing Cactus test files.
541
542- Tidying FF-A Memory Sharing tests.
543
544- Use CACTUS_ECHO_CMD in direct message tests.
545
546- Refactor handling of commands.
547 - Added helper macros to define a command handler, build a command table
548 in which each element is a pair of the handler and respective command
549 ID. Available tests have been moved to their own command handler.
550
551- Extend arguments in commands responses.
552 - In the test commands framework, added template to extend number of
553 values to include in a command response.
554
555- Check FF-A return is a valid direct response.
556 - Added a helper function to check if return of FFA_MSG_SEND_DIRECT_REQ
557 is FFA_MSG_SEND_DIRECT_RESP.
558
559- FFA_MSG_DIRECT_RESP call extended to use 5 registers.
560
561- Added accessors for arguments from FF-A calls.
562 - Some accessors for arguments from FF-A calls, namely for func id, error
563 code, and direct message destination/source.
564
565- Use virtual counter for sp_sleep.
566 - Changes sp_sleep() to use virtual counter instead of physical counter.
567
568- Checks if SIMD vectors are preserved in the normal world while transitioning
569 from normal world to secure world and back to normal world.
570
571- Tidying common code to tftf and cactus.
572
573- Refactor cactus_test_cmds.h to incorporate static inline functions instead
574 of macros to enforce type checking.
575
576- Removed reference to Hafnium in name from helper function and macro to
577 make them generic.
578
579- For consistency added the cmd id 'CACTUS_MEM_SEND_CMD'.
580
581- Add command to request memory sharing between SPs.
582
583- Add & handle commands 'CACTUS_REQ_ECHO_CMD' and 'CACTUS_ECHO_CMD'.
584
585- Update README with list of sample partitions.
586
587- Remove reference to PSA from xml test file.
588
589- Reduce tests verbosity in release mode.
590 - Update few NOTICE messages to VERBOSE/INFO.
591
592- Fix conversion issues on cactus responses.
593
594- Create RXTX map/configure helper macros and use them.
595
596- Update OP-TEE version used for testing to 3.10.
597 - SPMC as S-EL1 tests using OP-TEE depend on a static binary stored as
598 a CI file. This binary corresponds to a build of OP-TEE v3.10.
599
600- Add uart2 to device-regions node.
601 - First SP no longer has an open access to the full system peripheral
602 range and devices must be explicitly declared in the SP manifest.
603
604- New tests:
605
606 - Test for exercising SMMUv3 driver to perform stage2 translation.
607
608 - Test handling of non-secure interrupt while running SP.
609
610 - Add secondary cores direct messaging test for SPM.
611
612 - Testing deadlock by FF-A direct message.
613 - Added command CACTUS_DEADLOCK_CMD to file cactus_test_cmds.h to create
614 a deadlock scenario using FF-A direct message interfaces.
615
616 - Test SP-to-SP memory share operations
617 - Handle 'CACTUS_REQ_MEM_SEND_CMD' by sending memory to the receiver SP.
618
619 - Implemented test to validate FFA_RXTX_MAP ABI.
620
Manish V Badarkhe86600a82020-10-04 18:33:44 +0100621Version 2.4
622-----------
623
624New features
625^^^^^^^^^^^^
626- More tests are made available in this release to help validate the
627 functionalities in the following areas:
628 - SMCCC.
629 - New architecture specific features.
630 - FF-A features.
631 - New platform ports.
632
633- Various improvements to test framework and test suite such as documentation,
634 removing un-necessary dependencies, etc.
635
636TFTF
637~~~~
638
639- Remove dependencies from FVP to generic code by converting some FVP platform
640 specific macros to the common macros.
641
642- Remove make as a package dependency to compile TF-A test code.
643
644- Move defaults values and macro defs in a separate folder from Makefile.
645
646- Allow alternate stdout to be used apart from pl011 UART.
647
648- Get FVP platform's topology from build options to make FVP platform
649 configuration more flexible and eliminate test errors when the platform
650 is configured with number of CPUs less than default values in the makefile.
651
652- Update the FIP corrupt address which is used to corrupt BL2 image that helps
653 to trigger firmware update process.
654
655- Add explicit barrier before sev() in tftf_send_event_common API to avoid
656 core hang.
657
658- Align output properly on issuing make help_tests by removing dashes
659 and sort tests.
660
661- Moved a few FVP and Juno specific defined from common header files to platform
662 specific header files.
663
664- Replace SPCI with PSA FF-A in code as SPCI is now called as FF-A.
665
666- Add owner field to sp_layout generation to differentiate owner of SP which
667 could either be Silicon Provider or Platform provider.
668
669- Add v8.5 Branch Target Identifier(BTI) support in TFTF.
670
671- Remove dependency on SYS_CNT_BASE1 to read the memory mapped timers.
672
673- Enables SError aborts for all CPUs, during their power on sequence.
674
675- Documentation:
676
677 - Use conditional assignment on sphinx variables so that they can be
678 overwritten by environment and/or command line.
679
680 - Add support for documentation build as a target in Makefile.
681
682 - Update list of maintainers.
683
684 - Update documentation to explain how to locally build the documentation.
685
686 - Add .editorconfig from TF-A to define the coding style.
687
688 - Fix documentation to include 'path/to' prefix when specifying tftf.bin on
689 make fip cmd.
690
691 - Use docker to build documentation.
692
693 - Replace SPCI with PSA FF-A in documentation as SPCI is now called
694 as FF-A.
695
696- NVIDIA Tegra194:
697
698 - Skip CPU suspend tests requiring SGI as wake source as Tegra194 platforms
699 do not support CPU suspend power down and cannot be woken up with an SGI.
700
701 - Disable some system suspend test cases.
702
703 - Create dummy SMMU context for system resume to allow the System Resume
704 Firmware to complete without any errors or warnings.
705
706 - Increase RTC step value to 5ms as RTC consumes 250us for each register
707 read/write. Increase the step value to 5ms to cover all the register
708 read/write in program_timer().
709
710 - Skip some timer framework validation tests as CPUs on Tegra194 platforms
711 cannot be woken up with the RTC timer interrupt after power off.
712
713 - Introduce per-CPU Hypervisor Timer Interrupt ID.
714
715 - Skip PSCI STAT tests requiring PSTATE_TYPE_POWERDOWN as Tegra194 platforms
716 do not support CPU suspend with state type as PSTATE_TYPE_POWERDOWN.
717
718 - Disable boot requirement tests as Tegra194 platforms do not support memory
719 mapped timers.
720
721 - Skips the test "Create all power states and validate EL3 power state parsing"
722 from the "EL3 power state parser validation" test suite as it is not in
723 sync with this expectation.
724
725 - Moved reset, timers. wake, watchdog drivers from Tegra194 specific folder to
726 common driver folder so that these drivers can be used for other NVIDIA platforms.
727
728- New tests:
729
730 - Add test for SDEI RM_ANY routing mode.
731
732 - Add initial platform support for TC0.
733
734 - Add SMC fuzzing module test.
735
736 - Add test case for SMCCC_ARCH_SOC_ID feature.
737
738 - Add test that supports ARMv8.6-FGT in TF-A.
739
740 - Add test that supports ARMv8.6-ECV in TF-A.
741
742 - Add test for FFA_VERSION interface.
743
744 - Add test for FFA_FEATURES interface.
745
746 - Add console driver for the TI UART 16550.
747
748 - Add tests for FF-A memory sharing interfaces between tftf
749 and cactus secure partitions.
750
751 - NVIDIA Tegra194:
752
753 - Introduce platform port for Tegra194 to to initialize the tftf
754 framework and execute tests on the CPUs.
755
756 - Introduce power management support.
757
758 - Introduce support for RTC as wake source.
759
760 - Introduce system reset functionality test.
761
762 - Introduce watchdog timer test.
763
764 - Introduce support for NVIDIA Denver CPUs.
765
766 - Introduce RAS uncorrectable error injection test.
767
768 - Introduce tests to verify the Video Memory resize interface.
769
770 - Introduce test to inject RAS corrected errors for all supported
771 nodes from all CPUs.
772
773 - Introduce a test to get return value from SMC SiP function
774 TEGRA_SIP_GET_SMMU_PER.
775
776 - NVIDIA Tegra196:
777
778 - Introduce initial support for Tegra186 platforms.
779
780 - NVIDIA Tegra210:
781
782 - Introduce initial support for Tegra210 platforms.
783
784Secure partition - Cactus
785~~~~~~~~~~~~~~~~~~~~~~~~~
786
787- TFTF doesn't need to boot Secondary Cactus as Hafnium now boots all
788 partitions according to "boot-order" field value in the partition
789 manifests.
790
791- Remove test files related to deprecated SPCI Alpha specification and
792 SPRT interface.
793
794- Select different stdout device at runtime as primary VM can access
795 to UART while secondary VM's use hypervisor call to SPM for debug
796 logging.
797
798- An SP maps its RX/TX buffers in its EL1&0 Stage-1 translation regime.
799 The same RX/TX buffers are mapped by the SPMC in the SP's EL1&0
800 Stage-2 translation regime during boot time.
801
802- Update memory/device region nodes in manifest. Memory region has 3
803 entries such as RX buffer, TX buffer and dummy. These memory region
804 entries are mapped with attributes as "RX buffer: read-only",
805 "TX buffer: read-write" and "dummy: read-write-execute".
806 Device region mapped with read-write attribute.
807
808- Create tertiary partition without RX_TX region specified to test the
809 RXTX_MAP API.
810
811- Add third partition to ffa_partition_info_get test to test that a
812 partition can successfully get information about the third cactus
813 partition.
814
815- Map RXTX region to third partition to point the mailbox to this RXTX
816 region.
817
818- Adjust the number of EC context to max number of PEs as per the FF-A
819 specification mandating that a SP must either "Implement as many ECs
820 as the number of PEs (in case of a "multi-processor" SP with pinned
821 contexts)" or "Implement a single EC (in case of a migratable
822 "uni-processor" SP).
823
824- Updated cactus test payload and TFTF ids as it is decided to have
825 secure partition FF-A ids in the range from 0x8001 to 0xfffe, 0x8000
826 and 0xffff FF-A ids are reserved for the SPMC and the SPMD respectively
827 and in the non-secure worlds, FF-A id 0 is reserved for the hypervisor
828 and 1 to 0x7fff FF-A ids are reserved for VMs.
829
830- Break the message loop on bad message request instead of replying
831 with the FF-A error ABI to the SPMC.
832
833- Remove deprecated hypervisor calls spm_vm_get_count and spm_vcpu_get_count.
834 Instead use FFA_PARTITION_INFO_GET discovery ABI.
835
836- Implement hvc call 'SPM_INTERRUPT_GET' to get interrupt id.
837
838- Re-structure platform dependent files by moving platform dependent files
839 and macros to platform specific folder.
840
841- Adjust partition info get properties to support receipt of direct
842 message request.
843
844- New tests:
845
846 - Add FFA Version Test.
847
848 - Add FFA_FEATURES test.
849
850 - Add FFA_MEM_SHARE test
851
852 - Add FFA_MEM_LEND test.
853
854 - Add FFA_MEM_DONATE test.
855
856 - Add FFA_PARTITION_INFO_GET test.
857
858 - Add exception/interrupt framework.
859
860 - Add cactus support for TC0 platform.
861
862Issues resolved since last release
863^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
864
865- Update link to SMCCC specification.
866
867- Trim down the top-level readme file to give brief overview of the project
868 and also fix/update a number of broken/out-dated links in it.
869
870- Bug fix in Multicore IRQ spurious test.
871
872- Fix memory regions mapping with no NS bit set.
873
874- Reenable PSCI NODE_HW_STATE test which was disabled earlier due to
875 outdated SCP firmware.
876
877- Fix Aarch32 zeromem() function by avoiding infinite loop in 'zeromem'
878 function and optimizing 'memcpy4' function.
879
880- Add missing help_tests info on help target in the top-level Makefile.
881
882- Trim down the readme file as it does not need to provide detailed
883 information, instead it can simply be a landing page providing a brief
884 overview of the project and redirecting the reader to RTD for further
885 information.
886
887- Fix maximum number of CPUs in DSU cluster by setting maximum number of CPUs
888 in DSU cluster to 8.
889
Madhukar Pappireddybced2f32020-04-08 15:44:38 -0500890Version 2.3
891-----------
892
893New features
894^^^^^^^^^^^^
895
896- More tests are made available in this release to help validate
897 the functionality of TF-A.
898
899- CI upgraded to use GCC 9.2-2019.12 toolchain for tf-a-tests.
900
901- Various improvements to test framework and test suite.
902
903TFTF
904~~~~
905
906- Support for extended register usage as per SMCCC v1.2 specification.
907
908- Support for FVP platforms with SMT capabilities.
909
910- Improved support for documentation through addition of basic Sphinx
911 configuration and Makefile similar to TF-A repository.
912
913- Enhancement to libc library synchronous to TF-A code base.
914
915- ARMv8.3-PAuth enabled for all FWU tests in TFTF.
916
917- TFTF made RFC 4122 compliant by converting UUIDs to network order format.
918
919- Build improvement by deprecating custom AARCH64/AARCH32 macros in favor of
920 __arch64__ macro provided by compiler.
921
922- Support for HVC as a SMCCC conduit in TFTF.
923
924- New tests:
925
926 - AArch32 tests for checking if PMU counters leak in secure world.
927
928 - Add new debug filesystem (debugfs) test.
929
930 - Add a SPCI direct messaging test targeting bare-metal cactus SP.
931
932
933Secure partitions
934~~~~~~~~~~~~~~~~~
935
936Cactus
937~~~~~~
938
939- Several build improvements and symbol relocation fixup to make it position
940 independent executable.
941
942- Update of sample manifest to SPCI Beta1 format.
943
944- Support for generating JSON file as required by TF-A.
945
946Issues resolved since last release
947^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
948
949- Makefile bug fix for performing parallel builds.
950
951- Add missing D-cache invalidation of RW memory in tftf_entrypoint to safeguard
952 against possible corruption.
953
954- Fixes in GIC drivers to support base addresses beyond 4G range.
955
956- Fix build with XML::LibXML 2.0202 Perl module
957
958Known issues and limitations
959^^^^^^^^^^^^^^^^^^^^^^^^^^^^
960
961The sections below list the known issues and limitations of each test image
962provided in this repository. Unless and otherwise stated, issues and limitations
963stated in previous release continue to exist in this release.
964
965TFTF
966~~~~
967- NODE_HW_STATE test has been temporarily disabled for sgi575 platform due to a
968 dependency on SCP binaries version 2.5
969
Paul Beesley5c928952019-10-24 11:57:00 +0000970Version 2.2
971-----------
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500972
973New features
Paul Beesley5c928952019-10-24 11:57:00 +0000974^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500975
976- A wide range of tests are made available in this release to help validate
977 the functionality of TF-A.
978
979- Various improvements to test framework and test suite.
980
981TFTF
Paul Beesley5c928952019-10-24 11:57:00 +0000982~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -0500983
984- Enhancement to xlat table library synchronous to TF-A code base.
985
986- Enabled strict alignment checks (SCTLR.A & SCTLR.SA) in all images.
987
988- Support for a simple console driver. Currently it serves as a placeholder
989 with empty functions.
990
991- A topology helper API is added in the framework to get parent node info.
992
993- Support for FVP with clusters having upto 8 CPUs.
994
995- Enhanced linker script to separate code and RO data sections.
996
997- Relax SMC calls tests. The SMCCC specification recommends Trusted OSes to
998 mitigate the risk of leaking information by either preserving the register
999 state over the call, or returning a constant value, such as zero, in each
1000 register. Tests only allowed the former behaviour and have been extended to
1001 allow the latter as well.
1002
1003- Pointer Authentication enabled on warm boot path with individual APIAKey
1004 generation for each CPU.
1005
1006- New tests:
1007
1008 - Basic unit tests for xlat table library v2.
1009
1010 - Tests for validating SVE support in TF-A.
1011
1012 - Stress tests for dynamic xlat table library.
1013
1014 - PSCI test to measure latencies when turning ON a cluster.
1015
1016 - Series of AArch64 tests that stress the secure world to leak sensitive
1017 counter values.
1018
1019 - Test to validate PSCI SYSTEM_RESET call.
1020
1021 - Basic tests to validate Memory Tagging Extensions are being enabled and
1022 ensuring no undesired leak of sensitive data occurs.
1023
1024- Enhanced tests:
1025
1026 - Improved tests for Pointer Authentication support. Checks are performed
1027 to see if pointer authentication keys are accessible as well as validate
1028 if secure keys are being leaked after a PSCI version call or TSP call.
1029
1030 - Improved AMU test to remove unexecuted code iterating over Group1 counters
1031 and fix the conditional check of AMU Group0 counter value.
1032
1033Secure partitions
Paul Beesley5c928952019-10-24 11:57:00 +00001034~~~~~~~~~~~~~~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001035
1036A new Secure Partition Quark is introduced in this release.
1037
1038Quark
Paul Beesley5c928952019-10-24 11:57:00 +00001039~~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001040
1041The Quark test secure partition provided is a simple service which returns a
1042magic number. Further, a simple test is added to test if Quark is functional.
1043
1044Issues resolved since last release
Paul Beesley5c928952019-10-24 11:57:00 +00001045^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001046
1047- Bug fix in libc memchr implementation.
1048
1049- Bug fix in calculation of number of CPUs.
1050
1051- Streamlined SMC WORKAROUND_2 test and fixed a false fail on Cortex-A76 CPU.
1052
1053- Pointer Authentication support is now available for secondary CPUs and the
1054 corresponding tests are stable in this release.
1055
1056Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001057^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001058
1059The sections below list the known issues and limitations of each test image
1060provided in this repository. Unless and otherwise stated, issues and limitations
1061stated in previous release continue to exist in this release.
1062
1063TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001064~~~~
Madhukar Pappireddy74ccffa2019-10-10 02:30:40 -05001065- Multicore spurious interrupt test is observed to have unstable behavior. As a
1066 temporary solution, this test is skipped for AArch64 Juno configurations.
1067
1068- Generating SVE instructions requires `O3` compilation optimization. Since the
1069 current build structure does not allow compilation flag modification for
1070 specific files, the function which tests support for SVE has been pre-compiled
1071 and added as an assembly file.
1072
1073
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001074
Paul Beesley5c928952019-10-24 11:57:00 +00001075Version 2.1
1076-----------
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001077
1078New features
Paul Beesley5c928952019-10-24 11:57:00 +00001079^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001080
1081- Add initial support for testing Secure Partition Client Interface (SPCI)
1082 and Secure Partition Run-Time (SPRT) standards.
1083
1084 Exercise the full communication flow throughout the software stack, involving:
1085
1086 - A Secure-EL0 test partition as the Trusted World agent.
1087
1088 - TFTF as the Normal World agent.
1089
1090 - The Secure Partition Manager (SPM) in TF-A.
1091
1092- Various stability improvements, code refactoring and clean ups.
1093
1094TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001095~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001096
1097- Reorganize tests build infrastructure to allow the selection of a subset of
1098 tests.
1099
1100- Reorganize the platform layer for improved clarity and simplicity.
1101
1102- Sanitise inclusion of drivers header files.
1103
1104- Enhance the test report format for improved clarity and conciseness.
1105
1106- Dump CPU registers when hitting an unexpected exception. Previously, this
1107 would silently loop forever.
1108
1109- Import libc from TF-A to better align the two code bases.
1110
1111- New tests:
1112
1113 - SPM tests for exercising communication through either the MM or SPCI/SPRT
1114 interfaces.
1115
1116 - SMC calling convention tests.
1117
1118 - Initial tests for Armv8.3 Pointer Authentication support (experimental).
1119
1120- New platform ports:
1121
1122 - `Arm SGI-575`_ FVP.
1123
1124 - Hikey960 board (experimental).
1125
1126 - `Arm Neoverse Reference Design N1 Edge (RD-N1-Edge)`_ FVP (experimental).
1127
1128Secure partitions
Paul Beesley5c928952019-10-24 11:57:00 +00001129~~~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001130
1131We now have 3 Secure Partitions to test the SPM implementation in TF-A.
1132
1133Cactus-MM
1134'''''''''
1135
1136The Cactus test secure partition provided in version 2.0 has been renamed into
1137"*Cactus-MM*". It is still responsible for testing the SPM implementation based
1138on the Arm Management Mode Interface.
1139
1140Cactus
1141''''''
1142
1143This is a new test secure partition (as the former "*Cactus*" has been renamed
1144into "*Cactus-MM*", see above).
1145
1146Unlike *Cactus-MM*, this image tests the SPM implementation based on the SPCI
1147and SPRT draft specifications.
1148
1149It runs in Secure-EL0 and performs the following tasks:
1150
1151- Test that TF-A has correctly setup the secure partition environment (access
1152 to cache maintenance operations, to floating point registers, etc.)
1153
1154- Test that TF-A accepts to change data access permissions and instruction
1155 permissions on behalf of Cactus for memory regions the latter owns.
1156
1157- Test communication with SPM through SPCI/SPRT interfaces.
1158
1159Ivy
1160'''
1161
1162This is also a new test secure partition. It is provided in order to test
1163multiple partitions support in TF-A. It is derived from Cactus and essentially
1164provides the same services but with different identifiers at the moment.
1165
1166EL3 payload
Paul Beesley5c928952019-10-24 11:57:00 +00001167~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001168
1169- New platform ports:
1170
1171 - `Arm SGI-575`_ FVP.
1172
1173 - `Arm Neoverse Reference Design N1 Edge (RD-N1-Edge)`_ FVP (experimental).
1174
1175Issues resolved since last release
Paul Beesley5c928952019-10-24 11:57:00 +00001176^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001177
1178- The GICv2 spurious IRQ test is no longer Juno-specific. It is now only
1179 GICv2-specific.
1180
1181- The manual tests in AArch32 state now work properly. After investigation,
1182 we identified that this issue was not AArch32 specific but concerned any
1183 test relying on state information persisting across reboots. It was due to
1184 an incorrect build configuration.
1185
1186- Cactus-MM now successfully links with GNU toolchain 7.3.1.
1187
1188Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001189^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001190
1191The sections below lists the known issues and limitations of each test image
1192provided in this repository.
1193
1194TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001195~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001196
1197The TFTF test image might be conceptually sub-divided further in 2 parts: the
1198tests themselves, and the test framework they are based upon.
1199
1200Test framework
Paul Beesley5c928952019-10-24 11:57:00 +00001201~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001202
1203- Some stability issues.
1204
1205- No mechanism to abort tests when they time out (e.g. this could be
1206 implemented using a watchdog).
1207
1208- No convenient way to include or exclude tests on a per-platform basis.
1209
1210- Power domains and affinity levels are considered equivalent but they may
1211 not necessarily be.
1212
1213- Need to provide better support to alleviate duplication of test code. There
1214 are some recurrent test patterns for which helper functions should be
1215 provided. For example, bringing up all CPUs on the platform and executing the
1216 same function on all of them, or programming an interrupt and waiting for it
1217 to trigger.
1218
1219- Every CPU that participates in a test must return from the test function. If
1220 it does not - e.g. because it powered itself off for testing purposes - then
1221 the test framework will wait forever for this CPU. This limitation is too
1222 restrictive for some tests.
1223
1224- No protection against interrupted flash operations. If the target is reset
1225 while some data is written to flash, the test framework might behave
1226 incorrectly on reset.
1227
1228- When compiling the code, if the generation of the ``tests_list.c`` and/or
1229 ``tests_list.h`` files fails, the build process is not aborted immediately
1230 and will only fail later on.
1231
1232- The directory layout requires further improvements. Most of the test
1233 framework code has been moved under the ``tftf/`` directory to better isolate
1234 it but this effort is not complete. As a result, there are still some TFTF
1235 files scattered around.
1236
1237- Pointer Authentication testing is experimental and incomplete at this stage.
1238 It is only enabled on the primary CPU on the cold boot.
1239
1240Tests
Paul Beesley5c928952019-10-24 11:57:00 +00001241~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001242
1243- Some tests are implemented for AArch64 only and are skipped on AArch32.
1244
1245- Some tests are not robust enough:
1246
1247 - Some tests might hang in some circumstances. For example, they might wait
1248 forever for a condition to become true.
1249
1250 - Some tests rely on arbitrary time delays instead of proper synchronization
1251 when executing order-sensitive steps.
1252
1253 - Some tests have been implemented in a practical manner: they seem to work
1254 on actual hardware but they make assumptions that are not guaranteed by
1255 the Arm architecture. Therefore, they might fail on some other platforms.
1256
1257- PSCI stress tests are very unreliable and will often hang. The root cause is
1258 not known for sure but this might be due to bad synchronization between CPUs.
1259
1260- The GICv2 spurious IRQ test sometimes fails with the following error message:
1261
1262 ``SMC @ lead CPU returned 0xFFFFFFFF 0x8 0xC``
1263
1264 The root cause is unknown.
1265
1266- The FWU tests take a long time to complete. This is because they wait for the
1267 watchdog to reset the system. On FVP, TF-A configures the watchdog period to
1268 about 4 min. This limit is excessive for an automated testing context and
1269 leaves the user without feedback and unable to determine if the tests are
1270 proceeding properly.
1271
1272- The test "Target timer to a power down cpu" sometimes fails with the
1273 following error message:
1274
1275 ``Expected timer switch: 4 Actual: 3``
1276
1277 The root cause is unknown.
1278
1279FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001280~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001281
1282- The FWU tests do not work on the revC of the Base AEM FVP. They only work on
1283 the revB.
1284
1285- NS-BL1U and NS-BL2U images reuse TFTF-specific code for legacy reasons. This
1286 is not a clean design and may cause confusion.
1287
1288Test secure partitions (Cactus, Cactus-MM, Ivy)
Paul Beesley5c928952019-10-24 11:57:00 +00001289~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001290
1291- This is experimental code. It's likely to change a lot as the secure
1292 partition software architecture evolves.
1293
1294- Supported on AArch64 FVP platform only.
1295
1296All test images
Paul Beesley5c928952019-10-24 11:57:00 +00001297~~~~~~~~~~~~~~~
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001298
1299- TF-A Tests are derived from a fork of TF-A so:
1300
1301 - they've got some code in common but lag behind on some features.
1302
1303 - there might still be some irrelevant references to TF-A.
1304
1305- Some design issues.
1306 E.g. TF-A Tests inherited from the I/O layer of TF-A, which still needs a
1307 major rework.
1308
1309- Cannot build TF-A Tests with Clang. Only GCC is supported.
1310
1311- The build system does not cope well with parallel building. The user should
1312 not attempt to run multiple jobs in parallel with the ``-j`` option of `GNU
1313 make`.
1314
1315- The build system does not properly track build options. A clean build must be
1316 performed every time a build option changes.
1317
1318- UUIDs are not compliant to RFC 4122.
1319
1320- No floating point support. The code is compiled with GCC flag
1321 ``-mgeneral-regs-only``, which prevents the compiler from generating code
1322 that accesses floating point registers. This might limit some test scenarios.
1323
1324- The documentation is too lightweight.
1325
1326- Missing instruction barriers in some places before reading the system counter
1327 value. As a result, the CPU could speculatively read it and any delay loop
1328 calculations might be off (because based on stale values). We need to examine
1329 all such direct reads of the ``CNTPCT_EL0`` register and replace them with a
1330 call to ``syscounter_read()`` where appropriate.
1331
Paul Beesley5c928952019-10-24 11:57:00 +00001332Version 2.0
1333-----------
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001334
1335New features
Paul Beesley5c928952019-10-24 11:57:00 +00001336^^^^^^^^^^^^
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001337
1338This is the first public release of the Trusted Firmware-A Tests source code.
1339
1340TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001341~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001342
1343- Provides a baremetal test framework to exercise TF-A features through its
1344 ``SMC`` interface.
1345
1346- Integrates easily with TF-A: the TFTF binary is packaged in the FIP image
1347 as a ``BL33`` component.
1348
1349- Standalone binary that runs on the target without human intervention (except
1350 for some specific tests that require a manual target reset).
1351
1352- Designed for multi-core testing. The various sub-frameworks allow maximum
1353 parallelism in order to stress the firmware.
1354
1355- Displays test results on the UART output. This may then be parsed by an
1356 external tool and integrated in a continuous integration system.
1357
1358- Supports running in AArch64 (NS-EL2 or NS-EL1) and AArch32 states.
1359
1360- Supports parsing a tests manifest (XML file) listing the tests to include in
1361 the binary.
1362
1363- Detects most platform features at run time (e.g. topology, GIC version, ...).
1364
1365- Provides a topology enumeration framework. Allows tests to easily go through
1366 affinity levels and power domain nodes.
1367
1368- Provides an event framework to synchronize CPU operations in a multi-core
1369 context.
1370
1371- Provides a timer framework. Relies on a single global timer to generate
1372 interrupts for all CPUs in the system. This allows tests to easily program
1373 interrupts on demand to use as a wake-up event source to come out of CPU
1374 suspend state for example.
1375
1376- Provides a power-state enumeration framework. Abstracts the valid power
1377 states supported on the platform.
1378
1379- Provides helper functions for power management operations (CPU hotplug,
1380 CPU suspend, system suspend, ...) with proper saving of the hardware state.
1381
1382- Supports rebooting the platform at the end of each test for greater
1383 independence between tests.
1384
1385- Supports interrupting and resuming a test session. This relies on storing
1386 test results in non-volatile memory (e.g. flash).
1387
1388FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001389~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001390
1391- Provides example code to exercise the Firmware Update feature of TF-A.
1392
1393- Tests the robustness of the FWU state machine implemented in the TF-A by
1394 sending valid and invalid authentication, copy and image execution requests
1395 to the TF-A BL1 image.
1396
1397EL3 test payload
Paul Beesley5c928952019-10-24 11:57:00 +00001398~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001399
1400- Tests the ability of TF-A to load an EL3 payload.
1401
1402Cactus test secure partition
Paul Beesley5c928952019-10-24 11:57:00 +00001403~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001404
1405- Tests that TF-A has correctly setup the secure partition environment: it
1406 should be allowed to perform cache maintenance operations, access floating
1407 point registers, etc.
1408
1409- Tests the ability of a secure partition to request changing data access
1410 permissions and instruction permissions of memory regions it owns.
1411
1412- Tests the ability of a secure partition to handle StandaloneMM requests.
1413
1414Known issues and limitations
Paul Beesley5c928952019-10-24 11:57:00 +00001415^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001416
1417The sections below lists the known issues and limitations of each test image
1418provided in this repository.
1419
1420TFTF
Paul Beesley5c928952019-10-24 11:57:00 +00001421~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001422
1423The TFTF test image might be conceptually sub-divided further in 2 parts: the
1424tests themselves, and the test framework they are based upon.
1425
1426Test framework
Paul Beesley5c928952019-10-24 11:57:00 +00001427~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001428
1429- Some stability issues.
1430
1431- No mechanism to abort tests when they time out (e.g. this could be
1432 implemented using a watchdog).
1433
1434- No convenient way to include or exclude tests on a per-platform basis.
1435
1436- Power domains and affinity levels are considered equivalent but they may
1437 not necessarily be.
1438
1439- Need to provide better support to alleviate duplication of test code. There
1440 are some recurrent test patterns for which helper functions should be
1441 provided. For example, bringing up all CPUs on the platform and executing the
1442 same function on all of them, or programming an interrupt and waiting for it
1443 to trigger.
1444
1445- Every CPU that participates in a test must return from the test function. If
1446 it does not - e.g. because it powered itself off for testing purposes - then
1447 the test framework will wait forever for this CPU. This limitation is too
1448 restrictive for some tests.
1449
1450- No protection against interrupted flash operations. If the target is reset
1451 while some data is written to flash, the test framework might behave
1452 incorrectly on reset.
1453
1454- When compiling the code, if the generation of the tests_list.c and/or
1455 tests_list.h files fails, the build process is not aborted immediately and
1456 will only fail later on.
1457
1458- The directory layout is confusing. Most of the test framework code has been
1459 moved under the ``tftf/`` directory to better isolate it but this effort is
1460 not complete. As a result, there are still some TFTF files scattered around.
1461
1462Tests
Paul Beesley5c928952019-10-24 11:57:00 +00001463~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001464
1465- Some tests are implemented for AArch64 only and are skipped on AArch32.
1466
1467- Some tests are not robust enough:
1468
1469 - Some tests might hang in some circumstances. For example, they might wait
1470 forever for a condition to become true.
1471
1472 - Some tests rely on arbitrary time delays instead of proper synchronization
1473 when executing order-sensitive steps.
1474
1475 - Some tests have been implemented in a practical manner: they seem to work
1476 on actual hardware but they make assumptions that are not guaranteed by
1477 the Arm architecture. Therefore, they might fail on some other platforms.
1478
1479- PSCI stress tests are very unreliable and will often hang. The root cause is
1480 not known for sure but this might be due to bad synchronization between CPUs.
1481
1482- The GICv2 spurious IRQ test is Juno-specific. In reality, it should only be
1483 GICv2-specific. It should be reworked to remove any platform-specific
1484 assumption.
1485
1486- The GICv2 spurious IRQ test sometimes fails with the following error message:
1487
1488 ``SMC @ lead CPU returned 0xFFFFFFFF 0x8 0xC``
1489
1490 The root cause is unknown.
1491
1492- The manual tests in AArch32 mode do not work properly. They save some state
1493 information into non-volatile memory in order to detect the reset reason but
1494 this state does not appear to be retained. As a result, these tests keep
1495 resetting infinitely.
1496
1497- The FWU tests take a long time to complete. This is because they wait for the
1498 watchdog to reset the system. On FVP, TF-A configures the watchdog period to
1499 about 4 min. This is way too long in an automated testing context. Besides,
1500 the user gets not feedback, which may let them think that the tests are not
1501 working properly.
1502
1503- The test "Target timer to a power down cpu" sometimes fails with the
1504 following error message:
1505
1506 ``Expected timer switch: 4 Actual: 3``
1507
1508 The root cause is unknown.
1509
1510FWU images
Paul Beesley5c928952019-10-24 11:57:00 +00001511~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001512
Sandrine Bailleux7af6c6d2018-10-22 17:09:07 +02001513- The FWU tests do not work on the revC of the Base AEM FVP. They only work on
1514 the revB.
1515
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001516- NS-BL1U and NS-BL2U images reuse TFTF-specific code for legacy reasons. This
1517 is not a clean design and may cause confusion.
1518
1519Cactus test secure partition
Paul Beesley5c928952019-10-24 11:57:00 +00001520~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001521
1522- Cactus is experimental code. It's likely to change a lot as the secure
1523 partition software architecture evolves.
1524
1525- Fails to link with GNU toolchain 7.3.1.
1526
1527- Cactus is supported on AArch64 FVP platform only.
1528
1529All test images
Paul Beesley5c928952019-10-24 11:57:00 +00001530~~~~~~~~~~~~~~~
Sandrine Bailleux3cd87d72018-10-09 11:12:55 +02001531
1532- TF-A Tests are derived from a fork of TF-A so:
1533
1534 - they've got some code in common but lag behind on some features.
1535
1536 - there might still be some irrelevant references to TF-A.
1537
1538- Some design issues.
1539 E.g. TF-A Tests inherited from the I/O layer of TF-A, which still needs a
1540 major rework.
1541
1542- Cannot build TF-A Tests with Clang. Only GCC is supported.
1543
1544- The build system does not cope well with parallel building. The user should
1545 not attempt to run multiple jobs in parallel with the ``-j`` option of `GNU
1546 make`.
1547
1548- The build system does not properly track build options. A clean build must be
1549 performed every time a build option changes.
1550
1551- SMCCC v2 is not properly supported.
1552
1553- UUIDs are not compliant to RFC 4122.
1554
1555- No floating point support. The code is compiled with GCC flag
1556 ``-mgeneral-regs-only``, which prevents the compiler from generating code
1557 that accesses floating point registers. This might limit some test scenarios.
1558
1559- The documentation is too lightweight.
1560
1561--------------
1562
Jayanth Dodderi Chidanand090491b2022-05-03 15:05:11 +01001563*Copyright (c) 2018-2022, Arm Limited. All rights reserved.*
Sandrine Bailleux88e96a12019-03-28 09:34:55 +01001564
1565.. _Arm Neoverse Reference Design N1 Edge (RD-N1-Edge): https://developer.arm.com/products/system-design/reference-design/neoverse-reference-design
1566.. _Arm SGI-575: https://developer.arm.com/products/system-design/fixed-virtual-platforms