refactor(psci): unify coherency exit between AArch64 and AArch32
The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't, then we
must turn data caches off, handle the confusion that causes with the
stack, and call into the cpu driver which will flush the caches that
need flushing.
On AArch32 the above happens in common code. On AArch64, however, the
turning off of the caches happens in the cpu driver. Since we're dealing
with the stack, we must exercise control over it and implement this in
assembly. But as the two implementations are nominally different (in the
ordering of operations), the part that is in assembly is quite large as
jumping back to C to handle the difference might involve the stack.
Presumably, the AArch difference was introduced in order to cater for a
possible implementation where turning off the caches requires an IMP DEF
sequence. Well, Arm no longer makes cores without hardware assisted
coherency, so this eventually is not possible.
So take this part out of the cpu driver and put it into common code,
just like in AArch32. With this, there is no longer a need call
prepare_cpu_pwr_dwn() in a different order either - we can delay it a
bit to happen after the stack management. So the two AArch-s flows
become identical. We can convert prepare_cpu_pwr_dwn() to C and leave
psci_do_pwrdown_cache_maintenance() only to exercise control over stack.
Change-Id: Ie4759ebe20bb74b60533c6a47dbc2b101875900f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
diff --git a/lib/psci/psci_common.c b/lib/psci/psci_common.c
index 7c83a79..15a32f1 100644
--- a/lib/psci/psci_common.c
+++ b/lib/psci/psci_common.c
@@ -1196,6 +1196,26 @@
return (n_valid > 1U) ? 1 : 0;
}
+static void call_cpu_pwr_dwn(unsigned int power_level)
+{
+ struct cpu_ops *ops = get_cpu_data(cpu_ops_ptr);
+
+ /* Call the last available power down handler */
+ if (power_level > CPU_MAX_PWR_DWN_OPS - 1) {
+ power_level = CPU_MAX_PWR_DWN_OPS - 1;
+ }
+
+ assert(ops != NULL);
+ assert(ops->pwr_dwn_ops[power_level] != NULL);
+
+ return ops->pwr_dwn_ops[power_level]();
+}
+
+static void prepare_cpu_pwr_dwn(unsigned int power_level)
+{
+ call_cpu_pwr_dwn(power_level);
+}
+
/*******************************************************************************
* Initiate power down sequence, by calling power down operations registered for
* this CPU.
@@ -1213,26 +1233,24 @@
PMF_CACHE_MAINT);
#endif
-#if HW_ASSISTED_COHERENCY
+#if !HW_ASSISTED_COHERENCY
/*
- * With hardware-assisted coherency, the CPU drivers only initiate the
- * power down sequence, without performing cache-maintenance operations
- * in software. Data caches enabled both before and after this call.
- */
- prepare_cpu_pwr_dwn(power_level);
-#else
- /*
- * Without hardware-assisted coherency, the CPU drivers disable data
- * caches, then perform cache-maintenance operations in software.
+ * Disable data caching and handle the stack's cache maintenance.
*
- * This also calls prepare_cpu_pwr_dwn() to initiate power down
- * sequence, but that function will return with data caches disabled.
- * We must ensure that the stack memory is flushed out to memory before
- * we start popping from it again.
+ * If the core can't automatically exit coherency, the cpu driver needs
+ * to flush caches and exit coherency. We can't do this with data caches
+ * enabled. The cpu driver will decide which caches to flush based on
+ * the power level.
+ *
+ * If automatic coherency management is possible, we can keep data
+ * caches on until the very end and let hardware do cache maintenance.
*/
- psci_do_pwrdown_cache_maintenance(power_level);
+ psci_do_pwrdown_cache_maintenance();
#endif
+ /* Initiate the power down sequence by calling into the cpu driver. */
+ prepare_cpu_pwr_dwn(power_level);
+
#if ENABLE_RUNTIME_INSTRUMENTATION
PMF_CAPTURE_TIMESTAMP(rt_instr_svc,
RT_INSTR_EXIT_CFLUSH,