Fix mm_ptable_defrag to use break-before-make when required.
mm_ptable_defrag and related functions write/replace page table entries
directly without using break-before-make sequence. At least for aarch64
architecture, break-before-make sequence must be used when there is a
change in size of block used in the translation which is what happens
during a defrag. This patch fixes this issue by updating the defrag code
to use mm_replace_entry when replacing entries to ensure
break-before-make is used.
This patch also fixes couple of other issues:
1) mm_replace_entry only invalidates the TLB when an existing entry is
replaced with a new valid entry. However, the TLB invalidation also
needs to be done when replacing an entry from a valid entry to an
invalid/absent entry as well.
2) mm_ptable_identity_map includes a TLB invalidate at the end, but is
not necessary because mm_replace_entry has the appropriate TLB
invalidates for valid entries and it is expected that invalid entries
are never cached in the TLB, thus making the invalidate at the end of
this function unnecessary.
3) The aarch64 tlb invalidate functions arch_mm_invalidate*, used
dsh(ishst) for order page table writes, but ordering stores is not
sufficient since an older load could be reordered after the dsb(ishst)
and fault. Changed the code to use a full dsb(ish) to order both loads
and stores. This is also the sequence used in the ARMv8 ARM.
This issue was reported on the mailing list at:
https://lists.trustedfirmware.org/pipermail/hafnium/2021-January/000120.html
Signed-off-by: Raghu Krishnamurthy <raghu.ncstate@icloud.com>
Change-Id: Icaab09b61facdfe7d5f208377551f5ee35655c11
diff --git a/inc/hf/arch/mm.h b/inc/hf/arch/mm.h
index 1d6f841..3c4e27a 100644
--- a/inc/hf/arch/mm.h
+++ b/inc/hf/arch/mm.h
@@ -161,3 +161,9 @@
* Return the arch specific mm mode for send/recv pages of given VM ID.
*/
uint32_t arch_mm_extra_attributes_from_vm(ffa_vm_id_t id);
+
+/**
+ * Execute any barriers or synchronization that is required
+ * by a given architecture, after page table writes.
+ */
+void arch_mm_sync_table_writes(void);