blob: 851fbd67bdd79e3340a9c796bf7182ea61a07583 [file] [log] [blame] [view]
Andrew Walbran6e524d72019-11-12 17:36:57 +00001# Scheduler VM expectations
2
3Hafnium requires there to be a special 'primary' or 'scheduler' VM which is
4responsible for scheduling the other VMs. There are some particular expectations
5on this VM that are required for the rest of the system to function normally.
6
Andrew Walbranb7849972019-11-15 15:23:43 +00007[TOC]
8
Andrew Walbran6e524d72019-11-12 17:36:57 +00009## Scheduling
10
11The scheduler VM is responsible for scheduling the vCPUs of all the other VMs.
12It should request information about the VMs in the system using the
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010013`FFA_PARTITION_INFO_GET` function, and then schedule their vCPUs as it wishes.
Andrew Walbran6e524d72019-11-12 17:36:57 +000014The recommended way of doing this is to create a kernel thread for each vCPU,
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010015which will repeatedly run that vCPU by calling `FFA_RUN`.
Andrew Walbran6e524d72019-11-12 17:36:57 +000016
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010017`FFA_RUN` will return one of several possible functions, which must be handled
Andrew Walbran6e524d72019-11-12 17:36:57 +000018as follows:
19
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010020### `FFA_INTERRUPT`
Andrew Walbran6e524d72019-11-12 17:36:57 +000021
22The vCPU has been preempted but still has work to do. If the scheduling quantum
Andrew Walbrana0e30b02020-10-22 15:55:45 +010023has not expired, the scheduler MUST call `FFA_RUN` on the vCPU to allow it to
24continue.
Andrew Walbran6e524d72019-11-12 17:36:57 +000025
Andrew Walbran7e824602020-10-22 16:51:40 +010026If `w1` is non-zero, then Hafnium would like `FFA_RUN` to be called on the vCPU
27specified there. The scheduler MUST either wake the vCPU in question up if it is
28blocked, or preempt and re-run it if it is already running somewhere. This gives
29Hafnium a chance to update any CPU state which might have changed. The scheduler
30should call `FFA_RUN` again on the sending VM as usual.
31
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010032### `FFA_YIELD`
Andrew Walbran6e524d72019-11-12 17:36:57 +000033
34The vCPU has voluntarily yielded the CPU. The scheduler SHOULD take a scheduling
Andrew Walbrana0e30b02020-10-22 15:55:45 +010035decision to give cycles to those that need them but MUST call `FFA_RUN` on the
36vCPU at a later point.
Andrew Walbran6e524d72019-11-12 17:36:57 +000037
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010038### `FFA_MSG_WAIT`
Andrew Walbran6e524d72019-11-12 17:36:57 +000039
40The vCPU is blocked waiting for a message. The scheduler MUST take it off the
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010041run queue and not call `FFA_RUN` on the vCPU until it has either:
Andrew Walbran6e524d72019-11-12 17:36:57 +000042
43* injected an interrupt
44* sent it a message
Andrew Walbran7e824602020-10-22 16:51:40 +010045* received `FFA_INTERRUPT` for it from another vCPU
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010046* the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
Andrew Walbran6e524d72019-11-12 17:36:57 +000047 specified duration has expired.
48
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010049### `FFA_MSG_SEND`
Andrew Walbran6e524d72019-11-12 17:36:57 +000050
51A message has been sent by the vCPU. If the recipient is the scheduler VM itself
52then it can handle it as it pleases. Otherwise the scheduler MUST run a vCPU
53from the recipient VM and priority SHOULD be given to those vCPUs that are
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010054waiting for a message. The scheduler should call FFA_RUN again on the sending
Andrew Walbran6e524d72019-11-12 17:36:57 +000055VM as usual.
56
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010057### `FFA_RX_RELEASE`
Andrew Walbran6e524d72019-11-12 17:36:57 +000058
59The vCPU has made the mailbox writable and there are pending waiters. The
60scheduler MUST call `hf_mailbox_waiter_get()` repeatedly and notify all waiters
61by injecting an `HF_MAILBOX_WRITABLE_INTID` interrupt. The scheduler should call
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010062FFA_RUN again on the sending VM as usual.
Andrew Walbran6e524d72019-11-12 17:36:57 +000063
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010064### `HF_FFA_RUN_WAIT_FOR_INTERRUPT`
Andrew Walbran6e524d72019-11-12 17:36:57 +000065
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010066_This is a Hafnium-specific function not part of the FF-A standard._
Andrew Walbran6e524d72019-11-12 17:36:57 +000067
68The vCPU is blocked waiting for an interrupt. The scheduler MUST take it off the
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010069run queue and not call `FFA_RUN` on the vCPU until it has either:
Andrew Walbran6e524d72019-11-12 17:36:57 +000070
71* injected an interrupt
Andrew Walbran7e824602020-10-22 16:51:40 +010072* received `FFA_INTERRUPT` for it from another vCPU
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010073* the timeout provided in `w2` is not `FFA_SLEEP_INDEFINITE` and the
Andrew Walbran6e524d72019-11-12 17:36:57 +000074 specified duration has expired.
75
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010076### `FFA_ERROR`
Andrew Walbran6e524d72019-11-12 17:36:57 +000077
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010078#### `FFA_ABORTED`
Andrew Walbran6e524d72019-11-12 17:36:57 +000079
80The vCPU has aborted triggering the whole VM to abort. The scheduler MUST treat
Andrew Walbran7e824602020-10-22 16:51:40 +010081this the same as `FFA_INTERRUPT` for all the other vCPUs of the VM. For this
82vCPU the scheduler SHOULD either never call FFA_RUN on the vCPU again, or treat
83it the same as `HF_FFA_RUN_WAIT_FOR_INTERRUPT`.
Andrew Walbran6e524d72019-11-12 17:36:57 +000084
85#### Any other error code
86
Andrew Walbranb5ab43c2020-04-30 11:32:54 +010087This should not happen if the scheduler VM has called `FFA_RUN` correctly, but
Andrew Walbran6e524d72019-11-12 17:36:57 +000088in case there is some other error it should be logged. The scheduler SHOULD
89either try again or suspend the vCPU indefinitely.
90
91## Interrupt handling
92
93The scheduler VM is responsible for handling all hardware interrupts. Many of
94these will be intended for the scheduler VM itself and it can handle them as
95usual. However, it must also:
96
97* Enable, handle and ignore interrupts for the non-secure hypervisor physical
98 timer (PPI 10, IRQ 26).
99* Forward interrupts intended for secondary VMs to an appropriate vCPU of the
100 VM by calling `hf_interrupt_inject` and then running the vCPU as usual with
Andrew Walbranb5ab43c2020-04-30 11:32:54 +0100101 `FFA_RUN`. (If the vCPU is already running at the time that
Andrew Walbran6e524d72019-11-12 17:36:57 +0000102 `hf_interrupt_inject` is called then it must be preempted and run again so
103 that Hafnium can inject the interrupt.)