docs: fault-injection evidence for recovery ladder

This commit is contained in:
Lars Nolden
2026-09-01 11:21:04 +02:00
parent e131687264
commit 70762362a4
2 changed files with 45 additions and 0 deletions
+30
View File
@@ -72,3 +72,33 @@ Every run records `SDR sensitivity: full | degraded` from
`hil/sdr_health.py`. Treat cross-run comparisons as invalid unless both
runs report the same state -- this dongle loses ~17 dB within minutes of
streaming (docs/HARDWARE.md).
## Long reliability soak (DUT only, no SDR)
```
nix develop --command ./hil/soak.sh 9000 # seconds
```
Measures firmware-side reliability over hours: repeat success rate, timing
stability, radio recoveries, config-integrity failures and MCU resets, and
reports progress against the milestone targets. Deliberately does not use
the SDR, which cannot stream for hours without faulting.
## Fault injection (verifying the recovery ladder)
```
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject standby"
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject sleep"
```
`standby` leaves the radio out of RX with its configuration intact;
`sleep` additionally destroys the configuration. The heartbeat probe must
detect both and the recovery ladder must restore RX. Verified behaviour:
```
inject standby -> heartbeat chip_mode=2, config_intact=true
-> radio_recovery level 1 -> rx_restored
inject sleep -> heartbeat chip_mode=0, config_intact=false
-> radio_recovery level 1 -> failed (correctly refuses)
-> radio_recovery level 2 -> rx_restored
```