# Testing ## One-command HIL test ``` nix develop --command ./hil/run.sh --duration 300 ``` Performs: device discovery (aborts on ambiguity) -> SDR USB reset -> firmware build + flash -> DUT configuration -> concurrent serial + SDR capture -> correlation -> report (text + report.json). Exit 0 = PASS. Useful flags: - `--no-flash` reuse the already-flashed firmware - `--duration N` capture window seconds - `--tx-power N` configured SX1262 dBm (-9..14; FEM adds gain!) - `--delay MS` repeat delay (20..200) - `--gain G` SDR tuner gain - `--min-repeats N` PASS threshold ## Artifacts Each run writes `captures/-hil/`: ``` serial.log DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats) rtlwmbus.log rtl_wmbus decodes report.json machine-readable correlation result report.txt human-readable report setup.log DUT configuration acks ``` ## Manual pieces ``` ./tools/discover_devices.sh -v # device inventory ./tools/flash.sh # build + flash ./hil/capture_sdr.sh 60 /tmp/sdr.log # SDR-only capture ./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats ./result-py/bin/python3 hil/iq_energy.py iq.raw # burst energy probe ./result-py/bin/python3 hil/iq_spectrum.py iq.raw # spectrum sanity ./result-py/bin/python3 hil/sdr_health.py # observer sensitivity ./firmware/test/run_tests.sh # frame-layer tests (ASan) ``` ## DUT serial commands All output is single-line JSON; every line the HIL parses starts with `{`. ``` status current configuration + duty-cycle usage stats all counters (incl. tx_airtime_us, tx_duty_cycle_ppm) clear_stats radio_status chip mode, command status, device errors, RSSI radio_reset queue a full radio re-init probe low-level scripted SPI/IRQ/TX diagnostic with raw dumps test_tx transmit one synthetic C1 frame test_burst transmit the same frame twice, repeat_delay_ms apart inject standby fault injection: leave the radio out of RX inject sleep fault injection: destroy the radio configuration reboot set repeat_delay_ms N 20..200 set tx_power N -9..14 (configured SX1262 level; FEM adds gain) set repeat_enabled 0|1 set verbose 0|1 set fem_rx_csd 0|1 KCT8103L CSD level while in RX (LNA experiment) set pa_low 0|1 1 = lowest documented PA table set preamble_bits N 0|8|16|24|32 preamble detector length set duty_budget_ppm N rolling-hour TX airtime cap; 1000 = 0.1 %, 0 = off set test_lfield N 19..200, length of synthetic test frames set whitelist ID[,ID...] | set whitelist clear ``` ## Known pitfalls - SDR PLL wedge -> gain-independent white noise; `usbreset 0bda:2838`. - Never wrap the rtl pipeline in `timeout bash -c` (orphans children); apply timeout to `rtl_sdr` itself. - ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash scripts rediscover the port on failure. ## LNA / FEM A/B experiment (no SDR needed) ``` nix develop --command ./hil/lna_ab.sh 600 # seconds per arm ``` Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled and reports frame counts, CRC success, unique meters and RSSI per arm. Result: docs/RF_FRONTEND_TESTS.md. ## Interpreting observer state Every run records `SDR sensitivity: full | degraded` from `hil/sdr_health.py`. Treat cross-run comparisons as invalid unless both runs report the same state -- this dongle loses ~17 dB within minutes of streaming (docs/HARDWARE.md). ## Long reliability soak (DUT only, no SDR) ``` nix develop --command ./hil/soak.sh 9000 # seconds ``` Measures firmware-side reliability over hours: repeat success rate, timing stability, radio recoveries, config-integrity failures and MCU resets, and reports progress against the milestone targets. Deliberately does not use the SDR, which cannot stream for hours without faulting. ## Fault injection (verifying the recovery ladder) ``` ./result-py/bin/python3 hil/dut_cmd.py --listen 26 "inject standby" ./result-py/bin/python3 hil/dut_cmd.py --listen 26 "inject sleep" ``` `standby` leaves the radio out of RX with its configuration intact; `sleep` additionally destroys the configuration. The heartbeat probe must detect both and the recovery ladder must restore RX. Verified behaviour: ``` inject standby -> heartbeat chip_mode=2, config_intact=true -> radio_recovery level 1 -> rx_restored inject sleep -> heartbeat chip_mode=0, config_intact=false -> radio_recovery level 1 -> failed (correctly refuses) -> radio_recovery level 2 -> rx_restored ``` ## Build profiles ``` ./tools/build.sh test # default; the configuration all HIL numbers use ./tools/build.sh debug # verbose diagnostics, assertions, per-TX dumps ./tools/flash.sh [test|debug] ``` Differences are behavioural, not cosmetic: | | test | debug | |---|---|---| | `verbose` at boot | off | on | | `tx_diag` after every TX | only on failure | always | | raw frame hex in events | yes (HIL needs it) | yes | | optimisation / log level | size, warn | debug, info | | task watchdog | 15 s | 30 s | `hil/run.sh` always builds and flashes the `test` profile, so reported timing is never perturbed by debug serial traffic. Build directories are separate (`build-test`, `build-debug`) so profiles cannot silently reuse a stale sdkconfig.