6.2 KiB
Testing
One-command HIL test
nix develop --command ./hil/run.sh --duration 300
Performs: device discovery (aborts on ambiguity) -> SDR USB reset -> firmware build + flash -> DUT configuration -> concurrent serial + SDR capture -> correlation -> report (text + report.json). Exit 0 = PASS.
Useful flags:
--no-flashreuse the already-flashed firmware--duration Ncapture window seconds--tx-power Nconfigured SX1262 dBm (-9..14; FEM adds gain!)--delay MSrepeat delay (20..200)--gain GSDR tuner gain--min-repeats NPASS threshold
Artifacts
Each run writes captures/<timestamp>-hil/:
serial.log DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats)
rtlwmbus.log rtl_wmbus decodes
report.json machine-readable correlation result
report.txt human-readable report
setup.log DUT configuration acks
Manual pieces
./tools/discover_devices.sh -v # device inventory
./tools/flash.sh # build + flash
./hil/capture_sdr.sh 60 /tmp/sdr.log # SDR-only capture
./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats
./result-py/bin/python3 hil/iq_energy.py iq.raw # burst energy probe
./result-py/bin/python3 hil/iq_spectrum.py iq.raw # spectrum sanity
./result-py/bin/python3 hil/sdr_health.py # observer sensitivity
./firmware/test/run_tests.sh # frame-layer tests (ASan)
DUT serial commands
All output is single-line JSON; every line the HIL parses starts with {.
status current configuration + duty-cycle usage
stats all counters (incl. tx_airtime_us, tx_duty_cycle_ppm)
clear_stats
radio_status chip mode, command status, device errors, RSSI
radio_reset queue a full radio re-init
probe low-level scripted SPI/IRQ/TX diagnostic with raw dumps
test_tx transmit one synthetic C1 frame
test_burst transmit the same frame twice, repeat_delay_ms apart
inject standby fault injection: leave the radio out of RX
inject sleep fault injection: destroy the radio configuration
reboot
set repeat_delay_ms N 20..200
set tx_power N -9..14 (configured SX1262 level; FEM adds gain)
set repeat_enabled 0|1
set verbose 0|1
set fem_rx_csd 0|1 KCT8103L CSD level while in RX (LNA experiment)
set pa_low 0|1 1 = lowest documented PA table
set preamble_bits N 0|8|16|24|32 preamble detector length
set duty_budget_ppm N rolling-hour TX airtime cap; 1000 = 0.1 %, 0 = off
set test_lfield N 19..200, length of synthetic test frames
set whitelist ID[,ID...] | set whitelist clear
Known pitfalls
- SDR PLL wedge -> gain-independent white noise;
usbreset 0bda:2838. - Never wrap the rtl pipeline in
timeout bash -c(orphans children); apply timeout tortl_sdritself. - ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash scripts rediscover the port on failure.
LNA / FEM A/B experiment (no SDR needed)
nix develop --command ./hil/lna_ab.sh 600 # seconds per arm
Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled and reports frame counts, CRC success, unique meters and RSSI per arm. Result: docs/RF_FRONTEND_TESTS.md.
Interpreting observer state
Every run records SDR sensitivity: full | degraded from
hil/sdr_health.py. Treat cross-run comparisons as invalid unless both
runs report the same state -- this dongle loses ~17 dB within minutes of
streaming (docs/HARDWARE.md).
Long reliability soak (DUT only, no SDR)
nix develop --command ./hil/soak.sh 9000 # seconds
Measures firmware-side reliability over hours: repeat success rate, timing stability, radio recoveries, config-integrity failures and MCU resets, and reports progress against the milestone targets. Deliberately does not use the SDR, which cannot stream for hours without faulting.
Fault injection (verifying the recovery ladder)
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject standby"
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject sleep"
standby leaves the radio out of RX with its configuration intact;
sleep additionally destroys the configuration. The heartbeat probe must
detect both and the recovery ladder must restore RX. Verified behaviour:
inject standby -> heartbeat chip_mode=2, config_intact=true
-> radio_recovery level 1 -> rx_restored
inject sleep -> heartbeat chip_mode=0, config_intact=false
-> radio_recovery level 1 -> failed (correctly refuses)
-> radio_recovery level 2 -> rx_restored
Build profiles
./tools/build.sh test # default; the configuration all HIL numbers use
./tools/build.sh debug # verbose diagnostics, assertions, per-TX dumps
./tools/flash.sh [test|debug]
Differences are behavioural, not cosmetic:
| test | debug | |
|---|---|---|
verbose at boot |
off | on |
tx_diag after every TX |
only on failure | always |
| raw frame hex in events | yes (HIL needs it) | yes |
| optimisation / log level | size, warn | debug, info |
| task watchdog | 15 s | 30 s |
hil/run.sh always builds and flashes the test profile, so reported
timing is never perturbed by debug serial traffic. Build directories are
separate (build-test, build-debug) so profiles cannot silently reuse a
stale sdkconfig.
Verifying a standalone repeater (no serial connection)
When the DUT runs on external power with no USB host, firmware telemetry is unavailable and RF is the only evidence. Record IQ once, then derive both decodes and RF-level repeat pairs from the same samples:
timeout 80 rtl_sdr -f 868950000 -s 1600000 -g 42.1 -n 96000000 iq.raw
cat iq.raw | rtl_wmbus -f > decodes.log
./result-py/bin/python3 hil/verify_standalone.py --iq iq.raw --decodes decodes.log
Reports decoded repeats (same payload twice) and RF-level repeat pairs (burst pairs of matching duration inside the delay window). The latter matters because our transmission usually overloads the dongle and fails CRC even though the RF is correct.
Note: only one process can hold the RTL-SDR. If another tool (e.g. an
oms-tui / wmbusmeters pipeline) is running, rtl_sdr fails with
usb_claim_interface error -6 and it must be stopped first.