172 lines
6.2 KiB
Markdown
172 lines
6.2 KiB
Markdown
# Testing
|
|
|
|
## One-command HIL test
|
|
|
|
```
|
|
nix develop --command ./hil/run.sh --duration 300
|
|
```
|
|
|
|
Performs: device discovery (aborts on ambiguity) -> SDR USB reset ->
|
|
firmware build + flash -> DUT configuration -> concurrent serial + SDR
|
|
capture -> correlation -> report (text + report.json). Exit 0 = PASS.
|
|
|
|
Useful flags:
|
|
- `--no-flash` reuse the already-flashed firmware
|
|
- `--duration N` capture window seconds
|
|
- `--tx-power N` configured SX1262 dBm (-9..14; FEM adds gain!)
|
|
- `--delay MS` repeat delay (20..200)
|
|
- `--gain G` SDR tuner gain
|
|
- `--min-repeats N` PASS threshold
|
|
|
|
## Artifacts
|
|
|
|
Each run writes `captures/<timestamp>-hil/`:
|
|
```
|
|
serial.log DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats)
|
|
rtlwmbus.log rtl_wmbus decodes
|
|
report.json machine-readable correlation result
|
|
report.txt human-readable report
|
|
setup.log DUT configuration acks
|
|
```
|
|
|
|
## Manual pieces
|
|
|
|
```
|
|
./tools/discover_devices.sh -v # device inventory
|
|
./tools/flash.sh # build + flash
|
|
./hil/capture_sdr.sh 60 /tmp/sdr.log # SDR-only capture
|
|
./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats
|
|
./result-py/bin/python3 hil/iq_energy.py iq.raw # burst energy probe
|
|
./result-py/bin/python3 hil/iq_spectrum.py iq.raw # spectrum sanity
|
|
./result-py/bin/python3 hil/sdr_health.py # observer sensitivity
|
|
./firmware/test/run_tests.sh # frame-layer tests (ASan)
|
|
```
|
|
|
|
## DUT serial commands
|
|
|
|
All output is single-line JSON; every line the HIL parses starts with `{`.
|
|
|
|
```
|
|
status current configuration + duty-cycle usage
|
|
stats all counters (incl. tx_airtime_us, tx_duty_cycle_ppm)
|
|
clear_stats
|
|
radio_status chip mode, command status, device errors, RSSI
|
|
radio_reset queue a full radio re-init
|
|
probe low-level scripted SPI/IRQ/TX diagnostic with raw dumps
|
|
test_tx transmit one synthetic C1 frame
|
|
test_burst transmit the same frame twice, repeat_delay_ms apart
|
|
inject standby fault injection: leave the radio out of RX
|
|
inject sleep fault injection: destroy the radio configuration
|
|
reboot
|
|
|
|
set repeat_delay_ms N 20..200
|
|
set tx_power N -9..14 (configured SX1262 level; FEM adds gain)
|
|
set repeat_enabled 0|1
|
|
set verbose 0|1
|
|
set fem_rx_csd 0|1 KCT8103L CSD level while in RX (LNA experiment)
|
|
set pa_low 0|1 1 = lowest documented PA table
|
|
set preamble_bits N 0|8|16|24|32 preamble detector length
|
|
set duty_budget_ppm N rolling-hour TX airtime cap; 1000 = 0.1 %, 0 = off
|
|
set test_lfield N 19..200, length of synthetic test frames
|
|
set whitelist ID[,ID...] | set whitelist clear
|
|
```
|
|
|
|
## Known pitfalls
|
|
|
|
- SDR PLL wedge -> gain-independent white noise; `usbreset 0bda:2838`.
|
|
- Never wrap the rtl pipeline in `timeout bash -c` (orphans children);
|
|
apply timeout to `rtl_sdr` itself.
|
|
- ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash
|
|
scripts rediscover the port on failure.
|
|
|
|
## LNA / FEM A/B experiment (no SDR needed)
|
|
|
|
```
|
|
nix develop --command ./hil/lna_ab.sh 600 # seconds per arm
|
|
```
|
|
|
|
Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled
|
|
and reports frame counts, CRC success, unique meters and RSSI per arm.
|
|
Result: docs/RF_FRONTEND_TESTS.md.
|
|
|
|
## Interpreting observer state
|
|
|
|
Every run records `SDR sensitivity: full | degraded` from
|
|
`hil/sdr_health.py`. Treat cross-run comparisons as invalid unless both
|
|
runs report the same state -- this dongle loses ~17 dB within minutes of
|
|
streaming (docs/HARDWARE.md).
|
|
|
|
## Long reliability soak (DUT only, no SDR)
|
|
|
|
```
|
|
nix develop --command ./hil/soak.sh 9000 # seconds
|
|
```
|
|
|
|
Measures firmware-side reliability over hours: repeat success rate, timing
|
|
stability, radio recoveries, config-integrity failures and MCU resets, and
|
|
reports progress against the milestone targets. Deliberately does not use
|
|
the SDR, which cannot stream for hours without faulting.
|
|
|
|
## Fault injection (verifying the recovery ladder)
|
|
|
|
```
|
|
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject standby"
|
|
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject sleep"
|
|
```
|
|
|
|
`standby` leaves the radio out of RX with its configuration intact;
|
|
`sleep` additionally destroys the configuration. The heartbeat probe must
|
|
detect both and the recovery ladder must restore RX. Verified behaviour:
|
|
|
|
```
|
|
inject standby -> heartbeat chip_mode=2, config_intact=true
|
|
-> radio_recovery level 1 -> rx_restored
|
|
inject sleep -> heartbeat chip_mode=0, config_intact=false
|
|
-> radio_recovery level 1 -> failed (correctly refuses)
|
|
-> radio_recovery level 2 -> rx_restored
|
|
```
|
|
|
|
## Build profiles
|
|
|
|
```
|
|
./tools/build.sh test # default; the configuration all HIL numbers use
|
|
./tools/build.sh debug # verbose diagnostics, assertions, per-TX dumps
|
|
./tools/flash.sh [test|debug]
|
|
```
|
|
|
|
Differences are behavioural, not cosmetic:
|
|
|
|
| | test | debug |
|
|
|---|---|---|
|
|
| `verbose` at boot | off | on |
|
|
| `tx_diag` after every TX | only on failure | always |
|
|
| raw frame hex in events | yes (HIL needs it) | yes |
|
|
| optimisation / log level | size, warn | debug, info |
|
|
| task watchdog | 15 s | 30 s |
|
|
|
|
`hil/run.sh` always builds and flashes the `test` profile, so reported
|
|
timing is never perturbed by debug serial traffic. Build directories are
|
|
separate (`build-test`, `build-debug`) so profiles cannot silently reuse a
|
|
stale sdkconfig.
|
|
|
|
## Verifying a standalone repeater (no serial connection)
|
|
|
|
When the DUT runs on external power with no USB host, firmware telemetry is
|
|
unavailable and RF is the only evidence. Record IQ once, then derive both
|
|
decodes and RF-level repeat pairs from the same samples:
|
|
|
|
```
|
|
timeout 80 rtl_sdr -f 868950000 -s 1600000 -g 42.1 -n 96000000 iq.raw
|
|
cat iq.raw | rtl_wmbus -f > decodes.log
|
|
./result-py/bin/python3 hil/verify_standalone.py --iq iq.raw --decodes decodes.log
|
|
```
|
|
|
|
Reports decoded repeats (same payload twice) and RF-level repeat pairs
|
|
(burst pairs of matching duration inside the delay window). The latter
|
|
matters because our transmission usually overloads the dongle and fails CRC
|
|
even though the RF is correct.
|
|
|
|
Note: only one process can hold the RTL-SDR. If another tool (e.g. an
|
|
oms-tui / wmbusmeters pipeline) is running, `rtl_sdr` fails with
|
|
`usb_claim_interface error -6` and it must be stopped first.
|