128 lines
4.2 KiB
Markdown
128 lines
4.2 KiB
Markdown
# Testing
|
|
|
|
## One-command HIL test
|
|
|
|
```
|
|
nix develop --command ./hil/run.sh --duration 300
|
|
```
|
|
|
|
Performs: device discovery (aborts on ambiguity) -> SDR USB reset ->
|
|
firmware build + flash -> DUT configuration -> concurrent serial + SDR
|
|
capture -> correlation -> report (text + report.json). Exit 0 = PASS.
|
|
|
|
Useful flags:
|
|
- `--no-flash` reuse the already-flashed firmware
|
|
- `--duration N` capture window seconds
|
|
- `--tx-power N` configured SX1262 dBm (-9..14; FEM adds gain!)
|
|
- `--delay MS` repeat delay (20..200)
|
|
- `--gain G` SDR tuner gain
|
|
- `--min-repeats N` PASS threshold
|
|
|
|
## Artifacts
|
|
|
|
Each run writes `captures/<timestamp>-hil/`:
|
|
```
|
|
serial.log DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats)
|
|
rtlwmbus.log rtl_wmbus decodes
|
|
report.json machine-readable correlation result
|
|
report.txt human-readable report
|
|
setup.log DUT configuration acks
|
|
```
|
|
|
|
## Manual pieces
|
|
|
|
```
|
|
./tools/discover_devices.sh -v # device inventory
|
|
./tools/flash.sh # build + flash
|
|
./hil/capture_sdr.sh 60 /tmp/sdr.log # SDR-only capture
|
|
./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats
|
|
./result-py/bin/python3 hil/iq_energy.py iq.raw # burst energy probe
|
|
./result-py/bin/python3 hil/iq_spectrum.py iq.raw # spectrum sanity
|
|
```
|
|
|
|
## DUT serial commands
|
|
|
|
```
|
|
status | stats | clear_stats | radio_status | radio_reset | test_tx
|
|
set repeat_delay_ms N | set tx_power N | set repeat_enabled 0|1
|
|
set verbose 0|1 | set whitelist ID[,ID...] | set whitelist clear | reboot
|
|
```
|
|
|
|
## Known pitfalls
|
|
|
|
- SDR PLL wedge -> gain-independent white noise; `usbreset 0bda:2838`.
|
|
- Never wrap the rtl pipeline in `timeout bash -c` (orphans children);
|
|
apply timeout to `rtl_sdr` itself.
|
|
- ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash
|
|
scripts rediscover the port on failure.
|
|
|
|
## LNA / FEM A/B experiment (no SDR needed)
|
|
|
|
```
|
|
nix develop --command ./hil/lna_ab.sh 600 # seconds per arm
|
|
```
|
|
|
|
Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled
|
|
and reports frame counts, CRC success, unique meters and RSSI per arm.
|
|
Result: docs/RF_FRONTEND_TESTS.md.
|
|
|
|
## Interpreting observer state
|
|
|
|
Every run records `SDR sensitivity: full | degraded` from
|
|
`hil/sdr_health.py`. Treat cross-run comparisons as invalid unless both
|
|
runs report the same state -- this dongle loses ~17 dB within minutes of
|
|
streaming (docs/HARDWARE.md).
|
|
|
|
## Long reliability soak (DUT only, no SDR)
|
|
|
|
```
|
|
nix develop --command ./hil/soak.sh 9000 # seconds
|
|
```
|
|
|
|
Measures firmware-side reliability over hours: repeat success rate, timing
|
|
stability, radio recoveries, config-integrity failures and MCU resets, and
|
|
reports progress against the milestone targets. Deliberately does not use
|
|
the SDR, which cannot stream for hours without faulting.
|
|
|
|
## Fault injection (verifying the recovery ladder)
|
|
|
|
```
|
|
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject standby"
|
|
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject sleep"
|
|
```
|
|
|
|
`standby` leaves the radio out of RX with its configuration intact;
|
|
`sleep` additionally destroys the configuration. The heartbeat probe must
|
|
detect both and the recovery ladder must restore RX. Verified behaviour:
|
|
|
|
```
|
|
inject standby -> heartbeat chip_mode=2, config_intact=true
|
|
-> radio_recovery level 1 -> rx_restored
|
|
inject sleep -> heartbeat chip_mode=0, config_intact=false
|
|
-> radio_recovery level 1 -> failed (correctly refuses)
|
|
-> radio_recovery level 2 -> rx_restored
|
|
```
|
|
|
|
## Build profiles
|
|
|
|
```
|
|
./tools/build.sh test # default; the configuration all HIL numbers use
|
|
./tools/build.sh debug # verbose diagnostics, assertions, per-TX dumps
|
|
./tools/flash.sh [test|debug]
|
|
```
|
|
|
|
Differences are behavioural, not cosmetic:
|
|
|
|
| | test | debug |
|
|
|---|---|---|
|
|
| `verbose` at boot | off | on |
|
|
| `tx_diag` after every TX | only on failure | always |
|
|
| raw frame hex in events | yes (HIL needs it) | yes |
|
|
| optimisation / log level | size, warn | debug, info |
|
|
| task watchdog | 15 s | 30 s |
|
|
|
|
`hil/run.sh` always builds and flashes the `test` profile, so reported
|
|
timing is never perturbed by debug serial traffic. Build directories are
|
|
separate (`build-test`, `build-debug`) so profiles cannot silently reuse a
|
|
stale sdkconfig.
|