Files
heltec-repeater/docs/TESTING.md
T

6.2 KiB

Testing

One-command HIL test

nix develop --command ./hil/run.sh --duration 300

Performs: device discovery (aborts on ambiguity) -> SDR USB reset -> firmware build + flash -> DUT configuration -> concurrent serial + SDR capture -> correlation -> report (text + report.json). Exit 0 = PASS.

Useful flags:

  • --no-flash reuse the already-flashed firmware
  • --duration N capture window seconds
  • --tx-power N configured SX1262 dBm (-9..14; FEM adds gain!)
  • --delay MS repeat delay (20..200)
  • --gain G SDR tuner gain
  • --min-repeats N PASS threshold

Artifacts

Each run writes captures/<timestamp>-hil/:

serial.log      DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats)
rtlwmbus.log    rtl_wmbus decodes
report.json     machine-readable correlation result
report.txt      human-readable report
setup.log       DUT configuration acks

Manual pieces

./tools/discover_devices.sh -v                 # device inventory
./tools/flash.sh                               # build + flash
./hil/capture_sdr.sh 60 /tmp/sdr.log           # SDR-only capture
./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats
./result-py/bin/python3 hil/iq_energy.py iq.raw     # burst energy probe
./result-py/bin/python3 hil/iq_spectrum.py iq.raw   # spectrum sanity
./result-py/bin/python3 hil/sdr_health.py           # observer sensitivity
./firmware/test/run_tests.sh                        # frame-layer tests (ASan)

DUT serial commands

All output is single-line JSON; every line the HIL parses starts with {.

status              current configuration + duty-cycle usage
stats               all counters (incl. tx_airtime_us, tx_duty_cycle_ppm)
clear_stats
radio_status        chip mode, command status, device errors, RSSI
radio_reset         queue a full radio re-init
probe               low-level scripted SPI/IRQ/TX diagnostic with raw dumps
test_tx             transmit one synthetic C1 frame
test_burst          transmit the same frame twice, repeat_delay_ms apart
inject standby      fault injection: leave the radio out of RX
inject sleep        fault injection: destroy the radio configuration
reboot

set repeat_delay_ms N     20..200
set tx_power N            -9..14 (configured SX1262 level; FEM adds gain)
set repeat_enabled 0|1
set verbose 0|1
set fem_rx_csd 0|1        KCT8103L CSD level while in RX (LNA experiment)
set pa_low 0|1            1 = lowest documented PA table
set preamble_bits N       0|8|16|24|32 preamble detector length
set duty_budget_ppm N     rolling-hour TX airtime cap; 1000 = 0.1 %, 0 = off
set test_lfield N         19..200, length of synthetic test frames
set whitelist ID[,ID...] | set whitelist clear

Known pitfalls

  • SDR PLL wedge -> gain-independent white noise; usbreset 0bda:2838.
  • Never wrap the rtl pipeline in timeout bash -c (orphans children); apply timeout to rtl_sdr itself.
  • ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash scripts rediscover the port on failure.

LNA / FEM A/B experiment (no SDR needed)

nix develop --command ./hil/lna_ab.sh 600     # seconds per arm

Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled and reports frame counts, CRC success, unique meters and RSSI per arm. Result: docs/RF_FRONTEND_TESTS.md.

Interpreting observer state

Every run records SDR sensitivity: full | degraded from hil/sdr_health.py. Treat cross-run comparisons as invalid unless both runs report the same state -- this dongle loses ~17 dB within minutes of streaming (docs/HARDWARE.md).

Long reliability soak (DUT only, no SDR)

nix develop --command ./hil/soak.sh 9000     # seconds

Measures firmware-side reliability over hours: repeat success rate, timing stability, radio recoveries, config-integrity failures and MCU resets, and reports progress against the milestone targets. Deliberately does not use the SDR, which cannot stream for hours without faulting.

Fault injection (verifying the recovery ladder)

./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject standby"
./result-py/bin/python3 hil/dut_cmd.py <port> --listen 26 "inject sleep"

standby leaves the radio out of RX with its configuration intact; sleep additionally destroys the configuration. The heartbeat probe must detect both and the recovery ladder must restore RX. Verified behaviour:

inject standby -> heartbeat chip_mode=2, config_intact=true
               -> radio_recovery level 1 -> rx_restored
inject sleep   -> heartbeat chip_mode=0, config_intact=false
               -> radio_recovery level 1 -> failed   (correctly refuses)
               -> radio_recovery level 2 -> rx_restored

Build profiles

./tools/build.sh test      # default; the configuration all HIL numbers use
./tools/build.sh debug     # verbose diagnostics, assertions, per-TX dumps
./tools/flash.sh [test|debug]

Differences are behavioural, not cosmetic:

test debug
verbose at boot off on
tx_diag after every TX only on failure always
raw frame hex in events yes (HIL needs it) yes
optimisation / log level size, warn debug, info
task watchdog 15 s 30 s

hil/run.sh always builds and flashes the test profile, so reported timing is never perturbed by debug serial traffic. Build directories are separate (build-test, build-debug) so profiles cannot silently reuse a stale sdkconfig.

Verifying a standalone repeater (no serial connection)

When the DUT runs on external power with no USB host, firmware telemetry is unavailable and RF is the only evidence. Record IQ once, then derive both decodes and RF-level repeat pairs from the same samples:

timeout 80 rtl_sdr -f 868950000 -s 1600000 -g 42.1 -n 96000000 iq.raw
cat iq.raw | rtl_wmbus -f > decodes.log
./result-py/bin/python3 hil/verify_standalone.py --iq iq.raw --decodes decodes.log

Reports decoded repeats (same payload twice) and RF-level repeat pairs (burst pairs of matching duration inside the delay window). The latter matters because our transmission usually overloads the dongle and fails CRC even though the RF is correct.

Note: only one process can hold the RTL-SDR. If another tool (e.g. an oms-tui / wmbusmeters pipeline) is running, rtl_sdr fails with usb_claim_interface error -6 and it must be stopped first.