Files
heltec-repeater/docs/FINDINGS.md
T

15 KiB

Findings

CONFIRMED

  • Heltec V4.3 SX1262 pin mapping (NSS 8, SCK 9, MOSI 10, MISO 11, RST 12, BUSY 13, DIO1 14): SPI write/readback passes, radio enters RX (radio_init: spi:true, reset:true, tcxo:true, result:ok).
  • TCXO is DIO3-controlled at 1.8 V. The first Calibrate after SetDIO3AsTCXOCtrl latches a benign XOSC_START_ERR (0x20). Correct sequence: enable TCXO -> clear errors -> STDBY_XOSC (verify chip mode 3) -> clear -> Calibrate(0x7F) -> judge remaining errors. With that sequence: device_errors=0, TCXO verified running.
  • KCT8103L FEM on GPIO7 (power), GPIO2 (CSD), GPIO5 (CTX), CPS on DIO2. RX works with CSD high + CTX low + DIO2-as-RF-switch; TX RF confirmed with CTX high during TX.
  • SX1262 IRQ flags latch ONLY for interrupts enabled in the SetDioIrqParams mask. An unconfigured radio produces zero flags forever — indistinguishable from "no traffic" without the raw probe.
  • TX proof of life: SDR decoded our synthetic C1 format-A frame as valid (crc_ok=1, mfr 0x4D8E "SLN", ID 00000001) at configured -9 dBm, and the wM-Bus CRC blocks computed by firmware matched rtl_wmbus's checks.
  • First real repeat observed: a 100-byte Techem frame was received and the firmware entered its TX routine 50.002 ms after the reconstructed frame end (configured 50 ms). This timestamp precedes radio preparation; it is not the exact on-air start time.
  • The Nooelec NESDR SMArt v5 can enter gain-independent, railed-noise states until USB reset or physical replug. The tune-time [R82XX] PLL not locked! message is benign and rtl_test -t was not the cause. hil/sdr_health.py diagnoses the actual failure from gain response and spectral structure.
  • Real C1 traffic exists at this site: 8 meters in 10 min baseline (docs/SDR_BASELINE.md); one strong plaintext meter (61471660), several marginal encrypted Techem meters.

ROOT-CAUSED BUGS (fixed)

  1. XOSC_START_ERR treated as fatal -> init "failed" -> GFSK config was skipped entirely -> radio sat in LoRa defaults; recovery level 1 reported "rx_restored" on an unconfigured radio. Fix: proper TCXO verify + always apply configuration + recovery judges combined result.
  2. DIO1 ISR callback was never registered (sx126x_set_irq_callback not called) -> no notifications despite correct radio-side IRQ config.
  3. timeout bash -c "rtl_sdr | rtl_wmbus" orphans the pipeline children on timeout (device stays claimed). Apply timeout to rtl_sdr directly.
  4. The pa_low diagnostic selected SetPaConfig (0x01,0x00,0x00,0x01), documented in code as a low SX1262 row. Its first two parameters came from the SX1261 +10 dBm row, but deviceSel=0x00 selected SX1262, creating an undocumented hybrid. Same-capture A/B testing showed about 32 dB less RF than the valid SX1262 row (0x02,0x02,0x00,0x01). Fix: remove the invalid row and diagnostic toggle; always use a valid SX1262 row.
  5. The rolling-hour duty ledger retained only 384 individual events. At beacon rates it wrapped after about 2.7 s of airtime and silently under-counted the hour. Fix: 61 keyed minute buckets retain all airtime in the overlapping hour; host tests cover high event counts, expiry, boundary overlap, and invalid timestamps.

LIKELY

  • Fixed 255-byte RX capture adds up to ~20 ms artificial dead time per frame (by construction; measured airtimes are consistent with it).
  • The remaining CRC failures on long Techem frames are bit errors accumulating over frame length, not a hard sensitivity floor: the same meters decode cleanly when a few dB stronger (docs/RF_FRONTEND_TESTS.md).

CONFIRMED (measurement methodology)

  • WITHDRAWN: an earlier entry claimed "Heltec matched 124 of 127 SDR originals (97 %)". That measurement was taken while the dongle was in its degraded state, so the SDR decoded few originals and the Heltec matched nearly all of them. With the observer state recorded, the honest figure is 58.5 % of SDR originals matched (600 s run, observer "degraded"), with each receiver also decoding meters the other missed. See docs/RF_FRONTEND_TESTS.md.
  • Repeat payloads are byte-exact: 63/63 logged tx_payload values equalled mode bytes + the received raw frame. No corruption in the RX->TX path.
  • Real repeats are emitted as correct RF: aligning firmware TX-operation timestamps to bursts in the same stream's IQ confirmed 48 of 57 TX events (84 %), with alignment error median 1.3 ms (max 3.6 ms) and burst duration within 0.5 ms (median) of computed airtime. Firmware entered the TX routine at 200.0 ms against a 200 ms setting; radio preparation shifts actual RF onset later.
  • Firmware TX reliability: 58/58 TX success over 600 s and 72/72 over 1200 s; tx_timeout 0, tx_failures 0, radio recoveries 0, MCU resets 0.
  • The KCT8103L RX path is essential: holding CSD low in RX costs ~41 dB of chip-referred level and drops the visible meter population from 18 to 1 (600 s per arm, docs/RF_FRONTEND_TESTS.md). No evidence of excessive front-end gain hurting at this site.
  • Repeat-operation scheduling: +1 to +8 us against a 200 ms setting across 72 repeats. This is accuracy at entry to the TX routine, not RF onset. The historical 13.4 ms value called “TX airtime” was operation latency through observed TxDone; computed channel occupancy was 8.8 ms for the representative 102-byte radio payload (docs/TIMING.md).
  • The SX1262 GFSK engine cannot express wM-Bus length semantics, so RX uses a fixed 255-byte capture; RxDone therefore fires 9.4-17.1 ms after the real end of frame and the firmware reconstructs the true timestamp.
  • The observer, not the repeater, limits repeat DECODING. In the dongle's full-sensitivity state our TX arrives ~36 dB stronger than meter traffic and reaches RTL-SDR full scale; rtl-wmbus then receives our repeat with 1-2 byte errors and fails CRC (observed directly: transmitted ...78534f4c5458... decoded as ...78514f4c5448..., rest bit-exact). Raising the repeat delay 50 -> 200 ms improved SDR repeat decoding from 3 % to 30 % (demodulator level-estimate recovery).
  • The dongle's sensitivity drifts ~17 dB within minutes of streaming, so captures taken at "the same gain" at different times are NOT comparable. Meter decodes and our own TX measurements must come from one stream (hil/capture_both.sh).
  • rtl_sdr -g 0 selects AUTOMATIC gain, not 0 dB; lowest real fixed gain is 0.9 dB. Low fixed gain is not a usable workaround while degraded: at 0.9 and 7.7 dB nothing decoded at all.
  • Synthetic C1 bursts from the Heltec (L = 19/40/60/98) are decoded by rtl-wmbus as valid C1 with correct CRCs, including two airings of the same frame 50/100/200 ms apart. TX framing is therefore correct and standards-shaped independently of meter traffic.
  • rtl_wmbus runs two demodulators, so one airing yields 1-2 output lines; and rtl_sdr delivers ~81.92 ms buffers, so decode timestamps are quantized - SDR timestamps cannot be trusted below ~100 ms resolution. Correlation must use byte identity plus occurrence counting, not timestamps alone.
  • The Nooelec intermittently enters a deaf state (R820T I2C init failure / PLL wedge). hil/sdr_health.py detects it by checking that noise tracks tuner gain and the spectrum has structure; the harness aborts rather than reporting an unattributable empty capture.
  • "[R82XX] PLL not locked!" printed during tuning is BENIGN (emitted while rtl_sdr probes and disables direct-sampling mode); it is not a wedge indicator. The real symptom is railed, gain-independent noise.

CONFIRMED (failure recovery)

  • The recovery ladder works and escalates correctly, verified by fault injection rather than assumption: inject standby -> heartbeat sees chip_mode 2 -> level 1 restores RX; inject sleep (configuration destroyed) -> heartbeat sees chip_mode 0 and config_intact:false -> level 1 reports "failed" -> level 2 (full radio reinit) restores RX, confirmed by chip_mode 5 + config intact + plausible RSSI on the following heartbeats.
  • Checking chip mode alone is NOT sufficient to detect a broken receiver: a radio that lost its configuration can sit in RX looking healthy while deaf to C1. The firmware therefore reads the sync-word registers back every heartbeat (config_intact), and recovery levels only report success when both the mode and the configuration verify.

CONFIRMED (frame layer, host tests)

  • firmware/test/run_tests.sh validates the frame layer against real captured meter frames under AddressSanitizer: CRCs of third-party frames verify, every single-bit flip in them is detected, the builder's own frames validate for all buildable L, and 200k adversarial length/buffer combinations produce no out-of-bounds access.
  • The tests found a real internal inconsistency: the format-A length formula copied from rtl-wmbus disagrees with the greedy 16-byte block walk used for CRC validation whenever L = 10 (mod 16). Fixed to 1 + L + 2*(1 + ceil((L-9)/16)), which matches wmbusmeters' trimming. Both forms agree on every valid length observed here, and no valid L = 10 (mod 16) frame was ever captured -- the 10 candidates in the logs all fail their length-independent block-1 CRC, i.e. they are corrupted receptions. So this is a latent-correctness fix, not a measured improvement (docs/OPEN_QUESTIONS.md).

NEEDS TESTING

  • Narrower RX bandwidth (156 kHz) vs current 234.3 kHz on weak meters.
  • Long-run stability at >=100 and >=500 verified repeat events.
  • Early-stop RX (poll GetRxBufferStatus) to cut the ~20 ms deaf tail.

Location 3 follow-up (2026-09-01 evening): PA configuration postmortem

  • WITHDRAWN: the conclusion that this board had an intermittent post-SX1262 hardware TX death. The canonical “0/25 beacons produced any RF” capture was judged only by total energy in 1 ms windows. Reanalysis of the retained IQ finds a repeatable event-correlated FSK signature: TX median +1.5 dB over floor, nearby controls +0.2 dB, and FSK-selective excess +7.6 dB across 25/25 windows. It was weak RF, not zero RF.
  • Root cause in firmware: the pa_low diagnostic combined the SX1261 +10 dBm PA-duty/hpMax values with SX1262 deviceSel=0x00: (0x01,0x00,0x00,0x01). In a same-stream A/B test, documented SX1262 (0x02,0x02,0x00,0x01) raised the event envelope by about 32 dB. The invalid toggle and row are now deleted. Configured power is restricted to -9..+2 dBm; the external KCT8103L still follows.
  • The old logs recorded configured SetTxParams power but not which PA row the diagnostic had selected. Exact row state cannot be reconstructed for every old soak segment, and the old soak deleted each IQ file after its scalar verdict. Therefore its claimed +8 -> +6 -> zero thermal decay is not auditable and is not evidence of a board fault.
  • Fixed-firmware RF smoke test at configured -9 dBm: 9/9 scheduled beacons, median +27.81 dB over floor (range +27.02..+27.90), nearby controls +0.00 dB, FSK-selective excess +40.73 dB, and no ADC clipping at 0.9 dB SDR gain. Firmware also reported 9/9 TxDone and exactly 24,480 us of computed airtime (9 x 2,720 us).
  • hil/meter_anchored.py now requires event-correlated FSK spectral excess before calling RF present and reports strong, weak, or not_observed. The historical weak capture classifies weak; a known live repeat capture classifies strong. hil/beacon_soak.sh deletes IQ only for strong segments and retains every anomaly.
  • Observer health must still be judged per capture from ambient meter levels. TxDone remains only chip-level evidence. Neither fact supports a board-fault conclusion without the RF signature and retained raw IQ.

Field test 2026-09-01 (onsite standalone deployment)

Full narrative with the evidence table: docs/FIELD_TEST_2026-09-01.md.

  • WITHDRAWN: an earlier entry here blamed "a weak USB supply" (wall-adapter rail sag) for killing radiated TX. Disproven the same afternoon: the adapter is a 100 W GaN unit, a LiPo (stiffest possible rail) showed the same invisibility, and a frozen-position mid-capture unplug test failed on laptop power too. The supply was never the cause (except the power bank's genuine low-load auto-off, which the LED catches at a glance).
  • CONFIRMED: the repeater repeats standalone. With the SDR antenna moved to ~20 cm of the free-hanging, wall-powered board, same-duration burst pairs appeared at the 50 ms repeat delay (10-10, 17-17, 16-16 ms; received +14..+24 dB over floor).
  • WITHDRAWN: the claimed laptop-connected “+37 dB at ~50 cm” reference and the inferred USB-cable/counterpoise delta. The +37 dB number came from the repeat-borrowing clock-fit bug, so it cannot support a cable contribution estimate.
  • NOT ESTABLISHED: that the stock whip itself explains the weak standalone results. Antenna geometry unquestionably changes an over-the-air measurement, but the invalid SX1261-derived PA configuration is sufficient to explain a roughly 32 dB deficit and was not controlled in those runs.
  • CONFIRMED: TxDone (and therefore the LED double-blink) proves chip-level packet completion only. It is indistinguishable between a well-matched antenna and a shadowed/mismatched one -- §36's lesson extended to antenna geometry.
  • CONFIRMED: the recurring 10-24 ms bursts at +30..+43 dB ("monster singletons", 23-24 ms trains at ~30 ms spacing) are a neighbouring meter railing the dongle: rate invariant across firmware versions, reboots and power sources. Excluded from pair detection by fingerprint.

T-mode + long-frame support (2026-09-02)

  • CONFIRMED: T1 transmit works end to end. Synthetic T1 frames (3-of-6 coded in software, 50 kHz TX deviation via lazy SetModulationParams swap, same 0x543D sync) were decoded by rtl_wmbus as valid T1;... telegrams, including a long L=140 frame (12 decode lines over 6 airings, captures/2026-09-02T-t1-verify).
  • CONFIRMED: the T1 receive path decodes standard on-air chips. With no ambient T1 available (below), the proof is: our own T1 airings -- independently accepted by rtl_wmbus -- were FSK-demodulated from raw IQ by a separate host tool, and the post-sync byte stream was fed through the unmodified firmware decode code (wmbus_3of6_first_byte -> wmbus_3of6_decode -> wmbus_check_crc): 5/5 CRC-OK with the exact transmitted content. A C1 burst pushed through the same path is correctly rejected at the first coded byte.
  • CONFIRMED (site): there is NO ambient T-mode traffic at this site -- every ambient rtl_wmbus decode is C1. Correction to earlier belief: the large rx_not_c1 bucket (44 % of sync matches overnight) is NOT T1 meters; it is C1 frames whose two NRZ mode bytes were corrupted by bit errors, plus false syncs from the deliberately short 8-bit preamble detector. The Techem meters here are C-mode.
  • Long frames: format B >128-byte CRC chunking and format A up to L=200 are host-tested (sanitizers) and the L=200 format A frame was decoded CRC-OK over the air by rtl_wmbus.
  • T1 length cap: the fixed 255-byte capture holds ceil(total * 1.5) coded bytes, so T1 receive/repeat covers format A total <= 170 (L <= ~150). Long C1 covers total <= 253.
  • 3-of-6 tables verified against rtl-wmbus t1_c1_packet_decoder.h; codes are constant weight 3, so every single-chip corruption is detected (host test flips all 384 chip positions of a 32-byte payload).