diff --git a/README.md b/README.md new file mode 100644 index 0000000..78cafeb --- /dev/null +++ b/README.md @@ -0,0 +1,118 @@ +# Wireless M-Bus C1 Repeater — Heltec WiFi LoRa 32 V4.3 (RF feasibility experiment) + +Experimental firmware and hardware-in-the-loop test rig answering one +question: **can the SX1262 + KCT8103L front-end on the Heltec V4.3 serve as +the basis for a Wireless M-Bus gateway/repeater?** + +Short answer: yes for RX and TX at the PHY level. It receives real C1 +meter telegrams, retransmits them bit-exact after a configurable delay, and +returns to RX reliably. Quantitative limits and caveats are below. + +This is **not** a standards-compliant repeater. See +[docs/STANDARDS_GAPS.md](docs/STANDARDS_GAPS.md). + +## What is here + +``` +firmware/ ESP-IDF app: SX1262 GFSK driver, wM-Bus C1 RX/TX, + repeater state machine, recovery ladder, JSON telemetry +hil/ hardware-in-the-loop harness (run.sh is the entry point) +tools/ device discovery, flashing, reset +docs/ hardware evidence, RF config, findings, timing, gaps +captures/ per-run artifacts (logs, reports, IQ) +flake.nix ESP-IDF + rtl-sdr + rtl-wmbus + wmbusmeters + python env +``` + +## Quick start + +```bash +nix build .#hil-python -o result-py # once +nix develop --command ./hil/run.sh --duration 1200 +``` + +Exit code 0 = PASS. Artifacts land in `captures/-hil/` +(`report.txt`, `report.json`, `serial.log`, `rtlwmbus.log`, `iq_proof.txt`). + +Serial console (scriptable, JSON output): + +```bash +./result-py/bin/python3 hil/dut_cmd.py --listen 5 status stats +# status | stats | clear_stats | radio_status | radio_reset | probe +# test_tx | test_burst | reboot +# set repeat_delay_ms N | set tx_power N | set repeat_enabled 0|1 +# set verbose 0|1 | set fem_rx_csd 0|1 | set pa_low 0|1 +# set test_lfield N | set whitelist ID[,ID...] | set whitelist clear +``` + +## Results (measured, not inferred) + +Representative 1200 s run, `captures/2026-09-01T10-34-24-hil`: + +| Metric | Value | +|---|---| +| Heltec RX candidates / CRC-valid | 214 / 72 | +| SDR-classified originals | 129 (44 meters) | +| Heltec matched SDR originals | 47 (36.4 %) | +| Heltec TX attempts / success | 72 / 72 (100 %) | +| TX events RF-confirmed in IQ | 5 / 5 in the IQ window (100 %) | +| Firmware repeat delay (set 200 ms) | 200.001 ms min/median, 200.008 ms max | +| Radio recoveries / MCU resets | 0 / 0 | + +Key hardware findings: + +- **Pin mapping and FEM control verified** on real hardware — SX1262 on + GPIO8-14, KCT8103L on GPIO7/2/5 with CPS driven by DIO2, TCXO on DIO3 at + 1.8 V. [docs/HARDWARE.md](docs/HARDWARE.md) +- **The KCT8103L RX path is essential**: disabling it (CSD low) costs ~41 dB + and drops visible meters from 18 to 1. No overload penalty from maximum + front-end gain at this site. + [docs/RF_FRONTEND_TESTS.md](docs/RF_FRONTEND_TESTS.md) +- **Repeats are bit-exact**: 63/63 logged TX payloads equalled the received + raw frame plus C-mode bytes. +- **RX deaf time is ~225.7 ms** per repeat at a 200 ms delay, including a + 12.2 ms fixed-capture tail imposed by the SX1262 packet engine. + [docs/TIMING.md](docs/TIMING.md) + +## Measurement caveats — read before trusting any number + +The RTL-SDR observer is the weakest instrument in this setup and the +harness is built around that: + +1. Its sensitivity drifts ~17 dB within minutes of streaming, so captures + at "the same gain" minutes apart are not comparable. +2. Its tuner dies mid-stream (`i2c wr failed`) without `rtl_sdr` exiting — + a naive long capture silently returns "no traffic". The harness uses + bounded segments with reset-and-resume. +3. It can de-enumerate entirely, needing a physical replug. + +`hil/sdr_health.py` gates every run and every report states the observer +state (`full` / `degraded`), so an empty capture is always attributable. +An earlier claimed "97 % RX match rate" was withdrawn after it turned out +to have been measured against a degraded observer. + +Our transmission also arrives far stronger than meter traffic (up to ~36 dB +at full observer sensitivity), which pushes the dongle into compression so +rtl-wmbus receives our repeat with 1-2 byte errors and fails CRC. Repeats +are therefore verified two ways that are immune to this: + +- synthetic C1 bursts prove the framing is decodable as C1 (`verify_tx.py`); +- firmware TX timestamps are aligned to bursts in raw IQ to prove real + repeats radiate with the right timing and airtime + (`verify_repeats_iq.py`). + +Closing the remaining gap needs physical antenna separation or an inline +attenuator, which is not scriptable from the host. + +## Documentation + +- [HARDWARE.md](docs/HARDWARE.md) — board, pin mapping, FEM, observer faults +- [RF_CONFIG.md](docs/RF_CONFIG.md) — C1 PHY and SX1262 register settings +- [RF_FRONTEND_TESTS.md](docs/RF_FRONTEND_TESTS.md) — RX comparisons, LNA A/B +- [TIMING.md](docs/TIMING.md) — latency and deaf-time instrumentation +- [SDR_BASELINE.md](docs/SDR_BASELINE.md) — Phase 0 observer baseline +- [FINDINGS.md](docs/FINDINGS.md) — confirmed / disproven / needs testing +- [OPEN_QUESTIONS.md](docs/OPEN_QUESTIONS.md) — open and closed questions +- [STANDARDS_GAPS.md](docs/STANDARDS_GAPS.md) — deviations from EN 13757/OMS +- [TESTING.md](docs/TESTING.md) — how to run things, known pitfalls +- [DEVBOARD_REQUIREMENTS.md](docs/DEVBOARD_REQUIREMENTS.md) + + [devboard_signals.csv](docs/devboard_signals.csv) — future PCB needs diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index a95b2bc..d71828f 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -66,8 +66,18 @@ with alignment error median 1.3 ms (max 3.6 ms) and burst duration within 0.5 ms (median) of the computed airtime. Firmware repeat delay measured 200.0 ms min/median/max against a 200 ms setting. -- Firmware TX reliability over 600 s: tx_attempts 58, tx_success 58, - tx_timeout 0, tx_failures 0, recoveries 0. +- Firmware TX reliability: 58/58 TX success over 600 s and 72/72 over + 1200 s; tx_timeout 0, tx_failures 0, radio recoveries 0, MCU resets 0. +- The KCT8103L RX path is essential: holding CSD low in RX costs ~41 dB of + chip-referred level and drops the visible meter population from 18 to 1 + (600 s per arm, docs/RF_FRONTEND_TESTS.md). No evidence of excessive + front-end gain hurting at this site. +- Repeat-delay accuracy: +1 to +8 us against a 200 ms setting across 72 + repeats. RX deaf time per repeat ~225.7 ms median, of which 12.2 ms is + the fixed-255-byte RX tail and 13.4 ms the TX airtime (docs/TIMING.md). +- The SX1262 GFSK engine cannot express wM-Bus length semantics, so RX uses + a fixed 255-byte capture; RxDone therefore fires 9.4-17.1 ms after the + real end of frame and the firmware reconstructs the true timestamp. - The observer, not the repeater, limits repeat DECODING. In the dongle's full-sensitivity state our TX arrives ~36 dB stronger than meter traffic and reaches RTL-SDR full scale; rtl-wmbus then receives our repeat with @@ -101,7 +111,6 @@ ## NEEDS TESTING -- LNA on/off comparison (`set fem_rx_csd 0|1` hook exists, not yet measured). - Narrower RX bandwidth (156 kHz) vs current 234.3 kHz on weak meters. - Long-run stability at >=100 and >=500 verified repeat events. - Early-stop RX (poll GetRxBufferStatus) to cut the ~20 ms deaf tail. diff --git a/docs/OPEN_QUESTIONS.md b/docs/OPEN_QUESTIONS.md index 22aaf5f..c057649 100644 --- a/docs/OPEN_QUESTIONS.md +++ b/docs/OPEN_QUESTIONS.md @@ -5,14 +5,22 @@ drifted meters? - Preamble detector length: 8 bits chosen for sensitivity; false-sync rate in the field? (false syncs cost ~20 ms deaf time each at fixed-255 RX). -- Does the KCT8103L LNA help or hurt with strong 868 MHz interferers? - (see planned docs/RF_FRONTEND_TESTS.md) -- RX dead time during retransmit: measured distribution? +- CLOSED: Does the KCT8103L LNA help or hurt with strong 868 MHz + interferers? -> It helps decisively. CSD low costs ~41 dB and reduces + visible meters from 18 to 1; no overload degradation observed + (docs/RF_FRONTEND_TESTS.md). +- CLOSED: RX dead time during retransmit -> ~225.7 ms median at a 200 ms + repeat delay, of which 12.2 ms is the fixed-255 RX tail and 13.4 ms the + TX airtime (docs/TIMING.md). Early-stop RX would cut the tail. - Can the SX1262 re-arm RX fast enough to catch a meter's immediate follow-up telegram after our TX? - Exact production repeater framing/hop requirements (EN 13757-5). - Regulatory duty-cycle budget for a production repeater at 868.95 MHz. - Modulation shaping for TX: none vs GFSK BT=0.5 — which does rtl-wmbus / real receivers prefer? (TX currently unshaped 2-FSK.) -- Why did the R820T PLL wedge? (usbreset workaround in place; root cause - unknown, suspected rtl_test direct-sampling probe.) +- Why does the R820T degrade and eventually die during streaming? Three + failure modes are characterised in docs/HARDWARE.md; `usbreset` recovers + the first two, the third needs a physical replug. Root cause unknown + (thermal suspected). The `rtl_test -t` theory was DISPROVEN. +- Can the SX1262 RX be stopped early (poll GetRxBufferStatus after + SyncWordValid) to remove the ~12 ms fixed-capture tail? diff --git a/docs/RF_FRONTEND_TESTS.md b/docs/RF_FRONTEND_TESTS.md index 51128eb..cefab97 100644 --- a/docs/RF_FRONTEND_TESTS.md +++ b/docs/RF_FRONTEND_TESTS.md @@ -73,16 +73,44 @@ Firmware counters from the 600 s run corroborate the shape: (`rx_not_c1` are sync matches that are not C-mode: T1 traffic and false syncs from the deliberately short 8-bit preamble detector). -## LNA / CSD experiment +## LNA / CSD experiment - MEASURED `set fem_rx_csd 0|1` controls the KCT8103L CSD level while in RX (high = FEM enabled/LNA path, low = FEM disabled). TX always forces CSD high. -`hil/lna_ab.sh` runs two equal-length RX-only arms and reports frame -counts, CRC success and RSSI distribution per arm. +`hil/lna_ab.sh` runs two equal-length RX-only arms (retransmission +disabled, so the receiver is measured in isolation). -Results: see the "KCT8103L RX front-end A/B" section appended below once a -run has completed. Do not assume a direction before measuring - meter -traffic varies between arms, so only large differences are meaningful. +Result, 600 s per arm, `captures/2026-09-01T10-55-00-lna-ab`: + +| | CSD high (FEM enabled) | CSD low (FEM disabled) | +|---|---|---| +| candidate frames | 108 | 5 | +| CRC-valid frames | 35 | 5 | +| unique meters | 18 | 1 | +| median RSSI (all) | -61 dBm | -102 dBm | +| median RSSI (CRC ok) | -53 dBm | -102 dBm | +| strongest RSSI | -45 dBm | -101 dBm | +| firmware counters | candidates=190, valid=34, crc_fail=73, not_c1=83 | candidates=8, valid=5, crc_fail=0, not_c1=3 | + +**Conclusion: the KCT8103L RX path is essential.** Disabling it costs about +41 dB of chip-referred signal level and cuts the visible meter population +from 18 to 1. CSD must be held high in RX. + +Two secondary observations: + +- With CSD low the few frames that still arrive decode with 100 % CRC + success at -102 dBm. Signal and noise both drop with the FEM disabled, so + SNR survives for the single strongest local meter; chip-referred RSSI is + therefore not comparable across arms as an SNR measure. +- No sign of the "too much gain hurts" failure mode at this site: the + high-gain arm saw strictly more meters with no CRC collapse on strong + frames. Excessive front-end gain was a plausible risk given strong + 868 MHz neighbours, but it is not what limits this bench. + +Cost of the sensitive configuration: with CSD high, 83 of 190 sync matches +(44 %) were not C-mode (`rx_not_c1`) - T1 traffic plus false syncs from the +deliberately short 8-bit preamble detector. Each false sync consumes one +fixed-255-byte RX window (~12 ms, docs/TIMING.md). ## Bench dynamic-range limitation diff --git a/docs/TESTING.md b/docs/TESTING.md index 64d9ca2..8f56f6c 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -55,3 +55,20 @@ set verbose 0|1 | set whitelist ID[,ID...] | set whitelist clear | reboot apply timeout to `rtl_sdr` itself. - ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash scripts rediscover the port on failure. + +## LNA / FEM A/B experiment (no SDR needed) + +``` +nix develop --command ./hil/lna_ab.sh 600 # seconds per arm +``` + +Runs two RX-only arms (CSD high vs CSD low) with retransmission disabled +and reports frame counts, CRC success, unique meters and RSSI per arm. +Result: docs/RF_FRONTEND_TESTS.md. + +## Interpreting observer state + +Every run records `SDR sensitivity: full | degraded` from +`hil/sdr_health.py`. Treat cross-run comparisons as invalid unless both +runs report the same state -- this dongle loses ~17 dB within minutes of +streaming (docs/HARDWARE.md). diff --git a/docs/TIMING.md b/docs/TIMING.md new file mode 100644 index 0000000..d75a11d --- /dev/null +++ b/docs/TIMING.md @@ -0,0 +1,63 @@ +# Timing Instrumentation + +All values measured on the DUT with `esp_timer_get_time()` (monotonic +microseconds), from `captures/2026-09-01T10-34-24-hil` (1200 s run, 72 +repeats, 214 RX candidates, repeat delay set to 200 ms). + +## Measured + +| Quantity | min | median | max | n | +|---|---|---|---|---| +| RxDone -> reconstructed end of frame ("deaf tail") | 9 360 us | 12 240 us | 17 120 us | 214 | +| TX airtime (SetTx to TxDone) | 10 421 us | 13 422 us | 14 560 us | 72 | +| Programmed repeat delay (frame end -> TX start) | 200 001 us | 200 001 us | 200 008 us | 72 | +| Repeated frame length (over-the-air bytes) | 63 | 100 | 113 | 72 | + +Independent confirmation from the SDR side (raw IQ, same run): observed +burst duration differs from the computed airtime by 0.2-1.5 ms +(median 1.2 ms), and firmware TX events align to observed RF within +0-3.1 ms (median 2.4 ms). + +## Derived + +``` +repeat delay accuracy : +1 us to +8 us against a 200 ms setting + (jitter well under 10 us) +RX deaf time per repeat : ~225.7 ms median + = 12.2 ms deaf tail + + 200.0 ms programmed delay + + 13.4 ms TX airtime +``` + +The programmed delay dominates by design; it exists to make SDR +correlation possible during development. + +## The 12 ms deaf tail + +The SX1262's GFSK packet engine cannot express Wireless M-Bus length +semantics (format A carries per-block CRCs that the L-field excludes), so +RX uses a fixed 255-byte capture and the firmware truncates in software +using the L-field formulas. Consequences: + +- RxDone fires `(255 - frame_bytes) * 80 us` after the frame really ended; + the firmware subtracts this to recover the true end-of-frame timestamp, + which is what the repeat delay is measured from. +- During that tail the receiver is not re-armed for sync search, costing + 9.4-17.1 ms of receive availability per detected frame -- including for + false syncs from the deliberately short 8-bit preamble detector. + +Optimisation candidate (not implemented): poll `GetRxBufferStatus` after +`SyncWordValid`, read the L-field as soon as it lands, then stop RX early +once the computed frame length has been received. That would cut the tail +to roughly the frame's own airtime and materially raise the duty cycle of +a production repeater. + +## Relevance to the production gateway + +- A production repeater using randomized 5-25 s delays will have a very + different deaf-time profile; the 200 ms figure here is a development + artefact, not a design target (see docs/STANDARDS_GAPS.md). +- The fixed-255 tail is an SX1262 packet-engine limitation, so it applies + to any SX126x-based design. Budget for it, or implement early-stop RX. +- Repeat-delay accuracy of a few microseconds shows the ESP32-S3 side has + ample timing margin; timing precision will not be the constraint.