fix: TCXO XOSC_START_ERR handling, always apply config, register DIO1 ISR callback; TX proof of life confirmed by SDR

This commit is contained in:
Lars Nolden
2026-08-31 22:43:55 +02:00
parent 3b8547fb8a
commit f7c8614a3b
9 changed files with 283 additions and 14 deletions
+30
View File
@@ -0,0 +1,30 @@
# Future Solenos/Gebos dev-board requirements
Derived from actual debugging needs during this experiment. Machine-readable
pin list: devboard_signals.csv.
## Debug access that proved necessary
- SX1262 BUSY + DIO1 on test points: needed to reason about command timing,
IRQ latency, and stuck-BUSY recovery without guessing.
"I needed direct access to this signal because the only alternative was
inferring radio state from SPI responses."
- Full SPI bus (NSS/SCK/MOSI/MISO) on a header: logic-analyzer decode of
the SX1262 command stream is the ground truth when firmware and radio
disagree.
- FEM control lines (PWR_EN, CSD, CTX): RX sensitivity and TX output both
silently break if these are wrong; scoping them is the fastest check.
CSD doubles as board-revision detect (pull level) — keep it probeable.
- Native USB pins must remain dedicated to flash/console; the DUT USB
identity changes between app and ROM bootloader, so the dev board should
also expose UART0 TX as an always-on log output (the Heltec does not,
which made early-boot debugging blind).
## Wishlist beyond the current board
- Always-available secondary UART for logs (independent of native USB).
- Measurable radio power rail (current shunt) to correlate TX events with
actual PA draw — would have settled "is the FEM PA active?" instantly.
- RF test connector (u.FL tap or directional coupler) for conducted
measurements instead of over-the-air-only verification.
- ESP32 BOOT/RESET buttons brought to a header for automated strapping.
+40
View File
@@ -0,0 +1,40 @@
# Findings
## CONFIRMED
- Heltec V4.3 SX1262 pin mapping (NSS 8, SCK 9, MOSI 10, MISO 11, RST 12,
BUSY 13, DIO1 14): SPI write/readback passes, radio enters RX
(`radio_init: spi:true, reset:true, tcxo:true, result:ok`).
- TCXO is DIO3-controlled at 1.8 V; without post-TCXO recalibration the
XOSC_START_ERR device error appears. With recalibrate: no device errors.
- KCT8103L FEM on GPIO7 (power), GPIO2 (CSD), GPIO5 (CTX), CPS on DIO2.
RX works with CSD high + CTX low + DIO2-as-RF-switch.
- Firmware heartbeat RSSI in RX: -101..-107 dBm noise floor -> RX chain
with boosted gain is plausibly alive.
- Nooelec NESDR SMArt v5 R820T PLL wedges ("PLL not locked"), producing
gain-independent white noise on all frequencies until USB reset.
`usbreset 0bda:2838` reliably restores operation. Trigger correlated
with `rtl_test -t` direct-sampling probe.
- rtl-sdr-blog and osmocom rtl_sdr builds behave identically w.r.t. the
wedge; not a driver-fork issue.
- Real C1 traffic exists at this site (first telegram decoded 22:23 local,
meter ID 04767752).
## LIKELY
- Fixed 255-byte RX capture adds up to ~20 ms artificial dead time per
frame (by construction; to be measured).
## DISPROVEN
- "rtl_sdr pipeline works out of the box": the PLL wedge cost ~20 minutes
of empty captures. Always energy-probe the SDR before trusting a null
result.
## NEEDS TESTING
- Heltec C1 RX of real meter frames (SDR sees traffic; Heltec match TBD).
- TX proof of life via SDR.
- LNA on/off comparison (KCT8103L CSD low in RX = bypass?).
- Repeat delay accuracy, RX deaf time distribution.
- Long-run stability (>=100, >=500 repeats).
+18
View File
@@ -0,0 +1,18 @@
# Open Questions
- Optimal SX1262 RX bandwidth for C1: 234.3 kHz chosen (signal ~190 kHz
incl. tolerances); would 156 kHz improve sensitivity without clipping
drifted meters?
- Preamble detector length: 8 bits chosen for sensitivity; false-sync rate
in the field? (false syncs cost ~20 ms deaf time each at fixed-255 RX).
- Does the KCT8103L LNA help or hurt with strong 868 MHz interferers?
(see planned docs/RF_FRONTEND_TESTS.md)
- RX dead time during retransmit: measured distribution?
- Can the SX1262 re-arm RX fast enough to catch a meter's immediate
follow-up telegram after our TX?
- Exact production repeater framing/hop requirements (EN 13757-5).
- Regulatory duty-cycle budget for a production repeater at 868.95 MHz.
- Modulation shaping for TX: none vs GFSK BT=0.5 — which does rtl-wmbus /
real receivers prefer? (TX currently unshaped 2-FSK.)
- Why did the R820T PLL wedge? (usbreset workaround in place; root cause
unknown, suspected rtl_test direct-sampling probe.)
+57
View File
@@ -0,0 +1,57 @@
# Testing
## One-command HIL test
```
nix develop --command ./hil/run.sh --duration 300
```
Performs: device discovery (aborts on ambiguity) -> SDR USB reset ->
firmware build + flash -> DUT configuration -> concurrent serial + SDR
capture -> correlation -> report (text + report.json). Exit 0 = PASS.
Useful flags:
- `--no-flash` reuse the already-flashed firmware
- `--duration N` capture window seconds
- `--tx-power N` configured SX1262 dBm (-9..14; FEM adds gain!)
- `--delay MS` repeat delay (20..200)
- `--gain G` SDR tuner gain
- `--min-repeats N` PASS threshold
## Artifacts
Each run writes `captures/<timestamp>-hil/`:
```
serial.log DUT JSON-lines (wmbus_rx / wmbus_tx / heartbeats / stats)
rtlwmbus.log rtl_wmbus decodes
report.json machine-readable correlation result
report.txt human-readable report
setup.log DUT configuration acks
```
## Manual pieces
```
./tools/discover_devices.sh -v # device inventory
./tools/flash.sh # build + flash
./hil/capture_sdr.sh 60 /tmp/sdr.log # SDR-only capture
./result-py/bin/python3 hil/dut_cmd.py PORT --listen 10 status stats
./result-py/bin/python3 hil/iq_energy.py iq.raw # burst energy probe
./result-py/bin/python3 hil/iq_spectrum.py iq.raw # spectrum sanity
```
## DUT serial commands
```
status | stats | clear_stats | radio_status | radio_reset | test_tx
set repeat_delay_ms N | set tx_power N | set repeat_enabled 0|1
set verbose 0|1 | set whitelist ID[,ID...] | set whitelist clear | reboot
```
## Known pitfalls
- SDR PLL wedge -> gain-independent white noise; `usbreset 0bda:2838`.
- Never wrap the rtl pipeline in `timeout bash -c` (orphans children);
apply timeout to `rtl_sdr` itself.
- ESP32-S3 USB re-enumerates between app CDC and ROM bootloader; flash
scripts rediscover the port on failure.
+4
View File
@@ -88,6 +88,10 @@ static void console_task(void *arg)
repeater_request_radio_reset(); repeater_request_radio_reset();
ack("radio_reset", true, "queued"); ack("radio_reset", true, "queued");
} }
else if (strcmp(line, "probe") == 0) {
repeater_request_probe();
ack("probe", true, "queued");
}
else if (strcmp(line, "test_tx") == 0) { else if (strcmp(line, "test_tx") == 0) {
repeater_request_test_tx(); repeater_request_test_tx();
ack("test_tx", true, "queued"); ack("test_tx", true, "queued");
+95 -6
View File
@@ -9,6 +9,7 @@
#include "freertos/FreeRTOS.h" #include "freertos/FreeRTOS.h"
#include "freertos/task.h" #include "freertos/task.h"
#include "esp_timer.h" #include "esp_timer.h"
#include "driver/gpio.h"
#include "esp_task_wdt.h" #include "esp_task_wdt.h"
#include "esp_system.h" #include "esp_system.h"
@@ -31,6 +32,7 @@ static TaskHandle_t s_task;
static volatile int64_t s_irq_ts; static volatile int64_t s_irq_ts;
static volatile bool s_want_radio_reset; static volatile bool s_want_radio_reset;
static volatile bool s_want_test_tx; static volatile bool s_want_test_tx;
static volatile bool s_want_probe;
static uint8_t s_test_access_nr; static uint8_t s_test_access_nr;
static uint32_t s_seq; static uint32_t s_seq;
@@ -62,9 +64,13 @@ static void irq_cb(int64_t ts)
static bool radio_full_init(void) static bool radio_full_init(void)
{ {
sx126x_diag_t diag; sx126x_diag_t diag;
bool ok = sx126x_init(&diag); bool init_ok = sx126x_init(&diag);
bool cfg_ok = ok && sx126x_config_wmbus_c1(868950000u, RX_PAYLOAD_LEN) // Apply the wM-Bus configuration even when init reports soft failures:
&& sx126x_set_tx_power(g_cfg.tx_power_dbm); // an unconfigured radio in RX is worse than a partially degraded one,
// and the recovery ladder judges the combined result anyway.
bool cfg_ok = sx126x_config_wmbus_c1(868950000u, RX_PAYLOAD_LEN)
& sx126x_set_tx_power(g_cfg.tx_power_dbm);
cfg_ok = init_ok && cfg_ok;
LOGJ("\"event\":\"radio_init\",\"spi\":%s,\"reset\":%s,\"busy_low\":%s," LOGJ("\"event\":\"radio_init\",\"spi\":%s,\"reset\":%s,\"busy_low\":%s,"
"\"tcxo\":%s,\"device_errors\":%u,\"chip_mode\":%u," "\"tcxo\":%s,\"device_errors\":%u,\"chip_mode\":%u,"
"\"fem_power\":true,\"fem_mode\":\"rx\",\"frequency_hz\":868950000," "\"fem_power\":true,\"fem_mode\":\"rx\",\"frequency_hz\":868950000,"
@@ -181,12 +187,33 @@ static const char *do_tx(const uint8_t *payload, int payload_len,
} }
// airtime: 48 bit preamble + 16 bit sync + payload -> ~80us/byte + 640 // airtime: 48 bit preamble + 16 bit sync + payload -> ~80us/byte + 640
uint32_t airtime_ms = (payload_len * US_PER_BYTE + 640) / 1000; uint32_t airtime_ms = (payload_len * US_PER_BYTE + 640) / 1000;
uint16_t irq = wait_irq(airtime_ms + 200); // Instrumented wait: poll IRQ flags, DIO1 level and chip mode while
*tx_done_us = s_irq_ts; // also honoring the ISR notification. Gives ground truth about the
// TX sequence even when the interrupt path is broken.
int64_t deadline = esp_timer_get_time() + (airtime_ms + 200) * 1000LL;
uint16_t irq = 0;
bool notify_seen = false, dio1_seen = false;
uint8_t max_mode = 0;
while (esp_timer_get_time() < deadline) {
if (ulTaskNotifyTake(pdTRUE, 0) > 0) notify_seen = true;
if (gpio_get_level(BOARD_LORA_DIO1)) dio1_seen = true;
uint8_t st = sx126x_get_status();
uint8_t mode = (st >> 4) & 0x7;
if (mode > max_mode) max_mode = mode;
irq = sx126x_get_irq();
if (irq) break;
vTaskDelay(pdMS_TO_TICKS(1));
}
*tx_done_us = notify_seen ? s_irq_ts : esp_timer_get_time();
fem_set_tx(false); fem_set_tx(false);
sx126x_clear_irq(SX126X_IRQ_ALL);
LOGJ("\"event\":\"tx_diag\",\"irq\":%u,\"notify\":%s,\"dio1_high_seen\":%s,"
"\"max_chip_mode\":%u,\"device_errors\":%u",
irq, notify_seen ? "true" : "false", dio1_seen ? "true" : "false",
max_mode, sx126x_get_device_errors());
if (irq & SX126X_IRQ_TX_DONE) { if (irq & SX126X_IRQ_TX_DONE) {
g_stats.tx_success++; g_stats.tx_success++;
return "ok"; return notify_seen ? "ok" : "ok_polled";
} }
if (irq & SX126X_IRQ_TIMEOUT) { if (irq & SX126X_IRQ_TIMEOUT) {
g_stats.tx_timeout++; g_stats.tx_timeout++;
@@ -231,6 +258,62 @@ static void run_test_tx(void)
sx126x_restore_rx_payload_len(); sx126x_restore_rx_payload_len();
if (!sx126x_set_rx()) recover_ladder("test_tx_rx_restore"); if (!sx126x_set_rx()) recover_ladder("test_tx_rx_restore");
} }
// ---------------------------------------------------------------- probe
static void hexdump_json(const char *key, const uint8_t *d, int n)
{
printf("\"%s\":\"", key);
print_hex(d, n);
printf("\",");
}
// Scripted low-level diagnostic with raw SPI response dumps.
static void run_probe(void)
{
uint8_t r[16];
printf("{\"event\":\"probe\",");
sx126x_standby();
sx126x_raw(0xC0, NULL, 0, r, 2); // GetStatus
hexdump_json("status0", r, 2);
// SetDioIrqParams: all IRQs, all on DIO1
uint8_t di[8] = {0x03, 0xFF, 0x03, 0xFF, 0, 0, 0, 0};
sx126x_raw(0x08, di, 8, r, 9);
hexdump_json("setirq_resp", r, 9);
sx126x_raw(0xC0, NULL, 0, r, 2); // status after SetDioIrqParams
hexdump_json("status_after_setirq", r, 2);
sx126x_raw(0x02, (uint8_t[]){0x03, 0xFF}, 2, NULL, 0); // ClearIrq
sx126x_raw(0x12, NULL, 0, r, 4); // GetIrqStatus raw
hexdump_json("irq_cleared", r, 4);
// small TX: 8 byte payload
uint8_t pkt[8] = {0x54, 0xCD, 0xAA, 0x55, 0xDE, 0xAD, 0xBE, 0xEF};
sx126x_start_tx(pkt, 8);
printf("\"txpoll\":[");
for (int i = 0; i < 12; i++) {
vTaskDelay(pdMS_TO_TICKS(2));
uint8_t s2[2], q[4];
sx126x_raw(0xC0, NULL, 0, s2, 2);
sx126x_raw(0x12, NULL, 0, q, 4);
printf("\"%02X-%02X%02X%02X%02X\"%s", s2[1], q[0], q[1], q[2], q[3],
i < 11 ? "," : "");
}
printf("],");
fem_set_tx(false);
sx126x_raw(0x17, NULL, 0, r, 4); // GetDeviceErrors
hexdump_json("dev_errors", r, 4);
sx126x_clear_irq(SX126X_IRQ_ALL);
sx126x_restore_rx_payload_len();
sx126x_set_rx();
printf("\"done\":true}\n");
fflush(stdout);
}
// ------------------------------------------------------------------ RX path // ------------------------------------------------------------------ RX path
@@ -333,6 +416,7 @@ static void repeater_task(void *arg)
{ {
(void)arg; (void)arg;
s_task = xTaskGetCurrentTaskHandle(); s_task = xTaskGetCurrentTaskHandle();
sx126x_set_irq_callback(irq_cb);
esp_task_wdt_add(NULL); esp_task_wdt_add(NULL);
if (!radio_full_init()) { if (!radio_full_init()) {
@@ -354,6 +438,10 @@ static void repeater_task(void *arg)
s_want_test_tx = false; s_want_test_tx = false;
run_test_tx(); run_test_tx();
} }
if (s_want_probe) {
s_want_probe = false;
run_probe();
}
if (ulTaskNotifyTake(pdTRUE, pdMS_TO_TICKS(1000)) > 0) { if (ulTaskNotifyTake(pdTRUE, pdMS_TO_TICKS(1000)) > 0) {
uint16_t irq = sx126x_get_irq(); uint16_t irq = sx126x_get_irq();
@@ -388,6 +476,7 @@ void repeater_start(void)
void repeater_request_radio_reset(void) { s_want_radio_reset = true; } void repeater_request_radio_reset(void) { s_want_radio_reset = true; }
void repeater_request_test_tx(void) { s_want_test_tx = true; } void repeater_request_test_tx(void) { s_want_test_tx = true; }
void repeater_request_probe(void) { s_want_probe = true; }
// ------------------------------------------------------------------ status // ------------------------------------------------------------------ status
+1
View File
@@ -36,6 +36,7 @@ extern repeater_stats_t g_stats;
void repeater_start(void); void repeater_start(void);
void repeater_request_radio_reset(void); void repeater_request_radio_reset(void);
void repeater_request_test_tx(void); void repeater_request_test_tx(void);
void repeater_request_probe(void);
void repeater_print_status(void); void repeater_print_status(void);
void repeater_print_stats(void); void repeater_print_stats(void);
void repeater_print_radio_status(void); void repeater_print_radio_status(void);
+34 -8
View File
@@ -231,10 +231,22 @@ bool sx126x_init(sx126x_diag_t *diag)
read_regs(REG_SYNCWORD_0, verify, 2); read_regs(REG_SYNCWORD_0, verify, 2);
diag->spi_ok = (verify[0] == 0x54 && verify[1] == 0x3D); diag->spi_ok = (verify[0] == 0x54 && verify[1] == 0x3D);
// TCXO on DIO3 (1.8 V, ~4 ms), then recalibrate everything because // TCXO on DIO3 (1.8 V, ~4 ms). The first calibrate after enabling the
// enabling the TCXO invalidates the RC-based calibration. // TCXO latches a benign XOSC_START_ERR (the XOSC block briefly probes
// the crystal before DIO3 powers the TCXO). Correct sequence: enable,
// clear stale errors, force STDBY_XOSC (actually starts the TCXO),
// recalibrate, then judge the remaining error flags.
uint8_t tcxo[4] = {BOARD_TCXO_VOLTAGE_CODE, 0x00, 0x00, BOARD_TCXO_DELAY_STEPS}; uint8_t tcxo[4] = {BOARD_TCXO_VOLTAGE_CODE, 0x00, 0x00, BOARD_TCXO_DELAY_STEPS};
cmd(OP_SET_DIO3_TCXO_CTRL, tcxo, 4); cmd(OP_SET_DIO3_TCXO_CTRL, tcxo, 4);
cmd(OP_CLEAR_DEVICE_ERRORS, (uint8_t[]){0, 0}, 2);
uint8_t stdby_xosc = 0x01;
cmd(OP_SET_STANDBY, &stdby_xosc, 1);
vTaskDelay(pdMS_TO_TICKS(10));
busy_wait();
bool xosc_running = (((sx126x_get_status() >> 4) & 0x7) == 0x3);
cmd(OP_CLEAR_DEVICE_ERRORS, (uint8_t[]){0, 0}, 2);
uint8_t calib = 0x7F; uint8_t calib = 0x7F;
cmd(OP_CALIBRATE, &calib, 1); cmd(OP_CALIBRATE, &calib, 1);
vTaskDelay(pdMS_TO_TICKS(10)); vTaskDelay(pdMS_TO_TICKS(10));
@@ -242,9 +254,9 @@ bool sx126x_init(sx126x_diag_t *diag)
uint16_t errs = sx126x_get_device_errors(); uint16_t errs = sx126x_get_device_errors();
diag->device_errors = errs; diag->device_errors = errs;
// XOSC_START_ERR (bit 5) would indicate TCXO trouble diag->tcxo_ok = xosc_running && ((errs & 0x20) == 0);
diag->tcxo_ok = ((errs & 0x20) == 0);
if (errs) cmd(OP_CLEAR_DEVICE_ERRORS, (uint8_t[]){0, 0}, 2); if (errs) cmd(OP_CLEAR_DEVICE_ERRORS, (uint8_t[]){0, 0}, 2);
cmd(OP_SET_STANDBY, &(uint8_t){0x00}, 1);
// DC-DC regulator (Heltec module has the inductor; RNode uses default LDO, // DC-DC regulator (Heltec module has the inductor; RNode uses default LDO,
// but SX1262 modules from Heltec are DCDC-capable. Use DCDC for lower // but SX1262 modules from Heltec are DCDC-capable. Use DCDC for lower
@@ -312,11 +324,11 @@ bool sx126x_config_wmbus_c1(uint32_t freq_hz, uint8_t rx_payload_len)
ok &= cmd(OP_SET_BUFFER_BASE, (uint8_t[]){0x00, 0x00}, 2); ok &= cmd(OP_SET_BUFFER_BASE, (uint8_t[]){0x00, 0x00}, 2);
// IRQs on DIO1 // IRQs: enable ALL in the latch mask (diagnosis of missing TxDone/RxDone;
uint16_t irqs = SX126X_IRQ_TX_DONE | SX126X_IRQ_RX_DONE | SX126X_IRQ_TIMEOUT; // flags only latch when enabled here), route all to DIO1.
uint8_t di[8] = { uint8_t di[8] = {
irqs >> 8, irqs & 0xFF, // IRQ mask 0x03, 0xFF, // IRQ mask: all
irqs >> 8, irqs & 0xFF, // DIO1 0x02, 0x03, // DIO1: TxDone | RxDone | Timeout
0x00, 0x00, // DIO2 (RF switch) 0x00, 0x00, // DIO2 (RF switch)
0x00, 0x00, // DIO3 (TCXO) 0x00, 0x00, // DIO3 (TCXO)
}; };
@@ -445,6 +457,20 @@ uint16_t sx126x_get_device_errors(void)
if (!cmd_read(OP_GET_DEVICE_ERRORS, e, 2)) return 0xFFFF; if (!cmd_read(OP_GET_DEVICE_ERRORS, e, 2)) return 0xFFFF;
return ((uint16_t)e[0] << 8) | e[1]; return ((uint16_t)e[0] << 8) | e[1];
} }
bool sx126x_raw(uint8_t op, const uint8_t *args, size_t n_tx,
uint8_t *rx, size_t rx_len)
{
uint8_t tx[32] = {0}, rxb[32] = {0};
size_t total = 1 + n_tx;
if (rx_len > total) total = rx_len;
if (total > sizeof(tx)) return false;
tx[0] = op;
if (n_tx) memcpy(tx + 1, args, n_tx);
if (!xfer(tx, rxb, total)) return false;
if (rx && rx_len) memcpy(rx, rxb, rx_len);
return true;
}
uint8_t sx126x_get_status(void) uint8_t sx126x_get_status(void)
{ {
+4
View File
@@ -49,6 +49,10 @@ int sx126x_read_rx(uint8_t *out, size_t maxlen, int8_t *rssi_dbm);
int8_t sx126x_get_rssi_inst(void); int8_t sx126x_get_rssi_inst(void);
uint16_t sx126x_get_device_errors(void); uint16_t sx126x_get_device_errors(void);
uint8_t sx126x_get_status(void); uint8_t sx126x_get_status(void);
// Raw command exchange for diagnostics: sends op + n_tx arg bytes, captures
// the full-duplex MISO bytes into rx (same length). Returns false on error.
bool sx126x_raw(uint8_t op, const uint8_t *args, size_t n_tx,
uint8_t *rx, size_t rx_len);
// FEM control (exposed for diagnostics) // FEM control (exposed for diagnostics)
void fem_set_tx(bool tx); void fem_set_tx(bool tx);