Skip to content

BLE: ESP32 to NimBLE, shared BLE helpers, non-blocking nRF52 sends - #3633

Draft
liquidraver wants to merge 6 commits into
meshcore-dev:devfrom
liquidraver:unified-nimble
Draft

liquidraver wants to merge 6 commits into
meshcore-dev:devfrom
liquidraver:unified-nimble

Conversation

@liquidraver

Copy link
Copy Markdown
Contributor

Revives #1277. Six commits, each one builds, so it's easiest to review commit by commit.

What

  • ESP32: Bluedroid → NimBLE-Arduino 2.5.1. Same Nordic UART service, static passkey, MITM and bonding.
  • SerialBLECommon.h, shared by ESP32 / nRF52 / RP2040: UART UUIDs, connection and advertising parameters, the @@MAC name helper, debug macros and a single-producer/single-consumer frame queue.
  • Queue packs frames by size (2 KB). The app's "faster channel sync" fires about 40 GET_CHANNEL writes at once. That overflowed the old 12-slot queue, and channel sync froze at about 30 of 40 (current dev on nRF52 does this too).
  • nRF52: a send never blocks the main loop. We track the SoftDevice's 3 notification slots ourselves. bleuart.write() used to wait up to 100 ms, which is enough to drop LoRa packets on fast presets (SF7/500 kHz: 12–64 ms on air).
  • ESP32:
    • Notifies only after the phone subscribes.
    • Drops oversized frames once the MTU exchange has settled.
    • Lets the phone start pairing (Apple accessory guidelines 11.10).
    • Requests the larger data length before the MTU exchange (11.11).
  • Connection settings: 15–45 ms interval, slave latency 3, 5 s supervision timeout; data length 251, MTU 247.

Heads-up

  • ESP32 phones must re-pair. Old Bluedroid bonds are erased on first boot.
  • Slave latency 3: contact sync is just as fast, but command round trips are about 1.5–3× slower (measured against Windows). Setting it to 0 is a one-line change if we prefer prompt replies.
  • Private NimBLE call: ESP32 uses ble_gatts_set_clt_cfg_perm_flags(), which isn't in NimBLE's public headers, to require an authenticated link for subscribing.
  • Temporary workaround: the 1 s advertising check works around a NimBLE-Arduino ≤ 2.5.1 bug (Update to Mynewt-NimBLE version 1.9 h2zero/NimBLE-Arduino#1104). Drop it once a release includes the fix.

Tested

  • RAK4631:
    • Contact sync 350–400/s.
    • 40/40 channel burst and 100/100 command burst.
    • No main-loop stalls in sends.
    • Out-of-range and reconnect walk test, plus the MeshCore phone app.
  • Xiao S3: NimBLE migration and throughput, 370–510 contacts/s vs 17/s with the old 60 ms gap.
  • Builds: RAK4631, Xiao S3, Xiao C6. The Pico W BLE env already fails on dev.

I only tested it on RAK and Xiao S3, so it needs further testing on other hardware and with other real life scenarios rather than pure bench testing.

One header for the ESP32, nRF52 and RP2040 serial BLE interfaces:
Nordic UART UUIDs, connection and advertising parameters, the "@@MAC"
name helper, debug macros and the frame queue.

The queue is a single-producer/single-consumer ring of 2048 bytes that
stores frames packed by size, so a burst of small commands (the app's
"faster channel sync" sends ~40 GET_CHANNEL requests at once) fits
instead of overflowing fixed slots. RP2040 just switches to it.
Send logic unchanged. Settings now come from the shared header:
15-45 ms connection interval, slave latency 3, 5 s timeout (was
15-30 ms, 4, 2 s); slow advertising 211.25 ms (was 152.5 ms).

When the receive queue is full, only the frame that doesn't fit is
dropped instead of everything buffered. Also rejects a second
connection and restarts the advertising watchdog timer on disconnect.
bleuart.write() waits up to 100 ms for one of the SoftDevice's 3
notification slots, which stalls LoRa handling (at SF7/500 kHz a
packet is only 12-64 ms on air). Count notifications in flight, freed
by HVN_TX_COMPLETE, and only write while a slot is free. The 250 ms
backoff stays for other failures.

Queues are now cleared from the main loop only: clearing them in the
connect/disconnect callbacks could race a pop.

RAK4631, 3 contact syncs plus command bursts: waits inside writes
went from 131 (3.3 s total, longest 74 ms) to none.
Same Nordic UART service, static passkey with MITM and bonding.
Old Bluedroid bonds are erased once, so phones have to re-pair.

Notifications go out from the main loop without the old 60 ms gap,
retrying after 10 ms when the stack is out of buffers. Connection
parameters and the larger data length are requested once the link is
secured, and writes without response are accepted.

Includes a 1 s advertising check that works around NimBLE-Arduino
<= 2.5.1 (h2zero/NimBLE-Arduino#1104); remove it once released.
NimBLE's notify() to a connection handle skips the CCCD check, so
frames queued between pairing and subscription were lost. Send only
once the phone has subscribed.

Start our own MTU exchange and, once it has finished, drop frames too
big for the MTU instead of waiting forever when it stays at 23.
Apple accessory guidelines 11.10: don't request pairing until an ATT
request was rejected with Insufficient Authentication. NimBLE-Arduino
sent a security request when the phone subscribed on an unencrypted
link. The CCCD now needs an authenticated link, as on nRF52 with
Bluefruit, so the phone's subscribe is rejected and the phone pairs.

Guidelines 11.11: request the larger data length before the MTU
exchange.
@liquidraver

Copy link
Copy Markdown
Contributor Author

FYI this will allow setting "faster channel sync" without stalling on channel sync, thanks to the queue redesign

@liquidraver

Copy link
Copy Markdown
Contributor Author

A few numbers:

Board Flash Static RAM
Xiao ESP32-S3 1.56 MB → 1.17 MB (−401 KB, −25%) 190 KB → 178 KB (−12 KB)
Xiao ESP32-C3 1.20 MB → 0.72 MB (−482 KB, −39%) 161 KB → 149 KB (−12 KB)

The RAM figures are static RAM only. Bluedroid also allocates a lot of heap when Bluetooth starts; I haven't measured that saving on hardware yet.

On 4 MB boards that use min_spiffs.csv just to fit the firmware (128 KB filesystem), the C3 build now fits the default 1.25 MB app slot with about 540 KB to spare. That opens the way back to the default layout's 1.375 MB filesystem.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant