Skip to content

fix(mbedtls): preserve pending write parameters across retries - #619

Open
Anion3r wants to merge 2 commits into
machinezone:masterfrom
Anion3r:fix/mbedtls-pending-write
Open

Anion3r wants to merge 2 commits into
machinezone:masterfrom
Anion3r:fix/mbedtls-pending-write

Conversation

@Anion3r

@Anion3r Anion3r commented Oct 16, 2026

Copy link
Copy Markdown

When mbedtls_ssl_write() returns WANT_READ or WANT_WRITE, it must be retried with the same arguments until that write completes. WebSocketTransport can append to or reallocate its send queue between retries, while SocketMbedTLS::send() forwards the new pointer and length directly.

This can acknowledge more bytes than were actually delivered: in the deterministic regression, an 8-byte pending tail is retried after another buffer is queued, and the receiver obtains only 49,176 of 65,552 bytes. The resulting gap can remove a WebSocket frame header and cause a protocol-error close.

The adapter now owns a pending copy of at most 16 KiB and preserves its address and length across retryable writes. It returns the actual completed count, clears pending state on completion/error/close, rejects retries shorter than the pending input, and prevents a fatal TLS error from retaining a stale would-block status. Public method signatures and TLS verification policy are unchanged.

Adds a Catch2 memory-BIO test target that uses real Mbed TLS sessions, without external networking, to cover:

  • normal partial writes;
  • repeated WANT_WRITE and WANT_READ while the caller's queue grows, moves, and releases its original buffer;
  • a shorter retry;
  • closing a pending write and reusing the socket for a new session;
  • fatal errors after a would-block result.

The fixture derives its tail size from the active TLS record limit. A private friend accessor keeps session setup in test code. Windows test support links bcrypt for Mbed TLS builds using CNG entropy.

Validation

Tested against Mbed TLS 3.6.5, with a TLS 1.2 PSK memory fixture:

Configuration Regression before Regression after Full CTest suite
Linux x64 Debug 4 of 6 cases fail 6 cases / 214 assertions pass 22/22 targets pass
Windows x64 Release, MSVC 19.27 / SDK 10.0.19041 4 of 6 cases fail 6 cases / 214 assertions pass 19/19 targets pass

The default suite no longer depends on public hosts: DNS/connection checks use loopback addresses and a malformed IPv6 literal, WebSocket start/stop checks use a local echo server, and HTTP redirects use a second local server with an exact response-body assertion. The external Google HTTPS section was removed; existing local TLS-server tests retain real connection/CA verification coverage on Linux. The Windows suite currently has no equivalent local TLS-server target, so its memory-BIO regressions do not cover production TLS connection/CA loading. HTTP gzip assertions now follow IXWEBSOCKET_USE_ZLIB instead of failing when zlib is disabled.

The complete Linux run also passed with a temporary guard that aborts external hostname lookups and non-loopback connections (the explicitly malformed ::invalid input is allowed to exercise local resolver failure). Before the test cleanup, that same guard failed immediately on the existing Google DNS lookup. The guard is validation tooling outside the patch. The Windows complete run used local test endpoints without that guard. No default CTest targets were disabled.

Build with USE_TLS=ON, USE_MBED_TLS=ON, USE_TEST=ON, USE_WS=OFF, USE_ZLIB=OFF, and an installed Mbed TLS prefix. Then:

cmake --build build --parallel 2
ctest --test-dir build -R IXSocketMbedTLSWriteTest --output-on-failure
ctest --test-dir build --output-on-failure --timeout 60 -j 2

Mbed TLS 2.x/4.x and other TLS backends were not executed in this environment. The memory fixture explicitly skips configurations without its TLS 1.2 PSK/PSA prerequisites.

Reference: Mbed TLS write retry contract.

@Anion3r
Anion3r marked this pull request as ready for review October 16, 2026 11:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant