Skip to content

fix: drain final response buffers before completing - #497

Draft
atomicdeploy wants to merge 1 commit into
ESP32Async:mainfrom
atomicdeploy:fix/response-backpressure-completion
Draft

atomicdeploy wants to merge 1 commit into
ESP32Async:mainfrom
atomicdeploy:fix/response-backpressure-completion

Conversation

@atomicdeploy

Copy link
Copy Markdown

Summary

Keep AsyncAbstractResponse's final local buffer alive until its bytes have actually been admitted to the TCP queue, and finish only after admitted header/body bytes have been acknowledged.

The current implementation marks RESPONSE_END while filling the final source buffer. A short AsyncClient::add() can leave a tail in _send_buffer; the next request ACK then considers the response finished and closes it. Partial body adds also bypass _writtenLength accounting, while headers can be counted twice.

Changes

  • Separate source EOF from response completion with one boolean.
  • Account for every partial TCP add exactly once; retain zero/partial tails.
  • Preserve the current RESPONSE_TRY_AGAIN credit fix and avoid spending a header credit when zero bytes were admitted.
  • Wait for the final actual ACK; a poll or partial ACK cannot finish an unacknowledged tail.
  • Treat a failed send as an explicit transport failure, not successful completion.

Reproduction and verification

Prepared against exact upstream main 4d4eaca1833c3c11940a988fc8a6c0bdf734bbd2.

tests/run-response-backpressure.mjs extracts and compiles the actual write_send_buffs implementation from src/WebResponses.cpp with small transport/source doubles. It is not a separately rewritten response algorithm. Requires Node.js and a C++17 g++ (or CXX).

node tests/run-response-backpressure.mjs

The optional --baseline SOURCE --output EXECUTABLE mode verifies the pinned original source hash before reproduction. The original method fails deterministically with an eight-byte final source buffer, one admitted body byte and seven pending bytes while already exposing RESPONSE_END. Both in-flight-credit configurations reproduce the failure. Both patched configurations pass:

  • Partial/zero header and body adds, zero TCP window, and partial final ACKs/polls.
  • Empty, known-length, unknown-length, chunked/terminator and 224 KiB bodies with byte-exact reconstruction.
  • RESPONSE_TRY_AGAIN, failed sends and invalid sources.
  • Consistent actual admitted/acknowledged byte accounting.

This is focused host/mocked-transport evidence, not a full Arduino/lwIP build, packet-level measurement, hardware/Wi-Fi soak or proof that every reported network failure has this cause. No installed SDK was modified; this upstream patch follows a downstream immutable-response workaround and its reproducible short-add regression.

@mathieucarbou

Copy link
Copy Markdown
Member

@atomicdeploy please open an issue first describing the bug you saw and an MRE to reproduce it. Ideally written similar to a project exemple.
We’ll discuss in this issue how to fix it. Opening a PR from AI like that does not help understand the issue you had and also does not help us.
Thank you !

lvschouwen added a commit to lvschouwen/split-flap that referenced this pull request Oct 20, 2026
ESPAsyncWebServer 3.12.1 marks a response ended as soon as its last piece
is read from the source. When the TCP queue then refuses that piece
(tcp_write ERR_MEM, likely with the ESP-01's 536-byte segments), the next
ACK closes the connection with the piece still in the send buffer: the row
got a clean close 439 bytes short of the row image, three offers in a row.

patch_asyncweb.py now keeps the source's end apart from the response's
and takes an in-flight credit only for a pass that sent bytes. Every
anchor must occur exactly once. The test runs the patches on the pinned
v3.12.1 excerpts. Upstream PR ESP32Async/ESPAsyncWebServer#497 is the
same bug.

Refs #631

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants