Repository navigation
Conversation
…roxy's JoinSet The proxy in front of the Lambda Runtime API spawns two tasks per invocation into a JoinSet that is joined only at shutdown, so the set grows by two finished tasks per invocation. Remove finished tasks before each spawn, as the lifecycle listener does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The helper-only test would pass even without production task reaping, so a process-level regression is needed to check bounded memory growth after warmup on Linux and clean shutdown.
1 open issue
What changed in this PR
This PR prevents completed tasks from accumulating in the Lambda Runtime API proxy, applying the lifecycle listener’s cleanup pattern from #1508.
Changes:
- Drain completed tasks before spawning invocation-event and response tasks.
- Preserve shutdown handling for tasks still running.
- Add a unit test verifying that the cleanup helper removes completed tasks.
| File | Description |
|---|---|
bottlecap/src/proxy/interceptor.rs |
Adds completed-task cleanup at both spawn sites and a helper unit test. |
Review effort: Balanced
❌ ErrorsYour PR has failed checks. Please review the issues below and take necessary action before merging. 🚦 1 Pipeline job failed
Useful? React with 👍 / 👎 This comment will be updated automatically if new data arrives.🔗 Commit SHA: 0876a08 | Docs | View more details | Give us feedback! |
…on cycles Replace the helper-only test with one that sends 100 next/response cycles through the proxy's router to a fake Runtime API, then checks that the set holds at most two tasks and that shutdown finishes. Without the calls before each spawn, the set holds 200 tasks and the test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 263b912a95
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…the Runtime API proxy Add a process-level test that starts the real binary with the proxy on (the Datadog wrapper with Data Streams Monitoring) and runs 3,500 warm invocations, each with five next/response cycles through the proxy. On Linux, RSS must grow at most 3 MiB after a warm-up of 500 invocations. Without the fix, RSS grew 6.4 to 7.0 MiB over the 3,000 measured invocations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Problem
The extension can run a proxy in front of the Lambda Runtime API. It starts the proxy when the function uses Lambda Web Adapter or the Datadog wrapper, for example with App and API Protection. For each invocation, the proxy starts two tasks in a Tokio
JoinSet: one for the event and one for the response. AJoinSetkeeps each finished task until something joins it, and the proxy joins only at shutdown. So the set grows by two tasks per invocation for the life of the execution environment.This is the same pattern that #1508 fixes in the listener for the tracer's start and end calls. That leak made the extension's memory grow about 14 KB per invocation in #1496. I have not measured the memory growth from the proxy.
This PR
Before each new task, the proxy removes the tasks that have finished, with
JoinSet::try_join_next(). This is the same change as #1508. Each module keeps its own copy of the helper, as each already keeps its own shutdown function for its set. The proxy still joins the tasks that are running at shutdown.This PR is based on #1508. Merge #1508 first.
Testing
Unit test
Steps
cargo test --lib proxy::interceptor.spawn, and run the new test again.Result
The unit test starts a fake Lambda Runtime API and sends 100
nextandresponsecycles through the proxy's routes. It reads each body to the end, because the proxy starts its work for a body only when the whole body passes through. Then it counts the tasks left in the set and runs the proxy's shutdown.With the fix, the unit test passed in 20 of 20 runs.
Process-level test
The new test in the lifecycle contract tests starts the real binary with the Datadog wrapper and Data Streams Monitoring, which turns the proxy on. It runs 3,500 warm invocations. In each one, it sends five
nextandresponsecycles through the proxy on port 9000. On the first invocation, it checks that the fake Lambda API received both forwarded calls. On Linux, it checks that RSS grows at most 3 MiB from invocation 500 to the end, and then that the extension shuts down cleanly.The test sends five cycles per invocation because one leaked proxy task keeps less memory than one leaked listener task. With one cycle per invocation, RSS grew only 1.7 to 2.6 MiB without the fix, under the limit. The proxy does not link its calls to the extension's invocations.
Steps
cargo test --test lifecycle_contract_test runtime_api_proxy_memory_stays_boundedwith this PR.spawn, and run it again.Result
On macOS, the test runs 10 invocations and checks only the calls and the shutdown, because the harness measures memory only on Linux. I did not run it on the x86_64 CI runners yet. I did not run the proxy on Lambda.
🤖 Generated with Claude Code