Repository navigation
fix(agent): free all thread data when BoundedMemorySaver evicts - #108
Open
asasemahmed wants to merge 1 commit into
Open
asasemahmed wants to merge 1 commit into
asasemahmed wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BoundedMemorySaverhas two bugs, which together undermine what it is for (staying within a 512MB host, see #65).self.storage[thread_id].MemorySaveralso keeps every thread's channel values, including the messages, inself.blobs, and its pending writes inself.writes. Evicted conversations stay in memory for good. Withmax_threads=2and 50 threads,storageheld 2 threads whileblobsandwritesstill held all 50.len(self.storage). Butget_tuple()/list()index the storagedefaultdict, so just reading an unknown thread adds an entry. The AG-UI adapter does that before every run. Those entries count toward the limit but are never tracked for eviction. Once enough of them exist, everyput()evicts the thread it just wrote, and the conversation is lost right away.Eviction now counts the threads that
put()tracks and callsMemorySaver.delete_thread(), which removes the thread's checkpoints, writes and blobs.Tests: the new
tests/test_bounded_memory_saver.pycovers both cases with a small compiled graph. Full suite:uv run --directory apps/agent pytest(113 passed).