Skip to content

fix(agent): free all thread data when BoundedMemorySaver evicts - #108

Open
asasemahmed wants to merge 1 commit into
CopilotKit:mainfrom
asasemahmed:fix/agent-bounded-saver-eviction
Open

asasemahmed wants to merge 1 commit into
CopilotKit:mainfrom
asasemahmed:fix/agent-bounded-saver-eviction

Conversation

@asasemahmed

Copy link
Copy Markdown

BoundedMemorySaver has two bugs, which together undermine what it is for (staying within a 512MB host, see #65).

  1. Eviction only deletes self.storage[thread_id]. MemorySaver also keeps every thread's channel values, including the messages, in self.blobs, and its pending writes in self.writes. Evicted conversations stay in memory for good. With max_threads=2 and 50 threads, storage held 2 threads while blobs and writes still held all 50.
  2. The limit is checked against len(self.storage). But get_tuple()/list() index the storage defaultdict, so just reading an unknown thread adds an entry. The AG-UI adapter does that before every run. Those entries count toward the limit but are never tracked for eviction. Once enough of them exist, every put() evicts the thread it just wrote, and the conversation is lost right away.

Eviction now counts the threads that put() tracks and calls MemorySaver.delete_thread(), which removes the thread's checkpoints, writes and blobs.

Tests: the new tests/test_bounded_memory_saver.py covers both cases with a small compiled graph. Full suite: uv run --directory apps/agent pytest (113 passed).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant