Skip to content

fix(cache): scope exact invalidations - #2142

Open
stussysenik wants to merge 1 commit into
langfuse:mainfrom
stussysenik:fix/prompt-cache-invalidate-prefix
Open

stussysenik wants to merge 1 commit into
langfuse:mainfrom
stussysenik:fix/prompt-cache-invalidate-prefix

Conversation

@stussysenik

@stussysenik stussysenik commented Nov 19, 2026 •

Copy link
Copy Markdown

What does this PR do?

PromptCache.invalidate(prompt_name) deleted any cached key starting with the
given name. Since cache keys are f"{name}-version:{v}" or
f"{name}-label:{l}", invalidating foo also evicted foobar-label:production
(and my-prompt evicted my-prompt-extra). Invalidation now matches those two
exact key shapes, so only the named prompt is evicted.

No issue: langfuse/langfuse-python restricts issue creation to collaborators
(GraphQL CreateIssue permission error, REST POST 404). The bug and a runnable
repro are reproduced here:

from threading import RLock
from langfuse._utils.prompt_cache import PromptCache

c = object.__new__(PromptCache)
c._lock, c._cache = RLock(), {
    "foo-label:production": None, "foobar-label:production": None,
}
c.invalidate("foo")
assert "foobar-label:production" in c._cache  # fails before this PR

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

uv run --frozen ruff check .
uv run --frozen ruff format --check .
uv run --frozen mypy langfuse --no-error-summary
uv run --frozen pytest tests/unit/test_prompt_cache.py -q

Focused: 2 passed. Ruff: pass. Mypy: exit 0. Full unit suite: 1027 passed /
168 failed / 32 errors, the same failures present on clean main.

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed.
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

RetriggerView in GreptileConfidence Score: 4/5

The PR appears safe to merge, with a non-blocking edge case that can cause extra prompt fetches.

Summary

This PR narrows prompt cache invalidation and adds tests for overlapping names.

  • Prompt invalidation keeps similarly named prompts in the cache.

One edge case remains: names containing -label: or -version: can still lose unrelated cache entries.

Reviews (1) · Last reviewed commit: "fix(cache): scope exact invalidations" · Reviewed by Greptile

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Comment on lines +185 to +188
prefixes = (f"{prompt_name}-version:", f"{prompt_name}-label:")
with self._lock:
for key in list(self._cache):
if key.startswith(prompt_name):
if key.startswith(prefixes):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Some names still lose cache. Invalidating foo still removes the generated key foo-label:bar-label:production for a different prompt named foo-label:bar. Prompt names are plain strings, and the documented name rules do not exclude this case. This leaves the exact-name fix incomplete and causes unnecessary fetches. Store the prompt name separately, or use an unambiguous key format, and add a test for names containing these separators.

Prompt To Fix With AI
This is a comment left during a code review.
Path: langfuse/_utils/prompt_cache.py
Line: 185-188

Comment:
**Some names still lose cache.** Invalidating `foo` still removes the generated key `foo-label:bar-label:production` for a different prompt named `foo-label:bar`. Prompt names are plain strings, and the documented name rules do not exclude this case. This leaves the exact-name fix incomplete and causes unnecessary fetches. Store the prompt name separately, or use an unambiguous key format, and add a test for names containing these separators.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant