Skip to content

feat(mcp): Record side quest next-call fingerprints so taken quests can be measured - #2393

Draft
sentry-junior[bot] wants to merge 1 commit into
mainfrom
feat/mcp-side-quest-taken-fingerprints
Draft

sentry-junior[bot] wants to merge 1 commit into
mainfrom
feat/mcp-side-quest-taken-fingerprints

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

We record when a side quest is shown, but not whether the agent took it. This adds what a funnel needs, for every quest that has next calls, with no server-side state:

  • Offered: each shown quest records its next calls as app.side_quest.<id>.next, a comma-separated list of fingerprints. Only calls the session can make are recorded, which is the same list the agent sees.
  • Taken: every tool call records app.tool_call.fingerprint. For execute_sentry_tool, it goes on the child span of the tool it runs, not on the wrapper.

A fingerprint is a 32-bit FNV-1a hash of the tool name and its arguments, sorted by key. It leaves out scope keys (organization, project, region) and empty values. The server fills scope keys from constraints or the default org, so an agent may or may not pass them, and including them would miss real matches.

Measuring it: "taken" means a later span from the same user.id, within a window (say 60 min), whose fingerprint is in an earlier span's .next list. Remote spans are sampled at 0.3. Afaict each tool call is its own trace, so both sides are sampled independently. Start from sampled spans that showed the quest, count how many have a sampled matching follow-up, and divide that share by 0.3. stdio samples at 1.0, so its pairs are complete.

Not covered:

  • A follow-up with different arguments is missed. Examples: get_sentry_resource with a url or issueId alias instead of resourceId, or the agent going straight to update_issue on a related issue. Those count as not taken, so the rate is a lower bound.
  • related_issues_open lists up to 3 issues in details, but next (and so .next) holds only the first 2.
  • Quests with only a tip and a docs link have nothing to match.

Open decision: this uses span attributes only and does the matching in analytics (DATA-672). An exact, unsampled app.side_quest.taken metric would need the server to remember offered fingerprints per user: an in-memory map for stdio, and MCP_CACHE KV for remote. That is left out on purpose and can come later if the sampled estimate is too noisy.

via @Christinarlong.

--

View Session [Sentry]

…an be measured

Each shown side quest records fingerprints of the next calls it offered as
app.side_quest.<id>.next, and every tool call records its own fingerprint as
app.tool_call.fingerprint (on the inner span for execute_sentry_tool).
Analytics can then join a later call from the same user to the quest that
offered it, with no server-side state.

Co-Authored-By: Christina Long <60594860+Christinarlong@users.noreply.github.com>
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sentry-local Ready Ready Preview Nov 18, 2026 7:32pm UTC

Request Review

This branch was successfully deployed

2 active deployments
Preview — d9103ddf Deployed Nov 18, 2026 by vercel[bot]
Actions — d9103ddf Deployed Nov 18, 2026 by sentry-junior[bot] via eval #1965
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant