Skip to content

feat(mcp): Return setup knowledge from create_project - #2358

Merged
dcramer merged 5 commits into
mainfrom
feat/create-project-setup-knowledge
Nov 18, 2026
Merged

dcramer merged 5 commits into
mainfrom
feat/create-project-setup-knowledge

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

create_project now tells the calling agent which Sentry signals the new project can collect and where each one is documented. Before, it returned only a DSN and an instruction string.

"setup": {
  "platform": "Next.js",
  "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/",
  "signals": [
    { "name": "Errors", "usefulFor": "Unhandled exceptions and crashes, grouped into issues with stack traces and the state that led to them", "recommended": true, "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/" },
    { "name": "Traces", "usefulFor": "Debugging how requests, background jobs, and AI agent runs flow across services, and where they fail or slow down", "recommended": true, "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/tracing/" },
    { "name": "Logs", "usefulFor": "Structured app logs linked to the trace and error they came from, for context a stack trace lacks", "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/logs/" },
    { "name": "Metrics", "usefulFor": "Counters, gauges, and distributions to chart and alert on, such as signups, queue depth, or token usage", "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/metrics/" },
    { "name": "Session Replay", "usefulFor": "Understanding how users behave in a web or mobile UI: clicks, navigation, network calls, and errors in a replay, to reproduce bugs and find friction", "docsUrl": "https://docs.sentry.io/platforms/javascript/guides/nextjs/session-replay/" }
  ]
}

Behavior

  • Only the five core signals are listed, and only those the platform's SDK supports. For example, Node.js, Go, and Django get no Session Replay.
  • Errors and Traces are recommended, because the SDK's default init turns them on.
  • Signals the organization hasn't enabled are left out. This uses the same features Sentry's onboarding checks in getDisabledProducts: ourlogs-enabled, tracemetrics-enabled, and session-replay. The lookup runs in parallel with the DSN lookup and is best effort: if it fails, every supported signal is listed.
  • setup is null when the project has no platform, and in that case the organization isn't fetched.
  • The guidance string is removed. It told the agent what to do, which the tool-response design rules forbid. A failed DSN setup now returns a create_dsn suggested action. Otherwise suggestedActions is empty.

Platform data
platforms.json is generated and committed. Running pnpm --filter @sentry/mcp-core run generate-platform-setup reads:

  • Sentry's static/app/data/platforms.tsx: platform IDs, names, and docs links. It skips deprecated platforms and follows docs redirects.
  • docs.sentry.io/doctree.json: a platform supports a signal when its docs page has a tracing, logs, metrics, or session-replay child page. Guides with no signal pages of their own fall back to their docs fallbackGuide (Express → Node.js), or to the nearest parent page (Django → Python).

All 110 non-deprecated Sentry platforms resolve to a setup page. Every docs link is stored as a full https://docs.sentry.io/... URL, so agents never join paths.

Open questions

  • platforms.json is regenerated by hand, like the OTel namespace data. Should a scheduled job refresh it?
  • The sentry-instrument skill (skills.sentry.dev) currently reads only the DSN from this result. It can start using setup once this ships, but that change belongs in the skills repo.

via @dcramer.

--

View Session [Sentry]

create_project now returns `setup` for the new project's platform: its
setup guide on docs.sentry.io and the core signals the platform supports
(errors, traces, logs, metrics, session replay), each with a one-line
`usefulFor` and a docs link. Errors and traces are marked recommended,
matching the SDK's default init. Signals the organization has not
enabled are left out, using the same organization features Sentry's
onboarding checks. That lookup is best effort.

The `guidance` string is removed, since it told the agent what to do
instead of reporting facts. A failed DSN setup now returns a `create_dsn`
suggested action instead.

Platform data is generated into platforms.json by
`pnpm run generate-platform-setup` from Sentry's platform list and the
published docs tree.

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
sentry-local Skipped Skipped Nov 18, 2026 5:00pm UTC

Request Review

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
@dcramer
dcramer marked this pull request as ready for review November 18, 2026 16:15
@dcramer
dcramer requested a review from a team as a code owner November 18, 2026 16:15
@sentry-junior
sentry-junior Bot requested a review from dcramer November 18, 2026 16:15
@github-actions github-actions Bot added the risk: medium PR risk score: medium label Nov 18, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6939abb. Configure here.

@dcramer

dcramer commented Nov 18, 2026

Copy link
Copy Markdown
Member

remove the doc spec change from this PR because we're going to remove the specs in a different PR.

The specs are being removed in a separate PR.
@dcramer

dcramer commented Nov 18, 2026

Copy link
Copy Markdown
Member

we're likely going to need an automation setup that will handle our scheduled regenerations, but it likely should start with a github workflow, and that workflows should open a pr if no other pr exists (or update the existing pr), and do nothing if theres no updates that are needed. it could run daily

f/u in a new thread

@dcramer

dcramer commented Nov 18, 2026

Copy link
Copy Markdown
Member

Expand Session Replay here because it's not just about seeing; it's about understanding user behavior, particularly within a web UI or mobile UI. In general, the "useful for" should be pretty thoughtful but kept concise here. It shouldn't just be generic statements because agents need to understand if this will be helpful within the context of the application. Additionally, make sure the docs URLs are just full URLs instead of this relative path thing. It's too complicated for an agent.

Rewrite each signal's usefulFor so an agent can judge whether it fits the
app, and expand Session Replay to cover understanding user behavior in web
and mobile UIs. The generator now writes absolute docs.sentry.io URLs, so
platforms.json and the runtime no longer join relative paths.
@dcramer
dcramer merged commit 6c48d78 into main Nov 18, 2026
36 of 37 checks passed
@dcramer
dcramer deleted the feat/create-project-setup-knowledge branch November 18, 2026 17:15
dcramer added a commit that referenced this pull request Nov 18, 2026
)

`create_project`'s platform setup data (`platforms.json`), added in
#2358, is regenerated by hand today. This adds
`regenerate-platform-setup.yml`, which runs daily and on manual dispatch
from `main`. It runs `pnpm --filter @sentry/mcp-core run
generate-platform-setup` and keeps one PR with any changes on the fixed
`bot/regenerate-platform-setup` branch. `scripts/platform-setup-pr.mjs`
decides what to do:

- The generated data matches `main`, or matches the open PR: nothing. No
commit, no push, no PR.
- The data changed and no PR is open: commit it on a branch from `main`,
force-push the bot branch (this overwrites a leftover branch from a
closed PR), and open the PR.
- The data changed and the open PR has older data: rebuild the branch
from `main` and force-push it with a lease. The existing PR picks it up,
and no second PR is opened.

The PR is never merged automatically. It follows the
`bump-sentry-api.yml` rolling-PR pattern: the release bot
(`SENTRY_RELEASE_BOT_CLIENT_ID` / `SENTRY_RELEASE_BOT_PRIVATE_KEY`,
already used by `release.yml` and `bump-sentry-api.yml`) pushes and
opens the PR so `test.yml` runs on it. Commits use the
`github-actions[bot]` identity, like that bump. This assumes the release
bot can push branches and open PRs in this repo, which
`bump-sentry-api.yml` already relies on.

Running the generator locally against current sources left
`platforms.json` unchanged.

<!-- junior-request-attribution:start -->
via @dcramer.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:agent%3A892bc9b6695b269a47717d532623c628 -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/agent%3A892bc9b6695b269a47717d532623c628)
[[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A892bc9b6695b269a47717d532623c628/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: David Cramer <23610+dcramer@users.noreply.github.com>

This branch was successfully deployed

1 active and 1 inactive deployments
Actions — 192f1c9a Deployed Nov 18, 2026 by sentry-junior[bot] via eval #1961
Preview — 192f1c9a Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant