Skip to content

feat(docs): Add a cookie-free Fin Ask AI pilot - #20070

Open
rahulchhabria wants to merge 2 commits into
masterfrom
feat/fin-ask-ai-pilot
Open

rahulchhabria wants to merge 2 commits into
masterfrom
feat/fin-ask-ai-pilot

Conversation

@rahulchhabria

@rahulchhabria rahulchhabria commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

DESCRIBE YOUR PR

Add an opt-in Fin API pilot that preserves the docs Ask AI entry points and streams answers in a first-party dialog. This lets us evaluate Fin alongside our existing support workflow without loading Intercom Messenger or adding browser cookies or persistent storage. Kapa remains the default until the pilot is explicitly enabled.

The pilot includes sanitized answers and citations, follow-up questions, helpfulness feedback, transcript copying, and optional support handoff. Server-only Fin credentials, fixed-origin requests, shared Redis limits, and session locks bound the public API. Signed webhooks and a protected event export provide the input for a future internal agent that proposes docs changes; the docs-writing/PR worker and human-reply ingestion are outside this pilot.

The pilot guide lists API/SSE access, secrets, Redis, content configuration, support routing, privacy decisions, and permissions needed before a live test. Fin API access and streaming approval still need confirmation. Support handoff is disabled by default.

Validation: all 524 tests across 43 files passed in GitHub Actions, along with lint, spelling, links, CodeQL, and dependency/security checks. Warden and Bugbot completed with no issues found on the latest commit. Both Vercel preview builds passed.

Local checks passed 27 targeted tests, TypeScript, ESLint, and Prettier. Browser checks exercised the actual provider and dialog on desktop/mobile using simulated Fin responses, including sanitization, follow-ups, feedback, reset, focus, and support handoff. No new cookies, persistent browser storage, or third-party requests were observed in that harness. Regression tests cover preserving search until Kapa loads and releasing overlay scroll locks before opening Ask AI.

Live Fin responses still require the credentials and approvals listed in the pilot guide. The deployed preview has not been manually checked.

IS YOUR CHANGE URGENT?

  • Urgent deadline (GA date, etc.): YYYY-MM-DD
  • Other deadline: YYYY-MM-DD
  • No deadline: Not urgent, can wait up to 1 week+

PRE-MERGE CHECKLIST

  • Checked Vercel preview for correctness, including links
  • PR was reviewed and approved by any necessary SMEs (subject matter experts)
  • PR was reviewed and approved by a member of the Sentry docs team

Preserve Ask AI entry points with a first-party streaming dialog and server-only Fin calls. Add support handoff, durable feedback events, pilot limits, tests, and the access checklist.
@vercel

vercel Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
develop-docs Ready Ready Preview Oct 19, 2026 3:53am UTC
sentry-docs Ready Ready Preview Oct 19, 2026 3:53am UTC

Request Review

@github-actions github-actions Bot added the Priority: Normal Docs review has no urgent deadline label Oct 19, 2026
@codeowner-assignment
codeowner-assignment Bot requested a review from a team October 19, 2026 03:25
@cursor

cursor Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

The plan checks Ask AI reachability, search volume, and server failures. An issue escalates when Ask AI traffic collapses or search errors rise. The Fin dialog stays off until the deployment turns the Fin provider on.

Services: sentry-docs, develop-docs.

Mention @change-monitor in a comment to update the plan.

Plan

What changed

Ask AI buttons now open through a first-party provider. The default path still opens the current Kapa widget. Kapa stays loaded when ASK_AI_PROVIDER is not fin.

The Fin dialog is reachable when ASK_AI_PROVIDER is fin and FIN_PILOT_ENABLED is true. That dialog posts /api/ask-ai/ and streams sanitized answers. Support handoff stays off unless FIN_HANDOFF_ENABLED is true.

sentry-docs and develop-docs must deploy before this change is live. Those Vercel projects share app/ and src/. Catalog ids match that deploy map.

Risk

A provider crash can blank every page. The provider wraps the root layout. Broken click handlers can stop Ask AI or Algolia search. Turning Fin on too soon removes Kapa and can leave Ask AI with failing streams. Public scanners can hit the new routes and receive 503 while the gate stays off.

Intended effect

Confirm that Kapa widget requests continue while Fin stays off. Confirm that /api/ask-ai/ returns 200 streams only after Fin is on. Absent for Kapa is widget volume near zero while Kapa should still load. Absent for Fin is zero 200 responses after Fin is on. Zero /api/ask-ai/ traffic while Fin stays off is expected and does not confirm Fin.

Signal Baseline Window Rule Source
Kapa widget spans on docs 13023 spans in 24h. 2977 spans in the prior 24h. Band 2977-13023. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z and 2026-10-17T03:25:53Z to 2026-10-18T03:25:53Z While Kapa is default, hold in or above this band. A fall near 0 is absent. After Fin is on, expect a fall toward 0. Sentry org sentry project docs. Query environment:production (http.url:*widget.kapa.ai* OR http.url:*kapa-widget-proxy*).
Kapa widget spans on develop-docs 287 spans in 24h. 300 spans in the prior 24h. Band 287-300. Same windows Same rule on this project. Sentry org sentry project develop-docs. Same query.
/api/ask-ai/ HTTP spans 0 spans. This route is new. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z Hold at 0 while Fin is off. After Fin is on, 200 responses must rise above 0. All 503s after Fin is on is absent. Sentry org sentry projects docs and develop-docs. Query has:http.method http.url:*/api/ask-ai* environment:production.

Regression watch

Watch search, the header Ask AI path, and server 500s on the same page tree. The layout wrap can fail every page. Search click handlers can fail Algolia. Do not treat gated /api/ask-ai/ 503s as a regression while Fin stays off.

Signal Baseline Window Rule Source
docs.search.query 484 queries in 24h. 517 in the prior 24h. Band 484-517. 408 of 484 had results. 7d sum is 12150, about 1736 per day. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z, 2026-10-17T03:25:53Z to 2026-10-18T03:25:53Z, and 2026-10-12T03:25:53Z to 2026-10-19T03:25:53Z Hold in the 484-517 weekend band. Weekday volume may rise toward the 7d mean. A collapse toward 0 is a regression. Sentry metrics org sentry project docs. sum(value, docs.search.query) with environment:production.
Search and header client errors 6 events in 24h. 7 events in the prior 24h. Band 6-7. Share of search queries is 1.2-1.4%. Current titles are Algolia insights fetch failures and getItem on null. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z and 2026-10-17T03:25:53Z to 2026-10-18T03:25:53Z Hold in 6-7 per 24h. A rise above 15 events per 24h, or a new askAi stack, is a regression. Sentry errors org sentry project docs. Query environment:production stack.filename:[*search/index.tsx*, *header.tsx*].
http.server 500 and 502 0 spans in 24h and in 48h. Status mix in 24h is 4418×200, 63×307, 33×404. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z and 2026-10-17T03:25:53Z to 2026-10-19T03:25:53Z Hold at 0 on paths other than /api/ask-ai/. Any new 500 or 502 is a regression. Sentry traces org sentry project docs. Query span.op:http.server http.status_code:[500,502] environment:production.
LCP p75 420ms in 24h. 531ms over 48h. 865ms over 7d. Band 420-865ms. 2026-10-18T03:25:53Z to 2026-10-19T03:25:53Z, 2026-10-17T03:25:53Z to 2026-10-19T03:25:53Z, and 2026-10-12T03:25:53Z to 2026-10-19T03:25:53Z Hold in 420-865ms. A rise above 900ms is a regression. Sentry metrics org sentry project docs. p75(value, browser.web_vital.lcp) with environment:production.

Not observable

Ask AI open clicks are not in Sentry. Plausible Ask AI Referrer is not connected here. Cookie, storage, and third-party request absence cannot be checked from this telemetry. Answer quality, sanitization, and citations have no production signal. Signed webhook and event-export outcomes have no measured traffic. Datadog is not connected.

Get Slack DMs about this change: cursor.com/automations/rollouts

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bc83804. Configure here.

Comment thread src/components/search/index.tsx Outdated
Check provider readiness before releasing search and navigation overlays. Keep the next-frame handoff for scroll locks and cover delayed Kapa loading with regression tests.

This branch was successfully deployed

2 active deployments
Preview – sentry-docs — 80c2c58a Deployed Oct 19, 2026 by vercel[bot]
Preview – develop-docs — 80c2c58a Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority: Normal Docs review has no urgent deadline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant