Skip to content

fix: always filter credentials from URLs, and apply DataCollection to failed HTTP request events - #5935

Open
jamescrosswell wants to merge 12 commits into
mainfrom
ai/dc-5429-failed-request-0ffcba
Open

jamescrosswell wants to merge 12 commits into
mainfrom
ai/dc-5429-failed-request-0ffcba

Conversation

@jamescrosswell

@jamescrosswell jamescrosswell commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Phase 2 of #5420. This moves SentryHttpFailedRequestHandler from SendDefaultPii to the resolved DataCollection categories. It also strips URL credentials from the HTTP URLs the SDK records, whatever SendDefaultPii is.

Field Reads
request / response headers RequestHeadersWithLegacyPiiGate / ResponseHeadersWithLegacyPiiGate: Configured ?? (SendDefaultPii ? DenyList : Off)
request cookies (Cookie), response cookies (Set-Cookie) Cookies
query in Request.Url, Request.QueryString UrlQueryParams. No gated variant, because the query was never gated
user:pass@ in Request.Url always dropped
#fragment moved out of Request.Url into the new SentryRequest.Fragment, filtered like the query by UrlFragmentWithLegacyPiiGate: Configured?.UrlQueryParams ?? (SendDefaultPii ? DenyList : Off)

The two *WithLegacyPiiGate accessors, CookieFilter.FilterSetCookieHeader, GetSetCookies and the behaviour-aware AddHeaders / QueryStringFilter overloads are shared with SentryGraphQLHttpFailedRequestHandler. #5432 migrates that handler using them; it is untouched here.

What changes for apps that don't call ConfigureDataCollection

  • URL credentials are stripped where the SDK records a URL. Previously, with SendDefaultPii = true, user:pass@ reached Sentry in the failed-request Request.Url and in outgoing HTTP spans and breadcrumbs.
    • SentryMessageHandler now passes its subclasses the URL with credentials replaced by [Filtered]:[Filtered]@, Structuring Data's placeholder. Span descriptions and breadcrumbs use that. Matching TracePropagationTargets still uses the original.
    • The failed-request handler always drops userinfo, as its SendDefaultPii = false path already did.
    • This is what sentry-java and sentry-python do, at the point each integration records a URL, and Structuring Data requires it "regardless of sendDefaultPii". See the decision on #5429.
  • The Redact() pass in SentryClient is unchanged. It still runs only with SendDefaultPii = false. It rewrites every breadcrumb and span description, including ones the app set itself, and the Data Collection spec says the SDK "MUST NOT modify" those.
  • The fragment moves to request.fragment. This follows sentry-java's Request.fragment and Relay's request.fragment. It is still sent only with SendDefaultPii = true, as before, when it was part of Request.Url. It now goes through the sensitive deny list like the query, so #access_token=abc becomes access_token=[Filtered].
  • Response cookies. With SendDefaultPii = true, failed-request events now carry the response's Set-Cookie cookies, filtered, in contexts.response.cookies. The handler used to read a Cookie header, which responses never send. Each cookie loses its attributes, sensitive names get [Filtered] values, and a nameless or malformed cookie becomes [Filtered], per DataCollection phase 0: parse cookies individually and filter sensitive cookie names #5423.
  • Unchanged: headers and cookies are still collected only with SendDefaultPii = true, and the query is kept and filtered as before.

With ConfigureDataCollection, headers, cookies and the query follow HttpHeaders.Request / HttpHeaders.Response, Cookies and UrlQueryParams, and SendDefaultPii is ignored. The fragment follows UrlQueryParams too. UrlQueryParams = Off strips the query from Request.Url and drops Request.QueryString and Request.Fragment. Deny and allow lists filter the fragment's key=value pairs, and a fragment without any, such as #top, is kept.

Changelog Entry

  • URL credentials (user:pass@) are no longer sent in outgoing HTTP spans, breadcrumbs or failed HTTP request events from SentryHttpMessageHandler, even with SendDefaultPii = true. Custom SentryMessageHandler subclasses now receive the URL with credentials replaced by [Filtered] in ProcessRequest and HandleResponse.
  • With ConfigureDataCollection, failed HTTP request events follow its HttpHeaders, Cookies and UrlQueryParams settings, and UrlQueryParams applies to the URL fragment as well as the query. Without it, they still follow SendDefaultPii.
  • Failed HTTP request events send the URL fragment in request.fragment instead of request.url, with sensitive values such as #access_token=... replaced by [Filtered], and request.url has the same unescaped path whatever SendDefaultPii is.
  • With SendDefaultPii = true, failed HTTP request events now include the response's Set-Cookie cookies, filtered, in contexts.response.cookies.

Worth a look in review

  • RedactUrl was greedy. It matched from https:// to the last @ in the string, so https://example.com/search?q=a@b.com became https://[Filtered]@b.com. It now matches only the authority, as sentry-java parses it. That matters more now that every outgoing HTTP URL goes through it.
  • SentryMessageHandler subclasses now get the credential-free URL in ProcessRequest / HandleResponse. The signatures are unchanged, and request.RequestUri still has the original.
  • Placeholder vs. dropping. Java and Python replace userinfo with [Filtered]:[Filtered]@ in request URLs. Request.Url here drops it instead, which keeps the SendDefaultPii = false output byte-identical. Breadcrumb and span text get the placeholder.
  • The fragment is filtered; other SDKs don't. Java, Python and JS send it as is. Here it follows UrlQueryParams, because fragments can carry tokens (#access_token=...). For SendDefaultPii = true apps without ConfigureDataCollection, that means the deny list now applies to a fragment that used to go out unfiltered.
  • One URL format for everyone. Request.Url now always comes from FilteredHttpRequestUrl, so SendDefaultPii = true apps get the same form SendDefaultPii = false apps always had: an unescaped path, rather than AbsoluteUri's escaped one.
  • Cookie format. Response.Cookies is a string on the event, so Set-Cookie cookies are joined as name=value; name=value, the same way the server-side request path does it. The string-array form from DataCollection phase 0: parse cookies individually and filter sensitive cookie names #5423 is for span attributes.

Closes #5429

🤖 Generated with Claude Code

jamescrosswell and others added 3 commits November 16, 2026 13:36
…aware filters

Adds RequestHeadersWithLegacyPiiGate / ResponseHeadersWithLegacyPiiGate to
DataCollectionResolver for the client-side failed-request handlers, restores
CookieFilter.FilterSetCookieHeader, and lets the header, cookie and query
string helpers take a KeyValueCollectionBehavior. All new parameters default
to the current behaviour.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Collection

Headers use the legacy-gated accessors, cookies use Cookies and the query
string (in Request.Url and Request.QueryString) uses UrlQueryParams. Response
cookies now come from Set-Cookie, which responses send, rather than Cookie,
which they never do.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
URL userinfo used to reach Sentry whenever SendDefaultPii was true. It is now
filtered regardless of SendDefaultPii or DataCollection, as sentry-java and
sentry-python already do and as the span-description spec requires:

- the Redact() cascade over breadcrumbs, transaction and span descriptions runs
  for every event and transaction
- RedactUrl only matches userinfo inside the URL authority, so an '@' in the
  path, query or later text no longer swallows the host
- the failed-request Request.Url always drops userinfo and the fragment, and
  the fragment moves to the new SentryRequest.Fragment, as in sentry-java

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.76%. Comparing base (2e62117) to head (9d4e9f8).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5935      +/-   ##
==========================================
+ Coverage   77.44%   77.76%   +0.31%     
==========================================
  Files         546      535      -11     
  Lines       20053    19762     -291     
  Branches     3962     3950      -12     
==========================================
- Hits        15531    15368     -163     
+ Misses       3658     3543     -115     
+ Partials      864      851      -13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ureDataCollection

Without ConfigureDataCollection, failed-request events with
SendDefaultPii = false never carried the fragment: it was dropped along
with the rest of the unfiltered URL. Keep that, and always send it once
the app opts into DataCollection, as Java, Python and JS do.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jamescrosswell added a commit that referenced this pull request Nov 16, 2026
…or ConfigureDataCollection

Apply #5935's UrlFragmentWithLegacyPiiGate to the GraphQL failed-request
handler, which sets Request.Fragment itself.

Refs #5432

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The single method with a nullable behavior only matched the old rule
because an empty DenyList adds nothing to the built-in terms. Restore the
original one-liner for callers without a behavior and give the
behavior-aware path its own overload, so AddHeaders chooses explicitly.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jamescrosswell and others added 2 commits November 17, 2026 13:15
Matches QueryStringFilter.Filter and FilterHeaderValue.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Point each at #5439, which removes them together with SendDefaultPii.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the addition of Fragment here to match what other SDKs are doing.

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing breaks the lines a PR changes on purpose, one mutation at a time, and checks that a test fails each time. A changed line no test runs fails the check, and a mutant that survives the tests only warns. Full report

Sentry: tests caught 79 of 84 mutants on the changed lines

Stryker couldn't build 4 mutants on the changed lines, so no test ran against them.

Location Mutation Replaced with Status
src/Sentry/Internal/DataCollection/CookieFilter.cs:47 Equality mutation attributesStart <= 0 Survived
src/Sentry/Internal/DataCollection/CookieFilter.cs:49 Equality mutation separator <= 0 Survived
src/Sentry/Internal/DataCollection/CookieFilter.cs:50 Conditional (false) mutation (false?cookie :cookie[(separator + 1)..].TrimStart()) Survived
src/Sentry/Internal/DataCollection/CookieFilter.cs:50 Equality mutation separator <= 0 Survived
src/Sentry/Internal/DataCollection/QueryStringFilter.cs:35 Equality mutation separator <= 0 Survived

jamescrosswell and others added 2 commits November 17, 2026 13:47
…yClient

Running Redact() on every event and transaction also rewrote breadcrumbs
and span descriptions the app set itself, which the Data Collection spec
forbids ("the SDK MUST NOT modify it"). Restore the SendDefaultPii gate in
SentryClient, as on main, and instead have SentryMessageHandler pass its
subclasses the URL with credentials replaced, per Structuring Data. Trace
propagation targets still match against the original URL. sentry-java
applies the same rule where each integration records a URL.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kills two mutants the mutation-testing check reported as surviving.

Refs #5429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/Sentry/SentryMessageHandler.cs Resolved
Comment thread src/Sentry/SentryMessageHandler.cs Resolved
Co-authored-by: James Crosswell <jamescrosswell@users.noreply.github.com>

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to approve, with optional comments

Moves SentryHttpFailedRequestHandler onto the resolved DataCollection categories and strips URL credentials wherever the SDK records an outgoing URL. This covers #5429 as scoped by the decision recorded there, and the fix is at the cause (the handler's SendDefaultPii branch and the greedy RedactUrl regex), not the symptom.

  • Optional: src/Sentry/SentryGraphQLHttpFailedRequestHandler.cs:76 still records AbsoluteUri, so with SendDefaultPii = true GraphQL failed-request events still carry user:pass@ until #5432 lands. Either apply the same Url = uri?.FilteredHttpRequestUrl(...) line here (it doesn't depend on the rest of #5432), or keep the "always" claim out of the changelog wording until then.
  • Optional: There's no ### Changelog Entry, so the auto changelog gets only the title. Users will see several changes: credentials dropped with SendDefaultPii = true, the fragment moving from request.url to request.fragment, Set-Cookie cookies now collected in contexts.response.cookies, the unescaped Request.Url path for PII-on apps, and SentryMessageHandler subclasses (public, with protected ctors) now getting the redacted url in ProcessRequest/HandleResponse (src/Sentry/SentryMessageHandler.cs:86, :109). A short entry listing them would help upgraders.
  • Optional: Fragment isn't filtered at all, and with ConfigureDataCollection it's always sent, even with UrlQueryParams = Off (src/Sentry/SentryHttpFailedRequestHandler.cs:69, src/Sentry/Internal/DataCollectionResolver.cs:61). Fragments can carry tokens (e.g. #access_token=...). Consider running it through the same key/value filter as the query, or dropping it when UrlQueryParams is Off.

This is an automated preliminary review.

Fragments can carry tokens such as #access_token=..., so they now go
through QueryStringFilter with the query's behaviour. With
ConfigureDataCollection the fragment follows UrlQueryParams (Off drops
it); without it, it is still sent only with SendDefaultPii = true, now
through the sensitive deny list.

Raised in sentry-junior's review of #5935.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9d4e9f8. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: high PR risk score: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataCollection phase 2: migrate redact cascade and HTTP failed-request handler

1 participant