Skip to content

feat: DataCollection now sanitizes SQL statements on SqlClient, EF Core and EF6 spans and breadcrumbs - #5966

Draft
jamescrosswell wants to merge 5 commits into
mainfrom
ai/dc-5964-sql-sanitize-5d7980
Draft

jamescrosswell wants to merge 5 commits into
mainfrom
ai/dc-5964-sql-sanitize-5d7980

Conversation

@jamescrosswell

@jamescrosswell jamescrosswell commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

When an app calls the experimental ConfigureDataCollection(...), the SDK now replaces string and numeric literals in SQL statements with ? and removes comments before attaching them to:

  • SqlClient db.query span descriptions
  • EF Core command span descriptions
  • EF6 command span descriptions and breadcrumb messages
SELECT * FROM [Users] WHERE [Email] = N'jane@example.com' AND [Id] > 42 -- from checkout
SELECT * FROM [Users] WHERE [Email] = ? AND [Id] > ?

Apps that don't call ConfigureDataCollection send exactly what they send today. None of these call sites puts the statement anywhere else (no db.query.text span data, no breadcrumb data), so the description and the breadcrumb message are the only places it changes.

The sanitizer is a small internal tokenizer (Sentry.Internal.DataCollection.SqlSanitizer), modelled on sentry-go's sql integration. It adds no package dependency.

Part of #5420.

Decisions for review

The issue is labelled Needs Discussion. These are the defaults this draft takes.

  1. Rollout: only for apps that opt in. DataCollectionResolver.SanitizeDbStatements is true once SentryOptions.DataCollection is set, whatever the categories say. There is no new public option, and SendDefaultPii plays no part. It isn't gated on DatabaseQueryData either: ref: remove the experimental DataCollectionOptions.DatabaseQueryData until .NET collects query data #5933 removed that option, and the spec puts statements outside it. Sanitizing for every app, as Go does, would change every app's span descriptions.
  2. Lexer: our own. It handles '...' with '' escapes, N'...', X'...', B'...', PostgreSQL E'...' and $$...$$ / $tag$...$tag$, decimal, exponent and hex/binary/octal numbers, and --, /* */ (nested) and MySQL # comments. It leaves "...", [...] and backtick identifiers, digits inside identifiers (t1, col2) and placeholders (@p0, :name, $1, ?) alone. OpenTelemetry's contrib repo has a fuller one, src/Shared/SqlProcessor.cs (Apache-2.0), that also builds db.query.summary. Vendoring it is an alternative if we want the summary later.
  3. Dialects: one tokenizer, four rule sets. The dialects genuinely conflict in five places: "..." (a string in MySQL, an identifier elsewhere), backslash escapes (MySQL), [...] (an identifier in SQL Server and SQLite, an array subscript in PostgreSQL), # (a comment in MySQL, a temp table in SQL Server) and $12.50 (money in SQL Server, against PostgreSQL's $1 parameter). Each call site picks the rules from what it already knows:
    • SqlClient uses SQL Server rules.
    • EF Core maps the db.system it already sets: mssql → SQL Server, mysql and spanner → MySQL, sqlite → SQLite, anything else (PostgreSQL included) → generic.
    • EF6 has no db.system, so it maps the ADO.NET command's namespace instead. System.Data.SqlClient, Microsoft.Data.SqlClient and System.Data.SqlServerCe → SQL Server; MySql.Data.MySqlClient, MySqlConnector and Devart.Data.MySql → MySQL; System.Data.SQLite, Microsoft.Data.Sqlite and Devart.Data.SQLite → SQLite; anything else, including a wrapped command such as MiniProfiler's, → generic. For breadcrumbs, SentryCommandInterceptor passes the dialect to the SDK's own SentryQueryLogger; a custom IQueryLogger still receives the text through the public Log(string), as before.
    • Generic rules treat [ as punctuation. That never leaks: at worst a digit inside a bracketed identifier becomes ?. They also treat @ or # before a digit as PostgreSQL's abs and XOR operators, so @5 becomes @?. SQL Server rules keep #1 and @1 as names.
    • SQLite keeps "..." as an identifier. sentry-go treats it as a value, but EF Core's SQLite provider quotes every identifier that way, so its statements would become SELECT ?.? FROM ? AS ?.
  4. Failure: sanitize what we can, then ? for the rest. An unterminated literal or quoted identifier replaces everything after it with one ?, and an unterminated block comment is dropped. go-sqllexer and OpenTelemetry's SqlProcessor do the same. A fixed description would lose the operation and table for every malformed statement. This keeps the part that could be read and never emits the part that couldn't. The tokenizer can't fail outright: every character is either copied as structure or consumed by a literal or comment.
  5. Formatting is kept, not normalised. Whitespace stays as it was, a removed comment leaves the whitespace around it (SELECT /* x */ 1 → SELECT ?), and the result is trimmed. OpenTelemetry's processor does the same. Oracle's positional :1 becomes :?.
  6. EF Core query-compile spans are left alone. Their description is the LINQ expression, and EF Core's printer doesn't produce anything a SQL tokenizer can read safely. Captured on EF Core 10 with SQLite:
    'DbSet<Item>()
        .Where(i => i.Name == "it's "quoted" \ back")'
    'DbSet<Item>()
        .Where(i => i.Name.StartsWith("abc") && i.Name.Contains(x))'
    'DbSet<Item>().FromSql(SELECT * FROM Items WHERE Name = 'raw', @p)
    '
    
    The whole expression is wrapped in '...', string constants are printed in "..." without escaping, a char constant is printed with no quotes (Contains(x) for 'x'), and FromSql text is inlined unquoted. Captured variables already print as @captured. Sanitizing these would need its own approach, such as replacing ConstantExpression nodes in QueryExpressionEventData.Expression before printing, or a fixed description.
  7. Query summary is out of scope. db.query.summary and the operation name are tracked by sentry-java in Generate low-cardinality SQL summaries for database spans sentry-java#5981.

Notes

  • OpenTelemetry already sanitizes. Its SqlClient instrumentation always sanitizes db.query.text, with no opt-out (since 1.11.0-beta.1; first stable 1.15.0). Its EF Core instrumentation does the same for SQL-like providers, in prerelease builds only (since 1.13.0-beta.1). This PR doesn't touch Sentry.OpenTelemetry.
  • EF Core's own log messages aren't covered. Sentry.Extensions.Logging turns EF Core's Information-level Executed DbCommand log into a breadcrumb that carries the raw SQL. That is a log message rather than a statement the SDK attaches, so it is left for a separate decision.

Closes #5964

🤖 Generated with Claude Code

…re and EF6 spans and breadcrumbs

When an app calls ConfigureDataCollection, string and numeric literals in
SQL statements are replaced with ? and comments are removed before the
statement becomes a span description or EF6 breadcrumb message. Apps that
don't opt in are unchanged.

The sanitizer is an internal tokenizer modelled on sentry-go's sql
integration, with per-dialect rules where SQL Server, MySQL, SQLite and
PostgreSQL quoting conflict. Unterminated literals replace the rest of the
statement with ?.

Refs #5964

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@sentry-junior sentry-junior Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to approve, with optional comments

The PR fixes the cause: the three SQL call sites now run the statement through an internal tokenizer before it goes into a span description or breadcrumb, and only for apps that call ConfigureDataCollection. The tokenizer always moves forward and cannot throw, there is no public API change, and the tests cover each call site with the option on and off, plus the EF Core dialect mapping.

  • Optional: Closes #5964 will close the issue when this merges, but #5964 lists EF Core query-compile spans (EFQueryCompilerDiagnosticSourceHelper.GetDescription, line 11) as one of the places statements are attached, and this PR leaves them raw on purpose (decision 6). Use Part of #5964 or open a follow-up so that path stays tracked.
  • Optional: EF6 uses generic rules, so on MySQL a backslash-escaped quote also leaks part of a literal, not only "..." strings: 'O\'Brien' becomes ?Brien?. Both EF6 call sites have the DbCommand (SentryQueryPerformanceListener.CreateSpan, and SentryCommandInterceptor.Log at lines 61/65 before it calls IQueryLogger.Log(string)), so the dialect could come from the command or connection type, the way EF Core uses db.system. If you keep generic rules, a test that records this would make the limitation visible.

This is an automated preliminary review.

EF6 has no db.system, so the span and breadcrumb paths used generic rules
and a MySQL statement like 'O\'Brien' leaked part of its literal. Map the
command's namespace to SQL Server, MySQL or SQLite rules instead, and pass
the dialect from SentryCommandInterceptor to SentryQueryLogger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.89%. Comparing base (f08b02c) to head (c0ed2ba).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5966      +/-   ##
==========================================
+ Coverage   77.69%   77.89%   +0.20%     
==========================================
  Files         535      537       +2     
  Lines       19727    19903     +176     
  Branches     3932     3985      +53     
==========================================
+ Hits        15326    15504     +178     
  Misses       3545     3545              
+ Partials      856      854       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jamescrosswell and others added 2 commits November 19, 2026 10:41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PostgreSQL uses @ and # as the abs and XOR operators, so @5 and x#5 kept
the digit as part of a name and sent it unsanitized. The generic rules now
replace the number, and no longer let @ or # continue a word. SQL Server,
MySQL and SQLite rules are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing breaks the lines a PR changes on purpose, one mutation at a time, and checks that a test fails each time. A changed line no test runs fails the check, and a mutant that survives the tests only warns. Full report

Stryker didn't finish, so not every changed line was checked. The job log has the details.

CI's mutation run timed out, so it was run locally. The surviving mutants
pointed at untested cases (the digit 9, lowercase hex, one-digit exponents,
a leading .5, block comments at either end or touching a token, and most of
the EF6 namespace map) and at two redundant branches, which are removed:
the N'/X'/B' case re-scanned a string the quote case already handles, and
SQL Server's $.5 money branch added nothing ($.5 still becomes $.?).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sanitize SQL statements before attaching them to database spans and breadcrumbs

1 participant