Repository navigation
feat: DataCollection now sanitizes SQL statements on SqlClient, EF Core and EF6 spans and breadcrumbs - #5966
jamescrosswell wants to merge 5 commits into
Conversation
…re and EF6 spans and breadcrumbs When an app calls ConfigureDataCollection, string and numeric literals in SQL statements are replaced with ? and comments are removed before the statement becomes a span description or EF6 breadcrumb message. Apps that don't opt in are unchanged. The sanitizer is an internal tokenizer modelled on sentry-go's sql integration, with per-dialect rules where SQL Server, MySQL, SQLite and PostgreSQL quoting conflict. Unterminated literals replace the rest of the statement with ?. Refs #5964 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Good to approve, with optional comments
The PR fixes the cause: the three SQL call sites now run the statement through an internal tokenizer before it goes into a span description or breadcrumb, and only for apps that call ConfigureDataCollection. The tokenizer always moves forward and cannot throw, there is no public API change, and the tests cover each call site with the option on and off, plus the EF Core dialect mapping.
- Optional:
Closes #5964will close the issue when this merges, but #5964 lists EF Core query-compile spans (EFQueryCompilerDiagnosticSourceHelper.GetDescription, line 11) as one of the places statements are attached, and this PR leaves them raw on purpose (decision 6). UsePart of #5964or open a follow-up so that path stays tracked. - Optional: EF6 uses generic rules, so on MySQL a backslash-escaped quote also leaks part of a literal, not only
"..."strings:'O\'Brien'becomes?Brien?. Both EF6 call sites have theDbCommand(SentryQueryPerformanceListener.CreateSpan, andSentryCommandInterceptor.Logat lines 61/65 before it callsIQueryLogger.Log(string)), so the dialect could come from the command or connection type, the way EF Core usesdb.system. If you keep generic rules, a test that records this would make the limitation visible.
This is an automated preliminary review.
EF6 has no db.system, so the span and breadcrumb paths used generic rules and a MySQL statement like 'O\'Brien' leaked part of its literal. Map the command's namespace to SQL Server, MySQL or SQLite rules instead, and pass the dialect from SentryCommandInterceptor to SentryQueryLogger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5966 +/- ##
==========================================
+ Coverage 77.69% 77.89% +0.20%
==========================================
Files 535 537 +2
Lines 19727 19903 +176
Branches 3932 3985 +53
==========================================
+ Hits 15326 15504 +178
Misses 3545 3545
+ Partials 856 854 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PostgreSQL uses @ and # as the abs and XOR operators, so @5 and x#5 kept the digit as part of a name and sent it unsanitized. The generic rules now replace the number, and no longer let @ or # continue a word. SQL Server, MySQL and SQLite rules are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Mutation testing breaks the lines a PR changes on purpose, one mutation at a time, and checks that a test fails each time. A changed line no test runs fails the check, and a mutant that survives the tests only warns. Full report Stryker didn't finish, so not every changed line was checked. The job log has the details. |
CI's mutation run timed out, so it was run locally. The surviving mutants pointed at untested cases (the digit 9, lowercase hex, one-digit exponents, a leading .5, block comments at either end or touching a token, and most of the EF6 namespace map) and at two redundant branches, which are removed: the N'/X'/B' case re-scanned a string the quote case already handles, and SQL Server's $.5 money branch added nothing ($.5 still becomes $.?). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When an app calls the experimental
ConfigureDataCollection(...), the SDK now replaces string and numeric literals in SQL statements with?and removes comments before attaching them to:db.queryspan descriptionsApps that don't call
ConfigureDataCollectionsend exactly what they send today. None of these call sites puts the statement anywhere else (nodb.query.textspan data, no breadcrumb data), so the description and the breadcrumb message are the only places it changes.The sanitizer is a small internal tokenizer (
Sentry.Internal.DataCollection.SqlSanitizer), modelled on sentry-go'ssqlintegration. It adds no package dependency.Part of #5420.
Decisions for review
The issue is labelled Needs Discussion. These are the defaults this draft takes.
DataCollectionResolver.SanitizeDbStatementsis true onceSentryOptions.DataCollectionis set, whatever the categories say. There is no new public option, andSendDefaultPiiplays no part. It isn't gated onDatabaseQueryDataeither: ref: remove the experimental DataCollectionOptions.DatabaseQueryData until .NET collects query data #5933 removed that option, and the spec puts statements outside it. Sanitizing for every app, as Go does, would change every app's span descriptions.'...'with''escapes,N'...',X'...',B'...', PostgreSQLE'...'and$$...$$/$tag$...$tag$, decimal, exponent and hex/binary/octal numbers, and--,/* */(nested) and MySQL#comments. It leaves"...",[...]and backtick identifiers, digits inside identifiers (t1,col2) and placeholders (@p0,:name,$1,?) alone. OpenTelemetry's contrib repo has a fuller one,src/Shared/SqlProcessor.cs(Apache-2.0), that also buildsdb.query.summary. Vendoring it is an alternative if we want the summary later."..."(a string in MySQL, an identifier elsewhere), backslash escapes (MySQL),[...](an identifier in SQL Server and SQLite, an array subscript in PostgreSQL),#(a comment in MySQL, a temp table in SQL Server) and$12.50(money in SQL Server, against PostgreSQL's$1parameter). Each call site picks the rules from what it already knows:db.systemit already sets:mssql→ SQL Server,mysqlandspanner→ MySQL,sqlite→ SQLite, anything else (PostgreSQL included) → generic.db.system, so it maps the ADO.NET command's namespace instead.System.Data.SqlClient,Microsoft.Data.SqlClientandSystem.Data.SqlServerCe→ SQL Server;MySql.Data.MySqlClient,MySqlConnectorandDevart.Data.MySql→ MySQL;System.Data.SQLite,Microsoft.Data.SqliteandDevart.Data.SQLite→ SQLite; anything else, including a wrapped command such as MiniProfiler's, → generic. For breadcrumbs,SentryCommandInterceptorpasses the dialect to the SDK's ownSentryQueryLogger; a customIQueryLoggerstill receives the text through the publicLog(string), as before.[as punctuation. That never leaks: at worst a digit inside a bracketed identifier becomes?. They also treat@or#before a digit as PostgreSQL's abs and XOR operators, so@5becomes@?. SQL Server rules keep#1and@1as names."..."as an identifier. sentry-go treats it as a value, but EF Core's SQLite provider quotes every identifier that way, so its statements would becomeSELECT ?.? FROM ? AS ?.?for the rest. An unterminated literal or quoted identifier replaces everything after it with one?, and an unterminated block comment is dropped. go-sqllexer and OpenTelemetry'sSqlProcessordo the same. A fixed description would lose the operation and table for every malformed statement. This keeps the part that could be read and never emits the part that couldn't. The tokenizer can't fail outright: every character is either copied as structure or consumed by a literal or comment.SELECT /* x */ 1→SELECT ?), and the result is trimmed. OpenTelemetry's processor does the same. Oracle's positional:1becomes:?.'...', string constants are printed in"..."without escaping, acharconstant is printed with no quotes (Contains(x)for'x'), andFromSqltext is inlined unquoted. Captured variables already print as@captured. Sanitizing these would need its own approach, such as replacingConstantExpressionnodes inQueryExpressionEventData.Expressionbefore printing, or a fixed description.db.query.summaryand the operation name are tracked by sentry-java in Generate low-cardinality SQL summaries for database spans sentry-java#5981.Notes
db.query.text, with no opt-out (since 1.11.0-beta.1; first stable 1.15.0). Its EF Core instrumentation does the same for SQL-like providers, in prerelease builds only (since 1.13.0-beta.1). This PR doesn't touchSentry.OpenTelemetry.Sentry.Extensions.Loggingturns EF Core's Information-levelExecuted DbCommandlog into a breadcrumb that carries the raw SQL. That is a log message rather than a statement the SDK attaches, so it is left for a separate decision.Closes #5964
🤖 Generated with Claude Code