Repository navigation
fix(store): fill in missing span timestamps on every open - #11
Merged
Merged
Conversation
Retention deletes a trace once MAX(julianday(ts)) is older than the cutoff. A trace whose spans all have a NULL ts gives NULL, so it was never deleted. That state is reachable. 0.2.3 adds the ts column and fills it in, but only while adding the column. 0.2.2 can still write to a migrated database (its INSERT names its columns) and leaves ts empty. So after a rollback to 0.2.2 and an upgrade again, the spans 0.2.2 wrote kept no timestamp, and their traces outlived any retention window. The backfill now runs every time the store opens. A partial index of just the spans missing ts keeps that from scanning the table: inserts always set ts, so the index stays empty (the query plan goes from SCAN spans to the index, which the test checks). The new test writes a span, empties its ts as 0.2.2 would, reopens the store and purges with a cutoff in the future. It fails on the previous code (nothing is purged) and passes with this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 task done
# Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Retention could keep a trace forever. It deletes a trace once
MAX(julianday(ts))is older than the cutoff; a trace whose spans all have a NULLtsgives NULL, so it was never deleted.That state is reachable: 0.2.3 fills in
tsonly while adding the column, but 0.2.2 can still write to a migrated database (its INSERT names its columns) and leavestsempty. After a rollback to 0.2.2 and an upgrade again, those traces outlived any retention window. Found while writing the 0.2.3 upgrade note.The backfill now runs on every open. A partial index of just the spans missing
ts(empty in normal use, since every insert sets it) keeps the check from scanning the table.Testing
tsas 0.2.2 would, reopen, purge with a future cutoff. Fails on the previous code (nothing purged), passes now. It also asserts the startup UPDATE usesspans_missing_ts; without the index the plan isSCAN spans(checked).ruffandmypy --strictclean.Security invariants
tscolumn is written, from the span's owntimestamp. Retention still deletes whole traces only.🤖 Generated with Claude Code