Repository navigation
test(jni): pin the vortex.list layout with golden files, and rename its edition to qc2026.11.0 - #85
Merged
Conversation
The list layout is in the draft jni2026.11.0 edition, so nothing upstream promises it stays readable. Add golden files written through the vortex.listLayout write path, and check on every build that: - each file still reads back to its values, through full scans, row ranges, row indices, projections, filters and list_transform pushdown; - today's writer still writes the same list layouts: metadata, children, dtypes and row counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
The family was named after its crate, so an upstream jni family in vortex-jni would collide with it. Edition names aren't stored in files, so files already written read the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
Declare it immutable with a min_library_version, so it promises its files stay readable. The golden files enforce the promise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
Keep it immutable but unfrozen until the fork's first 0.91.0 build, so min_library_version names a release whose readers all read it. The golden files already hold the format fixed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
gitRasheed
approved these changes
Nov 4, 2026
gitRasheed
left a comment
Collaborator
There was a problem hiding this comment.
lgtm, fyi hours written so far record jni2026.11.0, so its checkReadable will reject them on this build but don't think it matters
Collaborator
Author
|
think we're good - but we should clear out the bucket anyway of all the old files we've let everywhere |
This was referenced Nov 4, 2026
ChristopherWirt
pushed a commit
that referenced
this pull request
Nov 5, 2026
## Summary Freezes `qc2026.11.0` in 0.91.0, the follow-up #85 left for when the fork's first 0.91.0 build is cut. Until then the edition is `immutable` but still a draft to `vortex-edition`, which promises nothing about reading its files. **Merge after upstream 0.91.0 is synced into develop**, so the first build that records the freeze is a 0.91.0 one. ## Changes - `min_library_version: Some("0.91.0")` on the list layout edition. - `list_layout_edition_is_frozen` (was `list_layout_edition_becomes_immutable`) also checks it isn't a draft. ## Testing - `cargo test -p vortex-jni --lib`: 71 passed. Signed-off-by: gitRasheed <99349713+gitRasheed@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
vortex.listLayoutwrites thevortex.listlayout, in our draft list layout edition.vortex.listin itsunstableedition, so it can change the format in any merge.What
Golden files in
vortex-jni/goldenfiles/list_layout/, written once through thevortex.listLayoutwrite path. They cover primitive, nullable, string, struct and nested list elements, a list inside a struct, and all-empty lists, each 20,000 rows across several blocks.Every build checks two things:
list_containsandlist_transformpushdown.The files are never regenerated by the check. A separate
#[ignore]d test only writes missing files, for adding cases. The README explains what to do when a check fails.Rename the edition to
qc2026.11.0jni2026.11.0, named after thevortex-jnicrate. An upstreamjnifamily in that crate would collide with it.qc2026.11.0together.Make
qc2026.11.0immutable, freezing in 0.91.0immutable: true, withmin_library_versionstillNone. Vortex freezes an edition by settingmin_library_versionto the release that freezes it."0.91.0"once the fork's first 0.91.0 build is cut. Changes landed after 0.90.0, so"0.90.0"would claim readers that may not read these files.list_layout_edition_becomes_immutablefails if a merge makes it mutable.Checked
I broke the format three ways, and the tests caught each one:
i64offsetsListLayoutMetadatafield renumberedTesting
cargo test -p vortex-jni --lib: 71 passed.cargo clippy -p vortex-jni --all-targets --all-features -- -D warnings: clean.cargo +nightly-2026-09-10 fmt -p vortex-jni: applied.🤖 Generated with Claude Code