Repository navigation
Conversation
jeanlyn
force-pushed
the
fix-ORC-2226-seek-eof
branch
from
October 22, 2026 06:37
e63c171 to
810e161
Compare
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
4 open issues
This test expects anIOExceptionwhen seeking past the end, but the stream implementations… · New These tests useString.format(...)without specifying a locale, which makes output depend on the… · New These tests useString.format(...)without specifying a locale, which makes output depend on the… · New These tests useString.format(...)without specifying a locale, which makes output depend on the… · New
What changed in this PR
Fixes ORC-2226 by allowing InStream.seek(length) for streams that have no planned read ranges (zero bytes assigned), which is needed when predicate pushdown selects an all-null trailing row group whose index points at end-of-stream.
Changes:
- Allow seek-to-EOF (
desired == length) whenbytes == nullin bothUncompressedStreamandCompressedStream(and reset compressed cached state). - Null-guard
UncompressedStream.toString()to avoid masking the real exception on zero-byte streams. - Add unit + regression tests covering seek-to-EOF behavior and SARG selection of all-null row groups.
| File | Description |
|---|---|
| java/core/src/java/org/apache/orc/impl/InStream.java | Accept seek(length) for zero-byte planned streams; reset compressed cached state; guard toString() against currentRange == null. |
| java/core/src/test/org/apache/orc/impl/TestInStream.java | Add focused unit tests for seek-to-EOF on uncompressed/compressed streams and seeking back after EOF. |
| java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java | Add end-to-end regression tests for SARG selection of all-null trailing/middle row groups. |
Review effort: Lite
The row group index of an all-null trailing row group points exactly at the end of the stream. When predicate pushdown selects only such a row group, the read planner assigns zero bytes to the stream, and UncompressedStream.seek() throws IllegalArgumentException because the position is not covered by any disk range. Accept seeking to the end of the stream as a valid EOF position in both UncompressedStream.seek() and CompressedStream.seek(), scoped to streams that have no backing bytes (bytes == null) so that genuinely out-of-bounds seeks on real data still throw. CompressedStream also resets the cached chunk header position since the buffer is dropped. Also null-guard UncompressedStream.toString() for streams without backing bytes.
jeanlyn
force-pushed
the
fix-ORC-2226-seek-eof
branch
from
October 22, 2026 09:57
810e161 to
3209741
Compare
Copilot stopped reviewing on behalf of
cxzl25 due to an error
October 22, 2026 13:00
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What changes were proposed in this pull request?
Scope the fix to exactly the failing condition:
InStream.UncompressedStream.seek(long): acceptdesired == lengthonly when no bytes were read for the stream (bytes == null) — i.e. the read planner assigned a zero-byte range to it — and park at EOF. All other seeks keep the existing behavior, so genuinely out-of-bounds seeks on real data still throw.InStream.CompressedStream.seek(long): same condition. It also resetscurrentCompressedStart, because the buffered chunk is dropped and the cached header position is no longer valid (otherwise a later seek back to that cached position would silently stay at EOF).InStream.UncompressedStream.toString(): null-guardcurrentRange(pre-existing NPE for streams without backing bytes, which would otherwise mask the real exception message).Why are the changes needed?
The row group index of an all-null trailing row group legitimately points at the end of the stream (both the Java and C++ writers record such positions). When predicate pushdown (SARG) selects only that row group,
StripePlanner.processStreamcomputes a zero-byte read range for the stream (start == end), so the stream is created withbytes == null.UncompressedStream.seek()then only special-casesdesired == 0 && bytes == null, and a seek todesired == lengthfalls through the range loop and throws:A full scan (no SARG) reads the same file fine, as does the C++ reader. See ORC-2226 for a minimal, self-contained reproduction (also reproduced on 2.1.2 and 2.3.1).
The condition is deliberately narrowed to
bytes == nullso that no behavior changes for streams that actually have data: forbytes != null, seeking to the stream end keeps the pre-existing semantics (accepted only at the end of the last disk range, rejected otherwise), and corrupt/truncated files still fail loudly instead of silently reading EOF.How was this patch tested?
New unit tests:
TestInStream#testUncompressedSeekToEndOfStream— anUncompressedStreamwith no backing bytes:seek(length)is accepted,read()then returns-1,available()returns0, and seeking past the end (length + 1) still throws.TestInStream#testCompressedSeekToEndOfStream— same forCompressedStream.TestInStream#testCompressedSeekBackFromEndOfStream— read, seek to the end, then seek back to the start: the cached chunk-header state must stay consistent and the re-read must return the correct value.TestRecordReaderImpl#testSargOnAllNullTrailingRowGroup— end-to-end regression: writes an uncompressed file whose trailing row group is all-NULL for a string column, then reads it with a SARG that selects only that row group; asserts both the row count and that the values are all NULL (fails before the fix, passes after).TestRecordReaderImpl#testSargOnAllNullMiddleRowGroup— boundary guard: an all-null row group in the middle of a stripe keeps working via the following row group's estimated end offset.Full
orc-coresuite passes locally: 1532 tests, 0 failures, 0 errors.Was this patch authored or co-authored using generative AI tooling?
Yes, co-authored using generative AI tooling. The analysis, fix and tests were reviewed and verified by a human before submission.
Generated-by: Kimi Code 2.1.1
Notes for reviewers
desired > length), and any non-zero seeks other thandesired == lengthon streams with no backing bytes, still throw — genuinely corrupt files are not masked.EncryptedStream,EncryptedCompressedStream) inherit the fix.