Skip to content

Separate container input handles from buffered element input - #826

Open
antiguru wants to merge 9 commits into
TimelyDataflow:masterfrom
antiguru:input-handle-split
Open

antiguru wants to merge 9 commits into
TimelyDataflow:masterfrom
antiguru:input-handle-split

Conversation

@antiguru

@antiguru antiguru commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

The input handle currently mixes element and container input, forcing send_batch to finish its element builder to preserve FIFO order. This change makes Handle<T, C> accept complete containers without a builder. Consuming it with with_builder::<CB>() or with_default_builder() produces an element-only BufferedHandle<T, CB>. The wrapper retains the timestamp and attached streams while flushing buffered records on timestamp changes and drop.

Stream registration stays on the existing Input trait and accepts only Handle<T, C>. Callers attach streams with scope.input_from or scope.new_input before adding buffering. The redundant core new_input_with_builder method is removed, and both close methods explicitly drop the consumed handle. Examples, documentation, and the Vec adapter use the simplified construction order.

Workspace tests passed with incremental compilation disabled. Documentation tests passed with the existing capture example on port 8000 excluded. Input integration tests cover fan-out, registration boundaries, conversion, progress, and flushing. The documentation examples cover both shared containers and buffered element input.

Posted by Codex.

@antiguru
antiguru marked this pull request as ready for review October 23, 2026 19:56

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant