Skip to content

Avoid storing or transferring discarded each results - #379

Closed
OskarEichler wants to merge 3 commits into
grosser:masterfrom
OskarEichler:codex/avoid-discarded-result-storage
Closed

OskarEichler wants to merge 3 commits into
grosser:masterfrom
OskarEichler:codex/avoid-discarded-result-storage

Conversation

@OskarEichler

Copy link
Copy Markdown
Contributor

Summary

Keep each result storage constant-sized, preserve the original return object with finish callbacks, and discard unused Ractor values before transferring them.

Reproduction and verification

Parallel.each([1, 2], in_threads: 0, finish: ->(*) {}) { |x| x } returns [nil, nil] instead of the original input. All execution modes also retain one nil slot per processed item. With 10,000 items, direct/thread result arrays occupy 89,712/80,464 bytes on this Ruby; the candidate keeps an empty 40-byte array. Callbacks still receive actual values. Ractor each without a finish callback can now discard a Proc result instead of attempting to transfer it.

Eight direct/thread/process/Ractor ownership-storage-callback cases pass; 36 existing each examples pass with two pending. A Proc-returning Ractor fixture exceeds a three-second isolated-process deadline on baseline and finishes on the candidate; the whole baseline process group was killed and waited for.

The combined installed-release candidate passes 800 model checks across direct, thread, process and Ractor modes. Existing selected suite: 165 examples, zero failures, nine pending, baseline and candidate on rbenv Ruby 4.0.6. Global process-name kill tests and the 200-process stress example were excluded for shared-host safety. SQLite examples use temporary databases. An external verification bundle uses already-installed dependencies; upstream Gemfile/lockfile/tests remain unchanged.

Runtime RuboCop passes. Full upstream RuboCop reports the same pre-existing RSpec/MatchWithSimpleRegex offense in spec/parallel_spec.rb:278 on both versions; no full green-lint claim. Gem packaging retains 2.1.0, Ruby >= 3.3, all four published paths and metadata, with no dependency changes.

Compatibility and limitations

Intentional correction: each with a finish callback returns its original source rather than an array of nils. Result-preserving map and callback values remain unchanged. No elapsed-time or production-throughput claim. Input materialization and ordered-callback buffering are separate concerns.

No tests were added or modified under the consuming repository's no-new-tests policy. Reproductions/model checks are external scratch scripts. No production operations or claim of complete concurrency/platform coverage.

@grosser

grosser commented Aug 28, 2026

Copy link
Copy Markdown
Owner

merge mess and some sus-ness making a fresh PR

@grosser grosser closed this Aug 28, 2026
@grosser

grosser commented Aug 29, 2026

Copy link
Copy Markdown
Owner

@OskarEichler lmk if #382 looks right

@OskarEichler

Copy link
Copy Markdown
Contributor Author

#382 looks right. I checked the full diff and the green Ruby 3.3/3.4/4.0 matrix: it preserves each's source return, still provides real values to finish, avoids storing discarded values in direct/thread/process modes, and clears Ractor results before transfer. preserve_results was internal-only, so the rename should not remove a documented API. Thanks for rebuilding it cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants