Skip to content

Seed worker model sampling per iteration (#554) - #561

Open
scttfrdmn wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
scttfrdmn:fix-554-per-iteration-model-seed
Open

scttfrdmn wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
scttfrdmn:fix-554-per-iteration-model-seed

Conversation

@scttfrdmn

Copy link
Copy Markdown

Fixes #554.

Problem

OpenEvolve.__init__ derives one llm_seed and writes it into every model config. Each worker's _lazy_init_worker_components then builds an LLMEnsemble seeded from it. So every worker draws the same sequence of models, and a recycled worker (max_tasks_per_child) restarts that sequence. With N workers, the realized model mix is K draws repeated N times rather than N·K independent draws.

Change

_run_iteration_worker calls _seed_worker_ensemble(iteration) after lazy init. It reseeds the worker's sampler from md5(f"{llm_seed}:{iteration}"), derived the same way llm_seed itself is:

  • draws are independent across workers;
  • runs stay reproducible, and an iteration's draw doesn't depend on which worker runs it or whether that worker was recycled;
  • with random_seed unset, the sampler is left alone (already independent).

The evaluator-model ensemble isn't touched: its only use is generate_all_with_context, which doesn't sample.

Tests

tests/test_worker_model_seeding.py (6 tests) simulates workers as _worker_init builds them:

  • draws are identical for 1, 3 or 4 workers and any iteration order;
  • workers no longer repeat one sequence;
  • realized mix follows weights (0.8 ± 0.06 over 500 iterations);
  • a recycled worker continues the sequence;
  • an unseeded sampler is untouched;
  • _run_iteration_worker seeds with its iteration.

With the reseed made a no-op (the old behaviour), the three behavioural tests fail. Full suite: python -m unittest discover tests, 918 tests, OK (2 skipped). Black-formatted.

Also checked end to end: in openevolve-bedrock, a 40-iteration spawn-mode run with 3 workers against a fake endpoint matches per-model call counts predicted from this derivation exactly. That project currently patches this in from outside, and will drop the patch once a release contains this fix.

…e#554)

Every worker builds its LLMEnsemble from the same llm_seed, so all workers
drew the same sequence of models, and a recycled worker (max_tasks_per_child)
restarted it. _run_iteration_worker now reseeds the worker's sampler from
md5(f"{llm_seed}:{iteration}"), derived like llm_seed itself: draws are
independent across workers and reproducible whichever worker runs which
iteration. With random_seed unset the sampler is left alone.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ensemble model selection is identical across worker processes when random_seed is set

2 participants