Repository navigation
Conversation
A NaN or +inf entry in f_values made computed_c1 NaN, so the C1 consistency check (a NaN comparison, always False) never fired and the candidate's claimed c1 was scored as is. Reject non-finite values before the non-negativity check. -inf was already rejected there and now reports the finite error instead. Fixes algorithmicsuperintelligence#549
|
|
|
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
examples/alphaevolve_math_problems/first_autocorr_ineq/evaluator.pydid not check thatf_valuesis finite. With NaN (or+inf) valuescomputed_c1is NaN, so the consistency checkabs(computed_c1 - c1_achieved) > 1e-6is always False and the candidate's claimedc1is scored as is (run()returningnp.full(10, np.nan), 0.5, 0.0, 10scoredcombined_score3.01).This adds
np.isfinite(f_values).all()right after the shape check and before the non-negativity check.-infwas already rejected by the non-negativity check and now reports the finite error instead.Tests
New
tests/test_first_autocorr_ineq_evaluator.py(same layout astests/test_example_evaluator_integrity.py): all-NaN values (the repro from the issue),+inf, a single NaN entry,-infstill rejected, and a valid finite function that still scores.Fixes #549