Repository navigation
Conversation
- mark.minimizer takes deprecated_options: old option names keep working in the constructor, with_option and algo_options and emit a FutureWarning with the new spelling. - option_group validates dataclasses that bundle related options. - resolve_changed_default announces a default that is about to change. - Tests with a *_wrappers marker only run if selected with -m; a new workflow runs them on pull requests that touch wrappers and monthly against the newest releases of the wrapped libraries. - Raise the minimal versions of fides, DFO-LS and Py-BOBYQA. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Update to fides 0.8.2 and add stepback_strategy="refine". - Rename options to stay closer to fides and to the new trust-region convention; the old names are deprecated until 0.7.0. - Announce the change of the stepback_strategy default to "reflect". - Reject Hessian approximations that need exact Hessians or residuals based on their properties instead of a list of classes. - Document every option by its effect. - Add wrapper tests: contract with fides, behavior, forwarding and deprecations. - Add a tutorial on the advanced options of fides. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Update to DFO-LS 1.6.5 and Py-BOBYQA 1.5.0. - Bundle options in groups that mirror the solvers and expose all solver options optimagic can serve; the old names and dictionaries are deprecated until 0.7.0. - Fix n_evals_per_point, n_fun_evals, stopping_maxfun of nag_pybobyqa and options that were never passed to the solvers. - Stop switching on the diagnostics of the solvers, which slowed them down; n_iterations is no longer reported. - Raise InvalidAlgoOptionError for options the solvers reject. - Document every option by its effect. - Add wrapper tests: contract with the solvers, behavior, forwarding and deprecations. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Rewrite the guide on adding optimizers as a markdown page with the standards for wrappers: option table, coverage, grouping, names, defaults, deprecations, tests and tutorials. - Rewrite the guide on documenting optimizers around an effect-first template for option docstrings. - Add the user-invoked wrap-optimizer skill and a pointer in AGENTS.md. - Update the algorithm documentation and the changelog. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Move the handling of deprecated options from an argument of mark.minimizer to the separate decorator mark.deprecated_options. - Option groups are only accepted as instances of their class. Dictionaries are rejected in the constructor, with_option and algo_options. - nag_dfols and nag_pybobyqa report n_iterations again. The count comes from the diagnostic information of the solvers, with the expensive poisedness computation switched off. - Add the rule that a wrapper keeps reporting what it reported before to the guide on adding optimizers and to the wrap-optimizer skill. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- nag_dfols and nag_pybobyqa: `restarts` and `noisy` each take a boolean or an option group instead of a flag plus a separate group. - Deprecated options can move into an option of the form `bool | <group>`, and converters see the other options of the call. - fides reports status, n_fun_evals and n_jac_evals; the NAG wrappers report the solvers' estimate of the derivative as jac. - Guide and skill: add the rule for features that can be switched on and off, and the rule that a wrapper fills as many fields of InternalOptimizeResult as it can. - The wrapper-tests workflow only runs monthly and by hand. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Notebooks changed:
Review visual diffs and comment on any notebook cell. All comments sync back to this PR. Powered by ReviewNB |
This is the case that #736 fixed for nag_pybobyqa. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem do you want to solve?
Many optimizer wrappers were added long ago and have not been touched since. Their options are only tested on trivial problems, some names are verbose or hide the structure of the wrapped library, some defaults are unexplained, and several options silently do nothing.
This PR sets a standard for wrappers and applies it to the first three:
mark.deprecated_optionsdecorator, validated option groups, and wrapper tests that are selected with library-level markers.fidesis updated to fides 0.8.2 and follows the standard. A new tutorial shows its advanced options.nag_dfolsandnag_pybobyqaare updated to DFO-LS 1.6.5 and Py-BOBYQA 1.5.0 and follow the standard.wrap-optimizerskill for coding agents runs the same workflow for the next wrappers. It stops for a human decision on the option table.Follow-ups are tracked in #744 (least-squares version of fides) and #745 (scipy option aliases).
Changes for users
All old option names keep working until 0.7.0 and emit a
FutureWarningthat shows the new spelling.fides
fides>=0.8.2.stepback_strategy="refine"is new.stepback_strategywill change from"truncate"to"reflect"(the fides default) in 0.7.0. Until then a warning is emitted if the option is not set.HybridFixed,HybridFraction) are now rejected with a clear error instead of failing inside fides.status(the exit flag of fides),n_fun_evalsandn_jac_evals.nag_dfols and nag_pybobyqa
DFO-LS>=1.6.5andPy-BOBYQA>=1.5.0.trustregion_options=NagTrustregionOptions(shrinking_factor=0.9). A group is passed as an instance of its class; dictionaries are rejected.restarts=Trueorrestarts=DfolsRestartOptions(max_unsuccessful_restarts=3), andnoisy=Trueornoisy=NagNoiseOptions(additive_noise_level=0.1). Passing the group switches the feature on. With the defaultrestarts=Nonethe solvers restart if the function is noisy.n_evals_per_point(formerlynoise_n_evals_per_point) could not be used.n_fun_evalscounted points instead of evaluations for DFO-LS and was missing for Py-BOBYQA.stopping_maxfunpassed viaalgo_optionswas ignored bynag_pybobyqa.n_iterationsis now the number of iterations of all runs. The old wrappers reported one less, because they read a zero-based index. The wrappers switch off the most expensive part of the solvers' diagnostics (poisedness), which is not needed for counting iterations.jac: the solvers' estimate of the Jacobian of the residuals (DFO-LS) or of the gradient (Py-BOBYQA) at the solution.algorithm_outputcontainsn_runsandn_evaluated_points.InvalidAlgoOptionErrorinstead of returning NaN parameters.Option tables
fides
hessian_updatehessian_update"bfgs"hessian_update_strategymaxiterstopping_maxitermaxtimestopping_maxtimestopping_max_secondsfatolconvergence_ftol_absfrtolconvergence_ftol_relxtolconvergence_xtol_absgatolconvergence_gtol_absgrtolconvergence_gtol_relsubspace_solversubspace_solver"full"(fides:"2D")trustregion_subspace_dimensionstepback_strategystepback_strategy"truncate","reflect"from 0.7.0trustregion_stepback_strategytheta_maxmax_stepback_fractiontrustregion_max_stepback_fractiondelta_inittrustregion_initial_radiusmutrustregion_shrinking_thresholdtrustregion_decrease_thresholdetatrustregion_expansion_thresholdtrustregion_increase_thresholdgamma1trustregion_shrinking_factortrustregion_decrease_factorgamma2trustregion_expansion_factortrustregion_increase_factorNot exposed:
history_file,verbose,funargs(handled by optimagic);resfun(#744);hess0,x_names,start_id.nag_dfols (D) and nag_pybobyqa (P)
Nonemeans that the option is not passed and the solver uses its own default, which depends on the problem.maxfunstopping_maxfunstopping_max_criterion_evaluationsrhoendconvergence_min_trustregion_radiusconvergence_minimal_trustregion_radius_tolerancerhobegtrustregion_initial_radiusnptn_interpolation_pointstrustregion_n_interpolation_pointsobjfun_has_noisenoisy(Trueor an instance ofNagNoiseOptions)nsamplesn_evals_per_pointnoise_n_evals_per_pointmodel.abs_tolconvergence_target_valueconvergence_criterion_valuemodel.rel_tolconvergence_ftol_scaledseek_global_minimumseek_global_optimumgeneral.rounding_error_constantrounding_error_constantinterpolation_rounding_errorgeneral.safety_step_threshsafety_step_thresholdthreshold_for_safety_stepgeneral.check_objfun_for_overflowcheck_fun_for_overflowclip_criterion_if_overflowinginit.random_initial_directionsinit_options.random_initial_directionsinit.random_directions_make_orthogonalinit_options.random_directions_make_orthogonalrandom_directions_orthogonaltr_radius.eta1trustregion_options.shrinking_thresholdtrustregion_threshold_successfultr_radius.eta2trustregion_options.expansion_thresholdtrustregion_threshold_very_successfultr_radius.gamma_dectrustregion_options.shrinking_factortrustregion_shrinking_factor_not_successfultr_radius.gamma_inctrustregion_options.expansion_factortrustregion_expansion_factor_successfultr_radius.gamma_inc_overlinetrustregion_options.step_expansion_factortrustregion_expansion_factor_very_successfultr_radius.alpha1trustregion_options.min_radius_shrinking_factortrustregion_shrinking_factor_lower_radiustr_radius.alpha2trustregion_options.radius_factor_after_min_radius_shrinkingtrustregion_shrinking_factor_upper_radiusslow.history_for_slowslow_progress_options.history_for_slowslow.thresh_for_slowslow_progress_options.threshold_for_slowslow.max_slow_itersslow_progress_options.max_slow_iterationsnoise.quit_on_noise_levelnoisy.quit_on_noise_levelnoise.scale_factor_for_quitnoisy.scale_factor_for_quitnoise.multiplicative_noise_levelnoisy.multiplicative_noise_levelnoise.additive_noise_levelnoisy.additive_noise_levelinterpolation.preconditioninterpolation_options.preconditiontrustregion_precondition_interpolationinterpolation.throw_error_on_nansinterpolation_options.throw_error_on_nansinterpolation.minimum_change_hessianinterpolation_options.minimum_change_hessiantrustregion_minimum_change_hession_for_underdetermined_interpolationregression.num_extra_stepsregression_options.n_extra_stepstrustregion_n_extra_points_to_replace_successfulregression.increase_num_extra_steps_with_restartregression_options.increase_n_extra_steps_with_restartregression.momentum_extra_stepsregression_options.momentum_extra_stepsrestarts.use_restartsrestarts(True,Falseor an instance of the restart options)restarts.max_unsuccessful_restartsrestarts.max_unsuccessful_restartsrestarts.max_unsuccessful_restarts_totalrestarts.max_unsuccessful_restarts_totalrestarts.rhobeg_scale_after_unsuccessful_restartrestarts.initial_radius_scale_after_unsuccessful_restartrestarts.rhoend_scalerestarts.convergence_min_radius_scalerestarts.use_soft_restartsrestarts.use_soft_restartsrestarts.soft.num_geom_stepsrestarts.soft_n_geometry_stepsrestarts.soft.move_xkrestarts.soft_move_iteraterestarts.soft.max_fake_successful_stepsrestarts.soft_max_fake_successful_stepsrestarts.hard.use_old_rk/use_old_fkrestarts.hard_use_old_fun_valuerestarts.auto_detectrestarts.auto_detectrestarts.auto_detect.historyrestarts.auto_detect_historyrestarts.auto_detect.min_chgJ_slope/min_chg_model_sloperestarts.auto_detect_min_model_change_sloperestarts.auto_detect.min_correlrestarts.auto_detect_min_correlationrestarts.increase_nptrestarts.increase_n_interpolation_pointsrestarts.increase_npt_amtrestarts.increase_n_interpolation_points_amountrestarts.hard.increase_ndirs_initial_amtrestarts.hard_increase_n_initial_directions_amountrestarts.max_nptrestarts.max_n_interpolation_pointsrestarts.throw_error_on_nansrestarts.throw_error_on_nansgrowing.ndirs_initialgrowing_options.n_initial_directionsgrowing.num_new_dirns_each_itergrowing_options.n_new_directions_each_iterationgrowing.delta_scale_new_dirnsgrowing_options.radius_scale_new_directionsgrowing.do_geom_stepsgrowing_options.do_geometry_stepsgrowing.reset_deltagrowing_options.reset_radiusgrowing.reset_rhogrowing_options.reset_min_radiusgrowing.gamma_decgrowing_options.shrinking_factorgrowing.safety.do_safety_stepgrowing_options.do_safety_stepgrowing.safety.reduce_deltagrowing_options.safety_reduce_radiusgrowing.safety.full_geom_stepgrowing_options.safety_full_geometry_stepgrowing.full_rank.use_full_rank_interpgrowing_options.use_full_rank_interpolationgrowing.perturb_trust_region_stepgrowing_options.perturb_trust_region_stepgrowing.full_rank.scale_factorgrowing_options.full_rank_scale_factorgrowing.full_rank.svd_scale_factorgrowing_options.full_rank_svd_scale_factorgrowing.full_rank.min_sing_valgrowing_options.full_rank_min_singular_valuegrowing.full_rank.svd_max_jac_condgrowing_options.full_rank_max_jacobian_condition_numberDeprecated options that are converted instead of renamed: the dictionaries
trustregion_reset_options,trustregion_fast_start_options(D) andconvergence_slow_progress, andinitial_directions,noise_additive_level,noise_multiplicative_level,convergence_noise_corrected_criterion_tolerance,trustregion_method_to_replace_extra_points(D),clip_criterion_if_overflowing(P, no replacement).Not exposed:
scaling_within_bounds: optimagic has its ownscaling.init.run_in_parallel: the solvers still evaluate the function one point at a time.general.check_objfun_for_overflow(P): Py-BOBYQA only catches anOverflowErrorof the function, which optimagic handles before the solver sees it.h,lh,prox_uh,func_tol.*,sfista.*), convex constraints given as projections (projections,dykstra.*,matrix_rank.*) and the evaluation database: not supported yet.Evidence for defaults
All numbers are from the 54 Moré-Wild problems.
fides convergence criteria (BFGS). "Stopped early" means not within 1e-6 of the best value any setting reached.
The harmonized optimagic defaults stay.
fides
subspace_solver."full"and"2D"need about the same number of evaluations (median 16.5 vs. 16.0). On Rosenbrock and random quadratic problems with 100 to 200 parameters, an iteration with"full"takes less than half the time of one with"2D". optimagic therefore keeps"full".fides
stepback_strategy. With synthetic bounds,"truncate"and"reflect"solve the same problems with median 15.0 vs. 14.5 evaluations. There is no evidence for deviating from fides, so the default moves to"reflect".DFO-LS slow-progress threshold. With optimagic's 1e-8 and with the DFO-LS default of 1e-4, 4 of 54 runs end by slow progress. With 1e-8 they stop at a relative error of 1e-10 to 1e-9, with 1e-4 at 5e-8 to 7e-6. The stricter value stays. For both NAG solvers the trust-region radius ends most runs.
Validation and remaining gaps
pixi run tests -m wrappers, about 10 seconds): contract tests against the installed libraries, behavior tests throughom.minimize, forwarding tests and deprecation tests for every old name.ty, the pre-commit hooks and the docs build pass locally.wrapper-testsworkflow runs them monthly against the newest releases of the wrapped libraries and can be started by hand; it can only run after the merge. The patch coverage reported by codecov is therefore lower than the real coverage of the wrappers.optimagic.optimizers.nag_optimizers. There is no shorter public import path yet.trustregion_reset_optionswithoutuse_resets. They are translated correctly if everything is passed in one call. If they are set withwith_optionon an algorithm that is already noisy, they have no effect and the warning says so.restarts.throw_error_on_nans,growing_options.use_full_rank_interpolation) only have forwarding tests.AI assistance
The code, tests, documentation, benchmarks and this description were written with Claude Code. The design decisions (naming and grouping rules, deprecation policy, defaults, test strategy) were made by me in an interview at the start and at a checkpoint on the option tables. The results were checked with the tests and benchmarks described above. I have not yet reviewed the diff line by line; that is the remaining work before this leaves draft status.
Checklist
Closes #XXXX.🤖 Generated with Claude Code