Skip to content

fix(models): save and continue with the fitted preprocessor - #562

Open
ChrisW09 wants to merge 2 commits into
mainfrom
fix/save-fitted-preprocessor
Open

ChrisW09 wants to merge 2 commits into
mainfrom
fix/save-fitted-preprocessor

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #561

Problem

set_params(preprocessing_config=...) / set_params(preprocessing_config__...=...) prepare the next fit(). They replace estimator._preprocessor with a new, unfitted preprocessor. The fitted preprocessor that the in-memory model keeps using lives on estimator._data_module. Two code paths read estimator._preprocessor instead:

  • build_save_bundle() saved the unfitted preprocessor, so the artifact's predict() raised NotFittedError after load();
  • _prepare_continuation() built the continuation data with it, so fit(rebuild=False) raised the same error.

Fix

  • deeptab/core/serialization.py (build_save_bundle): the artifact and its metadata take the data module's fitted preprocessor, falling back to estimator._preprocessor when there is none. preprocessor_kwargs and preprocessing_config are still saved as set, so a refit after load() uses the new settings. As a consequence, artifact_metadata["preprocessing"] holds the pending kwargs next to the fitted preprocessor's spec and fingerprint when settings changed after fit().
  • deeptab/models/_mixins/fit.py (_prepare_continuation): continuation reuses the previous data module's fitted preprocessor, as the docs describe ("Continuation also reuses fitted preprocessing").

Tests

tests/integration/test_preprocessing_changes_after_fit.py (new), for MLPRegressor and MLPLSS:

  • test_save_after_set_params_keeps_the_fitted_preprocessor: the reloaded artifact reproduces the in-memory predictions.
  • test_continuation_after_set_params_uses_the_fitted_preprocessor

Both tests change cat_cutoff after fit(): on main, n_bins set through set_params() doesn't reach the new preprocessor (#489), while cat_cutoff does. All four cases fail on main and pass with this change.

Docs

No change needed.

Checks

  • just test: 3240 passed, 46 skipped, 28 xfailed; coverage 91.28%
  • just docs: build succeeded (warnings treated as errors)
  • just check: all hooks passed (ruff, prettier, pyright)

set_params() on preprocessing_config replaces _preprocessor with an
unfitted one for the next fit(), but save() and fit(rebuild=False) read
_preprocessor, so the saved artifact could not predict and continuation
raised NotFittedError. Use the data module's fitted preprocessor in both
places.

Fixes #561
PreprocessingConfig on main ignores n_bins set through set_params()
(#489), so the tests change cat_cutoff, which reaches the new
preprocessor.

Refs #561

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] After set_params(preprocessing_config__...) on a fitted model, save() writes an unfitted preprocessor and fit(rebuild=False) fails

1 participant