[ChrisW09]

Fixes #78.

A Preprocessor fitted on a pandas DataFrame raised columns are missing at transform for the same data passed as a NumPy array of the same width, and one fitted on an array raised for a DataFrame. Pipelines containing a Preprocessor therefore failed at predict with NumPy-based serving, shap.KernelExplainer or df.to_numpy(), all of which scikit-learn's own transformers handle.

Root cause

transform always relabelled array input feature_0, feature_1, ... and then routed columns by name through the fitted ColumnTransformer. Those names never match the labels of a DataFrame fit, and a DataFrame's labels never match the synthetic names of an array fit.

Fix

Preprocessor.transform now converts its input with the new private Preprocessor._align_input instead of to_dataframe:

• The fitted labels come from the inner ColumnTransformer's feature_names_in_. Labels equal to feature_0, ..., feature_{n-1} mark a fit on an array.
• A NumPy array, and a DataFrame or dict passed after an array fit with other labels, are matched by position: they take the fitted labels, with scikit-learn's usual warning (X does not have valid feature names, but Preprocessor was fitted with feature names, or X has feature names, but Preprocessor was fitted without feature names). Numeric columns of an object array are re-inferred with infer_objects, so X.to_numpy() of a mixed frame works.
• Input matched by position whose width differs from n_features_in_ raises PretabDataError with scikit-learn's message X has n features, but Preprocessor is expecting m features as input. This covers any array of the wrong width, which is the check the PR for Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72 (fix(preprocessor): reject NumPy input of the wrong width at transform #112) adds.
• Any other DataFrame is matched by name, as before.

Behaviour changes

• After a fit on string-labelled columns, an array of the fitted width is accepted with a UserWarning (X does not have valid feature names, ...). It raised ValueError: columns are missing before.
• After a fit on an array, a DataFrame or dict of the fitted width with other labels is accepted by position with a UserWarning (X has feature names, ...). It raised columns are missing before.
• Positional input of a different width raises PretabDataError (a ValueError) with the scikit-learn message. Before, a narrower one raised columns are missing, and a wider array after an array fit was silently misaligned (Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72).
• A preprocessor fitted on a DataFrame whose labels are literally feature_0, feature_1, ... is treated like one fitted on an array. A same-width frame with other labels is then matched by position with a warning instead of raising columns are missing (see the Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 follow-up below).

Documentation

• Preprocessor.transform docstring: the X parameter describes matching by name and by position and the two warnings; new Raises section for positional input of the wrong width.
• _align_input docstring.
• docs/core_concepts/configuration.md ("NumPy array input"): a paragraph on a DataFrame fit with array input and an array fit with DataFrame input, the warnings, re-inference of object arrays, and the width error.

Tests

• test_frame_fit_accepts_an_array_of_the_fitted_width: a Preprocessor + Ridge pipeline fitted on a mixed numeric/string frame predicts X.to_numpy() (an object array) with the warning and the same values as for X. Fails on main, passes here.
• test_array_fit_accepts_a_frame_of_the_fitted_width: after an array fit, a renamed frame warns and gives exactly the array's output. Fails on main, passes here.
• test_positional_input_of_the_wrong_width_is_rejected: a narrower array after a frame fit and a narrower frame after an array fit both raise PretabDataError with the scikit-learn message. Fails on main, passes here.

Reproduction

Output of the script from #78 on this branch, with the predictions in the try block printed:

sklearn: [-0.119 -0.096]
fit DataFrame, transform array: [0.112 0.12 ]
fit array, transform DataFrame: [0.112 0.12 ]

Next to scikit-learn's warning for StandardScaler, the two Preprocessor pipelines warn X does not have valid feature names, but Preprocessor was fitted with feature names and X has feature names, but Preprocessor was fitted without feature names.

Checks

• just test: 1441 passed, 59 skipped, 7 xfailed; coverage 93.68% (gate 90%).
• Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1428 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
• just docs: strict build succeeds.
• just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
• Import direction between subpackages is unchanged.

Merge notes

• Base: main.
• This branch contains the same width check that the PR for Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72 (fix(preprocessor): reject NumPy input of the wrong width at transform #112) adds. This fix needs it, because input matched by position must raise at the wrong width, and it covers Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72's array case completely. Whichever PR lands second conflicts in Preprocessor.transform (Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72's inline check and its Raises text). Take this PR's side (X = self._align_input(X) and its Raises text), which keeps Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72's behaviour. Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72's tests pass unchanged on the result, and the full suite passes (1444 passed, 59 skipped, 7 xfailed). docs/core_concepts/configuration.md merges without a conflict but then states the width error twice, so drop Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned #72's sentence. tests/integration/test_preprocessor.py has an append/append conflict: keep both blocks.
• The PR for Preprocessor routes integer column labels positionally, silently swapping columns #55 (fix(preprocessor): route columns to the ColumnTransformer by label #100) also labels array input by position at transform (its _label_array_by_position, after a fit on non-string labels). It conflicts in transform, in the X docstring and next to _align_input. Keep _align_input and this PR's docstring, drop _label_array_by_position, and keep Preprocessor routes integer column labels positionally, silently swapping columns #55's _column_transformer_input and _input_labels. Resolved this way, the full suite passes (1450 passed, 59 skipped, 7 xfailed), and an array after a fit on pd.DataFrame({1: [...], 0: [...]}) is matched by position with the right values. Until the Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 follow-up below, that case also warns that the preprocessor was fitted with feature names, which scikit-learn does not do for non-string labels. Once Preprocessor routes integer column labels positionally, silently swapping columns #55 has landed, a regression test for an integer-labelled fit with array input can be added.
• The PR for Preprocessor.fit crashes on any bool column under default settings #54 (fix(preprocessor): encode boolean columns as binary categoricals #99) adds _column_transformer_input where this PR adds _align_input: keep both methods and both test blocks.
• The PR for Preprocessor with one-hot encoding or preset="expanded" fails under set_config(transform_output="pandas") #76 (fix(preprocessor): keep the internal ColumnTransformer on default output #114) extends the Returns text of transform directly above the new Raises section: keep both.
• The PR for Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94 (fix(preprocessor): accept polars DataFrames as input #129) conflicts in the X docstring of transform (keep Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94's polars type and this PR's text), in docs/core_concepts/configuration.md (keep this paragraph at the end of "NumPy array input", before Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94's new "Polars DataFrame input" section), and at the end of tests/integration/test_preprocessor.py (keep both blocks).
• The PRs for Preprocessor(numerical_method="box-cox") raises at transform for values below the training minimum #59 (fix(preprocessor): keep box-cox finite below the training minimum #104), Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 (fix(preprocessor): accept input_features after fitting on a NumPy array #108) and A 2-D y (multi-output target) crashes the B/M/I splines even when unsupervised, and target-aware methods with unrelated errors #88 (fix(splines): ignore y when unsupervised and reject a multi-output target #123) also append tests at the end of tests/integration/test_preprocessor.py: keep both blocks.
• Follow-up once Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 lands: Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 adds Preprocessor.feature_names_in_. Add not hasattr(self, "feature_names_in_") to fitted_on_array in _align_input, so a DataFrame whose labels are literally feature_0, feature_1, ... counts as a named fit. Choose the warning by hasattr(self, "feature_names_in_") as well, as in fix: resolve 10 high-priority bugs found in a library audit (#76-#85) #86, so a fit on non-string labels does not warn.
• Follow-up once Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94 lands: _align_input detects positional input with isinstance(X, np.ndarray). Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94 reads a list of rows like an array, so the check should become not has_column_labels(X). Otherwise a list of rows is not matched by position after a named fit, and its width is not checked after an array fit.

This change was previously part of #86, which is closed in favour of one pull request per issue.