Skip to content

fix(preprocessor): reject NumPy input of the wrong width at transform - #112

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/72-ndarray-width-check
Open

ChrisW09 wants to merge 1 commit into
mainfrom
fix/72-ndarray-width-check

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #72.

A Preprocessor fitted on a NumPy array accepted a transform-time array with extra columns without an error. A leading extra column (for example an accidental ID column) shifted every feature and dropped the last one, and a trailing extra column was ignored, so downstream predictions were silently wrong.

Root cause

Preprocessor.transform never compared the input width with n_features_in_. Array columns are named feature_0, feature_1, ... by position, and the inner ColumnTransformer, fitted on a DataFrame, selects those names and ignores any further columns. An array with extra columns therefore still contained every fitted name. A narrower array raised only an indirect columns are missing error.

Fix

Preprocessor.transform checks 2-D NumPy input before converting it to a DataFrame. If the number of columns differs from n_features_in_, it raises scikit-learn's usual X has n features, but Preprocessor is expecting m features as input. as a PretabDataError, which is a ValueError. DataFrame and dict input is still matched by column name.

Behaviour changes

  • transform with a NumPy array of a different width than at fit now raises PretabDataError with the scikit-learn message. Before, a wider array was accepted and misaligned, and a narrower one raised ValueError: columns are missing: {...}. Code that catches ValueError still catches the new error.

Documentation

  • Preprocessor.transform docstring: new Raises section for an array whose width differs from n_features_in_.
  • docs/core_concepts/configuration.md ("NumPy array input"): array columns are matched by position, so transform raises the usual scikit-learn error for an array with a different number of columns.

Tests

  • test_transform_rejects_an_ndarray_of_a_different_width[2], [4]: a narrower and a wider array both raise the scikit-learn-style message. Fails on main (width 2 raises columns are missing, width 4 is accepted), passes here.
  • test_transform_accepts_an_ndarray_of_the_fitted_width: an array of the fitted width is still accepted. Guard test: passes on both by design.

Reproduction

Output of the script from #72 on this branch:

n_features_in_: 3
predict(X)    : [222.66 100.1  199.4 ]
predict(X_bad): PretabDataError: X has 4 features, but Preprocessor is expecting 3 features as input.
ValueError: X has 4 features, but StandardScaler is expecting 3 features as input.

Checks

  • just test: 1441 passed, 59 skipped, 7 xfailed; coverage 93.65% (gate 90%).
  • Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1428 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
  • just docs: strict build succeeds.
  • just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
  • Import direction between subpackages is unchanged.

Merge notes

This change was previously part of #75, which is closed in favour of one pull request per issue.

Preprocessor recorded n_features_in_ at fit but never checked it. Array
columns are named feature_0, feature_1, ... by position, and the inner
ColumnTransformer selects those names, so an array with an extra leading
column was shifted by one (dropping the last feature) and a trailing
extra column was ignored, with no error. An array with fewer columns
raised an indirect 'columns are missing' error.

transform now raises the usual 'X has n features, but Preprocessor is
expecting m features as input.' error (a PretabDataError, which is a
ValueError) for NumPy input whose width differs from n_features_in_.
DataFrame input is still matched by column name.

Fixes #72

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preprocessor.transform does not check n_features_in_ for ndarray input; extra columns are silently misaligned

1 participant