[ChrisW09]

Fixes #95.

A column that is missing on every fit row, such as an optional field that is empty in a training window or a cross-validation fold, broke the Preprocessor. Numerical columns and "one-hot" crashed with scikit-learn's "Found array with 0 feature(s)". With "int" and "none", the column silently disappeared from the output while the metadata still listed it, so get_feature_info() and verbose=2 fits raised IndexError.

Root cause

Each per-column pipeline starts with a SimpleImputer built with scikit-learn's default keep_empty_features=False, which drops a feature without any observed value. A pipeline holds exactly one column, so the next step received an (n, 0) array. The scalers, the numerical methods and OneHotEncoder reject it. "int" and "none" accept it and return a 0-width block. build_feature_info then indexed mapping_ of an ordinal encoder that had been fitted on no column.

Fix

• pretab/compose/factory.py: the new _imputer() helper builds the leading imputer with keep_empty_features=True. get_numerical_transformer_steps and get_categorical_transformer_steps use it. An empty column is filled with 0, or with fill_value for strategy="constant", and keeps its block. imputer_kwargs can still override the setting.
• Preprocessor.fit finds the columns with no observed value and emits a DataWarning naming them. The constant policy sees an empty numerical column as the constant it becomes once imputed.
• When fitting the ColumnTransformer raises a ValueError other than a PretabConfigError and an empty column exists, the new Preprocessor._raise_for_empty_features refits each empty column's block on its own. If the block fails, but fits a stand-in column of observed values with the same y, it raises a PretabDataError naming the column and its method, chained to the method's error. Otherwise the original error propagates, so an unrelated failure (for example a y of the wrong length) is not blamed on the empty column.
• build_feature_info reports an "int" block fitted on no column as dimension 0 with no categories instead of raising IndexError.

Behaviour changes

• fit emits a DataWarning naming each column with no observed value. scikit-learn's "Skipping features without any observed values" UserWarning no longer appears.
• An empty column is fitted as a constant: PLE, the scalers, the feature maps, "int" and "one-hot" keep a block for it, and later data where the column is observed transforms with the fitted width.
• Methods that cannot be fitted on a constant column (B/M/I, P, natural and cubic regression splines, Box-Cox) raise a PretabDataError ("Column 'x' has no observed (non-missing) value at fit, and its 'bspline' representation cannot be fitted on such a column ...") instead of scikit-learn's "0 feature(s)" error. With imputation disabled, the column stays missing: the scalers, "quantile" and "none" keep its block as missing values, and every other method raises the same error.
• policy={"constant": "error"} / "warn" now also fires for an all-missing numerical column.
• The step builders get_numerical_transformer_steps / get_categorical_transformer_steps default to keep_empty_features=True. Passing imputer_kwargs={"keep_empty_features": False} restores scikit-learn's default.
• The golden regression baselines pass unchanged.

Documentation

• docs/core_concepts/missing_values.md: a note that a column with no observed value at fit keeps its block, how the imputers fill it, and the DataWarning.
• docs/representations/choosing_a_method.md (edge-case behaviour): a new "Column with no observed value at fit" entry. It lists which methods fit such a column with and without imputation, which ones raise the typed error naming the column, and that policy={"constant": "error"} rejects it. This backs the page's statement that "PreTab is explicit about degenerate inputs rather than failing silently", which the issue quotes.
• Docstrings: a paragraph in the Preprocessor class docstring; Raises (PretabDataError) and Warns (DataWarning) sections in Preprocessor.fit; _imputer and _raise_for_empty_features.

Tests

• test_imputer_keeps_a_column_without_observed_values: an all-NaN column keeps width 1 through the numerical ("minmax") and "one-hot" step pipelines, and imputer_kwargs can restore keep_empty_features=False. Fails on main, passes here.
• test_build_feature_info_reports_a_zero_width_block: an "int" block whose encoder received no column reports dimension 0. Fails on main (IndexError), passes here.
• test_verbose_2_lists_a_column_without_observed_values: a verbose=2 fit warns and lists the empty column in the feature table. Fails on main (IndexError), passes here.
• test_empty_numerical_column_keeps_its_block (27 cases: 14 methods, target-aware on and off): fit warns, the column keeps a block, and output_dims_, feature names, lineage, dict blocks and get_feature_info agree with the output. Data where the column is observed transforms with the fitted width. Fails on main, passes here.
• test_empty_numerical_column_names_the_column_when_the_method_needs_spread (B/M/I, P, natural and cubic regression splines, Box-Cox): PretabDataError naming the column and the method, chained to the method's error. Fails on main, passes here.
• test_empty_column_is_not_blamed_for_an_unrelated_fit_failure (3 cases): with a y that is too short or holds strings, the original error is raised, not the empty-column one. Fails on main, passes here.
• test_empty_numerical_column_under_missing_value_settings (8 settings) and test_empty_categorical_column_keeps_its_block ("int", "one-hot", "none" under 7 settings): the same metadata checks across every missing_policy, with a missing indicator, without imputation, and with constant imputation. Fail on main, pass here.
• test_empty_numerical_column_counts_as_constant_for_the_policy: policy={"constant": "error"} rejects an empty numerical column. Fails on main, passes here.
• test_column_empty_in_a_cross_fitting_fold_keeps_the_width: under CrossFittedTransformer, a column that is observed only in the last fold gives out-of-fold and refit outputs of the same width. Fails on main, passes here.

Reproduction

Output of the script from #95 on this branch. The script was adapted to print each call that now succeeds and the warnings fit emits, where the original hid all warnings:

   fit warns DataWarning: Preprocessor received column(s) ['income'] with no observed (non-missing) value at fit. Their blocks are kept but fitted on an entirely missing column (a single constant value once imputed), so they carry no information until the preprocessor is refit on data where those columns are observed.
numerical all-NaN column -> output_dims_: {'age': 7, 'income': 7, 'city': 1} | output shape: (100, 15)
   fit warns DataWarning: Preprocessor received column(s) ['segment'] with no observed (non-missing) value at fit. Their blocks are kept but fitted on an entirely missing column (a single constant value once imputed), so they carry no information until the preprocessor is refit on data where those columns are observed.
categorical all-NaN column -> categorical_features_: ['city', 'segment'] | output_dims_: {'age': 7, 'city': 1, 'segment': 1} | blocks: ['cat_city', 'cat_segment', 'num_age']
   get_feature_info() -> segment: {'preprocessing': 'imputer -> continuous_ordinal', 'dimension': 1, 'categories': 2}
   fit warns DataWarning: Preprocessor received column(s) ['segment'] with no observed (non-missing) value at fit. Their blocks are kept but fitted on an entirely missing column (a single constant value once imputed), so they carry no information until the preprocessor is refit on data where those columns are observed.
one-hot, all-NaN column -> output_dims_: {'age': 7, 'city': 3, 'segment': 1}

Checks

• just test: 1509 passed, 59 skipped, 7 xfailed; coverage 93.95% (gate 90%).
• Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1496 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
• just docs: strict build succeeds.
• just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
• Import direction between subpackages is unchanged.

Merge notes

• Base: main.
• The PRs for Preprocessor.fit crashes on any bool column under default settings #54 (fix(preprocessor): encode boolean columns as binary categoricals #99), Preprocessor routes integer column labels positionally, silently swapping columns #55 (fix(preprocessor): route columns to the ColumnTransformer by label #100), output_format="auto" chooses sparse vs dense per transform batch, breaking dense-only estimators at predict #73 (fix(preprocessor): resolve output_format='auto' once per fit #113) and Preprocessor with one-hot encoding or preset="expanded" fails under set_config(transform_output="pandas") #76 (fix(preprocessor): keep the internal ColumnTransformer on default output #114) change the same self.column_transformer_.fit(X, y) line in Preprocessor.fit that this PR wraps in try/except, so each conflicts there. Resolve it to X_ct = self._column_transformer_input(X) (Preprocessor.fit crashes on any bool column under default settings #54, Preprocessor routes integer column labels positionally, silently swapping columns #55), Preprocessor with one-hot encoding or preset="expanded" fails under set_config(transform_output="pandas") #76's set_output(transform="default") line, and output_format="auto" chooses sparse vs dense per transform batch, breaking dense-only estimators at predict #73's training_output = self.column_transformer_.fit_transform(X_ct, y) inside this PR's try, with self._raise_for_empty_features(X_ct, y, empty_features, config) in the except and resolve_output_format after it. Preprocessor.fit crashes on any bool column under default settings #54 also conflicts in the Preprocessor class docstring, where both add a paragraph after the alias list: keep both. Preprocessor routes integer column labels positionally, silently swapping columns #55 conflicts in the Raises section of fit: keep both PretabDataError conditions.
• Follow-up once Preprocessor routes integer column labels positionally, silently swapping columns #55 lands: the ColumnTransformer selectors become the string form of each label. _raise_for_empty_features must then look up the empty feature by that string form (empty = {str(feature): feature for feature in empty_features}, feature = empty.get(columns[0])). Otherwise an empty column with a non-string label is not matched, and the method's own error surfaces instead of the one naming the column.
• The PR for add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62 (fix(preprocessor): keep the missing indicator out of the representation #106) changes the same imputer construction in factory.py and the feature-info code in inspection.py; git merges the two cleanly. Follow-up once add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62 lands: _assert_metadata_matches_output in tests/integration/test_edge_case_contract.py currently skips the get_feature_info dimension check for blocks with the imputer's missing indicator. On main that indicator runs through the representation, and get_feature_info miscounts it for "int" and "none". On a trial merge with add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62's branch, the dimensions agree for every indicator setting these tests cover, so the skip should then be removed.
• The PR for Standalone transformers drop DataFrame feature names, so reordered columns are silently misaligned #85 (fix(transformers): record and check DataFrame feature names #135) conflicts in tests/compose/test_inspection.py, where both add a test after test_build_feature_info_does_not_infer_kind_from_cat_in_feature_name: keep both. The PR for A 2-D y (multi-output target) crashes the B/M/I splines even when unsupervised, and target-aware methods with unrelated errors #88 (fix(splines): ignore y when unsupervised and reject a multi-output target #123) conflicts in tests/integration/test_edge_case_contract.py, in the import lines (keep CrossFittedTransformer and IncompatibleParamsError) and in the section both append at the end (keep both).
• On main, CrossFittedTransformer converts the frame with np.asarray, so test_column_empty_in_a_cross_fitting_fold_keeps_the_width fits every column as categorical. After the PR for CrossFittedTransformer converts DataFrames to ndarray; a wrapped Preprocessor treats every column as categorical #58 (fix(cross-fitting): pass DataFrames to the wrapped transformer unchanged #103) it runs on the DataFrame. The test needs no change.

This change was previously part of #97, which is closed in favour of one pull request per issue.