[ChrisW09]

Fixes #63.

MSplineTransformer clipped its basis at absolute thresholds, so the basis depended on the units of the feature. On small-scale features, or with tight knots, the peaks were flattened at 1000 and the basis functions no longer integrated to one; on tiny-range features the whole output was zero. The same happened through Preprocessor(numerical_method="mspline") with quantile knots on a skewed feature.

Root cause

MSplineTransformer._mspline_basis scales each B-spline by (degree + 1) / knot_span, so its values scale like 1 / span in the feature's units. They were then compared against absolute constants: a basis function with knot_span <= 1e-6 was returned as zeros, values were clipped to 1e6, and _design_matrix clipped the whole design to ±1e3. Once a support is narrower than (degree + 1) / 1000, its peak exceeds 1000 and is cut off; once every span is below 1e-6, every column is zero.

Fix

• MSplineTransformer._mspline_basis no longer clips the values. A basis function is treated as degenerate, and returned as zeros, only when its span is at most np.finfo(float).eps times the knot range, which happens only for a repeated knot.
• MSplineTransformer._design_matrix no longer clips the design to ±1e3 or replaces ±inf. It keeps np.nan_to_num(design, nan=0.0), which clears the NaN that BSpline(extrapolate=False) returns outside the knot span.

Each basis function now integrates to one at any feature scale, and rescaling x by c rescales the basis by 1 / c.

Behaviour changes

• M-spline values are no longer capped at 1000. In the reproduction below, the peak at scale 0.01 is 2001 instead of 1000, and the 234 Preprocessor values that were clipped at exactly 1000 keep their actual value.
• A basis function with a positive span of at most 1e-6 (in the feature's units) is evaluated instead of returned as zeros, so tiny-range features get a non-zero basis.
• Where no value exceeded 1000 and every span was above 1e-6, the arithmetic is unchanged and so is the output.

Documentation

• Docstring of MSplineTransformer._mspline_basis: the formula, that the values scale like 1 / span and are left unclipped, and when a basis function is returned as zeros.
• No other changes were needed. The module and class docstrings and docs/representations/spline_expansions.md already state that each M-spline integrates to one, which now holds at any scale, and no page mentions the removed thresholds.

Tests

• test_mspline_basis_integrates_to_one_at_any_feature_scale[0.01|0.001|1e-07]: on uniform data scaled by 0.01, 0.001 and 1e-7, every column integrates to 1 over the fitted range (atol 1e-3). Fails on main (smallest integrals 0.809, 0.145 and 0.0), passes here.
• test_mspline_basis_integrates_to_one_at_any_feature_scale[1.0]: the unit-scale control. Guard test: passes on both by design.
• test_mspline_basis_is_scale_equivariant: on lognormal(0, 2) data, fitting and transforming X * 1e-4 gives the unit-scale basis times 1e4 (rtol 1e-8). Fails on main, passes here.
• test_mspline_basis_is_not_clipped_after_the_preprocessor_scaling: the issue's Preprocessor case (quantile knots on a lognormal(0, 2) feature, default min-max scaling to [-1, 1]). The basis exceeds 1000 and every column integrates to 1 over the scaled range. Fails on main, passes here.

Reproduction

Output of the script from #63 on this branch:

scale=1: integrals=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]  max=20.01
scale=0.01: integrals=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]  max=2001
scale=0.001: integrals=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]  max=2.001e+04
scale=1e-07: integrals=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]  max=2.001e+08
Preprocessor: values clipped at exactly 1000: 0

Checks

• just test: 1444 passed, 59 skipped, 7 xfailed; coverage 93.64% (gate 90%).
• Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1431 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
• just docs: strict build succeeds.
• just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
• Import direction between subpackages is unchanged.

Merge notes

• Base: main.
• The PR for A partial spline policy mapping drops the "clip" default, and "extrapolate"/"warn" give all-zero basis rows #84 (fix(splines): keep family policy defaults and extrapolate out-of-range rows #121) changes the BSpline(...) call in the same MSplineTransformer._mspline_basis to extrapolate=True. The hunks do not overlap, and git merges them without conflict. With both, an out-of-range value under an "extrapolate" or "warn" policy is scaled like an in-range one and is not capped at 1000 either.
• The PR for B/M/I spline and two-feature thin-plate spline transformers encode a missing value as an all-zero row #87 (fix(splines): expand a missing value to a NaN row in the B/M/I and thin-plate bases #122) relies on the np.nan_to_num(design, nan=0.0) that this PR keeps in _design_matrix. It also updates test_mspline_handles_nan, directly above the tests added here. The PRs for ISplineTransformer: 200-point trapezoid integration produces wrong and all-zero basis columns #53 (fix(splines): evaluate the I-spline basis in closed form #98), Target-aware splines trim selector knots by position, dropping the dominant split, and cap adaptive width at 15 #69 (fix(splines): search the resolved knot window in target-aware placement #109), B/M/I spline knot_locations are sized by output_dim and clipped to the data range instead of used as given #81 (fix(splines): honour explicit knot_locations regardless of output_dim #118) and A 2-D y (multi-output target) crashes the B/M/I splines even when unsupervised, and target-aware methods with unrelated errors #88 (fix(splines): ignore y when unsupervised and reject a multi-output target #123) add tests elsewhere in tests/expansion/spline/test_spline_expansions.py. Each of these branches merges with this one without conflict. With the branches for A partial spline policy mapping drops the "clip" default, and "extrapolate"/"warn" give all-zero basis rows #84, B/M/I spline and two-feature thin-plate spline transformers encode a missing value as an all-zero row #87 and ISplineTransformer: 200-point trapezoid integration produces wrong and all-zero basis columns #53 applied on top of this one, the full test suite passes (1499 passed, 59 skipped, 7 xfailed).
• ISplineTransformer: 200-point trapezoid integration produces wrong and all-zero basis columns #53 fixes the sibling defect in ISplineTransformer, which computes its own M-spline values and does not use MSplineTransformer. The two fixes are independent.

This change was previously part of #75, which is closed in favour of one pull request per issue.