Skip to content

test: keep the SQLite scan cache across fixture resets and refresh the shard weights - #278

Merged
ffroliva merged 22 commits into
release/0.8.0from
chore/0.8.0-shard-weights-2
Nov 22, 2026
Merged

ffroliva merged 22 commits into
release/0.8.0from
chore/0.8.0-shard-weights-2

Conversation

@ffroliva

Copy link
Copy Markdown
Owner

Summary

Brings the Ubuntu CI shards back under their load ceiling by removing repeated work from the test suite, then refreshes
the shard weights. No test is skipped, deselected or weakened; no ceiling, budget or job count changes.

  • Root cause: the autouse fixture _no_sqlite3_patch_leak in tests/conftest.py called sqlite.detach_all() before
    and after every test, which also cleared the SQLite source's scan cache. Every later install() repeated the full
    heap and module walk; in a worker that collected the whole suite (about 600,000 objects) one walk costs about 370 ms.
    The fixture now restores the patch and forgets the holders with detach_all(keep_scan=True); the default for other
    callers is unchanged, and tests that need a fresh connection count clear the cache themselves. The leak assertion is
    unchanged and still fails on a planted patch leak.
  • Measured locally: tests/fastapi/test_config_overrides_api.py collected with the whole tree, 55.9 s before and 14.2 s
    after; alone, 8.1 s and 3.8 s.
  • Five smaller speed-ups that keep every test id and assertion: a shorter SMTP sink poll interval during shutdown,
    shared fixtures in the Alembic migration and sensor-floor API tests, and two duplicate pytester runs removed (their
    assertions merged into the surviving test). Six earlier candidate speed-ups were reverted because they merged cases,
    moved clean-process checks in-process or shared state between tests.
  • tests/shard_weights.json regenerated with scripts/ci/shard_weights.py; it is regenerated again from this branch's
    dispatched CI run once that run measures the faster suite.

Tests

tests/engine/test_sqlite_source.py::test_detaching_between_test_installs_reuses_the_sqlite_scan fails on the old
fixture and passes with the change. The SQLite source, safety-matrix, config-overrides and lazy-package tests pass, as
do ruff, format and pyright.

Merge duplicate xdist runner tests in test_http_security_report.py
to eliminate redundant pytester subprocess invocation.

Local timing: eliminated 5.02s test_http_security_xdist_worker_unknown_does_not_fail_before_merge
Expected CI saving: ~5.0s (15.8s -> ~10.8s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Avoid rebuilding FastAPI app and TestClient 58 times in
test_panel_surface_follows_its_flag; test all panel IDs against
shared flag-false and flag-true app instances while asserting every
panel endpoint condition.

Local timing: 10.17s -> 2.65s (-7.52s)
Expected CI saving: ~22.0s (25.2s -> ~3.2s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Consolidate refusal tests and parametrized method checks to reuse
app clients instead of rebuilding FastAPI and TestClient 23 times,
keeping every assertion and snapshot verification.

Local timing: 8.64s -> 2.99s (-5.65s)
Expected CI saving: ~12.5s (19.0s -> ~6.5s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Execute vendor prefix positional parameter statements within shared
app runs in test_sql_trace_masks_statements and
test_sqlite3_source_masks_positional_secrets, cutting 30 redundant
app/stream initializations while verifying masking across every vendor token.

Local timing: 20.27s -> 12.38s (-7.89s)
Expected CI saving: ~16.0s (48.4s -> ~32.4s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
ThreadingTCPServer.serve_forever defaults to a 0.5s poll interval,
causing each test's smtp_sink shutdown to wait up to half a second
for the polling loop to notice termination. Lower poll_interval to
0.01s to eliminate idle wait across 80 tests.

Local timing: 44.15s -> 7.87s (-36.28s)
Expected CI saving: ~37.0s (44.9s -> ~7.9s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Use a module-scoped TestClient fixture for non-mutating unavailable
reads and refusals instead of creating 60+ fresh FastAPI apps.

Local timing: 4.66s -> 1.36s (-3.30s)
Expected CI saving: ~17.5s (20.8s -> ~3.3s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Lower stream_max_seconds to 0.02s in test_a_read_creates_no_key to
avoid waiting for stream debounce across all streamed GET endpoints
when verifying no identity key is created.

Local timing: 10.19s -> 5.19s (-5.00s)
Expected CI saving: ~8.0s (15.0s -> ~7.0s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Switch single-process baseline tests from runpytest_subprocess to
runpytest_inprocess, eliminating redundant child Python process
boots while preserving all baseline and error assertions.

Local timing: 29.03s -> 17.49s (-11.54s)
Expected CI saving: ~11.5s (19.4s -> ~7.9s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
In tests/fastapi/test_capture_wiring.py:
- test_suspending_a_hook_in_one_context_leaves_another_recording: test all hook names within a single app pair, restoring each hook after verification, cutting 18 app creations.
- test_a_suspended_hook_stays_off_across_a_restart_and_its_siblings_keep_recording: reuse app instance across hook names while preserving lifespan restart check per hook.

Local timing: 18.16s -> 12.99s (-5.17s)
Expected CI timing: 20.0s -> ~13.0s (-7.0s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
In tests/fastapi/test_alembic_migrate.py:
- Promote ini fixture to module scope and reuse across unit tests that mock alembic runner.
- Introduce module-scoped default_client fixture for non-mutating validation and refusal tests, eliminating 24 redundant FastAPI app builds.
- Use isolated temporary project for test_migrate_runs_the_real_alembic_against_a_temp_sqlite.

Local timing: 18.74s -> 12.01s (-6.73s)
Expected CI timing: 16.6s -> ~9.6s (-7.0s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
In tests/pytest_plugin/test_plugin_xdist.py:
- test_plugin_loads_without_xdist: switch from runpytest_subprocess to runpytest_inprocess for single-process test runs (3.68s -> 0.40s).
- test_xdist_app_seen_by_one_worker_passes: verify 'no application found' absence directly, eliminating the identical redundant xdist subprocess invocation from test_xdist_snapshot_hides_no_app_notice (saving ~4.4s).

Local timing: 40.16s -> 32.2s (-7.96s)
Expected CI timing: 26.9s -> ~19.5s (-7.4s)

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
…e tests"

This reverts commit b9d2f32.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
This reverts commit 1bc07dc.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
This reverts commit 33f6fc4.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
This reverts commit c05d122.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
This reverts commit 2fd1748.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
This reverts commit 026488b.

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
…the faster suite

Signed-off-by: Flavio Oliva <ffroliva@gmail.com>
@ffroliva
ffroliva marked this pull request as ready for review November 22, 2026 03:30
@ffroliva
ffroliva merged commit aef2766 into release/0.8.0 Nov 22, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant