fix(search): stop penalising production modules named test_*.py

AGENT_DIARY.md

@@ -57,6 +57,10 @@
   Пункты «RU→EN словарь» и «concept→symbol словарь» сняты: их заменяет контракт.
 - **Урок:** не учить сервер языкам, которые знает хост; отдавать `hints` и честно
   просить их, когда в вопросе нет code-имени.
+- **E67 (бакет тестов):** `src/core/test_impact.py` штрафовался на −2.0 как тестовый
+  файл, потому что начинается с `test_`. Исправлено проверкой содержимого (тесты или
+  импорт фреймворка), дефолт переключён; откат `MSCODEBASE_TEST_BUCKET=path`.
+  Замер: 3/3 пострадавших, якоря 21/41 без изменений, sample 40/40; точечно 0/9 → 9/9.
 - **E68 (бенчмарк из истории git):** вопрос = сообщение коммита, gold = `git show`,
   корпус на коммите `B` старше задач. Dev/71: tool 19, neighbours 37, search 24,
   grep 2 (hit@5); sealed/37: 7 / 21 / 5 / 1. P1 и P2 провалены: дефолт **хуже**

EXPERIMENTS_LOG.md

@@ -1,4 +1,23 @@
 # EXPERIMENTS_LOG.md — Audit Verification (2026-07-22)
+## [2026-11-10] — E67: a production module was penalised as a test file
+
+**Bug (found by the E66 generator):** `_bucket_penalty` classifies by path, and
+`test_*.py` matched `src/core/test_impact.py` — a production module that
+*predicts* test impact. Its definitions lost 2.0 rank on code questions, so three
+E66 tasks failed there while a plain grep found them. Blast radius: 926 `.py`
+files, 271 match the regex, 260 are real tests, **1 production module** is
+misclassified.
+
+**Fix:** outside `tests/`, a `test_*.py` is a test only if it defines tests or
+imports pytest/unittest; unreadable files keep the strict behaviour;
+`MSCODEBASE_TEST_BUCKET=path` is the rollback.
+
+**Measurement** (3 affected + 41 anchors + 40 sampled positives, 3 repeats):
+T1 3/3 fixed, T2 anchors 21/41 vs 21/41, T3 sample 40/40 vs 40/40, T4 identical
+on unrelated tasks. Targeted re-check: `path` 0/9, `content` 9/9 top-1.
+
+**Shipped:** default flipped to `content`.
+
 ## [2026-11-10] — E68: commit-localization benchmark — the default answers *less* than plain search
 
 **Why:** every earlier panel was authored by us, so "tuned to the test" was a

experiments/test_bucket_2026_11/REPORT.md

@@ -0,0 +1,50 @@
+# E67 — test-bucket classification
+
+Frozen: `frozen/PLAN.md`. Source: `experiments/scale_2026_11/REDTEAM.md` A2.
+
+## The bug
+
+`_bucket_penalty` treats any `test_*.py` as a test file and subtracts 2.0 from
+its rows on a code question. `src/core/test_impact.py` is a production module (it
+*predicts* test impact), so its definitions were pushed out of the answer: three
+E66 tasks failed there and a plain grep found them.
+
+Blast radius, measured: 926 tracked `.py` files, 271 match the path regex, 260 of
+them are real tests (they define `def test_*` or import a test framework), **1**
+production module is misclassified.
+
+## Fix
+
+The path rule stays; a content check overrides it outside `tests/`: a `test_*.py`
+counts as a test only if it defines tests or imports pytest/unittest. Unreadable
+files keep the old (penalise) behaviour, so nothing becomes more permissive by
+accident. `MSCODEBASE_TEST_BUCKET=path` restores the previous rule.
+
+## Measurement
+
+Arms: `path` vs `content`, one binary, interleaved. Panel: the 3 affected tasks +
+41 anchors + 40 sampled positives, 3 repeats, per-task majority.
+
+```
+T1  affected fixed (any@8)            3/3    PASS
+T2  anchors not worse                 21/41 vs 21/41   PASS
+T3  sampled positives not worse       40/40 vs 40/40   PASS
+T4  identical on unrelated tasks      PASS
+```
+
+Re-check of the load-bearing claim, 3 repeats per task per arm:
+
+```
+path     risk_level / module_of / referenced_modules   (valid, any, top1) x3
+         all three: (True, False, False) x3              -> 0/9
+content  all three: (True, True, True) x3                -> 9/9
+```
+
+One of 504 calls in the full A/B returned an invalid answer (a degraded run); the
+majority of 3 covered it, and the targeted re-check above has 18 valid calls.
+
+## Result
+
+The default is now `content`; `path` remains as a rollback via the env var. The
+change is a classification fix, not ranking: T3/T4 confirm it does not reorder
+anything that does not involve a `test_*.py` file.

experiments/test_bucket_2026_11/frozen/PLAN.md

@@ -0,0 +1,57 @@
+# Test-bucket classification — frozen plan (before the code change)
+
+Date: 2026-11-10. Source: `experiments/scale_2026_11/REDTEAM.md` A2.
+
+**Bug.** `_bucket_penalty` treats any `test_*.py` as a test file and subtracts
+2.0 from its rows on a code question. `src/core/test_impact.py` is a production
+module (it *predicts* test impact), so all its definitions are pushed out of the
+answer: three tasks of the E66 panel fail because of it, and grep finds them.
+
+**Change.** The path rule stays, and a content check overrides it: a `test_*.py`
+outside `tests/` counts as a test only if it defines `def test_*` or imports
+pytest/unittest. Unknown/unreadable files keep the old (penalise) behaviour, so
+nothing becomes *more* permissive by accident.
+
+Measured: 926 tracked `.py` files; 271 match the path regex; 260 carry test
+markers; **1** production module is misclassified.
+
+## Arms
+
+One binary, `MSCODEBASE_TEST_BUCKET`:
+
+- `path` — shipped behaviour (default; no change until the measurement says so);
+- `content` — the override described above.
+
+## Panel
+
+The change can only affect rows whose file matches `test_*.py`, and exactly one
+such file is outside `tests/`. So the panel is:
+
+- the 3 affected E66 tasks (`risk_level`, `module_of`, `referenced_modules`);
+- the 41 anchors (regression);
+- 40 random positives from the E66 panel (regression).
+
+84 tasks x 3 repeats, per-task majority, both arms interleaved.
+
+## Predictions (frozen)
+
+| # | Prediction | Falsifier |
+|---|---|---|
+| T1 | `content` fixes all 3 affected tasks (`any@8` true) | any of the 3 still misses |
+| T2 | no anchor loses `top1` or `any` | any loss |
+| T3 | the 40 sampled positives do not lose | any loss |
+| T4 | per-task rows are identical between arms on tasks whose files do not match `test_*.py` | any difference |
+
+T4 is the guard against a scalar change leaking into unrelated ranking.
+
+## Kill
+
+T2 or T3 fails -> keep `path` (the shipped behaviour) and record the bug as a
+KNOWN_ISSUE instead of a fix. T1 failing means the hypothesis is wrong about the
+cause, not about the penalty.
+
+## Note on what this is not
+
+This is a **classification fix**, not ranking tuning. The bar is "the same
+answers, plus the three that a production module was losing"; it is not allowed
+to reorder the corpus. If it does, T4 trips.