Repository navigation
fix(search): stop penalising production modules named test_*.py - #107
Merged
Merged
Conversation
`_bucket_penalty` classifies by path, and any `test_*.py` was treated as a test
file and lost 2.0 rank on code questions. `src/core/test_impact.py` is a
production module - it *predicts* test impact - so its definitions were pushed
out of the answer: three tasks of the E66 panel failed there while a plain grep
found them.
Blast radius, measured: 926 tracked .py files, 271 match the path regex, 260 of
them are real tests (they define `def test_*` or import a test framework), and 1
production module is misclassified.
Outside `tests/`, a `test_*.py` now counts as a test only if its content says so.
Unreadable files keep the strict behaviour, so nothing becomes more permissive by
accident. `MSCODEBASE_TEST_BUCKET=path` restores the old rule as a rollback.
Measured (3 affected + 41 anchors + 40 sampled positives, 3 repeats, majority):
T1 affected fixed (any@8) 3/3 PASS
T2 anchors not worse 21/41 vs 21/41
T3 sampled positives not worse 40/40 vs 40/40
T4 identical on unrelated tasks PASS
Targeted re-check of the load-bearing claim, 3 repeats per task per arm:
`path` 0/9, `content` 9/9 top-1 on risk_level, module_of, referenced_modules.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(search): stop penalising production modules named test_*.py
_bucket_penaltyclassifies by path, and anytest_*.pywas treated as a testfile and lost 2.0 rank on code questions.
src/core/test_impact.pyis aproduction module - it predicts test impact - so its definitions were pushed
out of the answer: three tasks of the E66 panel failed there while a plain grep
found them.
Blast radius, measured: 926 tracked .py files, 271 match the path regex, 260 of
them are real tests (they define
def test_*or import a test framework), and 1production module is misclassified.
Outside
tests/, atest_*.pynow counts as a test only if its content says so.Unreadable files keep the strict behaviour, so nothing becomes more permissive by
accident.
MSCODEBASE_TEST_BUCKET=pathrestores the old rule as a rollback.Measured (3 affected + 41 anchors + 40 sampled positives, 3 repeats, majority):
Targeted re-check of the load-bearing claim, 3 repeats per task per arm:
path0/9,content9/9 top-1 on risk_level, module_of, referenced_modules.