[hanhainebula]

Summary

Revise three existing retrieval tasks so their quality baselines, resource budgets, timing, and scores match the documented optimization objectives. This changes task settings and graders together. Fixed input assets, splits, and asset manifests are unchanged.

• tasks/sparse-retrieval-pruning — v0.5.2, CPU: Replace the old 4× full-score threshold with normalized reward clip((speedup - 4) / 12, 0, 1). Measure speedup using the ratio of medians from five complete search calls, alternating starter/candidate order, and require NDCG@10 ≥ 0.89 and Recall@100 ≥ 0.99 in every round. Isolate each invocation with fresh writable directories and frozen inputs/indexes. Fix interpreter/dependency setup, process-exit timing, resource accounting, and persistent diagnostic export. Budgets remain one CPU, 32 GiB RAM, and no swap or GPU.
• tasks/query-encoder-alignment — v0.2.0, one GPU: Replace the unsupported quality anchor with a reproducible Qwen3-Embedding-0.6B plus centered-ridge baseline, fitted without relevance labels and recomputed from locked inputs/code. Exclude both identical IDs and normalized identical text from retrieval results. Score clip((A - A0) / (1 - A0), 0, 1) when candidate runtime is at most 60% of the 4B reference runtime; the verified hidden baseline is A0 = 0.62. Repair external timing, isolated determinism checks, offline execution, and credential isolation. Both phases use an explicit GPU device mapping; the shared image test now validates either a single device_ids entry or count: 1, rejecting their combination.
• tasks/memory-constrained-dense-retrieval — v0.7.0, CPU: Set 512 MiB total memory with no swap, a kernel-enforced 64 MiB persisted-index workspace, 16 CPUs for building, and one logical CPU for queries and descendants. Restart the submission after building and include cold-start cost. Allow 150 seconds to build and 0.5 seconds per query. Require Accuracy@3 ≥ 0.95 and score normalized reward A * min(1, 0.030 / P95). Fix process-exit timing, completed-output snapshots, two-phase file limits, cleanup, and reward finalization.

This is the second PR, following the separate Docker lifecycle-timeout fix (1bd88b5). It contains these three task commits:

• cc49163 — Calibrate sparse retrieval speedup scoring and isolate repeated runs.
• 29b4686 — Calibrate query encoder quality and repair verifier execution.
• de82d40 — Revise dense retrieval resource limits and latency grading.

Existing task.toml author records (Search-SWE) and prior history are preserved. The only shared-file change in these commits is the GPU reservation regression test required by query-encoder-alignment. The two lifecycle-timeout files belong to the preceding PR.

Validation evidence

Checks rerun for this PR preparation: An isolated worktree contained exactly the 68 committed files over 21ab7a32ed75d4d463b6bc91deb0f304bcb3e3b5, excluding unrelated local changes and unfinished task directories.

python scripts/check_release.py
python scripts/check_tasks.py --merge-ready --base 21ab7a32ed75d4d463b6bc91deb0f304bcb3e3b5
python -m unittest discover -s scripts/tests -p 'test_*.py'
python .agents/skills/create-searchswe-task/scripts/test_scaffold_task.py
python -m unittest discover -s tasks/sparse-retrieval-pruning/tests -p 'test_*.py'
python -m unittest discover -s tasks/query-encoder-alignment/tests -p 'test_*.py'
python -m unittest discover -s tasks/memory-constrained-dense-retrieval/tests -p 'test_*.py'
git diff --check 21ab7a32ed75d4d463b6bc91deb0f304bcb3e3b5..HEAD

Release: zero errors/warnings; merge-ready: zero errors; shared tests: 174 passed; scaffolder: 5 passed; maintainer helper suite: 12 passed. Task regressions: sparse 9 passed, query 14 passed, dense 11 passed with 5 Docker-only cases skipped on the host. Asset --verify-only checks passed for all three packages (5, 30, and 8 files respectively). Committed content hashes match the reviewed scope and frozen task handoffs.

Existing runtime evidence retained and reviewed: Both task images were built during task revision. Dense's final Docker regression suite passed all 16 cases; negative cases include a 65 MiB index rejected with ENOSPC and a 700 MiB allocation killed for OOM. Its frozen reference scored 100 on public queries and 95 on hidden queries. Sparse has a quality-passing frozen reference at 7.499× speedup (29.1591/100). Query's two independent baseline fits and public predictions were byte-identical; its baseline, isolation, and grading boundary checks are documented in the task README.

All three formal trials used Codex + OpenRouter openai/gpt-6-luna, reasoning effort max. Their saved agent trajectories and verifier outputs were inspected; all completed without trial exceptions and received integrity-judge value 1.

Task	Score / 100	Observed result
sparse-retrieval-pruning	0	NDCG@10 0.882981 and Recall@100 0.985 fail the quality gates; measured speedup 6.6595×. This is a valid quality failure.
query-encoder-alignment	18.4211	Accuracy@1 69% versus baseline 62%; candidate 12.1705 s versus 4B reference 25.2756 s, passing the runtime gate.
memory-constrained-dense-retrieval	20.4719	Accuracy@3 19/20; P95 139.215 ms; build 78.321 s; index 63.46 MiB; no OOM kill.

Archived trial directories under local jobs/ (outputs are intentionally excluded from Git):

• sparse-retrieval-pruning-codex-openrouter-luna-max-v051-20261022-r2/2026-10-22__19-57-28/sparse-retrieval-pruning__qMd8Mow
• query-encoder-alignment-codex-openrouter-luna-max-v020-20261022-r2/2026-10-22__20-15-17/query-encoder-alignment__bBNpj6k
• memory-constrained-dense-retrieval-codex-openrouter-luna-max-v070-20261023/2026-10-23__04-01-17/memory-constrained-dense-retriev__yqhYBm5

Each contains agent/trajectory.json, result.json, and verifier evidence. Sparse's agent ran v0.5.1 and its verifier ran v0.5.2 after removing an invalid UID-wide process limit; the task inputs and scoring were unchanged. The subsequent diagnostic-export-only change was checked by replaying the same submission with the final package, with 47 exported-file hashes verified; that replay was not a new model/judge trial. Query's evaluated v0.2.0 code remained frozen, with only README updates afterward. Dense's v0.7.0 source remained unchanged across its formal run.

Validation limits: Docker builds and paid model trials were not repeated while preparing these commits because the evaluated task implementations were preserved. Historical trials used the shared harness recorded in their handoffs. The Docker lifecycle-timeout fix is submitted in the preceding PR; the checks above cover the isolated task-only snapshot, and the historical trials are not fresh end-to-end runs of the combined PRs. Full-score feasibility remains unproven for sparse's 16× target, query, and dense's hidden workload (reference: 95/100). Dense's 20-query gate is a fixed-workload acceptance criterion, and shared-CPU timing may vary. General storage declarations are not filesystem quotas; dense's 64 MiB index limit is enforced. Query does not claim a container-level no-swap guarantee. CI and integration against the latest upstream base still require the normal PR review checks.

Contributor checklist

• Existing task names and package locations retained; no new tasks introduced.
• Existing author metadata and original history preserved.
• Enable Allow edits from maintainers when opening the PR, or agree on contributor-applied review fixes.
• No secrets, downloaded inputs/models, job outputs, or author-only solutions included.
• Static checks and applicable runtime evidence recorded with version and execution limits.
• Existing immutable asset pins/manifests retained and local file hashes verified; no new asset publication or expanded license claims.

Maintainer merge checklist

• Review task settings, verifier isolation, runtime evidence, and validation exceptions.
• Review existing provenance/usage-term limitations and verify immutable asset sources.
• After the lifecycle-timeout PR is merged, run required checks against the updated upstream base and final PR head.
• Review judge health, resource enforcement, and failure cases; confirm task-local docs/configuration are complete.
• Merge with a merge commit to preserve contributor history.