[Cooki-e]

Summary

Add two self-contained retrieval tasks with separate verifiers and pinned public inputs.

• lightweight-query-retrieval (author: Bingyu): build a fast batch retrieval service over 66,454 documents, using the original Llama-3.2-1B base and 80,000 labeled examples. Runs offline on one A100 40GB; scoring combines a macro nDCG@10 quality gate with retrieval time.
• paper-metadata-retrieval (author: Cooki-e): identify the paper satisfying a research need and its source conditions from 70,000 complete paper texts with bibliographic metadata. Runs on 16 CPUs with 3 public validation requests, 8 hidden requests, and a 30-second deadline per query.

Both packages include instructions, resource docs, manifests, container definitions, contract tests and validation records. No shared task inventory or infrastructure file changes are needed. Corpus/model bytes, author solutions and trained witness artifacts are excluded from Git; private grading inputs stay in the verifier package.

Package paths: tasks/lightweight-query-retrieval/ and tasks/paper-metadata-retrieval/.

Assets and provenance

Public data: search-swe/Search-SWE, lightweight inputs at ff16485accd6d258610c158fce45e36b50a47c3f, and paper inputs at 8ee1f03ffe70dd7eb57b0bcd8627730fc4232dd3 via HF publication. Original model pins are retained. Fresh downloads verified all five public data files; all 14 local runtime assets also passed size/SHA-256 checks. Source terms and attribution remain documented per task and in the HF provenance.

Validation evidence

Commands include python scripts/check_release.py, python scripts/check_tasks.py --merge-ready --base 11a170be17a648511233e69a8ce1a9ae756aa0d9, task-local unittest discovery, fresh download_assets.py --task <name> --kind data restoration, and python -m unittest scripts.tests.test_task_images scripts.tests.test_rewardkit_deepseek scripts.tests.test_network_policies -v.

Twelve contract tests per task passed. Shared image/network/RewardKit checks ran 20 tests: 18 passed, 2 skipped because the host lacks harbor-rewardkit 0.2.0. Final release and merge-ready static checks passed. Existing task-local validation records include CPU/container isolation and timeout cases; the legal GPU witness reaches the quality gate (0.454410), with median 0.860873 seconds and reward 0 under the frozen speed rule. CPU grader fixtures cover all-correct/mixed/wrong results, malformed outputs, credential boundaries and real timeout continuation; fixtures are not retrieval solutions.

Configured coding-agent and live trajectory-judge trials were not run: no additional paid run was requested. Model/effort, coding-agent trajectory, agent score and live judge health are therefore unavailable. The mechanics/witness checks do not replace this review layer. Final package versions are lightweight v0.3.0 and paper v0.1.1; instruction SHA-256 values are 2f774a970ec1e0ff840a48905cc540a2528bdcd4580b6f07682649cc845c0b98 and 6dbf8e9b1226419e87c86524368e6d459b0fc2661dc361d0daaca57f0b70e440 respectively. Publication changes only asset mappings/pins and author-facing records, preserving instructions, runtime, scoring and input bytes; no model-trial feedback or task-setting revision is claimed.

Structural precedents are documented in each task's VALIDATION.md: query-encoder-alignment and local methodology/decision-model packages for the GPU task, and reasoning-query-rewriting plus biomedical-graph-retrieval for the CPU task. The new tasks define their own inputs, splits, scoring, hardware and deadlines.

Contributor checklist

• New tasks use unique descriptive names of at most five lowercase hyphen-separated words and live at tasks/<task-name>.
• Actual authors are recorded in task.toml; existing history is preserved.
• Allow edits from maintainers; verified after PR creation.
• No secrets, downloaded inputs/models, job outputs, or author-only solutions enter Git.
• Static checks and applicable runtime evidence are recorded, with unrun layers explicit.
• Provenance, applicable terms, contributor publication authorization and immutable asset pins are documented and verified.

Maintainer merge checklist

• Design, verifier isolation, runtime evidence and validation exceptions reviewed.
• Names checked against the current base and queued PRs.
• Upstream/official asset downloads verified against immutable pins, sizes and SHA-256 hashes.
• Official publication and preservation of preceding assets reviewed; merged official SHA pinned.
• Task-local docs/configuration complete; no shared task inventory updates needed.
• Required judges, credential isolation and failure cases validated; checks and merge-ready pass.
• Merge commit; original contributor history retained.