Skip to content

feat(harness): add decoupled harness spec, concept_tracker receipts, AgentGraph loops, dynamic STT config, and verified benchmarks - #2681

Merged
groupthinking merged 4 commits into
mainfrom
feat/yc-harness-concept-tracker-and-stt-config
Nov 16, 2026
Merged

groupthinking merged 4 commits into
mainfrom
feat/yc-harness-concept-tracker-and-stt-config

Conversation

@groupthinking

@groupthinking groupthinking commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Canonical issue

Closes #2680

Outcome

Delivers five focused, verified capabilities on top of main:

  1. Decoupled Brain & Ephemeral Sandbox Architecture Spec (docs/architecture/HARNESS_DECOUPLING_SPEC.md) formalizing persistent orchestration planes, L1/L2/L3 context caching, and Grind Guard goal budgets.
  2. Neon concept_tracker Schema & Durable Receipts (data/concept_tracker/migrations/001_sync_concept_tracker.sql and data/concept_tracker/receipts/*.json) synchronized with live Neon project round-night-95394081 (neondb.concept_tracker), including recorded contradictions and prevention rules.
  3. AgentGraph & Handoff Envelopes (src/youtube_extension/services/agents/agent_graph.py, agent_handoff.py, adapters/agent_orchestrator.py) supporting pipelines, conditional routing, fork-join, self-correction loops, error short-circuiting, and max_steps cycle guards.
  4. Dynamic SpeechToTextConfig Environment Binding (src/youtube_extension/services/ai/speech_to_text_service.py) using field(default_factory=...) so per-instance environment overrides take effect without module reloads.
  5. Verified Benchmark Fixtures (apps/web/src/lib/verified-benchmarks.ts) with exact SHA-256 content-integrity digests, timed caption coverage, and linked SOP compilation.

Scope

  • Included:
    • docs/architecture/HARNESS_DECOUPLING_SPEC.md
    • data/concept_tracker/migrations/001_sync_concept_tracker.sql
    • data/concept_tracker/receipts/2026-09-08-yc-harness-deep-dive.json
    • data/concept_tracker/receipts/2026-11-16-verified-benchmarks-and-sync.json
    • src/youtube_extension/services/agents/agent_graph.py
    • src/youtube_extension/services/agents/agent_handoff.py
    • src/youtube_extension/services/agents/adapters/agent_orchestrator.py
    • src/youtube_extension/services/ai/speech_to_text_service.py
    • tests/unit/test_agent_graph_loops.py
    • tests/unit/test_speech_to_text_service.py
    • apps/web/src/lib/verified-benchmarks.ts
    • apps/web/src/lib/__tests__/verified-benchmarks.test.ts
  • Explicitly excluded:
    • Unrelated UI skin changes, .agents/skills directory moves, and co-mingled prototype folders from legacy branches.

Risk

  • Risk level: low
  • Failure mode: Additive modules (agent_graph.py, agent_handoff.py, verified-benchmarks.ts, concept_tracker SQL/receipts) and default-preserving dataclass default_factory evaluation in SpeechToTextConfig.
  • Rollback: git revert 6b25fefff3c6c7f33bda40dc2d2e2410846ffdb0..f59b519947384d5e8ab0f2d3734811335dc92ef5 cleanly reverts all changes without side effects on existing routes or schemas.

Verification

Verified on exact head f59b519947384d5e8ab0f2d3734811335dc92ef5:

  • Focused tests:
    • pytest -o addopts="" tests/unit/test_speech_to_text_service.py tests/unit/test_agent_graph_loops.py (96 passed)
    • vitest run src/lib/__tests__/verified-benchmarks.test.ts src/lib/__tests__/linked-sop.test.ts (11 passed, including SHA-256 content-integrity check)
    • python -m compileall -q src/ (0 syntax errors)
    • npx eslint src/lib/verified-benchmarks.ts src/lib/__tests__/verified-benchmarks.test.ts (0 errors, 0 warnings)
  • Required CI: Triggered on push for head f59b519947384d5e8ab0f2d3734811335dc92ef5 (prior head 6b25fefff passed 29/29 checks including 16/16 live E2E)
  • Review threads resolved: All 3 vercel[bot] review findings resolved in f59b51994

Production evidence

Live Neon database (round-night-95394081, database neondb, schema concept_tracker) migration applied and verified via psql (7 canonical concepts, 1 separated term collision for MHS, 1 verified relationship, 2 recorded contradictions, and 2 durable execution receipts in concept_tracker.execution_receipts).

Agent handoff

  • One canonical issue is linked
  • No competing PR implements the same issue
  • Acceptance criteria are satisfied
  • Required checks pass on the current head
  • Human decision is requested only for product, security, irreversible infrastructure, or production approval

…AgentGraph loops, dynamic STT config, and verified benchmarks
@vercel

vercel Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
v0-uvai Ready Ready Preview, v0 Nov 16, 2026 11:13pm UTC

@github-actions github-actions Bot added documentation Improvements or additions to documentation javascript Pull requests that update javascript code python tests labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

🔍 PR Validation

⚠️ Large PR detected (1520 lines changed)

@github-actions

Copy link
Copy Markdown
Contributor

✅ E2E Test Results: ALL TESTS PASSED

Metric Value
Status 🟢 GREEN
Total Tests
Passed 16
Failed
Deployment https://v0-uvai-lp9493q87-garv1.vercel.app
Test Output
s on every request without persisting it�[32m 1�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mhomepage returns 200 with HTML�[32m 114�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page contains template/workflow markup�[32m 167�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page surfaces at least 5 workflow/template indicators�[32m 124�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mPOST /api/video/pack accepts a YouTube URL without hanging�[33m 1912�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mGET /api/video/pack resolves to a ready pack with evidence lineage�[33m 537�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mmissing URL returns 400, not a hang�[32m 63�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22minvalid URL returns 400 quickly, not a hang�[32m 60�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mPOST /api/video/pack with no body returns 400�[32m 58�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard returns 200�[32m 121�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard contains agent or pipeline visualization markup�[32m 132�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mGET /api returns a response (not 404)�[32m 178�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mretired /api/pipeline routes are gone (no pipeline served)�[32m 41�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22mhomepage has proper meta tags�[32m 76�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22m/features page returns 200�[32m 91�[2mms�[22m�[39m

�[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
�[2m      Tests �[22m �[1m�[32m16 passed�[39m�[22m�[90m (16)�[39m
�[2m   Start at �[22m 11:33:44
�[2m   Duration �[22m 5.00s�[2m (transform 32ms, setup 0ms, import 43ms, tests 4.86s, environment 0ms)�[22m


Comment thread apps/web/src/lib/verified-benchmarks.ts Resolved
@github-actions

Copy link
Copy Markdown
Contributor

✅ E2E Test Results: ALL TESTS PASSED

Metric Value
Status 🟢 GREEN
Total Tests
Passed 16
Failed
Deployment https://v0-uvai-3pz9nbkc8-garv1.vercel.app
Test Output
every request without persisting it�[32m 1�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mhomepage returns 200 with HTML�[32m 209�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page contains template/workflow markup�[33m 311�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page surfaces at least 5 workflow/template indicators�[32m 166�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mPOST /api/video/pack accepts a YouTube URL without hanging�[33m 2150�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mGET /api/video/pack resolves to a ready pack with evidence lineage�[33m 635�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mmissing URL returns 400, not a hang�[32m 123�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22minvalid URL returns 400 quickly, not a hang�[32m 105�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mPOST /api/video/pack with no body returns 400�[32m 104�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard returns 200�[32m 266�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard contains agent or pipeline visualization markup�[32m 137�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mGET /api returns a response (not 404)�[32m 264�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mretired /api/pipeline routes are gone (no pipeline served)�[32m 71�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22mhomepage has proper meta tags�[32m 117�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22m/features page returns 200�[32m 187�[2mms�[22m�[39m

�[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
�[2m      Tests �[22m �[1m�[32m16 passed�[39m�[22m�[90m (16)�[39m
�[2m   Start at �[22m 12:37:24
�[2m   Duration �[22m 6.51s�[2m (transform 37ms, setup 0ms, import 47ms, tests 6.36s, environment 0ms)�[22m


Copilot AI max review requested due to automatic review settings November 16, 2026 23:12
@groupthinking

Copy link
Copy Markdown
Owner Author

Fixed in 67a89623e: propagated non-ok AgentResult.status in both execute_fork_join and execute_self_correction_loop, added max_iterations < 1 validation, preserved default UUID trace_id, and added regression tests in tests/unit/test_agent_graph_loops.py (100/100 pytest passed, ruff clean).

@github-actions

Copy link
Copy Markdown
Contributor

✅ E2E Test Results: ALL TESTS PASSED

Metric Value
Status 🟢 GREEN
Total Tests
Passed 16
Failed
Deployment https://v0-uvai-5nlny8nmj-garv1.vercel.app
Test Output
on every request without persisting it�[32m 1�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mhomepage returns 200 with HTML�[32m 140�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page contains template/workflow markup�[32m 140�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mTemplate Gallery�[2m > �[22mfeatures page surfaces at least 5 workflow/template indicators�[32m 151�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mPOST /api/video/pack accepts a YouTube URL without hanging�[33m 2125�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mVideo Pack Pipeline�[2m > �[22mGET /api/video/pack resolves to a ready pack with evidence lineage�[33m 610�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mmissing URL returns 400, not a hang�[32m 82�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22minvalid URL returns 400 quickly, not a hang�[32m 75�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mError Handling�[2m > �[22mPOST /api/video/pack with no body returns 400�[32m 65�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard returns 200�[32m 132�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mDashboard�[2m > �[22m/dashboard contains agent or pipeline visualization markup�[32m 102�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mGET /api returns a response (not 404)�[32m 194�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mAPI Health�[2m > �[22mretired /api/pipeline routes are gone (no pipeline served)�[32m 44�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22mhomepage has proper meta tags�[32m 112�[2mms�[22m�[39m
 �[32m✓�[39m tests/e2e/pipeline.test.ts�[2m > �[22mEventRelay E2E — Live Deployment�[2m > �[22mStatic Assets�[2m > �[22m/features page returns 200�[32m 124�[2mms�[22m�[39m

�[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
�[2m      Tests �[22m �[1m�[32m16 passed�[39m�[22m�[90m (16)�[39m
�[2m   Start at �[22m 23:15:06
�[2m   Duration �[22m 5.62s�[2m (transform 41ms, setup 0ms, import 56ms, tests 5.43s, environment 0ms)�[22m


@groupthinking
groupthinking merged commit d98a37d into main Nov 16, 2026
30 of 31 checks passed
@groupthinking
groupthinking deleted the feat/yc-harness-concept-tracker-and-stt-config branch November 16, 2026 23:18

This branch was successfully deployed

1 active deployment
Preview — 67a89623 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation javascript Pull requests that update javascript code python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(harness): integrate decoupled harness spec, Neon concept_tracker schema, AgentGraph loops, dynamic STT config, and verified benchmarks

1 participant