Skip to content

Add Factory Droid capture through hooks and bounded backfill - #2793

Merged
HypForge merged 2 commits into
masterfrom
codex/droid-transcript-capture
Oct 21, 2026
Merged

HypForge merged 2 commits into
masterfrom
codex/droid-transcript-capture

Conversation

@philcunliffe

Copy link
Copy Markdown
Contributor

Factory Droid conversations can now be collected into ai_gateway_messages from local version-2 transcripts. Native hooks notify a loopback listener; scheduled backfill recovers missed notifications using the same projector and durable message identities.

Changes

  • Add bundled @hypaware/droid, setup selection, client attachment, and neutral skills.
  • Preserve user hooks during attach/detach and honor FACTORY_HOME_OVERRIDE, .hypignore, local-only policy, and persistent session exclusions.
  • Capture text and linked tool calls/results without reading credential sidecars or treating cumulative token totals as billed usage.
  • Bound transcript discovery, file/line sizes, scan work, batching, and live cursor state. Recovery defaults to every five minutes and honors the existing provider/retention window.
  • Add deterministic adapter tests, a full capture smoke including scheduled recovery, and LLP 0551 with native-format evidence.

Validation

  • Authenticated Factory Droid 0.245.0 on macOS: 10 parts captured live; another 10 recovered after stopping capture; repeated backfill added zero duplicates and an excluding window imported zero rows.
  • Packed build under real macOS launchd and Debian Linux systemd: hook capture, normal 60-second scheduled recovery, one-day provider window against 30-day cache retention, conversation/trace/log SQL queries, restart, deduplication, detach, and uninstall all passed. Linux ran in a disposable container with a real user service manager.
  • All 21 documented release smokes and npm run smoke -- droid_capture passed. Typecheck, declaration build, package contents, npx startup, LLP number check, and final focused regressions passed.
  • CPU/memory review found no unbounded queue or payload cache: discovery is capped at 10,000 entries, files at 64 MiB, live reads at 8 MiB, batches at 64 records/1 MiB, cursors at 64, and recovery scans at 256 MiB.

Known baseline failure

npm test: 8,867 passed, seven skipped, one failed. The Hermes legacy-session catch-up test at test/plugins/hermes-source.test.js:660 timed out with 64 of 133 sessions. The identical failure reproduced on unchanged baseline ad8a40958df74517a7241e245390abc0afe19a54; this PR does not claim a green full-suite gate.

Scope is local Factory CLI transcripts. Hosted sessions without local transcript files and Devin collection remain outside this change.

@philcunliffe philcunliffe added the neutral:adopt Foreign PR adopted into neutral's reconcile scope label Oct 21, 2026 — with ChatGPT Codex Connector
@HypForge HypForge added the neutral:adopted Adoption completion record: merged while carrying neutral:adopt (LLP 0031) label Oct 21, 2026
@HypForge

Copy link
Copy Markdown
Contributor

R1 at 864184054247afd09eec84425e9e409a39d209db: FINDINGS / HIGH pending privacy fix F1 (new adapter/shared detach is MEDIUM by nature).

F1 P1: both live capture and recovery treat an unreadable cwd/.hypignore as absent through default existsSync. Real non-root filesystem probe: readable ignore folder produces zero recovery projections; chmod000 makes recovery emit one and live capture return HTTP200/rowsWritten1. Fail closed on unprobeable policy paths in both lanes and add regressions before landing.

Reviewer checks: 81 focused tests pass, typecheck and text/diff checks pass. Independent discovery, payload/batch, 65-session cursor, path and root/nested hook ownership controls pass. The committed Droid test covers root-level detach preserving user/fallback handlers; existing nested detach tests stay green. Projected rows retain cwd, do not bill cumulative totals, and use existing export/dedupe paths. LLP 0045 is forward-ref-only; 0551 is Draft.

CPU/memory: discovery, transcript payloads/batches and cursor counts are bounded as described in the report; 256 MiB is a pre-read reservation, not a whole-pipeline byte/heap ceiling. Per-notification exclusion refresh and per-batch cache/spool dedup still scale with stored data. No separate performance blocker found. Steward reports head/master full suites green with no Hermes reproduction; required smokes remain pending for the fix head. Real Factory and installed-service acceptance not repeated. Server-suite gate not triggered. Do not land this head.

🤖 Generated with Claude Code

Both capture lanes resolved usage policy with the default existsSync, so a
.hypignore inside an unreadable folder read as absent and the session was
captured under the implicit full default. Probe with a stat that counts any
error other than ENOENT/ENOTDIR as present, as Copilot and LangGraph do, so
the resolver's fail-closed read suppresses the session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HypForge

Copy link
Copy Markdown
Contributor

Final review through R2 at ef5d9cc4382d7568b95e35e7038ae0c887569d46: CLEAN / MEDIUM. The remaining risk is the new capture adapter and shared detach path; no open review findings remain.

F1 P1 fixed: live capture and recovery now fail closed when the session cwd cannot be probed for .hypignore. Independent non-root filesystem reproduction writes no rows in either lane (live HTTP 202/skipped). The new regression covers readable ignore, inaccessible cwd and restored access; removing only the fix makes it fail with live 5/recovered 1.

Checks: 82 focused tests pass, typecheck passes, independent privacy and helper controls pass, and the restored review checkout is clean. R1 discovery/read/batch/cursor/path and hook ownership evidence carries. The committed root-event detach regression and existing nested/other-client tests remain green. No accepted LLP decision was rewritten.

Steward reports full suite at this head 8,873 pass / 0 fail / 3 skip and all four required smokes passing (cli_bundled_plugins_activated, client_attach_idempotent, walkthrough_picker_to_first_query, droid_capture).

CPU/memory: the fix substitutes one stat for each existing existence probe with transient allocation, preserving resolver memoization and adding no retained state. No new performance blocker. R1 bounds and qualifications remain: recovery's 256 MiB is a pre-read reservation; exclusion refresh and cache/spool dedup costs scale with stored data. Real Factory/installed-service acceptance was not repeated. Server-suite gate not triggered.

🤖 Generated with Claude Code

@HypForge
HypForge added this pull request to the merge queue Oct 21, 2026
Merged via the queue into master with commit 1cbada7 Oct 21, 2026
9 checks passed
@HypForge
HypForge deleted the codex/droid-transcript-capture branch October 21, 2026 04:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:adopt Foreign PR adopted into neutral's reconcile scope neutral:adopted Adoption completion record: merged while carrying neutral:adopt (LLP 0031)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants