Skip to content

fix(mcp): separate log cutoffs from backend failures - #304

Merged
roodboi merged 76 commits into
nextfrom
wf/y1-logs-b6
Oct 12, 2026
Merged

roodboi merged 76 commits into
nextfrom
wf/y1-logs-b6

Conversation

@roodboi

@roodboi roodboi commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

HACK-1134 (unit Y1).

Final behavior

hack.project.logs.tail and hack.project.logs.snapshot now end in exactly one outcome:

  • Complete: the backend ended the stream and exited 0. ok: true, data.cutoff: null.
  • Cutoff: the server stopped collection at one of its own bounds. This is a success: ok: true, no isError, and data.cutoff = { bound, limit }:
    • events: maxEvents reached. Only type: "log" events count. start, heartbeat, error and end events are still returned.
    • time: maxMs for tail, or the 120 s command timeout for snapshot.
    • bytes: the shared 8 MiB stdout+stderr budget. outputTruncated: true, and data now keeps the events parsed before the cut. The cut backs off to a UTF-8 code point boundary.
  • Backend failure: an error event, an end event whose reason is not eof/closed, or a non-zero exit at end of stream. ok: false, isError, and the closed error: { code: "E_UNEXPECTED", reason: "backend_error", message: <fixed> }. The message never repeats backend output, caller input or the query.

Failure evidence counts only before the server stops the child. A child that writes error/end(error) or exits non-zero in response to the server's own SIGTERM still produces a cutoff. The byte bound is the exception for events: every retained byte was written before the child was signalled, and nothing after the cut is parsed, so an error or failing end event inside the returned prefix is still a backend_error. For the events and time bounds, an error the backend wrote just before the stop but that the server read after it is reported as a cutoff; the server cannot tell it apart from a reply to the stop, and the doc says so. After a cutoff exitCode is the child's real status, which is often non-zero. data.stop_reason keeps its values.

Implementation:

  • src/mcp/logs-contract.ts: pure event classification and outcome mapping.
  • src/mcp/server.ts: both tools go through one collector (runHackLogCollect) and buildLogToolResult. The output schema now admits backend_error/E_UNEXPECTED.
  • src/mcp/output-budget.ts: opt-in codePointSafe mode, used only by the logs collector. The default byte-exact mode and its existing tests are unchanged.

Docs: docs/guides/agent-first-setup.md now scopes the "8 MiB budget is an error with data omitted" rule to non-log tools, and links the logs contract for the log tools' bytes cutoff. The maxEvents input description says only type:log events count (this changes the tool's schema text).

Decision doc: docs/reference/mcp-logs.md records no-go on stream handles, resume cursors and idempotency keys, with the conditions for revisiting. No cursor code is added.

Validation

Rework at e2c41a98 (after independent verification of 42e7b3b7):

  • bun test tests/mcp-logs-contract.test.ts tests/mcp-lifetime.test.ts: 28 pass, 0 fail, 3 consecutive runs
  • bun test tests/mcp.test.ts tests/mcp-session-context.test.ts tests/mcp-output-budget.test.ts tests/mcp-command-admission.test.ts tests/mcp-prompts.test.ts tests/mcp-startup-contract.test.ts tests/integration-ownership.test.ts tests/agent-plugin-generation.test.ts: 50 pass, 0 fail. No guidance render changed, so no fingerprint was appended.
  • bun run --cwd packages/cli typecheck: exit 0. bun run --cwd packages/cli check: exit 0, 0 errors, 34 warnings (unchanged); changed files lint clean.
  • No new macOS DiagnosticReports entry after the runs.

Original head 42e7b3b7:

  • bun test tests/mcp-logs-contract.test.ts tests/mcp.test.ts tests/mcp-lifetime.test.ts tests/mcp-output-budget.test.ts tests/mcp-command-admission.test.ts tests/mcp-session-context.test.ts tests/mcp-socket-backend.test.ts: 62 pass, 0 fail
  • bun test tests/integration-ownership.test.ts tests/agent-plugin-generation.test.ts: 20 pass, 0 fail. No guidance render changed, so no fingerprint was appended.
  • bun run --cwd packages/cli typecheck: clean
  • bun run --cwd packages/cli check: 0 errors, 34 warnings (the base branch also has 34; the changed files have none)
  • The CLI surface is unchanged, so docs:cli-reference was not regenerated.

New tests (tests/mcp-logs-contract.test.ts) use in-process sessions against a scripted stub backend (HACK_MCP_COMMAND), with a temp HOME, HACK_HOME and registry. They do not touch Docker/OrbStack or agent client config:

  • Each outcome for tail and snapshot, including heartbeat-interleaved maxEvents, a post-stop error, an error event that ends 8 bytes before the byte limit in the same write as the over-limit bytes, and a byte cut 3 bytes into a 4-byte emoji (asserts no U+FFFD and the exact retained byte count).
  • No replay: consecutive calls each start a new stream. A mid-tail disconnect stops the backend process, and a reconnected session sees only its own invocation's events. A cancelled tail stops its backend, and the retry in the same session starts fresh.

Negative controls

  • Mutating the collector (code-point-safe off, count all events toward maxEvents, accept failure events after a stop) fails 6 of the new tests. Restoring it passes them.
  • Rework: restoring the old gate (ignore all failure events once any stop fires) fails the new "error event read in the same chunk as the byte cut" test on 3 of 3 runs. Dropping the gate entirely fails the post-stop error test. Both restored afterwards.
  • A unit test shows that the byte-exact default capture does split a code point (it emits U+FFFD), so the UTF-8 assertion can detect the bug.

Contract change to an existing assertion

tests/mcp-lifetime.test.ts "hack.project.logs.tail bounds oversized stdout/stderr" asserted ok: false, isError, and no data after the byte limit. This unit's brief requires that result to be a cutoff success with partial data. For logs.tail only, the test now asserts ok: true and data.cutoff = {bound: "bytes"}. The outputTruncated assertion, the bounded capture size, the stopped child and the usable session are still asserted. hack.projects.list keeps the original assertions unchanged. The timeout lifetime test now also asserts ok: true and cutoff: {bound: "time"}.

Remaining gaps

  • Not exercised against real Docker Compose or Loki backends. Only the stub stream is tested. The classification follows the end reasons the CLI writes (eof, exit:N, closed, error).
  • The native/authored logs path writes one JSON result object, not typed stream events. It completes as eof, and maxEvents does not apply to it.
  • The events/time race described above is accepted and documented, not closed.
  • The time-window tests use a 2 s window (was 300 ms) so Bun stub startup on a loaded host cannot flake them.
  • Hosted CI has not run yet on this head.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

roodboi and others added 30 commits October 11, 2026 21:28
Add a data-only matrix (src/agents/client-capabilities.ts) that classifies
packaging, skill discovery, hooks, plugin MCP, standalone MCP, rules and UI
for Codex, Claude Code, Cursor and standalone consumers. Each cell records
status, evidence class, evidence ref, observed client version, source SHA
and host label. The rendered reference doc carries companion-package layout
and model-policy ownership decisions.

Tests enforce that every cell is classified, verified cells cite an existing
fixture-strength test, manifest or listing evidence never verifies loading,
and manifest-declared components agree with the matrix. The plugin
generation test gains a negative-controlled scan of plugins/hack and
examples/basic for model, permission and output-style settings and
non-primer hooks, plus a skill inventory check against the generator and
docs/integrations.md, which now names all four shipped skills.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hack.project.run built `run <flags> <service> <cmd...>`, so hack's own
parser consumed child flags: `node --version` printed hack's version and
`--branch other` retargeted the instance. Run now emits
`run <flags> -- <service> <cmd...>` and every option value is fused as
`--opt=value` across all tools.

- Validate service, services, branch, profiles, logs service and open
  target against a compose-identifier pattern (no leading '-', no empty
  value; branches may also contain '/').
- tail, maxEvents, maxMs and timeoutMs are positive integers with hard
  ceilings (provisional F27 settings); effective limits are echoed in
  data.limits.
- Every tool carries annotations (readOnly for list/status/logs/open,
  destructive for down/run/init, openWorld false).
- Refusals return error{code, message} with existing codes (E_USAGE,
  E_PROJECT_NOT_FOUND) and spawned:false. A typed tools/call dispatcher
  validates before any handler runs, so refused calls leave no argv
  entry and no audit line.
- Children spawn with HACK_NO_INTERACTIVE=1 on a copy of the frozen
  session env.

Refs HACK-1133, HACK-1134

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Legacy project env key reads (checkout-local, shared common-dir and
inherited primary keys) now use the same lstat, O_NOFOLLOW, size-bound and
stable-descriptor discipline as the native reader, extracted into
src/lib/private-file.ts. Key priority is unchanged. A symlinked, dangling,
oversized or non-regular key file is refused with a fixed message.

New key files are created with open(O_CREAT|O_EXCL|O_NOFOLLOW|O_WRONLY,
0600), fchmod 0600 and fsync, so an existing entry or dangling symlink is
never followed or replaced. A concurrent creator's key is adopted.

Encryption (env add --secret and the legacy migration) refuses a git-tracked
selected or inherited key with a fixed, value-free message before any key
copy. Decryption with a tracked key stays allowed for v4 compatibility.

Legacy v1 values and the encrypted_file store now require canonical base64,
a 12-byte IV, a 16-byte tag (authTagLength pinned) and a bounded ciphertext;
all parse, tamper and wrong-key failures use fixed redacted errors. The
envelope format, key derivation and v1 decryptability are unchanged.

Refs HACK-1127

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nfig

Address verifier findings on the agent-client capability matrix:

- Move the matrix renderer into scripts/generate-agent-client-capabilities.ts
  and add `bun run docs:agent-client-capabilities` (with `--check`), which
  rewrites only the generated section. The test compares the committed
  section with the rendered one and covers marker failures with closed codes.
- Scope the model/permission policy scan to generated guidance surfaces
  (plugins/hack, .claude-plugin, .cursor-plugin, .agents through its skill
  symlinks, and the example agent configuration) instead of example compose
  files. Parse YAML and markdown frontmatter as YAML, fail closed on
  structured-config formats the scan cannot parse, and count a file as
  scanned only when it was parsed.
- Treat reasoning-effort keys (reasoning_effort, model_reasoning_effort,
  frontmatter `effort`, nested effort/reasoning keys) as forbidden model
  settings, with negative controls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every pre-spawn refusal now comes from a closed catalogue (MCP_REFUSALS):
error {code, reason, message} uses an existing code, a closed reason and a
fixed message, and stderr repeats that message. Project names, ids, paths
and registry values are no longer echoed. Schema refusals name only keys
of the tool's own input schema.

An unknown tool name returns the SDK's in-band isError text result again
instead of a JSON-RPC -32602 protocol error, with a fixed message that
omits the requested name.

A canary test drives every refusal reason and the unknown-tool path and
asserts the canary never appears in the result, with no spawn or audit line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting tests

Seal authentic envelopes with 8- and 16-byte IVs and with ciphertext one
byte over each parser bound, so only the parser guards can refuse them, and
assert through a createDecipheriv spy that structural refusals happen before
any decipher and that successful opens pass authTagLength 16. Covers the
legacy project env, encrypted_file store and native env value parsers, with
at-bound 12-byte-IV controls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Interactive `hack init` now prints a non-mutating note that points at the
optional Hack plugin and `hack agent onboard`. Repository-local export of
Cursor rules, Claude hooks, Codex skills, AGENTS.md/CLAUDE.md sections and
project MCP config is a separate opt-in whose confirm defaults to no and
whose target list starts empty. The decision moves out of project.ts into
src/agents/init-agent-integrations.ts.

Doctor and the agent-prime freshness inventory now treat absent optional
integrations as current. Doctor flags a path only when it exists and lacks
the current revision marker (AGENTS.md/CLAUDE.md only when they carry a
Hack-managed section). integration-sync counts only stale or errored
checks; user-owned docs without a Hack section are not drift; retired
legacy artifacts still are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
config plan and config validate only treated the literal "base" as the
base selection, so `--env default` (and DEFAULT/none) reached the native
compiler as an overlay named "default" while env get, env list and the
runtime commands selected base layers.

Every CLI command that accepts --env now resolves it through one shared
resolver (src/cli/env-selection.ts): omitted keeps the configured default
overlay, base/default/none in any case select base layers, other names are
normalized, and a non-empty value with no usable name characters is refused
with the fixed "Invalid --env value." usage error instead of silently
selecting base. Authored native dispatch uses the same resolver, and
session workspace names stay env-base.

docs/env.md gains a command and delivery matrix with one row per CLI_SPEC
command that accepts --env; tests/env-command-matrix.test.ts keeps the rows
and selector flags in sync with CLI_SPEC. The canonical agent guidance gains
an env-composition section linking to the matrix; the content revision,
plugin bundle, repository examples and CLI reference are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address the W10 verifier findings so integration sync and doctor agree:

- checkAgentDocs marks a doc `unmanaged` only when it has neither the
  Hack start nor end marker. A doc with exactly one marker is a corrupted
  managed section and now reports a malformed-marker error, which sync
  counts as drift. Doctor treats the same doc as stale.
- Add readOptionalTextFile, which uses lstat before reading so callers
  can tell "absent" from "present but unreadable". The Cursor rules,
  Codex skill, shared skill, hack-init skill and agent-doc checks now
  report an unreadable existing file as `error` (drift) instead of
  `missing` (optional, current). Doctor uses the same helper.
- Add regression tests for start-only and end-only docs, plus mode-000
  Hack-owned files and docs in fixture HOME/project roots, for both
  sync and doctor.

Default `hack init` still writes no repository files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The env add --secret refusal tests asserted the refusal text on stderr, but
the default logger backend depends on the host: gum (stderr) when installed,
clack (stdout) otherwise. Hosted macOS runners have no gum, so the refusal
reached stdout and stderr was empty only in CI. Pin HACK_LOGGER=console in
the CLI helper, matching the other CLI-spawning tests, so the stderr
assertions hold on every host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The project-name boundary suite still expected the old value-echoing
"Ambiguous project name/ID" stderr from MCP. Assert the closed
MCP_REFUSALS contract instead: exact code, reason and fixed message,
spawned: false, no argv, and none of the duplicated names, registry ids
or fixture root anywhere in the serialized result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Temporary measurement only; reverted before the final commit. Writes closed
labels and durations per native command to a per-scenario JSONL file and
prints a summary in the docker-e2e job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A storage-bearing saved exec ran three full witness proofs (nine helper
carriers) and a warm up/restart seven. Within one command and one mutation
lease, an adjacent repeat with only read-only checks between it and the
previous proof now reuses that completed kernel proof instead of three new
helper carriers:

- saved exec: the pre-spawn check reuses the preparation proof; the proof
  after the process stays fresh (3 -> 2 proofs);
- up/restart without build, routing or file material: the first pre-spawn
  guard reuses the enrollment proof (warm 7 -> 6, cold seed path 6 -> 5).

Reuse requires the same lease authority, engine, generation and reference
and a byte-identical opening material binding to the one that opened and
closed the recorded proof. Every host check (anchors, expectation,
completion, binding, carrier artifact and lifetime) still runs; any
verification consumes the record first, so a failure or binding change
forces a fresh proof. Ordinary verification always proves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts the hosted measurement commit (closed-label cost trace, phase-trace
routing, per-scenario summary step and artifact) now that the per-phase and
per-call attribution is recorded in the pull request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The networks scenario took 3409 s on next with three proofs per storage exec
and ran 2275 s on the hosted engine with the coalesced proofs, so the 80 min
budget raise from the witness recovery change returns to 60 min. The budget
bounds a hung engine; no assertion or command deadline changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Storage witness proofs keep every fence; three exact costs are cut:

- Engine-ID fences: for an explicit DOCKER_HOST=unix:///... selection, the
  first fence of a command under one exact binding (selection, docker
  executable, client config and socket identities) still uses `docker info`
  and must equal one direct GET /info on that socket; only then do later
  fences under that unchanged binding each make one fresh direct request
  instead of a CLI process that lists every client plugin. A refused
  admission keeps the CLI, a moved binding re-admits through the CLI, and an
  admitted direct failure refuses without fallback. Two exact back-to-back
  ID reads with nothing between them are collapsed (holder observation into
  carrier inspect; provision fence into the target observation).
- Recorded command executables: the content digest is cached for one command
  by canonical path and complete stat identity (dev, ino, size, mtime and
  ctime in ns, mode, uid); every identity check still runs, any change
  re-reads, and racily fresh files are never cached.
- Target observations: the witness's closing re-observation of a
  deferred-finish verify invocation is answered by the carrier's own closing
  observation of the same query when the rechecked material binding is
  unchanged and only reads ran between (one observation per proof).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Temporary measurement only; reverted before the final commit. Re-applies the
X01 closed-label trace (reverting eb9a678) and adds point events for direct
engine-ID reads and admissions, cached executable digests, target observations
and reused closing observations, plus a shard 1 identity fence benchmark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An admitted direct engine-ID read that fails because the caller cancelled or
its probe deadline expired now reports the CLI probe's existing closed
cancel/timeout classification; that probe refuses before spawning anything.
Every other admitted direct failure still refuses without CLI fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Per command on the synthetic engine: 81 of 86 engine-ID fences and 20 of 21
  target observations per proof, with every proof kept; an admitted explicit
  Unix selection uses one CLI fence per command.
- A changed daemon ID between two admitted fences refuses before any helper,
  workload or state change; a refused admission keeps every fence on the CLI.
- A target change right after the verify carrier's closing observation is
  refused by the next remaining observation.
- Admission, re-admission, no-fallback and cancellation controls on a real
  Unix server; executable digest reuse, replacement, same-size tamper, mode,
  racy-fresh and mid-command replacement controls.
- Reference docs describe the three cuts and their limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measurement is complete; the hosted trace run was 38211327829.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… catalogue

Plugin skills can now ship references at skills/<skill>/references/*.md from
a typed registry (src/agents/skill-references.ts). The generator writes each
reference with a recomputable content revision, removes stale ones, and links
them from the plugin SKILL.md. Standalone hack-cli installs ship no
references, so their render (and ownership fingerprint) is unchanged.

The first reference is the hack-cli pattern catalogue, rendered from the typed
register of all twelve HACK-P cards with dispositions, owners, missing
contracts and value-free provenance. Tests check completeness, CLI_SPEC
command resolution, a banned-recipe scan outside negative-example blocks, and
an offline link contract over every packaged skill markdown file. External
URLs in hack-install and hack-remote move under "Further reading".

docs/agent-guidance.md records the reference contract and the promote/defer
decision for the lifecycle, cleanup, checks, sessions, hack-run and
hack-feedback skill candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seed an isolated HOME with sentinel Claude, Codex, Cursor and shared-skill
config (including stale hack-cli copies) and two fixtures with customized
AGENTS.md, CLAUDE.md, rules, skill copies, settings.local.json and .mcp.json.
Run init --auto, init --auto --with codex, agent prime, agent prime --check,
agent onboard, agent patterns, doctor --json, doctor --fix --no-interactive
and setup sync --check twice each with HOME and HACK_HOME overridden, and
require byte-identical hashes, no new client entries, git status changes only
under .hack/, and each command's expected exit code. An explicit
setup agents --agents-md is the positive control and must change exactly
AGENTS.md.

Docker is pointed at a missing socket and Mutagen at a stub so doctor --fix
stays free of Docker and network effects; the hackd it starts is stopped.
The scenario is registered in ALL_SCENARIOS and shard.ts places it in
exactly one docker-e2e shard via the default duration hint.

Refs: HACK-1124, HACK-1132, HACK-1155

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HACK-1211 needs a matched, bounded way to reproduce the intermittent Bun
crash and a repeatable record of crashes per hosted run. This adds the
instruments only; it changes no test, timeout, assertion or pin.

- bun-crash-experiment.yml runs only on workflow_dispatch or when the
  bun-crash-experiment label is added to a pull request. A labeled pull
  request runs the matched pair (1.3.9 baseline, 1.4.2 candidate) on its
  head; a dispatch runs one version at a chosen revision. It uses
  blacksmith-6vcpu-macos-15, contents: read, and timeout = budget + 15.
  The instrument comes from the workflow's commit, so older revisions
  can be measured.
- scripts/bun-crash-loop.ts alternates the native HTTPS owner test with
  the full bun test. It never starts an iteration whose estimate (its
  initial guess or longest observed run) ends past the budget, and
  records a run still going at the deadline as overrun, not a crash. It
  writes a closed summary and keeps logs plus bun*.ips reports for a
  7-day artifact. It never opens crash-report links.
- scripts/ci-bun-crash-census.ts reads every attempt's test job through
  the caller's gh, downloads only failed-job logs into an owned temp dir
  that it removes, and prints per-version counts and crashing ids only.
- Both use scripts/lib/bun-crash-classify.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tions

Storage-witness engine-ID admissions now resolve the Docker CLI's endpoint
selection the way the CLI does: a nonempty DOCKER_HOST, then DOCKER_CONTEXT,
then the client config's currentContext, then the default context at
unix:///var/run/docker.sock. A named context is read from its
contexts/meta/<sha256>/meta.json through a no-follow binding; only a plain
unix:/// docker endpoint with no TLS option, no unknown or duplicate field and
no entry at the context's TLS path is eligible. TLS material is checked for
absence by name and never opened. Missing, malformed, linked, TCP, SSH, TLS or
header-bearing contexts, and DOCKER_HOST with DOCKER_CONTEXT, keep the CLI.

Admission is unchanged: the first fence under a binding is the CLI read and
must equal one direct GET /info on the resolved socket. The binding now also
pins the context store entry and the absent TLS path, so a changed config,
context entry, socket or executable is admitted through the CLI again. The
ingress and VM-file observers, trusted without a CLI comparison, still accept
only an explicit Unix DOCKER_HOST.

Tests that run commands against a synthetic docker now pin an isolated client
config naming a missing context, so they never read the contributor's Docker
config or dial a real engine socket. A new opt-in test (HACK_TEST_DOCKER_CLI=1,
run on docker-e2e shard 4) runs the real Docker CLI against two fake Unix
engines for every selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Temporary measurement only; reverted before the final commit. Re-applies the
X03 closed-label trace (6a4e118) without its identity benchmark, labels each
engine-ID admission with its selection kind (host, default or context), counts
direct reads as trace leaves, and prints the runner's Docker client selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measurement is complete; the hosted trace run was 38225752202.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roodboi and others added 15 commits October 12, 2026 08:39
Merge origin/train/queue-preview (77da332) into the env-file branch.

Resolutions:
- No textual conflicts; the branch only touches env/secret-store files.
- src/agents/integration-revision.ts: taken from train; matches the
  revision tests/agent-instruction-source.test.ts computes.
- src/agents/integration-ownership.ts: KNOWN_RENDER_FINGERPRINTS is the
  union (train's appended fingerprints, nothing removed);
  tests/integration-ownership.test.ts asks for no further entries.
- Checked-in renders (.codex/skills/hack-cli, .cursor/rules/hack.mdc,
  examples/basic AGENTS.md/CLAUDE.md, plugins/hack) come from train;
  generate:agent-plugins produced no diff. No CLI surface change.

Verified: skill-references, integration-ownership, agent-plugin-generation,
agent-instruction-source, hack-init-skill, legacy-agent-artifacts,
private-env-file, secret-store-file-hygiene, aes-gcm-v1-context-binding
tests pass; packages/cli typecheck passes; packages/cli check 0 errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolutions:
- No textual conflicts; git merged all files automatically.
- src/agents/integration-revision.ts and KNOWN_RENDER_FINGERPRINTS in
  src/agents/integration-ownership.ts come from train/queue-preview only
  (this branch does not touch agent guidance); agent-instruction-source and
  integration-ownership tests pass with the merged values.
- generate:agent-plugins and docs:cli-reference produce no further changes.

Checks: skill-references, integration-ownership, agent-plugin-generation,
agent-instruction-source, mcp-lifetime, mcp-logs-contract (73 pass);
packages/cli typecheck clean; packages/cli check 0 errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…patterns-b6

Resolutions:
- No textual conflicts; git merged all files cleanly.
- src/agents/integration-revision.ts: took train value 51163ae32e56,
  which matches the revision tests/agent-instruction-source.test.ts
  recomputes from INSTRUCTION_SECTIONS on the merged tree.
- src/agents/integration-ownership.ts: KNOWN_RENDER_FINGERPRINTS is the
  union of both sides (train appended the offline env-matrix pointer
  fingerprints for hack-cli-skill and cursor rules); the ownership test
  requested no further fingerprints.
- Checked-in renders (.codex/skills/hack-cli, .cursor/rules/hack.mdc,
  examples/basic AGENTS.md/CLAUDE.md, hack-init skills) and the plugin
  bundle come from train and match their renderers byte-for-byte;
  generate:agent-plugins produced no diff. CLI surface unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…staleenv-b6

Resolutions:
- No textual conflicts; train/queue-preview brought the offline env
  guide bullet change (instruction source, integration revision,
  render fingerprints and checked-in renders) unchanged.
- src/agents/integration-revision.ts: kept the train value; it matches
  the revision tests/agent-instruction-source.test.ts computes.
- KNOWN_RENDER_FINGERPRINTS: train's append-only list; no extra
  fingerprint requested by tests/integration-ownership.test.ts.
- generate:agent-plugins produced no further changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosted CI failed "web should receive the env value resolved at up"
because `hack exec web -- printenv` stdout began with Clack warnings
(CoreDNS/Caddy not reachable) before the probe value. Every exec probe
now runs with HACK_LOGGER=console so Hack's logger writes to stderr,
and compares the last non-empty stdout line exactly. A wrong or missing
value still fails (missing output yields null).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the docker-tier compose-config-hash-drift scenario (HACK-1124, F03).
It starts app and dep, drifts only dep's Compose config hash, and requires
hack run and hack exec to keep every running container's full ID,
StartedAt, RestartCount and config-hash label, with no leftover one-off.
Positive controls: the run one-off sees the drifted value, and a final
hack up recreates dep.

Reproduce-only: the legacy Compose path already passes --no-deps when the
target is running and exec never converges, so no source change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`hack doctor` and `hack doctor --json` now run a read-only "env key" check
(HACK-1127). It audits the checkout-local, shared (git common dir) and
primary-checkout key files, or HACK_ENV_SECRET_KEY when no file exists, and
reports closed codes: env-key-ok, env-key-absent, env-key-weak-permissions,
env-key-wrong-owner, env-key-wrong-length, env-key-low-entropy,
env-key-symlink, env-key-not-regular, env-key-unreadable, env-key-tracked and
env-key-public-path. Findings carry project-relative subjects and the exact
command to run; key bytes and absolute paths are never emitted, symlinks are
never followed and --fix does not act on the check.

Also add scripts/dependabot-inventory.ts, a GET-only `gh api` inventory of
open Dependabot alerts (package, severity, manifest, fixed-in) that writes
nothing, documented in the docs contributor section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gum-backed logger passed the log message as a bare argument, so a
message starting with "-" or "--" (for example "--value-stdin requires
...") was parsed by gum's kong CLI as an unknown flag. gum printed its
usage and "gum: error: unknown flag" to stderr before the logger fell
back to clack.

Every gum call site in src/ui/gum.ts now places free text (log message
and structured fields, style/join text, confirm prompt, choose/filter
options, file path, pager content, format template, spin command,
version-check constraint) after "--", and binds valued flags such as
--header, --placeholder, --prompt, --value, --title and --affirmative
in single-token --flag=value form so a dash-leading value cannot be
read as an option.

gum invocations now go through a small GumRunner boundary so tests can
assert argv without gum installed or any process being spawned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in; no content change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in; no content change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in; no content change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in. next's tree equals train/queue-preview 77da332, which this branch already contains, so the branch tree is kept unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in. next's tree equals train/queue-preview 77da332, which this branch already contains, so the branch tree is kept unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in. next's tree equals train/queue-preview 77da332, which this branch already contains, so the branch tree is kept unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-parents onto next after the queued PRs squashed in. next's tree equals train/queue-preview 77da332, which this branch already contains, so the branch tree is kept unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi changed the base branch from train/queue-preview to next October 12, 2026 14:10
roodboi and others added 6 commits October 12, 2026 10:21
Stack PR #302 on PR #311.

Conflicted files: none. The two change sets are disjoint (#302:
env-file and secret-store hygiene; #311: src/ui/gum.ts with its tests
and fixture), so git merged them cleanly. No generated files changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. Git auto-merged the only shared file:
- docs/env.md: kept both sides' text (the env key doctor notes from this
  branch and the private env file and encrypted_file store notes from
  wf/y3-envfile-b6).

No generated files changed: neither branch alters CLI options or rendered
agent guidance, so plugins, docs/reference/cli.md, integration-revision.ts
and KNOWN_RENDER_FINGERPRINTS needed no update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. The merge applied cleanly; the two change sets
touch disjoint files.

Regenerated plugins (generate:agent-plugins) and docs/reference/cli.md
(docs:cli-reference) after the merge; both produced no changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. The merge was clean; the two change sets touch
disjoint files. Regenerated agent plugins and docs/reference/cli.md
produced no diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stack PR #301 on PR #307. The merge had no textual conflicts.
Git auto-merged docs/env.md, tests/e2e/run.ts and tests/e2e/shard.ts;
each keeps both sides (stale env warning plus env key/private file docs;
both exec-stale-env and compose-config-hash-drift scenarios and hints).
Regenerated plugins and docs/reference/cli.md produced no drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. The two change sets touch disjoint files, so git
merged them automatically. Generated surfaces (plugin bundle, CLI reference,
checked-in renders) were regenerated after the merge and showed no drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi changed the base branch from next to wf/y5-staleenv-b6 October 12, 2026 14:30
Base automatically changed from wf/y5-staleenv-b6 to next October 12, 2026 19:05
Re-parents onto next after the previous stacked PR squashed in; next's tree equals b0cde14, which this branch already contains, so the branch tree is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi marked this pull request as ready for review October 12, 2026 19:06
@roodboi
roodboi merged commit b8dbcd7 into next Oct 12, 2026
25 of 30 checks passed
@roodboi
roodboi deleted the wf/y1-logs-b6 branch October 12, 2026 19:06
roodboi added a commit that referenced this pull request Oct 12, 2026
Issue: HACK-1133 (unit Y2). Stacked on #304 (`wf/y1-logs-b6`), which sits on `train/queue-
preview`.

## Final behavior

Five new MCP tools, appended after the existing ten:

| Tool | Annotation | CLI command |
| --- | --- | --- |
| `hack.project.exec` | destructive | `hack exec [--project/--path] [--branch=] [--profile=] [--workdir=] -- <service> <cmd...>` |
| `hack.project.doctor` | readOnlyHint | `hack doctor [--path=<repo root>] [--branch=] --json` |
| `hack.project.env.explain` | readOnlyHint | `hack env explain [--project/--path] [--env=] [--service=] [--target=] --json -- <key>` |
| `hack.project.env.keys` | readOnlyHint | `hack env list [--project/--path] [--env=] [--service=] --json` |
| `hack.project.instances` | readOnlyHint | `hack projects --json --project=<registered name>` |

- **exec** is separate from `run`: it targets an already-running container and never starts one.
  `service`, `branch` and `profiles` use the existing identifier patterns. `cmd` must have at
  least one entry. All option values are fused, and `--` comes before the service, so child
  flags such as `--help` or `--branch` never reach hack's parser (checked against `CLI_SPEC`).
  Output is bounded by the shared 8 MiB capture limit and `timeoutMs` (default 120 s, ceiling
  600 s). `data.limits` echoes `{ timeoutMs, outputBytes }`.
- **doctor** uses a closed argv. `--fix`, `--migrate-env-config`, `--domain-migration`,
  `--native-*` and legacy rebind are never emitted. Extra input fields are stripped by the
  schema. A named project is passed as `--path=<repo root>`, because `hack doctor` has no
  `--project` option. When a check fails, the CLI exits 1, so `ok` is false, but the parsed
  report is kept in `data`.
- **env.explain / env.keys** are value-free. `stdout` is always `""`. `stderr` is `""` on
  success, or a fixed message on failure. `data` is an allowlisted projection: explain metadata
  (availability, secret flag, scope, source, precedence), or key rows `{ key, secret, required,
  source, resolved_from, services }`. Values, materialized values, storage locations and warning
  text are dropped. `--show-secrets` is never passed, and the tool has no input for it. Keys
  must match `^[A-Za-z_][A-Za-z0-9_]*$`.
- **instances** maps the selection to one registered project name. A path, or the session cwd,
  selects the deepest containing registration. Two different registrations at the same root are
  refused as `ambiguous_project`. An unregistered path is refused. The tool returns `{ project,
  found, runtime_ok, instances: [{ branch, compose_project, working_dir, services: [{ service,
  containers, running }] }] }`. `stdout` is withheld, because the raw listing includes container
  inspect data.
- Refusals reuse the existing closed catalogue. No new reasons or codes were added.
- `docs/reference/mcp-tools.md` is the new MCP tool contract. It covers all 15 tools:
  annotations, selection, identifiers, results, refusals, the value-free rules and the audit
  log. It is linked from `docs/reference.md` and the agent-first setup guide.

## Validation

- `bun test tests/mcp-inspect-tools.test.ts`: 16 pass. It checks:
  - argv for all five tools.
  - exec vs run separation.
  - CLI parser boundaries for exec and env explain.
  - 19 refusal probes. Each one spawns nothing, writes no audit line and echoes no input.
  - doctor repair flags are stripped.
  - env canary withholding on both success and failure.
  - instances selection by name, path, cwd and nested registration, plus its refusals.
  - `tools/list` annotations.
  - The doc table matches the registered tools and their annotation classes.
- `bun test tests/mcp.test.ts` (tools/list expectation extended, nothing removed), plus `mcp-
  logs-contract`, `mcp-lifetime`, `mcp-output-budget`, `mcp-session-context` and `mcp-prompts`:
  50 pass.
- `bun test tests/integration-ownership.test.ts tests/agent-plugin-generation.test.ts`: pass.
  The guidance render did not change, so no fingerprint was appended.
- `bun test tests/cli-reference-docs.test.ts`: pass. The CLI surface did not change, so no
  regeneration was needed.
- `bun run --cwd packages/cli typecheck`: clean. `bun run --cwd packages/cli check`: 0 errors.
  The 34 warnings were already there, and none are in the changed files.

Tests use stub executors only. Each fixture has a temp HOME, HACK_HOME and registry, and the
stub `hack` binary is selected with `HACK_MCP_COMMAND`. The tests do not touch Docker/OrbStack,
the keychain, real agent client config or the real home directory.

## Negative controls

- The same scripted env payload sent through `hack.projects.list`, which forwards raw output,
  *does* contain the secret canary. The scripted payload also contains it before projection. So
  the canary's absence from the env tools comes from the withholding, not from the stub or the
  transport.
- Calling doctor with `fix`, `migrateEnvConfig`, `domainMigration` and `nativeRunMapping`
  produces `["doctor", "--json"]`.
- Calling env.keys with `showSecrets: true` produces argv without `--show-secrets`.
- The first instances run failed because raw stdout leaked container mounts. That failure is why
  `stdout` is now withheld, and the test asserts it.

## Remaining host-bound gaps

- None of this ran against a real `hack` binary, Docker/OrbStack or a live MCP client. Real
  `hack exec` TTY handling, `hack doctor --json` timing on a host, and the daemon-served `hack
  projects --json` shape (assumed to match `serializeProjectView`) are not proven here.
- A legacy-contract `env explain` or `env list` may read keychain-backed values to compute
  availability. That can raise a native macOS authorization prompt. The MCP layer does not
  change this.
- `hack projects` can auto-register runtime projects as a side effect. That is existing CLI
  behavior, shared with `hack.projects.list`, which is already annotated read-only.

## Secret scan

The first head (cea44f8) failed `secret-scan`: gitleaks' generic-api-key rule flagged the
synthetic test canary `SECRET_CANARY` in `tests/mcp-inspect-tools.test.ts` (entropy 3.79). It is
a fixed marker the tests use to prove env values are never echoed, not a credential.

85c67b0 adds a narrow AND allowlist to `.gitleaks.toml`: the exact test path plus the exact
canary line. No code or test changed. This clears the leak already in branch history without
rewriting the branch.

- gitleaks 8.30.1 with the CI flags (`--full-history --diff-filter=tuxdb HEAD`,
  `.gitleaks.toml`): 1 leak at cea44f8, then no leaks at 85c67b0.
- Negative control (scratch repository, same config): a token-like value on the next line of the
  same file, and the same canary line in another path, are both still reported.
- Re-ran the focused MCP, integration-ownership and agent-plugin-generation tests (47 pass), CLI
  typecheck (exit 0) and CLI check (0 errors, 34 existing warnings).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@blacksmith-sh

blacksmith-sh Bot commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
native-config-build (133.3s) — Native --profile must succeed/
native-config-build (133.3s) — Native --profile must succeed
View Logs
native-config-down-hooks (19.0s) — Each finite down mode starts a ready owned generatio
n/
native-config-down-hooks (19.0s) — Each finite down mode starts a ready owned generatio
n
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant