From d111802ce7432dbe2f940d3c541fd09ffd9143e3 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:28:47 -0400 Subject: [PATCH 01/52] docs: add versioned agent-client capability matrix and policy controls Add a data-only matrix (src/agents/client-capabilities.ts) that classifies packaging, skill discovery, hooks, plugin MCP, standalone MCP, rules and UI for Codex, Claude Code, Cursor and standalone consumers. Each cell records status, evidence class, evidence ref, observed client version, source SHA and host label. The rendered reference doc carries companion-package layout and model-policy ownership decisions. Tests enforce that every cell is classified, verified cells cite an existing fixture-strength test, manifest or listing evidence never verifies loading, and manifest-declared components agree with the matrix. The plugin generation test gains a negative-controlled scan of plugins/hack and examples/basic for model, permission and output-style settings and non-primer hooks, plus a skill inventory check against the generator and docs/integrations.md, which now names all four shipped skills. Co-Authored-By: Claude Opus 5.5 --- docs/agent-guidance.md | 6 + docs/integrations.md | 9 +- docs/reference/agent-client-capabilities.md | 143 ++++++ src/agents/client-capabilities.ts | 517 ++++++++++++++++++++ tests/agent-client-capabilities.test.ts | 465 ++++++++++++++++++ tests/agent-plugin-generation.test.ts | 262 +++++++++- 6 files changed, 1398 insertions(+), 4 deletions(-) create mode 100644 docs/reference/agent-client-capabilities.md create mode 100644 src/agents/client-capabilities.ts create mode 100644 tests/agent-client-capabilities.test.ts diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 44ae2c1d..2e7c1a2e 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -3,6 +3,10 @@ Hack owns Hack-specific instructions. User and project policy continues to own model selection, general coding standards, work-unit planning, evolution, and writeback. Do not ship provider model pins or copy global instruction packs into this plugin. +`tests/agent-plugin-generation.test.ts` enforces this for every file under +`plugins/hack` and `examples/basic`: no model or model-override keys, no permission, +approval or sandbox overrides, no output-style settings, and hooks that run only +`hack agent prime`. See [model-policy ownership](reference/agent-client-capabilities.md#model-policy-ownership). ## Sources and installed surfaces @@ -43,6 +47,8 @@ belong to their client adapters; shared behavior belongs to the canonical source 4. Run instruction-source, onboarding, hack-init-skill, shared-agent-skill, and plugin generation tests. Run normal typecheck/check/test gates for behavior changes. New packaging needs fresh client loading proof beyond a successful marketplace listing. + Record that proof, or the open gate, in `src/agents/client-capabilities.ts` and + regenerate the [capability matrix](reference/agent-client-capabilities.md). 5. Release preparation regenerates plugin assets after the version bump. Verify the published bundle and installed executable separately. diff --git a/docs/integrations.md b/docs/integrations.md index 90a7c599..b9dc0f48 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -61,8 +61,11 @@ trust and approval rules. ## Optional native plugins Hack also ships an optional native plugin for Codex, Claude Code, and Cursor in -`plugins/hack`. It bundles the `hack-cli` and `hack-init` skills, the `hack mcp serve` -server, Claude primer hooks, and a Cursor rule that the agent can select when relevant. +`plugins/hack`. It bundles four skills (`hack-cli`, `hack-init`, `hack-install`, and +`hack-remote`), the `hack mcp serve` server, Claude primer hooks, and a Cursor rule +that the agent can select when relevant. The +[agent client capability matrix](reference/agent-client-capabilities.md) records +which of these components have been observed loading in each client. Install the Hack CLI first and make `hack` available on the client's PATH. Choose either the native plugin or the existing standalone setup for each client. @@ -95,7 +98,7 @@ outside its local-plugin root; use a copy rather than weakening that boundary. S [Cursor's plugin documentation](https://cursor.com/docs/plugins) for marketplace and local development support in the installed client version. -After installing, verify in a fresh session that both skills are discoverable, +After installing, verify in a fresh session that all four skills are discoverable, `hack mcp serve` can initialize, and the client's hooks or rule load. A successful marketplace registration alone does not prove those components loaded. The plugin's MCP fallback is for clients without shell access; use the Hack CLI when shell is available. diff --git a/docs/reference/agent-client-capabilities.md b/docs/reference/agent-client-capabilities.md new file mode 100644 index 00000000..638c81db --- /dev/null +++ b/docs/reference/agent-client-capabilities.md @@ -0,0 +1,143 @@ +# Agent client capability matrix + +This page classifies what Hack's optional agent integrations are known to do in +each client. The data lives in `src/agents/client-capabilities.ts`. +`tests/agent-client-capabilities.test.ts` checks the generated section below +against that data and enforces the classification rules. + +## Classification rules + +- **verified**: a named test file observed the client load the component, at + `isolated-fixture` or `authenticated-session` strength. +- **untested**: Hack ships the component, but no fixture or session has observed + a client load it at the recorded version. Each untested cell names its open gate. +- **unsupported**: Hack does not ship the component for that client. This says + nothing about what the client itself supports. + +Evidence classes, weakest first: + +| Class | Meaning | +| --- | --- | +| `source-read` | Classified by reading shipped files or client documentation. | +| `schema` | A test parses or generates the shipped file; no client loads it. | +| `isolated-fixture` | A test runs the real client with an isolated home and configuration. | +| `authenticated-session` | A signed-in client session used the component. | + +A manifest parse, a schema check or a marketplace listing is never evidence that a +component loaded. Each cell records the client version, a host label and the source +commit at which it was classified. That is one observation on one host. It does not +cover other machines or later client versions. A status for one client implies +nothing about another client, including UI parity. + +To reclassify a cell, update the data, increment +`AGENT_CLIENT_CAPABILITY_MATRIX_VERSION`, and regenerate the section below so the +test passes. Promote a cell to `verified` only after its named test passes against +the recorded client version. + + + +Matrix version 1, observed 2026-10-11. + +### Clients + +| Client | Manifest | +| --- | --- | +| Codex | `plugins/hack/.codex-plugin/plugin.json` | +| Claude Code | `plugins/hack/.claude-plugin/plugin.json` | +| Cursor | `plugins/hack/.cursor-plugin/plugin.json` | +| Standalone (portable manifest or generic stdio MCP client) | `plugins/hack/plugin.json` | + +### Capabilities + +| Capability | Meaning | +| --- | --- | +| Packaging | The client accepts, installs and enables the Hack plugin package in a session. | +| Skill discovery | A fresh session discovers the plugin's bundled skills (hack-cli, hack-init, hack-install, hack-remote). | +| Hooks | The client runs plugin-shipped hooks; Hack hooks may only run `hack agent prime`. | +| Plugin MCP | The client starts the plugin-declared `hack mcp serve` server and lists its tools. | +| Standalone MCP | The client loads an MCP entry written by `hack mcp install` (stdio or a selected MCP bundle) without the plugin. | +| Rules | The client loads a plugin-shipped rule file. | +| UI | The client renders plugin presentation metadata (display name, default prompts, brand color). | + +### Status summary + +| Capability | Codex | Claude Code | Cursor | Standalone (portable manifest or generic stdio MCP client) | +| --- | --- | --- | --- | --- | +| Packaging | untested | untested | untested | untested | +| Skill discovery | untested | untested | untested | untested | +| Hooks | unsupported | untested | unsupported | unsupported | +| Plugin MCP | untested | untested | untested | untested | +| Standalone MCP | untested | untested | untested | untested | +| Rules | unsupported | unsupported | untested | unsupported | +| UI | untested | unsupported | unsupported | unsupported | + +### Cells + +| Client | Capability | Status | Evidence class | Evidence | Client version | Host | Source | Note | Open gate | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| codex | Packaging | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| codex | Skill discovery | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| codex | Hooks | unsupported | source-read | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Codex plugin hooks; the manifest declares no `hooks` key. This says nothing about Codex's own hook support. | none | +| codex | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| codex | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | In the opt-in fixture, `codex mcp get hack --json` resolved the generated bundle adapter from an isolated CODEX_HOME. The fixture then stopped at the Claude step, so no tool call was observed in that run. Configuration discovery alone does not prove a Codex model session loaded the server or accepted project trust. | Pass tests/mcp-bundle-client.test.ts end to end at this client version; a Codex model session tool call remains a separate authenticated-session gate. | +| codex | Rules | unsupported | source-read | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Codex plugin rule file. Project AGENTS.md snippets are standalone setup, not a plugin component. | none | +| codex | UI | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares an `interface` block (display name, default prompts, brand color); rendering was not observed. | Observe the installed plugin's presentation in the Codex UI for this client version. | +| claude | Packaging | untested | schema | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| claude | Skill discovery | untested | schema | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| claude | Hooks | untested | schema | `plugins/hack/hooks/claude-hooks.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | SessionStart and PreCompact hooks run only `hack agent prime`; the generation test enforces that command allowlist. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| claude | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| claude | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | The opt-in fixture failed at this version: in an isolated HOME, `claude mcp get hack` reports the project-scoped server as pending approval instead of connected, so no connection or tool call was observed. Project approval is a native trust gate and must not be bypassed. | Qualify Claude standalone MCP without bypassing project approval (for example, user scope in an isolated HOME) and pass the fixture end to end. | +| claude | Rules | unsupported | source-read | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Claude plugin rule file. Project CLAUDE.md snippets are standalone setup, not a plugin component. | none | +| claude | UI | unsupported | source-read | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares no presentation metadata. Codex UI metadata implies nothing for Claude Code. | none | +| cursor | Packaging | untested | schema | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts. docs/integrations.md records an earlier manual symlink-rejection observation (Cursor 3.18.25), not a fixture. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| cursor | Skill discovery | untested | schema | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| cursor | Hooks | unsupported | source-read | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Cursor plugin hooks; the manifest declares no `hooks` key. | none | +| cursor | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| cursor | Standalone MCP | untested | schema | `tests/mcp-install.test.ts` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Tests check the `.cursor/mcp.json` entry Hack writes; Cursor itself never loaded it in a fixture. | Add an isolated Cursor fixture that loads the written entry and lists Hack tools. | +| cursor | Rules | untested | schema | `plugins/hack/rules/hack.mdc` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Agent-selectable rule (`alwaysApply: false`) matches the canonical renderer; loading was not observed. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| cursor | UI | unsupported | source-read | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares no presentation metadata. Codex UI metadata implies nothing for Cursor. | none | +| standalone | Packaging | untested | schema | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Portable agent-plugins manifest is derived from the Codex manifest; no portable consumer was observed. | Name a portable consumer and observe it install the package. | +| standalone | Skill discovery | untested | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | The portable manifest declares no component paths; discovery of `skills/` by convention is unverified. | Name a portable consumer and observe skill discovery in a fresh session. | +| standalone | Hooks | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | The portable manifest carries hooks only through the OpenAI extension, and Hack ships none there. | none | +| standalone | Plugin MCP | untested | schema | `plugins/hack/mcp.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Portable MCP manifest adds an explicit stdio transport to the same launch as `.mcp.json`. | Name a portable consumer and observe it start the declared server. | +| standalone | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | none | `macos-arm64-contributor` | `ec56769bdc04` | The fixture's reference MCP SDK client calls a real tool through the generated entry, but that step was not reached in the recorded run. The SDK client is not an agent. | Pass tests/mcp-bundle-client.test.ts end to end, or add a client-independent stdio fixture. | +| standalone | Rules | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | No portable rule component is shipped. | none | +| standalone | UI | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Presentation metadata exists only in the OpenAI extension; no portable UI is claimed. | none | + + + +## Companion-package layout decision + +Hack keeps three separate companion layouts. Each layout has one authoritative +owner, and none of them replaces the others. Users choose one per client. + +| Layout | What it contains | Authoritative owner | Installed and updated by | +| --- | --- | --- | --- | +| Native plugin (`plugins/hack`) | Client manifests, four skills, Claude primer hooks, Cursor rule, `hack mcp serve` declaration | `scripts/generate-agent-plugins.ts`, rendering from `src/agents/instruction-source.ts` and the skill renderers in `src/agents/` | The client's plugin manager. Hack commands never install or update it. | +| Standalone setup | Project AGENTS.md/CLAUDE.md snippets, Cursor rules, shared skills, Claude primer hooks, MCP entries | `src/agents/instruction-source.ts` and its renderers, written by `hack setup` and `hack mcp install` | Explicit `hack setup` and `hack mcp install` commands in a chosen scope | +| MCP bundle | Native MCP adapter, owner and backend executables with a content-derived identity | `src/mcp/bundle.ts` (packaging and verification) | `hack mcp install --bundle ... --cli ...` writes the client entry; the bundle is never modified after selection | + +Decisions: + +- Shared guidance has one canonical source, `src/agents/instruction-source.ts`. The + plugin and the standalone setup are renderings of it, not separate sources. +- Installing the plugin does not migrate or remove standalone guidance. See + [integration setup](../integrations.md). +- The MCP bundle is an alternative launch for MCP only. It does not carry skills, + hooks or rules. +- This page records the current layout. Layout changes are separate work and must + update this matrix in the same change. + +## Model-policy ownership + +The user's own provider configuration and project policy own these settings. Hack +artifacts must not set them: + +- model selection, model overrides and reasoning effort +- permission modes, tool allowlists, approval policies and sandbox modes +- output styles + +Hack-shipped hooks may run only `hack agent prime`. +`tests/agent-plugin-generation.test.ts` enforces this for every file under +`plugins/hack` and `examples/basic`. It includes a negative control that proves the +check rejects each forbidden setting. See [maintaining agent guidance](../agent-guidance.md). diff --git a/src/agents/client-capabilities.ts b/src/agents/client-capabilities.ts new file mode 100644 index 00000000..288b8da8 --- /dev/null +++ b/src/agents/client-capabilities.ts @@ -0,0 +1,517 @@ +/** + * Versioned agent-client capability matrix for Hack's optional agent + * integrations. This module is data only. + * + * Invariants, enforced by tests/agent-client-capabilities.test.ts: + * - Every client x capability pair has exactly one cell. + * - A `verified` cell names an existing test file whose evidence class is + * `isolated-fixture` or stronger. A manifest parse, schema check or + * marketplace listing is never component-loading evidence. + * - For the named native-plugin clients, a component is `unsupported` exactly + * when that client's plugin manifest does not declare it. + * - docs/reference/agent-client-capabilities.md matches this data. + * + * Each cell records the client version, host label and source revision at + * which it was classified. Those values describe one observation on one host; + * they are not a cross-machine or long-term compatibility claim, and a status + * for one client implies nothing about another client's UI or loading behavior. + */ + +/** Increment when cells are reclassified or the cell shape changes. */ +export const AGENT_CLIENT_CAPABILITY_MATRIX_VERSION = 1; + +/** Date (UTC) of the observations recorded below. */ +export const AGENT_CLIENT_CAPABILITY_OBSERVED_ON = "2026-10-11"; + +export const AGENT_CLIENTS = [ + "codex", + "claude", + "cursor", + "standalone", +] as const; +export type AgentClient = (typeof AGENT_CLIENTS)[number]; + +export const AGENT_CLIENT_CAPABILITIES = [ + "packaging", + "skillDiscovery", + "hooks", + "pluginMcp", + "standaloneMcp", + "rules", + "ui", +] as const; +export type AgentClientCapability = (typeof AGENT_CLIENT_CAPABILITIES)[number]; + +export type CapabilityStatus = "verified" | "untested" | "unsupported"; + +/** + * Evidence classes, weakest first. `source-read` and `schema` describe files + * Hack ships; only `isolated-fixture` and `authenticated-session` observe a + * client loading a component. + */ +export const EVIDENCE_CLASSES = [ + "source-read", + "schema", + "isolated-fixture", + "authenticated-session", +] as const; +export type EvidenceClass = (typeof EVIDENCE_CLASSES)[number]; + +export interface AgentClientDefinition { + readonly client: AgentClient; + readonly label: string; + /** Repository-relative plugin manifest Hack ships for this client. */ + readonly manifestPath: string; +} + +export interface AgentClientCapabilityDefinition { + readonly capability: AgentClientCapability; + readonly label: string; + readonly meaning: string; +} + +export interface ClientCapabilityCell { + readonly client: AgentClient; + readonly capability: AgentClientCapability; + readonly status: CapabilityStatus; + readonly evidenceClass: EvidenceClass; + /** Repository-relative file that carries the evidence. */ + readonly evidenceRef: string; + /** Installed client version observed during classification, if any. */ + readonly observedClientVersion: string | null; + /** Commit of the Hack source that was classified. */ + readonly sourceSha: string; + /** Non-identifying label of the host that produced the observation. */ + readonly hostLabel: string; + readonly note: string; + /** Proof still required before the cell can be `verified`; null if none. */ + readonly openGate: string | null; +} + +export const AGENT_CLIENT_DEFINITIONS: readonly AgentClientDefinition[] = [ + { + client: "codex", + label: "Codex", + manifestPath: "plugins/hack/.codex-plugin/plugin.json", + }, + { + client: "claude", + label: "Claude Code", + manifestPath: "plugins/hack/.claude-plugin/plugin.json", + }, + { + client: "cursor", + label: "Cursor", + manifestPath: "plugins/hack/.cursor-plugin/plugin.json", + }, + { + client: "standalone", + label: "Standalone (portable manifest or generic stdio MCP client)", + manifestPath: "plugins/hack/plugin.json", + }, +]; + +export const AGENT_CLIENT_CAPABILITY_DEFINITIONS: readonly AgentClientCapabilityDefinition[] = + [ + { + capability: "packaging", + label: "Packaging", + meaning: + "The client accepts, installs and enables the Hack plugin package in a session.", + }, + { + capability: "skillDiscovery", + label: "Skill discovery", + meaning: + "A fresh session discovers the plugin's bundled skills (hack-cli, hack-init, hack-install, hack-remote).", + }, + { + capability: "hooks", + label: "Hooks", + meaning: + "The client runs plugin-shipped hooks; Hack hooks may only run `hack agent prime`.", + }, + { + capability: "pluginMcp", + label: "Plugin MCP", + meaning: + "The client starts the plugin-declared `hack mcp serve` server and lists its tools.", + }, + { + capability: "standaloneMcp", + label: "Standalone MCP", + meaning: + "The client loads an MCP entry written by `hack mcp install` (stdio or a selected MCP bundle) without the plugin.", + }, + { + capability: "rules", + label: "Rules", + meaning: "The client loads a plugin-shipped rule file.", + }, + { + capability: "ui", + label: "UI", + meaning: + "The client renders plugin presentation metadata (display name, default prompts, brand color).", + }, + ]; + +const SOURCE_SHA = "ec56769bdc046f5b64e621eb04613cfb00a65ced"; +const HOST = "macos-arm64-contributor"; +const CODEX_VERSION = "codex-cli 0.162.1"; +const CLAUDE_VERSION = "Claude Code 2.1.296"; +const CURSOR_VERSION = "Cursor 3.24.12"; +const BUNDLE_FIXTURE = "tests/mcp-bundle-client.test.ts"; + +const PLUGIN_LOAD_GATE = + "Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count."; + +export const AGENT_CLIENT_CAPABILITY_MATRIX: readonly ClientCapabilityCell[] = [ + // Codex + { + client: "codex", + capability: "packaging", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.codex-plugin/plugin.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "codex", + capability: "skillDiscovery", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.codex-plugin/plugin.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares `skills: ./skills/`; skill content matches the canonical renderers.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "codex", + capability: "hooks", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.codex-plugin/plugin.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Hack ships no Codex plugin hooks; the manifest declares no `hooks` key. This says nothing about Codex's own hook support.", + openGate: null, + }, + { + client: "codex", + capability: "pluginMcp", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.mcp.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "codex", + capability: "standaloneMcp", + status: "untested", + evidenceClass: "isolated-fixture", + evidenceRef: BUNDLE_FIXTURE, + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "In the opt-in fixture, `codex mcp get hack --json` resolved the generated bundle adapter from an isolated CODEX_HOME. The fixture then stopped at the Claude step, so no tool call was observed in that run. Configuration discovery alone does not prove a Codex model session loaded the server or accepted project trust.", + openGate: + "Pass tests/mcp-bundle-client.test.ts end to end at this client version; a Codex model session tool call remains a separate authenticated-session gate.", + }, + { + client: "codex", + capability: "rules", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.codex-plugin/plugin.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Hack ships no Codex plugin rule file. Project AGENTS.md snippets are standalone setup, not a plugin component.", + openGate: null, + }, + { + client: "codex", + capability: "ui", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.codex-plugin/plugin.json", + observedClientVersion: CODEX_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares an `interface` block (display name, default prompts, brand color); rendering was not observed.", + openGate: + "Observe the installed plugin's presentation in the Codex UI for this client version.", + }, + // Claude Code + { + client: "claude", + capability: "packaging", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.claude-plugin/plugin.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "claude", + capability: "skillDiscovery", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.claude-plugin/plugin.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares `skills: ./skills/`; skill content matches the canonical renderers.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "claude", + capability: "hooks", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/hooks/claude-hooks.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "SessionStart and PreCompact hooks run only `hack agent prime`; the generation test enforces that command allowlist.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "claude", + capability: "pluginMcp", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.mcp.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "claude", + capability: "standaloneMcp", + status: "untested", + evidenceClass: "isolated-fixture", + evidenceRef: BUNDLE_FIXTURE, + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "The opt-in fixture failed at this version: in an isolated HOME, `claude mcp get hack` reports the project-scoped server as pending approval instead of connected, so no connection or tool call was observed. Project approval is a native trust gate and must not be bypassed.", + openGate: + "Qualify Claude standalone MCP without bypassing project approval (for example, user scope in an isolated HOME) and pass the fixture end to end.", + }, + { + client: "claude", + capability: "rules", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.claude-plugin/plugin.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Hack ships no Claude plugin rule file. Project CLAUDE.md snippets are standalone setup, not a plugin component.", + openGate: null, + }, + { + client: "claude", + capability: "ui", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.claude-plugin/plugin.json", + observedClientVersion: CLAUDE_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares no presentation metadata. Codex UI metadata implies nothing for Claude Code.", + openGate: null, + }, + // Cursor + { + client: "cursor", + capability: "packaging", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.cursor-plugin/plugin.json", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest is generated and parsed by tests/agent-plugin-generation.test.ts. docs/integrations.md records an earlier manual symlink-rejection observation (Cursor 3.18.25), not a fixture.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "cursor", + capability: "skillDiscovery", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.cursor-plugin/plugin.json", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares `skills: ./skills/`; skill content matches the canonical renderers.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "cursor", + capability: "hooks", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.cursor-plugin/plugin.json", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Hack ships no Cursor plugin hooks; the manifest declares no `hooks` key.", + openGate: null, + }, + { + client: "cursor", + capability: "pluginMcp", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/.mcp.json", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "cursor", + capability: "standaloneMcp", + status: "untested", + evidenceClass: "schema", + evidenceRef: "tests/mcp-install.test.ts", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Tests check the `.cursor/mcp.json` entry Hack writes; Cursor itself never loaded it in a fixture.", + openGate: + "Add an isolated Cursor fixture that loads the written entry and lists Hack tools.", + }, + { + client: "cursor", + capability: "rules", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/rules/hack.mdc", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Agent-selectable rule (`alwaysApply: false`) matches the canonical renderer; loading was not observed.", + openGate: PLUGIN_LOAD_GATE, + }, + { + client: "cursor", + capability: "ui", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/.cursor-plugin/plugin.json", + observedClientVersion: CURSOR_VERSION, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Manifest declares no presentation metadata. Codex UI metadata implies nothing for Cursor.", + openGate: null, + }, + // Standalone: no named client; the portable manifest or a generic stdio MCP client. + { + client: "standalone", + capability: "packaging", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/plugin.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Portable agent-plugins manifest is derived from the Codex manifest; no portable consumer was observed.", + openGate: "Name a portable consumer and observe it install the package.", + }, + { + client: "standalone", + capability: "skillDiscovery", + status: "untested", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/plugin.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "The portable manifest declares no component paths; discovery of `skills/` by convention is unverified.", + openGate: + "Name a portable consumer and observe skill discovery in a fresh session.", + }, + { + client: "standalone", + capability: "hooks", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/plugin.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "The portable manifest carries hooks only through the OpenAI extension, and Hack ships none there.", + openGate: null, + }, + { + client: "standalone", + capability: "pluginMcp", + status: "untested", + evidenceClass: "schema", + evidenceRef: "plugins/hack/mcp.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Portable MCP manifest adds an explicit stdio transport to the same launch as `.mcp.json`.", + openGate: + "Name a portable consumer and observe it start the declared server.", + }, + { + client: "standalone", + capability: "standaloneMcp", + status: "untested", + evidenceClass: "isolated-fixture", + evidenceRef: BUNDLE_FIXTURE, + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "The fixture's reference MCP SDK client calls a real tool through the generated entry, but that step was not reached in the recorded run. The SDK client is not an agent.", + openGate: + "Pass tests/mcp-bundle-client.test.ts end to end, or add a client-independent stdio fixture.", + }, + { + client: "standalone", + capability: "rules", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/plugin.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "No portable rule component is shipped.", + openGate: null, + }, + { + client: "standalone", + capability: "ui", + status: "unsupported", + evidenceClass: "source-read", + evidenceRef: "plugins/hack/plugin.json", + observedClientVersion: null, + sourceSha: SOURCE_SHA, + hostLabel: HOST, + note: "Presentation metadata exists only in the OpenAI extension; no portable UI is claimed.", + openGate: null, + }, +]; diff --git a/tests/agent-client-capabilities.test.ts b/tests/agent-client-capabilities.test.ts new file mode 100644 index 00000000..4c5691c3 --- /dev/null +++ b/tests/agent-client-capabilities.test.ts @@ -0,0 +1,465 @@ +import { expect, test } from "bun:test"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; + +import { + AGENT_CLIENT_CAPABILITIES, + AGENT_CLIENT_CAPABILITY_DEFINITIONS, + AGENT_CLIENT_CAPABILITY_MATRIX, + AGENT_CLIENT_CAPABILITY_MATRIX_VERSION, + AGENT_CLIENT_CAPABILITY_OBSERVED_ON, + AGENT_CLIENT_DEFINITIONS, + AGENT_CLIENTS, + type AgentClient, + type AgentClientCapability, + type ClientCapabilityCell, + EVIDENCE_CLASSES, + type EvidenceClass, +} from "../src/agents/client-capabilities.ts"; +import { isRecord } from "../src/lib/guards.ts"; + +const repoRoot = join(import.meta.dir, ".."); +const referencePath = join( + repoRoot, + "docs", + "reference", + "agent-client-capabilities.md" +); + +type IssueCode = + | "missing-cell" + | "duplicate-cell" + | "missing-evidence-ref" + | "invalid-source-sha" + | "missing-host-label" + | "missing-client-version" + | "open-gate-missing" + | "verified-without-test" + | "verified-below-fixture" + | "verified-on-manifest-or-listing" + | "verified-with-open-gate" + | "manifest-disagreement"; + +interface ClassificationIssue { + readonly client: AgentClient; + readonly capability: AgentClientCapability; + readonly code: IssueCode; +} + +/** Manifest key that declares each plugin component for the named clients. */ +const MANIFEST_COMPONENT_KEYS: Partial> = + { + skillDiscovery: "skills", + hooks: "hooks", + pluginMcp: "mcpServers", + rules: "rules", + ui: "interface", + }; + +const NATIVE_PLUGIN_CLIENTS: readonly AgentClient[] = [ + "codex", + "claude", + "cursor", +]; + +function evidenceStrength(evidenceClass: EvidenceClass): number { + return EVIDENCE_CLASSES.indexOf(evidenceClass); +} + +/** Manifests, shipped component files and marketplace listings describe a + * package; they never show that a client loaded a component. */ +function isManifestOrListingRef(ref: string): boolean { + const name = ref.split("/").at(-1) ?? ref; + return ( + ref.startsWith("plugins/") || + ref.includes("marketplace") || + ["plugin.json", "mcp.json", ".mcp.json", "marketplace.json"].includes( + name + ) || + name.endsWith(".mdc") || + name === "SKILL.md" + ); +} + +function classificationIssues({ + cells, + manifests, + exists, +}: { + readonly cells: readonly ClientCapabilityCell[]; + readonly manifests: Readonly>>; + readonly exists: (repoRelativePath: string) => boolean; +}): ClassificationIssue[] { + const issues: ClassificationIssue[] = []; + for (const client of AGENT_CLIENTS) { + for (const capability of AGENT_CLIENT_CAPABILITIES) { + const matches = cells.filter( + (cell) => cell.client === client && cell.capability === capability + ); + if (matches.length === 0) { + issues.push({ client, capability, code: "missing-cell" }); + } else if (matches.length > 1) { + issues.push({ client, capability, code: "duplicate-cell" }); + } + } + } + for (const cell of cells) { + const report = (code: IssueCode) => + issues.push({ client: cell.client, capability: cell.capability, code }); + if (!(cell.evidenceRef && exists(cell.evidenceRef))) { + report("missing-evidence-ref"); + } + if (!/^[0-9a-f]{40}$/.test(cell.sourceSha)) { + report("invalid-source-sha"); + } + if (cell.hostLabel.trim() === "") { + report("missing-host-label"); + } + if (cell.client !== "standalone" && !cell.observedClientVersion) { + report("missing-client-version"); + } + if (cell.status === "untested" && !cell.openGate?.trim()) { + report("open-gate-missing"); + } + if (cell.status === "verified") { + if ( + !( + /^tests\/.+\.test\.ts$/.test(cell.evidenceRef) && + exists(cell.evidenceRef) + ) + ) { + report("verified-without-test"); + } + if ( + evidenceStrength(cell.evidenceClass) < + evidenceStrength("isolated-fixture") + ) { + report("verified-below-fixture"); + } + if (isManifestOrListingRef(cell.evidenceRef)) { + report("verified-on-manifest-or-listing"); + } + if (cell.openGate !== null) { + report("verified-with-open-gate"); + } + if (!cell.observedClientVersion) { + report("missing-client-version"); + } + } + if (NATIVE_PLUGIN_CLIENTS.includes(cell.client)) { + const manifest = manifests[cell.client] ?? {}; + const key = MANIFEST_COMPONENT_KEYS[cell.capability]; + const declared = + cell.capability === "packaging" + ? Object.keys(manifest).length > 0 + : key !== undefined && key in manifest; + if ( + (key !== undefined || cell.capability === "packaging") && + (cell.status === "unsupported") === declared + ) { + report("manifest-disagreement"); + } + } + } + return issues; +} + +async function loadManifests(): Promise< + Record> +> { + const manifests: Record> = {}; + for (const definition of AGENT_CLIENT_DEFINITIONS) { + const parsed: unknown = await Bun.file( + join(repoRoot, definition.manifestPath) + ).json(); + if (!isRecord(parsed)) { + throw new Error(`Expected a manifest object: ${definition.manifestPath}`); + } + manifests[definition.client] = parsed; + } + return manifests; +} + +const repoExists = (path: string) => existsSync(join(repoRoot, path)); + +function replaceCell( + client: AgentClient, + capability: AgentClientCapability, + change: Partial +): ClientCapabilityCell[] { + return AGENT_CLIENT_CAPABILITY_MATRIX.map((cell) => + cell.client === client && cell.capability === capability + ? { ...cell, ...change } + : cell + ); +} + +function codesFor( + issues: readonly ClassificationIssue[], + client: AgentClient, + capability: AgentClientCapability +): IssueCode[] { + return issues + .filter( + (issue) => issue.client === client && issue.capability === capability + ) + .map((issue) => issue.code); +} + +function escapeCell(value: string): string { + return value.replaceAll("|", "\\|").replaceAll("\n", " "); +} + +function clientLabel(client: AgentClient): string { + return ( + AGENT_CLIENT_DEFINITIONS.find((definition) => definition.client === client) + ?.label ?? client + ); +} + +function capabilityLabel(capability: AgentClientCapability): string { + return ( + AGENT_CLIENT_CAPABILITY_DEFINITIONS.find( + (definition) => definition.capability === capability + )?.label ?? capability + ); +} + +const GENERATED_BEGIN = ""; +const GENERATED_END = ""; + +function renderGeneratedSection(): string { + const lines = [ + GENERATED_BEGIN, + "", + `Matrix version ${AGENT_CLIENT_CAPABILITY_MATRIX_VERSION}, observed ${AGENT_CLIENT_CAPABILITY_OBSERVED_ON}.`, + "", + "### Clients", + "", + "| Client | Manifest |", + "| --- | --- |", + ...AGENT_CLIENT_DEFINITIONS.map( + (definition) => + `| ${escapeCell(definition.label)} | \`${definition.manifestPath}\` |` + ), + "", + "### Capabilities", + "", + "| Capability | Meaning |", + "| --- | --- |", + ...AGENT_CLIENT_CAPABILITY_DEFINITIONS.map( + (definition) => + `| ${definition.label} | ${escapeCell(definition.meaning)} |` + ), + "", + "### Status summary", + "", + `| Capability | ${AGENT_CLIENTS.map(clientLabel).map(escapeCell).join(" | ")} |`, + `| --- | ${AGENT_CLIENTS.map(() => "---").join(" | ")} |`, + ...AGENT_CLIENT_CAPABILITIES.map((capability) => { + const statuses = AGENT_CLIENTS.map( + (client) => + AGENT_CLIENT_CAPABILITY_MATRIX.find( + (cell) => cell.client === client && cell.capability === capability + )?.status ?? "missing" + ); + return `| ${capabilityLabel(capability)} | ${statuses.join(" | ")} |`; + }), + "", + "### Cells", + "", + "| Client | Capability | Status | Evidence class | Evidence | Client version | Host | Source | Note | Open gate |", + "| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |", + ...AGENT_CLIENT_CAPABILITY_MATRIX.map((cell) => + [ + "", + cell.client, + capabilityLabel(cell.capability), + cell.status, + cell.evidenceClass, + `\`${cell.evidenceRef}\``, + escapeCell(cell.observedClientVersion ?? "none"), + `\`${cell.hostLabel}\``, + `\`${cell.sourceSha.slice(0, 12)}\``, + escapeCell(cell.note), + escapeCell(cell.openGate ?? "none"), + "", + ] + .join(" | ") + .trim() + ), + "", + GENERATED_END, + ]; + return lines.join("\n"); +} + +test("every client and capability pair is classified with complete evidence", async () => { + expect(AGENT_CLIENT_CAPABILITY_MATRIX).toHaveLength( + AGENT_CLIENTS.length * AGENT_CLIENT_CAPABILITIES.length + ); + const issues = classificationIssues({ + cells: AGENT_CLIENT_CAPABILITY_MATRIX, + manifests: await loadManifests(), + exists: repoExists, + }); + expect(issues).toEqual([]); +}); + +test("verified cells require an existing fixture-strength test file", async () => { + const manifests = await loadManifests(); + const accepted = replaceCell("codex", "standaloneMcp", { + status: "verified", + evidenceClass: "isolated-fixture", + evidenceRef: "tests/mcp-bundle-client.test.ts", + openGate: null, + }); + expect( + classificationIssues({ cells: accepted, manifests, exists: repoExists }) + ).toEqual([]); + + const missingTest = replaceCell("codex", "standaloneMcp", { + status: "verified", + evidenceClass: "isolated-fixture", + evidenceRef: "tests/does-not-exist.test.ts", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ + cells: missingTest, + manifests, + exists: repoExists, + }), + "codex", + "standaloneMcp" + ) + ).toEqual(["missing-evidence-ref", "verified-without-test"]); + + const weakClass = replaceCell("codex", "standaloneMcp", { + status: "verified", + evidenceClass: "schema", + evidenceRef: "tests/mcp-install.test.ts", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ cells: weakClass, manifests, exists: repoExists }), + "codex", + "standaloneMcp" + ) + ).toEqual(["verified-below-fixture"]); +}); + +test("negative control: plugin skills verified on manifest-only evidence fail", async () => { + const manifests = await loadManifests(); + const cells = replaceCell("claude", "skillDiscovery", { + status: "verified", + evidenceClass: "isolated-fixture", + evidenceRef: "plugins/hack/.claude-plugin/plugin.json", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ cells, manifests, exists: repoExists }), + "claude", + "skillDiscovery" + ) + ).toEqual(["verified-without-test", "verified-on-manifest-or-listing"]); + + const listing = replaceCell("cursor", "packaging", { + status: "verified", + evidenceClass: "authenticated-session", + evidenceRef: ".claude-plugin/marketplace.json", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ + cells: listing, + manifests, + exists: () => true, + }), + "cursor", + "packaging" + ) + ).toEqual(["verified-without-test", "verified-on-manifest-or-listing"]); +}); + +test("manifest-declared components agree with the matrix", async () => { + const manifests = await loadManifests(); + expect("hooks" in (manifests.codex ?? {})).toBe(false); + const codexHooks = replaceCell("codex", "hooks", { + status: "verified", + evidenceClass: "isolated-fixture", + evidenceRef: "tests/mcp-bundle-client.test.ts", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ + cells: codexHooks, + manifests, + exists: repoExists, + }), + "codex", + "hooks" + ) + ).toEqual(["manifest-disagreement"]); + + const hiddenRules = replaceCell("cursor", "rules", { + status: "unsupported", + openGate: null, + }); + expect( + codesFor( + classificationIssues({ + cells: hiddenRules, + manifests, + exists: repoExists, + }), + "cursor", + "rules" + ) + ).toEqual(["manifest-disagreement"]); + + const missing = AGENT_CLIENT_CAPABILITY_MATRIX.filter( + (cell) => !(cell.client === "claude" && cell.capability === "ui") + ); + expect( + codesFor( + classificationIssues({ cells: missing, manifests, exists: repoExists }), + "claude", + "ui" + ) + ).toEqual(["missing-cell"]); + + const ungated = replaceCell("claude", "hooks", { openGate: null }); + expect( + codesFor( + classificationIssues({ cells: ungated, manifests, exists: repoExists }), + "claude", + "hooks" + ) + ).toEqual(["open-gate-missing"]); +}); + +test("reference doc matches the matrix and records ownership decisions", async () => { + const doc = await Bun.file(referencePath).text(); + const start = doc.indexOf(GENERATED_BEGIN); + const end = doc.indexOf(GENERATED_END); + expect(start).toBeGreaterThanOrEqual(0); + expect(end).toBeGreaterThan(start); + expect(doc.slice(start, end + GENERATED_END.length)).toBe( + renderGeneratedSection() + ); + expect(doc).toContain("## Companion-package layout decision"); + expect(doc).toContain("## Model-policy ownership"); + for (const owner of [ + "scripts/generate-agent-plugins.ts", + "src/agents/instruction-source.ts", + "src/mcp/bundle.ts", + ]) { + expect(doc).toContain(owner); + } +}); diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 858a3c90..38e4a30a 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -1,5 +1,5 @@ import { afterEach, expect, test } from "bun:test"; -import { cp, mkdir, mkdtemp, rm, symlink } from "node:fs/promises"; +import { cp, mkdir, mkdtemp, readdir, rm, symlink } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -13,6 +13,7 @@ import { renderCodexSkill } from "../src/agents/codex-skill.ts"; import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; +import { isRecord } from "../src/lib/guards.ts"; const repoRoot = join(import.meta.dir, ".."); @@ -217,3 +218,262 @@ test("release preparation updates both manifests and rendered guidance after bum expect(skill).toContain("hack CLI v9.8.7"); expect(skill).not.toContain("hack CLI v0.0.1"); }, 15_000); + +interface PolicyFinding { + readonly file: string; + readonly code: + | "model-setting" + | "permission-setting" + | "output-style-setting" + | "frontmatter-hooks" + | "hook-command"; +} + +const HOOK_COMMAND = "hack agent prime"; + +/** Model, permission and output-style policy belongs to the user's provider + * configuration, never to Hack-shipped artifacts. */ +function forbiddenSettingCode(key: string): PolicyFinding["code"] | null { + const normalized = key.toLowerCase().replace(/[^a-z]/g, ""); + if (normalized.startsWith("model") || normalized.endsWith("model")) { + return "model-setting"; + } + if ( + normalized.includes("permission") || + normalized.includes("approval") || + normalized.startsWith("sandbox") || + normalized === "allowedtools" || + normalized === "disallowedtools" || + normalized === "yolo" + ) { + return "permission-setting"; + } + if (normalized.includes("outputstyle")) { + return "output-style-setting"; + } + return null; +} + +/** Hook entries may only run the session primer as a plain command. */ +function hasDisallowedHook(value: unknown): boolean { + if (Array.isArray(value)) { + return value.some(hasDisallowedHook); + } + if (!isRecord(value)) { + return false; + } + if ( + ("type" in value || "command" in value) && + !(value.type === "command" && value.command === HOOK_COMMAND) + ) { + return true; + } + return Object.values(value).some(hasDisallowedHook); +} + +function structuredPolicyFindings( + file: string, + value: unknown +): PolicyFinding[] { + if (Array.isArray(value)) { + return value.flatMap((item) => structuredPolicyFindings(file, item)); + } + if (!isRecord(value)) { + return []; + } + const findings: PolicyFinding[] = []; + for (const [key, nested] of Object.entries(value)) { + const code = forbiddenSettingCode(key); + if (code) { + findings.push({ file, code }); + } + if (key === "hooks" && hasDisallowedHook(nested)) { + findings.push({ file, code: "hook-command" }); + } + findings.push(...structuredPolicyFindings(file, nested)); + } + return findings; +} + +function frontmatterKeys(text: string): string[] { + const match = /^---\n([\s\S]*?)\n---/.exec(text); + if (!match?.[1]) { + return []; + } + return match[1] + .split("\n") + .map((line) => /^([A-Za-z0-9_-]+):/.exec(line)?.[1]) + .filter((key): key is string => key !== undefined); +} + +/** Classify one shipped file with closed codes; never report file content. */ +function policyFindings(file: string, text: string): PolicyFinding[] { + const findings: PolicyFinding[] = []; + const name = file.split("/").at(-1) ?? file; + if (file.split("/").some((part) => /^output[-_]?styles?$/i.test(part))) { + findings.push({ file, code: "output-style-setting" }); + } + if (name.endsWith(".json")) { + findings.push(...structuredPolicyFindings(file, JSON.parse(text))); + } else if (name.endsWith(".jsonl")) { + for (const line of text.split("\n").filter((entry) => entry.trim())) { + findings.push(...structuredPolicyFindings(file, JSON.parse(line))); + } + } else if (name.endsWith(".toml")) { + findings.push(...structuredPolicyFindings(file, Bun.TOML.parse(text))); + } else if (name.endsWith(".md") || name.endsWith(".mdc")) { + for (const key of frontmatterKeys(text)) { + const code = forbiddenSettingCode(key); + if (code) { + findings.push({ file, code }); + } + if (key.toLowerCase() === "hooks") { + findings.push({ file, code: "frontmatter-hooks" }); + } + } + } + return findings; +} + +test("plugin bundle and examples never pin models, override permissions or run other hooks", async () => { + const findings: PolicyFinding[] = []; + let scanned = 0; + for (const root of ["plugins/hack", "examples/basic"]) { + const glob = new Bun.Glob("**/*"); + for await (const relative of glob.scan({ + cwd: join(repoRoot, root), + dot: true, + onlyFiles: true, + })) { + const file = `${root}/${relative}`; + findings.push( + ...policyFindings(file, await Bun.file(join(repoRoot, file)).text()) + ); + scanned += 1; + } + } + expect(scanned).toBeGreaterThan(10); + expect(findings).toEqual([]); + expect( + await Bun.file( + join(repoRoot, "plugins/hack/hooks/claude-hooks.json") + ).text() + ).toContain(HOOK_COMMAND); +}); + +test("negative control: the policy scan rejects each forbidden setting", () => { + const cases: readonly (readonly [string, string, PolicyFinding["code"]])[] = [ + ["plugin.json", JSON.stringify({ model: "opus" }), "model-setting"], + [ + "settings.json", + JSON.stringify({ env: { x: "1" }, modelOverride: "x" }), + "model-setting", + ], + ["config.toml", 'model = "gpt-5"\n', "model-setting"], + ["config.toml", 'model_reasoning_effort = "high"\n', "model-setting"], + ["config.toml", 'approval_policy = "never"\n', "permission-setting"], + [ + "config.toml", + 'sandbox_mode = "danger-full-access"\n', + "permission-setting", + ], + [ + "settings.json", + JSON.stringify({ permissions: { defaultMode: "bypassPermissions" } }), + "permission-setting", + ], + [ + "settings.json", + JSON.stringify({ outputStyle: "Explanatory" }), + "output-style-setting", + ], + ["output-styles/hack.md", "# Style\n", "output-style-setting"], + ["skills/x/SKILL.md", "---\nname: x\nmodel: opus\n---\n", "model-setting"], + [ + "skills/x/SKILL.md", + "---\nname: x\nallowed-tools: Bash\n---\n", + "permission-setting", + ], + [ + "skills/x/SKILL.md", + "---\nname: x\nhooks:\n Stop: []\n---\n", + "frontmatter-hooks", + ], + [ + "hooks/claude-hooks.json", + JSON.stringify({ + hooks: { + SessionStart: [ + { + hooks: [ + { type: "command", command: `${HOOK_COMMAND}; curl example` }, + ], + }, + ], + }, + }), + "hook-command", + ], + [ + "hooks/claude-hooks.json", + JSON.stringify({ + hooks: { Stop: [{ hooks: [{ type: "prompt", prompt: "x" }] }] }, + }), + "hook-command", + ], + ]; + for (const [file, text, code] of cases) { + expect(policyFindings(file, text).map((finding) => finding.code)).toContain( + code + ); + } + expect( + policyFindings( + "hooks/claude-hooks.json", + JSON.stringify({ + hooks: { + SessionStart: [ + { + matcher: "", + hooks: [{ type: "command", command: HOOK_COMMAND }], + }, + ], + }, + }) + ) + ).toEqual([]); +}); + +test("shipped skill inventory matches the generator and the integration doc", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-agent-skills-")); + const pluginRoot = join(tempDir, "plugins", "hack"); + for (const directory of [ + ".codex-plugin", + ".claude-plugin", + ".cursor-plugin", + ]) { + await mkdir(join(pluginRoot, directory), { recursive: true }); + await cp( + join(repoRoot, "plugins/hack", directory, "plugin.json"), + join(pluginRoot, directory, "plugin.json") + ); + } + await cp( + join(repoRoot, "plugins/hack/.mcp.json"), + join(pluginRoot, ".mcp.json") + ); + await generateAgentPlugins({ repoRoot: tempDir, version: "0.0.0" }); + const skillNames = async (root: string) => + (await readdir(join(root, "skills"), { withFileTypes: true })) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort(); + const generated = await skillNames(pluginRoot); + expect(generated.length).toBeGreaterThan(0); + expect(await skillNames(join(repoRoot, "plugins/hack"))).toEqual(generated); + const doc = await Bun.file(join(repoRoot, "docs/integrations.md")).text(); + for (const name of generated) { + expect(doc).toContain(`\`${name}\``); + } + expect(doc).not.toContain("both skills"); +}, 15_000); From 58c485a92a06e237f0cc416332fbfae3dcb48a78 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:32:18 -0400 Subject: [PATCH 02/52] fix(mcp): harden run argv and tool input contract hack.project.run built `run `, so hack's own parser consumed child flags: `node --version` printed hack's version and `--branch other` retargeted the instance. Run now emits `run -- ` and every option value is fused as `--opt=value` across all tools. - Validate service, services, branch, profiles, logs service and open target against a compose-identifier pattern (no leading '-', no empty value; branches may also contain '/'). - tail, maxEvents, maxMs and timeoutMs are positive integers with hard ceilings (provisional F27 settings); effective limits are echoed in data.limits. - Every tool carries annotations (readOnly for list/status/logs/open, destructive for down/run/init, openWorld false). - Refusals return error{code, message} with existing codes (E_USAGE, E_PROJECT_NOT_FOUND) and spawned:false. A typed tools/call dispatcher validates before any handler runs, so refused calls leave no argv entry and no audit line. - Children spawn with HACK_NO_INTERACTIVE=1 on a copy of the frozen session env. Refs HACK-1133, HACK-1134 Co-Authored-By: Claude Opus 5.5 --- src/mcp/server.ts | 834 ++++++++++++++++++++---------- tests/mcp-session-context.test.ts | 13 +- tests/mcp.test.ts | 483 ++++++++++++++++- tests/project-run-command.test.ts | 37 ++ 4 files changed, 1078 insertions(+), 289 deletions(-) diff --git a/src/mcp/server.ts b/src/mcp/server.ts index 779bd1b2..c4b9b86a 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -2,13 +2,23 @@ import { appendFile } from "node:fs/promises"; import { dirname, isAbsolute, relative, resolve } from "node:path"; import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; -import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; +import type { RequestHandlerExtra } from "@modelcontextprotocol/sdk/shared/protocol.js"; +import { + CallToolRequestSchema, + type CallToolResult, + ErrorCode, + McpError, + type ServerNotification, + type ServerRequest, + type ToolAnnotations, +} from "@modelcontextprotocol/sdk/types.js"; import * as z from "zod/v4"; import pkg from "../../package.json"; import { type OnboardingMode, renderOnboardingPrompt, } from "../agents/onboarding-prompt.ts"; +import type { HackErrorCode } from "../lib/cli-result.ts"; import { ensureDir, pathExists } from "../lib/fs.ts"; import { isRecord } from "../lib/guards.ts"; import { resolveHackInvocation } from "../lib/hack-cli.ts"; @@ -52,15 +62,75 @@ type HackCommandResult = { readonly stderr: string; }; +/** Existing CLI error codes reused for pre-spawn MCP refusals (no new codes). */ +type McpRefusalCode = Extract; + type ToolResultPayload = HackCommandResult & { readonly ok: boolean; readonly data?: unknown; + /** Present (and false) only on refusals returned before any child spawned. */ + readonly spawned?: false; + readonly error?: { + readonly code: McpRefusalCode; + readonly message: string; + }; }; +type ProjectArgsResolution = + | { readonly ok: true; readonly args: string[] } + | { + readonly ok: false; + readonly code: McpRefusalCode; + readonly message: string; + }; + const DEFAULT_CMD_TIMEOUT_MS = 120_000; const DEFAULT_LOG_TAIL_EVENTS = 200; const DEFAULT_LOG_TAIL_MS = 5000; +/** + * Hard ceilings for numeric MCP inputs. These are provisional F27 experiment + * settings: values are refused above the ceiling (never clamped), and the + * effective limits a call ran with are echoed in `data.limits`. + */ +export const MCP_INPUT_CEILINGS = Object.freeze({ + tail: 10_000, + maxEvents: 10_000, + maxMs: 60_000, + timeoutMs: 600_000, +}); + +/** + * Compose-style identifier: starts with an ASCII letter or digit, so a value + * can never be read as an option (`-x`) or be empty. Applied to service, + * services, profiles, logs service and open target. + */ +const COMPOSE_IDENTIFIER_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/; +/** Same rule for branch names, which may also carry `/` (the CLI slugs it). */ +const BRANCH_IDENTIFIER_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_./-]*$/; +const IDENTIFIER_MAX_LENGTH = 253; + +const READ_ONLY_TOOL_ANNOTATIONS = Object.freeze({ + readOnlyHint: true, + openWorldHint: false, +}) satisfies ToolAnnotations; + +/** Changes local runtime state without removing it (up, restart). */ +const ADDITIVE_TOOL_ANNOTATIONS = Object.freeze({ + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + openWorldHint: false, +}) satisfies ToolAnnotations; + +/** May stop, replace or overwrite local state (down, run, init). */ +const DESTRUCTIVE_TOOL_ANNOTATIONS = Object.freeze({ + readOnlyHint: false, + destructiveHint: true, + idempotentHint: false, + openWorldHint: false, +}) satisfies ToolAnnotations; + const toolOutputSchema = { ok: z.boolean(), outputTruncated: z.boolean().optional(), @@ -69,8 +139,50 @@ const toolOutputSchema = { stdout: z.string(), stderr: z.string(), data: z.unknown().optional(), + spawned: z.literal(false).optional(), + error: z + .object({ + code: z.enum(["E_USAGE", "E_PROJECT_NOT_FOUND"]), + message: z.string(), + }) + .optional(), } as const; +function composeIdentifierInput(description: string) { + return z + .string() + .max(IDENTIFIER_MAX_LENGTH) + .regex(COMPOSE_IDENTIFIER_PATTERN) + .describe(description); +} + +function branchIdentifierInput(description: string) { + return z + .string() + .max(IDENTIFIER_MAX_LENGTH) + .regex(BRANCH_IDENTIFIER_PATTERN) + .describe(description); +} + +function boundedLimitInput(opts: { + readonly ceiling: number; + readonly description: string; +}) { + return z + .number() + .int() + .positive() + .max(opts.ceiling) + .describe(`${opts.description} (1-${opts.ceiling})`); +} + +/** Child environment: the frozen session env plus forced non-interactivity. */ +function buildChildEnv( + env: Readonly> +): Record { + return { ...env, HACK_NO_INTERACTIVE: "1" }; +} + /** * Start the hack MCP server on stdio for local tool clients. */ @@ -193,6 +305,13 @@ function registerTools(opts: { readonly server: McpServer; readonly context: McpSessionContext; }): void { + const tools: HackToolRegistry = new Map(); + const register = ( + tool: HackToolDefinition + ): void => { + registerHackTool({ server: opts.server, tools, tool }); + }; + const projectSelectorInput = { projectName: z .string() @@ -209,39 +328,57 @@ function registerTools(opts: { }; const branchInput = { - branch: z.string().describe("Branch/worktree name").optional(), + branch: branchIdentifierInput("Branch/worktree name").optional(), }; const profileInput = { - profiles: z.array(z.string()).describe("Compose profile names").optional(), + profiles: z + .array(composeIdentifierInput("Compose profile name")) + .describe("Compose profile names") + .optional(), }; - opts.server.registerTool( - "hack.projects.list", - { - title: "List hack projects", - description: "List registered hack projects and runtime status.", - inputSchema: { - filter: z.string().describe("Filter by project name").optional(), - includeGlobal: z - .boolean() - .describe("Include global infra projects") - .optional(), - includeUnregistered: z - .boolean() - .describe("Include unregistered compose projects") - .optional(), - }, - outputSchema: toolOutputSchema, + const logSelectionInput = { + service: composeIdentifierInput("Single service name").optional(), + services: z + .array(composeIdentifierInput("Service name")) + .describe("Multiple service names") + .optional(), + query: z.string().describe("Raw LogQL selector/query").optional(), + loki: z.boolean().describe("Force Loki backend").optional(), + compose: z.boolean().describe("Force docker compose backend").optional(), + tail: boundedLimitInput({ + ceiling: MCP_INPUT_CEILINGS.tail, + description: "Tail last N log lines", + }).optional(), + since: z.string().describe("Start time (RFC3339 or duration)").optional(), + until: z.string().describe("End time (RFC3339 or duration)").optional(), + }; + + register({ + name: "hack.projects.list", + title: "List hack projects", + description: "List registered hack projects and runtime status.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + filter: z.string().describe("Filter by project name").optional(), + includeGlobal: z + .boolean() + .describe("Include global infra projects") + .optional(), + includeUnregistered: z + .boolean() + .describe("Include unregistered compose projects") + .optional(), }, - async ( + handler: async ( { filter, includeGlobal, includeUnregistered }, extra ): Promise => { const args = [ "projects", "--json", - ...(filter ? ["--project", filter] : []), + ...(filter ? [`--project=${filter}`] : []), ...(includeGlobal ? ["--include-global"] : []), ...(includeUnregistered ? ["--all"] : []), ]; @@ -257,29 +394,27 @@ function registerTools(opts: { result, data: parseJson(result.stdout), }); - } - ); - - opts.server.registerTool( - "hack.project.status", - { - title: "Project status", - description: "Get docker compose status for a project.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.status", + title: "Project status", + description: "Get docker compose status for a project.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -301,29 +436,27 @@ function registerTools(opts: { result, data: parseJson(result.stdout), }); - } - ); - - opts.server.registerTool( - "hack.project.up", - { - title: "Start project", - description: "Start project services in detached mode.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.up", + title: "Start project", + description: "Start project services in detached mode.", + annotations: ADDITIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -342,29 +475,27 @@ function registerTools(opts: { }); return buildToolResult({ result }); - } - ); - - opts.server.registerTool( - "hack.project.down", - { - title: "Stop project", - description: "Stop project services.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.down", + title: "Stop project", + description: "Stop project services.", + annotations: DESTRUCTIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -382,29 +513,27 @@ function registerTools(opts: { }); return buildToolResult({ result }); - } - ); - - opts.server.registerTool( - "hack.project.restart", - { - title: "Restart project", - description: "Restart project services.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.restart", + title: "Restart project", + description: "Restart project services.", + annotations: ADDITIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -422,97 +551,79 @@ function registerTools(opts: { }); return buildToolResult({ result }); - } - ); - - opts.server.registerTool( - "hack.project.run", - { - title: "Run command in service container", - description: "Run a one-off command inside a service container.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - service: z.string().describe("Compose service name"), - cmd: z.array(z.string()).describe("Command arguments").optional(), - workdir: z - .string() - .describe("Working directory inside container") - .optional(), - timeoutMs: z.number().describe("Command timeout in ms").optional(), - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.run", + title: "Run command in service container", + description: + "Run a one-off command inside a service container. Command arguments are passed to the child after `--` and are never parsed as hack options.", + annotations: DESTRUCTIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + service: composeIdentifierInput("Compose service name"), + cmd: z.array(z.string()).describe("Command arguments").optional(), + workdir: z + .string() + .describe("Working directory inside container") + .optional(), + timeoutMs: boundedLimitInput({ + ceiling: MCP_INPUT_CEILINGS.timeoutMs, + description: "Command timeout in ms", + }).optional(), + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } - const args = [ - "run", - ...resolved.args, - ...buildBranchArgs({ branch: input.branch }), - ...buildProfileArgs({ profiles: input.profiles }), - ...(input.workdir ? ["--workdir", input.workdir] : []), - input.service, - ...(input.cmd ?? []), - ]; - + const timeoutMs = input.timeoutMs ?? DEFAULT_CMD_TIMEOUT_MS; const result = await runHackCommand({ context: opts.context, signal: extra.signal, tool: "hack.project.run", - args, - timeoutMs: input.timeoutMs, + args: buildRunArgs({ + projectArgs: resolved.args, + branch: input.branch, + profiles: input.profiles, + workdir: input.workdir, + service: input.service, + cmd: input.cmd ?? [], + }), + timeoutMs, }); - return buildToolResult({ result }); - } - ); - - opts.server.registerTool( - "hack.project.logs.snapshot", - { - title: "Snapshot logs", - description: "Fetch a log snapshot and return structured log events.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - service: z.string().describe("Single service name").optional(), - services: z - .array(z.string()) - .describe("Multiple service names") - .optional(), - query: z.string().describe("Raw LogQL selector/query").optional(), - loki: z.boolean().describe("Force Loki backend").optional(), - compose: z - .boolean() - .describe("Force docker compose backend") - .optional(), - tail: z.number().describe("Tail last N log lines").optional(), - since: z - .string() - .describe("Start time (RFC3339 or duration)") - .optional(), - until: z.string().describe("End time (RFC3339 or duration)").optional(), - }, - outputSchema: toolOutputSchema, + return buildToolResult({ result, data: { limits: { timeoutMs } } }); }, - async (input, extra) => { + }); + + register({ + name: "hack.project.logs.snapshot", + title: "Snapshot logs", + description: "Fetch a log snapshot and return structured log events.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + ...logSelectionInput, + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -522,16 +633,7 @@ function registerTools(opts: { ...buildProfileArgs({ profiles: input.profiles }), "--json", "--no-follow", - ...(input.loki ? ["--loki"] : []), - ...(input.compose ? ["--compose"] : []), - ...(input.tail ? ["--tail", String(input.tail)] : []), - ...(input.query ? ["--query", input.query] : []), - ...(input.services - ? buildServicesArgs({ services: input.services }) - : []), - ...(input.since ? ["--since", input.since] : []), - ...(input.until ? ["--until", input.until] : []), - ...(input.service ? [input.service] : []), + ...buildLogSelectionArgs(input), ]; const result = await runHackCommand({ @@ -546,50 +648,39 @@ function registerTools(opts: { data: { events: parseJsonLines(result.stdout), count: countJsonLines(result.stdout), + limits: { tail: input.tail ?? null }, }, }); - } - ); - - opts.server.registerTool( - "hack.project.logs.tail", - { - title: "Tail logs", - description: "Stream logs briefly and return the collected events.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - ...profileInput, - service: z.string().describe("Single service name").optional(), - services: z - .array(z.string()) - .describe("Multiple service names") - .optional(), - query: z.string().describe("Raw LogQL selector/query").optional(), - loki: z.boolean().describe("Force Loki backend").optional(), - compose: z - .boolean() - .describe("Force docker compose backend") - .optional(), - tail: z.number().describe("Tail last N log lines").optional(), - since: z - .string() - .describe("Start time (RFC3339 or duration)") - .optional(), - until: z.string().describe("End time (RFC3339 or duration)").optional(), - maxEvents: z.number().describe("Max events to collect").optional(), - maxMs: z.number().describe("Max duration in ms").optional(), - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.logs.tail", + title: "Tail logs", + description: "Stream logs briefly and return the collected events.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + ...logSelectionInput, + maxEvents: boundedLimitInput({ + ceiling: MCP_INPUT_CEILINGS.maxEvents, + description: "Max events to collect", + }).optional(), + maxMs: boundedLimitInput({ + ceiling: MCP_INPUT_CEILINGS.maxMs, + description: "Max duration in ms", + }).optional(), + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -598,25 +689,18 @@ function registerTools(opts: { ...buildBranchArgs({ branch: input.branch }), ...buildProfileArgs({ profiles: input.profiles }), "--json", - ...(input.loki ? ["--loki"] : []), - ...(input.compose ? ["--compose"] : []), - ...(input.tail ? ["--tail", String(input.tail)] : []), - ...(input.query ? ["--query", input.query] : []), - ...(input.services - ? buildServicesArgs({ services: input.services }) - : []), - ...(input.since ? ["--since", input.since] : []), - ...(input.until ? ["--until", input.until] : []), - ...(input.service ? [input.service] : []), + ...buildLogSelectionArgs(input), ]; + const maxEvents = input.maxEvents ?? DEFAULT_LOG_TAIL_EVENTS; + const maxMs = input.maxMs ?? DEFAULT_LOG_TAIL_MS; const result = await runHackLogTail({ context: opts.context, signal: extra.signal, tool: "hack.project.logs.tail", args, - maxEvents: input.maxEvents, - maxMs: input.maxMs, + maxEvents, + maxMs, }); return buildToolResult({ @@ -632,31 +716,30 @@ function registerTools(opts: { count: result.events.length, stop_reason: result.stopReason, duration_ms: result.durationMs, + limits: { tail: input.tail ?? null, maxEvents, maxMs }, }, }); - } - ); - - opts.server.registerTool( - "hack.project.open", - { - title: "Resolve project URL", - description: "Return a URL for the project without launching a browser.", - inputSchema: { - ...projectSelectorInput, - ...branchInput, - target: z.string().describe("Target host/subdomain or URL").optional(), - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.open", + title: "Resolve project URL", + description: "Return a URL for the project without launching a browser.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + target: composeIdentifierInput("Target host or subdomain").optional(), + }, + handler: async (input, extra) => { const resolved = await resolveProjectArgs({ selection: toProjectSelection(input), cwd: opts.context.cwd, registryPath: opts.context.registryPath, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ @@ -678,35 +761,34 @@ function registerTools(opts: { result, data: parseJson(result.stdout), }); - } - ); - - opts.server.registerTool( - "hack.project.init", - { - title: "Initialize project", - description: "Initialize a repo using non-interactive defaults.", - inputSchema: { - ...projectSelectorInput, - name: z.string().describe("Project slug").optional(), - devHost: z.string().describe("DEV_HOST override").optional(), - oauth: z.boolean().describe("Enable OAuth alias host").optional(), - oauthTld: z.string().describe("OAuth alias TLD").optional(), - manual: z - .boolean() - .describe("Skip discovery and generate a minimal compose") - .optional(), - noDiscovery: z - .boolean() - .describe("Skip discovery even if available") - .optional(), - }, - outputSchema: toolOutputSchema, }, - async (input, extra) => { + }); + + register({ + name: "hack.project.init", + title: "Initialize project", + description: "Initialize a repo using non-interactive defaults.", + annotations: DESTRUCTIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + name: z.string().describe("Project slug").optional(), + devHost: z.string().describe("DEV_HOST override").optional(), + oauth: z.boolean().describe("Enable OAuth alias host").optional(), + oauthTld: z.string().describe("OAuth alias TLD").optional(), + manual: z + .boolean() + .describe("Skip discovery and generate a minimal compose") + .optional(), + noDiscovery: z + .boolean() + .describe("Skip discovery even if available") + .optional(), + }, + handler: async (input, extra) => { const selection = toProjectSelection(input); if (!(selection.path || selection.repoRoot)) { return buildToolError({ + code: "E_USAGE", message: "Provide repoRoot or path for initialization.", }); } @@ -718,17 +800,17 @@ function registerTools(opts: { allowUnregistered: true, }); if (!resolved.ok) { - return buildToolError({ message: resolved.message }); + return buildToolError(resolved); } const args = [ "init", ...resolved.args, "--auto", - ...(input.name ? ["--name", input.name] : []), - ...(input.devHost ? ["--dev-host", input.devHost] : []), + ...(input.name ? [`--name=${input.name}`] : []), + ...(input.devHost ? [`--dev-host=${input.devHost}`] : []), ...(input.oauth ? ["--oauth"] : []), - ...(input.oauthTld ? ["--oauth-tld", input.oauthTld] : []), + ...(input.oauthTld ? [`--oauth-tld=${input.oauthTld}`] : []), ...(input.manual ? ["--manual"] : []), ...(input.noDiscovery ? ["--no-discovery"] : []), ]; @@ -742,10 +824,137 @@ function registerTools(opts: { }); return buildToolResult({ result }); + }, + }); + + installTypedToolDispatch({ server: opts.server, tools }); +} + +type ToolExtra = RequestHandlerExtra; + +type HackToolDefinition = { + readonly name: string; + readonly title: string; + readonly description: string; + readonly annotations: ToolAnnotations; + readonly inputSchema: Shape; + readonly handler: ( + input: z.output>, + extra: ToolExtra + ) => Promise; +}; + +type HackToolEntry = { + /** Validates raw arguments and refuses before the handler can spawn. */ + readonly call: (args: unknown, extra: ToolExtra) => Promise; +}; + +type HackToolRegistry = Map; + +/** + * Register one tool for discovery (`tools/list`) and keep its validating entry + * point for {@link installTypedToolDispatch}. The schema advertised by the SDK + * and the schema enforced at call time are the same object. + */ +function registerHackTool(opts: { + readonly server: McpServer; + readonly tools: HackToolRegistry; + readonly tool: HackToolDefinition; +}): void { + const { tool } = opts; + const schema = z.object(tool.inputSchema); + const entry: HackToolEntry = { + call: async (args, extra) => { + const parsed = await schema.safeParseAsync(args ?? {}); + if (!parsed.success) { + return buildToolError({ + code: "E_USAGE", + message: describeInvalidArguments(parsed.error), + }); + } + return await tool.handler(parsed.data, extra); + }, + }; + opts.tools.set(tool.name, entry); + const listedSchema: z.ZodObject = schema; + opts.server.registerTool( + tool.name, + { + title: tool.title, + description: tool.description, + annotations: { title: tool.title, ...tool.annotations }, + inputSchema: listedSchema, + outputSchema: toolOutputSchema, + }, + (args: unknown, extra: ToolExtra) => entry.call(args, extra) + ); +} + +/** + * Replace the SDK's generic `tools/call` dispatcher. The SDK rejects schema + * failures with untyped text before a handler runs; this dispatcher keeps that + * ordering (validation precedes any spawn) but returns the typed + * `{error, spawned: false}` refusal. Thrown handler failures and output-schema + * mismatches keep the SDK's untyped `isError` text result, because they may + * occur after a child was spawned. + */ +function installTypedToolDispatch(opts: { + readonly server: McpServer; + readonly tools: HackToolRegistry; +}): void { + const outputSchema = z.object(toolOutputSchema); + opts.server.server.setRequestHandler( + CallToolRequestSchema, + async (request, extra): Promise => { + const entry = opts.tools.get(request.params.name); + if (!entry) { + throw new McpError( + ErrorCode.InvalidParams, + `Tool ${request.params.name} not found` + ); + } + try { + const result = await entry.call(request.params.arguments, extra); + if ( + !( + result.isError || + outputSchema.safeParse(result.structuredContent).success + ) + ) { + return buildUntypedToolError( + `Output validation error: Invalid structured content for tool ${request.params.name}` + ); + } + return result; + } catch (error) { + return buildUntypedToolError( + error instanceof Error ? error.message : String(error) + ); + } } ); } +/** Closed, value-free summary: names the rejected fields, never echoes input. */ +function describeInvalidArguments(error: z.ZodError): string { + const fields = [ + ...new Set( + error.issues + .map((issue) => issue.path[0]) + .filter((key): key is string => typeof key === "string") + ), + ]; + const subject = + fields.length > 0 + ? `Invalid arguments: ${fields.join(", ")}.` + : "Invalid arguments."; + return `${subject} Identifiers must match ${COMPOSE_IDENTIFIER_PATTERN.source} (branches may also contain "/"); limits must be positive integers within their documented ceiling.`; +} + +function buildUntypedToolError(message: string): CallToolResult { + return { content: [{ type: "text", text: message }], isError: true }; +} + function buildToolResult(opts: { readonly result: HackCommandResult; readonly data?: unknown; @@ -770,16 +979,24 @@ function buildToolResult(opts: { }; } +/** + * Typed refusal for a request rejected before any child process exists. + * `spawned: false` is a guarantee: every caller returns before + * `runHackCommand`/`runHackLogTail`, so no argv, audit line or mutation occurs. + * `stderr` repeats the message for clients reading the earlier contract. + */ function buildToolError(opts: { + readonly code: McpRefusalCode; readonly message: string; - readonly command?: string; }): CallToolResult { const payload: ToolResultPayload = { ok: false, - command: opts.command ?? "", + command: "", exitCode: 1, stdout: "", stderr: opts.message, + spawned: false, + error: { code: opts.code, message: opts.message }, }; return { @@ -789,29 +1006,74 @@ function buildToolError(opts: { }; } +/** + * `hack run` argv. Every option value is fused (`--opt=value`) and `--` ends + * hack's option parsing, so nothing in `cmd` (for example `--branch other` or + * `--version`) can be consumed by hack or retarget the instance. + */ +export function buildRunArgs(opts: { + readonly projectArgs: readonly string[]; + readonly branch?: string; + readonly profiles?: readonly string[]; + readonly workdir?: string; + readonly service: string; + readonly cmd: readonly string[]; +}): string[] { + return [ + "run", + ...opts.projectArgs, + ...buildBranchArgs({ branch: opts.branch }), + ...buildProfileArgs({ profiles: opts.profiles }), + ...(opts.workdir ? [`--workdir=${opts.workdir}`] : []), + "--", + opts.service, + ...opts.cmd, + ]; +} + +function buildLogSelectionArgs(input: { + readonly loki?: boolean; + readonly compose?: boolean; + readonly tail?: number; + readonly query?: string; + readonly services?: readonly string[]; + readonly since?: string; + readonly until?: string; + readonly service?: string; +}): string[] { + return [ + ...(input.loki ? ["--loki"] : []), + ...(input.compose ? ["--compose"] : []), + ...(input.tail === undefined ? [] : [`--tail=${input.tail}`]), + ...(input.query ? [`--query=${input.query}`] : []), + ...(input.services ? buildServicesArgs({ services: input.services }) : []), + ...(input.since ? [`--since=${input.since}`] : []), + ...(input.until ? [`--until=${input.until}`] : []), + ...(input.service ? [input.service] : []), + ]; +} + function buildBranchArgs(opts: { readonly branch?: string }): string[] { - const branch = (opts.branch ?? "").trim(); - return branch.length > 0 ? ["--branch", branch] : []; + return opts.branch ? [`--branch=${opts.branch}`] : []; } function buildProfileArgs(opts: { readonly profiles?: readonly string[]; }): string[] { - const profiles = (opts.profiles ?? []).map((p) => p.trim()).filter(Boolean); + const profiles = opts.profiles ?? []; if (profiles.length === 0) { return []; } - return ["--profile", profiles.join(",")]; + return [`--profile=${profiles.join(",")}`]; } function buildServicesArgs(opts: { readonly services: readonly string[]; }): string[] { - const services = opts.services.map((s) => s.trim()).filter(Boolean); - if (services.length === 0) { + if (opts.services.length === 0) { return []; } - return ["--services", services.join(",")]; + return [`--services=${opts.services.join(",")}`]; } function toProjectSelection(input: ProjectSelection): ProjectSelection { @@ -827,10 +1089,7 @@ async function resolveProjectArgs(opts: { readonly selection: ProjectSelection; readonly cwd: string; readonly allowUnregistered?: boolean; -}): Promise< - | { readonly ok: true; readonly args: string[] } - | { readonly ok: false; readonly message: string } -> { +}): Promise { const selection = opts.selection; const picked = [ selection.projectName, @@ -840,6 +1099,7 @@ async function resolveProjectArgs(opts: { if (picked.length > 1) { return { ok: false, + code: "E_USAGE", message: "Use only one of projectName, repoRoot, or path.", }; } @@ -855,7 +1115,11 @@ async function resolveProjectArgs(opts: { if (pathLike) { const absPath = resolve(opts.cwd, pathLike); if (!(await pathExists(absPath))) { - return { ok: false, message: `Path not found: ${absPath}` }; + return { + ok: false, + code: "E_PROJECT_NOT_FOUND", + message: `Path not found: ${absPath}`, + }; } if (!opts.allowUnregistered) { const allowed = await isPathAllowed({ @@ -865,11 +1129,12 @@ async function resolveProjectArgs(opts: { if (!allowed) { return { ok: false, + code: "E_PROJECT_NOT_FOUND", message: "Path is outside registered hack projects.", }; } } - return { ok: true, args: ["--path", absPath] }; + return { ok: true, args: [`--path=${absPath}`] }; } const context = await findProjectContext(opts.cwd); @@ -885,25 +1150,24 @@ async function resolveProjectArgs(opts: { if (!allowed) { return { ok: false, + code: "E_PROJECT_NOT_FOUND", message: "No project context found. Provide projectName or path.", }; } } - return { ok: true, args: ["--path", opts.cwd] }; + return { ok: true, args: [`--path=${opts.cwd}`] }; } async function resolveProjectNameArgs(opts: { readonly projectName: string; readonly registryPath: string; -}): Promise< - | { readonly ok: true; readonly args: string[] } - | { readonly ok: false; readonly message: string } -> { +}): Promise { const name = normalizeProjectName(opts.projectName); if (!name) { return { ok: false, + code: "E_USAGE", message: "Invalid projectName: use a name containing letters or digits.", }; } @@ -915,16 +1179,17 @@ async function resolveProjectNameArgs(opts: { if (!project) { return { ok: false, + code: "E_PROJECT_NOT_FOUND", message: `Unknown project "${opts.projectName}". Run 'hack init' or 'hack projects' first.`, }; } - return { ok: true, args: ["--project", name] }; + return { ok: true, args: [`--project=${name}`] }; } catch (error) { if ( error instanceof AmbiguousProjectNameError || error instanceof AmbiguousProjectIdError ) { - return { ok: false, message: error.message }; + return { ok: false, code: "E_USAGE", message: error.message }; } throw error; } @@ -979,7 +1244,7 @@ async function runHackCommand(opts: { const proc = Bun.spawn(cmd, { detached: true, cwd: opts.cwd ?? opts.context.cwd, - env: opts.context.env, + env: buildChildEnv(opts.context.env), stdin: "ignore", stdout: "pipe", stderr: "pipe", @@ -1038,8 +1303,8 @@ async function runHackLogTail(opts: { readonly signal: AbortSignal; readonly tool: string; readonly args: readonly string[]; - readonly maxEvents?: number; - readonly maxMs?: number; + readonly maxEvents: number; + readonly maxMs: number; }): Promise<{ readonly outputTruncated?: boolean; readonly command: string; @@ -1057,14 +1322,13 @@ async function runHackLogTail(opts: { const proc = Bun.spawn(cmd, { detached: true, cwd: opts.context.cwd, - env: opts.context.env, + env: buildChildEnv(opts.context.env), stdin: "ignore", stdout: "pipe", stderr: "pipe", }); - const maxEvents = opts.maxEvents ?? DEFAULT_LOG_TAIL_EVENTS; - const maxMs = opts.maxMs ?? DEFAULT_LOG_TAIL_MS; + const { maxEvents, maxMs } = opts; const start = Date.now(); const events: Record[] = []; diff --git a/tests/mcp-session-context.test.ts b/tests/mcp-session-context.test.ts index cd34d429..d9b47f70 100644 --- a/tests/mcp-session-context.test.ts +++ b/tests/mcp-session-context.test.ts @@ -58,7 +58,7 @@ async function fixture(marker: string, delayMs = 20) { const command = join(project, "command"); await writeFile( command, - `#!${process.execPath}\nawait Bun.write("request-started", String(process.pid));\nawait Bun.sleep(${delayMs});\nconsole.log(JSON.stringify({ marker: process.env.SYNTHETIC_SESSION_MARKER, cwd: process.cwd(), args: Bun.argv.slice(2) }));\n` + `#!${process.execPath}\nawait Bun.write("request-started", String(process.pid));\nawait Bun.sleep(${delayMs});\nconsole.log(JSON.stringify({ marker: process.env.SYNTHETIC_SESSION_MARKER, noInteractive: process.env.HACK_NO_INTERACTIVE ?? null, cwd: process.cwd(), args: Bun.argv.slice(2) }));\n` ); await chmod(command, 0o755); const env = { @@ -67,6 +67,9 @@ async function fixture(marker: string, delayMs = 20) { HACK_HOME: state, HACK_GLOBAL_CONFIG_PATH: join(registryDir, "config.json"), HACK_MCP_COMMAND: "./command", + // Children must still see HACK_NO_INTERACTIVE=1: the server overrides + // this per spawn on a copy, never on the frozen session env. + HACK_NO_INTERACTIVE: "0", SYNTHETIC_SESSION_MARKER: marker, }; const server = createMcpServer({ cwd: project, env }); @@ -106,8 +109,9 @@ test("MCP sessions preserve independent cwd, environment, registry selection and ok: true, data: { marker: index === 0 ? "first" : "second", + noInteractive: "1", cwd: await realpath(item.project), - args: ["ps", "--project", "shared-name", "--json"], + args: ["ps", "--project=shared-name", "--json"], }, }); const audit = await readFile(join(item.state, "mcp-audit.log"), "utf8"); @@ -131,6 +135,11 @@ test("MCP path authorization uses only that session's registry", async () => { expect(denied.isError).toBe(true); expect(denied.structuredContent).toMatchObject({ stderr: "Path is outside registered hack projects.", + spawned: false, + error: { + code: "E_PROJECT_NOT_FOUND", + message: "Path is outside registered hack projects.", + }, }); const accepted = await second.client.callTool({ name: "hack.project.status", diff --git a/tests/mcp.test.ts b/tests/mcp.test.ts index 9d9a6e12..1b0227b2 100644 --- a/tests/mcp.test.ts +++ b/tests/mcp.test.ts @@ -1,11 +1,26 @@ -import { afterEach, expect } from "bun:test"; -import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { afterEach, expect, test } from "bun:test"; +import { + chmod, + mkdir, + mkdtemp, + readFile, + rm, + writeFile, +} from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; +import { parseCliArgv, resolveCommand } from "../src/cli/command.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { + buildRunArgs, + createMcpServer, + MCP_INPUT_CEILINGS, +} from "../src/mcp/server.ts"; import { testIntegration } from "./helpers/ci.ts"; let tempDir: string | null = null; @@ -147,3 +162,467 @@ function buildHackStubScript(): string { "process.exit(1)", ].join("\n"); } + +// --------------------------------------------------------------------------- +// In-process argv and contract checks (HACK-1133 / HACK-1134). A stub hack +// binary records every argv it receives plus the child's interactivity env; +// the session writes its audit log under the fixture's HACK_HOME. +// --------------------------------------------------------------------------- + +const contractClients: Client[] = []; +const contractRoots: string[] = []; + +afterEach(async () => { + await Promise.all(contractClients.splice(0).map((mcp) => mcp.close())); + await Promise.all( + contractRoots + .splice(0) + .map((root) => rm(root, { recursive: true, force: true })) + ); +}); + +type ContractFixture = { + readonly client: Client; + readonly argvLog: string; + readonly auditLog: string; +}; + +type StubInvocation = { + readonly args: readonly string[]; + readonly noInteractive: string | null; +}; + +async function startContractFixture(): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-mcp-contract-")); + contractRoots.push(root); + const project = join(root, "repo"); + const projectDir = join(project, ".hack"); + const state = join(root, "state"); + const registryDir = join(root, "registry"); + await mkdir(projectDir, { recursive: true }); + await mkdir(registryDir); + await writeFile(join(projectDir, "docker-compose.yml"), "services: {}\n"); + await writeFile( + join(projectDir, "hack.config.json"), + JSON.stringify({ name: "demo" }) + ); + await writeFile( + join(registryDir, "projects.json"), + JSON.stringify({ + version: 1, + projects: [ + { + id: "demo", + createdAt: "2026-01-01T00:00:00Z", + name: "demo", + repoRoot: project, + projectDirName: ".hack", + projectDir, + }, + ], + }) + ); + const argvLog = join(root, "argv.jsonl"); + const stub = join(root, "hack-stub"); + await writeFile( + stub, + [ + `#!${process.execPath}`, + 'import { appendFileSync } from "node:fs";', + "const args = Bun.argv.slice(2);", + `appendFileSync(${JSON.stringify(argvLog)}, \`\${JSON.stringify({ args, noInteractive: process.env.HACK_NO_INTERACTIVE ?? null })}\\n\`);`, + "console.log(JSON.stringify({ ok: true }));", + ].join("\n") + ); + await chmod(stub, 0o755); + const env: Record = { + ...process.env, + HOME: root, + HACK_HOME: state, + HACK_GLOBAL_CONFIG_PATH: join(registryDir, "config.json"), + HACK_MCP_COMMAND: stub, + }; + Reflect.deleteProperty(env, "HACK_NO_INTERACTIVE"); + const server = createMcpServer({ cwd: project, env }); + const client = new Client({ name: "hack-mcp-contract", version: "1" }); + contractClients.push(client); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + await client.connect(clientTransport); + return { client, argvLog, auditLog: join(state, "mcp-audit.log") }; +} + +async function readStubInvocations( + argvLog: string +): Promise { + if (!(await Bun.file(argvLog).exists())) { + return []; + } + return (await readFile(argvLog, "utf8")) + .split("\n") + .filter((line) => line.length > 0) + .map((line) => JSON.parse(line) as StubInvocation); +} + +test("hack.project.run ends hack options with -- and passes child flags intact", async () => { + const fixture = await startContractFixture(); + const result = await fixture.client.callTool({ + name: "hack.project.run", + arguments: { + projectName: "demo", + branch: "feature/x", + profiles: ["tools", "extra"], + workdir: "/srv/app dir", + service: "api", + cmd: ["node", "--version", "--branch", "other"], + }, + }); + expect(result.isError).toBeUndefined(); + expect(result.structuredContent).toMatchObject({ + ok: true, + data: { limits: { timeoutMs: 120_000 } }, + }); + expect(await readStubInvocations(fixture.argvLog)).toEqual([ + { + args: [ + "run", + "--project=demo", + "--branch=feature/x", + "--profile=tools,extra", + "--workdir=/srv/app dir", + "--", + "api", + "node", + "--version", + "--branch", + "other", + ], + noInteractive: "1", + }, + ]); +}); + +test("hack's own parser never reads run child flags as hack options", () => { + const argv = buildRunArgs({ + projectArgs: ["--project=demo"], + service: "api", + cmd: ["node", "--version", "--branch", "other"], + }); + const parsed = parseCliArgv(CLI_SPEC, argv); + expect(parsed.values.version).toBeUndefined(); + expect(parsed.values.branch).toBeUndefined(); + expect(parsed.values.project).toBe("demo"); + const resolved = resolveCommand(CLI_SPEC, parsed.positionals); + expect(resolved.command?.name).toBe("run"); + expect(resolved.remainingPositionals).toEqual([ + "api", + "node", + "--version", + "--branch", + "other", + ]); +}); + +test("spawning tools force a non-interactive child and echo effective limits", async () => { + const markerBefore = process.env.HACK_NO_INTERACTIVE; + const fixture = await startContractFixture(); + const tail = await fixture.client.callTool({ + name: "hack.project.logs.tail", + arguments: { projectName: "demo", maxEvents: 5, maxMs: 1000, tail: 3 }, + }); + expect(tail.structuredContent).toMatchObject({ + ok: true, + data: { limits: { tail: 3, maxEvents: 5, maxMs: 1000 } }, + }); + const snapshot = await fixture.client.callTool({ + name: "hack.project.logs.snapshot", + arguments: { + projectName: "demo", + services: ["api", "worker"], + query: '{app="x"} |= "-y"', + }, + }); + expect(snapshot.structuredContent).toMatchObject({ + ok: true, + data: { limits: { tail: null } }, + }); + const ceiling = await fixture.client.callTool({ + name: "hack.project.run", + arguments: { + projectName: "demo", + service: "api", + timeoutMs: MCP_INPUT_CEILINGS.timeoutMs, + }, + }); + expect(ceiling.structuredContent).toMatchObject({ + ok: true, + data: { limits: { timeoutMs: MCP_INPUT_CEILINGS.timeoutMs } }, + }); + const invocations = await readStubInvocations(fixture.argvLog); + expect(invocations.map((entry) => entry.args)).toEqual([ + ["logs", "--project=demo", "--json", "--tail=3"], + [ + "logs", + "--project=demo", + "--json", + "--no-follow", + '--query={app="x"} |= "-y"', + "--services=api,worker", + ], + ["run", "--project=demo", "--", "api"], + ]); + expect(invocations.map((entry) => entry.noInteractive)).toEqual([ + "1", + "1", + "1", + ]); + // The child env is a copy: the server process env is never mutated. + expect(process.env.HACK_NO_INTERACTIVE).toBe(markerBefore); +}); + +const REFUSED_MARKER = "-REFUSED-VALUE-MARKER"; + +type RefusedCall = { + readonly label: string; + readonly name: string; + readonly arguments: Record; +}; + +const LIMIT_PROBES = [0, -1, 1.5] as const; + +function limitProbes(opts: { + readonly name: string; + readonly field: keyof typeof MCP_INPUT_CEILINGS; + readonly base: Record; +}): RefusedCall[] { + return [...LIMIT_PROBES, MCP_INPUT_CEILINGS[opts.field] + 1].map((value) => ({ + label: `${opts.name} ${opts.field}=${value}`, + name: opts.name, + arguments: { ...opts.base, [opts.field]: value }, + })); +} + +const refusedCalls: readonly RefusedCall[] = [ + { + label: "run leading-dash service", + name: "hack.project.run", + arguments: { projectName: "demo", service: REFUSED_MARKER }, + }, + { + label: "run empty service", + name: "hack.project.run", + arguments: { projectName: "demo", service: "" }, + }, + { + label: "run missing service", + name: "hack.project.run", + arguments: { projectName: "demo" }, + }, + { + label: "run leading-dash branch", + name: "hack.project.run", + arguments: { projectName: "demo", service: "api", branch: REFUSED_MARKER }, + }, + { + label: "status empty branch", + name: "hack.project.status", + arguments: { projectName: "demo", branch: "" }, + }, + { + label: "up leading-dash profile", + name: "hack.project.up", + arguments: { projectName: "demo", profiles: [REFUSED_MARKER] }, + }, + { + label: "down empty profile", + name: "hack.project.down", + arguments: { projectName: "demo", profiles: [""] }, + }, + { + label: "snapshot leading-dash service", + name: "hack.project.logs.snapshot", + arguments: { projectName: "demo", service: REFUSED_MARKER }, + }, + { + label: "snapshot leading-dash services entry", + name: "hack.project.logs.snapshot", + arguments: { projectName: "demo", services: ["api", REFUSED_MARKER] }, + }, + { + label: "open leading-dash target", + name: "hack.project.open", + arguments: { projectName: "demo", target: REFUSED_MARKER }, + }, + ...limitProbes({ + name: "hack.project.logs.tail", + field: "tail", + base: { projectName: "demo" }, + }), + ...limitProbes({ + name: "hack.project.logs.tail", + field: "maxEvents", + base: { projectName: "demo" }, + }), + ...limitProbes({ + name: "hack.project.logs.tail", + field: "maxMs", + base: { projectName: "demo" }, + }), + ...limitProbes({ + name: "hack.project.logs.snapshot", + field: "tail", + base: { projectName: "demo" }, + }), + ...limitProbes({ + name: "hack.project.run", + field: "timeoutMs", + base: { projectName: "demo", service: "api" }, + }), +]; + +test("invalid identifiers and limits are typed refusals that never spawn or audit", async () => { + const fixture = await startContractFixture(); + for (const call of refusedCalls) { + const result = await fixture.client.callTool({ + name: call.name, + arguments: call.arguments, + }); + expect({ label: call.label, isError: result.isError }).toEqual({ + label: call.label, + isError: true, + }); + expect(result.structuredContent).toMatchObject({ + ok: false, + spawned: false, + command: "", + exitCode: 1, + stdout: "", + error: { code: "E_USAGE" }, + }); + // Closed diagnostics: the rejected value is never echoed back. + expect(JSON.stringify(result)).not.toContain(REFUSED_MARKER); + } + expect(await readStubInvocations(fixture.argvLog)).toEqual([]); + expect(await Bun.file(fixture.auditLog).exists()).toBe(false); +}); + +test("project resolution refusals carry existing codes and never spawn", async () => { + const fixture = await startContractFixture(); + const unknown = await fixture.client.callTool({ + name: "hack.project.down", + arguments: { projectName: "missing-project" }, + }); + expect(unknown.structuredContent).toMatchObject({ + ok: false, + spawned: false, + error: { code: "E_PROJECT_NOT_FOUND" }, + }); + const ambiguous = await fixture.client.callTool({ + name: "hack.project.run", + arguments: { projectName: "demo", path: ".", service: "api" }, + }); + expect(ambiguous.structuredContent).toMatchObject({ + ok: false, + spawned: false, + stderr: "Use only one of projectName, repoRoot, or path.", + error: { + code: "E_USAGE", + message: "Use only one of projectName, repoRoot, or path.", + }, + }); + expect(await readStubInvocations(fixture.argvLog)).toEqual([]); + expect(await Bun.file(fixture.auditLog).exists()).toBe(false); +}); + +const READ_ONLY = { readOnlyHint: true, openWorldHint: false }; +const ADDITIVE = { + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + openWorldHint: false, +}; +const DESTRUCTIVE = { + readOnlyHint: false, + destructiveHint: true, + idempotentHint: false, + openWorldHint: false, +}; + +test("tools/list publishes names, annotations and required inputs for every tool", async () => { + const fixture = await startContractFixture(); + const listed = await fixture.client.listTools(); + expect( + listed.tools.map((tool) => ({ + name: tool.name, + annotations: tool.annotations, + required: tool.inputSchema.required ?? [], + })) + ).toEqual([ + { + name: "hack.projects.list", + annotations: { title: "List hack projects", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.status", + annotations: { title: "Project status", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.up", + annotations: { title: "Start project", ...ADDITIVE }, + required: [], + }, + { + name: "hack.project.down", + annotations: { title: "Stop project", ...DESTRUCTIVE }, + required: [], + }, + { + name: "hack.project.restart", + annotations: { title: "Restart project", ...ADDITIVE }, + required: [], + }, + { + name: "hack.project.run", + annotations: { + title: "Run command in service container", + ...DESTRUCTIVE, + }, + required: ["service"], + }, + { + name: "hack.project.logs.snapshot", + annotations: { title: "Snapshot logs", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.logs.tail", + annotations: { title: "Tail logs", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.open", + annotations: { title: "Resolve project URL", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.init", + annotations: { title: "Initialize project", ...DESTRUCTIVE }, + required: [], + }, + ]); + const run = listed.tools.find((tool) => tool.name === "hack.project.run"); + expect(run?.inputSchema.properties?.timeoutMs).toMatchObject({ + type: "integer", + exclusiveMinimum: 0, + maximum: MCP_INPUT_CEILINGS.timeoutMs, + }); + expect(run?.inputSchema.properties?.service).toMatchObject({ + type: "string", + pattern: "^[A-Za-z0-9][A-Za-z0-9_.-]*$", + }); + expect(run?.outputSchema?.properties).toHaveProperty("spawned"); + expect(run?.outputSchema?.properties).toHaveProperty("error"); +}); diff --git a/tests/project-run-command.test.ts b/tests/project-run-command.test.ts index ac79e70c..5b148b3d 100644 --- a/tests/project-run-command.test.ts +++ b/tests/project-run-command.test.ts @@ -12,6 +12,9 @@ import { registerScopedModuleMock } from "./helpers/scoped-module-mock.ts"; const runCalls: Array<{ readonly composeFiles: readonly string[]; + readonly composeProject: string | null | undefined; + readonly service: string; + readonly cmdArgs: readonly string[]; readonly env: Record | undefined; readonly noDeps: boolean | undefined; }> = []; @@ -36,11 +39,17 @@ const runtimeBackendMock = await registerScopedModuleMock({ ps: async () => 0, run: async (opts: { readonly composeFiles: readonly string[]; + readonly composeProject?: string | null; + readonly service: string; + readonly cmdArgs: readonly string[]; readonly env?: Record; readonly noDeps?: boolean; }) => { runCalls.push({ composeFiles: [...opts.composeFiles], + composeProject: opts.composeProject, + service: opts.service, + cmdArgs: [...opts.cmdArgs], env: opts.env ? { ...opts.env } : undefined, noDeps: opts.noDeps, }); @@ -68,6 +77,7 @@ const loggerMock = await registerScopedModuleMock({ }); const { runCommand } = await import("../src/commands/project.ts"); +const { runCli } = await import("../src/cli/run.ts"); beforeAll(() => { runtimeBackendMock.activate(); @@ -117,6 +127,33 @@ test("run reaches the runtime without writing a contended registry", async () => ); }); +test("hack run -- passes child flags intact and keeps the default compose project", async () => { + const projectRoot = await createProject(); + // Without `--`, hack would consume --version (print and exit) and --branch + // (retarget the instance); the MCP run tool relies on this boundary. + const exitCode = await runCli([ + "run", + `--path=${projectRoot}`, + "--", + "api", + "node", + "--version", + "--branch", + "other", + ]); + + expect(exitCode).toBe(0); + expect(runCalls).toHaveLength(1); + expect(runCalls[0]?.service).toBe("api"); + expect(runCalls[0]?.cmdArgs).toEqual([ + "node", + "--version", + "--branch", + "other", + ]); + expect(runCalls[0]?.composeProject).toBeNull(); +}); + async function runPrintenv({ projectRoot, envName, From c6c0db35ab75ae02aa0a4b5d307425211efb1992 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:40:00 -0400 Subject: [PATCH 03/52] fix(env): harden legacy env key files and AES-GCM envelope parsing Legacy project env key reads (checkout-local, shared common-dir and inherited primary keys) now use the same lstat, O_NOFOLLOW, size-bound and stable-descriptor discipline as the native reader, extracted into src/lib/private-file.ts. Key priority is unchanged. A symlinked, dangling, oversized or non-regular key file is refused with a fixed message. New key files are created with open(O_CREAT|O_EXCL|O_NOFOLLOW|O_WRONLY, 0600), fchmod 0600 and fsync, so an existing entry or dangling symlink is never followed or replaced. A concurrent creator's key is adopted. Encryption (env add --secret and the legacy migration) refuses a git-tracked selected or inherited key with a fixed, value-free message before any key copy. Decryption with a tracked key stays allowed for v4 compatibility. Legacy v1 values and the encrypted_file store now require canonical base64, a 12-byte IV, a 16-byte tag (authTagLength pinned) and a bounded ciphertext; all parse, tamper and wrong-key failures use fixed redacted errors. The envelope format, key derivation and v1 decryptability are unchanged. Refs HACK-1127 Co-Authored-By: Claude Opus 5.5 --- src/lib/private-file.ts | 239 ++++++++++++++++ src/lib/project-env-config.ts | 379 +++++++++++++++---------- src/lib/secret-store.ts | 82 ++++-- tests/native-env-values.test.ts | 23 ++ tests/project-env-config.test.ts | 205 +++++++++++++ tests/project-env-worktree-key.test.ts | 266 +++++++++++++++++ tests/secret-store.test.ts | 152 ++++++++++ 7 files changed, 1171 insertions(+), 175 deletions(-) create mode 100644 src/lib/private-file.ts diff --git a/src/lib/private-file.ts b/src/lib/private-file.ts new file mode 100644 index 00000000..fd7f02d6 --- /dev/null +++ b/src/lib/private-file.ts @@ -0,0 +1,239 @@ +import { constants, type Stats } from "node:fs"; +import { + type FileHandle, + lstat, + open, + realpath, + unlink, +} from "node:fs/promises"; +import { dirname } from "node:path"; +import { isRecord } from "./guards.ts"; + +/** + * Bounded, no-follow reads and exclusive creates for private key material. + * + * Ownership: callers own the path choice and the refusal message. These helpers + * never follow a symlink at the final path component, never replace an existing + * entry, and never include file contents in errors. + */ + +export type BoundedPrivateFileReadOptions = { + readonly path: string; + /** + * Directories whose identity must stay stable across the read. With + * `canonical`, each one and the file must also be spelled by its realpath. + */ + readonly directories: readonly string[]; + /** Require the canonical (realpath) spelling for `path` and `directories`. */ + readonly canonical: boolean; + /** Largest accepted file size in bytes. */ + readonly maxBytes: number; + /** Builds the caller's fixed, value-free refusal. */ + readonly refusal: () => Error; + /** Runs between steps, e.g. to observe cancellation. */ + readonly checkpoint?: () => void; +}; + +function sameFileVersion(before: Stats, after: Stats): boolean { + return ( + before.dev === after.dev && + before.ino === after.ino && + before.size === after.size && + before.mtimeMs === after.mtimeMs && + before.ctimeMs === after.ctimeMs + ); +} + +function isMissingEntryError(error: unknown): boolean { + return isRecord(error) && error.code === "ENOENT"; +} + +/** + * Reads a small private text file without following a final-component symlink. + * Returns null when the entry is absent or contains only whitespace; otherwise + * the trimmed UTF-8 text. A symlink (dangling or not), a non-regular file, an + * unreadable or oversized file, invalid UTF-8, or a file that changes during + * the read throws `refusal()`; other filesystem errors propagate unchanged. + */ +export async function readBoundedPrivateFile( + opts: BoundedPrivateFileReadOptions +): Promise { + const checkpoint = opts.checkpoint ?? (() => undefined); + checkpoint(); + const directories = await Promise.all( + opts.directories.map(async (path) => { + const stats = await lstat(path); + if ( + !stats.isDirectory() || + (opts.canonical && (await realpath(path)) !== path) + ) { + throw opts.refusal(); + } + return { path, stats }; + }) + ); + const recheckDirectories = async () => { + for (const { path, stats } of directories) { + const current = await lstat(path); + if ( + !current.isDirectory() || + current.dev !== stats.dev || + current.ino !== stats.ino || + (opts.canonical && (await realpath(path)) !== path) + ) { + throw opts.refusal(); + } + } + checkpoint(); + }; + const observed = await lstat(opts.path).catch((error: unknown) => { + if (isMissingEntryError(error)) { + return null; + } + throw error; + }); + if (!observed) { + await recheckDirectories(); + return null; + } + if ( + !observed.isFile() || + observed.size > opts.maxBytes || + (observed.mode & 0o444) === 0 || + (opts.canonical && (await realpath(opts.path)) !== opts.path) + ) { + throw opts.refusal(); + } + const file = await open( + opts.path, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK + ); + try { + checkpoint(); + const before = await file.stat(); + if (!(before.isFile() && sameFileVersion(observed, before))) { + throw opts.refusal(); + } + const buffer = Buffer.alloc(before.size + 1); + let size = 0; + while (size < buffer.length) { + checkpoint(); + const { bytesRead } = await file.read( + buffer, + size, + buffer.length - size, + size + ); + if (bytesRead === 0) { + break; + } + size += bytesRead; + } + const after = await file.stat(); + const current = await lstat(opts.path); + if ( + size !== before.size || + !current.isFile() || + !sameFileVersion(before, after) || + !sameFileVersion(before, current) || + (opts.canonical && (await realpath(opts.path)) !== opts.path) + ) { + throw opts.refusal(); + } + await recheckDirectories(); + let text: string; + try { + text = new TextDecoder("utf-8", { fatal: true }).decode( + buffer.subarray(0, size) + ); + } catch { + throw opts.refusal(); + } + return text.trim() || null; + } finally { + await file.close(); + } +} + +/** + * Creates `path` with mode 0600 and writes `text`, durably. The open uses + * O_CREAT|O_EXCL|O_NOFOLLOW, so an existing entry, including a dangling + * symlink, fails with EEXIST and its target is never created or written. + * On a failed write the partially written file is removed only while the + * path still names the file this call created. + */ +export async function createPrivateFileExclusive(opts: { + readonly path: string; + readonly text: string; +}): Promise { + const file = await open( + opts.path, + constants.O_CREAT | + constants.O_EXCL | + constants.O_NOFOLLOW | + constants.O_WRONLY, + 0o600 + ); + let created: Stats | null = null; + let complete = false; + try { + created = await file.stat(); + // The umask can only remove bits; restore owner read/write explicitly. + await file.chmod(0o600); + await file.writeFile(opts.text, "utf8"); + await file.sync(); + complete = true; + } finally { + await file.close(); + if (!complete && created) { + const current = await lstat(opts.path).catch(() => null); + if ( + current?.isFile() && + current.dev === created.dev && + current.ino === created.ino + ) { + await unlink(opts.path).catch(() => undefined); + } + } + } + await syncDirectory({ path: dirname(opts.path) }); +} + +/** Best effort: persist the directory entry where the platform allows it. */ +async function syncDirectory(opts: { readonly path: string }): Promise { + let directory: FileHandle | null = null; + try { + directory = await open(opts.path, constants.O_RDONLY); + await directory.sync(); + } catch { + // Some filesystems refuse directory fsync; the file itself is already synced. + } finally { + await directory?.close().catch(() => undefined); + } +} + +/** + * Decodes standard padded base64 only when the text is its canonical encoding + * (re-encoding the bytes reproduces the text exactly). Whitespace, URL-safe or + * invalid characters, missing or extra padding, and non-zero trailing bits are + * refused. The length is bounded before any decoding allocation. + */ +export function decodeCanonicalBase64(opts: { + readonly text: string; + readonly maxBytes: number; +}): Buffer | null { + if ( + opts.text.length % 4 !== 0 || + opts.text.length > Math.ceil(opts.maxBytes / 3) * 4 + ) { + return null; + } + const decoded = Buffer.from(opts.text, "base64"); + if ( + decoded.length > opts.maxBytes || + decoded.toString("base64") !== opts.text + ) { + return null; + } + return decoded; +} diff --git a/src/lib/project-env-config.ts b/src/lib/project-env-config.ts index 2e9c9895..1d756962 100644 --- a/src/lib/project-env-config.ts +++ b/src/lib/project-env-config.ts @@ -4,16 +4,7 @@ import { createHash, randomBytes, } from "node:crypto"; -import { constants, type Stats } from "node:fs"; -import { - chmod, - lstat, - open, - readdir, - realpath, - rm, - stat, -} from "node:fs/promises"; +import { lstat, readdir, rm, stat } from "node:fs/promises"; import { basename, dirname, relative, resolve } from "node:path"; import { YAML } from "bun"; import { @@ -63,6 +54,11 @@ import { acquireNativeManagedEnvFile, assertNativeProjectInputRoot, } from "./native-project-inputs.ts"; +import { + createPrivateFileExclusive, + decodeCanonicalBase64, + readBoundedPrivateFile, +} from "./private-file.ts"; import { readProjectDefaultEnvConfig } from "./project.ts"; import { assertLegacyProjectDirectory, @@ -87,6 +83,9 @@ const PROJECT_ENV_SCOPE_PATTERN = /^[a-z0-9][a-z0-9._-]*$/; const PROJECT_ENV_SECRET_PREFIX = "v1" as const; const PROJECT_ENV_ALGORITHM = "aes-256-gcm"; const PROJECT_ENV_IV_BYTES = 12; +const PROJECT_ENV_TAG_BYTES = 16; +/** Decoded ciphertext bound; a value never exceeds its 1 MiB env layer. */ +const PROJECT_ENV_CIPHERTEXT_LIMIT = NATIVE_CONFIG_INPUT_LIMIT; const PROJECT_ENV_SECRET_KEY_ENV = "HACK_ENV_SECRET_KEY"; export const PROJECT_ENV_HOST_SCOPE = "host" as const; @@ -2699,28 +2698,29 @@ async function resolveProjectEnvKey(opts: { signal: opts.nativeSelection.signal, }); } + // Priority: checkout-local, shared common-dir, primary checkout, environment. const keyPath = resolveProjectEnvKeyPath({ projectRoot: opts.projectRoot }); + const localKey = await readLegacyProjectEnvKeyFile({ path: keyPath }); + if (localKey) { + return localKey; + } const sharedKeyPath = await resolveProjectEnvSharedKeyPath({ projectRoot: opts.projectRoot, }); - const text = await readTextFile(keyPath); - const sharedText = - sharedKeyPath === null ? null : await readTextFile(sharedKeyPath); - const envFallback = process.env[PROJECT_ENV_SECRET_KEY_ENV]?.trim() ?? ""; + const sharedKey = + sharedKeyPath === null + ? null + : await readLegacyProjectEnvKeyFile({ path: sharedKeyPath }); + if (sharedKey) { + return sharedKey; + } const inheritedKey = await resolveInheritedProjectEnvKey({ projectRoot: opts.projectRoot, }); - const trimmed = text?.trim() ?? ""; - if (trimmed.length > 0) { - return trimmed; - } - const sharedTrimmed = sharedText?.trim() ?? ""; - if (sharedTrimmed.length > 0) { - return sharedTrimmed; - } if (inheritedKey) { return inheritedKey.keyText; } + const envFallback = process.env[PROJECT_ENV_SECRET_KEY_ENV]?.trim() ?? ""; if (envFallback.length > 0) { return envFallback; } @@ -2787,108 +2787,80 @@ async function resolveNativeProjectEnvKey(opts: { return null; } -function sameNativeKeyFile(before: Stats, after: Stats): boolean { - return ( - before.dev === after.dev && - before.ino === after.ino && - before.size === after.size && - before.mtimeMs === after.mtimeMs && - before.ctimeMs === after.ctimeMs - ); -} - /** Keys stay inside their existing owner; reject redirected, oversized or changing descriptor reads. */ async function readNativeProjectEnvKeyFile(opts: { readonly path: string; readonly directories: readonly string[]; readonly signal?: AbortSignal; }): Promise { - checkNativeEnvCancellation(opts.signal); - const directories = await Promise.all( - opts.directories.map(async (path) => { - const stats = await lstat(path); - if (!stats.isDirectory() || (await realpath(path)) !== path) { - throw nativeEnvValuesError(); - } - return { path, stats }; - }) - ); - const recheckDirectories = async () => { - for (const { path, stats } of directories) { - const current = await lstat(path); - if ( - !current.isDirectory() || - current.dev !== stats.dev || - current.ino !== stats.ino || - (await realpath(path)) !== path - ) { - throw nativeEnvValuesError(); - } - } - checkNativeEnvCancellation(opts.signal); - }; - const observed = await lstat(opts.path).catch((error: unknown) => { - if (isRecord(error) && error.code === "ENOENT") { - return null; - } - throw error; + return await readBoundedPrivateFile({ + path: opts.path, + directories: opts.directories, + canonical: true, + maxBytes: NATIVE_CONFIG_INPUT_LIMIT, + refusal: nativeEnvValuesError, + checkpoint: () => checkNativeEnvCancellation(opts.signal), }); - if (!observed) { - await recheckDirectories(); - return null; +} + +/** + * Legacy key files are refused unless they are regular, non-symlink, bounded + * UTF-8 files, so a committed `.hack.secret.key` symlink never selects another + * file as the key. Directory components keep their legacy spelling (temporary + * and home directories are often reached through symlinks); only the final + * component is pinned. Absent and whitespace-only files return null. + */ +async function readLegacyProjectEnvKeyFile(opts: { + readonly path: string; +}): Promise { + try { + return await readBoundedPrivateFile({ + path: opts.path, + directories: [], + canonical: false, + maxBytes: NATIVE_CONFIG_INPUT_LIMIT, + refusal: projectEnvKeyFileError, + }); + } catch { + throw projectEnvKeyFileError(); } - if ( - !observed.isFile() || - observed.size > NATIVE_CONFIG_INPUT_LIMIT || - (observed.mode & 0o444) === 0 || - (await realpath(opts.path)) !== opts.path - ) { - throw nativeEnvValuesError(); +} + +/** Fixed, value-free refusal for legacy key selection, creation and encryption. */ +export class ProjectEnvKeyRefusalError extends Error { + constructor(message: string) { + super(message); + this.name = "ProjectEnvKeyRefusalError"; } - const file = await open( - opts.path, - constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK +} + +function projectEnvKeyFileError(): ProjectEnvKeyRefusalError { + return new ProjectEnvKeyRefusalError( + `Refusing project env key: ${PROJECT_ENV_KEY_FILENAME} must be a regular, readable, non-symlink, non-empty UTF-8 file of at most 1 MiB. Remove or replace it; key contents are never included in this message.` ); - try { - checkNativeEnvCancellation(opts.signal); - const before = await file.stat(); - if (!(before.isFile() && sameNativeKeyFile(observed, before))) { - throw nativeEnvValuesError(); - } - const buffer = Buffer.alloc(before.size + 1); - let size = 0; - while (size < buffer.length) { - checkNativeEnvCancellation(opts.signal); - const { bytesRead } = await file.read( - buffer, - size, - buffer.length - size, - size - ); - if (bytesRead === 0) { - break; - } - size += bytesRead; - } - const after = await file.stat(); - const current = await lstat(opts.path); - if ( - size !== before.size || - !current.isFile() || - !sameNativeKeyFile(before, after) || - !sameNativeKeyFile(before, current) || - (await realpath(opts.path)) !== opts.path - ) { - throw nativeEnvValuesError(); - } - await recheckDirectories(); - return ( - new TextDecoder("utf-8", { fatal: true }) - .decode(buffer.subarray(0, size)) - .trim() || null - ); - } finally { - await file.close(); +} + +function projectEnvTrackedKeyError(): ProjectEnvKeyRefusalError { + return new ProjectEnvKeyRefusalError( + `Refusing to encrypt with a git-tracked ${PROJECT_ENV_KEY_FILENAME}: a committed key is not secret. Run 'hack doctor --fix' to untrack it, rotate the key, and re-encrypt existing secrets.` + ); +} + +/** + * Encryption refuses a key file that git tracks. Decryption with such a key + * stays allowed for v4 compatibility; `hack doctor` reports the tracked key. + * An undeterminable status (git unavailable, path outside a work tree) is not + * treated as tracked. + */ +async function assertProjectEnvKeyUntracked(opts: { + readonly keyPath: string; +}): Promise { + const tracked = await isGitTrackedProjectEnvPath({ + projectRoot: dirname(opts.keyPath), + path: opts.keyPath, + }); + if (tracked === true) { + throw projectEnvTrackedKeyError(); } } @@ -2913,6 +2885,11 @@ export type EnsureProjectEnvSecretKeyResult = { * A checkout-local key is never silently created in a linked worktree: if the * shared location cannot be resolved (degraded git) or written, the fallback * local write carries a loud divergence warning (also returned in `warnings`). + * + * This is the encryption-side key owner. Key files are read without following + * symlinks, a git-tracked selected or adopted key is refused before any write, + * and new key files are created exclusively with mode 0600 (an existing entry, + * including a dangling symlink, is never replaced or followed). */ export async function ensureProjectEnvSecretKey(opts: { readonly projectRoot: string; @@ -2923,25 +2900,25 @@ export async function ensureProjectEnvSecretKey(opts: { projectRoot: opts.projectRoot, }); const sharedKeyPath = await resolveSharedKeyPathFromLocation(sharedLocation); - const existing = await readTextFile(keyPath); - if (existing !== null && existing.trim().length > 0) { + const existing = await readLegacyProjectEnvKeyFile({ path: keyPath }); + if (existing !== null) { + await assertProjectEnvKeyUntracked({ keyPath }); return { keyPath, - keyText: existing.trim(), + keyText: existing, created: false, warnings: [], }; } const sharedExisting = - sharedKeyPath === null ? null : await readTextFile(sharedKeyPath); - if ( - sharedKeyPath !== null && - sharedExisting !== null && - sharedExisting.trim().length > 0 - ) { + sharedKeyPath === null + ? null + : await readLegacyProjectEnvKeyFile({ path: sharedKeyPath }); + if (sharedKeyPath !== null && sharedExisting !== null) { + await assertProjectEnvKeyUntracked({ keyPath: sharedKeyPath }); return { keyPath: sharedKeyPath, - keyText: sharedExisting.trim(), + keyText: sharedExisting, created: false, warnings: [], }; @@ -2951,6 +2928,8 @@ export async function ensureProjectEnvSecretKey(opts: { projectRoot: opts.projectRoot, }); if (inheritedKey) { + // Refuse before copying a committed key into the shared location. + await assertProjectEnvKeyUntracked({ keyPath: inheritedKey.keyPath }); const written = await writeProjectEnvKeyWithFallback({ projectRoot: opts.projectRoot, preferredKeyPath: sharedKeyPath, @@ -2960,7 +2939,7 @@ export async function ensureProjectEnvSecretKey(opts: { }); return { keyPath: written.keyPath, - keyText: inheritedKey.keyText, + keyText: written.keyText, created: false, warnings: written.warnings, }; @@ -2976,8 +2955,9 @@ export async function ensureProjectEnvSecretKey(opts: { }); return { keyPath: written.keyPath, - keyText, - created: true, + keyText: written.keyText, + // A concurrent writer's key is adopted rather than replaced. + created: written.keyText === keyText, warnings: written.warnings, }; } @@ -3005,19 +2985,26 @@ async function writeProjectEnvKeyWithFallback(opts: { readonly localKeyPath: string; readonly keyText: string; readonly sharedLocation: ProjectEnvSharedKeyLocation | null; -}): Promise<{ readonly keyPath: string; readonly warnings: string[] }> { +}): Promise<{ + readonly keyPath: string; + readonly keyText: string; + readonly warnings: string[]; +}> { const warnings: string[] = []; if (opts.preferredKeyPath !== null) { try { - await writeProjectEnvKeyFile({ + const keyText = await writeProjectEnvKeyFile({ projectRoot: opts.projectRoot, path: opts.preferredKeyPath, keyText: opts.keyText, }); - return { keyPath: opts.preferredKeyPath, warnings }; + return { keyPath: opts.preferredKeyPath, keyText, warnings }; } catch (error: unknown) { - if (error instanceof ProjectInputSelectionError) { + if ( + error instanceof ProjectInputSelectionError || + error instanceof ProjectEnvKeyRefusalError + ) { throw error; } const message = error instanceof Error ? error.message : String(error); @@ -3034,7 +3021,7 @@ async function writeProjectEnvKeyWithFallback(opts: { ); } - await writeProjectEnvKeyFile({ + const keyText = await writeProjectEnvKeyFile({ projectRoot: opts.projectRoot, path: opts.localKeyPath, keyText: opts.keyText, @@ -3046,19 +3033,68 @@ async function writeProjectEnvKeyWithFallback(opts: { comment: "# project env key", }); emitProjectEnvKeyWarnings({ warnings }); - return { keyPath: opts.localKeyPath, warnings }; + return { keyPath: opts.localKeyPath, keyText, warnings }; } +/** + * Creates the key file exclusively (O_CREAT|O_EXCL|O_NOFOLLOW, mode 0600, + * fsync) and returns the key text now stored at `path`. When the path is + * already occupied, a valid untracked regular key file (e.g. a concurrent + * writer's) is adopted; anything else, including a symlink, is refused. + */ async function writeProjectEnvKeyFile(opts: { readonly projectRoot: string; readonly path: string; readonly keyText: string; -}): Promise { +}): Promise { await assertLegacyProjectInputFamily({ projectRoot: opts.projectRoot }); await ensureDir(dirname(opts.path)); await assertLegacyProjectInputFamily({ projectRoot: opts.projectRoot }); - await writeTextFile(opts.path, `${opts.keyText}\n`); - await chmod(opts.path, 0o600); + try { + await createPrivateFileExclusive({ + path: opts.path, + text: `${opts.keyText}\n`, + }); + return opts.keyText; + } catch (error: unknown) { + if (!(isRecord(error) && error.code === "EEXIST")) { + throw error; + } + } + return await adoptExistingProjectEnvKeyFile({ path: opts.path }); +} + +const PROJECT_ENV_KEY_ADOPTION_ATTEMPTS = 20; +const PROJECT_ENV_KEY_ADOPTION_DELAY_MS = 25; + +/** + * A concurrent creator may expose its key file before writing the key, so a + * regular file is re-read for a bounded period (about half a second). A + * symlink or other entry, or a file still empty or invalid afterwards, is + * refused; it is never replaced. + */ +async function adoptExistingProjectEnvKeyFile(opts: { + readonly path: string; +}): Promise { + for ( + let attempt = 0; + attempt < PROJECT_ENV_KEY_ADOPTION_ATTEMPTS; + attempt += 1 + ) { + const entry = await lstat(opts.path).catch(() => null); + if (!entry?.isFile()) { + break; + } + const existing = await readLegacyProjectEnvKeyFile({ + path: opts.path, + }).catch(() => null); + if (existing !== null) { + await assertProjectEnvKeyUntracked({ keyPath: opts.path }); + return existing; + } + await Bun.sleep(PROJECT_ENV_KEY_ADOPTION_DELAY_MS); + } + throw projectEnvKeyFileError(); } function emitProjectEnvKeyWarnings(opts: { @@ -3080,8 +3116,8 @@ async function resolveInheritedProjectEnvKey(opts: { } const keyPath = resolveProjectEnvKeyPath({ projectRoot: primaryRoot }); - const keyText = (await readTextFile(keyPath))?.trim() ?? ""; - if (keyText.length === 0) { + const keyText = await readLegacyProjectEnvKeyFile({ path: keyPath }); + if (keyText === null) { return null; } @@ -3097,7 +3133,9 @@ function encryptProjectEnvValue(opts: { }): string { const iv = randomBytes(PROJECT_ENV_IV_BYTES); const key = deriveProjectEnvKey({ keyText: opts.keyText }); - const cipher = createCipheriv(PROJECT_ENV_ALGORITHM, key, iv); + const cipher = createCipheriv(PROJECT_ENV_ALGORITHM, key, iv, { + authTagLength: PROJECT_ENV_TAG_BYTES, + }); const ciphertext = Buffer.concat([ cipher.update(opts.plaintext, "utf8"), cipher.final(), @@ -3111,34 +3149,65 @@ function encryptProjectEnvValue(opts: { ].join(":"); } +function invalidSecureEnvValueError(): Error { + return new Error("Invalid secure env value."); +} + +/** + * Parses and opens a v1 value `v1:::`. Each segment must + * be canonical padded base64; the IV must be 12 bytes, the tag 16 bytes and the + * ciphertext at most the env input bound. An empty ciphertext segment is the + * authenticated empty plaintext. Malformed, oversized, truncated, tampered and + * wrong-key inputs all throw the same fixed, value-free error. + */ function decryptProjectEnvValue(opts: { readonly ciphertext: string; readonly keyText: string; }): string { const parts = opts.ciphertext.split(":"); const [prefix, ivText, tagText, ciphertextText] = parts; - // AES-GCM authenticates an empty plaintext with an empty ciphertext segment. if ( parts.length !== 4 || prefix !== PROJECT_ENV_SECRET_PREFIX || - !ivText || - !tagText || + ivText === undefined || + tagText === undefined || ciphertextText === undefined ) { - throw new Error("Invalid secure env value."); + throw invalidSecureEnvValueError(); + } + const iv = decodeCanonicalBase64({ + text: ivText, + maxBytes: PROJECT_ENV_IV_BYTES, + }); + const tag = decodeCanonicalBase64({ + text: tagText, + maxBytes: PROJECT_ENV_TAG_BYTES, + }); + const ciphertext = decodeCanonicalBase64({ + text: ciphertextText, + maxBytes: PROJECT_ENV_CIPHERTEXT_LIMIT, + }); + if ( + iv?.length !== PROJECT_ENV_IV_BYTES || + tag?.length !== PROJECT_ENV_TAG_BYTES || + ciphertext === null + ) { + throw invalidSecureEnvValueError(); + } + try { + const key = deriveProjectEnvKey({ keyText: opts.keyText }); + const decipher = createDecipheriv(PROJECT_ENV_ALGORITHM, key, iv, { + authTagLength: PROJECT_ENV_TAG_BYTES, + }); + decipher.setAuthTag(tag); + const plaintext = Buffer.concat([ + decipher.update(ciphertext), + decipher.final(), + ]); + return plaintext.toString("utf8"); + } catch { + throw invalidSecureEnvValueError(); } - const key = deriveProjectEnvKey({ keyText: opts.keyText }); - const decipher = createDecipheriv( - PROJECT_ENV_ALGORITHM, - key, - Buffer.from(ivText, "base64") - ); - decipher.setAuthTag(Buffer.from(tagText, "base64")); - const plaintext = Buffer.concat([ - decipher.update(Buffer.from(ciphertextText, "base64")), - decipher.final(), - ]); - return plaintext.toString("utf8"); } function deriveProjectEnvKey(opts: { readonly keyText: string }): Buffer { diff --git a/src/lib/secret-store.ts b/src/lib/secret-store.ts index bc82e9ae..84749224 100644 --- a/src/lib/secret-store.ts +++ b/src/lib/secret-store.ts @@ -15,6 +15,7 @@ import type { import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { resolveGlobalHackDir } from "./config-paths.ts"; import { ensureDir, readTextFile } from "./fs.ts"; +import { decodeCanonicalBase64 } from "./private-file.ts"; const ENCRYPTED_FILE_KEY_ENV = "HACK_SECRETS_FILE_KEY"; const DISABLE_ENCRYPTED_FILE_KEYCHAIN_FALLBACK_ENV = @@ -27,6 +28,9 @@ const ENCRYPTED_STORE_VERSION = 1 as const; const PLAINTEXT_STORE_VERSION = 1 as const; const ENCRYPTED_ALGORITHM = "aes-256-gcm"; const ENCRYPTED_IV_BYTES = 12; +const ENCRYPTED_TAG_BYTES = 16; +/** Decoded ciphertext bound for the whole encrypted store. */ +const ENCRYPTED_CIPHERTEXT_LIMIT = 16 * 1024 * 1024; let cachedEncryptedFileKey: string | null = null; let encryptedFileKeyFailureCooldownUntilMs = 0; @@ -38,6 +42,13 @@ type EncryptedStorePayload = { readonly ciphertext: string; }; +/** Envelope fields decoded and size-checked; the bytes are not yet authenticated. */ +type ParsedEncryptedEnvelope = { + readonly iv: Buffer; + readonly tag: Buffer; + readonly ciphertext: Buffer; +}; + type PlaintextStore = { readonly version: typeof PLAINTEXT_STORE_VERSION; readonly entries: Record; @@ -712,10 +723,17 @@ async function readEncryptedEntries(input: { `Invalid encrypted secret store payload at ${input.filePath}.` ); } - const plaintext = decryptStore({ - envelope, - encryptionKey: input.encryptionKey, - }); + let plaintext: string; + try { + plaintext = decryptStore({ + envelope, + encryptionKey: input.encryptionKey, + }); + } catch { + throw new Error( + `Cannot decrypt encrypted secret store at ${input.filePath}: the key does not match or the payload was modified.` + ); + } return parsePlaintextStore(plaintext).entries; } @@ -736,7 +754,14 @@ async function writeEncryptedEntries(input: { await Bun.write(input.filePath, `${JSON.stringify(envelope, null, 2)}\n`); } -function parseEncryptedEnvelope(value: unknown): EncryptedStorePayload | null { +/** + * Accepts only the v1 envelope with canonical padded base64 fields, a 12-byte + * IV, a 16-byte tag and a bounded ciphertext. Anything else returns null so the + * caller reports a fixed, value-free error. + */ +function parseEncryptedEnvelope( + value: unknown +): ParsedEncryptedEnvelope | null { if (!(value && typeof value === "object")) { return null; } @@ -750,13 +775,26 @@ function parseEncryptedEnvelope(value: unknown): EncryptedStorePayload | null { ) { return null; } - return { - version: ENCRYPTED_STORE_VERSION, - algorithm: ENCRYPTED_ALGORITHM, - iv: maybe.iv, - tag: maybe.tag, - ciphertext: maybe.ciphertext, - }; + const iv = decodeCanonicalBase64({ + text: maybe.iv, + maxBytes: ENCRYPTED_IV_BYTES, + }); + const tag = decodeCanonicalBase64({ + text: maybe.tag, + maxBytes: ENCRYPTED_TAG_BYTES, + }); + const ciphertext = decodeCanonicalBase64({ + text: maybe.ciphertext, + maxBytes: ENCRYPTED_CIPHERTEXT_LIMIT, + }); + if ( + iv?.length !== ENCRYPTED_IV_BYTES || + tag?.length !== ENCRYPTED_TAG_BYTES || + ciphertext === null + ) { + return null; + } + return { iv, tag, ciphertext }; } function parsePlaintextStore(value: string): PlaintextStore { @@ -797,7 +835,9 @@ function encryptStore(input: { }): EncryptedStorePayload { const iv = randomBytes(ENCRYPTED_IV_BYTES); const key = deriveEncryptionKey({ source: input.encryptionKey }); - const cipher = createCipheriv(ENCRYPTED_ALGORITHM, key, iv); + const cipher = createCipheriv(ENCRYPTED_ALGORITHM, key, iv, { + authTagLength: ENCRYPTED_TAG_BYTES, + }); const ciphertext = Buffer.concat([ cipher.update(input.plaintext, "utf8"), cipher.final(), @@ -813,17 +853,19 @@ function encryptStore(input: { } function decryptStore(input: { - readonly envelope: EncryptedStorePayload; + readonly envelope: ParsedEncryptedEnvelope; readonly encryptionKey: string; }): string { const key = deriveEncryptionKey({ source: input.encryptionKey }); - const iv = Buffer.from(input.envelope.iv, "base64"); - const tag = Buffer.from(input.envelope.tag, "base64"); - const ciphertext = Buffer.from(input.envelope.ciphertext, "base64"); - const decipher = createDecipheriv(ENCRYPTED_ALGORITHM, key, iv); - decipher.setAuthTag(tag); + const decipher = createDecipheriv( + ENCRYPTED_ALGORITHM, + key, + input.envelope.iv, + { authTagLength: ENCRYPTED_TAG_BYTES } + ); + decipher.setAuthTag(input.envelope.tag); const plaintext = Buffer.concat([ - decipher.update(ciphertext), + decipher.update(input.envelope.ciphertext), decipher.final(), ]); return plaintext.toString("utf8"); diff --git a/tests/native-env-values.test.ts b/tests/native-env-values.test.ts index a0d35af4..e3aaeef7 100644 --- a/tests/native-env-values.test.ts +++ b/tests/native-env-values.test.ts @@ -508,6 +508,29 @@ for (const kind of [ }); } +// GCM verifies a truncated tag unless the tag length is pinned; both owners pin 16 bytes. +for (const kind of ["truncated-tag", "noncanonical-base64"] as const) { + test(`authentic ciphertext with a ${kind} refuses with fixed redacted diagnostics`, async () => { + const p = await project(); + await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); + const authentic = await encrypted(); + if (!(isRecord(authentic) && typeof authentic.secure === "string")) { + throw new Error("Synthetic ciphertext absent"); + } + const [prefix, iv, tag = "", ciphertext] = authentic.secure.split(":"); + const changedTag = + kind === "truncated-tag" + ? Buffer.from(tag, "base64").subarray(0, 12).toString("base64") + : ` ${tag}`; + await layer(p, "hack.env.default.yaml", { + global: { + SECRET: { secure: [prefix, iv, changedTag, ciphertext].join(":") }, + }, + }); + await refuses(() => values(p)); + }); +} + test("an encrypted empty value preserves presence", async () => { const p = await project(); await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); diff --git a/tests/project-env-config.test.ts b/tests/project-env-config.test.ts index 0ed20d6d..ee3da7b8 100644 --- a/tests/project-env-config.test.ts +++ b/tests/project-env-config.test.ts @@ -10,6 +10,7 @@ import { } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; +import { YAML } from "bun"; import { PROJECT_COMPOSE_FILENAME, @@ -1548,3 +1549,207 @@ test("Compose env repair rechecks native selection after reading the repair inpu ).rejects.toThrow("E_NATIVE_PROJECT_CONFLICT"); expect(await readFile(repo.composeFile, "utf8")).toBe(contents); }); + +const ENVELOPE_KEY = "synthetic-envelope-key-for-tests-only"; +const ENVELOPE_SENTINEL = "synthetic-envelope-plaintext-never-in-errors"; + +type EnvelopeParts = { + readonly iv: Buffer; + readonly tag: Buffer; + readonly ciphertext: Buffer; +}; + +/** Encrypts the sentinel with the real mutation owner and returns the v1 value. */ +async function encryptedEnvelopeFixture(): Promise<{ + readonly repo: Awaited>; + readonly secure: string; +}> { + const repo = await createRepo(); + await writeFile( + resolve(repo.projectRoot, PROJECT_ENV_KEY_FILENAME), + `${ENVELOPE_KEY}\n` + ); + await setProjectEnvValue({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + scope: "global", + key: "TOKEN", + value: ENVELOPE_SENTINEL, + secret: true, + }); + const parsed: unknown = YAML.parse( + await readFile(resolve(repo.projectDir, "hack.env.default.yaml"), "utf8") + ); + const secure = ( + parsed as { values?: { global?: { TOKEN?: { secure?: unknown } } } } + ).values?.global?.TOKEN?.secure; + if (typeof secure !== "string") { + throw new Error("missing synthetic envelope"); + } + return { repo, secure }; +} + +function envelopeParts(secure: string): EnvelopeParts { + const [, iv = "", tag = "", ciphertext = ""] = secure.split(":"); + return { + iv: Buffer.from(iv, "base64"), + tag: Buffer.from(tag, "base64"), + ciphertext: Buffer.from(ciphertext, "base64"), + }; +} + +function envelope(parts: EnvelopeParts): string { + return [ + "v1", + parts.iv.toString("base64"), + parts.tag.toString("base64"), + parts.ciphertext.toString("base64"), + ].join(":"); +} + +async function writeSecureValue(opts: { + readonly repo: Awaited>; + readonly secure: string; +}): Promise { + await writeFile( + resolve(opts.repo.projectDir, "hack.env.default.yaml"), + JSON.stringify({ + version: 1, + environment: "default", + secretsprovider: "project_key", + values: { global: { TOKEN: { secure: opts.secure } } }, + }) + ); +} + +const ENVELOPE_MUTATIONS: ReadonlyArray< + readonly [string, (secure: string) => string] +> = [ + ...[4, 8, 12].map( + (bytes) => + [ + `${bytes}-byte tag`, + (secure: string) => { + const parts = envelopeParts(secure); + return envelope({ ...parts, tag: parts.tag.subarray(0, bytes) }); + }, + ] as const + ), + [ + "16-byte IV", + (secure) => { + const parts = envelopeParts(secure); + return envelope({ + ...parts, + iv: Buffer.concat([parts.iv, Buffer.alloc(4)]), + }); + }, + ], + [ + "base64 with !", + (secure) => { + const [prefix, iv, tag, ciphertext = ""] = secure.split(":"); + return [prefix, iv, tag, `!${ciphertext.slice(1)}`].join(":"); + }, + ], + [ + "base64 with whitespace", + (secure) => { + const [prefix, iv = "", tag, ciphertext] = secure.split(":"); + return [prefix, `${iv.slice(0, 4)} ${iv.slice(4)}`, tag, ciphertext].join( + ":" + ); + }, + ], + [ + "truncated ciphertext", + (secure) => { + const parts = envelopeParts(secure); + return envelope({ + ...parts, + ciphertext: parts.ciphertext.subarray(0, -1), + }); + }, + ], + [ + "bit-flipped ciphertext", + (secure) => { + const parts = envelopeParts(secure); + const ciphertext = Buffer.from(parts.ciphertext); + ciphertext[0] = (ciphertext[0] ?? 0) ^ 1; + return envelope({ ...parts, ciphertext }); + }, + ], + [ + "oversized ciphertext", + (secure) => + envelope({ + ...envelopeParts(secure), + ciphertext: Buffer.alloc(1024 * 1024 + 1, 7), + }), + ], +]; + +for (const [name, mutate] of ENVELOPE_MUTATIONS) { + test(`legacy secure env value with ${name} refuses with a fixed redacted error`, async () => { + const { repo, secure } = await encryptedEnvelopeFixture(); + const mutated = mutate(secure); + expect(mutated).not.toBe(secure); + await writeSecureValue({ repo, secure: mutated }); + + const error = await resolveProjectEnvConfig({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + serviceNames: ["api"], + }).then( + () => null, + (caught: unknown) => caught + ); + + expect(error).toBeInstanceOf(Error); + const message = error instanceof Error ? error.message : ""; + expect(message).toBe("Invalid secure env value."); + for (const privateText of [ENVELOPE_SENTINEL, ENVELOPE_KEY]) { + expect(String(error)).not.toContain(privateText); + } + }); +} + +test("legacy secure env value with the wrong key refuses with the same fixed error", async () => { + const { repo } = await encryptedEnvelopeFixture(); + await writeFile( + resolve(repo.projectRoot, PROJECT_ENV_KEY_FILENAME), + "synthetic-wrong-key\n" + ); + await expect( + resolveProjectEnvConfig({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + serviceNames: ["api"], + }) + ).rejects.toThrow("Invalid secure env value."); +}); + +test("legacy v1 secure values round-trip, including the empty value", async () => { + const { repo } = await encryptedEnvelopeFixture(); + await setProjectEnvValue({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + scope: "global", + key: "EMPTY", + value: "", + secret: true, + }); + const resolved = await resolveProjectEnvConfig({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + serviceNames: ["api"], + }); + expect(resolved?.globalEnv.TOKEN).toBe(ENVELOPE_SENTINEL); + expect(resolved?.globalEnv.EMPTY).toBe(""); +}); diff --git a/tests/project-env-worktree-key.test.ts b/tests/project-env-worktree-key.test.ts index cc723338..3bda56e8 100644 --- a/tests/project-env-worktree-key.test.ts +++ b/tests/project-env-worktree-key.test.ts @@ -1,11 +1,13 @@ import { afterEach, expect, test } from "bun:test"; import { chmod, + lstat, mkdir, mkdtemp, readFile, rm, stat, + symlink, writeFile, } from "node:fs/promises"; import { tmpdir } from "node:os"; @@ -16,8 +18,10 @@ import { PROJECT_CONFIG_FILENAME, PROJECT_ENV_KEY_FILENAME, } from "../src/constants.ts"; +import { createPrivateFileExclusive } from "../src/lib/private-file.ts"; import { ensureProjectEnvSecretKey, + ProjectEnvKeyRefusalError, resolveProjectEnvConfig, resolveProjectEnvSharedKeyLocation, setProjectEnvValue, @@ -241,3 +245,265 @@ test.skipIf(process.getuid?.() === 0)( expect(ensured.warnings[0]).toContain(location.path); } ); + +const KEY_SENTINEL = "synthetic-committed-key-text-never-in-output"; +const VALUE_SENTINEL = "synthetic-secret-value-never-in-output"; + +type CliResult = { + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; +}; + +/** Runs the current-branch CLI with an isolated HOME and global config. */ +async function runHackCli(opts: { + readonly sandbox: string; + readonly args: readonly string[]; +}): Promise { + const home = resolve(opts.sandbox, "home"); + await mkdir(home, { recursive: true }); + const env: Record = { + ...process.env, + HOME: home, + HACK_GLOBAL_CONFIG_PATH: resolve(home, "hack.config.json"), + }; + Reflect.deleteProperty(env, "HACK_ENV_SECRET_KEY"); + const proc = Bun.spawn( + ["bun", resolve(import.meta.dir, "../index.ts"), ...opts.args], + { + cwd: opts.sandbox, + env, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + } + ); + const [stdout, stderr, exitCode] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + return { exitCode, stdout, stderr }; +} + +function envAddSecretArgs(projectRoot: string): readonly string[] { + return [ + "env", + "add", + "API_TOKEN", + VALUE_SENTINEL, + "--secret", + "--path", + projectRoot, + ]; +} + +function expectNoPrivateText(result: CliResult): void { + for (const text of [KEY_SENTINEL, VALUE_SENTINEL]) { + expect(result.stdout).not.toContain(text); + expect(result.stderr).not.toContain(text); + } +} + +async function commitKeyFile(primaryRoot: string): Promise { + await runGit(["add", "-f", PROJECT_ENV_KEY_FILENAME], primaryRoot); + await runGit(["commit", "-m", "commit key"], primaryRoot); +} + +test("env add --secret refuses a dangling key symlink and never creates its target", async () => { + const fixture = await createWorktreeFixture(); + const leakPath = resolve(fixture.primaryRoot, "leak.txt"); + await symlink( + "leak.txt", + resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME) + ); + + const result = await runHackCli({ + sandbox: fixture.sandbox, + args: envAddSecretArgs(fixture.primaryRoot), + }); + + expect(result.exitCode).not.toBe(0); + expect(result.stderr).toContain("Refusing project env key"); + expect(await Bun.file(leakPath).exists()).toBe(false); + expect( + await Bun.file( + resolve(fixture.primaryRoot, ".hack", "hack.env.default.yaml") + ).exists() + ).toBe(false); + expectNoPrivateText(result); +}); + +test("env add --secret refuses a key symlink to a tracked file without disclosing it", async () => { + const fixture = await createWorktreeFixture(); + const trackedPath = resolve(fixture.primaryRoot, "committed-key.txt"); + await writeFile(trackedPath, `${KEY_SENTINEL}\n`); + await runGit(["add", "committed-key.txt"], fixture.primaryRoot); + await runGit(["commit", "-m", "track key text"], fixture.primaryRoot); + await symlink( + "committed-key.txt", + resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME) + ); + + const result = await runHackCli({ + sandbox: fixture.sandbox, + args: envAddSecretArgs(fixture.primaryRoot), + }); + + expect(result.exitCode).not.toBe(0); + expect(result.stderr).toContain("Refusing project env key"); + expect(await readFile(trackedPath, "utf8")).toBe(`${KEY_SENTINEL}\n`); + expectNoPrivateText(result); +}); + +test("env add --secret refuses to encrypt with a committed key and keeps the key text out of output", async () => { + const fixture = await createWorktreeFixture(); + await writeFile( + resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME), + `${KEY_SENTINEL}\n`, + { mode: 0o600 } + ); + await commitKeyFile(fixture.primaryRoot); + + const result = await runHackCli({ + sandbox: fixture.sandbox, + args: envAddSecretArgs(fixture.primaryRoot), + }); + + expect(result.exitCode).not.toBe(0); + expect(result.stderr).toContain("Refusing to encrypt with a git-tracked"); + expect( + await Bun.file( + resolve(fixture.primaryRoot, ".hack", "hack.env.default.yaml") + ).exists() + ).toBe(false); + expectNoPrivateText(result); +}); + +test("a linked worktree refuses a committed primary key before copying it to the shared location", async () => { + const fixture = await createWorktreeFixture(); + await writeFile( + resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME), + `${KEY_SENTINEL}\n` + ); + await commitKeyFile(fixture.primaryRoot); + + const error = await ensureProjectEnvSecretKey({ + projectRoot: fixture.linkedRoot, + }).then( + () => null, + (caught: unknown) => caught + ); + + expect(error).toBeInstanceOf(ProjectEnvKeyRefusalError); + expect(String(error)).not.toContain(KEY_SENTINEL); + const location = await resolveProjectEnvSharedKeyLocation({ + projectRoot: fixture.linkedRoot, + }); + expect(location).not.toBeNull(); + expect(await Bun.file(location?.path ?? "").exists()).toBe(false); +}); + +// v4 compatibility choice: decrypting with a tracked key keeps working (hack +// doctor reports the committed key); only encryption refuses it. +test("decrypting with a git-tracked key stays allowed for v4 compatibility while encryption refuses", async () => { + const fixture = await createWorktreeFixture(); + const projectDir = resolve(fixture.primaryRoot, ".hack"); + await setProjectEnvValue({ + projectRoot: fixture.primaryRoot, + projectDir, + envName: null, + scope: "api", + key: "API_TOKEN", + value: VALUE_SENTINEL, + secret: true, + }); + await commitKeyFile(fixture.primaryRoot); + + const resolved = await resolveProjectEnvConfig({ + projectRoot: fixture.primaryRoot, + projectDir, + envName: null, + serviceNames: ["api"], + }); + expect(resolved?.serviceEnv.api?.API_TOKEN).toBe(VALUE_SENTINEL); + + await expect( + setProjectEnvValue({ + projectRoot: fixture.primaryRoot, + projectDir, + envName: null, + scope: "api", + key: "OTHER_TOKEN", + value: VALUE_SENTINEL, + secret: true, + }) + ).rejects.toThrow("Refusing to encrypt with a git-tracked"); +}); + +test("a new checkout-local key is created 0600 even under a restrictive umask", async () => { + const fixture = await createWorktreeFixture(); + const previousUmask = process.umask(0o277); + let ensured: Awaited>; + try { + ensured = await ensureProjectEnvSecretKey({ + projectRoot: fixture.primaryRoot, + }); + } finally { + process.umask(previousUmask); + } + expect(ensured.created).toBe(true); + expect(ensured.keyPath).toBe( + resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME) + ); + const info = await lstat(ensured.keyPath); + expect(info.isFile()).toBe(true); + expect(info.mode & 0o777).toBe(0o600); + expect((await readFile(ensured.keyPath, "utf8")).trim()).toBe( + ensured.keyText + ); +}); + +test("concurrent key creation across checkouts converges on one shared key", async () => { + const fixture = await createWorktreeFixture(); + const results = await Promise.all([ + ensureProjectEnvSecretKey({ projectRoot: fixture.linkedRoot }), + ensureProjectEnvSecretKey({ projectRoot: fixture.linkedRoot }), + ]); + const location = await resolveProjectEnvSharedKeyLocation({ + projectRoot: fixture.linkedRoot, + }); + const stored = (await readFile(location?.path ?? "", "utf8")).trim(); + for (const result of results) { + expect(result.keyPath).toBe(location?.path ?? ""); + expect(result.keyText).toBe(stored); + } + expect(results.filter((result) => result.created).length).toBe(1); +}); + +test("a whitespace-only key file is refused instead of being replaced", async () => { + const fixture = await createWorktreeFixture(); + const keyPath = resolve(fixture.primaryRoot, PROJECT_ENV_KEY_FILENAME); + await writeFile(keyPath, "\n"); + + await expect( + ensureProjectEnvSecretKey({ projectRoot: fixture.primaryRoot }) + ).rejects.toBeInstanceOf(ProjectEnvKeyRefusalError); + expect(await readFile(keyPath, "utf8")).toBe("\n"); +}); + +// Negative control: replacing createPrivateFileExclusive with Bun.write makes +// this test fail, because Bun.write follows the dangling link and creates it. +test("exclusive private-file creation never follows a dangling symlink", async () => { + const sandbox = await mkdtemp(join(tmpdir(), "hack-private-file-")); + tempDirs.add(sandbox); + const leakPath = resolve(sandbox, "leak.txt"); + const linkPath = resolve(sandbox, PROJECT_ENV_KEY_FILENAME); + await symlink("leak.txt", linkPath); + + await expect( + createPrivateFileExclusive({ path: linkPath, text: `${KEY_SENTINEL}\n` }) + ).rejects.toMatchObject({ code: "EEXIST" }); + expect(await Bun.file(leakPath).exists()).toBe(false); + expect((await lstat(linkPath)).isSymbolicLink()).toBe(true); +}); diff --git a/tests/secret-store.test.ts b/tests/secret-store.test.ts index 0a41f02c..6db50d88 100644 --- a/tests/secret-store.test.ts +++ b/tests/secret-store.test.ts @@ -300,6 +300,158 @@ test("encrypted_file backend returns recovery guidance when key material is miss await expect(access(keyPath)).rejects.toThrow(); }); +const STORE_KEY = "synthetic-store-key-for-tests-only"; +const STORE_SENTINEL = "synthetic-store-value-never-in-errors"; + +type StoreEnvelope = { + readonly version: number; + readonly algorithm: string; + readonly iv: string; + readonly tag: string; + readonly ciphertext: string; +}; + +/** Writes one sentinel entry through the real store and returns its envelope. */ +async function encryptedStoreFixture(): Promise<{ + readonly storePath: string; + readonly envelope: StoreEnvelope; +}> { + if (!tempDir) { + throw new Error("Missing temp dir"); + } + process.env.HACK_SECRETS_FILE_KEY = STORE_KEY; + const storePath = join(tempDir, "secrets.enc.json"); + await writeGlobalConfig({ + value: { + controlPlane: { + secrets: { + backend: "encrypted_file", + encryptedFile: { path: storePath }, + }, + }, + }, + }); + const store = await resolveSecretStore({ projectName: "app" }); + await store.set({ key: "TOKEN", value: STORE_SENTINEL }); + await store.set({ key: "EMPTY", value: "" }); + const envelope = JSON.parse( + await readFile(storePath, "utf8") + ) as StoreEnvelope; + return { storePath, envelope }; +} + +function reencode( + envelope: StoreEnvelope, + field: "iv" | "tag" | "ciphertext", + change: (bytes: Buffer) => Buffer +): StoreEnvelope { + return { + ...envelope, + [field]: change(Buffer.from(envelope[field], "base64")).toString("base64"), + }; +} + +const STORE_MUTATIONS: ReadonlyArray< + readonly [string, (envelope: StoreEnvelope) => StoreEnvelope] +> = [ + ...[4, 8, 12].map( + (bytes) => + [ + `${bytes}-byte tag`, + (envelope: StoreEnvelope) => + reencode(envelope, "tag", (tag) => tag.subarray(0, bytes)), + ] as const + ), + [ + "16-byte IV", + (envelope) => + reencode(envelope, "iv", (iv) => Buffer.concat([iv, Buffer.alloc(4)])), + ], + [ + "base64 with !", + (envelope) => ({ + ...envelope, + ciphertext: `!${envelope.ciphertext.slice(1)}`, + }), + ], + [ + "base64 with whitespace", + (envelope) => ({ + ...envelope, + tag: `${envelope.tag.slice(0, 4)}\n${envelope.tag.slice(4)}`, + }), + ], + [ + "truncated ciphertext", + (envelope) => + reencode(envelope, "ciphertext", (ciphertext) => + ciphertext.subarray(0, -1) + ), + ], + [ + "bit-flipped ciphertext", + (envelope) => + reencode(envelope, "ciphertext", (ciphertext) => { + const flipped = Buffer.from(ciphertext); + flipped[0] = (flipped[0] ?? 0) ^ 1; + return flipped; + }), + ], + [ + "oversized ciphertext", + (envelope) => ({ + ...envelope, + ciphertext: Buffer.alloc(16 * 1024 * 1024 + 1, 7).toString("base64"), + }), + ], +]; + +for (const [name, mutate] of STORE_MUTATIONS) { + test(`encrypted_file envelope with ${name} refuses with a fixed redacted error`, async () => { + const { storePath, envelope } = await encryptedStoreFixture(); + const mutated = mutate(envelope); + expect(mutated).not.toEqual(envelope); + await writeFile(storePath, `${JSON.stringify(mutated)}\n`); + + const store = await resolveSecretStore({ projectName: "app" }); + const error = await store.get({ key: "TOKEN" }).then( + () => null, + (caught: unknown) => caught + ); + + expect(error).toBeInstanceOf(Error); + const message = error instanceof Error ? error.message : ""; + expect(message).toMatch( + /^(Invalid encrypted secret store payload|Cannot decrypt encrypted secret store) at / + ); + for (const privateText of [STORE_SENTINEL, STORE_KEY]) { + expect(String(error)).not.toContain(privateText); + } + }); +} + +test("encrypted_file envelope with the wrong key refuses with a fixed redacted error", async () => { + await encryptedStoreFixture(); + process.env.HACK_SECRETS_FILE_KEY = "synthetic-wrong-store-key"; + const store = await resolveSecretStore({ projectName: "app" }); + const error = await store.get({ key: "TOKEN" }).then( + () => null, + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Error); + expect(String(error)).toContain("Cannot decrypt encrypted secret store at "); + expect(String(error)).not.toContain(STORE_SENTINEL); +}); + +test("encrypted_file v1 envelope round-trips, including the empty value", async () => { + const { envelope } = await encryptedStoreFixture(); + expect(Buffer.from(envelope.iv, "base64").length).toBe(12); + expect(Buffer.from(envelope.tag, "base64").length).toBe(16); + const store = await resolveSecretStore({ projectName: "app" }); + expect(await store.get({ key: "TOKEN" })).toBe(STORE_SENTINEL); + expect(await store.get({ key: "EMPTY" })).toBe(""); +}); + async function writeGlobalConfig(input: { readonly value: Record; }): Promise { From ffeecce2c17b41c301fc9d4dcaf9664cda7cb148 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:50:29 -0400 Subject: [PATCH 04/52] fix(agents): generate capability matrix doc and parse all guidance config Address verifier findings on the agent-client capability matrix: - Move the matrix renderer into scripts/generate-agent-client-capabilities.ts and add `bun run docs:agent-client-capabilities` (with `--check`), which rewrites only the generated section. The test compares the committed section with the rendered one and covers marker failures with closed codes. - Scope the model/permission policy scan to generated guidance surfaces (plugins/hack, .claude-plugin, .cursor-plugin, .agents through its skill symlinks, and the example agent configuration) instead of example compose files. Parse YAML and markdown frontmatter as YAML, fail closed on structured-config formats the scan cannot parse, and count a file as scanned only when it was parsed. - Treat reasoning-effort keys (reasoning_effort, model_reasoning_effort, frontmatter `effort`, nested effort/reasoning keys) as forbidden model settings, with negative controls. Co-Authored-By: Claude Opus 5.5 --- docs/agent-guidance.md | 12 +- docs/reference/agent-client-capabilities.md | 28 ++- package.json | 1 + scripts/generate-agent-client-capabilities.ts | 219 +++++++++++++++++ src/agents/client-capabilities.ts | 3 +- tests/agent-client-capabilities.test.ts | 179 ++++++-------- tests/agent-plugin-generation.test.ts | 230 +++++++++++++++--- 7 files changed, 515 insertions(+), 157 deletions(-) create mode 100644 scripts/generate-agent-client-capabilities.ts diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 2e7c1a2e..c6b9cbd2 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -3,10 +3,11 @@ Hack owns Hack-specific instructions. User and project policy continues to own model selection, general coding standards, work-unit planning, evolution, and writeback. Do not ship provider model pins or copy global instruction packs into this plugin. -`tests/agent-plugin-generation.test.ts` enforces this for every file under -`plugins/hack` and `examples/basic`: no model or model-override keys, no permission, -approval or sandbox overrides, no output-style settings, and hooks that run only -`hack agent prime`. See [model-policy ownership](reference/agent-client-capabilities.md#model-policy-ownership). +`tests/agent-plugin-generation.test.ts` enforces this for the generated guidance +surfaces (`plugins/hack`, the client marketplace manifests, `.agents`, and the +example agent configuration): no model, model-override or reasoning-effort keys, no +permission, approval or sandbox overrides, no output-style settings, and hooks that +run only `hack agent prime`. See [model-policy ownership](reference/agent-client-capabilities.md#model-policy-ownership). ## Sources and installed surfaces @@ -48,7 +49,8 @@ belong to their client adapters; shared behavior belongs to the canonical source generation tests. Run normal typecheck/check/test gates for behavior changes. New packaging needs fresh client loading proof beyond a successful marketplace listing. Record that proof, or the open gate, in `src/agents/client-capabilities.ts` and - regenerate the [capability matrix](reference/agent-client-capabilities.md). + run `bun run docs:agent-client-capabilities` to regenerate the + [capability matrix](reference/agent-client-capabilities.md). 5. Release preparation regenerates plugin assets after the version bump. Verify the published bundle and installed executable separately. diff --git a/docs/reference/agent-client-capabilities.md b/docs/reference/agent-client-capabilities.md index 638c81db..8c408b2c 100644 --- a/docs/reference/agent-client-capabilities.md +++ b/docs/reference/agent-client-capabilities.md @@ -2,8 +2,10 @@ This page classifies what Hack's optional agent integrations are known to do in each client. The data lives in `src/agents/client-capabilities.ts`. -`tests/agent-client-capabilities.test.ts` checks the generated section below -against that data and enforces the classification rules. +`scripts/generate-agent-client-capabilities.ts` renders the generated section +below from that data. `tests/agent-client-capabilities.test.ts` fails when the +committed section differs from the rendered one, and enforces the classification +rules. ## Classification rules @@ -30,9 +32,11 @@ cover other machines or later client versions. A status for one client implies nothing about another client, including UI parity. To reclassify a cell, update the data, increment -`AGENT_CLIENT_CAPABILITY_MATRIX_VERSION`, and regenerate the section below so the -test passes. Promote a cell to `verified` only after its named test passes against -the recorded client version. +`AGENT_CLIENT_CAPABILITY_MATRIX_VERSION`, and run +`bun run docs:agent-client-capabilities` to rewrite the section below. Add +`-- --check` to report a stale section without writing. Edit only the prose +outside the generated markers by hand. Promote a cell to `verified` only after its +named test passes against the recorded client version. @@ -133,11 +137,17 @@ Decisions: The user's own provider configuration and project policy own these settings. Hack artifacts must not set them: -- model selection, model overrides and reasoning effort +- model selection, model overrides and reasoning effort (for example + `model_reasoning_effort`, `reasoning_effort`, or `effort` in skill frontmatter) - permission modes, tool allowlists, approval policies and sandbox modes - output styles Hack-shipped hooks may run only `hack agent prime`. -`tests/agent-plugin-generation.test.ts` enforces this for every file under -`plugins/hack` and `examples/basic`. It includes a negative control that proves the -check rejects each forbidden setting. See [maintaining agent guidance](../agent-guidance.md). +`tests/agent-plugin-generation.test.ts` enforces this for the generated guidance +surfaces: `plugins/hack`, `.claude-plugin`, `.cursor-plugin`, `.agents` (including +the skills it links to), and the agent configuration in `examples/basic/.claude`, +`.codex` and `.cursor`. It parses JSON, JSON Lines, TOML, YAML and markdown +frontmatter. It fails on structured-config formats it cannot parse, such as INI, +JSONC, XML or `.env`, and counts a file as scanned only when it parsed it. +A negative control proves the check rejects each forbidden setting. See +[maintaining agent guidance](../agent-guidance.md). diff --git a/package.json b/package.json index 73053cac..02d64946 100644 --- a/package.json +++ b/package.json @@ -72,6 +72,7 @@ "test:e2e:dispatch": "bun run scripts/dispatch-e2e.ts", "test:e2e:railway": "bun run scripts/railway-node-e2e.ts", "docs:cli-reference": "bun run scripts/generate-cli-reference.ts", + "docs:agent-client-capabilities": "bun run scripts/generate-agent-client-capabilities.ts", "generate:agent-plugins": "bun run scripts/generate-agent-plugins.ts" }, "devDependencies": { diff --git a/scripts/generate-agent-client-capabilities.ts b/scripts/generate-agent-client-capabilities.ts new file mode 100644 index 00000000..2ad6a7af --- /dev/null +++ b/scripts/generate-agent-client-capabilities.ts @@ -0,0 +1,219 @@ +import { resolve } from "node:path"; + +import { + AGENT_CLIENT_CAPABILITIES, + AGENT_CLIENT_CAPABILITY_DEFINITIONS, + AGENT_CLIENT_CAPABILITY_MATRIX, + AGENT_CLIENT_CAPABILITY_MATRIX_VERSION, + AGENT_CLIENT_CAPABILITY_OBSERVED_ON, + AGENT_CLIENT_DEFINITIONS, + AGENT_CLIENTS, + type AgentClient, + type AgentClientCapability, + type ClientCapabilityCell, +} from "../src/agents/client-capabilities.ts"; + +/** + * Renders the generated section of docs/reference/agent-client-capabilities.md + * from src/agents/client-capabilities.ts. + * + * Run `bun run docs:agent-client-capabilities` to rewrite the section in place, + * or add `--check` to exit non-zero when the committed section is stale. + * tests/agent-client-capabilities.test.ts fails on the same drift. Only the + * text between the markers is generated; the surrounding prose is hand-written. + */ + +export const AGENT_CLIENT_CAPABILITIES_REFERENCE_PATH = + "docs/reference/agent-client-capabilities.md"; +export const GENERATED_BEGIN = + ""; +export const GENERATED_END = + ""; + +/** Closed failure codes; messages never include document content or paths. */ +export type GeneratedSectionError = + | "missing-markers" + | "duplicate-markers" + | "markers-out-of-order"; + +const GENERATED_SECTION_ERROR_MESSAGES: Readonly< + Record +> = { + "missing-markers": "The reference doc is missing a generated-section marker.", + "duplicate-markers": + "The reference doc has more than one generated-section marker pair.", + "markers-out-of-order": + "The reference doc's generated-section end marker precedes its start marker.", +}; + +export class GeneratedSectionMarkerError extends Error { + readonly code: GeneratedSectionError; + + constructor(code: GeneratedSectionError) { + super(GENERATED_SECTION_ERROR_MESSAGES[code]); + this.name = "GeneratedSectionMarkerError"; + this.code = code; + } +} + +function escapeCell(value: string): string { + return value.replaceAll("|", "\\|").replaceAll("\n", " "); +} + +function clientLabel(client: AgentClient): string { + return ( + AGENT_CLIENT_DEFINITIONS.find((definition) => definition.client === client) + ?.label ?? client + ); +} + +function capabilityLabel(capability: AgentClientCapability): string { + return ( + AGENT_CLIENT_CAPABILITY_DEFINITIONS.find( + (definition) => definition.capability === capability + )?.label ?? capability + ); +} + +function renderCellRow(cell: ClientCapabilityCell): string { + return [ + "", + cell.client, + capabilityLabel(cell.capability), + cell.status, + cell.evidenceClass, + `\`${cell.evidenceRef}\``, + escapeCell(cell.observedClientVersion ?? "none"), + `\`${cell.hostLabel}\``, + `\`${cell.sourceSha.slice(0, 12)}\``, + escapeCell(cell.note), + escapeCell(cell.openGate ?? "none"), + "", + ] + .join(" | ") + .trim(); +} + +/** Renders the generated section, markers included, from the given cells. */ +export function renderAgentClientCapabilitySection({ + cells = AGENT_CLIENT_CAPABILITY_MATRIX, +}: { + readonly cells?: readonly ClientCapabilityCell[]; +} = {}): string { + const lines = [ + GENERATED_BEGIN, + "", + `Matrix version ${AGENT_CLIENT_CAPABILITY_MATRIX_VERSION}, observed ${AGENT_CLIENT_CAPABILITY_OBSERVED_ON}.`, + "", + "### Clients", + "", + "| Client | Manifest |", + "| --- | --- |", + ...AGENT_CLIENT_DEFINITIONS.map( + (definition) => + `| ${escapeCell(definition.label)} | \`${definition.manifestPath}\` |` + ), + "", + "### Capabilities", + "", + "| Capability | Meaning |", + "| --- | --- |", + ...AGENT_CLIENT_CAPABILITY_DEFINITIONS.map( + (definition) => + `| ${definition.label} | ${escapeCell(definition.meaning)} |` + ), + "", + "### Status summary", + "", + `| Capability | ${AGENT_CLIENTS.map(clientLabel).map(escapeCell).join(" | ")} |`, + `| --- | ${AGENT_CLIENTS.map(() => "---").join(" | ")} |`, + ...AGENT_CLIENT_CAPABILITIES.map((capability) => { + const statuses = AGENT_CLIENTS.map( + (client) => + cells.find( + (cell) => cell.client === client && cell.capability === capability + )?.status ?? "missing" + ); + return `| ${capabilityLabel(capability)} | ${statuses.join(" | ")} |`; + }), + "", + "### Cells", + "", + "| Client | Capability | Status | Evidence class | Evidence | Client version | Host | Source | Note | Open gate |", + "| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |", + ...cells.map(renderCellRow), + "", + GENERATED_END, + ]; + return lines.join("\n"); +} + +function markerRange(document: string): { start: number; end: number } { + const start = document.indexOf(GENERATED_BEGIN); + const end = document.indexOf(GENERATED_END); + if (start < 0 || end < 0) { + throw new GeneratedSectionMarkerError("missing-markers"); + } + if ( + document.indexOf(GENERATED_BEGIN, start + 1) >= 0 || + document.indexOf(GENERATED_END, end + 1) >= 0 + ) { + throw new GeneratedSectionMarkerError("duplicate-markers"); + } + if (end < start) { + throw new GeneratedSectionMarkerError("markers-out-of-order"); + } + return { start, end: end + GENERATED_END.length }; +} + +/** Returns the committed generated section, markers included. */ +export function extractGeneratedSection(document: string): string { + const { start, end } = markerRange(document); + return document.slice(start, end); +} + +/** Replaces the generated section, keeping all hand-written prose intact. */ +export function replaceGeneratedSection({ + document, + section, +}: { + readonly document: string; + readonly section: string; +}): string { + const { start, end } = markerRange(document); + return document.slice(0, start) + section + document.slice(end); +} + +async function main(): Promise { + const check = process.argv.slice(2).includes("--check"); + const docPath = resolve( + import.meta.dir, + "..", + AGENT_CLIENT_CAPABILITIES_REFERENCE_PATH + ); + const document = await Bun.file(docPath).text(); + const next = replaceGeneratedSection({ + document, + section: renderAgentClientCapabilitySection(), + }); + if (check) { + if (next !== document) { + process.stderr.write( + "The agent client capability reference is stale. Run `bun run docs:agent-client-capabilities`.\n" + ); + process.exit(1); + } + process.stdout.write("agent client capability reference is current\n"); + return; + } + if (next === document) { + process.stdout.write("agent client capability reference is current\n"); + return; + } + await Bun.write(docPath, next); + process.stdout.write(`wrote ${AGENT_CLIENT_CAPABILITIES_REFERENCE_PATH}\n`); +} + +if (import.meta.main) { + await main(); +} diff --git a/src/agents/client-capabilities.ts b/src/agents/client-capabilities.ts index 288b8da8..657ee65f 100644 --- a/src/agents/client-capabilities.ts +++ b/src/agents/client-capabilities.ts @@ -9,7 +9,8 @@ * marketplace listing is never component-loading evidence. * - For the named native-plugin clients, a component is `unsupported` exactly * when that client's plugin manifest does not declare it. - * - docs/reference/agent-client-capabilities.md matches this data. + * - docs/reference/agent-client-capabilities.md matches this data; rewrite its + * generated section with `bun run docs:agent-client-capabilities`. * * Each cell records the client version, host label and source revision at * which it was classified. Those values describe one observation on one host; diff --git a/tests/agent-client-capabilities.test.ts b/tests/agent-client-capabilities.test.ts index 4c5691c3..8b1653ea 100644 --- a/tests/agent-client-capabilities.test.ts +++ b/tests/agent-client-capabilities.test.ts @@ -2,12 +2,18 @@ import { expect, test } from "bun:test"; import { existsSync } from "node:fs"; import { join } from "node:path"; +import { + AGENT_CLIENT_CAPABILITIES_REFERENCE_PATH, + extractGeneratedSection, + GENERATED_BEGIN, + GENERATED_END, + GeneratedSectionMarkerError, + renderAgentClientCapabilitySection, + replaceGeneratedSection, +} from "../scripts/generate-agent-client-capabilities.ts"; import { AGENT_CLIENT_CAPABILITIES, - AGENT_CLIENT_CAPABILITY_DEFINITIONS, AGENT_CLIENT_CAPABILITY_MATRIX, - AGENT_CLIENT_CAPABILITY_MATRIX_VERSION, - AGENT_CLIENT_CAPABILITY_OBSERVED_ON, AGENT_CLIENT_DEFINITIONS, AGENT_CLIENTS, type AgentClient, @@ -19,12 +25,7 @@ import { import { isRecord } from "../src/lib/guards.ts"; const repoRoot = join(import.meta.dir, ".."); -const referencePath = join( - repoRoot, - "docs", - "reference", - "agent-client-capabilities.md" -); +const referencePath = join(repoRoot, AGENT_CLIENT_CAPABILITIES_REFERENCE_PATH); type IssueCode = | "missing-cell" @@ -206,94 +207,6 @@ function codesFor( .map((issue) => issue.code); } -function escapeCell(value: string): string { - return value.replaceAll("|", "\\|").replaceAll("\n", " "); -} - -function clientLabel(client: AgentClient): string { - return ( - AGENT_CLIENT_DEFINITIONS.find((definition) => definition.client === client) - ?.label ?? client - ); -} - -function capabilityLabel(capability: AgentClientCapability): string { - return ( - AGENT_CLIENT_CAPABILITY_DEFINITIONS.find( - (definition) => definition.capability === capability - )?.label ?? capability - ); -} - -const GENERATED_BEGIN = ""; -const GENERATED_END = ""; - -function renderGeneratedSection(): string { - const lines = [ - GENERATED_BEGIN, - "", - `Matrix version ${AGENT_CLIENT_CAPABILITY_MATRIX_VERSION}, observed ${AGENT_CLIENT_CAPABILITY_OBSERVED_ON}.`, - "", - "### Clients", - "", - "| Client | Manifest |", - "| --- | --- |", - ...AGENT_CLIENT_DEFINITIONS.map( - (definition) => - `| ${escapeCell(definition.label)} | \`${definition.manifestPath}\` |` - ), - "", - "### Capabilities", - "", - "| Capability | Meaning |", - "| --- | --- |", - ...AGENT_CLIENT_CAPABILITY_DEFINITIONS.map( - (definition) => - `| ${definition.label} | ${escapeCell(definition.meaning)} |` - ), - "", - "### Status summary", - "", - `| Capability | ${AGENT_CLIENTS.map(clientLabel).map(escapeCell).join(" | ")} |`, - `| --- | ${AGENT_CLIENTS.map(() => "---").join(" | ")} |`, - ...AGENT_CLIENT_CAPABILITIES.map((capability) => { - const statuses = AGENT_CLIENTS.map( - (client) => - AGENT_CLIENT_CAPABILITY_MATRIX.find( - (cell) => cell.client === client && cell.capability === capability - )?.status ?? "missing" - ); - return `| ${capabilityLabel(capability)} | ${statuses.join(" | ")} |`; - }), - "", - "### Cells", - "", - "| Client | Capability | Status | Evidence class | Evidence | Client version | Host | Source | Note | Open gate |", - "| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |", - ...AGENT_CLIENT_CAPABILITY_MATRIX.map((cell) => - [ - "", - cell.client, - capabilityLabel(cell.capability), - cell.status, - cell.evidenceClass, - `\`${cell.evidenceRef}\``, - escapeCell(cell.observedClientVersion ?? "none"), - `\`${cell.hostLabel}\``, - `\`${cell.sourceSha.slice(0, 12)}\``, - escapeCell(cell.note), - escapeCell(cell.openGate ?? "none"), - "", - ] - .join(" | ") - .trim() - ), - "", - GENERATED_END, - ]; - return lines.join("\n"); -} - test("every client and capability pair is classified with complete evidence", async () => { expect(AGENT_CLIENT_CAPABILITY_MATRIX).toHaveLength( AGENT_CLIENTS.length * AGENT_CLIENT_CAPABILITIES.length @@ -444,15 +357,18 @@ test("manifest-declared components agree with the matrix", async () => { ).toEqual(["open-gate-missing"]); }); -test("reference doc matches the matrix and records ownership decisions", async () => { +test("committed reference section matches the generator output", async () => { const doc = await Bun.file(referencePath).text(); - const start = doc.indexOf(GENERATED_BEGIN); - const end = doc.indexOf(GENERATED_END); - expect(start).toBeGreaterThanOrEqual(0); - expect(end).toBeGreaterThan(start); - expect(doc.slice(start, end + GENERATED_END.length)).toBe( - renderGeneratedSection() + expect(extractGeneratedSection(doc)).toBe( + renderAgentClientCapabilitySection() ); + expect( + replaceGeneratedSection({ + document: doc, + section: renderAgentClientCapabilitySection(), + }) + ).toBe(doc); + expect(doc).toContain("bun run docs:agent-client-capabilities"); expect(doc).toContain("## Companion-package layout decision"); expect(doc).toContain("## Model-policy ownership"); for (const owner of [ @@ -462,4 +378,59 @@ test("reference doc matches the matrix and records ownership decisions", async ( ]) { expect(doc).toContain(owner); } + const packageJson: unknown = await Bun.file( + join(repoRoot, "package.json") + ).json(); + const scripts = isRecord(packageJson) ? packageJson.scripts : undefined; + expect( + isRecord(scripts) ? scripts["docs:agent-client-capabilities"] : undefined + ).toBe("bun run scripts/generate-agent-client-capabilities.ts"); +}); + +test("negative control: a reclassified cell makes the committed section stale", async () => { + const doc = await Bun.file(referencePath).text(); + const reclassified = replaceCell("claude", "skillDiscovery", { + status: "verified", + openGate: null, + }); + const rendered = renderAgentClientCapabilitySection({ cells: reclassified }); + expect(extractGeneratedSection(doc)).not.toBe(rendered); + const rewritten = replaceGeneratedSection({ + document: doc, + section: rendered, + }); + expect(extractGeneratedSection(rewritten)).toBe(rendered); + expect(rewritten.slice(0, doc.indexOf(GENERATED_BEGIN))).toBe( + doc.slice(0, doc.indexOf(GENERATED_BEGIN)) + ); + expect(rewritten.slice(rewritten.indexOf(GENERATED_END))).toBe( + doc.slice(doc.indexOf(GENERATED_END)) + ); +}); + +test("generator refuses documents without exactly one ordered marker pair", () => { + const section = renderAgentClientCapabilitySection(); + const codeFor = (document: string): string | null => { + try { + replaceGeneratedSection({ document, section }); + return null; + } catch (error) { + return error instanceof GeneratedSectionMarkerError + ? error.code + : "other"; + } + }; + expect(codeFor("# Title\n")).toBe("missing-markers"); + expect(codeFor(`${GENERATED_BEGIN}\n`)).toBe("missing-markers"); + expect( + codeFor( + `${GENERATED_BEGIN}\n${GENERATED_END}\n${GENERATED_BEGIN}\n${GENERATED_END}\n` + ) + ).toBe("duplicate-markers"); + expect(codeFor(`${GENERATED_END}\n${GENERATED_BEGIN}\n`)).toBe( + "markers-out-of-order" + ); + expect(codeFor(`a\n${GENERATED_BEGIN}\nold\n${GENERATED_END}\nb\n`)).toBe( + null + ); }); diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 38e4a30a..c5560227 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -2,6 +2,7 @@ import { afterEach, expect, test } from "bun:test"; import { cp, mkdir, mkdtemp, readdir, rm, symlink } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { parse as parseYaml } from "yaml"; import { generateAgentPlugins, @@ -226,16 +227,60 @@ interface PolicyFinding { | "permission-setting" | "output-style-setting" | "frontmatter-hooks" - | "hook-command"; + | "hook-command" + | "unparsed-config"; +} + +/** How the scan treated one file. Only `parsed` files count as scanned. */ +type PolicyScanKind = "parsed" | "not-config"; + +interface PolicyScan { + readonly kind: PolicyScanKind; + readonly findings: readonly PolicyFinding[]; } const HOOK_COMMAND = "hack agent prime"; -/** Model, permission and output-style policy belongs to the user's provider - * configuration, never to Hack-shipped artifacts. */ +/** + * Generated agent guidance surfaces: the native plugin bundle, the client + * marketplace manifests, the shared `.agents` tree (skills reach it through + * symlinks) and the agent configuration in the basic example. Example compose + * and runtime files are application configuration, not agent guidance. + */ +const GUIDANCE_SURFACES = [ + "plugins/hack", + ".claude-plugin", + ".cursor-plugin", + ".agents", + "examples/basic/.claude", + "examples/basic/.codex", + "examples/basic/.cursor", +] as const; + +/** Structured-config formats the scan cannot parse. Shipping one fails the + * scan instead of counting it as inspected. */ +const UNPARSED_CONFIG_EXTENSIONS = new Set([ + "cfg", + "conf", + "env", + "ini", + "json5", + "jsonc", + "plist", + "properties", + "xml", +]); + +/** Model, reasoning-effort, permission and output-style policy belongs to the + * user's provider configuration, never to Hack-shipped artifacts. */ function forbiddenSettingCode(key: string): PolicyFinding["code"] | null { const normalized = key.toLowerCase().replace(/[^a-z]/g, ""); - if (normalized.startsWith("model") || normalized.endsWith("model")) { + if ( + normalized.startsWith("model") || + normalized.endsWith("model") || + normalized.includes("effort") || + normalized.startsWith("reasoning") + ) { return "model-setting"; } if ( @@ -295,65 +340,110 @@ function structuredPolicyFindings( return findings; } -function frontmatterKeys(text: string): string[] { - const match = /^---\n([\s\S]*?)\n---/.exec(text); - if (!match?.[1]) { - return []; - } - return match[1] - .split("\n") - .map((line) => /^([A-Za-z0-9_-]+):/.exec(line)?.[1]) - .filter((key): key is string => key !== undefined); +const FRONTMATTER = /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/; + +/** Parses markdown frontmatter as YAML; returns null when there is none. */ +function frontmatter(text: string): unknown { + const match = FRONTMATTER.exec(text); + return match ? parseYaml(match[1] ?? "") : null; +} + +function extensionOf(name: string): string { + const dot = name.lastIndexOf("."); + return dot > 0 ? name.slice(dot + 1).toLowerCase() : ""; } /** Classify one shipped file with closed codes; never report file content. */ -function policyFindings(file: string, text: string): PolicyFinding[] { +function policyScan(file: string, text: string): PolicyScan { const findings: PolicyFinding[] = []; const name = file.split("/").at(-1) ?? file; if (file.split("/").some((part) => /^output[-_]?styles?$/i.test(part))) { findings.push({ file, code: "output-style-setting" }); } - if (name.endsWith(".json")) { - findings.push(...structuredPolicyFindings(file, JSON.parse(text))); - } else if (name.endsWith(".jsonl")) { - for (const line of text.split("\n").filter((entry) => entry.trim())) { - findings.push(...structuredPolicyFindings(file, JSON.parse(line))); + const extension = extensionOf(name); + let parsed: unknown = null; + let kind: PolicyScanKind = "parsed"; + if (extension === "json") { + parsed = JSON.parse(text); + } else if (extension === "jsonl") { + parsed = text + .split("\n") + .filter((line) => line.trim()) + .map((line): unknown => JSON.parse(line)); + } else if (extension === "toml") { + parsed = Bun.TOML.parse(text); + } else if (extension === "yaml" || extension === "yml") { + parsed = parseYaml(text); + } else if (extension === "md" || extension === "mdc") { + parsed = frontmatter(text); + if (parsed === null) { + kind = "not-config"; + } else if (isRecord(parsed) && "hooks" in parsed) { + findings.push({ file, code: "frontmatter-hooks" }); } - } else if (name.endsWith(".toml")) { - findings.push(...structuredPolicyFindings(file, Bun.TOML.parse(text))); - } else if (name.endsWith(".md") || name.endsWith(".mdc")) { - for (const key of frontmatterKeys(text)) { - const code = forbiddenSettingCode(key); - if (code) { - findings.push({ file, code }); - } - if (key.toLowerCase() === "hooks") { - findings.push({ file, code: "frontmatter-hooks" }); - } + } else { + kind = "not-config"; + if ( + UNPARSED_CONFIG_EXTENSIONS.has(extension) || + name.toLowerCase().startsWith(".env") + ) { + findings.push({ file, code: "unparsed-config" }); } } - return findings; + findings.push(...structuredPolicyFindings(file, parsed)); + return { kind, findings }; } -test("plugin bundle and examples never pin models, override permissions or run other hooks", async () => { +function policyFindings(file: string, text: string): PolicyFinding[] { + return [...policyScan(file, text).findings]; +} + +test("generated guidance surfaces never pin models, override permissions or run other hooks", async () => { const findings: PolicyFinding[] = []; - let scanned = 0; - for (const root of ["plugins/hack", "examples/basic"]) { + const parsedFiles: string[] = []; + for (const root of GUIDANCE_SURFACES) { const glob = new Bun.Glob("**/*"); for await (const relative of glob.scan({ cwd: join(repoRoot, root), dot: true, onlyFiles: true, + followSymlinks: true, })) { const file = `${root}/${relative}`; - findings.push( - ...policyFindings(file, await Bun.file(join(repoRoot, file)).text()) + const scan = policyScan( + file, + await Bun.file(join(repoRoot, file)).text() ); - scanned += 1; + findings.push(...scan.findings); + if (scan.kind === "parsed") { + parsedFiles.push(file); + } } } - expect(scanned).toBeGreaterThan(10); expect(findings).toEqual([]); + // Scope guard: every known config surface was actually parsed, including + // YAML and markdown frontmatter reached through the `.agents` symlinks. + for (const file of [ + "plugins/hack/plugin.json", + "plugins/hack/.mcp.json", + "plugins/hack/.codex-plugin/plugin.json", + "plugins/hack/.claude-plugin/plugin.json", + "plugins/hack/.cursor-plugin/plugin.json", + "plugins/hack/hooks/claude-hooks.json", + "plugins/hack/rules/hack.mdc", + "plugins/hack/skills/hack-cli/SKILL.md", + ".claude-plugin/marketplace.json", + ".cursor-plugin/marketplace.json", + ".agents/plugins/marketplace.json", + ".agents/skills/hack-repo-tla/SKILL.md", + ".agents/skills/hack-repo-tla/agents/openai.yaml", + "examples/basic/.claude/settings.json", + "examples/basic/.codex/config.toml", + "examples/basic/.cursor/mcp.json", + ]) { + expect(parsedFiles).toContain(file); + } + expect(parsedFiles.some((file) => file.includes("compose"))).toBe(false); expect( await Bun.file( join(repoRoot, "plugins/hack/hooks/claude-hooks.json") @@ -361,6 +451,35 @@ test("plugin bundle and examples never pin models, override permissions or run o ).toContain(HOOK_COMMAND); }); +test("policy scan counts only parsed files and fails closed on unparsed config", () => { + expect(policyScan("plugins/hack/README.md", "# Hack\n").kind).toBe( + "not-config" + ); + expect(policyScan("skills/x/scripts/run", "#!/bin/sh\n").kind).toBe( + "not-config" + ); + expect(policyScan("skills/x/SKILL.md", "---\nname: x\n---\n").kind).toBe( + "parsed" + ); + expect(policyScan("skills/x/agents/openai.yaml", "a: 1\n").kind).toBe( + "parsed" + ); + for (const file of [ + "plugins/hack/settings.ini", + "plugins/hack/client.conf", + "plugins/hack/config.jsonc", + "plugins/hack/.env", + "plugins/hack/.env.local", + ]) { + const scan = policyScan(file, "model=x\n"); + expect(scan.kind).toBe("not-config"); + expect(scan.findings.map((finding) => finding.code)).toEqual([ + "unparsed-config", + ]); + } + expect(() => policyScan("skills/x/agents/openai.yaml", "a: [\n")).toThrow(); +}); + test("negative control: the policy scan rejects each forbidden setting", () => { const cases: readonly (readonly [string, string, PolicyFinding["code"]])[] = [ ["plugin.json", JSON.stringify({ model: "opus" }), "model-setting"], @@ -371,6 +490,41 @@ test("negative control: the policy scan rejects each forbidden setting", () => { ], ["config.toml", 'model = "gpt-5"\n', "model-setting"], ["config.toml", 'model_reasoning_effort = "high"\n', "model-setting"], + ["config.toml", 'reasoning_effort = "high"\n', "model-setting"], + ["config.toml", 'plan_mode_reasoning_effort = "low"\n', "model-setting"], + [ + "settings.json", + JSON.stringify({ env: { CLAUDE_CODE_EFFORT_LEVEL: "high" } }), + "model-setting", + ], + [ + "settings.json", + JSON.stringify({ reasoning: { effort: "high" } }), + "model-setting", + ], + ["skills/x/SKILL.md", "---\nname: x\neffort: high\n---\n", "model-setting"], + [ + "skills/x/SKILL.md", + "---\nname: x\nmetadata:\n reasoningEffort: high\n---\n", + "model-setting", + ], + ["agents/openai.yaml", "model: gpt-5\n", "model-setting"], + ["agents/openai.yaml", "model_reasoning_effort: high\n", "model-setting"], + [ + "agents/openai.yaml", + "policy:\n approval_policy: never\n", + "permission-setting", + ], + [ + "agents/openai.yml", + "sandbox_mode: danger-full-access\n", + "permission-setting", + ], + [ + "hooks/hooks.yaml", + "hooks:\n Stop:\n - type: command\n command: curl example\n", + "hook-command", + ], ["config.toml", 'approval_policy = "never"\n', "permission-setting"], [ "config.toml", From 56bac3a020b4aae71fbd901b5e127b0053e7a3c1 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:51:10 -0400 Subject: [PATCH 05/52] fix(mcp): make typed refusals value-free and keep unknown-tool in-band Every pre-spawn refusal now comes from a closed catalogue (MCP_REFUSALS): error {code, reason, message} uses an existing code, a closed reason and a fixed message, and stderr repeats that message. Project names, ids, paths and registry values are no longer echoed. Schema refusals name only keys of the tool's own input schema. An unknown tool name returns the SDK's in-band isError text result again instead of a JSON-RPC -32602 protocol error, with a fixed message that omits the requested name. A canary test drives every refusal reason and the unknown-tool path and asserts the canary never appears in the result, with no spawn or audit line. Co-Authored-By: Claude Opus 5.5 --- src/mcp/server.ts | 178 +++++++++++++++++++---------- tests/mcp-session-context.test.ts | 1 + tests/mcp.test.ts | 179 +++++++++++++++++++++++++++++- 3 files changed, 296 insertions(+), 62 deletions(-) diff --git a/src/mcp/server.ts b/src/mcp/server.ts index c4b9b86a..b3dcefcc 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -7,7 +7,6 @@ import { CallToolRequestSchema, type CallToolResult, ErrorCode, - McpError, type ServerNotification, type ServerRequest, type ToolAnnotations, @@ -65,6 +64,73 @@ type HackCommandResult = { /** Existing CLI error codes reused for pre-spawn MCP refusals (no new codes). */ type McpRefusalCode = Extract; +/** + * Closed catalogue of pre-spawn refusals. Each reason maps to one existing + * error code and one fixed message. Messages are value-free: they never repeat + * caller-supplied project names, ids, paths, services or argv, nor registry + * values derived from them. `invalid_arguments` may append the rejected schema + * field names, which are drawn only from the tool's own input schema keys. + */ +export const MCP_REFUSALS = Object.freeze({ + invalid_arguments: { + code: "E_USAGE", + message: "Invalid arguments.", + }, + conflicting_project_selection: { + code: "E_USAGE", + message: "Use only one of projectName, repoRoot, or path.", + }, + invalid_project_name: { + code: "E_USAGE", + message: "Invalid projectName: use a name containing letters or digits.", + }, + ambiguous_project: { + code: "E_USAGE", + message: + "Ambiguous project selection: multiple registrations match. Select by path and inspect 'hack projects' before renaming or pruning an entry.", + }, + project_not_found: { + code: "E_PROJECT_NOT_FOUND", + message: "Unknown project. Run 'hack init' or 'hack projects' first.", + }, + path_not_found: { + code: "E_PROJECT_NOT_FOUND", + message: "Path not found.", + }, + path_not_registered: { + code: "E_PROJECT_NOT_FOUND", + message: "Path is outside registered hack projects.", + }, + no_project_context: { + code: "E_PROJECT_NOT_FOUND", + message: "No project context found. Provide projectName or path.", + }, + init_path_required: { + code: "E_USAGE", + message: "Provide repoRoot or path for initialization.", + }, +} as const satisfies Record< + string, + { readonly code: McpRefusalCode; readonly message: string } +>); + +export type McpRefusalReason = keyof typeof MCP_REFUSALS; + +/** Reasons whose message is exactly the catalogue text. */ +type FixedRefusalReason = Exclude; + +const MCP_REFUSAL_REASONS = Object.keys(MCP_REFUSALS) as [ + McpRefusalReason, + ...McpRefusalReason[], +]; + +/** + * In-band result for a `tools/call` naming an unregistered tool. Matches the + * SDK dispatcher's earlier shape (untyped `isError` text carrying the + * InvalidParams code) but never repeats the requested tool name. + */ +export const MCP_UNKNOWN_TOOL_MESSAGE = `MCP error ${ErrorCode.InvalidParams}: Tool not found`; + type ToolResultPayload = HackCommandResult & { readonly ok: boolean; readonly data?: unknown; @@ -72,17 +138,14 @@ type ToolResultPayload = HackCommandResult & { readonly spawned?: false; readonly error?: { readonly code: McpRefusalCode; + readonly reason: McpRefusalReason; readonly message: string; }; }; type ProjectArgsResolution = | { readonly ok: true; readonly args: string[] } - | { - readonly ok: false; - readonly code: McpRefusalCode; - readonly message: string; - }; + | { readonly ok: false; readonly reason: FixedRefusalReason }; const DEFAULT_CMD_TIMEOUT_MS = 120_000; const DEFAULT_LOG_TAIL_EVENTS = 200; @@ -143,6 +206,7 @@ const toolOutputSchema = { error: z .object({ code: z.enum(["E_USAGE", "E_PROJECT_NOT_FOUND"]), + reason: z.enum(MCP_REFUSAL_REASONS), message: z.string(), }) .optional(), @@ -787,10 +851,7 @@ function registerTools(opts: { handler: async (input, extra) => { const selection = toProjectSelection(input); if (!(selection.path || selection.repoRoot)) { - return buildToolError({ - code: "E_USAGE", - message: "Provide repoRoot or path for initialization.", - }); + return buildToolError({ reason: "init_path_required" }); } const resolved = await resolveProjectArgs({ @@ -863,13 +924,19 @@ function registerHackTool(opts: { }): void { const { tool } = opts; const schema = z.object(tool.inputSchema); + const schemaKeys: ReadonlySet = new Set( + Object.keys(tool.inputSchema) + ); const entry: HackToolEntry = { call: async (args, extra) => { const parsed = await schema.safeParseAsync(args ?? {}); if (!parsed.success) { return buildToolError({ - code: "E_USAGE", - message: describeInvalidArguments(parsed.error), + reason: "invalid_arguments", + message: describeInvalidArguments({ + error: parsed.error, + schemaKeys, + }), }); } return await tool.handler(parsed.data, extra); @@ -897,6 +964,10 @@ function registerHackTool(opts: { * `{error, spawned: false}` refusal. Thrown handler failures and output-schema * mismatches keep the SDK's untyped `isError` text result, because they may * occur after a child was spawned. + * + * An unknown tool name keeps the SDK 1.26 contract: an in-band `isError` text + * result (not a JSON-RPC protocol error), with the fixed + * {@link MCP_UNKNOWN_TOOL_MESSAGE} that omits the requested name. */ function installTypedToolDispatch(opts: { readonly server: McpServer; @@ -908,10 +979,7 @@ function installTypedToolDispatch(opts: { async (request, extra): Promise => { const entry = opts.tools.get(request.params.name); if (!entry) { - throw new McpError( - ErrorCode.InvalidParams, - `Tool ${request.params.name} not found` - ); + return buildUntypedToolError(MCP_UNKNOWN_TOOL_MESSAGE); } try { const result = await entry.call(request.params.arguments, extra); @@ -935,19 +1003,29 @@ function installTypedToolDispatch(opts: { ); } -/** Closed, value-free summary: names the rejected fields, never echoes input. */ -function describeInvalidArguments(error: z.ZodError): string { +/** + * Closed, value-free summary: names the rejected fields, never echoes input. + * Field names are kept only when they are keys of the tool's own schema, so a + * caller-chosen key can never reach the message. + */ +function describeInvalidArguments(opts: { + readonly error: z.ZodError; + readonly schemaKeys: ReadonlySet; +}): string { const fields = [ ...new Set( - error.issues + opts.error.issues .map((issue) => issue.path[0]) - .filter((key): key is string => typeof key === "string") + .filter( + (key): key is string => + typeof key === "string" && opts.schemaKeys.has(key) + ) ), ]; const subject = fields.length > 0 ? `Invalid arguments: ${fields.join(", ")}.` - : "Invalid arguments."; + : MCP_REFUSALS.invalid_arguments.message; return `${subject} Identifiers must match ${COMPOSE_IDENTIFIER_PATTERN.source} (branches may also contain "/"); limits must be positive integers within their documented ceiling.`; } @@ -984,19 +1062,27 @@ function buildToolResult(opts: { * `spawned: false` is a guarantee: every caller returns before * `runHackCommand`/`runHackLogTail`, so no argv, audit line or mutation occurs. * `stderr` repeats the message for clients reading the earlier contract. + * The message is the reason's fixed text from {@link MCP_REFUSALS}; only + * `invalid_arguments` may pass its schema-key-only summary instead. */ -function buildToolError(opts: { - readonly code: McpRefusalCode; - readonly message: string; -}): CallToolResult { +function buildToolError( + opts: + | { readonly reason: FixedRefusalReason } + | { readonly reason: "invalid_arguments"; readonly message: string } +): CallToolResult { + const { code } = MCP_REFUSALS[opts.reason]; + const message = + opts.reason === "invalid_arguments" + ? opts.message + : MCP_REFUSALS[opts.reason].message; const payload: ToolResultPayload = { ok: false, command: "", exitCode: 1, stdout: "", - stderr: opts.message, + stderr: message, spawned: false, - error: { code: opts.code, message: opts.message }, + error: { code, reason: opts.reason, message }, }; return { @@ -1097,11 +1183,7 @@ async function resolveProjectArgs(opts: { selection.path, ].filter((value) => value !== undefined); if (picked.length > 1) { - return { - ok: false, - code: "E_USAGE", - message: "Use only one of projectName, repoRoot, or path.", - }; + return { ok: false, reason: "conflicting_project_selection" }; } if (selection.projectName !== undefined) { @@ -1115,11 +1197,7 @@ async function resolveProjectArgs(opts: { if (pathLike) { const absPath = resolve(opts.cwd, pathLike); if (!(await pathExists(absPath))) { - return { - ok: false, - code: "E_PROJECT_NOT_FOUND", - message: `Path not found: ${absPath}`, - }; + return { ok: false, reason: "path_not_found" }; } if (!opts.allowUnregistered) { const allowed = await isPathAllowed({ @@ -1127,11 +1205,7 @@ async function resolveProjectArgs(opts: { registryPath: opts.registryPath, }); if (!allowed) { - return { - ok: false, - code: "E_PROJECT_NOT_FOUND", - message: "Path is outside registered hack projects.", - }; + return { ok: false, reason: "path_not_registered" }; } } return { ok: true, args: [`--path=${absPath}`] }; @@ -1148,11 +1222,7 @@ async function resolveProjectArgs(opts: { registryPath: opts.registryPath, }); if (!allowed) { - return { - ok: false, - code: "E_PROJECT_NOT_FOUND", - message: "No project context found. Provide projectName or path.", - }; + return { ok: false, reason: "no_project_context" }; } } @@ -1165,11 +1235,7 @@ async function resolveProjectNameArgs(opts: { }): Promise { const name = normalizeProjectName(opts.projectName); if (!name) { - return { - ok: false, - code: "E_USAGE", - message: "Invalid projectName: use a name containing letters or digits.", - }; + return { ok: false, reason: "invalid_project_name" }; } try { const project = await resolveRegisteredProjectByName({ @@ -1177,11 +1243,7 @@ async function resolveProjectNameArgs(opts: { registryPath: opts.registryPath, }); if (!project) { - return { - ok: false, - code: "E_PROJECT_NOT_FOUND", - message: `Unknown project "${opts.projectName}". Run 'hack init' or 'hack projects' first.`, - }; + return { ok: false, reason: "project_not_found" }; } return { ok: true, args: [`--project=${name}`] }; } catch (error) { @@ -1189,7 +1251,7 @@ async function resolveProjectNameArgs(opts: { error instanceof AmbiguousProjectNameError || error instanceof AmbiguousProjectIdError ) { - return { ok: false, code: "E_USAGE", message: error.message }; + return { ok: false, reason: "ambiguous_project" }; } throw error; } diff --git a/tests/mcp-session-context.test.ts b/tests/mcp-session-context.test.ts index d9b47f70..0c6423df 100644 --- a/tests/mcp-session-context.test.ts +++ b/tests/mcp-session-context.test.ts @@ -138,6 +138,7 @@ test("MCP path authorization uses only that session's registry", async () => { spawned: false, error: { code: "E_PROJECT_NOT_FOUND", + reason: "path_not_registered", message: "Path is outside registered hack projects.", }, }); diff --git a/tests/mcp.test.ts b/tests/mcp.test.ts index 1b0227b2..e6570aa6 100644 --- a/tests/mcp.test.ts +++ b/tests/mcp.test.ts @@ -20,6 +20,9 @@ import { buildRunArgs, createMcpServer, MCP_INPUT_CEILINGS, + MCP_REFUSALS, + MCP_UNKNOWN_TOOL_MESSAGE, + type McpRefusalReason, } from "../src/mcp/server.ts"; import { testIntegration } from "./helpers/ci.ts"; @@ -187,12 +190,27 @@ type ContractFixture = { readonly auditLog: string; }; +type RegistryEntryFixture = { + readonly id: string; + readonly name: string; + readonly repoRoot: string; +}; + +type ContractFixtureOptions = { + /** Extra registry rows beside the registered `demo` project. */ + readonly extraProjects?: readonly RegistryEntryFixture[]; + /** Session cwd relative to the fixture root (default: the demo repo). */ + readonly cwdName?: string; +}; + type StubInvocation = { readonly args: readonly string[]; readonly noInteractive: string | null; }; -async function startContractFixture(): Promise { +async function startContractFixture( + opts: ContractFixtureOptions = {} +): Promise { const root = await mkdtemp(join(tmpdir(), "hack-mcp-contract-")); contractRoots.push(root); const project = join(root, "repo"); @@ -219,9 +237,22 @@ async function startContractFixture(): Promise { projectDirName: ".hack", projectDir, }, + ...(opts.extraProjects ?? []).map((entry) => ({ + id: entry.id, + createdAt: "2026-01-01T00:00:00Z", + name: entry.name, + repoRoot: join(root, entry.repoRoot), + projectDirName: ".hack", + projectDir: join(root, entry.repoRoot, ".hack"), + })), ], }) ); + let cwd = project; + if (opts.cwdName !== undefined) { + cwd = join(root, opts.cwdName); + await mkdir(cwd, { recursive: true }); + } const argvLog = join(root, "argv.jsonl"); const stub = join(root, "hack-stub"); await writeFile( @@ -243,14 +274,14 @@ async function startContractFixture(): Promise { HACK_MCP_COMMAND: stub, }; Reflect.deleteProperty(env, "HACK_NO_INTERACTIVE"); - const server = createMcpServer({ cwd: project, env }); + const server = createMcpServer({ cwd, env }); const client = new Client({ name: "hack-mcp-contract", version: "1" }); contractClients.push(client); const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); await server.connect(serverTransport); await client.connect(clientTransport); - return { client, argvLog, auditLog: join(state, "mcp-audit.log") }; + return { client, argvLog, auditLog: join(state, "mcp-audit.log"), root }; } async function readStubInvocations( @@ -516,7 +547,7 @@ test("project resolution refusals carry existing codes and never spawn", async ( expect(unknown.structuredContent).toMatchObject({ ok: false, spawned: false, - error: { code: "E_PROJECT_NOT_FOUND" }, + error: { code: "E_PROJECT_NOT_FOUND", reason: "project_not_found" }, }); const ambiguous = await fixture.client.callTool({ name: "hack.project.run", @@ -528,6 +559,7 @@ test("project resolution refusals carry existing codes and never spawn", async ( stderr: "Use only one of projectName, repoRoot, or path.", error: { code: "E_USAGE", + reason: "conflicting_project_selection", message: "Use only one of projectName, repoRoot, or path.", }, }); @@ -535,6 +567,145 @@ test("project resolution refusals carry existing codes and never spawn", async ( expect(await Bun.file(fixture.auditLog).exists()).toBe(false); }); +/** + * Lowercase alphanumeric so project-name normalization keeps it intact; any + * echo of caller input (or of registry values it selected) would expose it. + */ +const CANARY = "zqxcanary4821"; + +type CanaryRefusal = { + readonly reason: McpRefusalReason; + readonly name: string; + readonly arguments: Record; + readonly fixture?: ContractFixtureOptions; +}; + +const canaryRefusals: readonly CanaryRefusal[] = [ + { + reason: "invalid_arguments", + name: "hack.project.run", + arguments: { projectName: "demo", service: `-${CANARY}`, cmd: [CANARY] }, + }, + { + reason: "conflicting_project_selection", + name: "hack.project.status", + arguments: { projectName: CANARY, path: `/nonexistent/${CANARY}` }, + }, + { + reason: "invalid_project_name", + name: "hack.project.status", + arguments: { projectName: `%%%${"!".repeat(3)}%%%` }, + }, + { + reason: "project_not_found", + name: "hack.project.status", + arguments: { projectName: CANARY }, + }, + { + reason: "ambiguous_project", + name: "hack.project.status", + arguments: { projectName: CANARY }, + fixture: { + extraProjects: [ + { id: `${CANARY}-a`, name: CANARY, repoRoot: `${CANARY}-a` }, + { id: `${CANARY}-b`, name: CANARY, repoRoot: `${CANARY}-b` }, + ], + }, + }, + { + reason: "ambiguous_project", + name: "hack.project.status", + arguments: { projectName: "idclash" }, + fixture: { + extraProjects: [ + { id: CANARY, name: "idclash", repoRoot: `${CANARY}-a` }, + { id: CANARY, name: "idclash-twin", repoRoot: `${CANARY}-b` }, + ], + }, + }, + { + reason: "path_not_found", + name: "hack.project.status", + arguments: { path: `/nonexistent/${CANARY}` }, + }, + { + reason: "path_not_registered", + name: "hack.project.logs.snapshot", + arguments: { path: ".", service: "api" }, + fixture: { cwdName: CANARY }, + }, + { + reason: "no_project_context", + name: "hack.project.status", + arguments: {}, + fixture: { cwdName: CANARY }, + }, + { + reason: "init_path_required", + name: "hack.project.init", + arguments: { name: CANARY, devHost: `${CANARY}.hack` }, + }, +]; + +test("every typed refusal is value-free: fixed message, closed reason, no canary", async () => { + const covered = new Set(); + for (const refusal of canaryRefusals) { + const fixture = await startContractFixture(refusal.fixture); + const result = await fixture.client.callTool({ + name: refusal.name, + arguments: refusal.arguments, + }); + const expected = MCP_REFUSALS[refusal.reason]; + const structured = result.structuredContent as { + readonly stderr: string; + readonly error: { readonly message: string }; + }; + expect({ reason: refusal.reason, isError: result.isError }).toEqual({ + reason: refusal.reason, + isError: true, + }); + expect(structured).toMatchObject({ + ok: false, + spawned: false, + command: "", + stdout: "", + error: { code: expected.code, reason: refusal.reason }, + }); + if (refusal.reason === "invalid_arguments") { + expect(structured.error.message).toStartWith( + "Invalid arguments: service." + ); + } else { + expect(structured.error.message).toBe(expected.message); + } + expect(structured.stderr).toBe(structured.error.message); + const serialized = JSON.stringify(result); + expect(serialized).not.toContain(CANARY); + expect(serialized).not.toContain("%%%"); + expect(serialized).not.toContain(fixture.root); + expect(await readStubInvocations(fixture.argvLog)).toEqual([]); + expect(await Bun.file(fixture.auditLog).exists()).toBe(false); + covered.add(refusal.reason); + } + const coveredReasons: readonly string[] = [...covered].sort(); + expect(coveredReasons).toEqual(Object.keys(MCP_REFUSALS).sort()); +}); + +test("an unknown tool keeps the in-band isError shape without echoing its name", async () => { + const fixture = await startContractFixture(); + const result = await fixture.client.callTool({ + name: `hack.${CANARY}`, + arguments: { projectName: CANARY }, + }); + expect(result).toEqual({ + content: [{ type: "text", text: MCP_UNKNOWN_TOOL_MESSAGE }], + isError: true, + }); + expect(MCP_UNKNOWN_TOOL_MESSAGE).toBe("MCP error -32602: Tool not found"); + expect(JSON.stringify(result)).not.toContain(CANARY); + expect(await readStubInvocations(fixture.argvLog)).toEqual([]); +}); + const READ_ONLY = { readOnlyHint: true, openWorldHint: false }; const ADDITIVE = { readOnlyHint: false, From cd76b26cff38c860045917c3994e4099137d333e Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:51:41 -0400 Subject: [PATCH 06/52] test(env): pin AES-GCM IV, size and tag-length guards with discriminating tests Seal authentic envelopes with 8- and 16-byte IVs and with ciphertext one byte over each parser bound, so only the parser guards can refuse them, and assert through a createDecipheriv spy that structural refusals happen before any decipher and that successful opens pass authTagLength 16. Covers the legacy project env, encrypted_file store and native env value parsers, with at-bound 12-byte-IV controls. Co-Authored-By: Claude Opus 5.5 --- tests/native-env-values.test.ts | 98 ++++++++++++++ tests/project-env-config.test.ts | 159 ++++++++++++++++++++++- tests/secret-store.test.ts | 212 ++++++++++++++++++++++++++++++- 3 files changed, 467 insertions(+), 2 deletions(-) diff --git a/tests/native-env-values.test.ts b/tests/native-env-values.test.ts index e3aaeef7..4497f14d 100644 --- a/tests/native-env-values.test.ts +++ b/tests/native-env-values.test.ts @@ -1,4 +1,5 @@ import { afterEach, beforeEach, expect, mock, spyOn, test } from "bun:test"; +import * as nodeCrypto from "node:crypto"; import * as fs from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -531,6 +532,103 @@ for (const kind of ["truncated-tag", "noncanonical-base64"] as const) { }); } +/** + * Seals plaintext exactly as the v1 owner does but with a caller-chosen IV + * length. GCM accepts any non-empty IV, so refusing these authentic values can + * only come from the parser's own IV guard, not from GCM authentication. + */ +function sealed(opts: { + readonly plaintext: string; + readonly ivBytes: number; +}): ProjectEnvStoredValue { + const key = nodeCrypto.createHash("sha256").update(KEY, "utf8").digest(); + const iv = nodeCrypto.randomBytes(opts.ivBytes); + const cipher = nodeCrypto.createCipheriv("aes-256-gcm", key, iv, { + authTagLength: 16, + }); + const ciphertext = Buffer.concat([ + cipher.update(opts.plaintext, "utf8"), + cipher.final(), + ]); + return { + secure: [ + "v1", + iv.toString("base64"), + cipher.getAuthTag().toString("base64"), + ciphertext.toString("base64"), + ].join(":"), + }; +} + +test("a test-sealed 12-byte-IV value decrypts with the tag length pinned (control)", async () => { + const p = await project(); + await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); + await layer(p, "hack.env.default.yaml", { + global: { SECRET: sealed({ plaintext: SENTINEL, ivBytes: 12 }) }, + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + expect((await values(p)).globalEnv).toEqual({ SECRET: SENTINEL }); + expect(decipher).toHaveBeenCalled(); + for (const call of decipher.mock.calls) { + const options: unknown = call[3]; + expect(options).toEqual({ authTagLength: 16 }); + } +}); + +for (const ivBytes of [8, 16]) { + test(`an authentic value sealed with a ${ivBytes}-byte IV refuses before decipher`, async () => { + const p = await project(); + await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); + await layer(p, "hack.env.default.yaml", { + global: { SECRET: sealed({ plaintext: SENTINEL, ivBytes }) }, + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + await refuses(() => values(p)); + expect(decipher).not.toHaveBeenCalled(); + }); +} + +test("a truncated tag refuses before decipher on the native path", async () => { + const p = await project(); + await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); + const authentic = sealed({ plaintext: SENTINEL, ivBytes: 12 }); + if (!(isRecord(authentic) && typeof authentic.secure === "string")) { + throw new Error("Synthetic ciphertext absent"); + } + const [prefix, iv, tag = "", ciphertext] = authentic.secure.split(":"); + await layer(p, "hack.env.default.yaml", { + global: { + SECRET: { + secure: [ + prefix, + iv, + Buffer.from(tag, "base64").subarray(0, 12).toString("base64"), + ciphertext, + ].join(":"), + }, + }, + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + await refuses(() => values(p)); + expect(decipher).not.toHaveBeenCalled(); +}); + +// The native layer read is bounded at 1 MiB, below the encoded size of a +// ciphertext over the 1 MiB parser bound, so this refusal comes from the +// bounded read; the shared parser bound is discriminated on the legacy path. +test("an authentic value over the ciphertext bound refuses without decipher", async () => { + const p = await project(); + await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); + await layer(p, "hack.env.default.yaml", { + global: { + SECRET: sealed({ plaintext: "a".repeat(1024 * 1024 + 1), ivBytes: 12 }), + }, + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + await refuses(() => values(p)); + expect(decipher).not.toHaveBeenCalled(); +}); + test("an encrypted empty value preserves presence", async () => { const p = await project(); await fs.writeFile(join(p, PROJECT_ENV_KEY_FILENAME), KEY); diff --git a/tests/project-env-config.test.ts b/tests/project-env-config.test.ts index ee3da7b8..b0a555c9 100644 --- a/tests/project-env-config.test.ts +++ b/tests/project-env-config.test.ts @@ -1,4 +1,5 @@ -import { afterEach, expect, test } from "bun:test"; +import { afterEach, expect, spyOn, test } from "bun:test"; +import * as nodeCrypto from "node:crypto"; import { writeFileSync } from "node:fs"; import { mkdir, @@ -1753,3 +1754,159 @@ test("legacy v1 secure values round-trip, including the empty value", async () = expect(resolved?.globalEnv.TOKEN).toBe(ENVELOPE_SENTINEL); expect(resolved?.globalEnv.EMPTY).toBe(""); }); + +/** Byte bound shared by the legacy and native v1 ciphertext parsers. */ +const ENVELOPE_CIPHERTEXT_LIMIT = 1024 * 1024; + +/** + * Seals plaintext exactly as the v1 owner does (AES-256-GCM, SHA-256 of the + * trimmed key text, 16-byte tag) but with a caller-chosen IV length. Node and + * Bun accept any non-empty GCM IV, so a refusal of these authentic envelopes + * can only come from the parser's own IV and size guards, not from GCM + * authentication. + */ +function sealProjectEnvValue(opts: { + readonly plaintext: string; + readonly ivBytes: number; +}): string { + const key = nodeCrypto + .createHash("sha256") + .update(ENVELOPE_KEY, "utf8") + .digest(); + const iv = nodeCrypto.randomBytes(opts.ivBytes); + const cipher = nodeCrypto.createCipheriv("aes-256-gcm", key, iv, { + authTagLength: 16, + }); + const ciphertext = Buffer.concat([ + cipher.update(opts.plaintext, "utf8"), + cipher.final(), + ]); + return envelope({ iv, tag: cipher.getAuthTag(), ciphertext }); +} + +type LegacyTokenOutcome = + | { readonly value: string | undefined } + | { readonly error: unknown }; + +async function resolveLegacyToken(opts: { + readonly repo: Awaited>; +}): Promise { + try { + const resolved = await resolveProjectEnvConfig({ + projectRoot: opts.repo.projectRoot, + projectDir: opts.repo.projectDir, + envName: null, + serviceNames: ["api"], + }); + return { value: resolved?.globalEnv.TOKEN }; + } catch (error: unknown) { + return { error }; + } +} + +function expectFixedEnvelopeRefusal(outcome: LegacyTokenOutcome): void { + const error = "error" in outcome ? outcome.error : null; + expect(error).toBeInstanceOf(Error); + expect(error instanceof Error ? error.message : "").toBe( + "Invalid secure env value." + ); + for (const privateText of [ENVELOPE_SENTINEL, ENVELOPE_KEY]) { + expect(String(error)).not.toContain(privateText); + } +} + +test("legacy test-sealed 12-byte-IV value at the ciphertext bound decrypts with the tag length pinned (control)", async () => { + const { repo } = await encryptedEnvelopeFixture(); + const plaintext = `${ENVELOPE_SENTINEL}${"a".repeat( + ENVELOPE_CIPHERTEXT_LIMIT - ENVELOPE_SENTINEL.length + )}`; + await writeSecureValue({ + repo, + secure: sealProjectEnvValue({ plaintext, ivBytes: 12 }), + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + const outcome = await resolveLegacyToken({ repo }); + expect("value" in outcome ? outcome.value : null).toBe(plaintext); + // The tag length is pinned by option rather than left to the runtime default. + expect(decipher).toHaveBeenCalled(); + for (const call of decipher.mock.calls) { + const options: unknown = call[3]; + expect(options).toEqual({ authTagLength: 16 }); + } + } finally { + decipher.mockRestore(); + } +}); + +for (const ivBytes of [8, 16]) { + test(`legacy authentic value sealed with a ${ivBytes}-byte IV refuses before decipher`, async () => { + const { repo } = await encryptedEnvelopeFixture(); + await writeSecureValue({ + repo, + secure: sealProjectEnvValue({ plaintext: ENVELOPE_SENTINEL, ivBytes }), + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedEnvelopeRefusal(await resolveLegacyToken({ repo })); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } + }); +} + +test("legacy authentic value one byte over the ciphertext bound refuses before decipher", async () => { + const { repo } = await encryptedEnvelopeFixture(); + await writeSecureValue({ + repo, + secure: sealProjectEnvValue({ + plaintext: "a".repeat(ENVELOPE_CIPHERTEXT_LIMIT + 1), + ivBytes: 12, + }), + }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedEnvelopeRefusal(await resolveLegacyToken({ repo })); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } +}); + +// Structural mutations must be refused by the explicit parser guards before any +// decipher exists, so the refusal does not depend on runtime GCM defaults. +for (const name of [ + "4-byte tag", + "8-byte tag", + "12-byte tag", + "16-byte IV", + "base64 with !", + "base64 with whitespace", + "oversized ciphertext", +]) { + test(`legacy secure env value with ${name} refuses before decipher`, async () => { + const mutate = ENVELOPE_MUTATIONS.find(([label]) => label === name)?.[1]; + if (!mutate) { + throw new Error("missing envelope mutation"); + } + const { repo, secure } = await encryptedEnvelopeFixture(); + await writeSecureValue({ repo, secure: mutate(secure) }); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedEnvelopeRefusal(await resolveLegacyToken({ repo })); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } + }); +} + +test("legacy secure env value with a 16-byte tag of the wrong value refuses", async () => { + const { repo, secure } = await encryptedEnvelopeFixture(); + const parts = envelopeParts(secure); + const tag = Buffer.from(parts.tag); + tag[15] = (tag[15] ?? 0) ^ 1; + await writeSecureValue({ repo, secure: envelope({ ...parts, tag }) }); + expectFixedEnvelopeRefusal(await resolveLegacyToken({ repo })); +}); diff --git a/tests/secret-store.test.ts b/tests/secret-store.test.ts index 6db50d88..bf3e30a1 100644 --- a/tests/secret-store.test.ts +++ b/tests/secret-store.test.ts @@ -1,4 +1,5 @@ -import { afterEach, beforeEach, expect, test } from "bun:test"; +import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; +import * as nodeCrypto from "node:crypto"; import { access, mkdir, @@ -452,6 +453,215 @@ test("encrypted_file v1 envelope round-trips, including the empty value", async expect(await store.get({ key: "EMPTY" })).toBe(""); }); +/** Byte bound of the encrypted_file ciphertext parser. */ +const STORE_CIPHERTEXT_LIMIT = 16 * 1024 * 1024; + +function storeKey(): Buffer { + return nodeCrypto.createHash("sha256").update(STORE_KEY, "utf8").digest(); +} + +/** Opens a store envelope the way the v1 owner does, to reuse its plaintext. */ +function openStoreEnvelope(envelope: StoreEnvelope): string { + const decipher = nodeCrypto.createDecipheriv( + "aes-256-gcm", + storeKey(), + Buffer.from(envelope.iv, "base64"), + { authTagLength: 16 } + ); + decipher.setAuthTag(Buffer.from(envelope.tag, "base64")); + return Buffer.concat([ + decipher.update(Buffer.from(envelope.ciphertext, "base64")), + decipher.final(), + ]).toString("utf8"); +} + +/** + * Seals a store plaintext exactly as the v1 owner does but with a caller-chosen + * IV length. GCM accepts any non-empty IV, so a refusal of these authentic + * envelopes can only come from the parser's own IV and size guards. + */ +function sealStoreEnvelope(opts: { + readonly plaintext: string; + readonly ivBytes: number; +}): StoreEnvelope { + const iv = nodeCrypto.randomBytes(opts.ivBytes); + const cipher = nodeCrypto.createCipheriv("aes-256-gcm", storeKey(), iv, { + authTagLength: 16, + }); + const ciphertext = Buffer.concat([ + cipher.update(opts.plaintext, "utf8"), + cipher.final(), + ]); + return { + version: 1, + algorithm: "aes-256-gcm", + iv: iv.toString("base64"), + tag: cipher.getAuthTag().toString("base64"), + ciphertext: ciphertext.toString("base64"), + }; +} + +/** Pads the real store plaintext with an extra entry to exactly `bytes` bytes. */ +function storePlaintextOfLength(opts: { + readonly envelope: StoreEnvelope; + readonly bytes: number; +}): string { + const parsed = JSON.parse(openStoreEnvelope(opts.envelope)) as { + readonly version: number; + readonly entries: Record; + }; + const base = JSON.stringify({ + ...parsed, + entries: { ...parsed.entries, PAD: "" }, + }); + const padded = JSON.stringify({ + ...parsed, + entries: { ...parsed.entries, PAD: "a".repeat(opts.bytes - base.length) }, + }); + expect(Buffer.byteLength(padded, "utf8")).toBe(opts.bytes); + return padded; +} + +async function readStoreToken(): Promise< + { readonly value: string | null } | { readonly error: unknown } +> { + try { + const store = await resolveSecretStore({ projectName: "app" }); + return { value: await store.get({ key: "TOKEN" }) }; + } catch (error: unknown) { + return { error }; + } +} + +function expectFixedStoreRefusal( + outcome: { readonly value: string | null } | { readonly error: unknown } +): void { + const error = "error" in outcome ? outcome.error : null; + expect(error).toBeInstanceOf(Error); + expect(error instanceof Error ? error.message : "").toMatch( + /^Invalid encrypted secret store payload at / + ); + for (const privateText of [STORE_SENTINEL, STORE_KEY]) { + expect(String(error)).not.toContain(privateText); + } +} + +test("encrypted_file test-sealed 12-byte-IV envelope at the ciphertext bound opens with the tag length pinned (control)", async () => { + const { storePath, envelope } = await encryptedStoreFixture(); + await writeFile( + storePath, + `${JSON.stringify( + sealStoreEnvelope({ + plaintext: storePlaintextOfLength({ + envelope, + bytes: STORE_CIPHERTEXT_LIMIT, + }), + ivBytes: 12, + }) + )}\n` + ); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + const outcome = await readStoreToken(); + expect("value" in outcome ? outcome.value : null).toBe(STORE_SENTINEL); + // The tag length is pinned by option rather than left to the runtime default. + expect(decipher).toHaveBeenCalled(); + for (const call of decipher.mock.calls) { + const options: unknown = call[3]; + expect(options).toEqual({ authTagLength: 16 }); + } + } finally { + decipher.mockRestore(); + } +}); + +for (const ivBytes of [8, 16]) { + test(`encrypted_file authentic envelope sealed with a ${ivBytes}-byte IV refuses before decipher`, async () => { + const { storePath, envelope } = await encryptedStoreFixture(); + await writeFile( + storePath, + `${JSON.stringify( + sealStoreEnvelope({ plaintext: openStoreEnvelope(envelope), ivBytes }) + )}\n` + ); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedStoreRefusal(await readStoreToken()); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } + }); +} + +test("encrypted_file authentic envelope one byte over the ciphertext bound refuses before decipher", async () => { + const { storePath, envelope } = await encryptedStoreFixture(); + await writeFile( + storePath, + `${JSON.stringify( + sealStoreEnvelope({ + plaintext: storePlaintextOfLength({ + envelope, + bytes: STORE_CIPHERTEXT_LIMIT + 1, + }), + ivBytes: 12, + }) + )}\n` + ); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedStoreRefusal(await readStoreToken()); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } +}); + +// Structural mutations must be refused by the explicit parser guards before any +// decipher exists, so the refusal does not depend on runtime GCM defaults. +for (const name of [ + "4-byte tag", + "8-byte tag", + "12-byte tag", + "16-byte IV", + "base64 with !", + "base64 with whitespace", + "oversized ciphertext", +]) { + test(`encrypted_file envelope with ${name} refuses before decipher`, async () => { + const mutate = STORE_MUTATIONS.find(([label]) => label === name)?.[1]; + if (!mutate) { + throw new Error("missing store mutation"); + } + const { storePath, envelope } = await encryptedStoreFixture(); + await writeFile(storePath, `${JSON.stringify(mutate(envelope))}\n`); + const decipher = spyOn(nodeCrypto, "createDecipheriv"); + try { + expectFixedStoreRefusal(await readStoreToken()); + expect(decipher).not.toHaveBeenCalled(); + } finally { + decipher.mockRestore(); + } + }); +} + +test("encrypted_file envelope with a 16-byte tag of the wrong value refuses", async () => { + const { storePath, envelope } = await encryptedStoreFixture(); + const tag = Buffer.from(envelope.tag, "base64"); + tag[15] = (tag[15] ?? 0) ^ 1; + await writeFile( + storePath, + `${JSON.stringify({ ...envelope, tag: tag.toString("base64") })}\n` + ); + const outcome = await readStoreToken(); + const error = "error" in outcome ? outcome.error : null; + expect(error).toBeInstanceOf(Error); + expect(String(error)).toContain("Cannot decrypt encrypted secret store at "); + for (const privateText of [STORE_SENTINEL, STORE_KEY]) { + expect(String(error)).not.toContain(privateText); + } +}); + async function writeGlobalConfig(input: { readonly value: Record; }): Promise { From bcf88ec9a35a35225f4882dd5f108c0ae74ca7cf Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 22:05:39 -0400 Subject: [PATCH 07/52] fix(init): stop writing repo-local agent files by default Interactive `hack init` now prints a non-mutating note that points at the optional Hack plugin and `hack agent onboard`. Repository-local export of Cursor rules, Claude hooks, Codex skills, AGENTS.md/CLAUDE.md sections and project MCP config is a separate opt-in whose confirm defaults to no and whose target list starts empty. The decision moves out of project.ts into src/agents/init-agent-integrations.ts. Doctor and the agent-prime freshness inventory now treat absent optional integrations as current. Doctor flags a path only when it exists and lacks the current revision marker (AGENTS.md/CLAUDE.md only when they carry a Hack-managed section). integration-sync counts only stale or errored checks; user-owned docs without a Hack section are not drift; retired legacy artifacts still are. Co-Authored-By: Claude Opus 5.5 --- docs/guides/init-project.md | 18 ++ src/agents/init-agent-integrations.ts | 258 +++++++++++++++++++++ src/cli/integration-sync.ts | 17 +- src/commands/doctor.ts | 73 ++++-- src/commands/project.ts | 155 +------------ src/mcp/agent-docs.ts | 3 + tests/doctor-command.test.ts | 69 ++++++ tests/init-agent-integrations.test.ts | 318 ++++++++++++++++++++++++++ tests/integration-sync.test.ts | 123 ++++++++++ 9 files changed, 866 insertions(+), 168 deletions(-) create mode 100644 src/agents/init-agent-integrations.ts create mode 100644 tests/init-agent-integrations.test.ts create mode 100644 tests/integration-sync.test.ts diff --git a/docs/guides/init-project.md b/docs/guides/init-project.md index 86b91774..6072fc28 100644 --- a/docs/guides/init-project.md +++ b/docs/guides/init-project.md @@ -41,6 +41,24 @@ Note: - HTTP services via `https://*.hack.local` hostnames (matching your Caddy labels) - non-HTTP services via Compose service hostnames (e.g. `db`, `redis`) +## Coding agents + +Interactive `hack init` does not write coding-agent files by default. It prints a +note recommending the optional Hack plugin for your client (see +[Integrations](../integrations.md#optional-native-plugins)) and +`hack agent onboard` for agent-assisted project setup. Non-interactive runs neither +prompt nor write agent files. + +Repository-local export is a separate opt-in for clients that cannot use the plugin. +Its confirm defaults to "no" and its target list starts empty. Only the targets you +select are written into the repository: `.cursor/rules/hack.mdc`, +`.claude/settings.local.json` plus the `hack-init` skill, `.codex/skills/hack-cli`, +the managed section of `AGENTS.md`/`CLAUDE.md`, or project MCP config. + +`hack doctor` and `hack agent prime --check` treat absent optional integrations as +current. They report review needed only for an installed integration that is stale +or unreadable, or for a retired legacy artifact. + ## Local domain compatibility For new projects using the default `hack.local`, `--oauth` adds diff --git a/src/agents/init-agent-integrations.ts b/src/agents/init-agent-integrations.ts new file mode 100644 index 00000000..ec40da43 --- /dev/null +++ b/src/agents/init-agent-integrations.ts @@ -0,0 +1,258 @@ +import { confirm, isCancel, multiselect, note } from "@clack/prompts"; +import { upsertAgentDocs } from "../mcp/agent-docs.ts"; +import { installMcpConfig, type McpTarget } from "../mcp/install.ts"; +import { logger } from "../ui/logger.ts"; +import { installClaudeHooks } from "./claude.ts"; +import { installCodexSkill } from "./codex-skill.ts"; +import { installCursorRules } from "./cursor.ts"; + +/** + * Coding-agent step of interactive `hack init`. + * + * Contract: the default path is non-mutating. It prints a note that points at + * the optional native plugin and `hack agent onboard`, and writes nothing under + * `.cursor`, `.claude`, `.codex`, `AGENTS.md`, `CLAUDE.md` or `.mcp.json`. + * Repository-local export is a separate, explicit opt-in: its confirm defaults + * to "no" and its target list starts empty, so only targets the user selects + * are written. Non-interactive runs neither prompt nor write. + */ + +/** Repository-local agent files `hack init` can export on explicit request. */ +export type InitAgentExportTarget = + | "cursor" + | "claude" + | "codex" + | "agents" + | "mcp"; + +export type InitAgentExportOption = { + readonly value: InitAgentExportTarget; + readonly label: string; +}; + +export const INIT_AGENT_EXPORT_CONFIRM = { + message: + "Also write Hack agent files into this repository? (most setups only need the plugin)", + initialValue: false, +} as const; + +export const INIT_AGENT_EXPORT_SELECT_MESSAGE = + "Select repository files to write (none selected by default):"; + +export const INIT_AGENT_EXPORT_OPTIONS: readonly InitAgentExportOption[] = [ + { + value: "cursor", + label: "Cursor rules: writes .cursor/rules/hack.mdc into this repository", + }, + { + value: "claude", + label: + "Claude Code hooks: writes .claude/settings.local.json and .claude/skills/hack-init into this repository", + }, + { + value: "codex", + label: "Codex skill: writes .codex/skills/hack-cli into this repository", + }, + { + value: "agents", + label: "Agent docs: writes AGENTS.md and CLAUDE.md into this repository", + }, + { + value: "mcp", + label: + "MCP config (no-shell clients): writes .mcp.json, .cursor/mcp.json or .codex/config.toml into this repository", + }, +]; + +/** Repository writes resolved from an explicit export selection. */ +export type InitAgentExportPlan = { + readonly cursor: boolean; + readonly claude: boolean; + readonly codex: boolean; + readonly agentDocs: boolean; + readonly mcpTargets: readonly McpTarget[]; +}; + +const ALL_MCP_TARGETS: readonly McpTarget[] = ["cursor", "claude", "codex"]; + +/** + * Resolve which repository files an explicit selection writes. An empty + * selection writes nothing. MCP config follows the selected clients, or all + * three clients when MCP is the only client-neutral choice. + */ +export function planInitAgentExport(opts: { + readonly selected: readonly InitAgentExportTarget[]; +}): InitAgentExportPlan { + const selection = new Set(opts.selected); + const clientHints = ALL_MCP_TARGETS.filter((target) => selection.has(target)); + let mcpTargets: readonly McpTarget[] = []; + if (selection.has("mcp")) { + mcpTargets = clientHints.length > 0 ? clientHints : ALL_MCP_TARGETS; + } + return { + cursor: selection.has("cursor"), + claude: selection.has("claude"), + codex: selection.has("codex"), + agentDocs: selection.has("agents"), + mcpTargets, + }; +} + +/** Non-mutating default guidance shown by interactive `hack init`. */ +export function renderInitAgentIntegrationNote(): string { + return [ + "No coding-agent files were written to this repository.", + "Recommended: install the optional Hack plugin for your client, for example:", + " codex plugin add hack@hack-dance", + " claude plugin install hack@hack-dance", + "Cursor: install Hack from Customize. See docs/integrations.md.", + "", + "Then let your agent finish project setup:", + " hack agent onboard", + ].join("\n"); +} + +export type InitAgentIntegrationPrompts = { + readonly note: (opts: { + readonly message: string; + readonly title: string; + }) => void; + readonly confirm: (opts: { + readonly message: string; + readonly initialValue: boolean; + }) => Promise; + readonly multiselect: (opts: { + readonly message: string; + readonly options: readonly InitAgentExportOption[]; + readonly initialValues: readonly InitAgentExportTarget[]; + readonly required: boolean; + }) => Promise; +}; + +export type InitAgentIntegrationOutcome = + | { readonly kind: "skipped" } + | { readonly kind: "note-only" } + | { readonly kind: "exported"; readonly plan: InitAgentExportPlan }; + +const clackPrompts: InitAgentIntegrationPrompts = { + note: ({ message, title }) => note(message, title), + confirm: (opts) => + confirm({ message: opts.message, initialValue: opts.initialValue }), + multiselect: (opts) => + multiselect({ + message: opts.message, + options: opts.options.map((option) => ({ ...option })), + initialValues: [...opts.initialValues], + required: opts.required, + }), +}; + +/** + * Run the coding-agent step of `hack init`. `guard` runs before every write + * (and once up front) so legacy/native project input ownership is rechecked. + */ +export async function runInitAgentIntegrations(opts: { + readonly repoRoot: string; + readonly interactive: boolean; + readonly guard: (opts: { readonly projectRoot: string }) => Promise; + readonly prompts?: InitAgentIntegrationPrompts; +}): Promise { + await opts.guard({ projectRoot: opts.repoRoot }); + if (!opts.interactive) { + return { kind: "skipped" }; + } + const prompts = opts.prompts ?? clackPrompts; + prompts.note({ + message: renderInitAgentIntegrationNote(), + title: "Coding agents", + }); + + const shouldExport = await prompts.confirm(INIT_AGENT_EXPORT_CONFIRM); + if (isCancel(shouldExport) || shouldExport !== true) { + return { kind: "note-only" }; + } + + const selected = await prompts.multiselect({ + message: INIT_AGENT_EXPORT_SELECT_MESSAGE, + options: INIT_AGENT_EXPORT_OPTIONS, + initialValues: [], + required: false, + }); + if (isCancel(selected) || selected.length === 0) { + return { kind: "note-only" }; + } + + const plan = planInitAgentExport({ selected }); + await applyInitAgentExport({ + repoRoot: opts.repoRoot, + plan, + guard: opts.guard, + }); + return { kind: "exported", plan }; +} + +async function applyInitAgentExport(opts: { + readonly repoRoot: string; + readonly plan: InitAgentExportPlan; + readonly guard: (opts: { readonly projectRoot: string }) => Promise; +}): Promise { + const projectRoot = opts.repoRoot; + if (opts.plan.cursor) { + await opts.guard({ projectRoot }); + const result = await installCursorRules({ scope: "project", projectRoot }); + logInstallResult({ label: "Cursor rules", ...result }); + } + if (opts.plan.claude) { + await opts.guard({ projectRoot }); + const result = await installClaudeHooks({ scope: "project", projectRoot }); + logInstallResult({ label: "Claude hooks", ...result }); + } + if (opts.plan.codex) { + await opts.guard({ projectRoot }); + const result = await installCodexSkill({ scope: "project", projectRoot }); + logInstallResult({ label: "Codex skill", ...result }); + } + if (opts.plan.agentDocs) { + await opts.guard({ projectRoot }); + const results = await upsertAgentDocs({ + projectRoot, + targets: ["agents", "claude"], + }); + for (const result of results) { + logInstallResult({ label: "Agent docs", ...result }); + } + } + if (opts.plan.mcpTargets.length > 0) { + await opts.guard({ projectRoot }); + const results = await installMcpConfig({ + targets: opts.plan.mcpTargets, + scope: "project", + projectRoot, + }); + for (const result of results) { + logInstallResult({ + label: "MCP config", + status: result.status, + path: result.path ?? "unknown path", + message: result.message, + }); + } + } +} + +function logInstallResult(opts: { + readonly label: string; + readonly status: string; + readonly path: string; + readonly message?: string; +}): void { + if (opts.status === "error") { + logger.warn({ message: opts.message ?? `Failed to update ${opts.label}` }); + return; + } + if (opts.status === "noop") { + logger.info({ message: `No changes for ${opts.label} (${opts.path})` }); + return; + } + logger.success({ message: `Updated ${opts.label} at ${opts.path}` }); +} diff --git a/src/cli/integration-sync.ts b/src/cli/integration-sync.ts index f3a4e664..ac72dd36 100644 --- a/src/cli/integration-sync.ts +++ b/src/cli/integration-sync.ts @@ -44,7 +44,7 @@ export function renderAgentIntegrationFreshnessNotice(opts: { } return [ `Hack integration inventory: review needed (CLI v${opts.report.cliVersion}).`, - "Some integrations are missing, stale, or could not be checked. Missing optional integrations need not be installed.", + "Some installed integrations are stale, retired, or could not be checked. Missing optional integrations need not be installed.", `Inspect affected paths: ${opts.report.verifyCommand}`, "Repair only the affected integration and scope authorized for this task.", `For an explicitly requested full refresh: ${opts.report.fixCommand}`, @@ -101,18 +101,27 @@ async function detectIntegrationDrift(opts: { }; } +// Optional integrations are not required: an absent ("missing") integration is +// current. Only present-but-stale or unverifiable ("error") checks are drift. +// Retired legacy artifacts are drift whenever they are present. function hasSingleCheckDrift(status: string): boolean { - return status !== "noop" && status !== "absent"; + return status === "stale" || status === "error"; } function hasMcpDrift(opts: { readonly checks: readonly McpCheckResult[]; }): boolean { - return opts.checks.some((check) => check.status !== "present"); + // MCP config has no revision marker, so only unreadable configs count. + return opts.checks.some((check) => check.status === "error"); } function hasDocDrift(opts: { readonly checks: readonly AgentDocCheckResult[]; }): boolean { - return opts.checks.some((check) => check.status !== "present"); + // A user-owned AGENTS.md/CLAUDE.md without a Hack section is not drift. + return opts.checks.some( + (check) => + check.status === "stale" || + (check.status === "error" && check.unmanaged !== true) + ); } diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index ac9a1989..4b50f6ef 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -3526,44 +3526,87 @@ async function maybeRepairHackd(): Promise { } } +/** + * Optional integrations are current when absent: a path is stale only when it + * exists and lacks the current revision marker. AGENTS.md/CLAUDE.md are + * user-owned, so they count only when they carry a Hack-managed section. + * Retained legacy artifacts are always stale. + */ export async function inspectDoctorAgentIntegrations(opts: { readonly projectRoot: string | null; readonly homeDir?: string; }): Promise<{ readonly status: "current" | "stale" }> { const home = (opts.homeDir ?? process.env.HOME ?? homedir()).trim(); - const paths = [ + const hackOwnedPaths = [ ...(opts.projectRoot ? [ resolve(opts.projectRoot, ".cursor", "rules", "hack.mdc"), resolve(opts.projectRoot, ".codex", "skills", "hack-cli", "SKILL.md"), - resolve(opts.projectRoot, "AGENTS.md"), - resolve(opts.projectRoot, "CLAUDE.md"), ] : []), resolve(home, ".cursor", "rules", "hack.mdc"), resolve(home, ".codex", "skills", "hack-cli", "SKILL.md"), resolve(home, ".ai", "skills", "hack-cli", "SKILL.md"), ]; + const sharedDocPaths = opts.projectRoot + ? [ + resolve(opts.projectRoot, "AGENTS.md"), + resolve(opts.projectRoot, "CLAUDE.md"), + ] + : []; const marker = `Content revision: \`${HACK_AGENT_INTEGRATION_CONTENT_REVISION}\``; - const [contents, legacyProject, legacyUser] = await Promise.all([ - Promise.all(paths.map((path) => readTextFile(path))), - opts.projectRoot - ? checkLegacyProjectAgentArtifacts({ projectRoot: opts.projectRoot }) - : Promise.resolve([]), - checkLegacyUserAgentArtifacts({ home }), - ]); + const [ownedContents, docContents, legacyProject, legacyUser] = + await Promise.all([ + Promise.all( + hackOwnedPaths.map((path) => readOptionalIntegrationText(path)) + ), + Promise.all( + sharedDocPaths.map((path) => readOptionalIntegrationText(path)) + ), + opts.projectRoot + ? checkLegacyProjectAgentArtifacts({ projectRoot: opts.projectRoot }) + : Promise.resolve([]), + checkLegacyUserAgentArtifacts({ home }), + ]); const hasLegacyArtifacts = [...legacyProject, ...legacyUser].some( (check) => check.status !== "absent" ); + // Present but unreadable paths are treated as stale rather than absent. + const hasStalePath = + ownedContents.some( + (content) => + content.present && + (content.text === null || !content.text.includes(marker)) + ) || + docContents.some( + (content) => + content.present && + (content.text === null || + (content.text.includes(HACK_AGENT_DOCS_SECTION_MARKER) && + !content.text.includes(marker))) + ); return { - status: - contents.every((content) => content?.includes(marker)) && - !hasLegacyArtifacts - ? "current" - : "stale", + status: hasStalePath || hasLegacyArtifacts ? "stale" : "current", }; } +const HACK_AGENT_DOCS_SECTION_MARKER = ""; + +async function readOptionalIntegrationText( + path: string +): Promise< + | { readonly present: false } + | { readonly present: true; readonly text: string | null } +> { + const text = await readTextFile(path); + if (text !== null) { + return { present: true, text }; + } + return (await pathExists(path)) + ? { present: true, text: null } + : { present: false }; +} + async function resolveDaemonReportForDoctorFix(): Promise< ReturnType > { diff --git a/src/commands/project.ts b/src/commands/project.ts index 38e6bf88..2eb690db 100644 --- a/src/commands/project.ts +++ b/src/commands/project.ts @@ -21,16 +21,13 @@ import { autocompleteMultiselect, confirm, isCancel, - multiselect, note, password, select, text, } from "@clack/prompts"; import { YAML } from "bun"; -import { installClaudeHooks } from "../agents/claude.ts"; -import { installCodexSkill } from "../agents/codex-skill.ts"; -import { installCursorRules } from "../agents/cursor.ts"; +import { runInitAgentIntegrations } from "../agents/init-agent-integrations.ts"; import { type OnboardingWith, parseOnboardingWith, @@ -274,9 +271,6 @@ import { buildWorktreeRetargetWarning, findSameCheckoutRetargetConflicts, } from "../lib/worktree-runtime-target.ts"; -import { upsertAgentDocs } from "../mcp/agent-docs.ts"; -import type { McpTarget } from "../mcp/install.ts"; -import { installMcpConfig } from "../mcp/install.ts"; import { resolveMux } from "../mux/mux-resolver.ts"; import { renderProjectConfigJson, @@ -3160,7 +3154,11 @@ async function handleInit({ ); } - await maybeSetupAgentIntegrations({ repoRoot }); + await runInitAgentIntegrations({ + repoRoot, + interactive: canPrompt(), + guard: assertLegacyProjectInputFamily, + }); note( [ @@ -3503,147 +3501,6 @@ function validateOauthTld(opts: { readonly value: string }): string | null { return null; } -type SetupIntegration = "cursor" | "claude" | "codex" | "agents" | "mcp"; - -async function maybeSetupAgentIntegrations(opts: { - readonly repoRoot: string; -}): Promise { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - if (!canPrompt()) { - return; - } - const shouldSetup = await confirm({ - message: "Set up coding agent integrations? (Cursor/Claude/Codex)", - initialValue: true, - }); - if (isCancel(shouldSetup) || !shouldSetup) { - return; - } - - const selected = await multiselect({ - message: "Select integrations to install:", - required: true, - options: [ - { value: "cursor", label: "Cursor rules (.cursor/rules/hack.mdc)" }, - { - value: "claude", - label: "Claude Code hooks (.claude/settings.local.json)", - }, - { value: "codex", label: "Codex skill (.codex/skills/hack-cli)" }, - { value: "agents", label: "AGENTS.md / CLAUDE.md snippets" }, - { value: "mcp", label: "MCP config (no-shell clients)" }, - ], - initialValues: ["cursor", "claude", "codex"], - }); - if (isCancel(selected) || selected.length === 0) { - return; - } - - const selection = new Set(selected); - - if (selection.has("cursor")) { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - const result = await installCursorRules({ - scope: "project", - projectRoot: opts.repoRoot, - }); - logInstallResult({ - label: "Cursor rules", - status: result.status, - path: result.path, - message: result.message, - }); - } - - if (selection.has("claude")) { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - const result = await installClaudeHooks({ - scope: "project", - projectRoot: opts.repoRoot, - }); - logInstallResult({ - label: "Claude hooks", - status: result.status, - path: result.path, - message: result.message, - }); - } - - if (selection.has("codex")) { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - const result = await installCodexSkill({ - scope: "project", - projectRoot: opts.repoRoot, - }); - logInstallResult({ - label: "Codex skill", - status: result.status, - path: result.path, - message: result.message, - }); - } - - if (selection.has("agents")) { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - const results = await upsertAgentDocs({ - projectRoot: opts.repoRoot, - targets: ["agents", "claude"], - }); - for (const result of results) { - logInstallResult({ - label: "Agent docs", - status: result.status, - path: result.path, - message: result.message, - }); - } - } - - if (selection.has("mcp")) { - await assertLegacyProjectInputFamily({ projectRoot: opts.repoRoot }); - const targetHints = selected.filter( - (value) => value === "cursor" || value === "claude" || value === "codex" - ); - const targets = ( - targetHints.length > 0 ? targetHints : ["cursor", "claude", "codex"] - ) as McpTarget[]; - - const results = await installMcpConfig({ - targets, - scope: "project", - projectRoot: opts.repoRoot, - }); - - for (const result of results) { - logInstallResult({ - label: "MCP config", - status: result.status, - path: result.path ?? "unknown path", - message: result.message, - }); - } - } -} - -function logInstallResult(opts: { - readonly label: string; - readonly status: string; - readonly path: string; - readonly message?: string; -}): void { - if (opts.status === "error") { - logger.warn({ message: opts.message ?? `Failed to update ${opts.label}` }); - return; - } - - if (opts.status === "noop") { - logger.info({ message: `No changes for ${opts.label} (${opts.path})` }); - return; - } - - logger.success({ message: `Updated ${opts.label} at ${opts.path}` }); -} - // Exported for direct unit-testing of the auto (non-interactive) discovery // + validation pipeline without going through the CLI prompt flow. export interface ComposeWizardInput { diff --git a/src/mcp/agent-docs.ts b/src/mcp/agent-docs.ts index eb3e24ca..7668f882 100644 --- a/src/mcp/agent-docs.ts +++ b/src/mcp/agent-docs.ts @@ -20,6 +20,8 @@ export type AgentDocCheckResult = { readonly status: "present" | "stale" | "missing" | "error"; readonly path: string; readonly message?: string; + /** Set when the file exists but has no Hack-managed section at all. */ + readonly unmanaged?: true; }; export type AgentDocRemoveResult = { @@ -118,6 +120,7 @@ export async function checkAgentDocs(opts: { status: "error", path, message: "Missing hack agent-docs markers.", + unmanaged: true, }); continue; } diff --git a/tests/doctor-command.test.ts b/tests/doctor-command.test.ts index d67bc805..9f3f9d32 100644 --- a/tests/doctor-command.test.ts +++ b/tests/doctor-command.test.ts @@ -300,6 +300,75 @@ test("doctor audits global agent guidance without a project", async () => { } }); +test("doctor treats absent optional agent integrations as current", async () => { + const root = await mkdtemp(join(tmpdir(), "hack-doctor-absent-agents-")); + const home = join(root, "home"); + const project = join(root, "repo"); + const marker = `Content revision: \`${HACK_AGENT_INTEGRATION_CONTENT_REVISION}\``; + try { + await mkdir(home, { recursive: true }); + await mkdir(project, { recursive: true }); + + // Empty project plus empty HOME. + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "current" }); + + // Plugin-only: client plugin state and user-authored docs, but no + // standalone Hack rules, skills or managed doc sections. + const pluginDir = join(home, ".claude", "plugins", "hack"); + await mkdir(pluginDir, { recursive: true }); + await writeFile(join(pluginDir, "plugin.json"), "{}\n"); + await writeFile(join(project, "AGENTS.md"), "# Team notes\n"); + await writeFile(join(project, "CLAUDE.md"), "Use bun.\n"); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "current" }); + + // A current Hack-managed section stays current. + await writeFile( + join(project, "AGENTS.md"), + `# Team notes\n\n${marker}\n\n` + ); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "current" }); + + // One present Hack-managed section without the current revision is stale. + await writeFile( + join(project, "AGENTS.md"), + "# Team notes\n\nold\n\n" + ); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "stale" }); + await writeFile(join(project, "AGENTS.md"), "# Team notes\n"); + + // One present Hack-owned file without the current revision is stale. + const projectRules = join(project, ".cursor", "rules", "hack.mdc"); + await mkdir(join(projectRules, ".."), { recursive: true }); + await writeFile(projectRules, "stale\n"); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "stale" }); + await writeFile(projectRules, `${marker}\n`); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "current" }); + + // A retired legacy project artifact is stale even when nothing else is. + await writeFile( + join(project, "CLAUDE.md"), + "\nRetired\n\n" + ); + await expect( + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }) + ).resolves.toEqual({ status: "stale" }); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test("doctor guidance routes runtime hygiene drift to projects prune", () => { const guidance = buildDoctorRecoveryGuidance({ results: [ diff --git a/tests/init-agent-integrations.test.ts b/tests/init-agent-integrations.test.ts new file mode 100644 index 00000000..61519df0 --- /dev/null +++ b/tests/init-agent-integrations.test.ts @@ -0,0 +1,318 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, relative } from "node:path"; +import { + INIT_AGENT_EXPORT_CONFIRM, + INIT_AGENT_EXPORT_OPTIONS, + type InitAgentExportTarget, + type InitAgentIntegrationPrompts, + planInitAgentExport, + renderInitAgentIntegrationNote, + runInitAgentIntegrations, +} from "../src/agents/init-agent-integrations.ts"; + +/** + * Every test runs against a fixture project root and a fixture HOME so the + * real installers can execute without touching the contributor's + * ~/.claude, ~/.codex or ~/.cursor configuration. + */ + +const AGENT_PATHS = [ + ".cursor", + ".claude", + ".codex", + "AGENTS.md", + "CLAUDE.md", + ".mcp.json", +] as const; + +let root: string | null = null; +let savedHome: string | undefined; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-init-agents-")); + await mkdir(join(root, "home"), { recursive: true }); + await mkdir(join(root, "repo", ".hack"), { recursive: true }); + await writeFile( + join(root, "repo", ".hack", "docker-compose.yml"), + "services: {}\n" + ); + savedHome = process.env.HOME; + process.env.HOME = join(root, "home"); +}); + +afterEach(async () => { + if (savedHome === undefined) { + // biome-ignore lint/performance/noDelete: restoring an unset env var + delete process.env.HOME; + } else { + process.env.HOME = savedHome; + } + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } +}); + +function fixture(): { readonly repo: string; readonly home: string } { + if (!root) { + throw new Error("fixture not initialized"); + } + return { repo: join(root, "repo"), home: join(root, "home") }; +} + +async function listFiles(dir: string): Promise { + const entries = await readdir(dir, { recursive: true, withFileTypes: true }); + return entries + .filter((entry) => entry.isFile()) + .map((entry) => relative(dir, join(entry.parentPath, entry.name))) + .sort(); +} + +type PromptLog = { + notes: string[]; + confirms: { message: string; initialValue: boolean }[]; + multiselects: { + initialValues: readonly InitAgentExportTarget[]; + required: boolean; + labels: readonly string[]; + }[]; +}; + +/** Simulates a user pressing Enter on every prompt (accepting each default). */ +function defaultAnswerPrompts(log: PromptLog): InitAgentIntegrationPrompts { + return { + note: ({ message }) => { + log.notes.push(message); + }, + confirm: (opts) => { + log.confirms.push({ ...opts }); + return Promise.resolve(opts.initialValue); + }, + multiselect: (opts) => { + log.multiselects.push({ + initialValues: opts.initialValues, + required: opts.required, + labels: opts.options.map((option) => option.label), + }); + return Promise.resolve([...opts.initialValues]); + }, + }; +} + +/** Simulates a user who opts in and selects exactly `selected`. */ +function optInPrompts(opts: { + readonly log: PromptLog; + readonly selected: readonly InitAgentExportTarget[]; +}): InitAgentIntegrationPrompts { + const base = defaultAnswerPrompts(opts.log); + return { + ...base, + confirm: (confirmOpts) => { + opts.log.confirms.push({ ...confirmOpts }); + return Promise.resolve(true); + }, + multiselect: async (selectOpts) => { + await base.multiselect(selectOpts); + return [...opts.selected]; + }, + }; +} + +function emptyLog(): PromptLog { + return { notes: [], confirms: [], multiselects: [] }; +} + +const noopGuard = (): Promise => Promise.resolve(); + +test("default answers print a note and write no agent files", async () => { + const { repo, home } = fixture(); + const log = emptyLog(); + const before = await listFiles(repo); + + const outcome = await runInitAgentIntegrations({ + repoRoot: repo, + interactive: true, + guard: noopGuard, + prompts: defaultAnswerPrompts(log), + }); + + expect(outcome).toEqual({ kind: "note-only" }); + expect(log.notes).toEqual([renderInitAgentIntegrationNote()]); + expect(log.notes[0]).toContain("hack agent onboard"); + expect(log.notes[0]).toContain("plugin"); + expect(log.confirms).toHaveLength(1); + expect(log.confirms[0]?.initialValue).toBe(false); + expect(log.multiselects).toEqual([]); + + expect(await listFiles(repo)).toEqual(before); + for (const path of AGENT_PATHS) { + expect(await Bun.file(join(repo, path)).exists()).toBe(false); + expect( + await readdir(join(repo, path)).then( + () => true, + () => false + ) + ).toBe(false); + } + expect(await listFiles(home)).toEqual([]); +}); + +test("export prompt has no preselection and labels repository writes", () => { + expect(INIT_AGENT_EXPORT_CONFIRM.initialValue).toBe(false); + for (const option of INIT_AGENT_EXPORT_OPTIONS) { + expect(option.label).toContain("into this repository"); + } + expect(INIT_AGENT_EXPORT_OPTIONS.map((option) => option.value)).toEqual([ + "cursor", + "claude", + "codex", + "agents", + "mcp", + ]); +}); + +test("opting in without selecting a target writes nothing", async () => { + const { repo, home } = fixture(); + const log = emptyLog(); + const before = await listFiles(repo); + + const outcome = await runInitAgentIntegrations({ + repoRoot: repo, + interactive: true, + guard: noopGuard, + prompts: optInPrompts({ log, selected: [] }), + }); + + expect(outcome).toEqual({ kind: "note-only" }); + expect(log.multiselects).toHaveLength(1); + expect(log.multiselects[0]?.initialValues).toEqual([]); + expect(log.multiselects[0]?.required).toBe(false); + expect(await listFiles(repo)).toEqual(before); + expect(await listFiles(home)).toEqual([]); +}); + +test("non-interactive init neither prompts nor writes", async () => { + const { repo, home } = fixture(); + const log = emptyLog(); + const before = await listFiles(repo); + let guardCalls = 0; + + const outcome = await runInitAgentIntegrations({ + repoRoot: repo, + interactive: false, + guard: () => { + guardCalls += 1; + return Promise.resolve(); + }, + prompts: defaultAnswerPrompts(log), + }); + + expect(outcome).toEqual({ kind: "skipped" }); + expect(guardCalls).toBe(1); + expect(log).toEqual(emptyLog()); + expect(await listFiles(repo)).toEqual(before); + expect(await listFiles(home)).toEqual([]); +}); + +test("explicit Cursor opt-in writes only the Cursor rules file", async () => { + const { repo, home } = fixture(); + const log = emptyLog(); + const before = await listFiles(repo); + let guardCalls = 0; + + const outcome = await runInitAgentIntegrations({ + repoRoot: repo, + interactive: true, + guard: () => { + guardCalls += 1; + return Promise.resolve(); + }, + prompts: optInPrompts({ log, selected: ["cursor"] }), + }); + + expect(outcome.kind).toBe("exported"); + expect(guardCalls).toBe(2); + expect(await listFiles(repo)).toEqual( + [...before, join(".cursor", "rules", "hack.mdc")].sort() + ); + expect(await listFiles(home)).toEqual([]); +}); + +test("explicit Claude opt-in writes only Claude project files", async () => { + const { repo, home } = fixture(); + const before = await listFiles(repo); + + await runInitAgentIntegrations({ + repoRoot: repo, + interactive: true, + guard: noopGuard, + prompts: optInPrompts({ log: emptyLog(), selected: ["claude"] }), + }); + + const added = (await listFiles(repo)).filter( + (path) => !before.includes(path) + ); + expect(added.length).toBeGreaterThan(0); + for (const path of added) { + expect(path.startsWith(".claude/")).toBe(true); + } + expect(added).toContain(join(".claude", "settings.local.json")); + expect(await listFiles(home)).toEqual([]); +}); + +test("a failing ownership guard prevents every export write", async () => { + const { repo, home } = fixture(); + const before = await listFiles(repo); + let guardCalls = 0; + + await expect( + runInitAgentIntegrations({ + repoRoot: repo, + interactive: true, + guard: () => { + guardCalls += 1; + return guardCalls > 1 + ? Promise.reject(new Error("project input changed")) + : Promise.resolve(); + }, + prompts: optInPrompts({ + log: emptyLog(), + selected: ["cursor", "claude", "codex", "agents", "mcp"], + }), + }) + ).rejects.toThrow("project input changed"); + + expect(await listFiles(repo)).toEqual(before); + expect(await listFiles(home)).toEqual([]); +}); + +test("planInitAgentExport maps selections to exactly the selected writes", () => { + expect(planInitAgentExport({ selected: [] })).toEqual({ + cursor: false, + claude: false, + codex: false, + agentDocs: false, + mcpTargets: [], + }); + expect(planInitAgentExport({ selected: ["agents"] })).toEqual({ + cursor: false, + claude: false, + codex: false, + agentDocs: true, + mcpTargets: [], + }); + expect(planInitAgentExport({ selected: ["claude", "mcp"] })).toEqual({ + cursor: false, + claude: true, + codex: false, + agentDocs: false, + mcpTargets: ["claude"], + }); + expect(planInitAgentExport({ selected: ["mcp"] }).mcpTargets).toEqual([ + "cursor", + "claude", + "codex", + ]); +}); diff --git a/tests/integration-sync.test.ts b/tests/integration-sync.test.ts new file mode 100644 index 00000000..3559c550 --- /dev/null +++ b/tests/integration-sync.test.ts @@ -0,0 +1,123 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { installCursorRules } from "../src/agents/cursor.ts"; +import { inspectAgentIntegrationFreshness } from "../src/cli/integration-sync.ts"; + +/** + * Freshness treats optional integrations as current when absent. Every test + * uses a fixture project root and fixture HOME, never the contributor's HOME. + */ + +let root: string | null = null; +let savedHome: string | undefined; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-integration-sync-")); + await mkdir(join(root, "home"), { recursive: true }); + await mkdir(join(root, "repo"), { recursive: true }); + savedHome = process.env.HOME; + process.env.HOME = join(root, "home"); +}); + +afterEach(async () => { + if (savedHome === undefined) { + // biome-ignore lint/performance/noDelete: restoring an unset env var + delete process.env.HOME; + } else { + process.env.HOME = savedHome; + } + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } +}); + +function fixture(): { readonly repo: string; readonly home: string } { + if (!root) { + throw new Error("fixture not initialized"); + } + return { repo: join(root, "repo"), home: join(root, "home") }; +} + +async function status(projectRoot: string | null): Promise { + return (await inspectAgentIntegrationFreshness({ projectRoot })).status; +} + +test("empty project and empty HOME are current", async () => { + const { repo } = fixture(); + expect(await status(repo)).toBe("current"); + expect(await status(null)).toBe("current"); +}); + +test("plugin-only setups without standalone files are current", async () => { + const { repo, home } = fixture(); + const pluginDir = join(home, ".claude", "plugins", "hack"); + await mkdir(pluginDir, { recursive: true }); + await writeFile(join(pluginDir, "plugin.json"), "{}\n"); + await writeFile( + join(home, ".claude", "settings.json"), + `${JSON.stringify({ enabledPlugins: { "hack@hack-dance": true } })}\n` + ); + await mkdir(join(home, ".codex"), { recursive: true }); + await writeFile( + join(home, ".codex", "config.toml"), + '[plugins."hack@hack-dance"]\nenabled = true\n' + ); + await writeFile(join(repo, "AGENTS.md"), "# Team notes\n"); + await writeFile(join(repo, "CLAUDE.md"), "Use bun.\n"); + await writeFile( + join(repo, ".mcp.json"), + `${JSON.stringify({ mcpServers: { other: { command: "other" } } })}\n` + ); + + expect(await status(repo)).toBe("current"); +}); + +test("a current installed integration stays current", async () => { + const { repo } = fixture(); + await installCursorRules({ scope: "project", projectRoot: repo }); + expect(await status(repo)).toBe("current"); +}); + +test("one stale present file is stale", async () => { + const { repo } = fixture(); + await installCursorRules({ scope: "project", projectRoot: repo }); + const rules = join(repo, ".cursor", "rules", "hack.mdc"); + await writeFile(rules, `${await Bun.file(rules).text()}\nSTALE-PROBE\n`); + expect(await status(repo)).toBe("stale"); +}); + +test("a stale Hack-managed agent doc section is stale", async () => { + const { repo } = fixture(); + await writeFile( + join(repo, "AGENTS.md"), + "# Team notes\n\nold\n\n" + ); + expect(await status(repo)).toBe("stale"); +}); + +test("an unreadable MCP config counts as drift", async () => { + const { repo } = fixture(); + await writeFile(join(repo, ".mcp.json"), "{ not json\n"); + expect(await status(repo)).toBe("stale"); +}); + +test("a retired legacy artifact is stale", async () => { + const { repo, home } = fixture(); + await writeFile( + join(repo, "CLAUDE.md"), + "\nRetired\n\n" + ); + expect(await status(repo)).toBe("stale"); + + await rm(join(repo, "CLAUDE.md")); + const legacySkillDir = join(home, ".codex", "skills", "hack-tickets"); + await mkdir(legacySkillDir, { recursive: true }); + await writeFile( + join(legacySkillDir, "SKILL.md"), + "---\nname: hack-tickets\n---\n" + ); + expect(await status(null)).toBe("stale"); +}); From c005262b854373c0447f794f5116c5bb50f4ad5b Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 22:13:17 -0400 Subject: [PATCH 08/52] fix(env): resolve --env base and default the same way on every command config plan and config validate only treated the literal "base" as the base selection, so `--env default` (and DEFAULT/none) reached the native compiler as an overlay named "default" while env get, env list and the runtime commands selected base layers. Every CLI command that accepts --env now resolves it through one shared resolver (src/cli/env-selection.ts): omitted keeps the configured default overlay, base/default/none in any case select base layers, other names are normalized, and a non-empty value with no usable name characters is refused with the fixed "Invalid --env value." usage error instead of silently selecting base. Authored native dispatch uses the same resolver, and session workspace names stay env-base. docs/env.md gains a command and delivery matrix with one row per CLI_SPEC command that accepts --env; tests/env-command-matrix.test.ts keeps the rows and selector flags in sync with CLI_SPEC. The canonical agent guidance gains an env-composition section linking to the matrix; the content revision, plugin bundle, repository examples and CLI reference are regenerated. Co-Authored-By: Claude Opus 5.5 --- .codex/skills/hack-cli/SKILL.md | 12 +- .cursor/rules/hack.mdc | 12 +- docs/env.md | 55 ++++- docs/reference/cli.md | 42 ++-- examples/basic/AGENTS.md | 11 +- examples/basic/CLAUDE.md | 11 +- plugins/hack/rules/hack.mdc | 10 +- plugins/hack/skills/hack-cli/SKILL.md | 10 +- src/agents/instruction-source.ts | 12 ++ src/agents/integration-revision.ts | 2 +- src/cli/env-selection.ts | 41 ++++ src/cli/options.ts | 4 +- src/commands/config-plan.ts | 6 +- src/commands/config-validate.ts | 7 +- src/commands/env.ts | 12 +- src/commands/project.ts | 12 +- src/commands/session.ts | 12 +- src/lib/native-compose-command.ts | 6 +- src/lib/project.ts | 14 +- tests/env-command-matrix.test.ts | 184 ++++++++++++++++ tests/env-selection-parity.test.ts | 288 ++++++++++++++++++++++++++ 21 files changed, 688 insertions(+), 75 deletions(-) create mode 100644 src/cli/env-selection.ts create mode 100644 tests/env-command-matrix.test.ts create mode 100644 tests/env-selection-parity.test.ts diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index 573e5544..8dd7c9d1 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -12,9 +12,9 @@ Use `hack` as the primary interface for local-first development. ## Integration freshness -- These instructions were generated by hack CLI v4.1.1; use the content revision and command behavior to assess freshness. +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). ## Product boundary @@ -144,6 +144,14 @@ Use `hack` as the primary interface for local-first development. - Execute in workspace with injected env: `hack session exec --env qa --service api "bun db:migrate"` - Stop workspace: `hack session stop ` +## Env composition + +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + ## Branch instances (parallel envs) - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 740f7aee..6795c812 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -7,9 +7,9 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is ## Integration freshness -- These instructions were generated by hack CLI v4.1.1; use the content revision and command behavior to assess freshness. +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). ## Product boundary @@ -64,6 +64,14 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Interactive host shell with injected env: `hack host shell --env --scope `. - Browser/host URL: use `hack open --json` (OAuth aliases are preferred when enabled). Container-to-container traffic should use Compose DNS. +## Env composition + +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. diff --git a/docs/env.md b/docs/env.md index 1d29bafb..a9e1eed5 100644 --- a/docs/env.md +++ b/docs/env.md @@ -136,7 +136,13 @@ Projects can set a default overlay in `.hack/hack.config.json`: } ``` -Use `--env=base` to bypass that default and read only `.hack/hack.env.default.yaml`. +Use `--env=base` to bypass that default and read only `.hack/hack.env.default.yaml` +(plus local overrides). `default` and `none` are aliases for `base`, in any case, on every +command that accepts `--env`; an overlay file named after one of those spellings is never +selected by name. Other names are normalized (for example ` QA ` selects `qa`), and a +non-empty value with no usable name characters is refused with `Invalid --env value.` +See the [command and delivery matrix](#command-and-delivery-matrix) for what each command +does with the selection. ## Metadata-only planning API @@ -206,6 +212,53 @@ Use `--target compose` if you explicitly want the container-oriented compose vie For host execution, `--scope` means "which env scope should Hack inject", not "where should this run." For example, `hack host exec --scope api -- bun db:migrate` still runs on the host. +## Command and delivery matrix + +Every command that accepts `--env` resolves it with the same rule: omitted selects +the configured default overlay (`env.defaultOverlay`); `base`, `default` or `none` +selects base layers only; any other value selects that normalized overlay name. The +table lists every such command; `tests/env-command-matrix.test.ts` fails when a +command gains or loses `--env` without a matching row, or when a row's selectors +drift from the CLI. Commands not listed here refuse `--env` as an option that is not +valid for them (arguments after a standalone `--` belong to the child command). + +The **Selectors** column lists the env-related flags each command accepts. +`--service` selects a Compose service scope, `--scope` selects the env scope injected +into a host command, and `--target` on env and host commands chooses the `host` or +`compose` env view. On `up`, `down` and `restart`, `--target` is execution routing +(`auto|local|remote`), not an env view. + +| Command | Selectors | Resolution timing | Refresh path | Native and compose limits | +| --- | --- | --- | --- | --- | +| `hack up` | `--env`, `--target` | Once, when services are created; values go into Compose containers and lifecycle hooks/processes, and the selection is recorded for the running stack. | `hack restart` or `hack env apply --service `; editing env files does not change running containers. | Authored native projects receive the same selection before dispatch. | +| `hack down` | `--env`, `--target` | At teardown, for lifecycle down hooks only; omitted reuses a named overlay recorded for the running stack, else the configured default. | None; nothing is delivered to containers. | The native runtime backend refuses `--env`. | +| `hack restart` | `--env`, `--target` | When services are recreated; omitted reuses a named overlay recorded for the running stack, else the configured default, and an explicit `--env` (including `base`) wins. | This is the refresh path for the whole stack or the named services. | Authored native projects receive the same selection before dispatch. | +| `hack run` | `--env` | Per invocation, for the new one-off container. | Each run resolves again. | The native runtime backend refuses `--env`; restart with the selection first. | +| `hack exec` | `--env` | Checked at invocation against the env recorded for the running stack; the command runs with the env the container was created with. | `hack restart --env ` | Compose refuses when the requested env differs from the env recorded for the running stack; the native runtime backend refuses `--env`. | +| `hack env list` | `--env`, `--service` | Per invocation, read-only; secret values are masked unless `--show-secrets`. | Not needed. | Reads files; no runtime required. | +| `hack env get` | `--env`, `--service` | Per invocation; prints exactly one plaintext value to stdout. | Not needed. | Reads files; refuses native project inputs. | +| `hack env explain` | `--env`, `--service`, `--target` | Per invocation; reports source, precedence and delivery without the value. | Not needed. | Reads files; no runtime required. | +| `hack env apply` | `--env`, `--service` | Runs `hack restart ` with the same `--env`, recreating one service. | This is the refresh path for one service. | `--service` is required. | +| `hack env add` | `--env`, `--service` | Selects the file written (`hack.env..yaml`, or the base file for `base`/`default`/`none`; `--local` writes the local variant). | Running workloads keep old values until `hack env apply` or `hack restart`. | Mutates only env files. | +| `hack env set` | `--env`, `--service` | Alias for `hack env add`. | Same as `hack env add`. | Mutates only env files. | +| `hack env unset` | `--env`, `--service` | Selects the file the key is removed from, as for `hack env add`. | Same as `hack env add`. | Mutates only env files. | +| `hack env materialize` | `--env`, `--service` | Writes `.hack/.env` once, at invocation. | Rerun `hack env materialize`. | Compatibility output for external tools; runtime injection does not depend on it. | +| `hack env exec` | `--env`, `--service`, `--target` | Per invocation, into one host command (host view by default). | Each invocation resolves again. | Runs on the host, not in a container. | +| `hack env shell` | `--env`, `--service`, `--target` | When the shell starts. | Start a new shell. | Runs on the host, not in a container. | +| `hack host exec` | `--env`, `--scope`, `--target` | Per invocation, into one host command (host view by default). | Each invocation resolves again. | Runs on the host, not in a container. | +| `hack host shell` | `--env`, `--scope`, `--target` | When the shell starts. | Start a new shell. | Runs on the host, not in a container. | +| `hack config validate` | `--env` | Per invocation; resolves the native selection without reading managed env documents. | Not needed. | Authored native projects only; cannot be combined with `--file`. | +| `hack config plan` | `--env` | Per invocation; metadata only (names, scopes, secret flags), never decrypted. | Not needed. | Authored native projects only. | +| `hack session start` | `--env`, `--service` | When the workspace is created; the name gains `.env-` (`.env-base` for base spellings), and `--up` runs `hack up -d` with the same selection. | Start a new workspace or pass `--env` to `hack session exec`. | Injects into the workspace shell, not into containers. | +| `hack session exec` | `--env`, `--service` | Per command; omitted reuses the selection encoded in the workspace name. | Each command resolves again. | Injects into the workspace command, not into containers. | + +MCP tools accept no env selector. They run the CLI without `--env`, so each row's +omitted-`--env` behavior applies. + +An overlay name is a label for a set of env files. It implies no production authority +and no network or instance isolation: a `prod` overlay still runs on the local stack, +and two overlays share the same instance unless you choose a separate branch instance. + ## Common commands Inspect resolved env: diff --git a/docs/reference/cli.md b/docs/reference/cli.md index a135c0c9..dddd00ec 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -107,7 +107,7 @@ hack up [services...] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--branch ` | Run against a branch-specific instance (compose name + hostnames) | | `--detach, -d` | Run in background (docker compose up -d) | | `--profile ` | Enable one or more compose profiles (comma-separated) | @@ -134,7 +134,7 @@ hack down [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--branch ` | Run against a branch-specific instance (compose name + hostnames) | | `--profile ` | Enable one or more compose profiles (comma-separated) | | `--target ` | Execution target routing (auto routes to remote when project execution mode requires it) | @@ -169,7 +169,7 @@ hack restart [services...] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--branch ` | Run against a branch-specific instance (compose name + hostnames) | | `--profile ` | Enable one or more compose profiles (comma-separated) | | `--target ` | Execution target routing (auto routes to remote when project execution mode requires it) | @@ -250,7 +250,7 @@ hack run [cmd...] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--branch ` | Run against a branch-specific instance (compose name + hostnames) | | `--workdir ` | Working directory inside the container (docker compose run -w) | | `--profile ` | Enable one or more compose profiles (comma-separated) | @@ -282,7 +282,7 @@ hack exec [cmd...] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--branch ` | Run against a branch-specific instance (compose name + hostnames) | | `--workdir ` | Working directory inside the container (docker compose run -w) | | `--profile ` | Enable one or more compose profiles (comma-separated) | @@ -593,7 +593,7 @@ Uses the matching bundled Rust compiler. Without --file, discovers a native proj | `--profile ` | Comma-separated declared native profiles | | `--domain ` | Select the generated native routing domain suffix | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--json` | Output JSON (machine-readable) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -618,7 +618,7 @@ Resolves the native project, permitted local env selection and routing preview, | `--profile ` | Comma-separated declared native profiles | | `--domain ` | Select the generated native routing domain suffix | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--json` | Output JSON (machine-readable) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -752,7 +752,7 @@ Reuse the default project workspace when it already exists, or create an isolate | `--new` | Create an isolated workspace instead of reusing the default project workspace | | `--name ` | Suffix for an isolated workspace name (for example: agent-1 -> project--agent-1) | | `--detach, -d` | Create or reuse the workspace without attaching (for GUI/non-TTY use) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Inject env for the selected scope (global or a service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -831,7 +831,7 @@ Queue a command in the workspace without opening a new interactive attach flow. | Option | Description | | --- | --- | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Inject env for the selected scope (global or a service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -2233,7 +2233,7 @@ hack env list [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--json` | Output JSON (machine-readable) | | `--show-secrets` | Print secret values (keychain) in plaintext | | `--service ` | Target scope (global or a discovered service name) | @@ -2265,7 +2265,7 @@ Read one global or service value without starting a runtime. Empty values succee | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -2293,7 +2293,7 @@ hack env explain [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--json` | Output JSON (machine-readable) | | `--service ` | Target scope (global or a discovered service name) | | `--target ` | Env view for host commands (default: host rewrites container-oriented addresses for local host execution) | @@ -2317,7 +2317,7 @@ hack env apply [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--json` | Output JSON (machine-readable) | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | @@ -2347,7 +2347,7 @@ hack env add [key] [value] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--secret` | Store value as an encrypted secure entry | | `--local` | Write to the worktree-local env override file instead of the shared repo env file | | `--service ` | Target scope (global or a discovered service name) | @@ -2378,7 +2378,7 @@ hack env set [key] [value] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--secret` | Store value as an encrypted secure entry | | `--local` | Write to the worktree-local env override file instead of the shared repo env file | | `--service ` | Target scope (global or a discovered service name) | @@ -2402,7 +2402,7 @@ hack env materialize [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | @@ -2432,7 +2432,7 @@ Inject the selected Hack env overlay directly into a one-off host command withou | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Target scope (global or a discovered service name) | | `--target ` | Env view for host commands (default: host rewrites container-oriented addresses for local host execution) | | `--shell ` | Run a shell command string via /bin/sh -lc after env injection so `$VAR` expansion happens inside the child shell | @@ -2460,7 +2460,7 @@ Start an interactive host shell with the selected Hack env overlay injected into | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--service ` | Target scope (global or a discovered service name) | | `--target ` | Env view for host commands (default: host rewrites container-oriented addresses for local host execution) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | @@ -2489,7 +2489,7 @@ hack env unset [key] [options] | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--local` | Write to the worktree-local env override file instead of the shared repo env file | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | @@ -2594,7 +2594,7 @@ Run a one-off command on the host with the selected Hack env overlay injected. U | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--scope ` | Resolve values for one env scope while still running the command on the host | | `--target ` | Env view for host commands (default: host rewrites container-oriented addresses for local host execution) | | `--shell ` | Run a shell command string via /bin/sh -lc after env injection so `$VAR` expansion happens inside the child shell | @@ -2622,7 +2622,7 @@ Start an interactive host shell with the selected Hack env overlay injected. Use | --- | --- | | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | -| `--env ` | Apply an optional env overlay by name (use 'base' to bypass overlays) | +| `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--scope ` | Resolve values for one env scope while still running the command on the host | | `--target ` | Env view for host commands (default: host rewrites container-oriented addresses for local host execution) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index f6d65123..6c61ad36 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -4,9 +4,9 @@ Use `hack` as the single interface for local-first runtime orchestration (compose, DNS/TLS, logs, env, and persistent project workspaces). Integration freshness: -- These instructions were generated by hack CLI v4.1.1; use the content revision and command behavior to assess freshness. +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -122,6 +122,13 @@ Workspaces (mux-managed, tmux-first by default): - Execute in workspace with injected env: `hack session exec --env qa --service api "bun db:migrate"` - Stop workspace: `hack session stop ` +Env composition: +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. - Target one with `--branch ` on up/open/logs/down (for example: `hack up --branch --detach`). diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index f6d65123..6c61ad36 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -4,9 +4,9 @@ Use `hack` as the single interface for local-first runtime orchestration (compose, DNS/TLS, logs, env, and persistent project workspaces). Integration freshness: -- These instructions were generated by hack CLI v4.1.1; use the content revision and command behavior to assess freshness. +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -122,6 +122,13 @@ Workspaces (mux-managed, tmux-first by default): - Execute in workspace with injected env: `hack session exec --env qa --service api "bun db:migrate"` - Stop workspace: `hack session stop ` +Env composition: +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. - Target one with `--branch ` on up/open/logs/down (for example: `hack up --branch --detach`). diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index 59bf4694..cd07cd69 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). ## Product boundary @@ -69,6 +69,14 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Interactive host shell with injected env: `hack host shell --env --scope `. - Browser/host URL: use `hack open --json` (OAuth aliases are preferred when enabled). Container-to-container traffic should use Compose DNS. +## Env composition + +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index 56186dfb..8dd7c9d1 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `c2a3f9669bb6` (version alone is not a freshness guarantee). +- Content revision: `c262dbc28ebc` (version alone is not a freshness guarantee). ## Product boundary @@ -144,6 +144,14 @@ Use `hack` as the primary interface for local-first development. - Execute in workspace with injected env: `hack session exec --env qa --service api "bun db:migrate"` - Stop workspace: `hack session stop ` +## Env composition + +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix + ## Branch instances (parallel envs) - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 16a6c948..a5b1d3f4 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -230,6 +230,18 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Stop workspace: `hack session stop `", ], }, + { + id: "env-composition", + title: "Env composition", + surfaces: ALL_SURFACES, + bullets: [ + "Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last.", + "Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation.", + "Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret.", + "Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`.", + "Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix", + ], + }, { id: "branch-instances", diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index fbc9c7bd..19fe0790 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "c2a3f9669bb6"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "c262dbc28ebc"; diff --git a/src/cli/env-selection.ts b/src/cli/env-selection.ts new file mode 100644 index 00000000..e45b9d08 --- /dev/null +++ b/src/cli/env-selection.ts @@ -0,0 +1,41 @@ +import { + ENV_BASE_SELECTION_ALIASES, + parseEnvConfigSelection, +} from "../lib/project.ts"; +import { CliUsageError } from "./command.ts"; + +const INVALID_ENV_SELECTION_MESSAGE = "Invalid --env value."; + +/** + * Resolve a CLI `--env` value through the one shared selection grammar. + * + * - `undefined` (option omitted): callers apply the configured default overlay. + * - `null`: base layers only. `base`, `default` and `none` (any case) and an + * empty value bypass the configured default overlay. + * - `string`: the canonical overlay name (`normalizeEnvConfigName`). + * + * A non-empty value with no usable overlay characters is refused with a fixed, + * value-free usage error instead of silently selecting the base layers. + * Every CLI command that accepts `--env` must resolve it here so that + * `default`/`base` mean the same layers everywhere. + */ +export function resolveRequestedEnvName(input: { + readonly envOption: string | undefined; +}): string | null | undefined { + const selection = parseEnvConfigSelection(input.envOption); + if (input.envOption === undefined) { + return selection; + } + if (selection === undefined) { + throw new CliUsageError(INVALID_ENV_SELECTION_MESSAGE); + } + const trimmed = input.envOption.trim(); + if ( + selection === null && + trimmed.length > 0 && + !ENV_BASE_SELECTION_ALIASES.has(trimmed.toLowerCase()) + ) { + throw new CliUsageError(INVALID_ENV_SELECTION_MESSAGE); + } + return selection; +} diff --git a/src/cli/options.ts b/src/cli/options.ts index eff1d50b..ceb52ddd 100644 --- a/src/cli/options.ts +++ b/src/cli/options.ts @@ -23,9 +23,9 @@ export const optEnv = defineOption({ name: "env", type: "string", long: "--env", - valueHint: "", + valueHint: "", description: - "Apply an optional env overlay by name (use 'base' to bypass overlays)", + "Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only", } as const); export const optBranch = defineOption({ diff --git a/src/commands/config-plan.ts b/src/commands/config-plan.ts index 178d513b..52951068 100644 --- a/src/commands/config-plan.ts +++ b/src/commands/config-plan.ts @@ -5,6 +5,7 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPath } from "../cli/options.ts"; import { HackCliError } from "../lib/cli-result.ts"; import { @@ -49,6 +50,9 @@ export const configPlanCommand = withHandler(spec, async ({ ctx, args }) => { if (profiles?.some((name) => name.length === 0)) { throw new CliUsageError("Native profile names must not be empty."); } + const explicitOverlay = resolveRequestedEnvName({ + envOption: args.options.env, + }); const controller = new AbortController(); const cancel = () => controller.abort(); process.once("SIGINT", cancel); @@ -58,7 +62,7 @@ export const configPlanCommand = withHandler(spec, async ({ ctx, args }) => { startDir: resolve(ctx.cwd, args.options.path ?? "."), profiles, explicitDomain: args.options.domain, - explicitOverlay: args.options.env === "base" ? null : args.options.env, + explicitOverlay, signal: controller.signal, }); renderPlan({ result, json: args.options.json === true }); diff --git a/src/commands/config-validate.ts b/src/commands/config-validate.ts index 706e1368..3fe36e1d 100644 --- a/src/commands/config-validate.ts +++ b/src/commands/config-validate.ts @@ -5,6 +5,7 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPath } from "../cli/options.ts"; import { HackCliError } from "../lib/cli-result.ts"; import { @@ -77,6 +78,9 @@ export const configValidateCommand = withHandler( if (profiles?.some((name) => name.length === 0)) { throw new CliUsageError("Native profile names must not be empty."); } + const explicitOverlay = resolveRequestedEnvName({ + envOption: args.options.env, + }); const controller = new AbortController(); const cancel = () => controller.abort(); process.once("SIGINT", cancel); @@ -88,8 +92,7 @@ export const configValidateCommand = withHandler( startDir: resolve(ctx.cwd, args.options.path ?? "."), profiles, explicitDomain: args.options.domain, - explicitOverlay: - args.options.env === "base" ? null : args.options.env, + explicitOverlay, signal: controller.signal, }) : await compileNativeConfig({ diff --git a/src/commands/env.ts b/src/commands/env.ts index d867a9e8..3e9e3a23 100644 --- a/src/commands/env.ts +++ b/src/commands/env.ts @@ -9,6 +9,7 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPath, optProject } from "../cli/options.ts"; import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { updateGlobalConfig } from "../lib/config.ts"; @@ -50,7 +51,6 @@ import type { ProjectContext } from "../lib/project.ts"; import { defaultProjectSlugFromPath, findProjectContext, - parseEnvConfigSelection, readProjectConfig, sanitizeProjectSlug, } from "../lib/project.ts"; @@ -628,16 +628,6 @@ async function resolveProjectName(project: ProjectContext): Promise { return sanitizeProjectSlug(raw.length > 0 ? raw : derived); } -function resolveRequestedEnvName(input: { - readonly envOption: string | undefined; -}): string | null | undefined { - const envName = parseEnvConfigSelection(input.envOption); - if (input.envOption !== undefined && envName === undefined) { - throw new CliUsageError("Invalid --env value."); - } - return envName; -} - async function resolveConfiguredSecretStore(input: { readonly project: ProjectContext; readonly projectName: string; diff --git a/src/commands/project.ts b/src/commands/project.ts index 38e6bf88..112e5915 100644 --- a/src/commands/project.ts +++ b/src/commands/project.ts @@ -77,6 +77,7 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optBranch, optDetach, @@ -219,7 +220,6 @@ import { findProjectContext, findRepoRootForInit, type ProjectLifecycleCommand, - parseEnvConfigSelection, readProjectConfig, readProjectDevHost, resolveProjectOauthAliasHost, @@ -676,16 +676,6 @@ function resolveStartDir(ctx: CliContext, pathOpt: string | undefined): string { return pathOpt ? resolve(ctx.cwd, pathOpt) : ctx.cwd; } -function resolveRequestedEnvName(input: { - readonly envOption: string | undefined; -}): string | null | undefined { - const envName = parseEnvConfigSelection(input.envOption); - if (input.envOption !== undefined && envName === undefined) { - throw new CliUsageError("Invalid --env value."); - } - return envName; -} - export function resolveRequestedComposeServices(opts: { readonly requested: readonly string[] | undefined; readonly available: readonly string[]; diff --git a/src/commands/session.ts b/src/commands/session.ts index d260b5a8..d300eac4 100644 --- a/src/commands/session.ts +++ b/src/commands/session.ts @@ -11,6 +11,7 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPretty } from "../cli/options.ts"; import { readHackEnvContract, @@ -22,7 +23,6 @@ import { defaultProjectSlugFromPath, findProjectContext, type ProjectContext, - parseEnvConfigSelection, readProjectConfig, } from "../lib/project.ts"; import { @@ -880,16 +880,6 @@ function resolveRunUpCwd(opts: { return opts.project.repoRoot; } -function resolveRequestedEnvName(opts: { - readonly envOption: string | undefined; -}): string | null | undefined { - const envName = parseEnvConfigSelection(opts.envOption); - if (opts.envOption !== undefined && envName === undefined) { - throw new CliUsageError("Invalid --env value."); - } - return envName; -} - function resolveRequestedServiceName(opts: { readonly serviceOption: string | undefined; }): string | null { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index e6ee1586..a89a0f17 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -2,6 +2,7 @@ import { randomBytes } from "node:crypto"; import type { LogOutputFormat } from "../backends/log-backend.ts"; import type { RuntimeBaseOptions } from "../backends/runtime-backend.ts"; import { CliUsageError } from "../cli/command.ts"; +import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { emitCliResult, errorResult, @@ -135,7 +136,6 @@ import { import { waitNativeComposeReady } from "./native-compose-wait-ready.ts"; import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; -import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; @@ -2319,7 +2319,9 @@ async function dispatchNativeComposeCommand( const options = { ...input, overlay: - input.overlay === null ? null : parseEnvConfigSelection(input.overlay), + input.overlay === null + ? null + : resolveRequestedEnvName({ envOption: input.overlay }), }; const adopted = await tryLegacyComposeAdoptedCommand(input); if (adopted !== null) { diff --git a/src/lib/project.ts b/src/lib/project.ts index ba163a11..7a54cea5 100644 --- a/src/lib/project.ts +++ b/src/lib/project.ts @@ -122,6 +122,17 @@ export function normalizeEnvConfigName(input: string): string | null { return normalized.length > 0 ? normalized : null; } +/** + * `--env` spellings (case-insensitive) that select the base layers and bypass + * the configured default overlay. An overlay file named for one of these + * spellings is never selected by name. + */ +export const ENV_BASE_SELECTION_ALIASES: ReadonlySet = new Set([ + "base", + "default", + "none", +]); + export function parseEnvConfigSelection( input: string | undefined ): string | null | undefined { @@ -134,8 +145,7 @@ export function parseEnvConfigSelection( return null; } - const lowered = trimmed.toLowerCase(); - if (lowered === "base" || lowered === "default" || lowered === "none") { + if (ENV_BASE_SELECTION_ALIASES.has(trimmed.toLowerCase())) { return null; } diff --git a/tests/env-command-matrix.test.ts b/tests/env-command-matrix.test.ts new file mode 100644 index 00000000..2bfdadae --- /dev/null +++ b/tests/env-command-matrix.test.ts @@ -0,0 +1,184 @@ +import { describe, expect, test } from "bun:test"; +import { resolve } from "node:path"; +import type { AnyCommandSpec } from "../src/cli/command.ts"; +import { optEnv } from "../src/cli/options.ts"; +import { runCli } from "../src/cli/run.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; + +/** + * Source check for the "Command and delivery matrix" in docs/env.md: its rows + * must be exactly the CLI_SPEC command paths that accept `--env`, and each + * row's Selectors cell must list exactly the env-related flags the command + * accepts. A command that gains or loses `--env` fails here until the matrix + * row is added or removed. + */ + +const DOC_PATH = resolve(import.meta.dir, "..", "docs", "env.md"); +const SECTION_HEADING = "## Command and delivery matrix"; +const SELECTOR_FLAGS: ReadonlySet = new Set([ + "--env", + "--service", + "--scope", + "--target", +]); +const ROW_PATTERN = /^\| `hack ([a-z][a-z -]*)` \| ([^|]*) \|/; +const FLAG_PATTERN = /`(--[a-z-]+)`/g; + +type EnvCommand = { + readonly path: string; + readonly spec: AnyCommandSpec; +}; + +function envCommands( + commands: readonly AnyCommandSpec[], + prefix: readonly string[] = [] +): EnvCommand[] { + const found: EnvCommand[] = []; + for (const command of commands) { + const path = [...prefix, command.name]; + if (command.options.some((option) => option.long === "--env")) { + found.push({ path: path.join(" "), spec: command }); + } + found.push(...envCommands(command.subcommands, path)); + } + return found; +} + +function matrixRows(doc: string): Map> { + const start = doc.indexOf(SECTION_HEADING); + expect(start).toBeGreaterThanOrEqual(0); + const rest = doc.slice(start + SECTION_HEADING.length); + const end = rest.indexOf("\n## "); + const section = end >= 0 ? rest.slice(0, end) : rest; + const rows = new Map>(); + for (const line of section.split("\n")) { + const match = ROW_PATTERN.exec(line); + if (!match) { + continue; + } + const path = match[1] ?? ""; + expect(rows.has(path), `duplicate matrix row: hack ${path}`).toBe(false); + rows.set( + path, + new Set( + [...(match[2] ?? "").matchAll(FLAG_PATTERN)].map( + (flag) => flag[1] ?? "" + ) + ) + ); + } + return rows; +} + +function selectorFlags(spec: AnyCommandSpec): ReadonlySet { + return new Set( + spec.options + .map((option) => option.long) + .filter((long) => SELECTOR_FLAGS.has(long)) + ); +} + +/** Differences between the documented matrix and CLI_SPEC; empty when in sync. */ +function matrixDrift(doc: string): string[] { + const rows = matrixRows(doc); + const commands = envCommands(CLI_SPEC.commands); + const drift: string[] = []; + for (const { path, spec } of commands) { + const documented = rows.get(path); + if (!documented) { + drift.push(`missing row: hack ${path}`); + continue; + } + const expected = [...selectorFlags(spec)].sort(); + const actual = [...documented].sort(); + if (JSON.stringify(expected) !== JSON.stringify(actual)) { + drift.push( + `selector drift for hack ${path}: expected ${expected.join(",")}, documented ${actual.join(",")}` + ); + } + } + const known = new Set(commands.map((command) => command.path)); + for (const path of rows.keys()) { + if (!known.has(path)) { + drift.push(`row without --env in CLI_SPEC: hack ${path}`); + } + } + return drift; +} + +describe("env command and delivery matrix", () => { + test("docs/env.md rows equal the CLI_SPEC commands that accept --env", async () => { + const doc = await Bun.file(DOC_PATH).text(); + expect(matrixDrift(doc)).toEqual([]); + expect(matrixRows(doc).size).toBe(envCommands(CLI_SPEC.commands).length); + }); + + test("every --env option shares the one optEnv definition", () => { + for (const { path, spec } of envCommands(CLI_SPEC.commands)) { + const option = spec.options.find( + (candidate) => candidate.long === "--env" + ); + expect(option, `hack ${path} defines its own --env`).toBe(optEnv); + } + }); + + test("negative control: removing or adding a matrix row is detected", async () => { + const doc = await Bun.file(DOC_PATH).text(); + const configPlanRow = doc + .split("\n") + .find((line) => line.startsWith("| `hack config plan` |")); + expect(configPlanRow).toBeDefined(); + expect(matrixDrift(doc.replace(`${configPlanRow}\n`, ""))).toEqual([ + "missing row: hack config plan", + ]); + expect( + matrixDrift( + doc.replace( + `${configPlanRow}\n`, + `${configPlanRow}\n| \`hack logs\` | \`--env\` | x | x | x |\n` + ) + ) + ).toEqual(["row without --env in CLI_SPEC: hack logs"]); + expect( + matrixDrift( + doc.replace( + `${configPlanRow}\n`, + `${configPlanRow?.replace("| `--env` |", "| `--env`, `--service` |")}\n` + ) + ) + ).toEqual([ + "selector drift for hack config plan: expected --env, documented --env,--service", + ]); + }); + + test("negative control: a command without --env refuses the flag as unsupported", async () => { + const commands: readonly AnyCommandSpec[] = CLI_SPEC.commands; + const ps = commands.find((command) => command.name === "ps"); + expect(ps).toBeDefined(); + expect(ps?.options.some((option) => option.long === "--env")).toBe(false); + expect(ps?.allowUnknownOptions).not.toBe(true); + const stdout: Buffer[] = []; + const originalOut = process.stdout.write; + const originalErr = process.stderr.write; + process.stdout.write = ((chunk: string | Uint8Array) => { + stdout.push(Buffer.from(chunk)); + return true; + }) as typeof process.stdout.write; + process.stderr.write = (() => true) as typeof process.stderr.write; + let code: number; + try { + code = await runCli(["ps", "--env", "qa", "--json"]); + } finally { + process.stdout.write = originalOut; + process.stderr.write = originalErr; + } + expect(code).not.toBe(0); + expect(JSON.parse(Buffer.concat(stdout).toString())).toEqual({ + ok: false, + error: { + code: "E_USAGE", + message: 'Option(s) not valid for "ps": --env', + }, + }); + }); +}); diff --git a/tests/env-selection-parity.test.ts b/tests/env-selection-parity.test.ts new file mode 100644 index 00000000..52f174ba --- /dev/null +++ b/tests/env-selection-parity.test.ts @@ -0,0 +1,288 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { + mkdir, + mkdtemp, + readdir, + realpath, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { CliUsageError } from "../src/cli/command.ts"; +import { resolveRequestedEnvName } from "../src/cli/env-selection.ts"; +import { optEnv } from "../src/cli/options.ts"; +import { runCli } from "../src/cli/run.ts"; +import { __testOnlySessionCommand } from "../src/commands/session.ts"; +import { tryNativeComposeCommand } from "../src/lib/native-compose-command.ts"; +import { restoreEnv } from "./helpers/env.ts"; +import { requireTestConfigCompiler } from "./helpers/test-config-compiler.ts"; + +/** + * One `--env` grammar across commands: `base`, `default` and `none` (any case) + * select the base layers and bypass the configured default overlay, and an + * unusable name is refused by every surface instead of silently selecting base. + * + * Compose-style projects back `env get`/`env list`; the authored native + * project backs metadata-only `config plan` through the real compiler. Both + * configure `qa` as the default overlay so omission and base differ. + */ + +const realCompiler = requireTestConfigCompiler(); +const BASE_SPELLINGS: string[] = ["base", "default", "DEFAULT", "none"]; +const INVALID_NAME = "!!!"; +const ENV_NAMES = [ + "HACK_GLOBAL_CONFIG_PATH", + "HACK_CONFIG_COMPILER_BINARY", + "HACK_ENV_SECRET_KEY", + "HACK_SECRETS_FILE_KEY", + "HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK", + "HACK_EXECUTION_MODE", + "HACK_HOME", + "HOME", + "CI", +] as const; +const savedEnv = new Map(); +let root = ""; +let composeRoot = ""; +let nativeRoot = ""; + +beforeEach(async () => { + // Native inputs must live in unredirected directories (no /var symlink). + root = await realpath( + await mkdtemp(join(tmpdir(), "hack-env-selection-parity-")) + ); + composeRoot = join(root, "compose"); + nativeRoot = join(root, "native"); + for (const name of ENV_NAMES) { + savedEnv.set(name, process.env[name]); + Reflect.deleteProperty(process.env, name); + } + process.env.HOME = join(root, "home"); + process.env.HACK_HOME = join(root, "hack-home"); + process.env.HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK = "1"; + process.env.HACK_GLOBAL_CONFIG_PATH = join(root, "global.json"); + process.env.HACK_CONFIG_COMPILER_BINARY = realCompiler; + await writeFile(process.env.HACK_GLOBAL_CONFIG_PATH, "{}"); + + await mkdir(join(composeRoot, ".hack"), { recursive: true }); + await writeFile( + join(composeRoot, ".hack/hack.config.json"), + JSON.stringify({ name: "parity-compose", env: { defaultOverlay: "qa" } }) + ); + await writeFile( + join(composeRoot, ".hack/docker-compose.yml"), + "services:\n api:\n image: alpine:3.20\n" + ); + await writeLayers(composeRoot); + + await mkdir(join(nativeRoot, ".hack"), { recursive: true }); + await writeFile( + join(nativeRoot, ".hack/hack.project.json"), + JSON.stringify({ + schema_version: 1, + name: "parity-native", + environment: { default_overlay: "qa" }, + worktree: { inherit_local: false, auto_branch: false }, + services: { web: { image: `sha256:${"a".repeat(64)}` } }, + }) + ); + await writeLayers(nativeRoot); +}); + +afterEach(async () => { + for (const [name, value] of savedEnv) { + restoreEnv(name, value); + } + savedEnv.clear(); + await rm(root, { recursive: true, force: true }); +}); + +async function writeLayers(projectRoot: string): Promise { + for (const name of ["default", "qa"]) { + await writeFile( + join(projectRoot, `.hack/hack.env.${name}.yaml`), + JSON.stringify({ + version: 1, + environment: name, + secretsprovider: "project_key", + values: { global: { VALUE: name === "default" ? "base" : "qa" } }, + }) + ); + } +} + +async function capture(args: readonly string[]) { + const stdout: Buffer[] = []; + const stderr: Buffer[] = []; + const originalOut = process.stdout.write; + const originalErr = process.stderr.write; + process.stdout.write = ((chunk: string | Uint8Array) => { + stdout.push(Buffer.from(chunk)); + return true; + }) as typeof process.stdout.write; + process.stderr.write = ((chunk: string | Uint8Array) => { + stderr.push(Buffer.from(chunk)); + return true; + }) as typeof process.stderr.write; + try { + const code = await runCli(args); + return { + code, + stdout: Buffer.concat(stdout).toString(), + stderr: Buffer.concat(stderr).toString(), + }; + } finally { + process.stdout.write = originalOut; + process.stderr.write = originalErr; + } +} + +function envArgs(value: string | undefined): readonly string[] { + return value === undefined ? [] : ["--env", value]; +} + +/** Selected overlay per surface; `null` is base layers only. */ +async function selections(value: string | undefined) { + const plan = await capture([ + "config", + "plan", + "--path", + nativeRoot, + ...envArgs(value), + "--json", + ]); + const get = await capture([ + "env", + "get", + "VALUE", + "--path", + composeRoot, + ...envArgs(value), + ]); + const list = await capture([ + "env", + "list", + "--path", + composeRoot, + ...envArgs(value), + "--json", + ]); + return { plan, get, list }; +} + +function planOverlay(result: { readonly stdout: string }): unknown { + const parsed: unknown = JSON.parse(result.stdout); + expect(parsed).toHaveProperty("ok", true); + expect(parsed).toHaveProperty("environment_plan.overlay"); + return (parsed as { environment_plan: { overlay: unknown } }).environment_plan + .overlay; +} + +function listOverlay(result: { readonly stdout: string }): unknown { + const parsed: unknown = JSON.parse(result.stdout); + expect(parsed).toHaveProperty("env_selection.effective"); + return (parsed as { env_selection: { effective: unknown } }).env_selection + .effective; +} + +test("omitting --env selects the configured default overlay on every surface", async () => { + const { plan, get, list } = await selections(undefined); + expect([plan.code, get.code, list.code]).toEqual([0, 0, 0]); + expect(planOverlay(plan)).toBe("qa"); + expect(get.stdout).toBe("qa"); + expect(listOverlay(list)).toBe("qa"); +}); + +test.each( + BASE_SPELLINGS +)("--env %s selects base layers on config plan, env get and env list alike", async (spelling) => { + const { plan, get, list } = await selections(spelling); + expect([plan.code, get.code, list.code]).toEqual([0, 0, 0]); + // Regression: config plan previously forwarded `default`/`DEFAULT`/`none` + // as an overlay name, selecting `.hack/hack.env.default.yaml` as an overlay. + expect(planOverlay(plan)).toBeNull(); + expect(get.stdout).toBe("base"); + expect(listOverlay(list)).toBeNull(); +}); + +test("an unusable --env name is refused by every surface without reading values", async () => { + const before = await readdir(join(nativeRoot, ".hack")); + const { plan, get, list } = await selections(INVALID_NAME); + for (const result of [plan, get, list]) { + expect(result.code).not.toBe(0); + } + // JSON surfaces report the fixed usage envelope; `get` keeps stdout empty. + for (const result of [plan, list]) { + expect(JSON.parse(result.stdout)).toEqual({ + ok: false, + error: { code: "E_USAGE", message: "Invalid --env value." }, + }); + } + expect(get.stdout).toBe(""); + expect(get.stderr).toContain("Unable to read env value."); + const validate = await capture([ + "config", + "validate", + "--path", + nativeRoot, + "--env", + INVALID_NAME, + "--json", + ]); + expect(validate.code).not.toBe(0); + expect(JSON.parse(validate.stdout)).toHaveProperty("error.code", "E_USAGE"); + expect(await readdir(join(nativeRoot, ".hack"))).toEqual(before); +}); + +test("config validate shares the base spellings with config plan", async () => { + for (const spelling of BASE_SPELLINGS) { + const result = await capture([ + "config", + "validate", + "--path", + nativeRoot, + "--env", + spelling, + "--json", + ]); + expect(result.code).toBe(0); + expect(JSON.parse(result.stdout)).toHaveProperty( + "local_resolution.overlay", + null + ); + } +}); + +test("authored native dispatch refuses an unusable --env before selecting a project", async () => { + const error: unknown = await tryNativeComposeCommand({ + cwd: nativeRoot, + path: nativeRoot, + operation: "ps", + overlay: INVALID_NAME, + }).catch((value: unknown) => value); + expect(error).toBeInstanceOf(CliUsageError); + expect(error).toHaveProperty("message", "Invalid --env value."); +}); + +test("the shared resolver maps base spellings to null and keeps workspace names env-base", () => { + const { buildScopedWorkspaceBaseName } = __testOnlySessionCommand; + expect(resolveRequestedEnvName({ envOption: undefined })).toBeUndefined(); + expect(resolveRequestedEnvName({ envOption: " QA " })).toBe("qa"); + expect(() => resolveRequestedEnvName({ envOption: INVALID_NAME })).toThrow( + "Invalid --env value." + ); + for (const spelling of BASE_SPELLINGS) { + const envName = resolveRequestedEnvName({ envOption: spelling }); + expect(envName).toBeNull(); + expect( + buildScopedWorkspaceBaseName({ + projectName: "parity", + envOptionSpecified: true, + envName, + serviceName: null, + }) + ).toBe("parity.env-base"); + } + expect(optEnv.valueHint).toBe(""); +}); From 6136946efe6118e5bcd321bd0350e35b350bf55d Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 22:35:27 -0400 Subject: [PATCH 09/52] fix(agents): count corrupted and unreadable integrations as drift Address the W10 verifier findings so integration sync and doctor agree: - checkAgentDocs marks a doc `unmanaged` only when it has neither the Hack start nor end marker. A doc with exactly one marker is a corrupted managed section and now reports a malformed-marker error, which sync counts as drift. Doctor treats the same doc as stale. - Add readOptionalTextFile, which uses lstat before reading so callers can tell "absent" from "present but unreadable". The Cursor rules, Codex skill, shared skill, hack-init skill and agent-doc checks now report an unreadable existing file as `error` (drift) instead of `missing` (optional, current). Doctor uses the same helper. - Add regression tests for start-only and end-only docs, plus mode-000 Hack-owned files and docs in fixture HOME/project roots, for both sync and doctor. Default `hack init` still writes no repository files. Co-Authored-By: Claude Opus 5.5 --- src/agents/codex-skill.ts | 13 +++- src/agents/cursor.ts | 13 +++- src/agents/hack-init-skill.ts | 14 +++- src/agents/shared-skill.ts | 12 ++- src/cli/integration-sync.ts | 7 +- src/commands/doctor.ts | 44 ++++++++--- src/lib/fs.ts | 44 ++++++++++- src/mcp/agent-docs.ts | 133 +++++++++++++++++++++------------ tests/doctor-command.test.ts | 109 ++++++++++++++++++++++++++- tests/integration-sync.test.ts | 70 ++++++++++++++++- 10 files changed, 387 insertions(+), 72 deletions(-) diff --git a/src/agents/codex-skill.ts b/src/agents/codex-skill.ts index 4f8b80fa..ecfefa82 100644 --- a/src/agents/codex-skill.ts +++ b/src/agents/codex-skill.ts @@ -4,7 +4,7 @@ import { dirname, resolve } from "node:path"; import { ensureDir, pathExists, - readTextFile, + readOptionalTextFile, writeTextFileIfChanged, } from "../lib/fs.ts"; import { @@ -108,7 +108,16 @@ export async function checkCodexSkill(opts: { } const path = resolved.path; - const content = await readTextFile(path); + const file = await readOptionalTextFile(path); + if (file.kind === "unreadable") { + return { + scope: opts.scope, + status: "error", + path, + message: "Hack-managed file exists but could not be read.", + }; + } + const content = file.kind === "text" ? file.text : ""; if (!content) { return { scope: opts.scope, status: "missing", path }; } diff --git a/src/agents/cursor.ts b/src/agents/cursor.ts index 27462835..97cdb365 100644 --- a/src/agents/cursor.ts +++ b/src/agents/cursor.ts @@ -4,7 +4,7 @@ import { dirname, resolve } from "node:path"; import { ensureDir, pathExists, - readTextFile, + readOptionalTextFile, writeTextFileIfChanged, } from "../lib/fs.ts"; import { @@ -80,7 +80,16 @@ export async function checkCursorRules(opts: { } const path = resolved.path; - const content = await readTextFile(path); + const file = await readOptionalTextFile(path); + if (file.kind === "unreadable") { + return { + scope: opts.scope, + status: "error", + path, + message: "Hack-managed file exists but could not be read.", + }; + } + const content = file.kind === "text" ? file.text : ""; if (!content) { return { scope: opts.scope, status: "missing", path }; } diff --git a/src/agents/hack-init-skill.ts b/src/agents/hack-init-skill.ts index 2f3c9e67..1744320a 100644 --- a/src/agents/hack-init-skill.ts +++ b/src/agents/hack-init-skill.ts @@ -4,7 +4,7 @@ import { dirname, resolve } from "node:path"; import { ensureDir, pathExists, - readTextFile, + readOptionalTextFile, writeTextFileIfChanged, } from "../lib/fs.ts"; import { normalizeInstructionText } from "./instruction-source.ts"; @@ -129,7 +129,17 @@ export async function checkHackInitSkill(opts: { } const path = resolved.path; - const content = await readTextFile(path); + const file = await readOptionalTextFile(path); + if (file.kind === "unreadable") { + return { + client: opts.client, + scope: opts.scope, + status: "error", + path, + message: "Hack-managed file exists but could not be read.", + }; + } + const content = file.kind === "text" ? file.text : ""; if (!content) { return { client: opts.client, scope: opts.scope, status: "missing", path }; } diff --git a/src/agents/shared-skill.ts b/src/agents/shared-skill.ts index 5b5a8828..8175eef4 100644 --- a/src/agents/shared-skill.ts +++ b/src/agents/shared-skill.ts @@ -4,7 +4,7 @@ import { dirname, resolve } from "node:path"; import { ensureDir, pathExists, - readTextFile, + readOptionalTextFile, writeTextFileIfChanged, } from "../lib/fs.ts"; import { renderCodexSkill } from "./codex-skill.ts"; @@ -56,7 +56,15 @@ export async function checkSharedHackSkill(): Promise { return { status: "error", path: SKILL_FILENAME, message: resolved.message }; } - const content = await readTextFile(resolved.path); + const file = await readOptionalTextFile(resolved.path); + if (file.kind === "unreadable") { + return { + status: "error", + path: resolved.path, + message: "Hack-managed file exists but could not be read.", + }; + } + const content = file.kind === "text" ? file.text : ""; if (!content) { return { status: "missing", path: resolved.path }; } diff --git a/src/cli/integration-sync.ts b/src/cli/integration-sync.ts index ac72dd36..977ad121 100644 --- a/src/cli/integration-sync.ts +++ b/src/cli/integration-sync.ts @@ -103,7 +103,9 @@ async function detectIntegrationDrift(opts: { // Optional integrations are not required: an absent ("missing") integration is // current. Only present-but-stale or unverifiable ("error") checks are drift. -// Retired legacy artifacts are drift whenever they are present. +// A Hack-owned file that exists but cannot be read is reported as "error" by +// its check, so it counts as drift (matching doctor). Retired legacy +// artifacts are drift whenever they are present. function hasSingleCheckDrift(status: string): boolean { return status === "stale" || status === "error"; } @@ -118,7 +120,8 @@ function hasMcpDrift(opts: { function hasDocDrift(opts: { readonly checks: readonly AgentDocCheckResult[]; }): boolean { - // A user-owned AGENTS.md/CLAUDE.md without a Hack section is not drift. + // A user-owned AGENTS.md/CLAUDE.md with neither Hack marker is not drift. + // A doc with only one marker (corrupted section) or an unreadable doc is. return opts.checks.some( (check) => check.status === "stale" || diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index 4b50f6ef..77a49486 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -80,6 +80,7 @@ import { ensureDir, ensureGitignoreEntry, pathExists, + readOptionalTextFile, readTextFile, writeTextFileIfChanged, } from "../lib/fs.ts"; @@ -3528,9 +3529,10 @@ async function maybeRepairHackd(): Promise { /** * Optional integrations are current when absent: a path is stale only when it - * exists and lacks the current revision marker. AGENTS.md/CLAUDE.md are - * user-owned, so they count only when they carry a Hack-managed section. - * Retained legacy artifacts are always stale. + * exists and lacks the current revision marker, or exists but cannot be read. + * AGENTS.md/CLAUDE.md are user-owned, so they count only when they carry a + * Hack start or end marker; a section with only one of the two markers is + * corrupted and stale. Retained legacy artifacts are always stale. */ export async function inspectDoctorAgentIntegrations(opts: { readonly projectRoot: string | null; @@ -3582,15 +3584,35 @@ export async function inspectDoctorAgentIntegrations(opts: { (content) => content.present && (content.text === null || - (content.text.includes(HACK_AGENT_DOCS_SECTION_MARKER) && - !content.text.includes(marker))) + isStaleAgentDocSection({ text: content.text, marker })) ); return { status: hasStalePath || hasLegacyArtifacts ? "stale" : "current", }; } -const HACK_AGENT_DOCS_SECTION_MARKER = ""; +const HACK_AGENT_DOCS_SECTION_START = ""; +const HACK_AGENT_DOCS_SECTION_END = ""; + +/** + * A user-owned doc with neither Hack marker is not stale. A doc with only one + * marker is a corrupted managed section; a complete section is stale when it + * lacks the current revision marker. + */ +function isStaleAgentDocSection(opts: { + readonly text: string; + readonly marker: string; +}): boolean { + const hasStart = opts.text.includes(HACK_AGENT_DOCS_SECTION_START); + const hasEnd = opts.text.includes(HACK_AGENT_DOCS_SECTION_END); + if (!(hasStart || hasEnd)) { + return false; + } + if (!(hasStart && hasEnd)) { + return true; + } + return !opts.text.includes(opts.marker); +} async function readOptionalIntegrationText( path: string @@ -3598,13 +3620,11 @@ async function readOptionalIntegrationText( | { readonly present: false } | { readonly present: true; readonly text: string | null } > { - const text = await readTextFile(path); - if (text !== null) { - return { present: true, text }; + const file = await readOptionalTextFile(path); + if (file.kind === "absent") { + return { present: false }; } - return (await pathExists(path)) - ? { present: true, text: null } - : { present: false }; + return { present: true, text: file.kind === "text" ? file.text : null }; } async function resolveDaemonReportForDoctorFix(): Promise< diff --git a/src/lib/fs.ts b/src/lib/fs.ts index 7b0e49ef..f289e353 100644 --- a/src/lib/fs.ts +++ b/src/lib/fs.ts @@ -1,4 +1,4 @@ -import { mkdir, stat } from "node:fs/promises"; +import { lstat, mkdir, stat } from "node:fs/promises"; export async function pathExists(absolutePath: string): Promise { try { @@ -28,6 +28,48 @@ export async function readTextFile( } } +/** + * Result of reading an optional file that distinguishes a path that does not + * exist from one that exists but cannot be read. + */ +export type OptionalTextFile = + | { readonly kind: "absent" } + | { readonly kind: "unreadable" } + | { readonly kind: "text"; readonly text: string }; + +/** + * Read an optional text file without collapsing "present but unreadable" into + * "absent". + * + * Existence is decided with `lstat` before reading, so a path is `absent` only + * when it (or a parent component) does not exist. Any other lstat failure + * (for example a parent directory without search permission), a dangling + * symlink, or a read failure (for example mode 000) is `unreadable`. Callers + * that check Hack-managed files must treat `unreadable` as drift rather than + * as an optional integration that was never installed. + */ +export async function readOptionalTextFile( + absolutePath: string +): Promise { + try { + await lstat(absolutePath); + } catch (error: unknown) { + return isMissingPathError(error) + ? { kind: "absent" } + : { kind: "unreadable" }; + } + const text = await readTextFile(absolutePath); + return text === null ? { kind: "unreadable" } : { kind: "text", text }; +} + +function isMissingPathError(error: unknown): boolean { + if (typeof error !== "object" || error === null || !("code" in error)) { + return false; + } + const code = (error as { readonly code: unknown }).code; + return code === "ENOENT" || code === "ENOTDIR"; +} + export async function writeTextFile( absolutePath: string, content: string diff --git a/src/mcp/agent-docs.ts b/src/mcp/agent-docs.ts index 7668f882..ee529022 100644 --- a/src/mcp/agent-docs.ts +++ b/src/mcp/agent-docs.ts @@ -4,7 +4,12 @@ import { normalizeInstructionText, renderInstructionSections, } from "../agents/instruction-source.ts"; -import { pathExists, readTextFile, writeTextFileIfChanged } from "../lib/fs.ts"; +import { + pathExists, + readOptionalTextFile, + readTextFile, + writeTextFileIfChanged, +} from "../lib/fs.ts"; export type AgentDocTarget = "agents" | "claude"; @@ -20,7 +25,11 @@ export type AgentDocCheckResult = { readonly status: "present" | "stale" | "missing" | "error"; readonly path: string; readonly message?: string; - /** Set when the file exists but has no Hack-managed section at all. */ + /** + * Set only when the file is readable and carries neither Hack marker, so it + * is a user-owned doc without a Hack-managed section. A doc with exactly one + * marker is a corrupted managed section and is never `unmanaged`. + */ readonly unmanaged?: true; }; @@ -108,46 +117,7 @@ export async function checkAgentDocs(opts: { for (const target of opts.targets) { const path = resolveAgentDocPath({ projectRoot: opts.projectRoot, target }); try { - const existing = await readTextFile(path); - if (!existing) { - results.push({ target, status: "missing", path }); - continue; - } - - if (!hasAgentDocSnippet({ content: existing })) { - results.push({ - target, - status: "error", - path, - message: "Missing hack agent-docs markers.", - unmanaged: true, - }); - continue; - } - - const region = extractSnippetRegion({ content: existing }); - if (region === null) { - results.push({ - target, - status: "error", - path, - message: "Malformed hack agent-docs markers.", - }); - continue; - } - - if (normalizeInstructionText({ text: region }) !== expected) { - results.push({ - target, - status: "stale", - path, - message: - "hack agent-docs content is out of date. Run: hack setup sync", - }); - continue; - } - - results.push({ target, status: "present", path }); + results.push(await checkAgentDocTarget({ target, path, expected })); } catch (error: unknown) { const message = error instanceof Error ? error.message : "Failed to read file"; @@ -158,6 +128,72 @@ export async function checkAgentDocs(opts: { return results; } +/** + * Classify one agent doc. Absent or empty docs are `missing`; an unreadable + * doc is `error`; a readable doc with neither Hack marker is a user-owned + * `unmanaged` error; a doc with only one marker is a corrupted (managed) + * section and is `error` without `unmanaged`. + */ +async function checkAgentDocTarget(opts: { + readonly target: AgentDocTarget; + readonly path: string; + readonly expected: string; +}): Promise { + const { target, path } = opts; + const file = await readOptionalTextFile(path); + if (file.kind === "unreadable") { + return { + target, + status: "error", + path, + message: "Agent doc exists but could not be read.", + }; + } + const existing = file.kind === "text" ? file.text : ""; + if (!existing) { + return { target, status: "missing", path }; + } + + const markers = inspectAgentDocMarkers({ content: existing }); + if (!(markers.hasStart || markers.hasEnd)) { + return { + target, + status: "error", + path, + message: "Missing hack agent-docs markers.", + unmanaged: true, + }; + } + if (!(markers.hasStart && markers.hasEnd)) { + return { + target, + status: "error", + path, + message: + "Malformed hack agent-docs markers: only one of the start/end markers is present.", + }; + } + + const region = extractSnippetRegion({ content: existing }); + if (region === null) { + return { + target, + status: "error", + path, + message: "Malformed hack agent-docs markers.", + }; + } + if (normalizeInstructionText({ text: region }) !== opts.expected) { + return { + target, + status: "stale", + path, + message: "hack agent-docs content is out of date. Run: hack setup sync", + }; + } + return { target, status: "present", path }; +} + /** * Remove the hack snippet from agent docs. */ @@ -290,11 +326,14 @@ function extractSnippetRegion(opts: { return match ? match[0] : null; } -function hasAgentDocSnippet(opts: { readonly content: string }): boolean { - return ( - opts.content.includes(DOC_MARKER_START) && - opts.content.includes(DOC_MARKER_END) - ); +function inspectAgentDocMarkers(opts: { readonly content: string }): { + readonly hasStart: boolean; + readonly hasEnd: boolean; +} { + return { + hasStart: opts.content.includes(DOC_MARKER_START), + hasEnd: opts.content.includes(DOC_MARKER_END), + }; } function escapeRegex(opts: { readonly value: string }): string { diff --git a/tests/doctor-command.test.ts b/tests/doctor-command.test.ts index 9f3f9d32..6d7be786 100644 --- a/tests/doctor-command.test.ts +++ b/tests/doctor-command.test.ts @@ -1,5 +1,12 @@ import { expect, test } from "bun:test"; -import { mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises"; +import { + chmod, + mkdir, + mkdtemp, + readdir, + rm, + writeFile, +} from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; import { HACK_AGENT_INTEGRATION_CONTENT_REVISION } from "../src/agents/integration-revision.ts"; @@ -369,6 +376,106 @@ test("doctor treats absent optional agent integrations as current", async () => } }); +/** Run a doctor agent-integration check against a fixture HOME and project. */ +async function withDoctorAgentFixture( + run: (paths: { + readonly home: string; + readonly project: string; + readonly inspect: () => Promise<{ readonly status: string }>; + }) => Promise +): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-doctor-agent-edge-")); + const home = join(root, "home"); + const project = join(root, "repo"); + const savedHome = process.env.HOME; + process.env.HOME = home; + try { + await mkdir(home, { recursive: true }); + await mkdir(project, { recursive: true }); + await run({ + home, + project, + inspect: () => + inspectDoctorAgentIntegrations({ projectRoot: project, homeDir: home }), + }); + } finally { + if (savedHome === undefined) { + // biome-ignore lint/performance/noDelete: restoring an unset env var + delete process.env.HOME; + } else { + process.env.HOME = savedHome; + } + await rm(root, { recursive: true, force: true }); + } +} + +test("doctor treats a doc with only one Hack marker as stale", async () => { + const marker = `Content revision: \`${HACK_AGENT_INTEGRATION_CONTENT_REVISION}\``; + await withDoctorAgentFixture(async ({ project, inspect }) => { + const doc = join(project, "AGENTS.md"); + + // Start marker only, even with the current revision text inside. + await writeFile( + doc, + `# notes\n\n${marker}\nno end marker\n` + ); + await expect(inspect()).resolves.toEqual({ status: "stale" }); + + // End marker only. + await writeFile( + doc, + `# notes\n${marker}\nno start marker\n\n` + ); + await expect(inspect()).resolves.toEqual({ status: "stale" }); + + // Neither marker: user-owned doc, not stale. + await writeFile(doc, "# notes\n"); + await expect(inspect()).resolves.toEqual({ status: "current" }); + }); +}); + +// Root can read mode-000 files, so the unreadable fixture cannot be built. +const doctorRunsAsRoot = process.getuid?.() === 0; + +test.skipIf(doctorRunsAsRoot)( + "doctor treats a present but unreadable Hack-owned file as stale", + async () => { + const marker = `Content revision: \`${HACK_AGENT_INTEGRATION_CONTENT_REVISION}\``; + await withDoctorAgentFixture(async ({ project, inspect }) => { + const rules = join(project, ".cursor", "rules", "hack.mdc"); + await mkdir(join(rules, ".."), { recursive: true }); + await writeFile(rules, `${marker}\n`); + await expect(inspect()).resolves.toEqual({ status: "current" }); + + await chmod(rules, 0o000); + try { + await expect(inspect()).resolves.toEqual({ status: "stale" }); + } finally { + await chmod(rules, 0o644); + } + await expect(inspect()).resolves.toEqual({ status: "current" }); + }); + } +); + +test.skipIf(doctorRunsAsRoot)( + "doctor treats a present but unreadable agent doc as stale", + async () => { + await withDoctorAgentFixture(async ({ project, inspect }) => { + const doc = join(project, "CLAUDE.md"); + await writeFile(doc, "# notes\n"); + await expect(inspect()).resolves.toEqual({ status: "current" }); + + await chmod(doc, 0o000); + try { + await expect(inspect()).resolves.toEqual({ status: "stale" }); + } finally { + await chmod(doc, 0o644); + } + }); + } +); + test("doctor guidance routes runtime hygiene drift to projects prune", () => { const guidance = buildDoctorRecoveryGuidance({ results: [ diff --git a/tests/integration-sync.test.ts b/tests/integration-sync.test.ts index 3559c550..89a4155f 100644 --- a/tests/integration-sync.test.ts +++ b/tests/integration-sync.test.ts @@ -1,8 +1,9 @@ import { afterEach, beforeEach, expect, test } from "bun:test"; -import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { installCursorRules } from "../src/agents/cursor.ts"; +import { installSharedHackSkill } from "../src/agents/shared-skill.ts"; import { inspectAgentIntegrationFreshness } from "../src/cli/integration-sync.ts"; /** @@ -121,3 +122,70 @@ test("a retired legacy artifact is stale", async () => { ); expect(await status(null)).toBe("stale"); }); + +test("a doc with only the Hack start marker is a corrupted section and stale", async () => { + const { repo } = fixture(); + await writeFile( + join(repo, "AGENTS.md"), + "# notes\n\nold content without end\n" + ); + expect(await status(repo)).toBe("stale"); +}); + +test("a doc with only the Hack end marker is a corrupted section and stale", async () => { + const { repo } = fixture(); + await writeFile( + join(repo, "CLAUDE.md"), + "# notes\nold content without start\n\n" + ); + expect(await status(repo)).toBe("stale"); +}); + +// Root can read mode-000 files, so the unreadable fixture cannot be built. +const runsAsRoot = process.getuid?.() === 0; + +test.skipIf(runsAsRoot)( + "an installed but unreadable Hack-owned project file counts as drift", + async () => { + const { repo } = fixture(); + await installCursorRules({ scope: "project", projectRoot: repo }); + const rules = join(repo, ".cursor", "rules", "hack.mdc"); + expect(await status(repo)).toBe("current"); + await chmod(rules, 0o000); + try { + expect(await status(repo)).toBe("stale"); + } finally { + await chmod(rules, 0o644); + } + expect(await status(repo)).toBe("current"); + } +); + +test.skipIf(runsAsRoot)( + "an installed but unreadable Hack-owned HOME file counts as drift", + async () => { + const { home } = fixture(); + await installSharedHackSkill(); + const skill = join(home, ".ai", "skills", "hack-cli", "SKILL.md"); + expect(await status(null)).toBe("current"); + await chmod(skill, 0o000); + try { + expect(await status(null)).toBe("stale"); + } finally { + await chmod(skill, 0o644); + } + } +); + +test.skipIf(runsAsRoot)("an unreadable agent doc counts as drift", async () => { + const { repo } = fixture(); + const doc = join(repo, "AGENTS.md"); + await writeFile(doc, "# Team notes\n"); + expect(await status(repo)).toBe("current"); + await chmod(doc, 0o000); + try { + expect(await status(repo)).toBe("stale"); + } finally { + await chmod(doc, 0o644); + } +}); From 9854d1b5be525c6cd4308575480d06de3bfdfd2d Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 23:02:09 -0400 Subject: [PATCH 10/52] test(env): pin the console logger backend for CLI refusal tests The env add --secret refusal tests asserted the refusal text on stderr, but the default logger backend depends on the host: gum (stderr) when installed, clack (stdout) otherwise. Hosted macOS runners have no gum, so the refusal reached stdout and stderr was empty only in CI. Pin HACK_LOGGER=console in the CLI helper, matching the other CLI-spawning tests, so the stderr assertions hold on every host. Co-Authored-By: Claude Opus 5.5 --- tests/project-env-worktree-key.test.ts | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/tests/project-env-worktree-key.test.ts b/tests/project-env-worktree-key.test.ts index 3bda56e8..00cf7fa3 100644 --- a/tests/project-env-worktree-key.test.ts +++ b/tests/project-env-worktree-key.test.ts @@ -255,7 +255,14 @@ type CliResult = { readonly stderr: string; }; -/** Runs the current-branch CLI with an isolated HOME and global config. */ +/** + * Runs the current-branch CLI with an isolated HOME and global config. + * + * The logger backend is pinned to `console` (plain stderr). Without the pin + * the backend depends on the host: `gum` writes to stderr when installed, but + * hosted runners lack gum and fall back to clack, which writes to stdout, so + * refusal text would leave stderr empty only in CI. + */ async function runHackCli(opts: { readonly sandbox: string; readonly args: readonly string[]; @@ -266,6 +273,7 @@ async function runHackCli(opts: { ...process.env, HOME: home, HACK_GLOBAL_CONFIG_PATH: resolve(home, "hack.config.json"), + HACK_LOGGER: "console", }; Reflect.deleteProperty(env, "HACK_ENV_SECRET_KEY"); const proc = Bun.spawn( From 6e1ff6398112de11507cdb626089a77afd20df5c Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 23:12:24 -0400 Subject: [PATCH 11/52] test(mcp): assert closed ambiguous_project refusal in project-name suite The project-name boundary suite still expected the old value-echoing "Ambiguous project name/ID" stderr from MCP. Assert the closed MCP_REFUSALS contract instead: exact code, reason and fixed message, spawned: false, no argv, and none of the duplicated names, registry ids or fixture root anywhere in the serialized result. Co-Authored-By: Claude Opus 5.5 --- tests/project-name-command.test.ts | 74 ++++++++++++++++++++++++------ 1 file changed, 60 insertions(+), 14 deletions(-) diff --git a/tests/project-name-command.test.ts b/tests/project-name-command.test.ts index ef699cd8..e03ea752 100644 --- a/tests/project-name-command.test.ts +++ b/tests/project-name-command.test.ts @@ -3,7 +3,11 @@ import { chmod, readFile, rm, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; -import { createMcpServer } from "../src/mcp/server.ts"; +import { + createMcpServer, + MCP_REFUSALS, + type McpRefusalReason, +} from "../src/mcp/server.ts"; import { projectNameFixture } from "./helpers/project-name-fixture.ts"; // The same boundary suite can qualify the current branch's compiled binary. @@ -127,6 +131,37 @@ for (const selector of selectors) { }, 15_000); } +/** + * Asserts the closed pre-spawn refusal contract: fixed catalogue code and + * message, no child spawned, and none of the given caller or registry values + * anywhere in the serialized result. + */ +function expectClosedRefusal(opts: { + readonly result: Awaited>; + readonly reason: McpRefusalReason; + readonly absent: readonly string[]; +}) { + const expected = MCP_REFUSALS[opts.reason]; + expect(opts.result.isError).toBe(true); + expect(opts.result.structuredContent).toEqual({ + ok: false, + command: "", + exitCode: 1, + stdout: "", + stderr: expected.message, + spawned: false, + error: { + code: expected.code, + reason: opts.reason, + message: expected.message, + }, + }); + const serialized = JSON.stringify(opts.result); + for (const value of opts.absent) { + expect(serialized).not.toContain(value); + } +} + async function connectMcp() { const command = join(fixture.root, "fixture-cli"); // Spawn the actual current-source CLI (or explicitly selected current build). @@ -165,19 +200,25 @@ test("MCP selectors survive real CLI forwarding and reject aliases with multiple const contender = await fixture.createProject("other", "my-app"); await fixture.writeRegistry([target.entry, contender.entry]); const before = await readFile(fixture.registryPath, "utf8"); + const registered = [ + "my_app", + "my-app", + target.entry.id, + contender.entry.id, + fixture.root, + ]; for (const projectName of ["my_app", "my-app", "", "!!!"]) { const result = await client.callTool({ name: "hack.project.open", arguments: { projectName }, }); - expect(result.isError).toBe(true); - expect(result.structuredContent).toMatchObject({ - ok: false, - stderr: expect.stringContaining( + expectClosedRefusal({ + result, + reason: projectName === "" || projectName === "!!!" - ? "Invalid projectName" - : "Ambiguous project name" - ), + ? "invalid_project_name" + : "ambiguous_project", + absent: projectName === "" ? registered : [...registered, projectName], }); } expect(await readFile(fixture.registryPath, "utf8")).toBe(before); @@ -195,12 +236,17 @@ for (const reversed of [false, true]) { name: "hack.project.open", arguments: { projectName }, }); - expect(result.isError).toBe(true); - expect(result.structuredContent).toMatchObject({ - ok: false, - exitCode: 1, - stdout: "", - stderr: expect.stringContaining("Ambiguous project ID"), + expectClosedRefusal({ + result, + reason: "ambiguous_project", + absent: [ + projectName, + "my_app", + "my-app", + contender.entry.name, + target.entry.id, + fixture.root, + ], }); } expect(await readFile(fixture.registryPath, "utf8")).toBe(before); From 1f8b51e0c29e2213d609e8689cc923c512cdbeef Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 21:12:53 -0400 Subject: [PATCH 12/52] chore(x01): temporary witness cost trace for hosted measurement Temporary measurement only; reverted before the final commit. Writes closed labels and durations per native command to a per-scenario JSONL file and prints a summary in the docker-e2e job. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 19 ++ scripts/x01-trace-summary.ts | 268 ++++++++++++++++++ src/lib/native-compose-command-storage.ts | 12 +- src/lib/native-compose-command.ts | 8 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- .../native-compose-storage-witness-docker.ts | 43 ++- src/lib/native-compose-storage-witness.ts | 34 ++- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 171 +++++++++++ tests/e2e/harness.ts | 9 + .../e2e/native-config-networks-acceptance.ts | 4 + 12 files changed, 570 insertions(+), 28 deletions(-) create mode 100644 scripts/x01-trace-summary.ts create mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 457db89f..34c2387c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -306,7 +306,10 @@ jobs: HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 + # TEMPORARY X01 measurement; reverted before the final commit. + HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | + mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -315,6 +318,22 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" + # TEMPORARY X01 measurement; reverted before the final commit. + - name: X01 witness cost summary + if: always() + run: | + for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do + [ -f "$f" ] || continue + bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true + done + - name: X01 witness cost trace artifact + if: always() + uses: actions/upload-artifact@v4 + with: + name: x01-trace-shard-${{ matrix.shard }} + path: ${{ runner.temp }}/x01-trace + if-no-files-found: ignore + retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts new file mode 100644 index 00000000..ba276c07 --- /dev/null +++ b/scripts/x01-trace-summary.ts @@ -0,0 +1,268 @@ +// TEMPORARY X01 measurement summary. Reverted before the final commit. +// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. +type Event = { + readonly p: number; + readonly v: string; + readonly k: string; + readonly s?: string; + readonly l?: string; + readonly ms?: number; + readonly t?: number; + readonly c?: number; + readonly line?: { + readonly phase?: string; + readonly boundary?: string; + readonly durationMs?: number | null; + }; +}; +const WITNESS = new Set([ + "storage-prepare", + "storage-verify", + "carrier-new", + "inspect", + "provision", + "finish", + "witness-verify", + "witness-prepare", + "witness-enroll", +]); +const isWitness = (label: string) => + WITNESS.has(label) || label.startsWith("invoke:"); +const LEAF = new Set(["probe", "run", "exec", "hash"]); +const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; + +const path = process.argv[2]; +const title = process.argv[3] ?? path; +if (!path) { + throw new Error("usage: x01-trace-summary [title]"); +} +const text = await Bun.file(path).text(); +const events: Event[] = []; +for (const line of text.split("\n")) { + if (line.trim() === "") { + continue; + } + try { + events.push(JSON.parse(line) as Event); + } catch { + // skip partial line + } +} +const byProcess = new Map(); +for (const event of events) { + const list = byProcess.get(event.p) ?? []; + list.push(event); + byProcess.set(event.p, list); +} +type Proc = { + readonly pid: number; + readonly verb: string; + readonly totalMs: number; + readonly witnessMs: number; + readonly events: Event[]; + readonly kind: string; +}; +function topWitnessMs(list: Event[]): number { + let total = 0; + for (const event of list) { + if (event.k !== "scope" || !event.l || !isWitness(event.l)) { + continue; + } + const parts = (event.s ?? "").split(">"); + parts.pop(); + if (parts.some(isWitness)) { + continue; + } + total += event.ms ?? 0; + } + return total; +} +const procs: Proc[] = []; +for (const [pid, list] of byProcess) { + const exit = list.find((event) => event.k === "exit"); + const verb = list[0]?.v ?? "other"; + const labels = new Set(list.map((event) => event.l ?? "")); + let kind = verb; + if (verb === "up" || verb === "restart" || verb === "run") { + if (labels.has("invoke:seed")) { + kind = `${verb}(cold)`; + } else if (labels.has("invoke:verify")) { + kind = `${verb}(warm)`; + } else { + kind = `${verb}(no-witness)`; + } + } else if (verb === "exec") { + kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; + } + procs.push({ + pid, + verb, + totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), + witnessMs: topWitnessMs(list), + events: list, + kind, + }); +} +const fmt = (ms: number) => (ms / 1000).toFixed(1); +const out: string[] = []; +out.push(`#### X01 trace: ${title}`); +out.push(`events=${events.length} processes=${procs.length}`); +out.push(""); +out.push( + "kind | n | avg total s | sum total s | avg witness s | sum witness s" +); +const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); +for (const kind of kinds) { + const selected = procs.filter((proc) => proc.kind === kind); + const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); + const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); + out.push( + `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` + ); +} +const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); +const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); +out.push( + `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` +); +out.push(""); + +function leafTable(selected: Proc[], heading: string, limit = 24) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const key = `${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} (per process avg over n=${selected.length})`); + out.push("leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function leafByScope(selected: Proc[], heading: string, limit = 40) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const parts = (event.s ?? "") + .split(">") + .filter( + (part) => + isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) + ); + const inner = parts.slice(-2).join(">") || "(none)"; + const key = `${inner} :: ${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push( + `${heading} leaves by scope (per process avg over n=${selected.length})` + ); + out.push("scope :: leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function scopeTable(selected: Proc[], heading: string, limit = 30) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (event.k !== "scope" || !event.l) { + continue; + } + const parts = (event.s ?? "").split(">"); + const key = parts + .filter( + (part) => + PHASE.test(part) || + isWitness(part) || + part.startsWith("xattr:") || + part.startsWith("cmd:") || + part.startsWith("exec-") + ) + .join(">"); + if (key === "") { + continue; + } + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} scopes (per process avg over n=${selected.length})`); + out.push("scope path | n/proc | s/proc | ms/each"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function phaseRows(proc: Proc) { + const rows = new Map(); + for (const event of proc.events) { + if ( + event.k === "phase" && + event.line?.boundary !== "begin" && + event.line?.phase + ) { + const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.line.durationMs ?? 0; + rows.set(event.line.phase, row); + } + } + return rows; +} +for (const kind of [ + "up(cold)", + "up(warm)", + "exec(storage)", + "restart(warm)", + "run(warm)", + "run(cold)", + "down", +]) { + const selected = procs.filter((proc) => proc.kind === kind); + if (selected.length === 0) { + continue; + } + const sample = selected[0] as Proc; + out.push( + `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` + ); + out.push("phase-trace phase | spans | s"); + for (const [phase, row] of phaseRows(sample)) { + out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); + } + out.push(""); + scopeTable([sample], `${kind} first sample`, 40); + leafTable([sample], `${kind} first sample leaves`); + scopeTable(selected, `${kind} all`, 30); + leafTable(selected, `${kind} all leaves`); + leafByScope(selected, `${kind} all`); +} +console.log(out.join("\n")); diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index 58bc48bf..ce45846c 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,6 +23,7 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -129,7 +130,12 @@ async function volumeNames(signal: AbortSignal): Promise { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export async function prepareNativeComposeCommandStorage(opts: { +export function prepareNativeComposeCommandStorage( + opts: Parameters[0] +): ReturnType { + return x01Scope("storage-prepare", () => x01Prepare(opts)); +} +async function x01Prepare(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -220,7 +226,9 @@ export async function prepareNativeComposeCommandStorage(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = async (generation: NativeComposeGeneration) => { + const verify = (generation: NativeComposeGeneration) => + x01Scope("storage-verify", () => x01Verify(generation)); + const x01Verify = async (generation: NativeComposeGeneration) => { await assertFresh?.(); const before = await store.loadCurrent(); if (before.storageWitnessesPending) { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index e6ee1586..9701043c 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -137,6 +137,7 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -942,7 +943,8 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = async () => { + const assertSaved = () => x01Scope("exec-assert-saved", x01AssertSaved); + const x01AssertSaved = async () => { const latest = await store.loadCurrent(); assertStartupAvailable(latest); if ( @@ -959,7 +961,9 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - runSavedProcess({ options, generation, document, base, signal }), + x01Scope("exec-process", () => + runSavedProcess({ options, generation, document, base, signal }) + ), }); }); } diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index 0019b3fc..d8bc9dc8 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,6 +3,7 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1232,7 +1233,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - return async (args) => { + const x01Inner = async (args: readonly string[]): Promise => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1310,6 +1311,8 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; + return (args) => + x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 540f833b..943b7932 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,5 +1,6 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; +import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -135,6 +136,8 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); + } else if (x01Enabled()) { + trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -147,5 +150,8 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return context.getStore()?.measure(phase, operation) ?? operation(); + return x01Scope( + phase, + () => context.getStore()?.measure(phase, operation) ?? operation() + ); } diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index 0c04e7ba..da785e7b 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -96,6 +96,7 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; +import { x01Label, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -813,7 +814,12 @@ async function groupAbsent( await new Promise((resolve) => setTimeout(resolve, 20)); } } -async function commandExecutable( +function commandExecutable( + path: string +): Promise { + return x01Time("hash", x01Label([path]), () => x01CommandExecutable(path)); +} +async function x01CommandExecutable( path: string ): Promise { const canonical = await realpath(path); @@ -960,7 +966,14 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export async function runNativeComposeStorageDockerCommand(supplied: { +export function runNativeComposeStorageDockerCommand( + supplied: Parameters[0] +): ReturnType { + return x01Scope(`cmd:${x01Label(supplied.args)}`, () => + x01RunCommandInner(supplied) + ); +} +async function x01RunCommandInner(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1305,7 +1318,12 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export async function createNativeComposeDockerStorageXattrCarrier( +export function createNativeComposeDockerStorageXattrCarrier( + opts: Context +): Promise { + return x01Scope("carrier-new", () => x01CreateCarrier(opts)); +} +async function x01CreateCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -2024,16 +2042,19 @@ export async function createNativeComposeDockerStorageXattrCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect, - provision, - invoke, + inspect: (selection) => x01Scope("inspect", () => inspect(selection)), + provision: (selection) => + x01Scope("provision", () => provision(selection)), + invoke: (input) => + x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), finish: context.originalCommandRecords === true - ? async ({ input, completion }) => { - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - } + ? ({ input, completion }) => + x01Scope("finish", async () => { + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + }) : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index b8a242f3..0650f4b4 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,6 +78,7 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -493,7 +494,16 @@ function requireXattrAdmission( refuse(); } } -async function invokeXattr(opts: { +function invokeXattr( + opts: Parameters[0] +): ReturnType { + const operation = + isRecord(opts.request) && typeof opts.request.operation === "string" + ? opts.request.operation + : "unknown"; + return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); +} +async function x01InvokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1387,17 +1397,21 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }); + return await x01Scope("witness-prepare", () => + prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }) + ); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }); + return await x01Scope("witness-enroll", () => + enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }) + ); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1408,7 +1422,9 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); + await x01Scope("witness-verify", () => + verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) + ); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ef9faf1b..ff27e6b8 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,6 +4,7 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -40,7 +41,13 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export async function exec( +export function exec( + cmd: readonly string[], + opts: ExecOptions = {} +): Promise { + return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); +} +async function x01ExecInner( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -123,7 +130,13 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export async function run( +export function run( + cmd: readonly string[], + opts: RunOptions = {} +): Promise { + return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); +} +async function x01RunInner( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts new file mode 100644 index 00000000..48848c68 --- /dev/null +++ b/src/lib/x01-cost-probe.ts @@ -0,0 +1,171 @@ +// TEMPORARY X01 measurement probe. Reverted before the final commit. +// Writes closed labels and durations only (no ids, paths or values). +import { AsyncLocalStorage } from "node:async_hooks"; +import { appendFileSync } from "node:fs"; + +const FILE = process.env.HACK_X01_TRACE_FILE; +const VERBS = new Set([ + "up", + "down", + "exec", + "run", + "restart", + "ps", + "logs", + "status", + "init", +]); +const SUBS = new Set([ + "info", + "image", + "container", + "volume", + "network", + "compose", + "create", + "start", + "rm", + "inspect", + "ls", + "ps", + "exec", + "run", + "logs", + "version", + "events", + "stop", + "kill", + "wait", + "up", + "down", + "config", + "pull", + "build", + "restart", + "port", + "cp", + "top", +]); +const verb = (() => { + for (const arg of process.argv.slice(2)) { + if (VERBS.has(arg)) { + return arg; + } + } + return "other"; +})(); +const scopes = new AsyncLocalStorage(); +let enabled = typeof FILE === "string" && FILE.startsWith("/"); +let wroteStart = false; + +function append(record: Record): void { + appendFileSync( + FILE as string, + `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` + ); +} + +function write(record: Record): void { + if (!enabled) { + return; + } + try { + if (!wroteStart) { + wroteStart = true; + append({ k: "start", t: Math.round(performance.now()) }); + process.on("exit", (code) => { + try { + append({ k: "exit", c: code, t: Math.round(performance.now()) }); + } catch { + // measurement only + } + }); + } + append({ + s: scopes.getStore() ?? "", + t: Math.round(performance.now()), + ...record, + }); + } catch { + enabled = false; + } +} + +export function x01Enabled(): boolean { + return enabled; +} + +/** Raw line from the phase trace, re-wrapped with process identity. */ +export function x01Phase(line: string): void { + try { + write({ k: "phase", line: JSON.parse(line) as unknown }); + } catch { + enabled = false; + } +} + +export function x01Label(argv: readonly string[], binary = true): string { + const parts: string[] = []; + let rest = argv; + if (binary) { + const name = (argv[0] ?? "").split("/").at(-1) ?? ""; + parts.push( + ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) + ? name + : "bin" + ); + rest = argv.slice(1); + } else { + parts.push("docker"); + } + for (const token of rest) { + if (parts.length >= 3) { + break; + } + if (token.startsWith("-")) { + continue; + } + if (SUBS.has(token)) { + parts.push(token); + } else { + break; + } + } + return parts.join(" "); +} + +export async function x01Time( + kind: string, + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const started = performance.now(); + let ok = false; + try { + const result = await operation(); + ok = true; + return result; + } finally { + write({ + k: kind, + l: label, + ms: Math.round((performance.now() - started) * 10) / 10, + ok, + }); + } +} + +export async function x01Scope( + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const parent = scopes.getStore(); + const path = parent ? `${parent}>${label}` : label; + return await scopes.run(path, () => x01Time("scope", label, operation)); +} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index 2201457b..e7599c52 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,6 +43,8 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -768,6 +770,13 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); + // TEMPORARY X01 measurement: one trace file per scenario. + if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { + process.env.HACK_X01_TRACE_FILE = join( + process.env.HACK_X01_TRACE_DIR, + `${scenario.name}.jsonl` + ); + } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 2728a605..a2157596 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1153,6 +1153,10 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + ...(process.env.HACK_X01_TRACE_FILE + ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } + : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From 0911e6de6f7c57a31c01296c40f20e238ac60b7d Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 23:36:51 -0400 Subject: [PATCH 13/52] perf(storage): coalesce witness helper work per native command A storage-bearing saved exec ran three full witness proofs (nine helper carriers) and a warm up/restart seven. Within one command and one mutation lease, an adjacent repeat with only read-only checks between it and the previous proof now reuses that completed kernel proof instead of three new helper carriers: - saved exec: the pre-spawn check reuses the preparation proof; the proof after the process stays fresh (3 -> 2 proofs); - up/restart without build, routing or file material: the first pre-spawn guard reuses the enrollment proof (warm 7 -> 6, cold seed path 6 -> 5). Reuse requires the same lease authority, engine, generation and reference and a byte-identical opening material binding to the one that opened and closed the recorded proof. Every host check (anchors, expectation, completion, binding, carrier artifact and lifetime) still runs; any verification consumes the record first, so a failure or binding change forces a fresh proof. Ordinary verification always proves. Co-Authored-By: Claude Opus 5.5 --- src/lib/native-compose-command-storage.ts | 35 ++++-- src/lib/native-compose-command.ts | 28 ++++- src/lib/native-compose-storage-witness.ts | 112 ++++++++++++++---- tests/helpers/native-compose-command.ts | 1 + ...compose-storage-activation-command.test.ts | 58 +++++++++ ...ompose-storage-witness-xattr-owner.test.ts | 108 +++++++++++++++++ 6 files changed, 308 insertions(+), 34 deletions(-) diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index ce45846c..d9b58c13 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -128,6 +128,11 @@ async function volumeNames(signal: AbortSignal): Promise { return names; } +export type NativeComposeCommandStorageVerifyOptions = { + /** Only when nothing but read-only checks ran since this lease's previous proof. */ + readonly adjacentProof?: true; +}; + /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ export function prepareNativeComposeCommandStorage( @@ -226,9 +231,14 @@ async function x01Prepare(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = (generation: NativeComposeGeneration) => - x01Scope("storage-verify", () => x01Verify(generation)); - const x01Verify = async (generation: NativeComposeGeneration) => { + const verify = ( + generation: NativeComposeGeneration, + options: NativeComposeCommandStorageVerifyOptions = {} + ) => x01Scope("storage-verify", () => x01Verify(generation, options)); + const x01Verify = async ( + generation: NativeComposeGeneration, + options: NativeComposeCommandStorageVerifyOptions = {} + ) => { await assertFresh?.(); const before = await store.loadCurrent(); if (before.storageWitnessesPending) { @@ -248,6 +258,9 @@ async function x01Prepare(opts: { engineId, reference: entry.reference, carrier: await carrier(), + ...(options.adjacentProof === true + ? { adjacentProof: true as const } + : {}), }); } const after = await store.loadCurrent(); @@ -328,18 +341,24 @@ export type NativeComposeCommandStorage = Awaited< ReturnType >; -/** Saved exec does not publish a ready receipt. Cancellation preserves its known - * exit and starts no late helper; every future admission re-verifies content. */ +/** Saved exec does not publish a ready receipt. The opening check directly + * follows command storage preparation under the same lease, with only read-only + * source and ownership checks between, so it may reuse that adjacent proof. The + * closing check after the process always performs a fresh proof. Cancellation + * preserves its known exit and starts no late helper; every future admission + * re-verifies content. */ export async function runNativeComposeStorageVerifiedExec(opts: { readonly signal: AbortSignal; - readonly assertSaved: () => Promise; + readonly assertSaved: ( + options: NativeComposeCommandStorageVerifyOptions + ) => Promise; readonly run: () => Promise; }): Promise { const { signal, assertSaved, run } = opts; - await assertSaved(); + await assertSaved({ adjacentProof: true }); const code = await run(); if (!signal.aborted) { - await assertSaved(); + await assertSaved({}); } return code; } diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 9701043c..0aa92b81 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -26,6 +26,7 @@ import { } from "./native-compose-build.ts"; import { type NativeComposeCommandStorage, + type NativeComposeCommandStorageVerifyOptions, nativeComposePlanStorage, nativeComposeSavedExecUsesReadLease, prepareNativeComposeCommandStorage, @@ -943,8 +944,11 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = () => x01Scope("exec-assert-saved", x01AssertSaved); - const x01AssertSaved = async () => { + const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => + x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); + const x01AssertSaved = async ( + proof: NativeComposeCommandStorageVerifyOptions + ) => { const latest = await store.loadCurrent(); assertStartupAvailable(latest); if ( @@ -955,7 +959,7 @@ async function savedCommand(opts: { } await store.readGenerationDocument(generation); await assertNativeComposeOwned(selection); - await storage?.verify(generation); + await storage?.verify(generation, proof); }; return await runNativeComposeStorageVerifiedExec({ signal, @@ -1665,6 +1669,15 @@ async function executePreparedGeneration(opts: { }); }, }); + // Up/restart enrollment ends with a storage proof. With no build, routing or + // file material, the first pre-spawn guard follows it with only read-only + // source and ownership checks (arming is a no-op), so that guard may reuse the + // adjacent proof. Every later guard and finalization proof stays fresh. + let adjacentGuardProof = + (operation === "up" || operation === "restart") && + builds.length === 0 && + routing === null && + files === null; const result = await mutation.runEffect({ generation, operation, @@ -1699,7 +1712,14 @@ async function executePreparedGeneration(opts: { assertFresh, assertOwned: () => ownership.assertOwned(), verifyStorage: storage - ? () => storage.verify(generation) + ? () => { + const adjacentProof = adjacentGuardProof; + adjacentGuardProof = false; + return storage.verify( + generation, + adjacentProof ? { adjacentProof } : {} + ); + } : undefined, }), }); diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 0650f4b4..bfcbfe4e 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -1245,7 +1245,62 @@ export async function enrollNativeComposeStorageWitness(opts: { }).catch((error) => normalizedRefusal(error, stage)); } -/** Read-only verification. No missing journal/marker, including interrupted enrollment, is seeded or repaired. */ +/** + * Completed directory-xattr kernel proofs of one mutation lease. A proof is keyed + * by its exact engine, generation and reference and holds the material binding + * that opened and closed it unchanged. The lease's authority object is the + * WeakMap key, so a later command, lease or revoked authority never sees it. + */ +const leaseProofs = new WeakMap< + NativeComposeMaterialAuthority, + Map +>(); +/** Take (and clear) this verification's lease record before any check, so a + * failed or refused verification never leaves an older proof reusable. */ +function takeLeaseProof(opts: { + readonly authority: NativeComposeMaterialAuthority; + readonly engineId: string; + readonly generation: NativeComposeGeneration; + readonly reference: NativeComposeStorageWitnessReference; + readonly adjacent: boolean; +}) { + const proofs = leaseProofs.get(opts.authority) ?? new Map(); + leaseProofs.set(opts.authority, proofs); + const key = JSON.stringify([ + opts.engineId, + opts.generation.generationId, + opts.generation.inputRevision, + opts.reference, + ]); + const recorded = proofs.get(key); + proofs.delete(key); + return { + /** Run the kernel proof unless an adjacent caller opens with the recorded binding. */ + async prove(opening: string, run: () => Promise): Promise { + if (!(opts.adjacent && recorded === opening)) { + await run(); + } + }, + /** Record a completed proof only when its binding never moved. */ + record(opening: string, closing: string): void { + if (opening === closing) { + proofs.set(key, opening); + } + }, + }; +} + +/** + * Read-only verification. No missing journal/marker, including interrupted enrollment, is seeded or repaired. + * + * `adjacentProof` is only for a caller whose code since its previous storage + * proof under this same lease performed no hook, workload, engine or storage + * effect. Such a repeat runs every host check (anchors, expectation, completion, + * binding) but reuses the completed kernel proof instead of three new helper + * carriers, and only when its opening material binding is byte-identical to the + * binding that opened and closed that proof. Any other verification, and any + * mismatch, performs the full proof and replaces the lease record. + */ export async function verifyNativeComposeStorageWitness(opts: { readonly authority: NativeComposeMaterialAuthority; readonly generation: NativeComposeGeneration; @@ -1256,6 +1311,7 @@ export async function verifyNativeComposeStorageWitness(opts: { readonly archive: Uint8Array; }>; readonly xattrCarrier?: NativeComposeStorageXattrCarrier; + readonly adjacentProof?: true; }): Promise { const { authority, generation, engineId, observe, xattrCarrier } = opts; if (!nativeComposeStorageWitnessReferenceValid(opts.reference)) { @@ -1284,11 +1340,19 @@ export async function verifyNativeComposeStorageWitness(opts: { return await runNativeComposeMaterialAction({ authority, run: async () => { + const lease = takeLeaseProof({ + authority, + engineId, + generation, + reference, + adjacent: opts.adjacentProof === true, + }); const current = await assertNativeComposeMaterialAuthority({ authority, generation, phase: "inspect", }); + const opening = JSON.stringify(current); const held = await directories({ binding: current, name: reference.volume.name, @@ -1334,27 +1398,29 @@ export async function verifyNativeComposeStorageWitness(opts: { if (!xattrCarrier) { return refuse(); } - await observeXattr({ - carrier: xattrCarrier, - saved, - volume: reference.volume, - stopped: false, - journal: last(held), - check: async () => { - await checkedExpectation(held, reference.expectation); - await readCompletion(); - await recheckDirectories(held); - const latest = await assertNativeComposeMaterialAuthority({ - authority, - generation, - phase: "inspect", - }); - if (!matchesBinding(saved.binding, latest, engineId)) { - return refuse(); - } - return latest; - }, - }); + await lease.prove(opening, () => + observeXattr({ + carrier: xattrCarrier, + saved, + volume: reference.volume, + stopped: false, + journal: last(held), + check: async () => { + await checkedExpectation(held, reference.expectation); + await readCompletion(); + await recheckDirectories(held); + const latest = await assertNativeComposeMaterialAuthority({ + authority, + generation, + phase: "inspect", + }); + if (!matchesBinding(saved.binding, latest, engineId)) { + return refuse(); + } + return latest; + }, + }) + ); } await checkedExpectation(held, reference.expectation); await readCompletion(); @@ -1369,6 +1435,8 @@ export async function verifyNativeComposeStorageWitness(opts: { } if (xattrCarrier) { checkNativeComposeStorageXattrCarrierLifetime(xattrCarrier); + // A v3 reference with a carrier always reached the kernel proof above. + lease.record(opening, JSON.stringify(latest)); } } finally { await close(held); diff --git a/tests/helpers/native-compose-command.ts b/tests/helpers/native-compose-command.ts index 37e01b08..460e9714 100644 --- a/tests/helpers/native-compose-command.ts +++ b/tests/helpers/native-compose-command.ts @@ -177,6 +177,7 @@ if(args[0]==="compose") { } if(oneoff && args.includes("run")) {const doc=await Bun.file(args[args.indexOf("-f")+1]).json();await Bun.write(engine,JSON.stringify(doc));await Bun.write(root+"/oneoff-name",args[args.indexOf("--name")+1]);process.exit(0);} if(args.includes("down")) {await rm(engine,{force:true});await appendFile(root+"/order","engine-stopped\\n");process.exit(0);} + if(args.includes("exec") && await Bun.file(engine).exists()) {${opts.storage ? 'await appendFile(root+"/storage-events","exec\\n");' : ""}process.exit(0);} process.exit(99); } if(oneoff && args[0]==="container" && args[1]==="rm") {if(args.length!==3||args[2]!=="c".repeat(64))process.exit(96);await Bun.write(root+"/oneoff-removed","removed");process.exit(0);} diff --git a/tests/native-compose-storage-activation-command.test.ts b/tests/native-compose-storage-activation-command.test.ts index 6ad088fd..93267d2d 100644 --- a/tests/native-compose-storage-activation-command.test.ts +++ b/tests/native-compose-storage-activation-command.test.ts @@ -378,3 +378,61 @@ test("public down recover completes only removed readonly original work and next } }); }, 500_000); + +// One proof is root, verify and root-after through three fresh helper carriers. +// Warm up/restart keep the opening, both effect-phase, enrollment and both +// finalization proofs; only the first pre-spawn guard reuses the adjacent +// enrollment proof (no build, routing or file material). Saved exec keeps its +// opening proof, reuses it for the adjacent pre-spawn check and proves again +// after the process. A cold up seeds once and keeps every later proof. +test( + "source CLI helper invocations per command keep every non-adjacent proof", + async () => + await ownedColdCase(async (owner) => { + const root = await fixture("", false, { + storage, + noHooks: true, + cleanupAllowed: owner.cleanupAllowed, + }); + const eventsFile = join(root, "storage-events"); + const events = async () => + (await Bun.file(eventsFile).exists()) + ? (await Bun.file(eventsFile).text()).trim().split("\n") + : []; + const counted = async (args: readonly string[]) => { + const before = (await events()).length; + const result = await owner.invoke(root, args); + expect(result.code, await disposition(root, result.code)).toBe(0); + return (await events()).slice(before); + }; + const proofs = (count: number) => + Array.from({ length: count }, () => ["root", "verify", "root"]).flat(); + expect(await counted(["up", "--detach", "--json"])).toEqual([ + "root", + "seed", + ...proofs(3), + "workload", + ...proofs(2), + ]); + expect(await counted(["up", "--detach", "--json"])).toEqual([ + ...proofs(4), + "workload", + ...proofs(2), + ]); + expect(await counted(["exec", "web", "--", "true"])).toEqual([ + ...proofs(1), + "exec", + ...proofs(1), + ]); + expect(await counted(["restart", "--json"])).toEqual([ + ...proofs(4), + "workload", + ...proofs(2), + ]); + expect( + (await operations(root)).filter((op) => op === "seed") + ).toHaveLength(1); + expect(await Bun.file(join(root, "carriers")).json()).toEqual([]); + }), + 390_000 +); diff --git a/tests/native-compose-storage-witness-xattr-owner.test.ts b/tests/native-compose-storage-witness-xattr-owner.test.ts index 7692b840..2cb45c0b 100644 --- a/tests/native-compose-storage-witness-xattr-owner.test.ts +++ b/tests/native-compose-storage-witness-xattr-owner.test.ts @@ -2869,3 +2869,111 @@ test("ordinary inspection authority cannot enter saved carrier recovery observat }); expect(observed).toBe(0); }, 30_000); + +function adjacentVerifier(opts: { + readonly transport: ReturnType; + readonly generation: NativeComposeGeneration; + readonly reference: Parameters< + typeof verifyNativeComposeStorageXattrWitness + >[0]["reference"]; +}) { + return (mutation: NativeComposeMutation, adjacent: boolean) => + verifyNativeComposeStorageXattrWitness({ + authority: mutation.materialAuthority, + generation: opts.generation, + engineId, + reference: opts.reference, + carrier: opts.transport.carrier, + ...(adjacent ? { adjacentProof: true as const } : {}), + }); +} + +test("adjacent storage proof is reused only within its own lease", async () => { + const { store, transport, generation, reference } = await active(); + const verify = adjacentVerifier({ transport, generation, reference }); + transport.calls.length = 0; + await store.withMutation(async (mutation) => { + // Nothing to reuse yet: the first adjacent request performs the proof. + await verify(mutation, true); + expect(helperOperations(transport)).toEqual(["root", "verify", "root"]); + await verify(mutation, true); + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(3); + // Ordinary verification always performs a fresh proof. + await verify(mutation, false); + expect(helperOperations(transport)).toHaveLength(6); + }); + await store.withMutation(async (mutation) => { + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(9); + }); + expect(transport.calls).not.toContain("seed"); + expect(transport.calls).not.toContain("provision"); +}, 30_000); + +test("adjacent storage proof still runs every host check and never replaces the next fresh proof", async () => { + const { store, transport, generation, reference } = await active(); + const verify = adjacentVerifier({ transport, generation, reference }); + const slot = join( + store.identity.checkoutRoot, + ".hack", + ".internal", + "native-compose", + store.identity.instanceId, + "storage-witnesses", + createHash("sha256").update(volume.name).digest("hex") + ); + transport.calls.length = 0; + await store.withMutation(async (mutation) => { + await verify(mutation, false); + // Host evidence is rechecked on reuse; a missing completion record refuses. + await rename(join(slot, "enrolled.json"), join(slot, "moved.json")); + await expect(verify(mutation, true)).rejects.toThrow("values omitted"); + await rename(join(slot, "moved.json"), join(slot, "enrolled.json")); + // The refusal cleared the lease record, so the repeat proves again. + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(6); + // A reused kernel proof stands only for its adjacent caller. + transport.attributes.clear(); + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(6); + await expect(verify(mutation, false)).rejects.toThrow("values omitted"); + await expect(verify(mutation, true)).rejects.toThrow("values omitted"); + }); + expect(transport.calls).not.toContain("seed"); +}, 30_000); + +test("adjacent storage proof is not reused across a changed material binding", async () => { + const { store, transport, generation, reference } = await active(); + const verify = adjacentVerifier({ transport, generation, reference }); + transport.calls.length = 0; + await store.withMutation(async (mutation) => { + await verify(mutation, false); + expect(helperOperations(transport)).toHaveLength(3); + // Intent publication changes the receipt before the effect body runs. + const result = await mutation.runEffect({ + generation, + operation: "up", + assertFresh: async () => {}, + assertOwned: async () => {}, + captureStorage: () => [volume], + storageWitnesses: { + kind: "directory-xattr", + engineId, + carrier: transport.carrier, + }, + effect: async () => { + const entry = helperOperations(transport).length; + // The effect-entry proof under the published intent is adjacent. + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(entry); + return { value: 0, outcome: "complete" }; + }, + }); + expect(result).toEqual({ value: 0, outcome: "complete" }); + // After completion the receipt moved again; nothing earlier is reusable. + const finished = helperOperations(transport).length; + await verify(mutation, true); + expect(helperOperations(transport)).toHaveLength(finished + 3); + }); +}, 30_000); From eb9a678597fe7ad46fb4ca938b70bea2b819e2c6 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Sun, 11 Oct 2026 23:42:17 -0400 Subject: [PATCH 14/52] chore(x01): revert temporary witness cost trace Reverts the hosted measurement commit (closed-label cost trace, phase-trace routing, per-scenario summary step and artifact) now that the per-phase and per-call attribution is recorded in the pull request. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 19 -- scripts/x01-trace-summary.ts | 268 ------------------ src/lib/native-compose-command-storage.ts | 14 +- src/lib/native-compose-command.ts | 9 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- .../native-compose-storage-witness-docker.ts | 43 +-- src/lib/native-compose-storage-witness.ts | 34 +-- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 171 ----------- tests/e2e/harness.ts | 9 - .../e2e/native-config-networks-acceptance.ts | 4 - 12 files changed, 28 insertions(+), 573 deletions(-) delete mode 100644 scripts/x01-trace-summary.ts delete mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 34c2387c..457db89f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -306,10 +306,7 @@ jobs: HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 - # TEMPORARY X01 measurement; reverted before the final commit. - HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | - mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -318,22 +315,6 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" - # TEMPORARY X01 measurement; reverted before the final commit. - - name: X01 witness cost summary - if: always() - run: | - for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do - [ -f "$f" ] || continue - bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true - done - - name: X01 witness cost trace artifact - if: always() - uses: actions/upload-artifact@v4 - with: - name: x01-trace-shard-${{ matrix.shard }} - path: ${{ runner.temp }}/x01-trace - if-no-files-found: ignore - retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts deleted file mode 100644 index ba276c07..00000000 --- a/scripts/x01-trace-summary.ts +++ /dev/null @@ -1,268 +0,0 @@ -// TEMPORARY X01 measurement summary. Reverted before the final commit. -// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. -type Event = { - readonly p: number; - readonly v: string; - readonly k: string; - readonly s?: string; - readonly l?: string; - readonly ms?: number; - readonly t?: number; - readonly c?: number; - readonly line?: { - readonly phase?: string; - readonly boundary?: string; - readonly durationMs?: number | null; - }; -}; -const WITNESS = new Set([ - "storage-prepare", - "storage-verify", - "carrier-new", - "inspect", - "provision", - "finish", - "witness-verify", - "witness-prepare", - "witness-enroll", -]); -const isWitness = (label: string) => - WITNESS.has(label) || label.startsWith("invoke:"); -const LEAF = new Set(["probe", "run", "exec", "hash"]); -const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; - -const path = process.argv[2]; -const title = process.argv[3] ?? path; -if (!path) { - throw new Error("usage: x01-trace-summary [title]"); -} -const text = await Bun.file(path).text(); -const events: Event[] = []; -for (const line of text.split("\n")) { - if (line.trim() === "") { - continue; - } - try { - events.push(JSON.parse(line) as Event); - } catch { - // skip partial line - } -} -const byProcess = new Map(); -for (const event of events) { - const list = byProcess.get(event.p) ?? []; - list.push(event); - byProcess.set(event.p, list); -} -type Proc = { - readonly pid: number; - readonly verb: string; - readonly totalMs: number; - readonly witnessMs: number; - readonly events: Event[]; - readonly kind: string; -}; -function topWitnessMs(list: Event[]): number { - let total = 0; - for (const event of list) { - if (event.k !== "scope" || !event.l || !isWitness(event.l)) { - continue; - } - const parts = (event.s ?? "").split(">"); - parts.pop(); - if (parts.some(isWitness)) { - continue; - } - total += event.ms ?? 0; - } - return total; -} -const procs: Proc[] = []; -for (const [pid, list] of byProcess) { - const exit = list.find((event) => event.k === "exit"); - const verb = list[0]?.v ?? "other"; - const labels = new Set(list.map((event) => event.l ?? "")); - let kind = verb; - if (verb === "up" || verb === "restart" || verb === "run") { - if (labels.has("invoke:seed")) { - kind = `${verb}(cold)`; - } else if (labels.has("invoke:verify")) { - kind = `${verb}(warm)`; - } else { - kind = `${verb}(no-witness)`; - } - } else if (verb === "exec") { - kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; - } - procs.push({ - pid, - verb, - totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), - witnessMs: topWitnessMs(list), - events: list, - kind, - }); -} -const fmt = (ms: number) => (ms / 1000).toFixed(1); -const out: string[] = []; -out.push(`#### X01 trace: ${title}`); -out.push(`events=${events.length} processes=${procs.length}`); -out.push(""); -out.push( - "kind | n | avg total s | sum total s | avg witness s | sum witness s" -); -const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); -for (const kind of kinds) { - const selected = procs.filter((proc) => proc.kind === kind); - const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); - const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); - out.push( - `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` - ); -} -const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); -const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); -out.push( - `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` -); -out.push(""); - -function leafTable(selected: Proc[], heading: string, limit = 24) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const key = `${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} (per process avg over n=${selected.length})`); - out.push("leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function leafByScope(selected: Proc[], heading: string, limit = 40) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const parts = (event.s ?? "") - .split(">") - .filter( - (part) => - isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) - ); - const inner = parts.slice(-2).join(">") || "(none)"; - const key = `${inner} :: ${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push( - `${heading} leaves by scope (per process avg over n=${selected.length})` - ); - out.push("scope :: leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function scopeTable(selected: Proc[], heading: string, limit = 30) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (event.k !== "scope" || !event.l) { - continue; - } - const parts = (event.s ?? "").split(">"); - const key = parts - .filter( - (part) => - PHASE.test(part) || - isWitness(part) || - part.startsWith("xattr:") || - part.startsWith("cmd:") || - part.startsWith("exec-") - ) - .join(">"); - if (key === "") { - continue; - } - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} scopes (per process avg over n=${selected.length})`); - out.push("scope path | n/proc | s/proc | ms/each"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function phaseRows(proc: Proc) { - const rows = new Map(); - for (const event of proc.events) { - if ( - event.k === "phase" && - event.line?.boundary !== "begin" && - event.line?.phase - ) { - const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.line.durationMs ?? 0; - rows.set(event.line.phase, row); - } - } - return rows; -} -for (const kind of [ - "up(cold)", - "up(warm)", - "exec(storage)", - "restart(warm)", - "run(warm)", - "run(cold)", - "down", -]) { - const selected = procs.filter((proc) => proc.kind === kind); - if (selected.length === 0) { - continue; - } - const sample = selected[0] as Proc; - out.push( - `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` - ); - out.push("phase-trace phase | spans | s"); - for (const [phase, row] of phaseRows(sample)) { - out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); - } - out.push(""); - scopeTable([sample], `${kind} first sample`, 40); - leafTable([sample], `${kind} first sample leaves`); - scopeTable(selected, `${kind} all`, 30); - leafTable(selected, `${kind} all leaves`); - leafByScope(selected, `${kind} all`); -} -console.log(out.join("\n")); diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index d9b58c13..c9384e64 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,7 +23,6 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -135,12 +134,7 @@ export type NativeComposeCommandStorageVerifyOptions = { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export function prepareNativeComposeCommandStorage( - opts: Parameters[0] -): ReturnType { - return x01Scope("storage-prepare", () => x01Prepare(opts)); -} -async function x01Prepare(opts: { +export async function prepareNativeComposeCommandStorage(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -231,11 +225,7 @@ async function x01Prepare(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = ( - generation: NativeComposeGeneration, - options: NativeComposeCommandStorageVerifyOptions = {} - ) => x01Scope("storage-verify", () => x01Verify(generation, options)); - const x01Verify = async ( + const verify = async ( generation: NativeComposeGeneration, options: NativeComposeCommandStorageVerifyOptions = {} ) => { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 0aa92b81..8c5dcaf8 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -138,7 +138,6 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -944,9 +943,7 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => - x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); - const x01AssertSaved = async ( + const assertSaved = async ( proof: NativeComposeCommandStorageVerifyOptions ) => { const latest = await store.loadCurrent(); @@ -965,9 +962,7 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - x01Scope("exec-process", () => - runSavedProcess({ options, generation, document, base, signal }) - ), + runSavedProcess({ options, generation, document, base, signal }), }); }); } diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index d8bc9dc8..0019b3fc 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,7 +3,6 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1233,7 +1232,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - const x01Inner = async (args: readonly string[]): Promise => { + return async (args) => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1311,8 +1310,6 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; - return (args) => - x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 943b7932..540f833b 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,6 +1,5 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; -import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -136,8 +135,6 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); - } else if (x01Enabled()) { - trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -150,8 +147,5 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return x01Scope( - phase, - () => context.getStore()?.measure(phase, operation) ?? operation() - ); + return context.getStore()?.measure(phase, operation) ?? operation(); } diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index da785e7b..0c04e7ba 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -96,7 +96,6 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; -import { x01Label, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -814,12 +813,7 @@ async function groupAbsent( await new Promise((resolve) => setTimeout(resolve, 20)); } } -function commandExecutable( - path: string -): Promise { - return x01Time("hash", x01Label([path]), () => x01CommandExecutable(path)); -} -async function x01CommandExecutable( +async function commandExecutable( path: string ): Promise { const canonical = await realpath(path); @@ -966,14 +960,7 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export function runNativeComposeStorageDockerCommand( - supplied: Parameters[0] -): ReturnType { - return x01Scope(`cmd:${x01Label(supplied.args)}`, () => - x01RunCommandInner(supplied) - ); -} -async function x01RunCommandInner(supplied: { +export async function runNativeComposeStorageDockerCommand(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1318,12 +1305,7 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export function createNativeComposeDockerStorageXattrCarrier( - opts: Context -): Promise { - return x01Scope("carrier-new", () => x01CreateCarrier(opts)); -} -async function x01CreateCarrier( +export async function createNativeComposeDockerStorageXattrCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -2042,19 +2024,16 @@ async function x01CreateCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect: (selection) => x01Scope("inspect", () => inspect(selection)), - provision: (selection) => - x01Scope("provision", () => provision(selection)), - invoke: (input) => - x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), + inspect, + provision, + invoke, finish: context.originalCommandRecords === true - ? ({ input, completion }) => - x01Scope("finish", async () => { - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - }) + ? async ({ input, completion }) => { + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + } : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index bfcbfe4e..269c2154 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,7 +78,6 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -494,16 +493,7 @@ function requireXattrAdmission( refuse(); } } -function invokeXattr( - opts: Parameters[0] -): ReturnType { - const operation = - isRecord(opts.request) && typeof opts.request.operation === "string" - ? opts.request.operation - : "unknown"; - return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); -} -async function x01InvokeXattr(opts: { +async function invokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1465,21 +1455,17 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await x01Scope("witness-prepare", () => - prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }) - ); + return await prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await x01Scope("witness-enroll", () => - enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }) - ); + return await enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1490,9 +1476,7 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await x01Scope("witness-verify", () => - verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) - ); + await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ff27e6b8..ef9faf1b 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,7 +4,6 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -41,13 +40,7 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export function exec( - cmd: readonly string[], - opts: ExecOptions = {} -): Promise { - return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); -} -async function x01ExecInner( +export async function exec( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -130,13 +123,7 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export function run( - cmd: readonly string[], - opts: RunOptions = {} -): Promise { - return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); -} -async function x01RunInner( +export async function run( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts deleted file mode 100644 index 48848c68..00000000 --- a/src/lib/x01-cost-probe.ts +++ /dev/null @@ -1,171 +0,0 @@ -// TEMPORARY X01 measurement probe. Reverted before the final commit. -// Writes closed labels and durations only (no ids, paths or values). -import { AsyncLocalStorage } from "node:async_hooks"; -import { appendFileSync } from "node:fs"; - -const FILE = process.env.HACK_X01_TRACE_FILE; -const VERBS = new Set([ - "up", - "down", - "exec", - "run", - "restart", - "ps", - "logs", - "status", - "init", -]); -const SUBS = new Set([ - "info", - "image", - "container", - "volume", - "network", - "compose", - "create", - "start", - "rm", - "inspect", - "ls", - "ps", - "exec", - "run", - "logs", - "version", - "events", - "stop", - "kill", - "wait", - "up", - "down", - "config", - "pull", - "build", - "restart", - "port", - "cp", - "top", -]); -const verb = (() => { - for (const arg of process.argv.slice(2)) { - if (VERBS.has(arg)) { - return arg; - } - } - return "other"; -})(); -const scopes = new AsyncLocalStorage(); -let enabled = typeof FILE === "string" && FILE.startsWith("/"); -let wroteStart = false; - -function append(record: Record): void { - appendFileSync( - FILE as string, - `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` - ); -} - -function write(record: Record): void { - if (!enabled) { - return; - } - try { - if (!wroteStart) { - wroteStart = true; - append({ k: "start", t: Math.round(performance.now()) }); - process.on("exit", (code) => { - try { - append({ k: "exit", c: code, t: Math.round(performance.now()) }); - } catch { - // measurement only - } - }); - } - append({ - s: scopes.getStore() ?? "", - t: Math.round(performance.now()), - ...record, - }); - } catch { - enabled = false; - } -} - -export function x01Enabled(): boolean { - return enabled; -} - -/** Raw line from the phase trace, re-wrapped with process identity. */ -export function x01Phase(line: string): void { - try { - write({ k: "phase", line: JSON.parse(line) as unknown }); - } catch { - enabled = false; - } -} - -export function x01Label(argv: readonly string[], binary = true): string { - const parts: string[] = []; - let rest = argv; - if (binary) { - const name = (argv[0] ?? "").split("/").at(-1) ?? ""; - parts.push( - ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) - ? name - : "bin" - ); - rest = argv.slice(1); - } else { - parts.push("docker"); - } - for (const token of rest) { - if (parts.length >= 3) { - break; - } - if (token.startsWith("-")) { - continue; - } - if (SUBS.has(token)) { - parts.push(token); - } else { - break; - } - } - return parts.join(" "); -} - -export async function x01Time( - kind: string, - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const started = performance.now(); - let ok = false; - try { - const result = await operation(); - ok = true; - return result; - } finally { - write({ - k: kind, - l: label, - ms: Math.round((performance.now() - started) * 10) / 10, - ok, - }); - } -} - -export async function x01Scope( - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const parent = scopes.getStore(); - const path = parent ? `${parent}>${label}` : label; - return await scopes.run(path, () => x01Time("scope", label, operation)); -} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index e7599c52..2201457b 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,8 +43,6 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -770,13 +768,6 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); - // TEMPORARY X01 measurement: one trace file per scenario. - if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { - process.env.HACK_X01_TRACE_FILE = join( - process.env.HACK_X01_TRACE_DIR, - `${scenario.name}.jsonl` - ); - } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index a2157596..2728a605 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1153,10 +1153,6 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - ...(process.env.HACK_X01_TRACE_FILE - ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } - : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From 7b2ccfc8db529200f09f0455dd90cc5d9df9d5e5 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 00:21:00 -0400 Subject: [PATCH 15/52] test(e2e): restore the networks scenario budget after witness coalescing The networks scenario took 3409 s on next with three proofs per storage exec and ran 2275 s on the hosted engine with the coalesced proofs, so the 80 min budget raise from the witness recovery change returns to 60 min. The budget bounds a hung engine; no assertion or command deadline changes. Co-Authored-By: Claude Opus 5.5 --- tests/e2e/native-config-networks-acceptance.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 2728a605..633cb044 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -42,8 +42,9 @@ const FIXTURE = "hack.e2e.native-config-networks"; const OUTPUT_LIMIT = 2 * 1024 * 1024; const COMMAND_TIMEOUT = 180_000; // Witness-bearing exec/up verification runs three fresh helper carriers per proof; on the -// hosted Linux engine that is ~12-15 s per exec and ~35 s per up across three checkouts. -const SCENARIO_TIMEOUT = 80 * 60_000; +// hosted Linux engine a storage exec (two proofs) takes ~10 s and a warm up (six or seven) +// ~30 s across three checkouts, about 38 minutes for this scenario. +const SCENARIO_TIMEOUT = 60 * 60_000; const CLEANUP_TIMEOUT = 3 * 60_000; const BUN_TAG = "oven/bun:1.4.2-slim"; const CADDY_TAG = "lucaslorentz/caddy-docker-proxy:2.10.0-alpine"; From 37f093a5118e7ba19eb5c5c7d6b96e7a5249f76e Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 02:10:55 -0400 Subject: [PATCH 16/52] perf(storage): make witness engine and executable fences cheaper Storage witness proofs keep every fence; three exact costs are cut: - Engine-ID fences: for an explicit DOCKER_HOST=unix:///... selection, the first fence of a command under one exact binding (selection, docker executable, client config and socket identities) still uses `docker info` and must equal one direct GET /info on that socket; only then do later fences under that unchanged binding each make one fresh direct request instead of a CLI process that lists every client plugin. A refused admission keeps the CLI, a moved binding re-admits through the CLI, and an admitted direct failure refuses without fallback. Two exact back-to-back ID reads with nothing between them are collapsed (holder observation into carrier inspect; provision fence into the target observation). - Recorded command executables: the content digest is cached for one command by canonical path and complete stat identity (dev, ino, size, mtime and ctime in ns, mode, uid); every identity check still runs, any change re-reads, and racily fresh files are never cached. - Target observations: the witness's closing re-observation of a deferred-finish verify invocation is answered by the carrier's own closing observation of the same query when the rechecked material binding is unchanged and only reads ran between (one observation per proof). Co-Authored-By: Claude Opus 5.5 --- src/lib/native-compose-engine-identity.ts | 146 ++++++++++++---- ...tive-compose-storage-command-executable.ts | 133 +++++++++++++++ ...compose-storage-witness-docker-identity.ts | 51 ++++++ .../native-compose-storage-witness-docker.ts | 161 ++++++++++-------- ...e-compose-storage-witness-xattr-carrier.ts | 16 +- src/lib/native-compose-storage-witness.ts | 27 ++- tests/helpers/native-compose-command.ts | 5 +- 7 files changed, 414 insertions(+), 125 deletions(-) create mode 100644 src/lib/native-compose-storage-command-executable.ts create mode 100644 src/lib/native-compose-storage-witness-docker-identity.ts diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index 6682491f..63618b2c 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -273,20 +273,16 @@ function readId(opts: { }); } -/** - * Replace only `docker info` transport for an explicit local Unix selection. - * Every call makes a fresh bounded daemon-ID request; no result is cached. Other - * selections return null before admission and retain their existing CLI path. - * Admitted errors never fall back. Socket, executable and selection bindings are - * checked before and after each request under the caller's existing time budget. - */ -export function createNativeComposeEngineIdentityObserver( - opts: { - readonly signal?: AbortSignal; - readonly timeoutMs?: number; - readonly environment?: Environment; - } = {} -): (() => Promise) | null { +type BoundObserver = { + /** Opaque digest of the exact selection, executable, config and socket binding. */ + readonly key: string; + readonly read: () => Promise; +}; +function bindObserver(opts: { + readonly signal?: AbortSignal; + readonly timeoutMs?: number; + readonly environment?: Environment; +}): BoundObserver | null { const environment = opts.environment ?? process.env; const signal = opts.signal; const selected = selection(environment); @@ -328,29 +324,107 @@ export function createNativeComposeEngineIdentityObserver( check(signal, deadline); }; recheck(); - return async () => { - if (reading) { - return refuse(); - } - reading = true; - try { - recheck(); - const result = await readId({ - socketPath: socket.canonical, - signal, - deadline, - limit: remaining, - }); - remaining -= result.bytes; - recheck(); - return result.id; - } catch { - return refuse(); - } finally { - reading = false; - } - }; + const key = new Bun.CryptoHasher("sha256") + .update(JSON.stringify([selected, binary, config, socket, executable])) + .digest("hex"); + return Object.freeze({ + key, + read: async () => { + if (reading) { + return refuse(); + } + reading = true; + try { + recheck(); + const result = await readId({ + socketPath: socket.canonical, + signal, + deadline, + limit: remaining, + }); + remaining -= result.bytes; + recheck(); + return result.id; + } catch { + return refuse(); + } finally { + reading = false; + } + }, + }); } catch { return refuse(); } } + +/** + * Replace only `docker info` transport for an explicit local Unix selection. + * Every call makes a fresh bounded daemon-ID request; no result is cached. Other + * selections return null before admission and retain their existing CLI path. + * Admitted errors never fall back. Socket, executable and selection bindings are + * checked before and after each request under the caller's existing time budget. + */ +export function createNativeComposeEngineIdentityObserver( + opts: { + readonly signal?: AbortSignal; + readonly timeoutMs?: number; + readonly environment?: Environment; + } = {} +): (() => Promise) | null { + return bindObserver(opts)?.read ?? null; +} + +/** + * Command-scoped transport for repeated `docker info --format '{{json .ID}}'` + * fences. Each fence still makes exactly one fresh engine-ID read; no ID is cached. + * + * Admission is per exact binding: the selection values, `docker` executable path + * and identity, client-config path/content and socket path identities. The first + * fence under an eligible binding returns the caller's own Docker CLI read and + * compares it with one direct `GET /info` on the same explicit Unix socket. Only + * an equal ID admits that binding; a direct failure or a different ID keeps the + * CLI for that binding for the rest of the command. Later fences under an admitted + * binding use one fresh direct request each, and its failures refuse without CLI + * fallback. A changed binding is a new binding and is admitted through the CLI + * again. Ineligible selections, and bindings that cannot be established, always + * use the CLI. The returned text has the CLI's `{{json .ID}}` line shape. + */ +export function createNativeComposeEngineIdentityAdmissions(): (opts: { + readonly cli: () => Promise; + readonly signal?: AbortSignal; + readonly timeoutMs: number; + readonly environment: Environment; +}) => Promise { + const admitted = new Map(); + return async (opts) => { + let bound: BoundObserver | null; + try { + bound = bindObserver({ + signal: opts.signal, + timeoutMs: opts.timeoutMs, + environment: opts.environment, + }); + } catch { + bound = null; + } + if (bound === null) { + return await opts.cli(); + } + const state = admitted.get(bound.key); + if (state === true) { + return `${JSON.stringify(await bound.read())}\n`; + } + const text = await opts.cli(); + if (state === undefined) { + let equal = false; + try { + const value: unknown = JSON.parse(text); + equal = typeof value === "string" && value === (await bound.read()); + } catch { + equal = false; + } + admitted.set(bound.key, equal); + } + return text; + }; +} diff --git a/src/lib/native-compose-storage-command-executable.ts b/src/lib/native-compose-storage-command-executable.ts new file mode 100644 index 00000000..a8817d68 --- /dev/null +++ b/src/lib/native-compose-storage-command-executable.ts @@ -0,0 +1,133 @@ +import { constants } from "node:fs"; +import { type FileHandle, lstat, open, realpath } from "node:fs/promises"; +import type { NativeComposeStorageCommandExecutable } from "./native-compose-storage-command-record.ts"; +import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; + +/** Coarser than any supported filesystem timestamp granularity (1 s on HFS+/ext3). */ +const RACY_MS = 2000; +const LIMIT = 268_435_456n; + +async function digestContent(file: FileHandle, size: number): Promise { + const hash = new Bun.CryptoHasher("sha256"); + const bytes = Buffer.alloc(65_536); + let position = 0; + while (position < size) { + const read = await file.read( + bytes, + 0, + Math.min(bytes.length, size - position), + position + ); + if (read.bytesRead <= 0) { + return refuse(); + } + hash.update(bytes.subarray(0, read.bytesRead)); + position += read.bytesRead; + } + return hash.digest("hex"); +} + +/** + * Identity and SHA-256 of a recorded command executable (the Docker CLI and + * `/bin/sh`), with every open, type, link-count, size, write-mode, before/after + * stat, named-path and canonical-path check run on each call. + * + * Only the content read is cached, for the life of one CLI process (one native + * command), keyed by the canonical path and the complete stat identity: device, + * inode, size, mtime and ctime in nanoseconds, mode and owner. Any change to any + * of them re-reads the content. Like Git's racily clean index entries, a file + * whose mtime or ctime is within two seconds of the read is never cached, so a + * write inside the same timestamp tick as the hashed bytes cannot be hidden. + * `now` and `digest` exist for focused tests only. + */ +export function createNativeComposeStorageCommandExecutableReader( + opts: { + readonly now?: () => number; + readonly digest?: (file: FileHandle, size: number) => Promise; + } = {} +): (path: string) => Promise { + const now = opts.now ?? Date.now; + const digest = opts.digest ?? digestContent; + const settled = new Map< + string, + { readonly identity: string; readonly hash: string } + >(); + return async (path) => { + const canonical = await realpath(path); + const file = await open( + canonical, + constants.O_RDONLY | constants.O_NOFOLLOW + ); + try { + const before = await file.stat({ bigint: true }); + if ( + !before.isFile() || + before.nlink !== 1n || + before.size <= 0n || + before.size > LIMIT || + (before.mode & 0o022n) !== 0n + ) { + return refuse(); + } + const identity = JSON.stringify( + [ + before.dev, + before.ino, + before.size, + before.mtimeNs, + before.ctimeNs, + before.mode, + before.uid, + ].map(String) + ); + const size = Number(before.size); + const started = now(); + const cached = settled.get(canonical); + const hash = + cached?.identity === identity ? cached.hash : await digest(file, size); + const after = await file.stat({ bigint: true }), + named = await lstat(canonical, { bigint: true }); + if ( + !( + before.dev === after.dev && + before.ino === after.ino && + before.dev === named.dev && + before.ino === named.ino + ) || + before.size !== after.size || + before.mtimeNs !== after.mtimeNs || + before.ctimeNs !== after.ctimeNs || + before.mode !== after.mode || + before.uid !== after.uid || + (await realpath(path)) !== canonical + ) { + settled.delete(canonical); + return refuse(); + } + const newest = Math.max( + Number(before.mtimeNs / 1_000_000n), + Number(before.ctimeNs / 1_000_000n) + ); + if (newest < started - RACY_MS) { + settled.set(canonical, { identity, hash }); + } else { + settled.delete(canonical); + } + return { + path: canonical, + dev: before.dev.toString(), + ino: before.ino.toString(), + uid: Number(before.uid), + mode: Number(before.mode), + size, + hash, + }; + } finally { + await file.close(); + } + }; +} + +/** The command's single reader; a CLI process runs exactly one native command. */ +export const readNativeComposeStorageCommandExecutable = + createNativeComposeStorageCommandExecutableReader(); diff --git a/src/lib/native-compose-storage-witness-docker-identity.ts b/src/lib/native-compose-storage-witness-docker-identity.ts new file mode 100644 index 00000000..e718b33c --- /dev/null +++ b/src/lib/native-compose-storage-witness-docker-identity.ts @@ -0,0 +1,51 @@ +import { createNativeComposeEngineIdentityAdmissions } from "./native-compose-engine-identity.ts"; +import type { createNativeComposeProbe } from "./native-compose-ownership.ts"; + +type Probe = ReturnType; +const IDENTITY = ["info", "--format", "{{json .ID}}"] as const; + +/** One admission table per CLI process; a process runs exactly one native command. */ +const commandAdmissions = createNativeComposeEngineIdentityAdmissions(); + +function identityQuery(args: readonly string[]): boolean { + return ( + args.length === IDENTITY.length && + args.every((value, index) => value === IDENTITY[index]) + ); +} + +/** + * Storage-witness probe whose exact engine-ID query keeps one fresh read per + * fence but may use the command's admitted direct Unix transport + * (`createNativeComposeEngineIdentityAdmissions`). Every other query, every + * ineligible selection and every unadmitted binding goes to the supplied CLI + * probe unchanged. The direct read shares this probe's signal and deadline, and + * the same environment snapshot the CLI probe captured. Recovery observation + * keeps its own CLI probes. + */ +export function nativeComposeStorageIdentityProbe(opts: { + readonly probe: Probe; + readonly signal?: AbortSignal; + readonly timeoutMs: number; + readonly admissions?: ReturnType< + typeof createNativeComposeEngineIdentityAdmissions + >; +}): Probe { + const { probe, signal } = opts; + const admissions = opts.admissions ?? commandAdmissions; + const deadline = Date.now() + opts.timeoutMs; + const environment = Object.freeze({ ...process.env }); + return (args) => { + const remaining = Math.min(60_000, deadline - Date.now()); + if (!identityQuery(args) || remaining < 1) { + // An expired probe keeps the CLI probe's own timeout refusal. + return probe(args); + } + return admissions({ + cli: () => probe(args), + signal, + timeoutMs: remaining, + environment, + }); + }; +} diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index 0c04e7ba..c8e06218 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -40,6 +40,7 @@ import { type NativeComposeStorageCarrierCompletion, type NativeComposeStorageReadonlyCarrierIntent, } from "./native-compose-storage-carrier-journal.ts"; +import { readNativeComposeStorageCommandExecutable } from "./native-compose-storage-command-executable.ts"; import { type NativeComposeStorageCommandObservation, readNativeComposeStorageCommandObservation, @@ -67,6 +68,7 @@ import { nativeComposeStorageDockerImageReference, selectNativeComposeStorageDockerDependency, } from "./native-compose-storage-witness-docker-artifact.ts"; +import { nativeComposeStorageIdentityProbe } from "./native-compose-storage-witness-docker-identity.ts"; import { assertNativeComposeStorageDockerCarrierVolume, inspectNativeComposeStorageDockerVolume, @@ -813,68 +815,9 @@ async function groupAbsent( await new Promise((resolve) => setTimeout(resolve, 20)); } } -async function commandExecutable( - path: string -): Promise { - const canonical = await realpath(path); - const file = await open(canonical, constants.O_RDONLY | constants.O_NOFOLLOW); - try { - const before = await file.stat({ bigint: true }); - if ( - !before.isFile() || - before.nlink !== 1n || - before.size <= 0n || - before.size > 268_435_456n || - (before.mode & 0o022n) !== 0n - ) { - return refuse(); - } - const hash = new Bun.CryptoHasher("sha256"); - const bytes = Buffer.alloc(65_536); - let position = 0; - const size = Number(before.size); - while (position < size) { - const read = await file.read( - bytes, - 0, - Math.min(bytes.length, size - position), - position - ); - if (read.bytesRead <= 0) { - return refuse(); - } - hash.update(bytes.subarray(0, read.bytesRead)); - position += read.bytesRead; - } - const after = await file.stat({ bigint: true }), - named = await lstat(canonical, { bigint: true }); - if ( - !( - before.dev === after.dev && - before.ino === after.ino && - before.dev === named.dev && - before.ino === named.ino - ) || - before.size !== after.size || - before.mtimeNs !== after.mtimeNs || - before.ctimeNs !== after.ctimeNs || - (await realpath(path)) !== canonical - ) { - return refuse(); - } - return { - path: canonical, - dev: before.dev.toString(), - ino: before.ino.toString(), - uid: Number(before.uid), - mode: Number(before.mode), - size, - hash: hash.digest("hex"), - }; - } finally { - await file.close(); - } -} +/** Every identity check runs per call; only an unchanged stat identity skips the + * content read for the rest of this command. */ +const commandExecutable = readNativeComposeStorageCommandExecutable; async function boundedHostFile(path: string): Promise { const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); try { @@ -1330,11 +1273,14 @@ export async function createNativeComposeDockerStorageXattrCarrier( if (!cleanup) { lifetime(context); } - return createNativeComposeProbe({ - signal: cleanup ? undefined : context.signal, - timeoutMs: cleanup - ? 10_000 - : Math.min(15_000, context.deadline - Date.now()), + const signal = cleanup ? undefined : context.signal; + const timeoutMs = cleanup + ? 10_000 + : Math.min(15_000, context.deadline - Date.now()); + return nativeComposeStorageIdentityProbe({ + probe: createNativeComposeProbe({ signal, timeoutMs }), + signal, + timeoutMs, }); }; let dependency: ReturnType; @@ -1368,7 +1314,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( throw normalized; } const selectedArtifact = dependency.artifact; - const inspect = async (selection: Selection) => { + const observe = async (selection: Selection) => { const stage: NativeComposeEffectRefusal["stage"] = "storage-target"; try { const before = await current(context, selection); @@ -1384,7 +1330,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( return refuse(); } lifetime(context); - return target; + return { binding: after, target }; } catch (error) { retainNativeComposeEffectRefusal(error, { stage, @@ -1393,6 +1339,49 @@ export async function createNativeComposeDockerStorageXattrCarrier( throw error; } }; + const inspect = async (selection: Selection) => + (await observe(selection)).target; + // The closing target observation of the latest deferred-finish verify + // invocation, after which that invocation performed no effect. Every port + // entry consumes it, so it can answer only the next port call. + let closing: { + readonly invocationId: string; + readonly name: string; + readonly storage: string; + readonly binding: NativeComposeMaterialBinding; + readonly target: unknown; + } | null = null; + /** An exact repeat (same query, same rechecked material binding, no effect + * since: the caller attests reads only) answers with that closing observation. */ + const inspectPort = async ( + selection: Selection, + options?: { readonly afterInvocation?: string } + ) => { + const closed = closing; + closing = null; + if ( + closed && + options?.afterInvocation === closed.invocationId && + selection.name === closed.name && + selection.storage === closed.storage + ) { + try { + tool(); + const binding = await current(context, selection); + if (same(binding, closed.binding)) { + lifetime(context); + return closed.target; + } + } catch (error) { + retainNativeComposeEffectRefusal(error, { + stage: "storage-target", + reason: nativeComposeEffectReason(error), + }); + throw error; + } + } + return await inspect(selection); + }; const provision = async ( selection: Selection & { readonly engineId: string; @@ -1400,6 +1389,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( readonly ownerToken: string; } ) => { + closing = null; const stage: NativeComposeEffectRefusal["stage"] = "storage-provision"; try { const before = await current(context, selection, true); @@ -1465,8 +1455,9 @@ export async function createNativeComposeDockerStorageXattrCarrier( requestText: "", captures: [], }; + // The target observation opens with this same engine-ID fence; nothing + // runs between, so a separate identical read here would be a duplicate. const read = probe(); - await engine(read, context); const finalAbsent = await observeNativeComposeStorageDockerTarget({ probe: read, current: before, @@ -1556,6 +1547,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( (completion: NativeComposeStorageCarrierCompletion) => Promise >(); const invoke = async (input: NativeComposeStorageXattrInvocation) => { + closing = null; let stage: NativeComposeEffectRefusal["stage"] = "storage-target"; try { const before = await current(context, input.target, !input.readonly); @@ -1668,12 +1660,17 @@ export async function createNativeComposeDockerStorageXattrCarrier( if (!(created.exitCode === 0 && ID.test(id))) { return refuse(); } + /** `engineFenced` only when the caller's immediately preceding step, with no + * await between, ended with the identical engine-ID fence. */ const inspectCarrier = async ( - cleanup = false + cleanup = false, + engineFenced = false ): Promise => { try { const observed = probe(cleanup); - await engine(observed, context); + if (!engineFenced) { + await engine(observed, context); + } const value: unknown = JSON.parse( await observed([ "container", @@ -1735,7 +1732,8 @@ export async function createNativeComposeDockerStorageXattrCarrier( if (!same(holders, input.target)) { return refuse(); } - const ready = await inspectCarrier(); + // The holder observation just closed with the same engine-ID fence. + const ready = await inspectCarrier(false, true); if ( !( ready.createdAt === known.createdAt && @@ -1931,7 +1929,8 @@ export async function createNativeComposeDockerStorageXattrCarrier( if (selected !== "") { return refuse(); } - const after = await inspect(input.target); + const closed = await observe(input.target); + const after = closed.target; if (!same(after, input.target)) { return refuse(); } @@ -1978,6 +1977,17 @@ export async function createNativeComposeDockerStorageXattrCarrier( await retireInvocationFiles(files); } completed = true; + if (deferred) { + // No effect follows this closing observation before return: retirement + // waits for the journal completion and finish(). + closing = { + invocationId: input.invocationId, + name: input.target.name, + storage: input.target.storage, + binding: closed.binding, + target: after, + }; + } return { artifact: selectedArtifact, carrierId: id, @@ -2024,12 +2034,13 @@ export async function createNativeComposeDockerStorageXattrCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect, + inspect: inspectPort, provision, invoke, finish: context.originalCommandRecords === true ? async ({ input, completion }) => { + closing = null; const finish = finalizers.get(input) ?? refuse(); finalizers.delete(input); await finish(completion); diff --git a/src/lib/native-compose-storage-witness-xattr-carrier.ts b/src/lib/native-compose-storage-witness-xattr-carrier.ts index b3fe8936..b229444a 100644 --- a/src/lib/native-compose-storage-witness-xattr-carrier.ts +++ b/src/lib/native-compose-storage-witness-xattr-carrier.ts @@ -112,10 +112,18 @@ export type NativeComposeStorageXattrInvocation = { }; /** Trusted engine transport. Shape-valid ports alone do not qualify artifacts or runtime behavior. */ export type NativeComposeStorageXattrPorts = { - readonly inspect: (selection: { - readonly name: string; - readonly storage: string; - }) => Promise; + /** `afterInvocation` attests that the caller performed only reads since that + * invocation of this carrier returned. A carrier whose invocation performed no + * effect after its own closing observation of the same selection may answer + * with that observation when the material binding is unchanged; any other + * call observes again. */ + readonly inspect: ( + selection: { + readonly name: string; + readonly storage: string; + }, + options?: { readonly afterInvocation?: string } + ) => Promise; /** Exactly one original cold attempt. Idempotent create alone cannot establish admission. */ readonly provision?: (selection: { readonly name: string; diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 269c2154..7586eb73 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -450,18 +450,23 @@ async function inspectXattr(opts: { readonly saved: XattrExpectation; readonly stopped: boolean; readonly check: XattrCheck; + /** Only when nothing but reads ran since this carrier invocation returned. */ + readonly afterInvocation?: string; }): Promise { checkXattrArtifact(opts.saved, opts.carrier); const before = await opts.check(); checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); - const value = await nativeComposeStorageXattrCarrierPorts( - opts.carrier - ).inspect( - Object.freeze({ - name: opts.saved.binding.name, - storage: opts.saved.binding.storage, - }) - ); + const selection = Object.freeze({ + name: opts.saved.binding.name, + storage: opts.saved.binding.storage, + }); + const ports = nativeComposeStorageXattrCarrierPorts(opts.carrier); + const value = await (opts.afterInvocation === undefined + ? ports.inspect(selection) + : ports.inspect( + selection, + Object.freeze({ afterInvocation: opts.afterInvocation }) + )); const current = await opts.check(); checkNativeComposeStorageXattrCarrierLifetime(opts.carrier); if (JSON.stringify(before) !== JSON.stringify(current)) { @@ -555,7 +560,11 @@ async function invokeXattr(opts: { carrier: opts.carrier, current: latest, }); - const after = await inspectXattr(opts); + // Only reads ran since invoke returned (binding check and result validation). + const after = await inspectXattr({ + ...opts, + afterInvocation: input.invocationId, + }); if (JSON.stringify(after) !== JSON.stringify(target)) { return refuse(); } diff --git a/tests/helpers/native-compose-command.ts b/tests/helpers/native-compose-command.ts index 460e9714..33dc84cc 100644 --- a/tests/helpers/native-compose-command.ts +++ b/tests/helpers/native-compose-command.ts @@ -226,7 +226,9 @@ export async function invoke( root: string, args = ["up", "--detach", "--json"], startupTimeoutMs = 1000, - commandTimeoutMs = 15_000 + commandTimeoutMs = 15_000, + /** Docker client selection only (for example an explicit Unix DOCKER_HOST). */ + extraEnv: Readonly> = {} ) { const child = Bun.spawn( [ @@ -250,6 +252,7 @@ export async function invoke( HACK_COMPOSE_STARTUP_TIMEOUT_MS: String(startupTimeoutMs), CI: "1", HACK_EXECUTION_MODE: "non_interactive", + ...extraEnv, }, stdin: "ignore", stdout: "pipe", From 6a4e118e853c3fb341af2ff7c24e41630b96135f Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 02:10:28 -0400 Subject: [PATCH 17/52] chore(x03): temporary witness probe trace for hosted measurement Temporary measurement only; reverted before the final commit. Re-applies the X01 closed-label trace (reverting eb9a6785) and adds point events for direct engine-ID reads and admissions, cached executable digests, target observations and reused closing observations, plus a shard 1 identity fence benchmark. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 23 ++ scripts/x01-trace-summary.ts | 268 ++++++++++++++++++ scripts/x03-identity-bench.sh | 29 ++ src/lib/native-compose-command-storage.ts | 14 +- src/lib/native-compose-command.ts | 9 +- src/lib/native-compose-engine-identity.ts | 9 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- ...tive-compose-storage-command-executable.ts | 4 + ...ompose-storage-witness-docker-inventory.ts | 8 +- .../native-compose-storage-witness-docker.ts | 45 ++- src/lib/native-compose-storage-witness.ts | 34 ++- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 176 ++++++++++++ tests/e2e/harness.ts | 9 + .../e2e/native-config-networks-acceptance.ts | 4 + 16 files changed, 630 insertions(+), 32 deletions(-) create mode 100644 scripts/x01-trace-summary.ts create mode 100644 scripts/x03-identity-bench.sh create mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 457db89f..09052d11 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -300,13 +300,20 @@ jobs: - name: Verify owned bridge policy and aliases through real compiler and Compose normalization if: matrix.shard == 4 run: HACK_TEST_COMPOSE_CONFIG=1 bun test tests/native-compose-renderer-config.test.ts --test-name-pattern '^one authored bridge keeps internal policy and static aliases through compiler and Compose normalization$' + # TEMPORARY X03 measurement; reverted before the final commit. + - name: X03 engine identity fence benchmark + if: matrix.shard == 1 + run: bash scripts/x03-identity-bench.sh || true - name: Run local and Docker E2E shard env: HACK_E2E_PROCESS_POLICY_REFUSAL_CAPSULE: "1" HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 + # TEMPORARY X01 measurement; reverted before the final commit. + HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | + mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -315,6 +322,22 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" + # TEMPORARY X01 measurement; reverted before the final commit. + - name: X01 witness cost summary + if: always() + run: | + for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do + [ -f "$f" ] || continue + bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true + done + - name: X01 witness cost trace artifact + if: always() + uses: actions/upload-artifact@v4 + with: + name: x01-trace-shard-${{ matrix.shard }} + path: ${{ runner.temp }}/x01-trace + if-no-files-found: ignore + retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts new file mode 100644 index 00000000..ba276c07 --- /dev/null +++ b/scripts/x01-trace-summary.ts @@ -0,0 +1,268 @@ +// TEMPORARY X01 measurement summary. Reverted before the final commit. +// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. +type Event = { + readonly p: number; + readonly v: string; + readonly k: string; + readonly s?: string; + readonly l?: string; + readonly ms?: number; + readonly t?: number; + readonly c?: number; + readonly line?: { + readonly phase?: string; + readonly boundary?: string; + readonly durationMs?: number | null; + }; +}; +const WITNESS = new Set([ + "storage-prepare", + "storage-verify", + "carrier-new", + "inspect", + "provision", + "finish", + "witness-verify", + "witness-prepare", + "witness-enroll", +]); +const isWitness = (label: string) => + WITNESS.has(label) || label.startsWith("invoke:"); +const LEAF = new Set(["probe", "run", "exec", "hash"]); +const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; + +const path = process.argv[2]; +const title = process.argv[3] ?? path; +if (!path) { + throw new Error("usage: x01-trace-summary [title]"); +} +const text = await Bun.file(path).text(); +const events: Event[] = []; +for (const line of text.split("\n")) { + if (line.trim() === "") { + continue; + } + try { + events.push(JSON.parse(line) as Event); + } catch { + // skip partial line + } +} +const byProcess = new Map(); +for (const event of events) { + const list = byProcess.get(event.p) ?? []; + list.push(event); + byProcess.set(event.p, list); +} +type Proc = { + readonly pid: number; + readonly verb: string; + readonly totalMs: number; + readonly witnessMs: number; + readonly events: Event[]; + readonly kind: string; +}; +function topWitnessMs(list: Event[]): number { + let total = 0; + for (const event of list) { + if (event.k !== "scope" || !event.l || !isWitness(event.l)) { + continue; + } + const parts = (event.s ?? "").split(">"); + parts.pop(); + if (parts.some(isWitness)) { + continue; + } + total += event.ms ?? 0; + } + return total; +} +const procs: Proc[] = []; +for (const [pid, list] of byProcess) { + const exit = list.find((event) => event.k === "exit"); + const verb = list[0]?.v ?? "other"; + const labels = new Set(list.map((event) => event.l ?? "")); + let kind = verb; + if (verb === "up" || verb === "restart" || verb === "run") { + if (labels.has("invoke:seed")) { + kind = `${verb}(cold)`; + } else if (labels.has("invoke:verify")) { + kind = `${verb}(warm)`; + } else { + kind = `${verb}(no-witness)`; + } + } else if (verb === "exec") { + kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; + } + procs.push({ + pid, + verb, + totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), + witnessMs: topWitnessMs(list), + events: list, + kind, + }); +} +const fmt = (ms: number) => (ms / 1000).toFixed(1); +const out: string[] = []; +out.push(`#### X01 trace: ${title}`); +out.push(`events=${events.length} processes=${procs.length}`); +out.push(""); +out.push( + "kind | n | avg total s | sum total s | avg witness s | sum witness s" +); +const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); +for (const kind of kinds) { + const selected = procs.filter((proc) => proc.kind === kind); + const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); + const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); + out.push( + `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` + ); +} +const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); +const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); +out.push( + `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` +); +out.push(""); + +function leafTable(selected: Proc[], heading: string, limit = 24) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const key = `${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} (per process avg over n=${selected.length})`); + out.push("leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function leafByScope(selected: Proc[], heading: string, limit = 40) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const parts = (event.s ?? "") + .split(">") + .filter( + (part) => + isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) + ); + const inner = parts.slice(-2).join(">") || "(none)"; + const key = `${inner} :: ${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push( + `${heading} leaves by scope (per process avg over n=${selected.length})` + ); + out.push("scope :: leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function scopeTable(selected: Proc[], heading: string, limit = 30) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (event.k !== "scope" || !event.l) { + continue; + } + const parts = (event.s ?? "").split(">"); + const key = parts + .filter( + (part) => + PHASE.test(part) || + isWitness(part) || + part.startsWith("xattr:") || + part.startsWith("cmd:") || + part.startsWith("exec-") + ) + .join(">"); + if (key === "") { + continue; + } + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} scopes (per process avg over n=${selected.length})`); + out.push("scope path | n/proc | s/proc | ms/each"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function phaseRows(proc: Proc) { + const rows = new Map(); + for (const event of proc.events) { + if ( + event.k === "phase" && + event.line?.boundary !== "begin" && + event.line?.phase + ) { + const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.line.durationMs ?? 0; + rows.set(event.line.phase, row); + } + } + return rows; +} +for (const kind of [ + "up(cold)", + "up(warm)", + "exec(storage)", + "restart(warm)", + "run(warm)", + "run(cold)", + "down", +]) { + const selected = procs.filter((proc) => proc.kind === kind); + if (selected.length === 0) { + continue; + } + const sample = selected[0] as Proc; + out.push( + `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` + ); + out.push("phase-trace phase | spans | s"); + for (const [phase, row] of phaseRows(sample)) { + out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); + } + out.push(""); + scopeTable([sample], `${kind} first sample`, 40); + leafTable([sample], `${kind} first sample leaves`); + scopeTable(selected, `${kind} all`, 30); + leafTable(selected, `${kind} all leaves`); + leafByScope(selected, `${kind} all`); +} +console.log(out.join("\n")); diff --git a/scripts/x03-identity-bench.sh b/scripts/x03-identity-bench.sh new file mode 100644 index 00000000..050dfd5c --- /dev/null +++ b/scripts/x03-identity-bench.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# TEMPORARY X03 measurement; reverted before the final commit. +# Decomposes one engine-ID fence on the hosted engine. Prints timings only. +set -euo pipefail +host="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" +socket="${host#unix://}" +n=40 +bench() { + local label="$1" + shift + local start end + start=$(date +%s%N) + for _ in $(seq "$n"); do + "$@" >/dev/null + done + end=$(date +%s%N) + echo "x03-bench ${label}: $(((end - start) / n / 1000)) us/call over ${n}" +} +echo "x03-bench selection: $(case "$host" in unix://*) echo unix ;; *) echo other ;; esac), DOCKER_HOST set: ${DOCKER_HOST:+yes}" +echo "x03-bench client plugins: $(docker info --format '{{len .ClientInfo.Plugins}}')" +bench "docker info --format {{json .ID}}" docker info --format '{{json .ID}}' +bench "docker version --format server" docker version --format '{{.Server.Version}}' +bench "docker volume ls" docker volume ls --format '{{json .Name}}' +bench "docker context inspect (client only)" docker context inspect --format '{{json .Endpoints.docker.Host}}' +bench "curl GET /info (direct Unix)" curl -sf --unix-socket "$socket" http://docker/info +bench "curl GET /_ping (direct Unix)" curl -sf --unix-socket "$socket" http://docker/_ping +sha="$(command -v docker)" +bench "sha256sum docker CLI" sha256sum "$sha" +echo "x03-bench docker CLI size: $(stat -Lc %s "$sha") bytes" diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index c9384e64..d9b58c13 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,6 +23,7 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -134,7 +135,12 @@ export type NativeComposeCommandStorageVerifyOptions = { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export async function prepareNativeComposeCommandStorage(opts: { +export function prepareNativeComposeCommandStorage( + opts: Parameters[0] +): ReturnType { + return x01Scope("storage-prepare", () => x01Prepare(opts)); +} +async function x01Prepare(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -225,7 +231,11 @@ export async function prepareNativeComposeCommandStorage(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = async ( + const verify = ( + generation: NativeComposeGeneration, + options: NativeComposeCommandStorageVerifyOptions = {} + ) => x01Scope("storage-verify", () => x01Verify(generation, options)); + const x01Verify = async ( generation: NativeComposeGeneration, options: NativeComposeCommandStorageVerifyOptions = {} ) => { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 8c5dcaf8..0aa92b81 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -138,6 +138,7 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -943,7 +944,9 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = async ( + const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => + x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); + const x01AssertSaved = async ( proof: NativeComposeCommandStorageVerifyOptions ) => { const latest = await store.loadCurrent(); @@ -962,7 +965,9 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - runSavedProcess({ options, generation, document, base, signal }), + x01Scope("exec-process", () => + runSavedProcess({ options, generation, document, base, signal }) + ), }); }); } diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index 63618b2c..205641d6 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -3,6 +3,7 @@ import { request } from "node:http"; import { dirname, isAbsolute, join, normalize, parse } from "node:path"; import { isRecord } from "./guards.ts"; import { bindNativeComposeEngineConfig } from "./native-compose-engine-config.ts"; +import { x01Mark, x01Time } from "./x01-cost-probe.ts"; const ENGINE_ID = /^[A-Za-z0-9:-]{1,128}$/; const AMBIGUOUS_PATH = /[%?#\0\r\n]/; @@ -411,18 +412,22 @@ export function createNativeComposeEngineIdentityAdmissions(): (opts: { return await opts.cli(); } const state = admitted.get(bound.key); + const direct = bound.read; if (state === true) { - return `${JSON.stringify(await bound.read())}\n`; + return `${JSON.stringify(await x01Time("ident", "direct", direct))}\n`; } const text = await opts.cli(); if (state === undefined) { let equal = false; try { const value: unknown = JSON.parse(text); - equal = typeof value === "string" && value === (await bound.read()); + equal = + typeof value === "string" && + value === (await x01Time("ident", "admission", direct)); } catch { equal = false; } + x01Mark("ident-admit", equal ? "admitted" : "refused"); admitted.set(bound.key, equal); } return text; diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index 0019b3fc..d8bc9dc8 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,6 +3,7 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1232,7 +1233,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - return async (args) => { + const x01Inner = async (args: readonly string[]): Promise => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1310,6 +1311,8 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; + return (args) => + x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 540f833b..943b7932 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,5 +1,6 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; +import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -135,6 +136,8 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); + } else if (x01Enabled()) { + trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -147,5 +150,8 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return context.getStore()?.measure(phase, operation) ?? operation(); + return x01Scope( + phase, + () => context.getStore()?.measure(phase, operation) ?? operation() + ); } diff --git a/src/lib/native-compose-storage-command-executable.ts b/src/lib/native-compose-storage-command-executable.ts index a8817d68..780dae04 100644 --- a/src/lib/native-compose-storage-command-executable.ts +++ b/src/lib/native-compose-storage-command-executable.ts @@ -2,6 +2,7 @@ import { constants } from "node:fs"; import { type FileHandle, lstat, open, realpath } from "node:fs/promises"; import type { NativeComposeStorageCommandExecutable } from "./native-compose-storage-command-record.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Label, x01Mark } from "./x01-cost-probe.ts"; /** Coarser than any supported filesystem timestamp granularity (1 s on HFS+/ext3). */ const RACY_MS = 2000; @@ -83,6 +84,9 @@ export function createNativeComposeStorageCommandExecutableReader( const size = Number(before.size); const started = now(); const cached = settled.get(canonical); + if (cached?.identity === identity) { + x01Mark("hash-cached", x01Label([canonical])); + } const hash = cached?.identity === identity ? cached.hash : await digest(file, size); const after = await file.stat({ bigint: true }), diff --git a/src/lib/native-compose-storage-witness-docker-inventory.ts b/src/lib/native-compose-storage-witness-docker-inventory.ts index b5bdb741..f292021c 100644 --- a/src/lib/native-compose-storage-witness-docker-inventory.ts +++ b/src/lib/native-compose-storage-witness-docker-inventory.ts @@ -8,6 +8,7 @@ import { type NativeComposeStorageXattrTarget, } from "./native-compose-storage-witness-xattr-carrier.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,254}$/; @@ -164,7 +165,12 @@ export async function assertNativeComposeStorageDockerCarrierVolume(opts: { /** Complete holder reads include foreign containers. The two inventories and selected * volume reread are observed fences; they do not claim an atomic daemon transaction. */ -export async function observeNativeComposeStorageDockerTarget(opts: { +export function observeNativeComposeStorageDockerTarget( + opts: Parameters[0] +): ReturnType { + return x01Scope("target", () => x01ObserveTarget(opts)); +} +async function x01ObserveTarget(opts: { readonly probe: Probe; readonly selection: Selection; readonly current: NativeComposeMaterialBinding; diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index c8e06218..d955faf0 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -98,6 +98,7 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; +import { x01Label, x01Mark, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -817,7 +818,10 @@ async function groupAbsent( } /** Every identity check runs per call; only an unchanged stat identity skips the * content read for the rest of this command. */ -const commandExecutable = readNativeComposeStorageCommandExecutable; +const commandExecutable = (path: string) => + x01Time("hash", x01Label([path]), () => + readNativeComposeStorageCommandExecutable(path) + ); async function boundedHostFile(path: string): Promise { const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); try { @@ -903,7 +907,14 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export async function runNativeComposeStorageDockerCommand(supplied: { +export function runNativeComposeStorageDockerCommand( + supplied: Parameters[0] +): ReturnType { + return x01Scope(`cmd:${x01Label(supplied.args)}`, () => + x01RunCommandInner(supplied) + ); +} +async function x01RunCommandInner(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1248,7 +1259,12 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export async function createNativeComposeDockerStorageXattrCarrier( +export function createNativeComposeDockerStorageXattrCarrier( + opts: Context +): Promise { + return x01Scope("carrier-new", () => x01CreateCarrier(opts)); +} +async function x01CreateCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -1370,6 +1386,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( const binding = await current(context, selection); if (same(binding, closed.binding)) { lifetime(context); + x01Mark("target-reused", "closing"); return closed.target; } } catch (error) { @@ -2034,17 +2051,21 @@ export async function createNativeComposeDockerStorageXattrCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect: inspectPort, - provision, - invoke, + inspect: (selection, options) => + x01Scope("inspect", () => inspectPort(selection, options)), + provision: (selection) => + x01Scope("provision", () => provision(selection)), + invoke: (input) => + x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), finish: context.originalCommandRecords === true - ? async ({ input, completion }) => { - closing = null; - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - } + ? ({ input, completion }) => + x01Scope("finish", async () => { + closing = null; + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + }) : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 7586eb73..1fc68abf 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,6 +78,7 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -498,7 +499,16 @@ function requireXattrAdmission( refuse(); } } -async function invokeXattr(opts: { +function invokeXattr( + opts: Parameters[0] +): ReturnType { + const operation = + isRecord(opts.request) && typeof opts.request.operation === "string" + ? opts.request.operation + : "unknown"; + return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); +} +async function x01InvokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1464,17 +1474,21 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }); + return await x01Scope("witness-prepare", () => + prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }) + ); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }); + return await x01Scope("witness-enroll", () => + enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }) + ); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1485,7 +1499,9 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); + await x01Scope("witness-verify", () => + verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) + ); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ef9faf1b..ff27e6b8 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,6 +4,7 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -40,7 +41,13 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export async function exec( +export function exec( + cmd: readonly string[], + opts: ExecOptions = {} +): Promise { + return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); +} +async function x01ExecInner( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -123,7 +130,13 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export async function run( +export function run( + cmd: readonly string[], + opts: RunOptions = {} +): Promise { + return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); +} +async function x01RunInner( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts new file mode 100644 index 00000000..17b2f555 --- /dev/null +++ b/src/lib/x01-cost-probe.ts @@ -0,0 +1,176 @@ +// TEMPORARY X01 measurement probe. Reverted before the final commit. +// Writes closed labels and durations only (no ids, paths or values). +import { AsyncLocalStorage } from "node:async_hooks"; +import { appendFileSync } from "node:fs"; + +const FILE = process.env.HACK_X01_TRACE_FILE; +const VERBS = new Set([ + "up", + "down", + "exec", + "run", + "restart", + "ps", + "logs", + "status", + "init", +]); +const SUBS = new Set([ + "info", + "image", + "container", + "volume", + "network", + "compose", + "create", + "start", + "rm", + "inspect", + "ls", + "ps", + "exec", + "run", + "logs", + "version", + "events", + "stop", + "kill", + "wait", + "up", + "down", + "config", + "pull", + "build", + "restart", + "port", + "cp", + "top", +]); +const verb = (() => { + for (const arg of process.argv.slice(2)) { + if (VERBS.has(arg)) { + return arg; + } + } + return "other"; +})(); +const scopes = new AsyncLocalStorage(); +let enabled = typeof FILE === "string" && FILE.startsWith("/"); +let wroteStart = false; + +function append(record: Record): void { + appendFileSync( + FILE as string, + `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` + ); +} + +function write(record: Record): void { + if (!enabled) { + return; + } + try { + if (!wroteStart) { + wroteStart = true; + append({ k: "start", t: Math.round(performance.now()) }); + process.on("exit", (code) => { + try { + append({ k: "exit", c: code, t: Math.round(performance.now()) }); + } catch { + // measurement only + } + }); + } + append({ + s: scopes.getStore() ?? "", + t: Math.round(performance.now()), + ...record, + }); + } catch { + enabled = false; + } +} + +export function x01Enabled(): boolean { + return enabled; +} + +/** Raw line from the phase trace, re-wrapped with process identity. */ +export function x01Phase(line: string): void { + try { + write({ k: "phase", line: JSON.parse(line) as unknown }); + } catch { + enabled = false; + } +} + +export function x01Label(argv: readonly string[], binary = true): string { + const parts: string[] = []; + let rest = argv; + if (binary) { + const name = (argv[0] ?? "").split("/").at(-1) ?? ""; + parts.push( + ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) + ? name + : "bin" + ); + rest = argv.slice(1); + } else { + parts.push("docker"); + } + for (const token of rest) { + if (parts.length >= 3) { + break; + } + if (token.startsWith("-")) { + continue; + } + if (SUBS.has(token)) { + parts.push(token); + } else { + break; + } + } + return parts.join(" "); +} + +export async function x01Time( + kind: string, + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const started = performance.now(); + let ok = false; + try { + const result = await operation(); + ok = true; + return result; + } finally { + write({ + k: kind, + l: label, + ms: Math.round((performance.now() - started) * 10) / 10, + ok, + }); + } +} + +/** TEMPORARY X03 measurement: a closed-label point event. */ +export function x01Mark(kind: string, label: string): void { + write({ k: kind, l: label }); +} + +export async function x01Scope( + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const parent = scopes.getStore(); + const path = parent ? `${parent}>${label}` : label; + return await scopes.run(path, () => x01Time("scope", label, operation)); +} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index 2201457b..e7599c52 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,6 +43,8 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -768,6 +770,13 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); + // TEMPORARY X01 measurement: one trace file per scenario. + if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { + process.env.HACK_X01_TRACE_FILE = join( + process.env.HACK_X01_TRACE_DIR, + `${scenario.name}.jsonl` + ); + } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 633cb044..6c519470 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1154,6 +1154,10 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + ...(process.env.HACK_X01_TRACE_FILE + ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } + : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From 2b11ed3c88ba06c0f2d291855bcba4a896fd587a Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 02:39:54 -0400 Subject: [PATCH 18/52] fix(storage): keep closed probe cancellation for direct engine-ID fences An admitted direct engine-ID read that fails because the caller cancelled or its probe deadline expired now reports the CLI probe's existing closed cancel/timeout classification; that probe refuses before spawning anything. Every other admitted direct failure still refuses without CLI fallback. Co-Authored-By: Claude Opus 5.5 --- src/lib/native-compose-storage-witness-docker-identity.ts | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/lib/native-compose-storage-witness-docker-identity.ts b/src/lib/native-compose-storage-witness-docker-identity.ts index e718b33c..b77d53f2 100644 --- a/src/lib/native-compose-storage-witness-docker-identity.ts +++ b/src/lib/native-compose-storage-witness-docker-identity.ts @@ -46,6 +46,13 @@ export function nativeComposeStorageIdentityProbe(opts: { signal, timeoutMs: remaining, environment, + }).catch((error: unknown) => { + // Admitted failures never fall back. Cancellation and expiry keep the CLI + // probe's closed classification: that probe refuses them before spawning. + if (signal?.aborted || Date.now() >= deadline) { + return probe(args); + } + throw error; }); }; } From a777bc40ff036d93992f13463ac9a247ef5d44ee Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 02:40:01 -0400 Subject: [PATCH 19/52] test(storage): pin witness fence counts and refusal controls - Per command on the synthetic engine: 81 of 86 engine-ID fences and 20 of 21 target observations per proof, with every proof kept; an admitted explicit Unix selection uses one CLI fence per command. - A changed daemon ID between two admitted fences refuses before any helper, workload or state change; a refused admission keeps every fence on the CLI. - A target change right after the verify carrier's closing observation is refused by the next remaining observation. - Admission, re-admission, no-fallback and cancellation controls on a real Unix server; executable digest reuse, replacement, same-size tamper, mode, racy-fresh and mid-command replacement controls. - Reference docs describe the three cuts and their limits. Co-Authored-By: Claude Opus 5.5 --- docs/reference/native-compose-ownership.md | 10 + .../native-compose-storage-xattr-carrier.md | 40 ++++ .../helpers/native-compose-storage-docker.ts | 18 +- tests/native-compose-engine-identity.test.ts | 214 +++++++++++++++++ ...compose-storage-command-executable.test.ts | 118 ++++++++++ ...ive-compose-storage-command-record.test.ts | 66 +++++- .../native-compose-storage-fence-cost.test.ts | 215 ++++++++++++++++++ 7 files changed, 679 insertions(+), 2 deletions(-) create mode 100644 tests/native-compose-storage-command-executable.test.ts create mode 100644 tests/native-compose-storage-fence-cost.test.ts diff --git a/docs/reference/native-compose-ownership.md b/docs/reference/native-compose-ownership.md index 0cc7a0ba..23aa7819 100644 --- a/docs/reference/native-compose-ownership.md +++ b/docs/reference/native-compose-ownership.md @@ -127,6 +127,16 @@ falls back to the CLI after a failure. Ambient HTTP proxy variables do not chang the Unix destination. Its deadline and cumulative 8 MiB response limit remain control-channel budgets; cancellation or failure closes only its owned connection. +Storage-witness fences in the Docker carrier repeat the engine-ID read hundreds +of times per command, so they use the same direct transport through +command-scoped admission instead of a fixed observer. Each fence still makes one +fresh read. The first fence under one exact binding (the selection values, +`docker` executable, config and socket identities above) is the existing Docker +CLI read and must equal one direct `GET /info`; only an equal ID admits that +binding for the rest of the command. A different or failed admission read keeps +the CLI for that binding, a changed binding is admitted through the CLI again, +and an admitted direct failure refuses without fallback. + Real Unix-server controls live in `tests/native-compose-engine-identity.test.ts`; existing CLI-transport and proxy ownership controls remain in `tests/native-compose-ingress.test.ts`. These tests establish the bounded transport diff --git a/docs/reference/native-compose-storage-xattr-carrier.md b/docs/reference/native-compose-storage-xattr-carrier.md index 8a44dada..059d03e7 100644 --- a/docs/reference/native-compose-storage-xattr-carrier.md +++ b/docs/reference/native-compose-storage-xattr-carrier.md @@ -225,6 +225,46 @@ retains the private journal and input. Journal bytes do not replace these actual stop and absence observations. Ordinary command admission calls this port on the cataloged platforms; PostgreSQL data roots and crash durability remain unqualified. +### Fence cost within one command + +Every engine-ID fence, target observation and executable identity check still +runs; three exact costs are removed. Each change is local to one CLI process, +which runs exactly one native command. + +- Engine-ID fences (`docker info --format '{{json .ID}}'`) in the Docker + carrier use command-scoped admission for an explicit `DOCKER_HOST=unix:///...` + selection that the ingress transport also admits (see + [ownership](native-compose-ownership.md#ingress-daemon-identity-transport)). + The first fence under one exact binding (selection values, `docker` + executable, client config and socket identities) is the Docker CLI read and + must equal one direct `GET /info` on that socket. Only then does each later + fence under the unchanged binding make one fresh direct request instead of a + CLI process that also lists every client plugin. A different or failed + admission read keeps the CLI for that binding; a moved binding is admitted + through the CLI again; an admitted direct failure refuses without fallback. + No ID is cached. Saved recovery observation keeps its CLI probes. +- Two identical engine-ID reads with nothing awaited between them are one fence: + the holder observation's closing read also fences the carrier inspect that + immediately follows it, and the cold-provision target observation's opening + read is the provision fence. +- The recorded command executables (Docker CLI and `/bin/sh`) are opened and + checked on every call (type, link count, size, write mode, before/after stat, + named path, canonical path). Only their content digest is reused, keyed by the + canonical path and the complete stat identity (device, inode, size, mtime and + ctime in nanoseconds, mode, owner); any change re-reads the content, and a + file changed within two seconds of the read is never reused. +- After a readonly verify invocation with deferred finish, the carrier performs + no effect after its own closing target observation. The witness's immediate + re-observation of the same query, with only reads between, is answered with + that observation when the rechecked material binding is unchanged. Every other + observation, including the finisher's before file retirement and the opening + observation of the next invocation, stays fresh: each effect is still + immediately preceded and followed by a fresh target observation. + +On the synthetic engine a proof keeps 81 of 86 engine-ID fences and 20 of 21 +target observations; `tests/native-compose-storage-fence-cost.test.ts` pins the +per-command counts and the refusal controls. + Holder selection includes normalized ancestor and descendant bind paths; similarly named sibling roots stay independent. Command stdin and captures use held no-follow descriptors, checked after the final admission await and held until owned group diff --git a/tests/helpers/native-compose-storage-docker.ts b/tests/helpers/native-compose-storage-docker.ts index f103f657..c65abf41 100644 --- a/tests/helpers/native-compose-storage-docker.ts +++ b/tests/helpers/native-compose-storage-docker.ts @@ -36,7 +36,17 @@ if(storageArgs[0]==="volume"&&storageArgs[1]==="create"){ console.log(name);process.exit(0); } if(storageArgs[0]==="volume"&&storageArgs[1]==="ls"&&argument("--format")==="{{json .Name}}"){ - for(const row of await readRows(storageVolumes))emit(row.name);process.exit(0); + for(const row of await readRows(storageVolumes))emit(row.name); + // The verify carrier's closing observation lists twice; then the target changes. + if(await Bun.file(storageRoot+"/fault-close-armed").exists()){ + const listed=Number(await Bun.file(storageRoot+"/fault-close-armed").text()||0)+1; + if(listed<2)await Bun.write(storageRoot+"/fault-close-armed",String(listed)); + else{ + const rows=await readRows(storageVolumes);rows[0].createdAt="2026-01-01T00:00:00Z";await writeRows(storageVolumes,rows); + await (await import("node:fs/promises")).rename(storageRoot+"/fault-close-armed",storageRoot+"/fault-target-changed"); + } + } + process.exit(0); } if(storageArgs[0]==="volume"&&storageArgs[1]==="inspect"&&argument("--format").includes(".Mountpoint")){ const rows=await readRows(storageVolumes); @@ -90,6 +100,12 @@ if(storageArgs[0]==="rm"){ await Bun.write(storageRoot+"/fault-post-remove",id); } } + if(await Bun.file(storageRoot+"/fault-after-verify-close").exists()&&removed.host.Mounts[1].ReadOnly){ + const program=removed.host.Mounts[0].Source,owner=program.slice(0,program.indexOf("/storage-carriers/")); + const hash=(await import("node:crypto")).createHash("sha256").update(removed.host.Mounts[1].Source).digest("hex"); + const journal=await Bun.file(owner+"/storage-witnesses/"+hash+"/carrier.json").json(); + if(journal.intent?.operation==="verify")await (await import("node:fs/promises")).rename(storageRoot+"/fault-after-verify-close",storageRoot+"/fault-close-armed"); + } await writeRows(storageCarriers,rows.filter(row=>row.id!==id));console.log(id);process.exit(0); } if(storageArgs[0]==="container"&&storageArgs[1]==="ls"&&argument("--format")==="{{json .ID}}"){ diff --git a/tests/native-compose-engine-identity.test.ts b/tests/native-compose-engine-identity.test.ts index e4275777..6b64461d 100644 --- a/tests/native-compose-engine-identity.test.ts +++ b/tests/native-compose-engine-identity.test.ts @@ -23,11 +23,14 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { bindNativeComposeEngineSocketPath, + createNativeComposeEngineIdentityAdmissions, createNativeComposeEngineIdentityObserver, NativeComposeEngineIdentityError, } from "../src/lib/native-compose-engine-identity.ts"; import { observeNativeComposeIngress } from "../src/lib/native-compose-ingress.ts"; +import { createNativeComposeProbe } from "../src/lib/native-compose-ownership.ts"; import { NativeComposeRoutingError } from "../src/lib/native-compose-routing.ts"; +import { nativeComposeStorageIdentityProbe } from "../src/lib/native-compose-storage-witness-docker-identity.ts"; import { restoreEnv } from "./helpers/env.ts"; const ENGINE = "synthetic-engine"; @@ -657,3 +660,214 @@ test("pure socket path binding pins mode and refuses ambiguous or relative selec expect(bound.path).toBe(socket); expect(requests).toHaveLength(0); }); + +// Command-scoped storage-witness admissions. A CLI stand-in counts each Docker +// CLI fence; the real Unix server counts each direct fence. +function cliStandIn(id: () => string = () => ENGINE) { + let calls = 0; + return { + calls: () => calls, + read: () => { + calls++; + return Promise.resolve(`${JSON.stringify(id())}\n`); + }, + }; +} +function fence( + admissions: ReturnType, + cli: ReturnType +) { + return admissions({ cli: cli.read, timeoutMs: 5000, environment }); +} +test("storage fences answer the first one with the CLI and admit only an equal direct ID", async () => { + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + expect([cli.calls(), requests.length]).toEqual([1, 1]); + for (let index = 0; index < 3; index++) { + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + } + // One fresh direct request per later fence; no further CLI process. + expect([cli.calls(), requests.length, connections.size]).toEqual([1, 4, 4]); +}); +test("a changed daemon ID between two admitted fences is returned, never cached", async () => { + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + await fence(admissions, cli); + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + handler = (_req, res) => + res.end(JSON.stringify({ ID: "replacement-engine", private: CANARY })); + expect(await fence(admissions, cli)).toBe('"replacement-engine"\n'); + handler = (_req, res) => res.end(JSON.stringify({ ID: ENGINE })); + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + expect([cli.calls(), requests.length]).toEqual([1, 4]); +}); +test("a different or failed admission read keeps the CLI for that binding", async () => { + for (const reply of [ + (_req: IncomingMessage, res: ServerResponse) => + res.end(JSON.stringify({ ID: "other-engine" })), + (_req: IncomingMessage, res: ServerResponse) => { + res.statusCode = 500; + res.end(); + }, + ]) { + handler = reply; + requests = []; + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + for (let index = 0; index < 3; index++) { + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + } + expect([cli.calls(), requests.length]).toEqual([3, 1]); + } +}); +test("admitted direct failures refuse without CLI fallback", async () => { + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + await fence(admissions, cli); + handler = (_req, res) => { + res.statusCode = 500; + res.end(CANARY); + }; + await refusal(() => fence(admissions, cli)); + expect([cli.calls(), requests.length]).toEqual([1, 2]); +}); +test("a moved config, socket or executable binding is admitted through the CLI again", async () => { + const path = await config("{}"); + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([1, 2]); + await rename(path, join(root, "previous-config.json")); + await writeFile(path, "{}", { mode: 0o600 }); + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([2, 4]); + await rename(socket, join(root, "original.sock")); + await listen(socket); + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([3, 6]); + await rename(docker, join(root, "original-docker")); + await writeFile(docker, "#!/bin/sh\nexit 98\n", { mode: 0o700 }); + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([4, 8]); +}); +test("ineligible or unbindable selections and CLI admission failures stay on the CLI", async () => { + for (const selection of [ + { DOCKER_HOST: undefined }, + { DOCKER_CONTEXT: "selected-context" }, + { DOCKER_HOST: `unix://${join(root, "missing.sock")}` }, + ]) { + const admissions = createNativeComposeEngineIdentityAdmissions(); + const cli = cliStandIn(); + for (let index = 0; index < 2; index++) { + expect( + await admissions({ + cli: cli.read, + timeoutMs: 5000, + environment: { ...environment, ...selection }, + }) + ).toBe(`"${ENGINE}"\n`); + } + expect(cli.calls()).toBe(2); + } + expect(requests).toHaveLength(0); + const admissions = createNativeComposeEngineIdentityAdmissions(); + const failure = new Error("synthetic CLI fence failure"); + await expect( + admissions({ + cli: () => Promise.reject(failure), + timeoutMs: 5000, + environment, + }) + ).rejects.toBe(failure); + // A failed CLI admission records nothing: the next fence admits again. + const cli = cliStandIn(); + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([1, 2]); +}); +/** The storage probe snapshots process.env, like the CLI probe it wraps. */ +async function withSelectedProcessEnvironment(body: () => Promise) { + const keys = [ + "PATH", + "HOME", + "DOCKER_HOST", + "DOCKER_CONFIG", + "DOCKER_CONTEXT", + "DOCKER_TLS", + "DOCKER_TLS_VERIFY", + "DOCKER_CUSTOM_HEADERS", + "DOCKER_CERT_PATH", + "DOCKER_API_VERSION", + ] as const; + const before = keys.map((key) => process.env[key]); + try { + for (const key of keys) { + restoreEnv(key, environment[key]); + } + await body(); + } finally { + for (const [index, key] of keys.entries()) { + restoreEnv(key, before[index]); + } + } +} +test("cancelling an admitted direct fence keeps the CLI probe's closed cancellation without a process", async () => { + await withSelectedProcessEnvironment(async () => { + const admissions = createNativeComposeEngineIdentityAdmissions(); + await fence(admissions, cliStandIn()); + const controller = new AbortController(); + const probe = nativeComposeStorageIdentityProbe({ + probe: createNativeComposeProbe({ + signal: controller.signal, + timeoutMs: 5000, + }), + signal: controller.signal, + timeoutMs: 5000, + admissions, + }); + handler = () => controller.abort(); + const spawn = spyOn(Bun, "spawn"); + try { + await expect( + probe(["info", "--format", "{{json .ID}}"]) + ).rejects.toMatchObject({ code: "E_NATIVE_COMPOSE_PROBE_CANCELLED" }); + expect(spawn).not.toHaveBeenCalled(); + expect(requests).toHaveLength(2); + } finally { + spawn.mockRestore(); + } + }); +}); +test("storage identity probe routes only the exact engine-ID query", async () => { + const routed: string[] = []; + await withSelectedProcessEnvironment(async () => { + const admissions = createNativeComposeEngineIdentityAdmissions(); + const probe = nativeComposeStorageIdentityProbe({ + probe: (args) => { + routed.push(args.join(" ")); + return Promise.resolve(`${JSON.stringify(ENGINE)}\n`); + }, + timeoutMs: 5000, + admissions, + }); + for (const args of [ + ["info", "--format", "{{json .ID}}"], + ["info", "--format", "{{json .ID}}"], + ["info", "--format", '{"id":{{json .ID}}}'], + ["volume", "ls", "--format", "{{json .Name}}"], + ]) { + await probe(args); + } + expect(routed).toEqual([ + "info --format {{json .ID}}", + 'info --format {"id":{{json .ID}}}', + "volume ls --format {{json .Name}}", + ]); + expect(requests).toHaveLength(2); + }); +}); diff --git a/tests/native-compose-storage-command-executable.test.ts b/tests/native-compose-storage-command-executable.test.ts new file mode 100644 index 00000000..28cc95d9 --- /dev/null +++ b/tests/native-compose-storage-command-executable.test.ts @@ -0,0 +1,118 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { + chmod, + link, + mkdtemp, + readFile, + realpath, + rename, + rm, + symlink, + utimes, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createNativeComposeStorageCommandExecutableReader } from "../src/lib/native-compose-storage-command-executable.ts"; + +let root: string; +let path: string; +beforeEach(async () => { + root = await realpath(await mkdtemp(join(tmpdir(), "command-executable-"))); + path = join(root, "docker"); + await writeFile(path, "#!/bin/sh\nexit 0\n", { mode: 0o700 }); +}); +afterEach(async () => { + await rm(root, { recursive: true, force: true }); +}); + +/** A reader whose clock treats every file as settled, counting content reads. */ +function reader(opts: { readonly settled?: boolean } = {}) { + let digests = 0; + const read = createNativeComposeStorageCommandExecutableReader({ + now: opts.settled === false ? Date.now : () => Date.now() + 60_000, + digest: async (file, size) => { + digests++; + const bytes = Buffer.alloc(size); + await file.read(bytes, 0, size, 0); + return new Bun.CryptoHasher("sha256").update(bytes).digest("hex"); + }, + }); + return { read, digests: () => digests }; +} + +test("an unchanged stat identity reuses the digest and still returns the full identity", async () => { + const { read, digests } = reader(); + const first = await read(path); + expect(await read(path)).toEqual(first); + expect(await read(path)).toEqual(first); + expect(digests()).toBe(1); + expect(first).toMatchObject({ + path, + hash: new Bun.CryptoHasher("sha256") + .update(await readFile(path)) + .digest("hex"), + }); +}); +test("replacing the executable between reads is a new identity and is re-read", async () => { + const { read, digests } = reader(); + const first = await read(path); + // Same bytes, new inode: the recorded identity no longer matches. + await rename(path, join(root, "original")); + await writeFile(path, "#!/bin/sh\nexit 0\n", { mode: 0o700 }); + const same = await read(path); + expect(same.ino).not.toBe(first.ino); + expect(same.hash).toBe(first.hash); + // Different bytes behind the same path: a different digest. + await rename(path, join(root, "second")); + await writeFile(path, "#!/bin/sh\nexit 7\n", { mode: 0o700 }); + const replaced = await read(path); + expect(replaced.hash).not.toBe(first.hash); + expect(digests()).toBe(3); +}); +test("same-size in-place tamper with restored mtime is re-read through ctime", async () => { + const { read, digests } = reader(); + const first = await read(path); + const before = await Bun.file(path).stat(); + const bytes = await readFile(path, "utf8"); + await writeFile(path, bytes.replace("exit 0", "exit 9")); + await utimes(path, before.atime, before.mtime); + const tampered = await read(path); + expect([tampered.ino, tampered.size]).toEqual([first.ino, first.size]); + expect(tampered.hash).not.toBe(first.hash); + expect(digests()).toBe(2); +}); +test("mode changes are re-read and group or other writable files still refuse", async () => { + const { read, digests } = reader(); + const first = await read(path); + await chmod(path, 0o500); + expect((await read(path)).mode).not.toBe(first.mode); + expect(digests()).toBe(2); + await chmod(path, 0o720); + await expect(read(path)).rejects.toThrow("values omitted"); + await chmod(path, 0o700); + await link(path, join(root, "second-link")); + await expect(read(path)).rejects.toThrow("values omitted"); +}); +test("a racily fresh executable is never cached", async () => { + const { read, digests } = reader({ settled: false }); + await read(path); + await read(path); + await read(path); + expect(digests()).toBe(3); +}); +test("a changed canonical target is a different cache entry", async () => { + const { read, digests } = reader(); + const other = join(root, "other"); + await writeFile(other, "#!/bin/sh\nexit 3\n", { mode: 0o700 }); + const alias = join(root, "alias"); + await symlink(path, alias); + const first = await read(alias); + expect(first.path).toBe(path); + await rm(alias); + await symlink(other, alias); + const second = await read(alias); + expect(second.path).toBe(other); + expect(second.hash).not.toBe(first.hash); + expect(digests()).toBe(2); +}); diff --git a/tests/native-compose-storage-command-record.test.ts b/tests/native-compose-storage-command-record.test.ts index c70a66b5..2bc06d10 100644 --- a/tests/native-compose-storage-command-record.test.ts +++ b/tests/native-compose-storage-command-record.test.ts @@ -1,5 +1,12 @@ import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; -import { existsSync, readFileSync } from "node:fs"; +import { + existsSync, + readFileSync, + renameSync, + statSync, + utimesSync, + writeFileSync, +} from "node:fs"; import { mkdtemp, readFile, @@ -409,6 +416,63 @@ async function transport( } } } +test.each(["unchanged", "replaced", "tampered"] as const)( + "a recorded executable %s after its digest was cached: only unchanged runs the target", + (change) => + ownedCase(async () => { + const f = await fixture(), + marker = join(f.root, "effect"); + const bin = await realpath( + await mkdtemp(join(tmpdir(), "native-command-executable-")) + ); + roots.push(bin); + const program = join(bin, "program"); + const text = '#!/bin/sh\nprintf complete > "$1"; printf value\n'; + await writeFile(program, text, { mode: 0o700 }); + // Settle past the racy window so the arming digest is cached for this command. + await new Promise((resolve) => setTimeout(resolve, 2100)); + try { + const run = runNativeComposeStorageDockerCommand({ + context: { + signal: new AbortController().signal, + deadline: Date.now() + 5000, + }, + files: f.files, + args: [program, marker], + timeoutMs: 5000, + // After the record is armed with the cached digest, before spawn. + beforeSpawn: () => { + if (change === "replaced") { + renameSync(program, join(bin, "original")); + writeFileSync(program, text, { mode: 0o700 }); + } else if (change === "tampered") { + const before = statSync(program); + writeFileSync(program, text.replace("value", "VALUE")); + utimesSync(program, before.atime, before.mtime); + } + }, + assertAdmitted: async () => {}, + originalCommand: { owner: f.owner, kind: "create", carrier: null }, + }); + if (change === "unchanged") { + expect(await run).toEqual({ exitCode: 0, stdout: "value" }); + expect(await readFile(marker, "utf8")).toBe("complete"); + expect((await f.read()).commands[0]?.settlement).toMatchObject({ + exitCode: 0, + }); + return; + } + await expect(run).rejects.toThrow("values omitted"); + expect(existsSync(marker)).toBe(false); + expect((await f.read()).commands[0]?.settlement).toBeNull(); + } finally { + if (!nativeComposeStorageCommandOwnerConfirmed(f.owner)) { + retained.add(f.root); + } + } + })(), + 30_000 +); test( "real fast command cannot execute before its durable stopped-child publication", ownedCase(async () => { diff --git a/tests/native-compose-storage-fence-cost.test.ts b/tests/native-compose-storage-fence-cost.test.ts new file mode 100644 index 00000000..3f096511 --- /dev/null +++ b/tests/native-compose-storage-fence-cost.test.ts @@ -0,0 +1,215 @@ +import { afterEach, expect, test } from "bun:test"; +import { mkdir, writeFile } from "node:fs/promises"; +import { createServer, type Server } from "node:http"; +import type { Socket } from "node:net"; +import { join } from "node:path"; +import { openNativeComposeGenerationStore } from "../src/lib/native-compose-generation.ts"; +import { fixture, invoke } from "./helpers/native-compose-command.ts"; + +const storage = { data: { kind: "persistent", scope: "worktree" } } as const; +const IDENTITY = JSON.stringify(["info", "--format", "{{json .ID}}"]); +// Every storage target observation lists volumes exactly twice (open and close). +const OBSERVATION = JSON.stringify([ + "volume", + "ls", + "--format", + "{{json .Name}}", +]); +// Per storage proof (root, verify, root-after carriers) on this synthetic engine. +// Before X03: 86 engine-ID fences and 21 target observations per proof. +const PROOF_FENCES = 81; +const PROOF_OBSERVATIONS = 20; +// The command's saved file-engine observation reads the ID twice, unchanged. +const FILE_ENGINE_FENCES = 2; + +const servers: Server[] = []; +const peers = new Set(); +afterEach(async () => { + for (const peer of peers) { + peer.destroy(); + } + peers.clear(); + for (const server of servers.splice(0)) { + await new Promise((resolve) => server.close(() => resolve())); + } +}); +/** A real Unix `GET /info` endpoint for the same engine the synthetic CLI reports. */ +async function engine( + root: string, + id: (request: number) => string = () => "fixture-engine" +) { + const socket = join(root, "engine.sock"); + const client = join(root, "client"); + await mkdir(client, { mode: 0o700 }); + await writeFile(join(client, "config.json"), "{}\n", { mode: 0o600 }); + let served = 0; + const server = createServer((req, res) => { + const request = served++; + if (req.method !== "GET" || req.url !== "/info") { + res.statusCode = 404; + res.end(); + return; + } + res.end(JSON.stringify({ ID: id(request) })); + }); + server.on("connection", (peer) => peers.add(peer)); + servers.push(server); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(socket, resolve); + }); + return { + env: { DOCKER_HOST: `unix://${socket}`, DOCKER_CONFIG: client }, + served: () => served, + }; +} +async function lines(root: string, name: string): Promise { + const file = Bun.file(join(root, name)); + return (await file.exists()) + ? (await file.text()).trim().split("\n").filter(Boolean) + : []; +} +async function counted( + root: string, + args: readonly string[], + selected?: Awaited> +) { + const before = (await lines(root, "requests")).length; + const direct = selected?.served() ?? 0; + const result = await invoke( + root, + [...args], + 30_000, + 120_000, + selected?.env ?? {} + ); + const calls = (await lines(root, "requests")).slice(before); + return { + code: result.code, + cli: calls.filter((line) => line === IDENTITY).length, + direct: (selected?.served() ?? 0) - direct, + observations: calls.filter((line) => line === OBSERVATION).length / 2, + }; +} +async function saved(root: string) { + const store = await openNativeComposeGenerationStore({ + projectRoot: root, + instance: null, + mode: "saved", + }); + try { + return await store.loadCurrent(); + } finally { + await store.close(); + } +} +async function enrolled() { + const root = await fixture("", false, { storage, noHooks: true }); + expect((await invoke(root, undefined, 30_000, 120_000)).code).toBe(0); + return root; +} + +test("storage fence count per command: every proof kept, exact duplicates gone, one CLI admission", async () => { + const root = await enrolled(); + // CLI transport (no explicit Unix selection): every fence is a `docker info`. + expect(await counted(root, ["exec", "web", "--", "true"])).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 2 * PROOF_FENCES, + direct: 0, + observations: 2 * PROOF_OBSERVATIONS, + }); + expect(await counted(root, ["up", "--detach", "--json"])).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 6 * PROOF_FENCES, + direct: 0, + observations: 6 * PROOF_OBSERVATIONS, + }); + // Admitted Unix selection: the same fences, one fresh read each. Only the + // first storage fence is a CLI read, checked against one extra direct read. + const selected = await engine(root); + expect(await counted(root, ["exec", "web", "--", "true"], selected)).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 1, + direct: 2 * PROOF_FENCES, + observations: 2 * PROOF_OBSERVATIONS, + }); + expect(await counted(root, ["up", "--detach", "--json"], selected)).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 1, + direct: 6 * PROOF_FENCES, + observations: 6 * PROOF_OBSERVATIONS, + }); +}, 390_000); + +test("a refused admission keeps every storage fence on the Docker CLI for the command", async () => { + const root = await enrolled(); + // The socket answers for a different engine than the CLI's selection. + const selected = await engine(root, () => "other-engine"); + expect(await counted(root, ["exec", "web", "--", "true"], selected)).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 2 * PROOF_FENCES, + direct: 1, + observations: 2 * PROOF_OBSERVATIONS, + }); +}, 390_000); + +test("a changed daemon ID between two admitted fences refuses before the workload", async () => { + const root = await enrolled(); + const before = await saved(root); + const carriers = await Bun.file(join(root, "carriers")).text(); + const operations = (await lines(root, "storage-operations")).length; + // Admission reads the original ID; every later fence sees another engine. + const selected = await engine(root, (request) => + request === 0 ? "fixture-engine" : "replacement-engine" + ); + const requestsBefore = (await lines(root, "requests")).length; + const result = await counted(root, ["up", "--detach", "--json"], selected); + expect(result.code).toBe(1); + // The admission and the next fence: the opening target observation refuses. + expect(result.direct).toBe(2); + const calls = (await lines(root, "requests")).slice(requestsBefore); + expect( + calls.some((line) => { + const args: string[] = JSON.parse(line); + return ( + args[0] === "compose" || + args[0] === "create" || + args[0] === "start" || + args[0] === "rm" + ); + }) + ).toBe(false); + expect((await lines(root, "storage-operations")).length).toBe(operations); + expect(await Bun.file(join(root, "carriers")).text()).toBe(carriers); + const after = await saved(root); + expect(after.storageWitnesses).toEqual(before.storageWitnesses); + expect(after.storageWitnessesPending).toBe(false); + expect(after.pending).toBeNull(); +}, 390_000); + +test("a target change after the verify carrier's closing observation is still detected", async () => { + const root = await enrolled(); + const events = (await lines(root, "storage-events")).length; + // Change the selected volume right after the verify carrier's own closing + // observation, where the witness's former duplicate re-observation stood. + await writeFile(join(root, "fault-after-verify-close"), "0"); + const requestsBefore = (await lines(root, "requests")).length; + const result = await counted(root, ["exec", "web", "--", "true"]); + expect(await Bun.file(join(root, "fault-target-changed")).exists()).toBe( + true + ); + expect(result.code).not.toBe(0); + // The carrier's closing observation (two listings, the change after the + // second) completed; a later complete observation saw the change and refused. + const calls = (await lines(root, "requests")).slice(requestsBefore); + const removed = calls.findLastIndex( + (line) => (JSON.parse(line) as string[])[0] === "rm" + ); + expect( + calls.slice(removed).filter((line) => line === OBSERVATION) + ).toHaveLength(4); + // The opening proof refused: no workload exec and no later proof ran. + const after = (await lines(root, "storage-events")).slice(events); + expect(after).not.toContain("exec"); + expect(after).toEqual(["root", "verify"]); +}, 390_000); From 994ddb894ef407319a2f6ba4bbdd04511e07ea40 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 02:41:07 -0400 Subject: [PATCH 20/52] chore(x03): revert temporary witness probe trace Measurement is complete; the hosted trace run was 38211327829. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 23 -- scripts/x01-trace-summary.ts | 268 ------------------ scripts/x03-identity-bench.sh | 29 -- src/lib/native-compose-command-storage.ts | 14 +- src/lib/native-compose-command.ts | 9 +- src/lib/native-compose-engine-identity.ts | 9 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- ...tive-compose-storage-command-executable.ts | 4 - ...ompose-storage-witness-docker-inventory.ts | 8 +- .../native-compose-storage-witness-docker.ts | 45 +-- src/lib/native-compose-storage-witness.ts | 34 +-- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 176 ------------ tests/e2e/harness.ts | 9 - .../e2e/native-config-networks-acceptance.ts | 4 - 16 files changed, 32 insertions(+), 630 deletions(-) delete mode 100644 scripts/x01-trace-summary.ts delete mode 100644 scripts/x03-identity-bench.sh delete mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 09052d11..457db89f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -300,20 +300,13 @@ jobs: - name: Verify owned bridge policy and aliases through real compiler and Compose normalization if: matrix.shard == 4 run: HACK_TEST_COMPOSE_CONFIG=1 bun test tests/native-compose-renderer-config.test.ts --test-name-pattern '^one authored bridge keeps internal policy and static aliases through compiler and Compose normalization$' - # TEMPORARY X03 measurement; reverted before the final commit. - - name: X03 engine identity fence benchmark - if: matrix.shard == 1 - run: bash scripts/x03-identity-bench.sh || true - name: Run local and Docker E2E shard env: HACK_E2E_PROCESS_POLICY_REFUSAL_CAPSULE: "1" HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 - # TEMPORARY X01 measurement; reverted before the final commit. - HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | - mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -322,22 +315,6 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" - # TEMPORARY X01 measurement; reverted before the final commit. - - name: X01 witness cost summary - if: always() - run: | - for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do - [ -f "$f" ] || continue - bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true - done - - name: X01 witness cost trace artifact - if: always() - uses: actions/upload-artifact@v4 - with: - name: x01-trace-shard-${{ matrix.shard }} - path: ${{ runner.temp }}/x01-trace - if-no-files-found: ignore - retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts deleted file mode 100644 index ba276c07..00000000 --- a/scripts/x01-trace-summary.ts +++ /dev/null @@ -1,268 +0,0 @@ -// TEMPORARY X01 measurement summary. Reverted before the final commit. -// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. -type Event = { - readonly p: number; - readonly v: string; - readonly k: string; - readonly s?: string; - readonly l?: string; - readonly ms?: number; - readonly t?: number; - readonly c?: number; - readonly line?: { - readonly phase?: string; - readonly boundary?: string; - readonly durationMs?: number | null; - }; -}; -const WITNESS = new Set([ - "storage-prepare", - "storage-verify", - "carrier-new", - "inspect", - "provision", - "finish", - "witness-verify", - "witness-prepare", - "witness-enroll", -]); -const isWitness = (label: string) => - WITNESS.has(label) || label.startsWith("invoke:"); -const LEAF = new Set(["probe", "run", "exec", "hash"]); -const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; - -const path = process.argv[2]; -const title = process.argv[3] ?? path; -if (!path) { - throw new Error("usage: x01-trace-summary [title]"); -} -const text = await Bun.file(path).text(); -const events: Event[] = []; -for (const line of text.split("\n")) { - if (line.trim() === "") { - continue; - } - try { - events.push(JSON.parse(line) as Event); - } catch { - // skip partial line - } -} -const byProcess = new Map(); -for (const event of events) { - const list = byProcess.get(event.p) ?? []; - list.push(event); - byProcess.set(event.p, list); -} -type Proc = { - readonly pid: number; - readonly verb: string; - readonly totalMs: number; - readonly witnessMs: number; - readonly events: Event[]; - readonly kind: string; -}; -function topWitnessMs(list: Event[]): number { - let total = 0; - for (const event of list) { - if (event.k !== "scope" || !event.l || !isWitness(event.l)) { - continue; - } - const parts = (event.s ?? "").split(">"); - parts.pop(); - if (parts.some(isWitness)) { - continue; - } - total += event.ms ?? 0; - } - return total; -} -const procs: Proc[] = []; -for (const [pid, list] of byProcess) { - const exit = list.find((event) => event.k === "exit"); - const verb = list[0]?.v ?? "other"; - const labels = new Set(list.map((event) => event.l ?? "")); - let kind = verb; - if (verb === "up" || verb === "restart" || verb === "run") { - if (labels.has("invoke:seed")) { - kind = `${verb}(cold)`; - } else if (labels.has("invoke:verify")) { - kind = `${verb}(warm)`; - } else { - kind = `${verb}(no-witness)`; - } - } else if (verb === "exec") { - kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; - } - procs.push({ - pid, - verb, - totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), - witnessMs: topWitnessMs(list), - events: list, - kind, - }); -} -const fmt = (ms: number) => (ms / 1000).toFixed(1); -const out: string[] = []; -out.push(`#### X01 trace: ${title}`); -out.push(`events=${events.length} processes=${procs.length}`); -out.push(""); -out.push( - "kind | n | avg total s | sum total s | avg witness s | sum witness s" -); -const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); -for (const kind of kinds) { - const selected = procs.filter((proc) => proc.kind === kind); - const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); - const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); - out.push( - `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` - ); -} -const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); -const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); -out.push( - `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` -); -out.push(""); - -function leafTable(selected: Proc[], heading: string, limit = 24) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const key = `${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} (per process avg over n=${selected.length})`); - out.push("leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function leafByScope(selected: Proc[], heading: string, limit = 40) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const parts = (event.s ?? "") - .split(">") - .filter( - (part) => - isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) - ); - const inner = parts.slice(-2).join(">") || "(none)"; - const key = `${inner} :: ${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push( - `${heading} leaves by scope (per process avg over n=${selected.length})` - ); - out.push("scope :: leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function scopeTable(selected: Proc[], heading: string, limit = 30) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (event.k !== "scope" || !event.l) { - continue; - } - const parts = (event.s ?? "").split(">"); - const key = parts - .filter( - (part) => - PHASE.test(part) || - isWitness(part) || - part.startsWith("xattr:") || - part.startsWith("cmd:") || - part.startsWith("exec-") - ) - .join(">"); - if (key === "") { - continue; - } - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} scopes (per process avg over n=${selected.length})`); - out.push("scope path | n/proc | s/proc | ms/each"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function phaseRows(proc: Proc) { - const rows = new Map(); - for (const event of proc.events) { - if ( - event.k === "phase" && - event.line?.boundary !== "begin" && - event.line?.phase - ) { - const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.line.durationMs ?? 0; - rows.set(event.line.phase, row); - } - } - return rows; -} -for (const kind of [ - "up(cold)", - "up(warm)", - "exec(storage)", - "restart(warm)", - "run(warm)", - "run(cold)", - "down", -]) { - const selected = procs.filter((proc) => proc.kind === kind); - if (selected.length === 0) { - continue; - } - const sample = selected[0] as Proc; - out.push( - `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` - ); - out.push("phase-trace phase | spans | s"); - for (const [phase, row] of phaseRows(sample)) { - out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); - } - out.push(""); - scopeTable([sample], `${kind} first sample`, 40); - leafTable([sample], `${kind} first sample leaves`); - scopeTable(selected, `${kind} all`, 30); - leafTable(selected, `${kind} all leaves`); - leafByScope(selected, `${kind} all`); -} -console.log(out.join("\n")); diff --git a/scripts/x03-identity-bench.sh b/scripts/x03-identity-bench.sh deleted file mode 100644 index 050dfd5c..00000000 --- a/scripts/x03-identity-bench.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash -# TEMPORARY X03 measurement; reverted before the final commit. -# Decomposes one engine-ID fence on the hosted engine. Prints timings only. -set -euo pipefail -host="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" -socket="${host#unix://}" -n=40 -bench() { - local label="$1" - shift - local start end - start=$(date +%s%N) - for _ in $(seq "$n"); do - "$@" >/dev/null - done - end=$(date +%s%N) - echo "x03-bench ${label}: $(((end - start) / n / 1000)) us/call over ${n}" -} -echo "x03-bench selection: $(case "$host" in unix://*) echo unix ;; *) echo other ;; esac), DOCKER_HOST set: ${DOCKER_HOST:+yes}" -echo "x03-bench client plugins: $(docker info --format '{{len .ClientInfo.Plugins}}')" -bench "docker info --format {{json .ID}}" docker info --format '{{json .ID}}' -bench "docker version --format server" docker version --format '{{.Server.Version}}' -bench "docker volume ls" docker volume ls --format '{{json .Name}}' -bench "docker context inspect (client only)" docker context inspect --format '{{json .Endpoints.docker.Host}}' -bench "curl GET /info (direct Unix)" curl -sf --unix-socket "$socket" http://docker/info -bench "curl GET /_ping (direct Unix)" curl -sf --unix-socket "$socket" http://docker/_ping -sha="$(command -v docker)" -bench "sha256sum docker CLI" sha256sum "$sha" -echo "x03-bench docker CLI size: $(stat -Lc %s "$sha") bytes" diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index d9b58c13..c9384e64 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,7 +23,6 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -135,12 +134,7 @@ export type NativeComposeCommandStorageVerifyOptions = { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export function prepareNativeComposeCommandStorage( - opts: Parameters[0] -): ReturnType { - return x01Scope("storage-prepare", () => x01Prepare(opts)); -} -async function x01Prepare(opts: { +export async function prepareNativeComposeCommandStorage(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -231,11 +225,7 @@ async function x01Prepare(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = ( - generation: NativeComposeGeneration, - options: NativeComposeCommandStorageVerifyOptions = {} - ) => x01Scope("storage-verify", () => x01Verify(generation, options)); - const x01Verify = async ( + const verify = async ( generation: NativeComposeGeneration, options: NativeComposeCommandStorageVerifyOptions = {} ) => { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 0aa92b81..8c5dcaf8 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -138,7 +138,6 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -944,9 +943,7 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => - x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); - const x01AssertSaved = async ( + const assertSaved = async ( proof: NativeComposeCommandStorageVerifyOptions ) => { const latest = await store.loadCurrent(); @@ -965,9 +962,7 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - x01Scope("exec-process", () => - runSavedProcess({ options, generation, document, base, signal }) - ), + runSavedProcess({ options, generation, document, base, signal }), }); }); } diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index 205641d6..63618b2c 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -3,7 +3,6 @@ import { request } from "node:http"; import { dirname, isAbsolute, join, normalize, parse } from "node:path"; import { isRecord } from "./guards.ts"; import { bindNativeComposeEngineConfig } from "./native-compose-engine-config.ts"; -import { x01Mark, x01Time } from "./x01-cost-probe.ts"; const ENGINE_ID = /^[A-Za-z0-9:-]{1,128}$/; const AMBIGUOUS_PATH = /[%?#\0\r\n]/; @@ -412,22 +411,18 @@ export function createNativeComposeEngineIdentityAdmissions(): (opts: { return await opts.cli(); } const state = admitted.get(bound.key); - const direct = bound.read; if (state === true) { - return `${JSON.stringify(await x01Time("ident", "direct", direct))}\n`; + return `${JSON.stringify(await bound.read())}\n`; } const text = await opts.cli(); if (state === undefined) { let equal = false; try { const value: unknown = JSON.parse(text); - equal = - typeof value === "string" && - value === (await x01Time("ident", "admission", direct)); + equal = typeof value === "string" && value === (await bound.read()); } catch { equal = false; } - x01Mark("ident-admit", equal ? "admitted" : "refused"); admitted.set(bound.key, equal); } return text; diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index d8bc9dc8..0019b3fc 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,7 +3,6 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1233,7 +1232,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - const x01Inner = async (args: readonly string[]): Promise => { + return async (args) => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1311,8 +1310,6 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; - return (args) => - x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 943b7932..540f833b 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,6 +1,5 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; -import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -136,8 +135,6 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); - } else if (x01Enabled()) { - trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -150,8 +147,5 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return x01Scope( - phase, - () => context.getStore()?.measure(phase, operation) ?? operation() - ); + return context.getStore()?.measure(phase, operation) ?? operation(); } diff --git a/src/lib/native-compose-storage-command-executable.ts b/src/lib/native-compose-storage-command-executable.ts index 780dae04..a8817d68 100644 --- a/src/lib/native-compose-storage-command-executable.ts +++ b/src/lib/native-compose-storage-command-executable.ts @@ -2,7 +2,6 @@ import { constants } from "node:fs"; import { type FileHandle, lstat, open, realpath } from "node:fs/promises"; import type { NativeComposeStorageCommandExecutable } from "./native-compose-storage-command-record.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Label, x01Mark } from "./x01-cost-probe.ts"; /** Coarser than any supported filesystem timestamp granularity (1 s on HFS+/ext3). */ const RACY_MS = 2000; @@ -84,9 +83,6 @@ export function createNativeComposeStorageCommandExecutableReader( const size = Number(before.size); const started = now(); const cached = settled.get(canonical); - if (cached?.identity === identity) { - x01Mark("hash-cached", x01Label([canonical])); - } const hash = cached?.identity === identity ? cached.hash : await digest(file, size); const after = await file.stat({ bigint: true }), diff --git a/src/lib/native-compose-storage-witness-docker-inventory.ts b/src/lib/native-compose-storage-witness-docker-inventory.ts index f292021c..b5bdb741 100644 --- a/src/lib/native-compose-storage-witness-docker-inventory.ts +++ b/src/lib/native-compose-storage-witness-docker-inventory.ts @@ -8,7 +8,6 @@ import { type NativeComposeStorageXattrTarget, } from "./native-compose-storage-witness-xattr-carrier.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,254}$/; @@ -165,12 +164,7 @@ export async function assertNativeComposeStorageDockerCarrierVolume(opts: { /** Complete holder reads include foreign containers. The two inventories and selected * volume reread are observed fences; they do not claim an atomic daemon transaction. */ -export function observeNativeComposeStorageDockerTarget( - opts: Parameters[0] -): ReturnType { - return x01Scope("target", () => x01ObserveTarget(opts)); -} -async function x01ObserveTarget(opts: { +export async function observeNativeComposeStorageDockerTarget(opts: { readonly probe: Probe; readonly selection: Selection; readonly current: NativeComposeMaterialBinding; diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index d955faf0..c8e06218 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -98,7 +98,6 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; -import { x01Label, x01Mark, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -818,10 +817,7 @@ async function groupAbsent( } /** Every identity check runs per call; only an unchanged stat identity skips the * content read for the rest of this command. */ -const commandExecutable = (path: string) => - x01Time("hash", x01Label([path]), () => - readNativeComposeStorageCommandExecutable(path) - ); +const commandExecutable = readNativeComposeStorageCommandExecutable; async function boundedHostFile(path: string): Promise { const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); try { @@ -907,14 +903,7 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export function runNativeComposeStorageDockerCommand( - supplied: Parameters[0] -): ReturnType { - return x01Scope(`cmd:${x01Label(supplied.args)}`, () => - x01RunCommandInner(supplied) - ); -} -async function x01RunCommandInner(supplied: { +export async function runNativeComposeStorageDockerCommand(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1259,12 +1248,7 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export function createNativeComposeDockerStorageXattrCarrier( - opts: Context -): Promise { - return x01Scope("carrier-new", () => x01CreateCarrier(opts)); -} -async function x01CreateCarrier( +export async function createNativeComposeDockerStorageXattrCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -1386,7 +1370,6 @@ async function x01CreateCarrier( const binding = await current(context, selection); if (same(binding, closed.binding)) { lifetime(context); - x01Mark("target-reused", "closing"); return closed.target; } } catch (error) { @@ -2051,21 +2034,17 @@ async function x01CreateCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect: (selection, options) => - x01Scope("inspect", () => inspectPort(selection, options)), - provision: (selection) => - x01Scope("provision", () => provision(selection)), - invoke: (input) => - x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), + inspect: inspectPort, + provision, + invoke, finish: context.originalCommandRecords === true - ? ({ input, completion }) => - x01Scope("finish", async () => { - closing = null; - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - }) + ? async ({ input, completion }) => { + closing = null; + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + } : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 1fc68abf..7586eb73 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,7 +78,6 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -499,16 +498,7 @@ function requireXattrAdmission( refuse(); } } -function invokeXattr( - opts: Parameters[0] -): ReturnType { - const operation = - isRecord(opts.request) && typeof opts.request.operation === "string" - ? opts.request.operation - : "unknown"; - return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); -} -async function x01InvokeXattr(opts: { +async function invokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1474,21 +1464,17 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await x01Scope("witness-prepare", () => - prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }) - ); + return await prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await x01Scope("witness-enroll", () => - enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }) - ); + return await enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1499,9 +1485,7 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await x01Scope("witness-verify", () => - verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) - ); + await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ff27e6b8..ef9faf1b 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,7 +4,6 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -41,13 +40,7 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export function exec( - cmd: readonly string[], - opts: ExecOptions = {} -): Promise { - return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); -} -async function x01ExecInner( +export async function exec( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -130,13 +123,7 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export function run( - cmd: readonly string[], - opts: RunOptions = {} -): Promise { - return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); -} -async function x01RunInner( +export async function run( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts deleted file mode 100644 index 17b2f555..00000000 --- a/src/lib/x01-cost-probe.ts +++ /dev/null @@ -1,176 +0,0 @@ -// TEMPORARY X01 measurement probe. Reverted before the final commit. -// Writes closed labels and durations only (no ids, paths or values). -import { AsyncLocalStorage } from "node:async_hooks"; -import { appendFileSync } from "node:fs"; - -const FILE = process.env.HACK_X01_TRACE_FILE; -const VERBS = new Set([ - "up", - "down", - "exec", - "run", - "restart", - "ps", - "logs", - "status", - "init", -]); -const SUBS = new Set([ - "info", - "image", - "container", - "volume", - "network", - "compose", - "create", - "start", - "rm", - "inspect", - "ls", - "ps", - "exec", - "run", - "logs", - "version", - "events", - "stop", - "kill", - "wait", - "up", - "down", - "config", - "pull", - "build", - "restart", - "port", - "cp", - "top", -]); -const verb = (() => { - for (const arg of process.argv.slice(2)) { - if (VERBS.has(arg)) { - return arg; - } - } - return "other"; -})(); -const scopes = new AsyncLocalStorage(); -let enabled = typeof FILE === "string" && FILE.startsWith("/"); -let wroteStart = false; - -function append(record: Record): void { - appendFileSync( - FILE as string, - `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` - ); -} - -function write(record: Record): void { - if (!enabled) { - return; - } - try { - if (!wroteStart) { - wroteStart = true; - append({ k: "start", t: Math.round(performance.now()) }); - process.on("exit", (code) => { - try { - append({ k: "exit", c: code, t: Math.round(performance.now()) }); - } catch { - // measurement only - } - }); - } - append({ - s: scopes.getStore() ?? "", - t: Math.round(performance.now()), - ...record, - }); - } catch { - enabled = false; - } -} - -export function x01Enabled(): boolean { - return enabled; -} - -/** Raw line from the phase trace, re-wrapped with process identity. */ -export function x01Phase(line: string): void { - try { - write({ k: "phase", line: JSON.parse(line) as unknown }); - } catch { - enabled = false; - } -} - -export function x01Label(argv: readonly string[], binary = true): string { - const parts: string[] = []; - let rest = argv; - if (binary) { - const name = (argv[0] ?? "").split("/").at(-1) ?? ""; - parts.push( - ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) - ? name - : "bin" - ); - rest = argv.slice(1); - } else { - parts.push("docker"); - } - for (const token of rest) { - if (parts.length >= 3) { - break; - } - if (token.startsWith("-")) { - continue; - } - if (SUBS.has(token)) { - parts.push(token); - } else { - break; - } - } - return parts.join(" "); -} - -export async function x01Time( - kind: string, - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const started = performance.now(); - let ok = false; - try { - const result = await operation(); - ok = true; - return result; - } finally { - write({ - k: kind, - l: label, - ms: Math.round((performance.now() - started) * 10) / 10, - ok, - }); - } -} - -/** TEMPORARY X03 measurement: a closed-label point event. */ -export function x01Mark(kind: string, label: string): void { - write({ k: kind, l: label }); -} - -export async function x01Scope( - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const parent = scopes.getStore(); - const path = parent ? `${parent}>${label}` : label; - return await scopes.run(path, () => x01Time("scope", label, operation)); -} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index e7599c52..2201457b 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,8 +43,6 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -770,13 +768,6 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); - // TEMPORARY X01 measurement: one trace file per scenario. - if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { - process.env.HACK_X01_TRACE_FILE = join( - process.env.HACK_X01_TRACE_DIR, - `${scenario.name}.jsonl` - ); - } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 6c519470..633cb044 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1154,10 +1154,6 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - ...(process.env.HACK_X01_TRACE_FILE - ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } - : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From 3edbd546e686a5573f26bd0e4473b28a34d37213 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:00:38 -0400 Subject: [PATCH 21/52] feat(agents): package offline skill references and the HACK-P pattern catalogue Plugin skills can now ship references at skills//references/*.md from a typed registry (src/agents/skill-references.ts). The generator writes each reference with a recomputable content revision, removes stale ones, and links them from the plugin SKILL.md. Standalone hack-cli installs ship no references, so their render (and ownership fingerprint) is unchanged. The first reference is the hack-cli pattern catalogue, rendered from the typed register of all twelve HACK-P cards with dispositions, owners, missing contracts and value-free provenance. Tests check completeness, CLI_SPEC command resolution, a banned-recipe scan outside negative-example blocks, and an offline link contract over every packaged skill markdown file. External URLs in hack-install and hack-remote move under "Further reading". docs/agent-guidance.md records the reference contract and the promote/defer decision for the lifecycle, cleanup, checks, sessions, hack-run and hack-feedback skill candidates. Co-Authored-By: Claude Opus 5.5 --- docs/agent-guidance.md | 29 ++ plugins/hack/skills/hack-cli/SKILL.md | 4 + .../skills/hack-cli/references/patterns.md | 217 +++++++++ plugins/hack/skills/hack-install/SKILL.md | 11 +- plugins/hack/skills/hack-remote/SKILL.md | 9 +- scripts/generate-agent-plugins.ts | 71 ++- src/agents/install-skill.ts | 11 +- src/agents/pattern-catalogue.ts | 423 ++++++++++++++++++ src/agents/remote-bootstrap-skill.ts | 9 +- src/agents/skill-references.ts | 98 ++++ tests/agent-plugin-generation.test.ts | 10 +- tests/helpers/skill-links.ts | 172 +++++++ tests/pattern-catalogue.test.ts | 217 +++++++++ tests/skill-references.test.ts | 130 ++++++ 14 files changed, 1398 insertions(+), 13 deletions(-) create mode 100644 plugins/hack/skills/hack-cli/references/patterns.md create mode 100644 src/agents/pattern-catalogue.ts create mode 100644 src/agents/skill-references.ts create mode 100644 tests/helpers/skill-links.ts create mode 100644 tests/pattern-catalogue.test.ts create mode 100644 tests/skill-references.test.ts diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 44ae2c1d..09f8622d 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -66,3 +66,32 @@ belong to their client adapters; shared behavior belongs to the canonical source preserve customized files. See [integration setup](integrations.md). - Historical Factory validation artifacts describe past runs; they are evidence, not current contributor policy. Do not rewrite history to resemble the current workflow. + +## Packaged references and skill candidates + +Plugin skills can ship markdown references at `skills//references/*.md`. +Register each one in `src/agents/skill-references.ts` (skill, file, render); the +generator writes it with a recomputable content revision and links it from the +plugin's SKILL.md. Standalone `hack setup` installs ship no references, so their +`hack-cli` render stays unlinked. References must resolve inside the installed +package and work offline: relative links stay inside `plugins/hack`, never point +at repository `docs/` or absolute paths, and external URLs appear only under a +"Further reading" heading. `tests/skill-references.test.ts` enforces this. + +The first reference is the `hack-cli` pattern catalogue, rendered from the typed +register in `src/agents/pattern-catalogue.ts`. It keeps all twelve HACK-P cards +(HACK-1145..HACK-1156). A card that is not supported names its missing contract +and domain owner; the disposition does not waive that owner's runtime gates. +`tests/pattern-catalogue.test.ts` checks completeness, CLI_SPEC command +resolution and the banned-recipe scan. + +Skill candidates and where their patterns live now: + +| Candidate | Decision | Patterns live in | Promotion owner and missing contract | +| --- | --- | --- | --- | +| lifecycle | Defer | `hack-cli` "Lifecycle + startup"; catalogue HACK-P01, P04, P05, P06 | WU29: identity-aware adoption and ownership-safe shared host lifecycles | +| cleanup | Defer | `hack-cli` "Lifecycle + startup" (`--prune-caches`); catalogue HACK-P09, P10 | WU12/WU30: reference-aware retention and cache provenance | +| checks | Defer | `hack-cli` "Running things"; catalogue HACK-P02, P10, P11 | WU11: qualified portable toolchain fixture with a distinct trigger | +| sessions | Defer | `hack-cli` "Workspaces" and [sessions](sessions.md) | WU18: trigger evaluations showing `hack-cli` misroutes session work | +| hack-run | Defer | `hack-cli` "Running things" | WU26: decides between a hack-run skill and references | +| hack-feedback | Defer | Not shipped | WU25: privacy-reviewed feedback drafts | diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index d9ded7b3..e35765aa 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -187,3 +187,7 @@ Use `hack` as the primary interface for local-first development. - Init patterns: `hack agent patterns` - MCP (no-shell only): `hack setup mcp` - MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect. + +## References + +- [Pattern catalogue](references/patterns.md): reusable project patterns with what Hack supports today, the missing contract and its owner. diff --git a/plugins/hack/skills/hack-cli/references/patterns.md b/plugins/hack/skills/hack-cli/references/patterns.md new file mode 100644 index 00000000..b191f818 --- /dev/null +++ b/plugins/hack/skills/hack-cli/references/patterns.md @@ -0,0 +1,217 @@ +# Hack pattern catalogue + +Reusable patterns observed across Hack projects, with what Hack supports today. +Use a card's commands only within its disposition. A partial, retained, deferred +or rejected card names the missing contract and its owner; do not improvise the +missing part with ad hoc scripts. + +| Id | Pattern | Disposition | Owner | +| --- | --- | --- | --- | +| HACK-P01 | One-shot dependency and generated-artifact bootstrap | Partial | WU07/WU11 | +| HACK-P02 | Portable toolchain without an always-on application | Partial | WU11 | +| HACK-P03 | Cloud prerequisite preflight | Partial | WU27 | +| HACK-P04 | Shared host service with identity-aware adoption | Retained as example | WU29 | +| HACK-P05 | Branch-owned external routing claim | Deferred | WU29 | +| HACK-P06 | Opt-in public tunnel with end-to-end readiness | Deferred | WU29 | +| HACK-P07 | Laptop and headless-runner capability overlays | Partial | WU27/WU30 | +| HACK-P08 | Tracked defaults plus idempotent machine-local hydration | Deferred | WU27/WU14 | +| HACK-P09 | Disposable run workspaces with reference-aware retention | Partial | WU12/WU30 | +| HACK-P10 | Pinned tool image plus independent dependency caches | Partial | WU30 | +| HACK-P11 | Stable agent guidance and typed operation results | Partial | WU19/WU21 | +| HACK-P12 | Multi-lane admission and durable run provenance | Deferred | WU13/WU04 | + +## HACK-P01: One-shot dependency and generated-artifact bootstrap + +- Use when: Services need installed dependencies or generated artifacts before the application starts. +- Disposition: Partial. Missing: locked-v1 covers one immutable payload per volume. Atomic publication of worktree-specific generated artifacts, env-dependent installers, and two-worktree or interrupted-writer qualification remain open. +- Owner: WU07/WU11; tracked in HACK-1145. +- Commands: `hack up --json`, `hack ps --json`, `hack logs --no-follow --tail 100` +- Prerequisites: A Compose initializer service that consumers require through `service_completed_successfully`; for the opt-in `hack.dependencies.cache-protocol: locked-v1` label, a pinned installer image and an immutable dependency payload. +- Env ownership: locked-v1 runs the initializer and verifier with a clean environment and refuses explicit `environment`/`env_file`; credential- or env-dependent installers keep their existing setup. +- Readiness: Installation, generated-artifact publication and application readiness are separate steps. Completion is the initializer's successful exit plus its verifier, not a populated cache. +- Cleanup: Ordinary `hack down` keeps dependency volumes. Keep worktree-specific generated output in its own always-run step and directory. +- Provenance: 4 consumers (HACK-P01 research register). + + +> Negative control: A populated cache directory is not a completion marker. Concurrent writers to one volume and failed publication need ownership and recovery; consumers must see a complete old or complete new artifact. + + +## HACK-P02: Portable toolchain without an always-on application + +- Use when: Checks, builds or tests need pinned tools in a container, but the project has no long-running app to start. +- Disposition: Partial. Missing: Cross-worktree cache reuse with bounded eviction, a source receipt for linked worktrees, and owned cache accounting remain open. +- Owner: WU11; tracked in HACK-1146. +- Commands: `hack run --profile -- ` +- Prerequisites: A one-shot Compose service behind an opt-in profile with pinned tool versions, explicit resource bounds and separate named volumes for reusable build caches. +- Env ownership: Mount no Docker socket, user home, SSH agent or credential directory; keep project secrets out of the image build context. +- Readiness: The task's own exit status is the result. Check the `hack run` exit status directly, not through a pipe filter, so a failed check stays a failure. +- Cleanup: The service is one-shot. Caches live in named volumes that `hack down` preserves; a source-identity override used for tests must not reach release evidence. +- Provenance: 4 consumers (HACK-P02 research register). + + +> Negative control: A test-only source identity override must not escape into release evidence, and a linked worktree's Git pointer needs deliberate handling rather than an assumed `.git` directory. + + +## HACK-P03: Cloud prerequisite preflight + +- Use when: Startup depends on cloud credentials or tokens that may be missing, expired or scoped wrongly. +- Disposition: Partial. Missing: Hack has no credential availability, expiry or scope check and no machine-identity path for headless runners; the WU27 env contract owns both. +- Owner: WU27; tracked in HACK-1147. +- Commands: `hack doctor --json`, `hack env explain --json`, `hack env explain --target compose --json` +- Prerequisites: Required key names declared in Hack env; the credential tool and its native sign-in owned by the user. +- Env ownership: Report key names, winning source and repair direction, never values. Check the container view separately from the host view. +- Readiness: Preflight runs before startup and fails with the missing name; a present value is not proof that it is valid or unexpired. +- Cleanup: Preflight starts nothing and leaves no state. +- Provenance: 3 consumers (HACK-P03 research register). + + +> Negative control: A token on the host does not prove the container receives it, and a headless runner cannot depend on interactive browser sign-in. + + +## HACK-P04: Shared host service with identity-aware adoption + +- Use when: Several projects or branches share a host helper such as a session issuer or local proxy. +- Disposition: Retained as example. Missing: Today's adoption is port-only. Health- and upstream-checked adoption with a recorded adopted-versus-owned state is the WU29 lifecycle ownership contract. +- Owner: WU29; tracked in HACK-1148. +- Commands: `hack up --json`, `hack logs --no-follow --tail 100` +- Prerequisites: A lifecycle process with `singleton` ports. Hack adopts an existing helper only when every configured port is occupied and the config opts into `onConflict: "adopt"`. +- Env ownership: The helper's credentials stay with its owner; adoption must not forward broad host env. +- Readiness: Pair adoption with a `lifecycle.up.before` hook that checks the listener's identity and expected upstream and fails on mismatch. +- Cleanup: Hack stops only processes it started; an adopted helper is not Hack-owned and is never stopped by Hack. +- Provenance: single-consumer generalization (HACK-P04 research register). + + +> Negative control: Port-only reuse can attach to a foreign service. Process liveness or a placeholder process is not readiness, and no foreign process may be killed to free a port. + + +## HACK-P05: Branch-owned external routing claim + +- Use when: A branch needs an external route or callback claim tied to its own instance. +- Disposition: Deferred. Missing: Hack has no branch-scoped external claim/release contract with bounded retry, stop-during-retry and PID-reuse safety. +- Owner: WU29; tracked in HACK-1149. +- Commands: none yet; do not substitute a hand-written recipe. +- Prerequisites: Explicit project and branch identity for the claim; no recipe is published until the contract exists. +- Env ownership: Claim credentials stay host-scoped and are not written into tracked files. +- Readiness: A claim must be a bounded, observable operation with a retry budget. +- Cleanup: Release only the claim and process group that this branch owns. +- Provenance: single-consumer generalization (HACK-P05 research register). + + +> Negative control: The observed consumer uses a fixed default URL, unbounded retry and broad name-based cleanup. Retain it as a negative example, not a copyable recipe. + + +## HACK-P06: Opt-in public tunnel with end-to-end readiness + +- Use when: A developer occasionally needs the local application reachable from the public internet. +- Disposition: Deferred. Missing: Hack has no opt-in tunnel lifecycle with external-route readiness and owned-child shutdown. +- Owner: WU29; tracked in HACK-1150. +- Commands: none yet; do not substitute a hand-written recipe. +- Prerequisites: Default off; host-scoped tunnel credentials, the tunnel tool and the intended CA. +- Env ownership: Pass only the tunnel's own credential; never forward broad host env. +- Readiness: Probe the exposed application route end to end; a missing credential, tool or CA fails clearly. +- Cleanup: Shutdown leaves no owned tunnel child; disabled mode starts no tunnel. +- Provenance: single-consumer generalization (HACK-P06 research register). + + +> Negative control: A running tunnel process or open TCP listener alone does not prove the external route reaches the right upstream. + + +## HACK-P07: Laptop and headless-runner capability overlays + +- Use when: The same application topology runs on a laptop and on a headless runner with different auth, network or helpers. +- Disposition: Partial. Missing: Hack has env overlays and value-free inspection but no laptop/headless capability contract or headless machine identity; WU27 and WU30 own them. +- Owner: WU27/WU30; tracked in HACK-1151. +- Commands: `hack env explain --env --json`, `hack env explain --env --target compose --json`, `hack up --env --json` +- Prerequisites: One shared Compose topology; env overlays that express only the auth, network and helper differences. +- Env ownership: Overlays own the differences; inspect the effective value source per target instead of reading host env. +- Readiness: Verify container metadata and dependency reachability on each target; the runner must not need browser sign-in. +- Cleanup: Overlays add no state of their own; inherited worktree overrides must be visible and deliberate. +- Provenance: single-consumer generalization (HACK-P07 research register). + + +> Negative control: Do not change locality flags just to bypass a readiness check. Host env is not evidence of container delivery. + + +## HACK-P08: Tracked defaults plus idempotent machine-local hydration + +- Use when: A repository tracks env defaults and each machine adds local values on top. +- Disposition: Deferred. Missing: Hack has no semantic no-op detection or atomic machine-local import contract for env hydration. +- Owner: WU27/WU14; tracked in HACK-1152. +- Commands: none yet; do not substitute a hand-written recipe. +- Prerequisites: Tracked defaults, a machine-local layer and preserved operator overrides. +- Env ownership: Defaults are tracked; hydration writes only the local layer and keeps values out of argv. +- Readiness: Repeat hydration leaves the tracked tree clean; an interrupted import still leaves a resolvable layer. +- Cleanup: Unchanged values cause no rewrite. +- Provenance: single-consumer generalization (HACK-P08 research register). + + +> Negative control: Re-encrypting unchanged entries causes diff and stash churn, and values passed in command arguments are not a secure import mechanism. + + +## HACK-P09: Disposable run workspaces with reference-aware retention + +- Use when: Runs or branches create workspaces and caches that should be reclaimed once nothing uses them. +- Disposition: Partial. Missing: Generation/run ownership, active references, holds and interrupted-reclamation recovery are not implemented for run workspaces. +- Owner: WU12/WU30; tracked in HACK-1153. +- Commands: `hack down --prune-caches`, `hack branch remove `, `hack projects prune --dry-run` +- Prerequisites: Explicitly disposable cache volumes (`hack.cache.disposable: "true"` or Next.js build output) and a reviewed dry run before pruning. +- Env ownership: Reclamation reads no env values. +- Readiness: Not applicable; retention is a cleanup contract. +- Cleanup: `--prune-caches` is confirmation-gated and removes only exact, project-owned disposable volumes; sessions and application data keep their own retention. +- Provenance: single-consumer generalization (HACK-P09 research register). + + +> Negative control: Stopped or Docker-reclaimable does not mean disposable. Active, pending, held or unknown resources must survive. + + +## HACK-P10: Pinned tool image plus independent dependency caches + +- Use when: A runner or toolchain image must stay pinned while application dependencies and credentials vary. +- Disposition: Partial. Missing: Hack does not yet distinguish L1, L2 and cold misses, detect a tampered cache, or emit stage timing and provenance receipts. +- Owner: WU30; tracked in HACK-1154. +- Commands: `hack run --profile -- `, `hack up --json` +- Prerequisites: Tool manifests pinned in the image; application dependencies in separate cache volumes; credentials outside both. +- Env ownership: Credentials are injected at run time and never baked into the tool image or caches. +- Readiness: Each run reports which stage ran; a changed lockfile or tool pin rebuilds through the same command. +- Cleanup: Tool images and dependency caches are retained; prune only explicitly disposable caches. +- Provenance: 3 consumers (HACK-P10 research register). + + +> Negative control: Code that has landed is not an enabled, verified deployment, and a cache hit does not prove the installer was skipped. + + +## HACK-P11: Stable agent guidance and typed operation results + +- Use when: Agents run project workflows and need versioned guidance plus machine-readable results. +- Disposition: Partial. Missing: A guidance freeze during execution and a typed terminal result for resumed background operations remain open. +- Owner: WU19/WU21; tracked in HACK-1155. +- Commands: `hack up --json`, `hack down --json`, `hack restart --json`, `hack doctor --json`, `hack run -- ` +- Prerequisites: Generated guidance from Hack's canonical source with a content revision; `--json` on lifecycle and diagnostic commands. +- Env ownership: Results report names and states, not values. +- Readiness: Pass child argv after `--` in `hack run -- `. Check the exit status or `--json` envelope directly rather than through a pipe filter. +- Cleanup: Guidance updates are explicit `hack setup` operations, never side effects of ordinary commands. +- Provenance: 2 consumers (HACK-P11 research register). + + +> Negative control: Filtered shell text can hide a failed child, and historical dispatch, PR or gateway requests must not silently expand current product scope. + + +## HACK-P12: Multi-lane admission and durable run provenance + +- Use when: Several heavy runs compete for scarce resources and need queued admission with durable ownership. +- Disposition: Deferred. Missing: Hack has no multi-lane admission, lease or durable run-provenance contract. +- Owner: WU13/WU04; tracked in HACK-1156. +- Commands: none yet; do not substitute a hand-written recipe. +- Prerequisites: Separate job admission, scarce-resource leases and project-registry operations. +- Env ownership: Admission and provenance records hold no env values. +- Readiness: Report queue time separately from execution time with the exact workload. +- Cleanup: Cancellation and worker restart must not duplicate ownership or lose a continuation. +- Provenance: 2 consumers (HACK-P12 research register). + + +> Negative control: A global registry lock is not a workload scheduler, and idle host, idle lane and idle application are different signals. + + +--- + +Content revision: `091b4e80a9b6`. Generated by Hack; edit the canonical source and regenerate the plugin. diff --git a/plugins/hack/skills/hack-install/SKILL.md b/plugins/hack/skills/hack-install/SKILL.md index c9da3dcd..02f401b7 100644 --- a/plugins/hack/skills/hack-install/SKILL.md +++ b/plugins/hack/skills/hack-install/SKILL.md @@ -25,8 +25,8 @@ On a Homebrew host, use the official formula `hack-dance/tap/hack`. Inspect `brew install hack-dance/tap/hack` or `brew upgrade hack-dance/tap/hack` as appropriate. Run Homebrew as the user, never with sudo. -For another supported host, read the installation instructions at -https://github.com/hack-dance/hack and inspect the official release's platform +For another supported host, read the installation instructions in the Hack +repository (see Further reading) and inspect the official release's platform assets. Pin a release for reproducible setup, save and review its installer, and follow its documented verification. Do not guess asset names or run an unreviewed download through a shell. Managed containers use the slim path in @@ -34,7 +34,7 @@ unreviewed download through a shell. Managed containers use the slim path in ## Native prerelease alongside stable -Read https://github.com/hack-dance/hack/blob/next/docs/guides/candidate-install.md +Read the candidate installation guide (see Further reading) and verify the selected release exists. The current packaged native installer supports Apple Silicon macOS, not arbitrary Linux cloud hosts. Use an explicit published `5.0.0-next.N` tag and its reviewed `scripts/install-prerelease.py`. @@ -79,3 +79,8 @@ Report the executable, version, channel and target. Inspect project status and an actual app request, and an appropriate log check. A successful install or MCP listing alone does not prove app readiness. Keep local HTTPS trust prompts native; browser access is a separate check from a successful CLI HTTPS request. + +## Further reading + +- [Hack repository and releases](https://github.com/hack-dance/hack) +- [Candidate installation guide](https://github.com/hack-dance/hack/blob/next/docs/guides/candidate-install.md) diff --git a/plugins/hack/skills/hack-remote/SKILL.md b/plugins/hack/skills/hack-remote/SKILL.md index 2f8dd3da..531d4d0b 100644 --- a/plugins/hack/skills/hack-remote/SKILL.md +++ b/plugins/hack/skills/hack-remote/SKILL.md @@ -30,9 +30,9 @@ Use `hack_cli` and `project_root` below as local shell variables containing thos Request `HACK_ENV_SECRET_KEY` through the target's approved secret manager or direct process injection. The user enters the value in its native secret UI, never in chat. Hack needs the real key bytes locally to decrypt the committed env; an HTTPS proxy placeholder cannot replace them. Do not print the key, pass it as a command argument, write it into a plugin, image, shell profile, or cached file, or export decrypted values for later reuse. Use non-production credentials and review the chosen command's own logging. -Current Codex Cloud uses an Install script and Start skill. Request the key as a direct environment variable, preferably a personal value scoped to the chosen environment; it is available to the task. Network secrets substitute HTTPS credentials and cannot decrypt Hack env. Keep prepared files free of keys and plaintext secrets. After approved publication, verify a fresh task. See [current Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environments). +Current Codex Cloud uses an Install script and Start skill. Request the key as a direct environment variable, preferably a personal value scoped to the chosen environment; it is available to the task. Network secrets substitute HTTPS credentials and cannot decrypt Hack env. Keep prepared files free of keys and plaintext secrets. After approved publication, verify a fresh task. See current Cloud environments under Further reading. -Legacy Codex Cloud secrets are setup-only and removed before the agent phase; a setup export does not persist. Do not bypass that boundary by saving the key or decrypted env in cached files. Runtime decryption needs a separately authorized injection mechanism, or the task must use a secret-free workflow. See [legacy Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environment). +Legacy Codex Cloud secrets are setup-only and removed before the agent phase; a setup export does not persist. Do not bypass that boundary by saving the key or decrypted env in cached files. Runtime decryption needs a separately authorized injection mechanism, or the task must use a secret-free workflow. See legacy Cloud environments under Further reading. Cloud tasks must have this skill in their repository or supported cloud environment; a personal plugin installed on a local computer is not proof it is loaded remotely. Confirm the available skills in the target task. @@ -55,3 +55,8 @@ When the selected runtime is ready and the project's lifecycle hooks have been r ``` Check an actual application endpoint or repository health command with a bounded timeout, then perform one meaningful development operation. Follow the project's approved network access and authentication flow; do not bypass provider restrictions. Report the CLI version/path, target, workflow, env overlay name, readiness result, and remaining blocker without secrets. Installation, a listed MCP server, or a running container alone does not prove the app works. Stop only resources created and owned by this task when cleanup is requested; preserve reusable caches and application data. + +## Further reading + +- [Current Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environments) +- [Legacy Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environment) diff --git a/scripts/generate-agent-plugins.ts b/scripts/generate-agent-plugins.ts index a0fa434e..1a8e8520 100644 --- a/scripts/generate-agent-plugins.ts +++ b/scripts/generate-agent-plugins.ts @@ -1,3 +1,4 @@ +import { readdir, rm } from "node:fs/promises"; import { dirname, resolve } from "node:path"; import { renderCodexSkill } from "../src/agents/codex-skill.ts"; @@ -5,6 +6,13 @@ import { renderCursorRules } from "../src/agents/cursor.ts"; import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; +import { + type PluginSkillName, + referencesForSkill, + renderSkillReference, + skillReferencePath, + withSkillReferenceLinks, +} from "../src/agents/skill-references.ts"; import { isRecord } from "../src/lib/guards.ts"; export async function generateAgentPlugins({ @@ -19,7 +27,7 @@ export async function generateAgentPlugins({ const generatedSkills = [ { path: resolve(pluginRoot, "skills", "hack-cli", "SKILL.md"), - content: renderCodexSkill(), + content: renderPluginSkill("hack-cli"), }, { path: resolve(pluginRoot, "skills", "hack-init", "SKILL.md"), @@ -43,6 +51,7 @@ export async function generateAgentPlugins({ await Bun.$`mkdir -p ${dirname(skill.path)}`; await Bun.write(skill.path, skill.content); } + await writeSkillReferences({ pluginRoot }); const manifestPaths = [ resolve(pluginRoot, ".codex-plugin", "plugin.json"), @@ -137,6 +146,66 @@ export function portableMcpManifest( }; } +/** + * Render a plugin skill's SKILL.md. The plugin variant of `hack-cli` adds + * links to its packaged references; standalone installs keep + * {@link renderCodexSkill} because they ship no references. + */ +export function renderPluginSkill(skill: "hack-cli"): string { + return withSkillReferenceLinks({ skill, content: renderCodexSkill() }); +} + +const PLUGIN_SKILLS: readonly PluginSkillName[] = [ + "hack-cli", + "hack-init", + "hack-install", + "hack-remote", +]; + +/** Every packaged skill reference keyed by its plugin-relative path. */ +export function renderPluginReferences(): ReadonlyMap { + return new Map( + PLUGIN_SKILLS.flatMap((skill) => + referencesForSkill(skill).map( + (reference) => + [ + skillReferencePath(reference), + renderSkillReference(reference), + ] as const + ) + ) + ); +} + +/** + * Write every registered reference and remove stale generated ones, so each + * bundled `references/` directory holds exactly the registry's markdown files. + */ +async function writeSkillReferences({ + pluginRoot, +}: { + readonly pluginRoot: string; +}): Promise { + const references = renderPluginReferences(); + for (const skill of PLUGIN_SKILLS) { + const directory = resolve(pluginRoot, "skills", skill, "references"); + const existing = await readdir(directory).catch((): string[] => []); + for (const name of existing) { + if ( + name.endsWith(".md") && + !references.has(`skills/${skill}/references/${name}`) + ) { + await rm(resolve(directory, name)); + } + } + } + for (const [relativePath, content] of references) { + const path = resolve(pluginRoot, relativePath); + await Bun.$`mkdir -p ${dirname(path)}`; + await Bun.write(path, content); + } +} + export function renderPluginRule(): string { return [ "---", diff --git a/src/agents/install-skill.ts b/src/agents/install-skill.ts index 9717ba9f..0ea2c097 100644 --- a/src/agents/install-skill.ts +++ b/src/agents/install-skill.ts @@ -27,8 +27,8 @@ On a Homebrew host, use the official formula \`hack-dance/tap/hack\`. Inspect \`brew install hack-dance/tap/hack\` or \`brew upgrade hack-dance/tap/hack\` as appropriate. Run Homebrew as the user, never with sudo. -For another supported host, read the installation instructions at -https://github.com/hack-dance/hack and inspect the official release's platform +For another supported host, read the installation instructions in the Hack +repository (see Further reading) and inspect the official release's platform assets. Pin a release for reproducible setup, save and review its installer, and follow its documented verification. Do not guess asset names or run an unreviewed download through a shell. Managed containers use the slim path in @@ -36,7 +36,7 @@ unreviewed download through a shell. Managed containers use the slim path in ## Native prerelease alongside stable -Read https://github.com/hack-dance/hack/blob/next/docs/guides/candidate-install.md +Read the candidate installation guide (see Further reading) and verify the selected release exists. The current packaged native installer supports Apple Silicon macOS, not arbitrary Linux cloud hosts. Use an explicit published \`5.0.0-next.N\` tag and its reviewed \`scripts/install-prerelease.py\`. @@ -81,5 +81,10 @@ Report the executable, version, channel and target. Inspect project status and an actual app request, and an appropriate log check. A successful install or MCP listing alone does not prove app readiness. Keep local HTTPS trust prompts native; browser access is a separate check from a successful CLI HTTPS request. + +## Further reading + +- [Hack repository and releases](https://github.com/hack-dance/hack) +- [Candidate installation guide](https://github.com/hack-dance/hack/blob/next/docs/guides/candidate-install.md) `; } diff --git a/src/agents/pattern-catalogue.ts b/src/agents/pattern-catalogue.ts new file mode 100644 index 00000000..82e36565 --- /dev/null +++ b/src/agents/pattern-catalogue.ts @@ -0,0 +1,423 @@ +/** + * Typed register of the twelve reusable project patterns (HACK-P01..P12) + * found across Hack consumers, with each card's current disposition. + * + * Invariants (enforced by tests/pattern-catalogue.test.ts): + * - All twelve ids are present exactly once; the register stays complete even + * for recipes Hack cannot support yet. + * - A card that is not `supported` names its domain owner and the missing + * contract. Such a disposition never waives the owner's runtime gates. + * - Every `hack` invocation a card names resolves in CLI_SPEC on this branch. + * - Unsafe recipes (broad process kills, kill-by-port, placeholder liveness, + * secrets in argv) appear only inside negative-example blocks. + * + * Consumer names stay in the Linear issues; the shipped reference records + * only how many consumers a card generalizes from. + */ + +/** How far Hack supports a pattern today. */ +export type PatternDisposition = + | "supported" + | "partial" + | "retain-as-example" + | "deferred" + | "rejected"; + +/** Where a card's generalization comes from. */ +export type PatternProvenance = { + /** Number of distinct consumer projects that exhibit the pattern. */ + readonly consumers: number; + /** Research register entry; consumer detail lives in the Linear issue. */ + readonly source: string; +}; + +export type PatternCard = { + /** Stable research id, `HACK-P01`..`HACK-P12`. */ + readonly id: string; + /** Linear issue that tracks the card. */ + readonly linearId: string; + readonly title: string; + readonly useWhen: string; + readonly disposition: PatternDisposition; + /** + * `hack` invocations that exist on this CLI and apply to the pattern. For a + * card that is not `supported`, these cover only the part Hack owns today. + */ + readonly supportedCommands: readonly string[]; + readonly prerequisites: string; + readonly envOwnership: string; + readonly readiness: string; + readonly cleanup: string; + /** What looks like success but is not; rendered as a negative example. */ + readonly negativeControl: string; + /** Work unit(s) that own the runtime mechanism. */ + readonly domainOwner: string; + /** Contract Hack still lacks; `null` only for `supported` cards. */ + readonly missingContract: string | null; + readonly provenance: PatternProvenance; +}; + +/** The complete pattern register, in id order. */ +export const PATTERN_CATALOGUE: readonly PatternCard[] = [ + { + id: "HACK-P01", + linearId: "HACK-1145", + title: "One-shot dependency and generated-artifact bootstrap", + useWhen: + "Services need installed dependencies or generated artifacts before the application starts.", + disposition: "partial", + supportedCommands: [ + "hack up --json", + "hack ps --json", + "hack logs --no-follow --tail 100", + ], + prerequisites: + "A Compose initializer service that consumers require through `service_completed_successfully`; for the opt-in `hack.dependencies.cache-protocol: locked-v1` label, a pinned installer image and an immutable dependency payload.", + envOwnership: + "locked-v1 runs the initializer and verifier with a clean environment and refuses explicit `environment`/`env_file`; credential- or env-dependent installers keep their existing setup.", + readiness: + "Installation, generated-artifact publication and application readiness are separate steps. Completion is the initializer's successful exit plus its verifier, not a populated cache.", + cleanup: + "Ordinary `hack down` keeps dependency volumes. Keep worktree-specific generated output in its own always-run step and directory.", + negativeControl: + "A populated cache directory is not a completion marker. Concurrent writers to one volume and failed publication need ownership and recovery; consumers must see a complete old or complete new artifact.", + domainOwner: "WU07/WU11", + missingContract: + "locked-v1 covers one immutable payload per volume. Atomic publication of worktree-specific generated artifacts, env-dependent installers, and two-worktree or interrupted-writer qualification remain open.", + provenance: { consumers: 4, source: "HACK-P01 research register" }, + }, + { + id: "HACK-P02", + linearId: "HACK-1146", + title: "Portable toolchain without an always-on application", + useWhen: + "Checks, builds or tests need pinned tools in a container, but the project has no long-running app to start.", + disposition: "partial", + supportedCommands: ["hack run --profile -- "], + prerequisites: + "A one-shot Compose service behind an opt-in profile with pinned tool versions, explicit resource bounds and separate named volumes for reusable build caches.", + envOwnership: + "Mount no Docker socket, user home, SSH agent or credential directory; keep project secrets out of the image build context.", + readiness: + "The task's own exit status is the result. Check the `hack run` exit status directly, not through a pipe filter, so a failed check stays a failure.", + cleanup: + "The service is one-shot. Caches live in named volumes that `hack down` preserves; a source-identity override used for tests must not reach release evidence.", + negativeControl: + "A test-only source identity override must not escape into release evidence, and a linked worktree's Git pointer needs deliberate handling rather than an assumed `.git` directory.", + domainOwner: "WU11", + missingContract: + "Cross-worktree cache reuse with bounded eviction, a source receipt for linked worktrees, and owned cache accounting remain open.", + provenance: { consumers: 4, source: "HACK-P02 research register" }, + }, + { + id: "HACK-P03", + linearId: "HACK-1147", + title: "Cloud prerequisite preflight", + useWhen: + "Startup depends on cloud credentials or tokens that may be missing, expired or scoped wrongly.", + disposition: "partial", + supportedCommands: [ + "hack doctor --json", + "hack env explain --json", + "hack env explain --target compose --json", + ], + prerequisites: + "Required key names declared in Hack env; the credential tool and its native sign-in owned by the user.", + envOwnership: + "Report key names, winning source and repair direction, never values. Check the container view separately from the host view.", + readiness: + "Preflight runs before startup and fails with the missing name; a present value is not proof that it is valid or unexpired.", + cleanup: "Preflight starts nothing and leaves no state.", + negativeControl: + "A token on the host does not prove the container receives it, and a headless runner cannot depend on interactive browser sign-in.", + domainOwner: "WU27", + missingContract: + "Hack has no credential availability, expiry or scope check and no machine-identity path for headless runners; the WU27 env contract owns both.", + provenance: { consumers: 3, source: "HACK-P03 research register" }, + }, + { + id: "HACK-P04", + linearId: "HACK-1148", + title: "Shared host service with identity-aware adoption", + useWhen: + "Several projects or branches share a host helper such as a session issuer or local proxy.", + disposition: "retain-as-example", + supportedCommands: ["hack up --json", "hack logs --no-follow --tail 100"], + prerequisites: + 'A lifecycle process with `singleton` ports. Hack adopts an existing helper only when every configured port is occupied and the config opts into `onConflict: "adopt"`.', + envOwnership: + "The helper's credentials stay with its owner; adoption must not forward broad host env.", + readiness: + "Pair adoption with a `lifecycle.up.before` hook that checks the listener's identity and expected upstream and fails on mismatch.", + cleanup: + "Hack stops only processes it started; an adopted helper is not Hack-owned and is never stopped by Hack.", + negativeControl: + "Port-only reuse can attach to a foreign service. Process liveness or a placeholder process is not readiness, and no foreign process may be killed to free a port.", + domainOwner: "WU29", + missingContract: + "Today's adoption is port-only. Health- and upstream-checked adoption with a recorded adopted-versus-owned state is the WU29 lifecycle ownership contract.", + provenance: { consumers: 1, source: "HACK-P04 research register" }, + }, + { + id: "HACK-P05", + linearId: "HACK-1149", + title: "Branch-owned external routing claim", + useWhen: + "A branch needs an external route or callback claim tied to its own instance.", + disposition: "deferred", + supportedCommands: [], + prerequisites: + "Explicit project and branch identity for the claim; no recipe is published until the contract exists.", + envOwnership: + "Claim credentials stay host-scoped and are not written into tracked files.", + readiness: + "A claim must be a bounded, observable operation with a retry budget.", + cleanup: "Release only the claim and process group that this branch owns.", + negativeControl: + "The observed consumer uses a fixed default URL, unbounded retry and broad name-based cleanup. Retain it as a negative example, not a copyable recipe.", + domainOwner: "WU29", + missingContract: + "Hack has no branch-scoped external claim/release contract with bounded retry, stop-during-retry and PID-reuse safety.", + provenance: { consumers: 1, source: "HACK-P05 research register" }, + }, + { + id: "HACK-P06", + linearId: "HACK-1150", + title: "Opt-in public tunnel with end-to-end readiness", + useWhen: + "A developer occasionally needs the local application reachable from the public internet.", + disposition: "deferred", + supportedCommands: [], + prerequisites: + "Default off; host-scoped tunnel credentials, the tunnel tool and the intended CA.", + envOwnership: + "Pass only the tunnel's own credential; never forward broad host env.", + readiness: + "Probe the exposed application route end to end; a missing credential, tool or CA fails clearly.", + cleanup: + "Shutdown leaves no owned tunnel child; disabled mode starts no tunnel.", + negativeControl: + "A running tunnel process or open TCP listener alone does not prove the external route reaches the right upstream.", + domainOwner: "WU29", + missingContract: + "Hack has no opt-in tunnel lifecycle with external-route readiness and owned-child shutdown.", + provenance: { consumers: 1, source: "HACK-P06 research register" }, + }, + { + id: "HACK-P07", + linearId: "HACK-1151", + title: "Laptop and headless-runner capability overlays", + useWhen: + "The same application topology runs on a laptop and on a headless runner with different auth, network or helpers.", + disposition: "partial", + supportedCommands: [ + "hack env explain --env --json", + "hack env explain --env --target compose --json", + "hack up --env --json", + ], + prerequisites: + "One shared Compose topology; env overlays that express only the auth, network and helper differences.", + envOwnership: + "Overlays own the differences; inspect the effective value source per target instead of reading host env.", + readiness: + "Verify container metadata and dependency reachability on each target; the runner must not need browser sign-in.", + cleanup: + "Overlays add no state of their own; inherited worktree overrides must be visible and deliberate.", + negativeControl: + "Do not change locality flags just to bypass a readiness check. Host env is not evidence of container delivery.", + domainOwner: "WU27/WU30", + missingContract: + "Hack has env overlays and value-free inspection but no laptop/headless capability contract or headless machine identity; WU27 and WU30 own them.", + provenance: { consumers: 1, source: "HACK-P07 research register" }, + }, + { + id: "HACK-P08", + linearId: "HACK-1152", + title: "Tracked defaults plus idempotent machine-local hydration", + useWhen: + "A repository tracks env defaults and each machine adds local values on top.", + disposition: "deferred", + supportedCommands: [], + prerequisites: + "Tracked defaults, a machine-local layer and preserved operator overrides.", + envOwnership: + "Defaults are tracked; hydration writes only the local layer and keeps values out of argv.", + readiness: + "Repeat hydration leaves the tracked tree clean; an interrupted import still leaves a resolvable layer.", + cleanup: "Unchanged values cause no rewrite.", + negativeControl: + "Re-encrypting unchanged entries causes diff and stash churn, and values passed in command arguments are not a secure import mechanism.", + domainOwner: "WU27/WU14", + missingContract: + "Hack has no semantic no-op detection or atomic machine-local import contract for env hydration.", + provenance: { consumers: 1, source: "HACK-P08 research register" }, + }, + { + id: "HACK-P09", + linearId: "HACK-1153", + title: "Disposable run workspaces with reference-aware retention", + useWhen: + "Runs or branches create workspaces and caches that should be reclaimed once nothing uses them.", + disposition: "partial", + supportedCommands: [ + "hack down --prune-caches", + "hack branch remove ", + "hack projects prune --dry-run", + ], + prerequisites: + 'Explicitly disposable cache volumes (`hack.cache.disposable: "true"` or Next.js build output) and a reviewed dry run before pruning.', + envOwnership: "Reclamation reads no env values.", + readiness: "Not applicable; retention is a cleanup contract.", + cleanup: + "`--prune-caches` is confirmation-gated and removes only exact, project-owned disposable volumes; sessions and application data keep their own retention.", + negativeControl: + "Stopped or Docker-reclaimable does not mean disposable. Active, pending, held or unknown resources must survive.", + domainOwner: "WU12/WU30", + missingContract: + "Generation/run ownership, active references, holds and interrupted-reclamation recovery are not implemented for run workspaces.", + provenance: { consumers: 1, source: "HACK-P09 research register" }, + }, + { + id: "HACK-P10", + linearId: "HACK-1154", + title: "Pinned tool image plus independent dependency caches", + useWhen: + "A runner or toolchain image must stay pinned while application dependencies and credentials vary.", + disposition: "partial", + supportedCommands: [ + "hack run --profile -- ", + "hack up --json", + ], + prerequisites: + "Tool manifests pinned in the image; application dependencies in separate cache volumes; credentials outside both.", + envOwnership: + "Credentials are injected at run time and never baked into the tool image or caches.", + readiness: + "Each run reports which stage ran; a changed lockfile or tool pin rebuilds through the same command.", + cleanup: + "Tool images and dependency caches are retained; prune only explicitly disposable caches.", + negativeControl: + "Code that has landed is not an enabled, verified deployment, and a cache hit does not prove the installer was skipped.", + domainOwner: "WU30", + missingContract: + "Hack does not yet distinguish L1, L2 and cold misses, detect a tampered cache, or emit stage timing and provenance receipts.", + provenance: { consumers: 3, source: "HACK-P10 research register" }, + }, + { + id: "HACK-P11", + linearId: "HACK-1155", + title: "Stable agent guidance and typed operation results", + useWhen: + "Agents run project workflows and need versioned guidance plus machine-readable results.", + disposition: "partial", + supportedCommands: [ + "hack up --json", + "hack down --json", + "hack restart --json", + "hack doctor --json", + "hack run -- ", + ], + prerequisites: + "Generated guidance from Hack's canonical source with a content revision; `--json` on lifecycle and diagnostic commands.", + envOwnership: "Results report names and states, not values.", + readiness: + "Pass child argv after `--` in `hack run -- `. Check the exit status or `--json` envelope directly rather than through a pipe filter.", + cleanup: + "Guidance updates are explicit `hack setup` operations, never side effects of ordinary commands.", + negativeControl: + "Filtered shell text can hide a failed child, and historical dispatch, PR or gateway requests must not silently expand current product scope.", + domainOwner: "WU19/WU21", + missingContract: + "A guidance freeze during execution and a typed terminal result for resumed background operations remain open.", + provenance: { consumers: 2, source: "HACK-P11 research register" }, + }, + { + id: "HACK-P12", + linearId: "HACK-1156", + title: "Multi-lane admission and durable run provenance", + useWhen: + "Several heavy runs compete for scarce resources and need queued admission with durable ownership.", + disposition: "deferred", + supportedCommands: [], + prerequisites: + "Separate job admission, scarce-resource leases and project-registry operations.", + envOwnership: "Admission and provenance records hold no env values.", + readiness: + "Report queue time separately from execution time with the exact workload.", + cleanup: + "Cancellation and worker restart must not duplicate ownership or lose a continuation.", + negativeControl: + "A global registry lock is not a workload scheduler, and idle host, idle lane and idle application are different signals.", + domainOwner: "WU13/WU04", + missingContract: + "Hack has no multi-lane admission, lease or durable run-provenance contract.", + provenance: { consumers: 2, source: "HACK-P12 research register" }, + }, +]; + +/** Markers that delimit negative examples in the rendered catalogue. */ +export const NEGATIVE_EXAMPLE_START = ""; +export const NEGATIVE_EXAMPLE_END = ""; + +const DISPOSITION_LABELS: Readonly> = { + supported: "Supported", + partial: "Partial", + "retain-as-example": "Retained as example", + deferred: "Deferred", + rejected: "Rejected", +}; + +/** Render the catalogue as the packaged `hack-cli` reference. */ +export function renderPatternCatalogue(): string { + const lines = [ + "# Hack pattern catalogue", + "", + "Reusable patterns observed across Hack projects, with what Hack supports today.", + "Use a card's commands only within its disposition. A partial, retained, deferred", + "or rejected card names the missing contract and its owner; do not improvise the", + "missing part with ad hoc scripts.", + "", + "| Id | Pattern | Disposition | Owner |", + "| --- | --- | --- | --- |", + ...PATTERN_CATALOGUE.map( + (card) => + `| ${card.id} | ${card.title} | ${DISPOSITION_LABELS[card.disposition]} | ${card.domainOwner} |` + ), + "", + ...PATTERN_CATALOGUE.flatMap((card) => renderCard(card)), + ]; + return `${lines.join("\n").trimEnd()}\n`; +} + +function renderCard(card: PatternCard): string[] { + const commands = + card.supportedCommands.length === 0 + ? "none yet; do not substitute a hand-written recipe." + : card.supportedCommands.map((command) => `\`${command}\``).join(", "); + const consumers = + card.provenance.consumers === 1 + ? "single-consumer generalization" + : `${card.provenance.consumers} consumers`; + const disposition = + card.missingContract === null + ? DISPOSITION_LABELS[card.disposition] + : `${DISPOSITION_LABELS[card.disposition]}. Missing: ${card.missingContract}`; + return [ + `## ${card.id}: ${card.title}`, + "", + `- Use when: ${card.useWhen}`, + `- Disposition: ${disposition}`, + `- Owner: ${card.domainOwner}; tracked in ${card.linearId}.`, + `- Commands: ${commands}`, + `- Prerequisites: ${card.prerequisites}`, + `- Env ownership: ${card.envOwnership}`, + `- Readiness: ${card.readiness}`, + `- Cleanup: ${card.cleanup}`, + `- Provenance: ${consumers} (${card.provenance.source}).`, + "", + NEGATIVE_EXAMPLE_START, + `> Negative control: ${card.negativeControl}`, + NEGATIVE_EXAMPLE_END, + "", + ]; +} diff --git a/src/agents/remote-bootstrap-skill.ts b/src/agents/remote-bootstrap-skill.ts index cf927428..0d93db79 100644 --- a/src/agents/remote-bootstrap-skill.ts +++ b/src/agents/remote-bootstrap-skill.ts @@ -37,9 +37,9 @@ export function renderHackRemoteSkill(): string { "", "Request `HACK_ENV_SECRET_KEY` through the target's approved secret manager or direct process injection. The user enters the value in its native secret UI, never in chat. Hack needs the real key bytes locally to decrypt the committed env; an HTTPS proxy placeholder cannot replace them. Do not print the key, pass it as a command argument, write it into a plugin, image, shell profile, or cached file, or export decrypted values for later reuse. Use non-production credentials and review the chosen command's own logging.", "", - "Current Codex Cloud uses an Install script and Start skill. Request the key as a direct environment variable, preferably a personal value scoped to the chosen environment; it is available to the task. Network secrets substitute HTTPS credentials and cannot decrypt Hack env. Keep prepared files free of keys and plaintext secrets. After approved publication, verify a fresh task. See [current Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environments).", + "Current Codex Cloud uses an Install script and Start skill. Request the key as a direct environment variable, preferably a personal value scoped to the chosen environment; it is available to the task. Network secrets substitute HTTPS credentials and cannot decrypt Hack env. Keep prepared files free of keys and plaintext secrets. After approved publication, verify a fresh task. See current Cloud environments under Further reading.", "", - "Legacy Codex Cloud secrets are setup-only and removed before the agent phase; a setup export does not persist. Do not bypass that boundary by saving the key or decrypted env in cached files. Runtime decryption needs a separately authorized injection mechanism, or the task must use a secret-free workflow. See [legacy Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environment).", + "Legacy Codex Cloud secrets are setup-only and removed before the agent phase; a setup export does not persist. Do not bypass that boundary by saving the key or decrypted env in cached files. Runtime decryption needs a separately authorized injection mechanism, or the task must use a secret-free workflow. See legacy Cloud environments under Further reading.", "", "Cloud tasks must have this skill in their repository or supported cloud environment; a personal plugin installed on a local computer is not proof it is loaded remotely. Confirm the available skills in the target task.", "", @@ -63,5 +63,10 @@ export function renderHackRemoteSkill(): string { "", "Check an actual application endpoint or repository health command with a bounded timeout, then perform one meaningful development operation. Follow the project's approved network access and authentication flow; do not bypass provider restrictions. Report the CLI version/path, target, workflow, env overlay name, readiness result, and remaining blocker without secrets. Installation, a listed MCP server, or a running container alone does not prove the app works. Stop only resources created and owned by this task when cleanup is requested; preserve reusable caches and application data.", "", + "## Further reading", + "", + "- [Current Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environments)", + "- [Legacy Cloud environments](https://learn.chatgpt.com/docs/environments/cloud-environment)", + "", ].join("\n"); } diff --git a/src/agents/skill-references.ts b/src/agents/skill-references.ts new file mode 100644 index 00000000..57792bea --- /dev/null +++ b/src/agents/skill-references.ts @@ -0,0 +1,98 @@ +import { createHash } from "node:crypto"; + +import { renderPatternCatalogue } from "./pattern-catalogue.ts"; + +/** Plugin skills that can carry packaged references. */ +export type PluginSkillName = + | "hack-cli" + | "hack-init" + | "hack-install" + | "hack-remote"; + +/** + * A markdown reference shipped inside the plugin package at + * `skills//references/`. + * + * References are offline-first: they resolve inside the installed package, + * link only to files inside it, and keep external URLs under a + * "Further reading" heading. Each one is linked from its skill's SKILL.md. + */ +export type SkillReference = { + readonly skill: PluginSkillName; + /** Lowercase kebab-case markdown file name, e.g. `patterns.md`. */ + readonly file: string; + /** Link text used in the skill's References section. */ + readonly title: string; + /** One-line description shown next to the link. */ + readonly summary: string; + /** Canonical body; the packaged file adds a revision footer. */ + readonly render: () => string; +}; + +/** Every packaged skill reference, in generation order. */ +export const SKILL_REFERENCES: readonly SkillReference[] = [ + { + skill: "hack-cli", + file: "patterns.md", + title: "Pattern catalogue", + summary: + "reusable project patterns with what Hack supports today, the missing contract and its owner.", + render: renderPatternCatalogue, + }, +]; + +const REFERENCE_FILE_PATTERN = /^[a-z0-9][a-z0-9-]*\.md$/; + +/** References that belong to one skill. */ +export function referencesForSkill( + skill: PluginSkillName +): readonly SkillReference[] { + return SKILL_REFERENCES.filter((reference) => reference.skill === skill); +} + +/** Path of a reference relative to the plugin root. */ +export function skillReferencePath(reference: SkillReference): string { + assertReferenceFile(reference.file); + return `skills/${reference.skill}/references/${reference.file}`; +} + +/** First 12 hex characters of the SHA-256 of a reference body. */ +export function skillReferenceRevision(body: string): string { + return createHash("sha256").update(body).digest("hex").slice(0, 12); +} + +/** + * Render the packaged reference file: the canonical body plus a content + * revision that a reader (or test) can recompute from the body above it. + */ +export function renderSkillReference(reference: SkillReference): string { + assertReferenceFile(reference.file); + const body = reference.render(); + return `${body.trimEnd()}\n\n---\n\nContent revision: \`${skillReferenceRevision(body)}\`. Generated by Hack; edit the canonical source and regenerate the plugin.\n`; +} + +/** + * Append a References section that links each of the skill's packaged + * references. Skills without references are returned unchanged, so the + * standalone installs (which ship no references) never gain a dangling link. + */ +export function withSkillReferenceLinks(opts: { + readonly skill: PluginSkillName; + readonly content: string; +}): string { + const references = referencesForSkill(opts.skill); + if (references.length === 0) { + return opts.content; + } + const links = references.map((reference) => { + assertReferenceFile(reference.file); + return `- [${reference.title}](references/${reference.file}): ${reference.summary}`; + }); + return `${opts.content.trimEnd()}\n\n## References\n\n${links.join("\n")}\n`; +} + +function assertReferenceFile(file: string): void { + if (!REFERENCE_FILE_PATTERN.test(file)) { + throw new Error("Invalid skill reference file name"); + } +} diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 858a3c90..891b7036 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -7,9 +7,10 @@ import { generateAgentPlugins, portableMcpManifest, portablePluginManifest, + renderPluginReferences, renderPluginRule, + renderPluginSkill, } from "../scripts/generate-agent-plugins.ts"; -import { renderCodexSkill } from "../src/agents/codex-skill.ts"; import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; @@ -84,8 +85,13 @@ test("semantic-release commits all generated client manifests", async () => { test("bundled skills and rule match current canonical guidance", async () => { const root = join(repoRoot, "plugins", "hack"); expect(await Bun.file(join(root, "skills/hack-cli/SKILL.md")).text()).toBe( - renderCodexSkill() + renderPluginSkill("hack-cli") ); + const references = renderPluginReferences(); + expect(references.size).toBeGreaterThan(0); + for (const [path, content] of references) { + expect(await Bun.file(join(root, path)).text()).toBe(content); + } expect(await Bun.file(join(root, "skills/hack-init/SKILL.md")).text()).toBe( renderHackInitSkill() ); diff --git a/tests/helpers/skill-links.ts b/tests/helpers/skill-links.ts new file mode 100644 index 00000000..fcd41f66 --- /dev/null +++ b/tests/helpers/skill-links.ts @@ -0,0 +1,172 @@ +import { existsSync } from "node:fs"; +import { dirname, isAbsolute, relative, resolve, sep } from "node:path"; + +/** + * Offline link contract for the packaged plugin skills. + * + * Every markdown file under `/skills` is checked: + * - relative links resolve to an existing file inside the plugin root (no + * `../` escape into the repository, no `docs/` path that mirrors repository + * docs, no absolute paths); + * - external URLs appear only under a "Further reading" heading, so core + * guidance never depends on the network; + * - every `skills//references/*.md` file is linked from that skill's + * SKILL.md. + * + * Inline code and fenced blocks are literals (commands, schema ids), not + * links, and are ignored. Issues carry closed codes and plugin-relative paths. + */ +export type SkillLinkIssueCode = + | "absolute-link" + | "escapes-plugin" + | "repo-docs-link" + | "missing-target" + | "external-outside-further-reading" + | "unlinked-reference" + | "missing-skill"; + +export type SkillLinkIssue = { + readonly file: string; + readonly code: SkillLinkIssueCode; +}; + +const FENCED_BLOCK = /```[\s\S]*?```/g; +const INLINE_CODE = /`[^`\n]*`/g; +const HEADING = /^#{1,6}\s+(.+?)\s*#*\s*$/; +const MARKDOWN_LINK = /\]\(\s*]+)>?(?:\s+"[^"]*")?\s*\)/g; +const URL = /\b[a-z][a-z0-9+.-]*:\/\/[^\s)>\]]+/gi; +const SCHEME = /^[a-z][a-z0-9+.-]*:/i; +const FURTHER_READING = /^further reading$/i; + +export async function findSkillLinkIssues(opts: { + readonly pluginRoot: string; +}): Promise { + const issues: SkillLinkIssue[] = []; + const files: string[] = []; + for await (const path of new Bun.Glob("skills/**/*.md").scan({ + cwd: opts.pluginRoot, + onlyFiles: true, + })) { + files.push(path); + } + files.sort(); + + const linkedFromSkill = new Map>(); + for (const file of files) { + const absolute = resolve(opts.pluginRoot, file); + const targets = new Set(); + for (const link of scanLinks(await Bun.file(absolute).text())) { + const outcome = classifyLink({ + pluginRoot: opts.pluginRoot, + from: absolute, + ...link, + }); + if (outcome.kind === "issue") { + issues.push({ file, code: outcome.code }); + } else if (outcome.kind === "local") { + targets.add(outcome.path); + } + } + if (file.endsWith("/SKILL.md")) { + linkedFromSkill.set(dirname(absolute), targets); + } + } + + for (const file of files) { + const code = referenceLinkIssue({ + pluginRoot: opts.pluginRoot, + file, + linkedFromSkill, + }); + if (code) { + issues.push({ file, code }); + } + } + return issues; +} + +type LinkOutcome = + | { readonly kind: "ignored" } + | { readonly kind: "local"; readonly path: string } + | { readonly kind: "issue"; readonly code: SkillLinkIssueCode }; + +function classifyLink(opts: { + readonly pluginRoot: string; + readonly from: string; + readonly target: string; + readonly furtherReading: boolean; +}): LinkOutcome { + if (SCHEME.test(opts.target) || opts.target.startsWith("//")) { + return opts.furtherReading + ? { kind: "ignored" } + : { kind: "issue", code: "external-outside-further-reading" }; + } + const path = opts.target.split("#")[0]?.split("?")[0] ?? ""; + if (path === "") { + return { kind: "ignored" }; + } + if (isAbsolute(path) || path.startsWith("~")) { + return { kind: "issue", code: "absolute-link" }; + } + const resolved = resolve(dirname(opts.from), path); + const inside = relative(opts.pluginRoot, resolved); + if (inside === ".." || inside.startsWith(`..${sep}`) || isAbsolute(inside)) { + return { kind: "issue", code: "escapes-plugin" }; + } + if (path.split("/").includes("docs")) { + return { kind: "issue", code: "repo-docs-link" }; + } + if (!existsSync(resolved)) { + return { kind: "issue", code: "missing-target" }; + } + return { kind: "local", path: resolved }; +} + +/** A `skills//references/.md` must be linked from its SKILL.md. */ +function referenceLinkIssue(opts: { + readonly pluginRoot: string; + readonly file: string; + readonly linkedFromSkill: ReadonlyMap>; +}): SkillLinkIssueCode | null { + const parts = opts.file.split("/"); + if (parts.length !== 4 || parts[2] !== "references") { + return null; + } + const links = opts.linkedFromSkill.get( + resolve(opts.pluginRoot, "skills", parts[1] ?? "") + ); + if (!links) { + return "missing-skill"; + } + return links.has(resolve(opts.pluginRoot, opts.file)) + ? null + : "unlinked-reference"; +} + +/** Yield link targets and URLs with whether they sit under "Further reading". */ +function* scanLinks( + text: string +): Generator<{ readonly target: string; readonly furtherReading: boolean }> { + const prose = text + .replace(FENCED_BLOCK, (block) => block.replace(/[^\n]/g, " ")) + .replace(INLINE_CODE, (code) => " ".repeat(code.length)); + let furtherReading = false; + for (const line of prose.split("\n")) { + const heading = HEADING.exec(line); + if (heading) { + furtherReading = FURTHER_READING.test(heading[1] ?? ""); + continue; + } + const seen = new Set(); + for (const match of line.matchAll(MARKDOWN_LINK)) { + const target = match[1] ?? ""; + seen.add(target); + yield { target, furtherReading }; + } + for (const match of line.matchAll(URL)) { + if (!seen.has(match[0])) { + yield { target: match[0], furtherReading }; + } + } + } +} diff --git a/tests/pattern-catalogue.test.ts b/tests/pattern-catalogue.test.ts new file mode 100644 index 00000000..96e6df6b --- /dev/null +++ b/tests/pattern-catalogue.test.ts @@ -0,0 +1,217 @@ +import { expect, test } from "bun:test"; +import { join } from "node:path"; + +import { + NEGATIVE_EXAMPLE_END, + NEGATIVE_EXAMPLE_START, + PATTERN_CATALOGUE, + type PatternCard, + type PatternDisposition, + renderPatternCatalogue, +} from "../src/agents/pattern-catalogue.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { + extractHackInvocations, + findGuidanceCommandIssues, +} from "./helpers/cli-guidance-commands.ts"; + +const EXPECTED_IDS = Array.from( + { length: 12 }, + (_, index) => `HACK-P${String(index + 1).padStart(2, "0")}` +); +const OWNER = /^WU\d+(?:\/WU\d+)*$/; + +/** Closed id problems: missing or duplicated cards fail the register. */ +function catalogueIdIssues(cards: readonly PatternCard[]): string[] { + const ids = cards.map((card) => card.id); + return [ + ...EXPECTED_IDS.filter((id) => !ids.includes(id)).map( + (id) => `missing ${id}` + ), + ...ids + .filter((id, index) => ids.indexOf(id) !== index) + .map((id) => `duplicate ${id}`), + ...ids + .filter((id) => !EXPECTED_IDS.includes(id)) + .map((id) => `unexpected ${id}`), + ]; +} + +/** Recipes that must never appear outside a negative-example block. */ +const BANNED_RECIPES: readonly (readonly [string, RegExp])[] = [ + ["broad-process-kill", /\b(?:pkill|killall)\b/], + ["kill-by-port", /\blsof\b[^\n|]*-t[^\n|]*\|\s*xargs\s+(?:-\S+\s+)*kill\b/], + ["placeholder-liveness-tail", /\btail\s+-f\s+\/dev\/null\b/], + ["placeholder-liveness-sleep", /\bsleep\s+infinity\b/], + [ + "argv-secret-flag", + /--(?:password|passwd|token|secret|api-key|apikey|access-key)(?:=|\s+)\S/i, + ], + [ + "argv-secret-assignment", + /\b[A-Z][A-Z0-9_]*(?:TOKEN|SECRET|PASSWORD|PASSWD|API_KEY)=\S/, + ], + ["argv-env-value", /\bhack\s+env\s+(?:set|add)\s+\S+\s+\S+/], +]; + +function stripNegativeExamples(text: string): string { + let result = ""; + let rest = text; + for (;;) { + const start = rest.indexOf(NEGATIVE_EXAMPLE_START); + if (start === -1) { + return result + rest; + } + const end = rest.indexOf(NEGATIVE_EXAMPLE_END, start); + if (end === -1) { + // An unterminated block exempts nothing. + return result + rest; + } + result += rest.slice(0, start); + rest = rest.slice(end + NEGATIVE_EXAMPLE_END.length); + } +} + +function bannedRecipeFindings(text: string): string[] { + const scanned = stripNegativeExamples(text); + return BANNED_RECIPES.filter(([, pattern]) => pattern.test(scanned)).map( + ([code]) => code + ); +} + +test("the catalogue holds exactly the twelve HACK-P cards in order", () => { + expect(catalogueIdIssues(PATTERN_CATALOGUE)).toEqual([]); + expect(PATTERN_CATALOGUE.map((card) => card.id)).toEqual(EXPECTED_IDS); + expect(PATTERN_CATALOGUE.map((card) => card.linearId)).toEqual( + Array.from({ length: 12 }, (_, index) => `HACK-${1145 + index}`) + ); +}); + +test("negative control: deleting or duplicating a card fails the id check", () => { + expect(catalogueIdIssues(PATTERN_CATALOGUE.slice(1))).toEqual([ + "missing HACK-P01", + ]); + const first = PATTERN_CATALOGUE[0]; + if (!first) { + throw new Error("empty catalogue"); + } + expect(catalogueIdIssues([...PATTERN_CATALOGUE, first])).toEqual([ + "duplicate HACK-P01", + ]); +}); + +test("starting dispositions match the recorded decisions", () => { + const expected: Readonly> = { + "HACK-P01": "partial", + "HACK-P02": "partial", + "HACK-P03": "partial", + "HACK-P04": "retain-as-example", + "HACK-P05": "deferred", + "HACK-P06": "deferred", + "HACK-P07": "partial", + "HACK-P08": "deferred", + "HACK-P09": "partial", + "HACK-P10": "partial", + "HACK-P11": "partial", + "HACK-P12": "deferred", + }; + expect( + Object.fromEntries( + PATTERN_CATALOGUE.map((card) => [card.id, card.disposition]) + ) + ).toEqual(expected); + expect( + PATTERN_CATALOGUE.find((card) => card.id === "HACK-P04")?.domainOwner + ).toBe("WU29"); +}); + +test("every card is complete and every unsupported card names its owner and missing contract", () => { + for (const card of PATTERN_CATALOGUE) { + for (const field of [ + card.title, + card.useWhen, + card.prerequisites, + card.envOwnership, + card.readiness, + card.cleanup, + card.negativeControl, + card.provenance.source, + ]) { + expect(field.trim().length, card.id).toBeGreaterThan(0); + } + expect(card.domainOwner, card.id).toMatch(OWNER); + expect(card.provenance.consumers, card.id).toBeGreaterThan(0); + if (card.disposition === "supported") { + expect(card.missingContract, card.id).toBeNull(); + } else { + expect(card.missingContract?.trim().length ?? 0, card.id).toBeGreaterThan( + 0 + ); + } + } +}); + +test("every named hack command resolves in CLI_SPEC", () => { + for (const card of PATTERN_CATALOGUE) { + for (const command of card.supportedCommands) { + expect(command.startsWith("hack "), `${card.id}: ${command}`).toBe(true); + expect( + findGuidanceCommandIssues({ + cli: CLI_SPEC, + rendered: `\`${command}\``, + }), + `${card.id}: ${command}` + ).toEqual([]); + } + } + const rendered = renderPatternCatalogue(); + expect(extractHackInvocations(rendered).length).toBeGreaterThan(0); + expect(findGuidanceCommandIssues({ cli: CLI_SPEC, rendered })).toEqual([]); +}); + +test("banned recipes appear only inside negative-example blocks", () => { + expect(bannedRecipeFindings(renderPatternCatalogue())).toEqual([]); + const rendered = renderPatternCatalogue(); + for (const card of PATTERN_CATALOGUE) { + expect(rendered).toContain( + `${NEGATIVE_EXAMPLE_START}\n> Negative control: ${card.negativeControl}\n${NEGATIVE_EXAMPLE_END}` + ); + } +}); + +test("negative control: the banned-recipe scan flags each recipe outside a negative block", () => { + const fixtures: Readonly> = { + "broad-process-kill": "pkill -f node", + "kill-by-port": "lsof -t -i :3000 | xargs kill", + "placeholder-liveness-tail": "command: tail -f /dev/null", + "placeholder-liveness-sleep": "command: sleep infinity", + "argv-secret-flag": "tool login --token abc123", + "argv-secret-assignment": "API_TOKEN=abc123 hack up", + "argv-env-value": "hack env set DATABASE_URL postgres://x", + }; + for (const [code, text] of Object.entries(fixtures)) { + expect(bannedRecipeFindings(text)).toEqual([code]); + expect( + bannedRecipeFindings( + `${NEGATIVE_EXAMPLE_START}\n${text}\n${NEGATIVE_EXAMPLE_END}` + ) + ).toEqual([]); + expect(bannedRecipeFindings(`${NEGATIVE_EXAMPLE_START}\n${text}`)).toEqual([ + code, + ]); + } +}); + +test("the packaged hack-cli reference is the rendered catalogue", async () => { + const packaged = await Bun.file( + join( + import.meta.dir, + "..", + "plugins/hack/skills/hack-cli/references/patterns.md" + ) + ).text(); + expect(packaged.startsWith(renderPatternCatalogue().trimEnd())).toBe(true); + for (const card of PATTERN_CATALOGUE) { + expect(packaged).toContain(`## ${card.id}: ${card.title}`); + } +}); diff --git a/tests/skill-references.test.ts b/tests/skill-references.test.ts new file mode 100644 index 00000000..c650199e --- /dev/null +++ b/tests/skill-references.test.ts @@ -0,0 +1,130 @@ +import { afterEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { renderPluginSkill } from "../scripts/generate-agent-plugins.ts"; +import { renderCodexSkill } from "../src/agents/codex-skill.ts"; +import { + renderSkillReference, + SKILL_REFERENCES, + skillReferencePath, + skillReferenceRevision, + withSkillReferenceLinks, +} from "../src/agents/skill-references.ts"; +import { findSkillLinkIssues } from "./helpers/skill-links.ts"; + +const pluginRoot = join(import.meta.dir, "..", "plugins", "hack"); + +let tempDir: string | null = null; + +afterEach(async () => { + if (tempDir) { + await rm(tempDir, { recursive: true, force: true }); + tempDir = null; + } +}); + +async function fixturePlugin( + files: Readonly> +): Promise { + tempDir = await mkdtemp(join(tmpdir(), "hack-skill-links-")); + const root = join(tempDir, "plugins", "hack"); + await mkdir(join(tempDir, "docs"), { recursive: true }); + await Bun.write(join(tempDir, "docs", "env.md"), "# Env\n"); + for (const [path, text] of Object.entries(files)) { + await mkdir(join(root, path, ".."), { recursive: true }); + await Bun.write(join(root, path), text); + } + return root; +} + +test("packaged skill links stay inside the plugin and work offline", async () => { + expect(await findSkillLinkIssues({ pluginRoot })).toEqual([]); +}); + +test("every packaged reference is linked and carries a recomputable revision", async () => { + expect(SKILL_REFERENCES.length).toBeGreaterThan(0); + for (const reference of SKILL_REFERENCES) { + const packaged = await Bun.file( + join(pluginRoot, skillReferencePath(reference)) + ).text(); + expect(packaged).toBe(renderSkillReference(reference)); + expect(packaged).toContain( + `Content revision: \`${skillReferenceRevision(reference.render())}\`` + ); + const skill = await Bun.file( + join(pluginRoot, "skills", reference.skill, "SKILL.md") + ).text(); + expect(skill).toContain(`](references/${reference.file})`); + } +}); + +test("only the plugin skill links references; standalone renders stay unchanged", () => { + expect(renderCodexSkill()).not.toContain("references/"); + const plugin = renderPluginSkill("hack-cli"); + expect(plugin.startsWith(renderCodexSkill().trimEnd())).toBe(true); + expect(plugin.slice(renderCodexSkill().trimEnd().length)).toBe( + "\n\n## References\n\n- [Pattern catalogue](references/patterns.md): reusable project patterns with what Hack supports today, the missing contract and its owner.\n" + ); + const unchanged = "---\nname: hack-init\n---\n"; + expect( + withSkillReferenceLinks({ skill: "hack-init", content: unchanged }) + ).toBe(unchanged); +}); + +test("negative control: a link into repository docs fails the link test", async () => { + const root = await fixturePlugin({ + "skills/x/SKILL.md": "# X\n\nSee [env](../../docs/env.md).\n", + "skills/y/SKILL.md": "# Y\n\nSee [env](../../../../docs/env.md).\n", + }); + expect(await findSkillLinkIssues({ pluginRoot: root })).toEqual([ + { file: "skills/x/SKILL.md", code: "repo-docs-link" }, + { file: "skills/y/SKILL.md", code: "escapes-plugin" }, + ]); +}); + +test("negative control: absolute, missing, external and unlinked references fail", async () => { + const root = await fixturePlugin({ + "skills/x/SKILL.md": [ + "# X", + "", + "- [abs](/etc/hosts)", + "- [gone](references/gone.md)", + "- [web](https://example.com/guide)", + "- Read https://example.com/bare too.", + "- `https://schemas.hack/literal.json` is a literal, not a link.", + "", + "```sh", + "curl https://example.com/in-a-fence", + "```", + "", + "## Further reading", + "", + "- [ok](https://example.com/ok)", + "", + ].join("\n"), + "skills/x/references/orphan.md": "# Orphan\n", + }); + expect(await findSkillLinkIssues({ pluginRoot: root })).toEqual([ + { file: "skills/x/SKILL.md", code: "absolute-link" }, + { file: "skills/x/SKILL.md", code: "missing-target" }, + { file: "skills/x/SKILL.md", code: "external-outside-further-reading" }, + { file: "skills/x/SKILL.md", code: "external-outside-further-reading" }, + { file: "skills/x/references/orphan.md", code: "unlinked-reference" }, + ]); +}); + +test("reference file names are constrained to plain markdown names", () => { + for (const file of ["../escape.md", "nested/x.md", "x.txt", "X.md"]) { + expect(() => + skillReferencePath({ + skill: "hack-cli", + file, + title: "x", + summary: "x", + render: () => "", + }) + ).toThrow("Invalid skill reference file name"); + } +}); From 726cc27601bf64657314289820f456c1e77b1c89 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:01:11 -0400 Subject: [PATCH 22/52] test(e2e): add agent-surface-freeze negative control for guidance writes Seed an isolated HOME with sentinel Claude, Codex, Cursor and shared-skill config (including stale hack-cli copies) and two fixtures with customized AGENTS.md, CLAUDE.md, rules, skill copies, settings.local.json and .mcp.json. Run init --auto, init --auto --with codex, agent prime, agent prime --check, agent onboard, agent patterns, doctor --json, doctor --fix --no-interactive and setup sync --check twice each with HOME and HACK_HOME overridden, and require byte-identical hashes, no new client entries, git status changes only under .hack/, and each command's expected exit code. An explicit setup agents --agents-md is the positive control and must change exactly AGENTS.md. Docker is pointed at a missing socket and Mutagen at a stub so doctor --fix stays free of Docker and network effects; the hackd it starts is stopped. The scenario is registered in ALL_SCENARIOS and shard.ts places it in exactly one docker-e2e shard via the default duration hint. Refs: HACK-1124, HACK-1132, HACK-1155 Co-Authored-By: Claude Opus 5.5 --- tests/e2e/README.md | 34 +- tests/e2e/fixture.ts | 184 ++++++++++- tests/e2e/run.ts | 2 + tests/e2e/scenarios/agent-surface-freeze.ts | 338 ++++++++++++++++++++ 4 files changed, 552 insertions(+), 6 deletions(-) create mode 100644 tests/e2e/scenarios/agent-surface-freeze.ts diff --git a/tests/e2e/README.md b/tests/e2e/README.md index d8804860..6e20ee84 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -443,10 +443,11 @@ any e2e result. - **Tier 1 (`local`)** — runs everywhere; no docker needed: `automation-check`, `init`, `env-secrets`, `worktree-secrets`, `worktree-registry`, `worktree-branch-default`, `agent-docs-sync`, - `doctor` (doctor tolerates missing docker — it must report, not crash), and - `lifecycle-session-recovery`. The lifecycle recovery scenario skips when - tmux is unavailable unless `HACK_E2E_REQUIRE_TMUX=1` makes that capability - mandatory (as it is in the dedicated Docker/tmux CI job). + `agent-surface-freeze`, `doctor` (doctor tolerates missing docker — it must + report, not crash), and `lifecycle-session-recovery`. The lifecycle recovery + scenario skips when tmux is unavailable unless `HACK_E2E_REQUIRE_TMUX=1` + makes that capability mandatory (as it is in the dedicated Docker/tmux CI + job). - **Tier 2 (`docker`)** — opt-in via `HACK_E2E_DOCKER=1`; requires a running docker daemon, the machine's global `hack-dev` network (`hack global install`), and pulls `oven/bun:1`: `cache-prune`, `up-down`, @@ -455,6 +456,31 @@ any e2e result. processes only start via `hack up` (no standalone lifecycle command), which is why that scenario is tier 2. +## Agent surface freeze + +`agent-surface-freeze` is the negative control for surprise agent config +writes (HACK-1124, HACK-1132, HACK-1155). It seeds a scenario-owned HOME with +sentinel client config (`~/.claude/settings.json`, `~/.claude.json`, +`~/.claude/skills/`, `~/.codex/config.toml`, `~/.codex/skills/`, +`~/.cursor/mcp.json`, `~/.cursor/rules/`, `~/.ai/skills/`, including stale +`hack-cli` copies). It also commits customized `AGENTS.md`, `CLAUDE.md`, +`.cursor/rules/hack.mdc`, `.codex/skills/hack-cli/SKILL.md`, +`.claude/settings.local.json` and `.mcp.json` into two fixtures. It then runs +`init --auto`, `init --auto --with codex`, `agent prime`, +`agent prime --check`, `agent onboard`, `agent patterns`, `doctor --json`, +`doctor --fix --no-interactive` and `setup sync --check` twice each. After +every run it requires byte-identical hashes for those trees, no new entries +under them, and `git status` changes only under `.hack/`. Each command must +also exit with its expected code, so a usage error cannot pass as "no write". + +Every call sets `HOME` and `HACK_HOME` to scenario temp dirs, so the +contributor's real HOME is never read or written. `DOCKER_HOST` points at a +missing socket and `HACK_MUTAGEN_PATH` at a stub. This keeps +`doctor --fix` free of Docker and network effects even on a Docker CI shard. +The scenario stops the isolated hackd that `doctor --fix` starts. The positive +control, an explicit `setup agents --agents-md`, must change exactly +`AGENTS.md`. + ## Fixture `fixture.ts` scaffolds, per scenario, a fresh git repo with: diff --git a/tests/e2e/fixture.ts b/tests/e2e/fixture.ts index eeceb935..d014f1ea 100644 --- a/tests/e2e/fixture.ts +++ b/tests/e2e/fixture.ts @@ -1,5 +1,6 @@ -import { mkdir } from "node:fs/promises"; -import { join, resolve } from "node:path"; +import type { Stats } from "node:fs"; +import { lstat, mkdir, readdir, readlink } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; import { runCommand } from "./harness.ts"; @@ -495,3 +496,182 @@ async function git(opts: { ); } } + +/** Content hash per path; directories map to "dir" so new empty dirs count. */ +export type FileHashSnapshot = ReadonlyMap; + +export type FileHashDiff = { + readonly changed: readonly string[]; + readonly added: readonly string[]; + readonly removed: readonly string[]; +}; + +/** + * Agent client config a default Hack flow must never touch, relative to an + * isolated HOME. Stale Hack-owned guidance (the most tempting "repair" + * target) sits next to unrelated user config. + */ +const HOME_CLIENT_SENTINELS: Readonly> = { + ".claude/settings.json": `${JSON.stringify({ env: { HACK_E2E_SENTINEL: "home-claude-settings" } }, null, 2)}\n`, + ".claude.json": `${JSON.stringify({ mcpServers: { "e2e-sentinel": { command: "true" } } }, null, 2)}\n`, + ".claude/skills/e2e-sentinel/SKILL.md": + "---\nname: e2e-sentinel\n---\nhack-e2e sentinel: home claude skill\n", + ".claude/skills/hack-cli/SKILL.md": + "---\nname: hack-cli\n---\nhack-e2e sentinel: stale home claude hack skill\n", + ".codex/config.toml": + '# hack-e2e sentinel: home codex config\nmodel = "e2e-sentinel"\n', + ".codex/skills/e2e-sentinel/SKILL.md": + "---\nname: e2e-sentinel\n---\nhack-e2e sentinel: home codex skill\n", + ".codex/skills/hack-cli/SKILL.md": + "---\nname: hack-cli\n---\nhack-e2e sentinel: stale home codex hack skill\n", + ".cursor/mcp.json": `${JSON.stringify({ mcpServers: { "e2e-sentinel": { command: "true" } } }, null, 2)}\n`, + ".cursor/rules/e2e-sentinel.mdc": "hack-e2e sentinel: home cursor rule\n", + ".cursor/rules/hack.mdc": "hack-e2e sentinel: stale home cursor hack rule\n", + ".ai/skills/e2e-sentinel/SKILL.md": + "---\nname: e2e-sentinel\n---\nhack-e2e sentinel: home shared skill\n", + ".ai/skills/hack-cli/SKILL.md": + "---\nname: hack-cli\n---\nhack-e2e sentinel: stale home shared hack skill\n", +}; + +/** HOME-relative client roots whose whole tree must stay byte-identical. */ +export const HOME_CLIENT_ROOTS: readonly string[] = [ + ".claude", + ".claude.json", + ".codex", + ".cursor", + ".ai", + ".agents", +]; + +/** + * Customized repo guidance a default Hack flow must never rewrite, relative + * to a fixture root. None carries Hack's managed markers, so any write is a + * surprise rewrite rather than an in-place marker refresh. + */ +const REPO_GUIDANCE_SENTINELS: Readonly> = { + "AGENTS.md": + "# Team agent rules\n\nhack-e2e sentinel: customized AGENTS.md\n", + "CLAUDE.md": + "# Team Claude rules\n\nhack-e2e sentinel: customized CLAUDE.md\n", + ".cursor/rules/hack.mdc": "hack-e2e sentinel: customized repo cursor rule\n", + ".codex/skills/hack-cli/SKILL.md": + "---\nname: hack-cli\n---\nhack-e2e sentinel: customized repo codex skill\n", + ".claude/settings.local.json": `${JSON.stringify({ permissions: { allow: ["Bash(true)"] } }, null, 2)}\n`, + ".mcp.json": `${JSON.stringify({ mcpServers: { "e2e-sentinel": { command: "true" } } }, null, 2)}\n`, +}; + +/** Root-relative repo guidance roots whose whole tree must stay byte-identical. */ +export const REPO_GUIDANCE_ROOTS: readonly string[] = [ + "AGENTS.md", + "CLAUDE.md", + ".mcp.json", + ".claude", + ".codex", + ".cursor", + ".ai", + ".agents", +]; + +async function writeSentinels(opts: { + readonly root: string; + readonly files: Readonly>; +}): Promise { + const written: string[] = []; + for (const [relative, content] of Object.entries(opts.files)) { + const path = join(opts.root, relative); + await mkdir(dirname(path), { recursive: true }); + await Bun.write(path, content); + written.push(path); + } + return written; +} + +/** Seed sentinel client config under an isolated HOME; returns the files. */ +export async function seedHomeClientSentinels(opts: { + readonly home: string; +}): Promise { + return await writeSentinels({ + root: opts.home, + files: HOME_CLIENT_SENTINELS, + }); +} + +/** Seed and commit customized repo guidance; returns the files. */ +export async function seedRepoGuidanceSentinels(opts: { + readonly root: string; +}): Promise { + const written = await writeSentinels({ + root: opts.root, + files: REPO_GUIDANCE_SENTINELS, + }); + // Force-add: settings.local.json is commonly ignored, but must be tracked + // here so `git status` would surface any edit. + await git({ root: opts.root, args: ["add", "-f", "--", ...written] }); + await git({ + root: opts.root, + args: ["commit", "-q", "-m", "fixture: customized agent guidance"], + }); + return written; +} + +/** + * Hash every entry under `roots` (files or directories; missing roots add + * nothing). Symlinks record their target text and are never followed. + */ +export async function snapshotFileHashes(opts: { + readonly roots: readonly string[]; +}): Promise { + const snapshot = new Map(); + async function visit(path: string): Promise { + let info: Stats; + try { + info = await lstat(path); + } catch { + return; + } + if (info.isSymbolicLink()) { + snapshot.set(path, `link:${await readlink(path)}`); + } else if (info.isDirectory()) { + snapshot.set(path, "dir"); + for (const entry of await readdir(path)) { + await visit(join(path, entry)); + } + } else { + const hasher = new Bun.CryptoHasher("sha256"); + hasher.update(await Bun.file(path).arrayBuffer()); + snapshot.set(path, `sha256:${hasher.digest("hex")}`); + } + } + for (const root of opts.roots) { + await visit(root); + } + return snapshot; +} + +/** Paths whose hash changed, appeared, or disappeared between snapshots. */ +export function diffFileHashes(opts: { + readonly before: FileHashSnapshot; + readonly after: FileHashSnapshot; +}): FileHashDiff { + const changed: string[] = []; + const added: string[] = []; + const removed: string[] = []; + for (const [path, hash] of opts.after) { + const prior = opts.before.get(path); + if (prior === undefined) { + added.push(path); + } else if (prior !== hash) { + changed.push(path); + } + } + for (const path of opts.before.keys()) { + if (!opts.after.has(path)) { + removed.push(path); + } + } + return { + changed: changed.sort(), + added: added.sort(), + removed: removed.sort(), + }; +} diff --git a/tests/e2e/run.ts b/tests/e2e/run.ts index 7e9f0668..e7151477 100644 --- a/tests/e2e/run.ts +++ b/tests/e2e/run.ts @@ -6,6 +6,7 @@ import { } from "./harness.ts"; import { nativeConfigNetworksScenario } from "./native-config-networks-acceptance.ts"; import { agentDocsSyncScenario } from "./scenarios/agent-docs-sync.ts"; +import { agentSurfaceFreezeScenario } from "./scenarios/agent-surface-freeze.ts"; import { automationCheckScenario } from "./scenarios/automation-check.ts"; import { cachePruneScenario } from "./scenarios/cache-prune.ts"; import { dependencyCacheScenario } from "./scenarios/dependency-cache.ts"; @@ -81,6 +82,7 @@ export const ALL_SCENARIOS: readonly Scenario[] = [ worktreeRegistryScenario, worktreeBranchDefaultScenario, agentDocsSyncScenario, + agentSurfaceFreezeScenario, doctorScenario, domainMigrationFilesScenario, lifecycleSessionRecoveryScenario, diff --git a/tests/e2e/scenarios/agent-surface-freeze.ts b/tests/e2e/scenarios/agent-surface-freeze.ts new file mode 100644 index 00000000..e9ae98c9 --- /dev/null +++ b/tests/e2e/scenarios/agent-surface-freeze.ts @@ -0,0 +1,338 @@ +import { mkdir } from "node:fs/promises"; +import { join, relative } from "node:path"; + +import { + createMonorepoFixture, + diffFileHashes, + type FileHashSnapshot, + HOME_CLIENT_ROOTS, + type MonorepoFixture, + REPO_GUIDANCE_ROOTS, + seedHomeClientSentinels, + seedRepoGuidanceSentinels, + snapshotFileHashes, +} from "../fixture.ts"; +import { + type CliResult, + expect, + expectExit, + runCommand, + type Scenario, + type ScenarioContext, +} from "../harness.ts"; + +const FREEZE_TIMEOUT_MS = 240_000; +const RUNS_PER_COMMAND = 2; + +/** + * Exit codes accepted for one ordinary command, per run. A wrong code means + * the command did not exercise its real path (usage error, crash), so a + * byte-identical tree would prove nothing. + */ +type FreezeCommand = { + readonly label: string; + readonly fixture: "project" | "handoff"; + readonly args: readonly string[]; + readonly codes: readonly [readonly number[], readonly number[]]; +}; + +/** + * Default flows covered by the freeze. `init --auto` refuses an existing + * .hack/ on its second run (exit 1); `--with codex` instead hands off the + * onboarding prompt for the existing setup (exit 0). `doctor` exits 1 for the + * unreachable Docker and missing global install, and `--fix` runs only its + * Docker-free remediations (see `freezeEnv`). `setup sync --check` exits 1 + * because the customized guidance differs from Hack's render; it reports the + * drift and must not repair it. + */ +const FREEZE_COMMANDS: readonly FreezeCommand[] = [ + { + label: "init --auto", + fixture: "project", + args: ["init", "--auto"], + codes: [[0], [1]], + }, + { + label: "init --auto --with codex", + fixture: "handoff", + args: ["init", "--auto", "--with", "codex"], + codes: [[0], [0]], + }, + { + label: "agent prime", + fixture: "project", + args: ["agent", "prime"], + codes: [[0], [0]], + }, + { + label: "agent prime --check", + fixture: "project", + args: ["agent", "prime", "--check"], + codes: [[0], [0]], + }, + { + label: "agent onboard", + fixture: "project", + args: ["agent", "onboard"], + codes: [[0], [0]], + }, + { + label: "agent patterns", + fixture: "project", + args: ["agent", "patterns"], + codes: [[0], [0]], + }, + { + label: "doctor --json", + fixture: "project", + args: ["doctor", "--json"], + codes: [ + [0, 1], + [0, 1], + ], + }, + { + label: "doctor --fix --no-interactive", + fixture: "project", + args: ["doctor", "--fix", "--no-interactive"], + codes: [ + [0, 1], + [0, 1], + ], + }, + { + label: "setup sync --check", + fixture: "project", + args: ["setup", "sync", "--check"], + codes: [[1], [1]], + }, +]; + +type FreezeState = { + readonly ctx: ScenarioContext; + readonly env: Readonly>; + readonly roots: readonly string[]; + readonly fixtures: Readonly< + Record + >; + readonly baseline: FileHashSnapshot; +}; + +/** + * Negative control for agent guidance writes (HACK-1124, HACK-1132, + * HACK-1155): default setup, onboarding, agent and doctor flows leave an + * isolated HOME's client config and a repo's customized guidance + * byte-identical, twice in a row. An explicit `setup agents` export is the + * positive control: it changes only its documented target. + * + * Every call overrides HOME and HACK_HOME with scenario temp dirs; the + * contributor's real HOME is never read or written. + */ +export const agentSurfaceFreezeScenario: Scenario = { + name: "agent-surface-freeze", + tier: "local", + summary: + "default flows leave isolated-HOME client config and repo guidance byte-identical", + run: async (ctx) => { + const home = join(ctx.tempRoot, "home"); + await mkdir(home, { recursive: true }); + const seededHome = await seedHomeClientSentinels({ home }); + const project = await createMonorepoFixture({ + parentDir: ctx.tempRoot, + withHackConfig: false, + }); + const handoff = await createMonorepoFixture({ + parentDir: ctx.tempRoot, + withHackConfig: false, + }); + const seededRepo = [ + ...(await seedRepoGuidanceSentinels({ root: project.root })), + ...(await seedRepoGuidanceSentinels({ root: handoff.root })), + ]; + + const roots = [ + ...HOME_CLIENT_ROOTS.map((root) => join(home, root)), + ...REPO_GUIDANCE_ROOTS.map((root) => join(project.root, root)), + ...REPO_GUIDANCE_ROOTS.map((root) => join(handoff.root, root)), + ]; + const baseline = await snapshotFileHashes({ roots }); + const missing = [...seededHome, ...seededRepo].filter( + (path) => !baseline.get(path)?.startsWith("sha256:") + ); + expect({ + that: missing.length === 0, + message: `baseline snapshot is missing seeded sentinels: ${missing.join(", ")}`, + }); + + const state: FreezeState = { + ctx, + env: freezeEnv({ ctx, home }), + roots, + fixtures: { project, handoff }, + baseline, + }; + + try { + for (const command of FREEZE_COMMANDS) { + for (let run = 0; run < RUNS_PER_COMMAND; run += 1) { + await runFrozenCommand({ state, command, run }); + } + } + await runPositiveControl({ state }); + } finally { + // `doctor --fix` starts hackd under the isolated HACK_HOME; never let it + // outlive the scenario. `daemon stop` exits 0 whether or not it ran. + const stop = await ctx.cli({ + args: ["daemon", "stop"], + cwd: project.root, + env: state.env, + }); + if (stop.timedOut || stop.exitCode !== 0) { + ctx.retainFixtures("isolated hackd stop failed"); + } + } + }, +}; + +/** + * Per-call env: isolated HOME + HACK_HOME, Docker pointed at a socket that + * does not exist (local tier: no Docker side effects even when the host runs + * Docker), and a stub Mutagen path so `doctor --fix` never downloads. + */ +function freezeEnv(opts: { + readonly ctx: ScenarioContext; + readonly home: string; +}): Readonly> { + return { + HOME: opts.home, + HACK_HOME: opts.ctx.hackHome, + DOCKER_HOST: `unix://${join(opts.ctx.tempRoot, "no-docker.sock")}`, + HACK_MUTAGEN_PATH: join(opts.ctx.tempRoot, "no-mutagen"), + }; +} + +async function runFrozenCommand(opts: { + readonly state: FreezeState; + readonly command: FreezeCommand; + readonly run: number; +}): Promise { + const { state, command } = opts; + const fixture = state.fixtures[command.fixture]; + const label = `${command.label} (run ${opts.run + 1})`; + const result = await state.ctx.cli({ + args: command.args, + cwd: fixture.root, + env: state.env, + timeoutMs: FREEZE_TIMEOUT_MS, + }); + expectExit({ + result, + codes: command.codes[opts.run] ?? [], + message: `${label} should exercise its real path`, + }); + await expectFrozen({ state, label, result }); + state.ctx.log(`${label}: exit ${result.exitCode}, guidance unchanged`); +} + +async function expectFrozen(opts: { + readonly state: FreezeState; + readonly label: string; + readonly result: CliResult; +}): Promise { + const diff = diffFileHashes({ + before: opts.state.baseline, + after: await snapshotFileHashes({ roots: opts.state.roots }), + }); + expect({ + that: + diff.changed.length === 0 && + diff.added.length === 0 && + diff.removed.length === 0, + message: `${opts.label} wrote agent client or guidance surfaces: changed=[${diff.changed.join(", ")}] added=[${diff.added.join(", ")}] removed=[${diff.removed.join(", ")}]`, + result: opts.result, + }); + for (const fixture of Object.values(opts.state.fixtures)) { + const unexpected = (await gitStatusPaths({ root: fixture.root })).filter( + (line) => !line.path.startsWith(".hack/") + ); + expect({ + that: unexpected.length === 0, + message: `${opts.label} left tracked or untracked changes outside .hack/ in ${fixture.name}: ${unexpected.map((line) => line.raw).join(", ")}`, + result: opts.result, + }); + } +} + +/** + * Positive control: the explicit AGENTS.md export must write exactly + * AGENTS.md, proving the hash and git-status probes can see a real write. + */ +async function runPositiveControl(opts: { + readonly state: FreezeState; +}): Promise { + const { state } = opts; + const project = state.fixtures.project; + const target = join(project.root, "AGENTS.md"); + const result = await state.ctx.cli({ + args: ["setup", "agents", "--agents-md"], + cwd: project.root, + env: state.env, + }); + expectExit({ + result, + codes: [0], + message: "explicit setup agents --agents-md should succeed", + }); + const diff = diffFileHashes({ + before: state.baseline, + after: await snapshotFileHashes({ roots: state.roots }), + }); + expect({ + that: + diff.changed.length === 1 && + diff.changed[0] === target && + diff.added.length === 0 && + diff.removed.length === 0, + message: `explicit setup agents --agents-md should change only ${target}: changed=[${diff.changed.join(", ")}] added=[${diff.added.join(", ")}] removed=[${diff.removed.join(", ")}]`, + result, + }); + const outside = (await gitStatusPaths({ root: project.root })).filter( + (line) => !line.path.startsWith(".hack/") + ); + expect({ + that: + outside.length === 1 && + outside[0]?.path === relative(project.root, target), + message: `explicit export should leave only AGENTS.md modified outside .hack/: ${outside.map((line) => line.raw).join(", ")}`, + result, + }); + state.ctx.log("positive control: setup agents --agents-md changed AGENTS.md"); +} + +type StatusLine = { readonly raw: string; readonly path: string }; + +/** Porcelain v1 entries, including every untracked file. */ +async function gitStatusPaths(opts: { + readonly root: string; +}): Promise { + const status = await runCommand({ + argv: [ + "git", + "-C", + opts.root, + "status", + "--porcelain=v1", + "--untracked-files=all", + ], + cwd: opts.root, + }); + expectExit({ + result: status, + codes: [0], + message: "git status should succeed in the fixture", + }); + return status.stdout + .split("\n") + .filter((line) => line.length > 3) + .map((raw) => ({ raw, path: raw.slice(3) })); +} From 9b971fa00e94e2a691d7855469aa2781dc006836 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:07:49 -0400 Subject: [PATCH 23/52] ci: add gated Bun crash experiment and read-only crash census HACK-1211 needs a matched, bounded way to reproduce the intermittent Bun crash and a repeatable record of crashes per hosted run. This adds the instruments only; it changes no test, timeout, assertion or pin. - bun-crash-experiment.yml runs only on workflow_dispatch or when the bun-crash-experiment label is added to a pull request. A labeled pull request runs the matched pair (1.3.9 baseline, 1.4.2 candidate) on its head; a dispatch runs one version at a chosen revision. It uses blacksmith-6vcpu-macos-15, contents: read, and timeout = budget + 15. The instrument comes from the workflow's commit, so older revisions can be measured. - scripts/bun-crash-loop.ts alternates the native HTTPS owner test with the full bun test. It never starts an iteration whose estimate (its initial guess or longest observed run) ends past the budget, and records a run still going at the deadline as overrun, not a crash. It writes a closed summary and keeps logs plus bun*.ips reports for a 7-day artifact. It never opens crash-report links. - scripts/ci-bun-crash-census.ts reads every attempt's test job through the caller's gh, downloads only failed-job logs into an owned temp dir that it removes, and prints per-version counts and crashing ids only. - Both use scripts/lib/bun-crash-classify.ts. Co-Authored-By: Claude Opus 5.5 --- .ai/skills/hack-repo-verify/SKILL.md | 3 +- .github/workflows/bun-crash-experiment.yml | 163 ++++++ .github/workflows/ci.yml | 2 +- package.json | 1 + scripts/bun-crash-loop.ts | 519 ++++++++++++++++++ scripts/ci-bun-crash-census.ts | 605 +++++++++++++++++++++ scripts/lib/bun-crash-classify.ts | 135 +++++ tests/bun-crash-loop.test.ts | 402 ++++++++++++++ tests/ci-bun-crash-census.test.ts | 446 +++++++++++++++ 9 files changed, 2274 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/bun-crash-experiment.yml create mode 100644 scripts/bun-crash-loop.ts create mode 100644 scripts/ci-bun-crash-census.ts create mode 100644 scripts/lib/bun-crash-classify.ts create mode 100644 tests/bun-crash-loop.test.ts create mode 100644 tests/ci-bun-crash-census.test.ts diff --git a/.ai/skills/hack-repo-verify/SKILL.md b/.ai/skills/hack-repo-verify/SKILL.md index 63634aa0..39a1657d 100644 --- a/.ai/skills/hack-repo-verify/SKILL.md +++ b/.ai/skills/hack-repo-verify/SKILL.md @@ -48,7 +48,7 @@ source provenance. Raw `bun test ` remains available for focused tests; it bypasses the suite preflight and does not prepare compiler-dependent fixtures. For release workflow edits, run the pinned syntax/context check also enforced by CI: -`mise x actionlint@1.7.12 -- actionlint -shellcheck= -pyflakes= .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/prerelease.yml`. +`mise x actionlint@1.7.12 -- actionlint -shellcheck= -pyflakes= .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/prerelease.yml .github/workflows/bun-crash-experiment.yml`. This validates event globs and reusable workflow contexts; Bun YAML parsing alone does not. `bun run privacy:check` scans Git-tracked files. Stage reviewed new source/tests @@ -70,6 +70,7 @@ untracked candidate additions. Keep private artifacts excluded from the index. | Native authored command slices | `tests/native-authored-command.test.ts` (unsupported matrix), `tests/native-authored-command-dispatch.test.ts`, `tests/native-authored-project-{start,run,observe,logs,exec,recovery}.test.ts` and the [command contract](../../../docs/reference/native-authored-commands.md); the dispatch "source CLI" cases need the prepared compiler (`bun run build:config-compiler` or absolute `HACK_CONFIG_COMPILER_BINARY`) and fail without it; `native-authored-project-start` and `native-authored-project-exec` are nondeterministic on a local macOS host, so a single local pass or failure is not proof; live acceptance runs host-serialized on one prepared macOS provider slot | | Native VM/routing/reclamation | [candidate guide](../../../docs/guides/native-candidate.md), relevant acceptance gate and isolated host fixture; Linux checks cannot qualify macOS effects | | Resource/performance claims | [performance skill](../hack-repo-performance/SKILL.md), matched workload and measured boundaries | +| Bun runtime crashes | Read-only history: `bun run ci:bun-crash-census -- --since --until ` (failed `test` job logs; `--scan-passing` for all). Matched reproduction: the gated [experiment](../../../.github/workflows/bun-crash-experiment.yml), which spends hosted macOS minutes and needs an explicit go-ahead; never loop on a shared host. Both share [the classifier](../../../scripts/lib/bun-crash-classify.ts) | Read [CI](../../../.github/workflows/ci.yml) for current hosted gates. A local pass does not mean CI ran. Existing unsupported remote/gateway/native-app checks are diff --git a/.github/workflows/bun-crash-experiment.yml b/.github/workflows/bun-crash-experiment.yml new file mode 100644 index 00000000..d7c60965 --- /dev/null +++ b/.github/workflows/bun-crash-experiment.yml @@ -0,0 +1,163 @@ +name: Bun crash experiment + +# HACK-1211 instrument. Nothing here runs on push or on ordinary pull-request +# activity. A run starts only from a manual dispatch (which GitHub registers once +# this file is on the default branch) or when someone adds the +# `bun-crash-experiment` label to a pull request. Every run spends hosted macOS +# minutes (about budget + 15 per Bun version), so start one only with an explicit +# go-ahead. +# +# A labeled pull request runs the matched pair on the pull request's head: the +# 1.3.9 baseline and the 1.4.2 candidate pinned on next, with the same job and +# budget. A dispatch runs one Bun version at a chosen source revision. +on: + workflow_dispatch: + inputs: + bun_version: + description: Bun version to loop under (major.minor.patch) + type: string + default: "1.3.9" + source_revision: + description: Full 40-character commit to test; empty means the dispatched commit + type: string + default: "" + budget_minutes: + description: Loop budget in minutes (5-240); the job timeout is budget + 15 + type: number + default: 30 + pull_request: + types: [labeled] + +permissions: + contents: read + +concurrency: + group: bun-crash-experiment-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: false + +jobs: + plan: + if: ${{ github.event_name == 'workflow_dispatch' || github.event.label.name == 'bun-crash-experiment' }} + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + outputs: + bun_versions: ${{ steps.plan.outputs.bun_versions }} + source_revision: ${{ steps.plan.outputs.source_revision }} + budget_seconds: ${{ steps.plan.outputs.budget_seconds }} + timeout_minutes: ${{ steps.plan.outputs.timeout_minutes }} + steps: + - name: Validate inputs and plan the runs + id: plan + env: + EVENT_NAME: ${{ github.event_name }} + INPUT_BUN_VERSION: ${{ inputs.bun_version }} + INPUT_SOURCE_REVISION: ${{ inputs.source_revision }} + INPUT_BUDGET_MINUTES: ${{ inputs.budget_minutes }} + PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} + EVENT_SHA: ${{ github.sha }} + run: | + set -euo pipefail + if [ "$EVENT_NAME" = "workflow_dispatch" ]; then + if ! [[ "$INPUT_BUN_VERSION" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then + echo "Invalid bun_version" + exit 1 + fi + versions="[\"$INPUT_BUN_VERSION\"]" + revision="${INPUT_SOURCE_REVISION:-$EVENT_SHA}" + budget="$INPUT_BUDGET_MINUTES" + else + versions='["1.3.9","1.4.2"]' + revision="$PR_HEAD_SHA" + budget=30 + fi + if ! [[ "$revision" =~ ^[0-9a-f]{40}$ ]]; then + echo "Invalid source_revision" + exit 1 + fi + if ! [[ "$budget" =~ ^[1-9][0-9]{0,2}$ ]] || [ "$budget" -lt 5 ] || [ "$budget" -gt 240 ]; then + echo "Invalid budget_minutes" + exit 1 + fi + { + echo "bun_versions=$versions" + echo "source_revision=$revision" + echo "budget_seconds=$((budget * 60))" + echo "timeout_minutes=$((budget + 15))" + } >> "$GITHUB_OUTPUT" + + loop: + name: Bun crash loop (${{ matrix.bun_version }}) + needs: plan + strategy: + fail-fast: false + matrix: + bun_version: ${{ fromJSON(needs.plan.outputs.bun_versions) }} + runs-on: blacksmith-6vcpu-macos-15 + timeout-minutes: ${{ fromJSON(needs.plan.outputs.timeout_minutes) }} + permissions: + contents: read + steps: + # The instrument comes from this workflow's commit, so an older source + # revision without these scripts can still be measured. + - name: Checkout instrument + uses: actions/checkout@v4 + with: + path: .bun-crash-instrument + persist-credentials: false + sparse-checkout: | + scripts/bun-crash-loop.ts + scripts/lib/bun-crash-classify.ts + sparse-checkout-cone-mode: false + - name: Move the instrument outside the tested tree + run: | + mkdir -p "$RUNNER_TEMP/bun-crash-instrument" + mv .bun-crash-instrument/scripts "$RUNNER_TEMP/bun-crash-instrument/scripts" + rm -rf .bun-crash-instrument + - name: Checkout source revision + uses: actions/checkout@v4 + with: + ref: ${{ needs.plan.outputs.source_revision }} + persist-credentials: false + - name: Setup Bun + uses: oven-sh/setup-bun@v1 + with: + bun-version: ${{ matrix.bun_version }} + - name: Install dependencies + run: bun install --frozen-lockfile + - uses: dtolnay/rust-toolchain@1.97.1 + - name: Prepare matching compiler for private file owner tests + run: | + if grep -q '"build:config-compiler":' package.json; then + bun run build:config-compiler + else + echo "Source revision has no config compiler build" + fi + - name: Check test preconditions + run: | + if grep -q '"test:preflight":' package.json; then + bun run test:preflight + else + echo "Source revision has no test preflight" + fi + - name: Loop the owner test and the full suite within the budget + env: + BUN_ENABLE_CRASH_REPORTING: "0" + EXPECTED_BUN_VERSION: ${{ matrix.bun_version }} + SOURCE_REVISION: ${{ needs.plan.outputs.source_revision }} + BUDGET_SECONDS: ${{ needs.plan.outputs.budget_seconds }} + run: >- + bun "$RUNNER_TEMP/bun-crash-instrument/scripts/bun-crash-loop.ts" + --budget-seconds="$BUDGET_SECONDS" + --expect-bun-version="$EXPECTED_BUN_VERSION" + --source-revision="$SOURCE_REVISION" + --out-dir="$RUNNER_TEMP/bun-crash-loop" + - name: Preserve iteration logs and Bun crash reports + if: ${{ always() }} + uses: actions/upload-artifact@v4 + with: + name: bun-crash-loop-${{ matrix.bun_version }} + path: ${{ runner.temp }}/bun-crash-loop + if-no-files-found: ignore + retention-days: 7 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 457db89f..1bef8411 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -130,7 +130,7 @@ jobs: fi sleep "$((attempt * 5))" done - mise x actionlint@1.7.12 -- actionlint -shellcheck= -pyflakes= .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/prerelease.yml + mise x actionlint@1.7.12 -- actionlint -shellcheck= -pyflakes= .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/prerelease.yml .github/workflows/bun-crash-experiment.yml - name: Fetch pinned model checker run: curl --fail --location --retry 2 --max-time 60 https://github.com/tlaplus/tlaplus/releases/download/v1.7.4/tla2tools.jar --output "$RUNNER_TEMP/tla2tools.jar" - name: Check evidence validation and models diff --git a/package.json b/package.json index 73053cac..7c3234d4 100644 --- a/package.json +++ b/package.json @@ -71,6 +71,7 @@ "test:e2e:gateway": "bun run scripts/gateway-e2e.ts", "test:e2e:dispatch": "bun run scripts/dispatch-e2e.ts", "test:e2e:railway": "bun run scripts/railway-node-e2e.ts", + "ci:bun-crash-census": "bun scripts/ci-bun-crash-census.ts", "docs:cli-reference": "bun run scripts/generate-cli-reference.ts", "generate:agent-plugins": "bun run scripts/generate-agent-plugins.ts" }, diff --git a/scripts/bun-crash-loop.ts b/scripts/bun-crash-loop.ts new file mode 100644 index 00000000..220aa1d2 --- /dev/null +++ b/scripts/bun-crash-loop.ts @@ -0,0 +1,519 @@ +#!/usr/bin/env bun +/** + * Fixed-budget Bun crash loop for the gated `bun-crash-experiment` workflow. + * + * It alternates `bun test tests/native-https-owner.test.ts` with the full + * `bun test`, classifies every run with the shared classifier, keeps each run's + * log, and writes a closed, value-free summary. + * + * Budget invariant: an iteration starts only when its estimated duration (the + * larger of its initial estimate and its longest observed run) ends at or before + * the deadline. An iteration that is still running at the deadline is stopped and + * recorded as `overrun`, never as a crash, unless its log carries the crash banner. + * + * The loop never opens or fetches crash-report links. It sets + * `BUN_ENABLE_CRASH_REPORTING=0` so Bun does not upload a report itself; the logs + * and macOS `.ips` reports stay local for offline decoding. + * + * Do not run this on a shared host: it is meant for an isolated hosted runner. + */ +import { closeSync, openSync } from "node:fs"; +import { + appendFile, + copyFile, + mkdir, + readdir, + readFile, + stat, +} from "node:fs/promises"; +import { homedir, tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { + type BunRunOutcome, + type BunRunReason, + classifyBunRun, + closedBunVersion, + closedSignalName, + hasBunCrashBanner, +} from "./lib/bun-crash-classify.ts"; + +export type LoopCommandId = "focused" | "full"; + +export interface LoopCommand { + readonly id: LoopCommandId; + readonly argv: readonly string[]; + readonly initialEstimateMs: number; +} + +export type IterationOutcome = BunRunOutcome | "overrun"; +export type IterationReason = BunRunReason | "deadline"; + +export interface ExecutedRun { + readonly exitCode: number | null; + readonly signal: string | null; + readonly output: string; + /** The harness stopped the run at the deadline. */ + readonly stoppedAtDeadline: boolean; +} + +export interface IterationRecord { + readonly index: number; + readonly command: LoopCommandId; + readonly outcome: IterationOutcome; + readonly reason: IterationReason; + readonly exitCode: number | null; + readonly signal: string | null; + readonly durationMs: number; +} + +export interface LoopResult { + readonly budgetMs: number; + readonly elapsedMs: number; + readonly iterations: readonly IterationRecord[]; + readonly estimatesMs: Readonly>; +} + +export const DEFAULT_FOCUSED_ESTIMATE_SECONDS = 120; +export const DEFAULT_FULL_ESTIMATE_SECONDS = 600; +const MIN_BUDGET_SECONDS = 60; +const MAX_BUDGET_SECONDS = 6 * 60 * 60; +const KILL_GRACE_MS = 10_000; +const REVISION = /^[0-9a-f]{40}$/; + +/** A closed, value-free diagnostic the loop may print. */ +export class LoopError extends Error {} + +export function loopCommands(opts: { + readonly bunExecutable: string; + readonly focusedEstimateSeconds: number; + readonly fullEstimateSeconds: number; +}): readonly LoopCommand[] { + return [ + { + id: "focused", + argv: [opts.bunExecutable, "test", "tests/native-https-owner.test.ts"], + initialEstimateMs: opts.focusedEstimateSeconds * 1000, + }, + { + id: "full", + argv: [opts.bunExecutable, "test"], + initialEstimateMs: opts.fullEstimateSeconds * 1000, + }, + ]; +} + +/** + * Picks the next command, or null when none can finish by the deadline. + * The command at `cursor` is preferred; when it does not fit, the next command in + * sequence order that fits runs instead. Equality with the deadline fits. + */ +export function planNextCommand(opts: { + readonly nowMs: number; + readonly deadlineMs: number; + readonly commands: readonly LoopCommand[]; + readonly cursor: number; + readonly estimatesMs: Readonly>; +}): { readonly command: LoopCommand; readonly cursor: number } | null { + const count = opts.commands.length; + if (count === 0 || opts.nowMs >= opts.deadlineMs) { + return null; + } + const preferred = ((opts.cursor % count) + count) % count; + for (let offset = 0; offset < count; offset += 1) { + const command = opts.commands[(preferred + offset) % count]; + if ( + command !== undefined && + opts.nowMs + opts.estimatesMs[command.id] <= opts.deadlineMs + ) { + return { command, cursor: preferred + 1 }; + } + } + return null; +} + +function classifyIteration(run: ExecutedRun): { + readonly outcome: IterationOutcome; + readonly reason: IterationReason; +} { + if (run.stoppedAtDeadline) { + return hasBunCrashBanner(run.output) + ? { outcome: "crash", reason: "banner" } + : { outcome: "overrun", reason: "deadline" }; + } + return classifyBunRun(run); +} + +/** Runs commands until none can finish within the budget. */ +export async function runBunCrashLoop(opts: { + readonly budgetMs: number; + readonly commands: readonly LoopCommand[]; + readonly now: () => number; + readonly execute: ( + command: LoopCommand, + context: { readonly index: number; readonly deadlineMs: number } + ) => Promise; + readonly onIteration?: (record: IterationRecord) => void; +}): Promise { + const startMs = opts.now(); + const deadlineMs = startMs + opts.budgetMs; + const estimatesMs = {} as Record; + for (const command of opts.commands) { + estimatesMs[command.id] = command.initialEstimateMs; + } + const iterations: IterationRecord[] = []; + let cursor = 0; + for (;;) { + const next = planNextCommand({ + nowMs: opts.now(), + deadlineMs, + commands: opts.commands, + cursor, + estimatesMs, + }); + if (next === null) { + break; + } + cursor = next.cursor; + const index = iterations.length + 1; + const started = opts.now(); + const run = await opts.execute(next.command, { index, deadlineMs }); + const durationMs = Math.max(0, opts.now() - started); + estimatesMs[next.command.id] = Math.max( + estimatesMs[next.command.id], + durationMs + ); + const { outcome, reason } = classifyIteration(run); + const record: IterationRecord = { + index, + command: next.command.id, + outcome, + reason, + exitCode: + typeof run.exitCode === "number" && Number.isInteger(run.exitCode) + ? run.exitCode + : null, + signal: run.stoppedAtDeadline ? null : closedSignalName(run.signal), + durationMs, + }; + iterations.push(record); + opts.onIteration?.(record); + } + return { + budgetMs: opts.budgetMs, + elapsedMs: Math.max(0, opts.now() - startMs), + iterations, + estimatesMs, + }; +} + +const OUTCOMES: readonly IterationOutcome[] = [ + "pass", + "failure", + "crash", + "overrun", +]; + +export function summarizeIterations( + iterations: readonly IterationRecord[] +): Record> { + const empty = (): Record => ({ + pass: 0, + failure: 0, + crash: 0, + overrun: 0, + }); + const totals = { focused: empty(), full: empty() }; + for (const record of iterations) { + totals[record.command][record.outcome] += 1; + } + return totals; +} + +const seconds = (ms: number): string => (ms / 1000).toFixed(1); + +/** Markdown for `$GITHUB_STEP_SUMMARY`. Only closed tokens, numbers and validated identifiers. */ +export function renderLoopSummary(opts: { + readonly result: LoopResult; + readonly bunVersion: string | null; + readonly sourceRevision: string | null; + readonly diagnosticReports: number; +}): string { + const totals = summarizeIterations(opts.result.iterations); + const lines = [ + "## Bun crash experiment", + "", + "| Field | Value |", + "| --- | --- |", + `| Bun | ${opts.bunVersion ?? "unknown"} |`, + `| Source revision | ${opts.sourceRevision ?? "unknown"} |`, + `| Budget seconds | ${seconds(opts.result.budgetMs)} |`, + `| Loop seconds | ${seconds(opts.result.elapsedMs)} |`, + `| Iterations | ${opts.result.iterations.length} |`, + `| Crash reports (bun*.ips) | ${opts.diagnosticReports} |`, + "", + "| Command | Runs | Pass | Failure | Crash | Overrun | Estimate seconds |", + "| --- | --- | --- | --- | --- | --- | --- |", + ]; + for (const id of ["focused", "full"] as const) { + const row = totals[id]; + const runs = OUTCOMES.reduce((sum, outcome) => sum + row[outcome], 0); + lines.push( + `| ${id} | ${runs} | ${row.pass} | ${row.failure} | ${row.crash} | ${row.overrun} | ${seconds(opts.result.estimatesMs[id] ?? 0)} |` + ); + } + lines.push( + "", + "| # | Command | Outcome | Reason | Exit | Signal | Seconds |", + "| --- | --- | --- | --- | --- | --- | --- |" + ); + for (const record of opts.result.iterations) { + lines.push( + `| ${record.index} | ${record.command} | ${record.outcome} | ${record.reason} | ${record.exitCode ?? "-"} | ${record.signal ?? "-"} | ${seconds(record.durationMs)} |` + ); + } + lines.push( + "", + "Logs and crash reports are in the run artifact. Crash-report links are never opened.", + "" + ); + return lines.join("\n"); +} + +/** Copies `bun*.ips` reports modified at or after `sinceMs` into `destination`. */ +export async function collectDiagnosticReports(opts: { + readonly source: string; + readonly destination: string; + readonly sinceMs: number; +}): Promise { + let names: string[]; + try { + names = await readdir(opts.source); + } catch { + return 0; + } + let copied = 0; + for (const name of names) { + if (!/^bun.*\.ips$/i.test(name)) { + continue; + } + const source = join(opts.source, name); + const info = await stat(source).catch(() => null); + if (!info?.isFile() || info.mtimeMs < opts.sinceMs) { + continue; + } + await mkdir(opts.destination, { recursive: true }); + await copyFile(source, join(opts.destination, name)); + copied += 1; + } + return copied; +} + +export function executeWithLog(opts: { + readonly cwd: string; + readonly logDir: string; +}): ( + command: LoopCommand, + context: { readonly index: number; readonly deadlineMs: number } +) => Promise { + return async (command, context) => { + const logPath = join( + opts.logDir, + `iteration-${String(context.index).padStart(3, "0")}-${command.id}.log` + ); + // Write straight to a file, so a child that keeps the descriptor open cannot block the loop. + const fd = openSync(logPath, "w", 0o600); + let stoppedAtDeadline = false; + let proc: ReturnType; + try { + proc = Bun.spawn([...command.argv], { + cwd: opts.cwd, + env: { ...process.env, BUN_ENABLE_CRASH_REPORTING: "0" }, + stdin: "ignore", + stdout: fd, + stderr: fd, + }); + } catch (error) { + closeSync(fd); + throw error; + } + let killTimer: ReturnType | undefined; + const stopTimer = setTimeout( + () => { + stoppedAtDeadline = true; + proc.kill("SIGTERM"); + killTimer = setTimeout(() => proc.kill("SIGKILL"), KILL_GRACE_MS); + }, + Math.max(0, context.deadlineMs - Date.now()) + ); + try { + await proc.exited; + } finally { + clearTimeout(stopTimer); + if (killTimer !== undefined) { + clearTimeout(killTimer); + } + closeSync(fd); + } + return { + exitCode: proc.exitCode, + signal: proc.signalCode ?? null, + output: await readFile(logPath, "utf8"), + stoppedAtDeadline, + }; + }; +} + +export interface LoopArguments { + readonly budgetSeconds: number; + readonly outDir: string; + readonly expectBunVersion: string | null; + readonly sourceRevision: string | null; + readonly focusedEstimateSeconds: number; + readonly fullEstimateSeconds: number; +} + +function parseSeconds(name: string, value: string, min: number, max: number) { + if (!/^\d{1,6}$/.test(value)) { + throw new LoopError(`Invalid ${name}`); + } + const parsed = Number(value); + if (parsed < min || parsed > max) { + throw new LoopError(`Invalid ${name}`); + } + return parsed; +} + +export function parseLoopArguments(argv: readonly string[]): LoopArguments { + const values = new Map(); + for (const arg of argv) { + const match = /^--([a-z-]+)=(.*)$/.exec(arg); + if (!match?.[1] || match[2] === undefined || values.has(match[1])) { + throw new LoopError("Invalid argument"); + } + values.set(match[1], match[2]); + } + const known = new Set([ + "budget-seconds", + "out-dir", + "expect-bun-version", + "source-revision", + "focused-estimate-seconds", + "full-estimate-seconds", + ]); + for (const key of values.keys()) { + if (!known.has(key)) { + throw new LoopError("Invalid argument"); + } + } + const budget = values.get("budget-seconds"); + if (budget === undefined) { + throw new LoopError("Missing --budget-seconds"); + } + const expect = values.get("expect-bun-version"); + if (expect !== undefined && closedBunVersion(expect) === null) { + throw new LoopError("Invalid --expect-bun-version"); + } + const revision = values.get("source-revision"); + if (revision !== undefined && !REVISION.test(revision)) { + throw new LoopError("Invalid --source-revision"); + } + const outDir = values.get("out-dir"); + if (outDir !== undefined && outDir.length === 0) { + throw new LoopError("Invalid --out-dir"); + } + return { + budgetSeconds: parseSeconds( + "--budget-seconds", + budget, + MIN_BUDGET_SECONDS, + MAX_BUDGET_SECONDS + ), + outDir: resolve(outDir ?? join(tmpdir(), "bun-crash-loop")), + expectBunVersion: expect ?? null, + sourceRevision: revision ?? null, + focusedEstimateSeconds: parseSeconds( + "--focused-estimate-seconds", + values.get("focused-estimate-seconds") ?? + String(DEFAULT_FOCUSED_ESTIMATE_SECONDS), + 1, + MAX_BUDGET_SECONDS + ), + fullEstimateSeconds: parseSeconds( + "--full-estimate-seconds", + values.get("full-estimate-seconds") ?? + String(DEFAULT_FULL_ESTIMATE_SECONDS), + 1, + MAX_BUDGET_SECONDS + ), + }; +} + +async function main(): Promise { + const args = parseLoopArguments(process.argv.slice(2)); + const bunVersion = closedBunVersion(Bun.version); + if (args.expectBunVersion !== null && bunVersion !== args.expectBunVersion) { + throw new LoopError("Running Bun does not match --expect-bun-version"); + } + const logDir = join(args.outDir, "logs"); + await mkdir(logDir, { recursive: true }); + const loopStartMs = Date.now(); + const result = await runBunCrashLoop({ + budgetMs: args.budgetSeconds * 1000, + commands: loopCommands({ + bunExecutable: process.execPath, + focusedEstimateSeconds: args.focusedEstimateSeconds, + fullEstimateSeconds: args.fullEstimateSeconds, + }), + now: Date.now, + execute: executeWithLog({ cwd: process.cwd(), logDir }), + onIteration: (record) => { + process.stdout.write( + `iteration=${record.index} command=${record.command} outcome=${record.outcome} reason=${record.reason} seconds=${seconds(record.durationMs)}\n` + ); + }, + }); + const diagnosticReports = await collectDiagnosticReports({ + source: join(homedir(), "Library", "Logs", "DiagnosticReports"), + destination: join(args.outDir, "diagnostic-reports"), + sinceMs: loopStartMs, + }); + const summary = renderLoopSummary({ + result, + bunVersion, + sourceRevision: args.sourceRevision, + diagnosticReports, + }); + await Bun.write( + join(args.outDir, "summary.json"), + `${JSON.stringify( + { + bunVersion, + sourceRevision: args.sourceRevision, + diagnosticReports, + ...result, + }, + null, + 2 + )}\n` + ); + const stepSummary = process.env.GITHUB_STEP_SUMMARY; + if (stepSummary) { + await appendFile(stepSummary, summary); + } else { + process.stdout.write(summary); + } + if (result.iterations.length === 0) { + throw new LoopError("No iteration fits the budget"); + } +} + +if (import.meta.main) { + try { + await main(); + } catch (error) { + process.stderr.write( + `${error instanceof LoopError ? error.message : "Bun crash loop failed"}\n` + ); + process.exitCode = 1; + } +} diff --git a/scripts/ci-bun-crash-census.ts b/scripts/ci-bun-crash-census.ts new file mode 100644 index 00000000..c6b85368 --- /dev/null +++ b/scripts/ci-bun-crash-census.ts @@ -0,0 +1,605 @@ +#!/usr/bin/env bun +/** + * Read-only census of Bun crashes in hosted CI history. + * + * For a `--since`/`--until` window it lists every run of the CI workflow, reads + * the `test` job of every run attempt, downloads only failed-job logs (all + * completed logs with `--scan-passing`) into an owned temporary directory, and + * classifies each with the shared classifier used by `scripts/bun-crash-loop.ts`. + * + * It uses the caller's existing `gh` authentication and never handles a token. + * It is strictly read-only: no reruns, comments, labels or uploads. It prints + * counts, closed tokens, versions and numeric run/job ids only, never log bodies. + * + * Bun version per job comes from the `Setup Bun` input in the job log when a log + * is read, and otherwise from the workflow file at the run's head commit. For a + * pull-request run the job actually used the merge commit's workflow, so a stale + * pull-request head can attribute a job to the older pin; `versionSources` shows + * how many jobs each source covers. + */ +import { closeSync, openSync, rmSync } from "node:fs"; +import { mkdtemp, readFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + type BunRunReason, + classifyHostedJobLog, + closedBunVersion, + hostedJobBunVersion, +} from "./lib/bun-crash-classify.ts"; + +/** A closed, value-free diagnostic the census may print. */ +export class CensusError extends Error {} + +/** The GitHub REST reads the census needs, through the caller's `gh` authentication. */ +export interface GhReader { + /** Paginated GET; one JSON value per output line of `jq`. */ + readonly lines: (path: string, jq: string) => Promise; + /** One GET returning JSON. */ + readonly json: (path: string) => Promise; + /** GET whose body is written to `destination`. */ + readonly download: (path: string, destination: string) => Promise; +} + +export type JobConclusion = + | "success" + | "failure" + | "timed_out" + | "cancelled" + | "other"; + +export type VersionSource = "log" | "workflow" | "unknown"; + +export interface JobRecord { + readonly runId: number; + readonly attempt: number; + readonly jobId: number; + readonly conclusion: JobConclusion; + readonly bunVersion: string; + readonly versionSource: VersionSource; + /** Null when no log was read. */ + readonly scan: "crash" | "failure" | "pass" | "unavailable" | null; + readonly reason: BunRunReason | null; +} + +export interface CrashingJob { + readonly runId: number; + readonly attempt: number; + readonly jobId: number; + readonly reason: BunRunReason; +} + +export interface VersionTotals { + readonly bunVersion: string; + executions: number; + successes: number; + failures: number; + cancelled: number; + crashes: number; + logsScanned: number; + logsUnavailable: number; + readonly versionSources: Record; + readonly crashing: CrashingJob[]; +} + +const ISO_INSTANT = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/; +const REPOSITORY = /^[A-Za-z0-9_.-]{1,100}\/[A-Za-z0-9_.-]{1,100}$/; +const WORKFLOW_FILE = /^[A-Za-z0-9_.-]{1,100}\.ya?ml$/; +const JOB_NAME = /^[A-Za-z0-9_. -]{1,100}$/; +const SHA = /^[0-9a-f]{40}$/; +/** The runs endpoint returns at most 1000 results for one `created` query. */ +const MAX_WINDOW_RUNS = 1000; +const DEFAULT_CONCURRENCY = 6; + +const FAILED: ReadonlySet = new Set(["failure", "timed_out"]); + +function closedConclusion(value: unknown): JobConclusion { + return value === "success" || + value === "failure" || + value === "timed_out" || + value === "cancelled" + ? value + : "other"; +} + +const positiveInteger = (value: unknown): number | null => + typeof value === "number" && Number.isSafeInteger(value) && value > 0 + ? value + : null; + +/** + * The `bun-version` given to the named job in a workflow file, or null. The + * job block runs from ` :` to the next two-space-indented key. + */ +export function workflowJobBunVersion( + workflow: string, + job: string +): string | null { + const lines = workflow.split(/\r?\n/); + const start = lines.findIndex((line) => line.trimEnd() === ` ${job}:`); + if (start === -1) { + return null; + } + for (const line of lines.slice(start + 1)) { + if (/^ {2}\S/.test(line)) { + return null; + } + const match = /^\s+bun-version:\s*["']?([^"'\s]+)["']?\s*$/.exec(line); + if (match) { + return closedBunVersion(match[1]); + } + } + return null; +} + +export function emptyTotals(bunVersion: string): VersionTotals { + return { + bunVersion, + executions: 0, + successes: 0, + failures: 0, + cancelled: 0, + crashes: 0, + logsScanned: 0, + logsUnavailable: 0, + versionSources: { log: 0, workflow: 0, unknown: 0 }, + crashing: [], + }; +} + +/** Per-Bun-version totals, versions in ascending order with `unknown` last. */ +export function aggregateCensus( + records: readonly JobRecord[] +): VersionTotals[] { + const byVersion = new Map(); + for (const record of records) { + const totals = + byVersion.get(record.bunVersion) ?? emptyTotals(record.bunVersion); + byVersion.set(record.bunVersion, totals); + totals.executions += 1; + totals.versionSources[record.versionSource] += 1; + if (record.conclusion === "success") { + totals.successes += 1; + } else if (FAILED.has(record.conclusion)) { + totals.failures += 1; + } else if (record.conclusion === "cancelled") { + totals.cancelled += 1; + } + if (record.scan === "unavailable") { + totals.logsUnavailable += 1; + } else if (record.scan !== null) { + totals.logsScanned += 1; + } + if (record.scan === "crash" && record.reason !== null) { + totals.crashes += 1; + totals.crashing.push({ + runId: record.runId, + attempt: record.attempt, + jobId: record.jobId, + reason: record.reason, + }); + } + } + const order = (version: string): number[] => + version === "unknown" + ? [Number.POSITIVE_INFINITY] + : version.split(".").map(Number); + return [...byVersion.values()].sort((left, right) => { + const a = order(left.bunVersion); + const b = order(right.bunVersion); + for (let index = 0; index < Math.max(a.length, b.length); index += 1) { + const difference = (a[index] ?? 0) - (b[index] ?? 0); + if (difference !== 0) { + return difference; + } + } + return 0; + }); +} + +export interface CensusOptions { + readonly repository: string; + readonly workflow: string; + readonly job: string; + readonly since: string; + readonly until: string; + readonly scanPassing: boolean; + readonly concurrency?: number; +} + +export interface CensusResult { + readonly runs: number; + readonly records: readonly JobRecord[]; + readonly totals: readonly VersionTotals[]; +} + +async function mapConcurrent( + items: readonly T[], + limit: number, + task: (item: T) => Promise +): Promise { + const results = new Array(items.length); + let next = 0; + const worker = async () => { + while (next < items.length) { + const index = next; + next += 1; + results[index] = await task(items[index] as T); + } + }; + await Promise.all( + Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, worker) + ); + return results; +} + +function parseLine(line: string): Record { + let value: unknown; + try { + value = JSON.parse(line); + } catch { + throw new CensusError("Unexpected GitHub response"); + } + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new CensusError("Unexpected GitHub response"); + } + return value as Record; +} + +interface ScannedLog { + readonly scan: Exclude; + readonly reason: BunRunReason | null; + readonly bunVersion: string | null; +} + +/** Downloads one job log into `workDir`, classifies it and removes it. */ +async function scanJobLog(opts: { + readonly gh: GhReader; + readonly repo: string; + readonly jobId: number; + readonly workDir: string; +}): Promise { + const destination = join(opts.workDir, `job-${opts.jobId}.log`); + try { + await opts.gh.download( + `repos/${opts.repo}/actions/jobs/${opts.jobId}/logs`, + destination + ); + const log = await readFile(destination, "utf8"); + const classification = classifyHostedJobLog(log); + return { + scan: classification.outcome, + reason: classification.reason, + bunVersion: hostedJobBunVersion(log), + }; + } catch { + return { scan: "unavailable", reason: null, bunVersion: null }; + } finally { + await rm(destination, { force: true }); + } +} + +function jobVersion( + fromLog: string | null, + fromWorkflow: string | null +): { readonly bunVersion: string; readonly versionSource: VersionSource } { + if (fromLog !== null) { + return { bunVersion: fromLog, versionSource: "log" }; + } + if (fromWorkflow !== null) { + return { bunVersion: fromWorkflow, versionSource: "workflow" }; + } + return { bunVersion: "unknown", versionSource: "unknown" }; +} + +/** + * Runs the census. Logs are written only under `workDir`, one at a time per + * worker, and each is removed as soon as it is classified. + */ +export async function runCensus( + options: CensusOptions, + gh: GhReader, + workDir: string +): Promise { + const repo = options.repository; + const created = `${options.since}..${options.until}`; + const runsPath = `repos/${repo}/actions/workflows/${options.workflow}/runs?created=${created}`; + const head = await gh.json(`${runsPath}&per_page=1`); + const total = + typeof head === "object" && head !== null && "total_count" in head + ? head.total_count + : null; + if (typeof total !== "number" || !Number.isSafeInteger(total) || total < 0) { + throw new CensusError("Unexpected GitHub response"); + } + if (total > MAX_WINDOW_RUNS) { + throw new CensusError("Window has more than 1000 runs; narrow it"); + } + const runs: { readonly id: number; readonly headSha: string | null }[] = []; + for (const line of await gh.lines( + `${runsPath}&per_page=100`, + ".workflow_runs[] | {id, head_sha}" + )) { + const run = parseLine(line); + const id = positiveInteger(run.id); + if (id === null) { + throw new CensusError("Unexpected GitHub response"); + } + const sha = + typeof run.head_sha === "string" && SHA.test(run.head_sha) + ? run.head_sha + : null; + if (!runs.some((existing) => existing.id === id)) { + runs.push({ id, headSha: sha }); + } + } + + const workflowVersions = new Map>(); + const workflowVersion = (sha: string | null): Promise => { + if (sha === null) { + return Promise.resolve(null); + } + const cached = workflowVersions.get(sha); + if (cached) { + return cached; + } + const pending = gh + .json( + `repos/${repo}/contents/.github/workflows/${options.workflow}?ref=${sha}` + ) + .then((value) => { + if ( + typeof value !== "object" || + value === null || + !("content" in value) || + typeof value.content !== "string" + ) { + return null; + } + return workflowJobBunVersion( + Buffer.from(value.content, "base64").toString("utf8"), + options.job + ); + }) + .catch(() => null); + workflowVersions.set(sha, pending); + return pending; + }; + + const perRun = await mapConcurrent( + runs, + options.concurrency ?? DEFAULT_CONCURRENCY, + async (run) => { + const seen = new Set(); + const records: JobRecord[] = []; + const lines = await gh.lines( + `repos/${repo}/actions/runs/${run.id}/jobs?filter=all&per_page=100`, + ".jobs[] | {id, name, run_attempt, status, conclusion}" + ); + for (const line of lines) { + const job = parseLine(line); + const jobId = positiveInteger(job.id); + const attempt = positiveInteger(job.run_attempt); + if ( + job.name !== options.job || + job.status !== "completed" || + job.conclusion === "skipped" || + jobId === null || + attempt === null || + seen.has(jobId) + ) { + continue; + } + seen.add(jobId); + const conclusion = closedConclusion(job.conclusion); + const scanned = + FAILED.has(conclusion) || + (options.scanPassing && conclusion !== "other") + ? await scanJobLog({ gh, repo, jobId, workDir }) + : null; + const fromWorkflow = scanned?.bunVersion + ? null + : await workflowVersion(run.headSha); + records.push({ + runId: run.id, + attempt, + jobId, + conclusion, + ...jobVersion(scanned?.bunVersion ?? null, fromWorkflow), + scan: scanned?.scan ?? null, + reason: scanned?.reason ?? null, + }); + } + return records; + } + ); + const records = perRun.flat(); + return { runs: runs.length, records, totals: aggregateCensus(records) }; +} + +/** Closed, value-free text report. */ +export function renderCensus(opts: { + readonly options: CensusOptions; + readonly result: CensusResult; +}): string { + const { options, result } = opts; + const lines = [ + `Bun crash census repo=${options.repository} workflow=${options.workflow} job=${options.job} since=${options.since} until=${options.until} runs=${result.runs} jobs=${result.records.length} scan_passing=${options.scanPassing}`, + ]; + for (const totals of result.totals) { + const sources = Object.entries(totals.versionSources) + .map(([source, count]) => `${source}:${count}`) + .join(","); + lines.push( + `bun=${totals.bunVersion} executions=${totals.executions} successes=${totals.successes} failures=${totals.failures} cancelled=${totals.cancelled} crashes=${totals.crashes} logs_scanned=${totals.logsScanned} logs_unavailable=${totals.logsUnavailable} version_sources=${sources}` + ); + for (const crash of totals.crashing) { + lines.push( + ` crash run=${crash.runId} attempt=${crash.attempt} job=${crash.jobId} reason=${crash.reason}` + ); + } + } + return `${lines.join("\n")}\n`; +} + +const VALUED_ARGUMENTS = new Set(["since", "until", "repo", "workflow", "job"]); +const BOOLEAN_ARGUMENTS = new Set(["scan-passing", "json"]); + +function tokenizeCensusArguments(argv: readonly string[]): { + readonly values: Map; + readonly flags: Set; +} { + const values = new Map(); + const flags = new Set(); + for (let index = 0; index < argv.length; index += 1) { + const match = /^--([a-z-]+)(?:=(.*))?$/.exec(argv[index] ?? ""); + const key = match?.[1]; + let value = match?.[2]; + if (key && BOOLEAN_ARGUMENTS.has(key) && value === undefined) { + if (flags.has(key)) { + throw new CensusError("Invalid argument"); + } + flags.add(key); + continue; + } + if (!(key && VALUED_ARGUMENTS.has(key)) || values.has(key)) { + throw new CensusError("Invalid argument"); + } + if (value === undefined) { + index += 1; + value = argv[index]; + } + if (value === undefined) { + throw new CensusError("Invalid argument"); + } + values.set(key, value); + } + return { values, flags }; +} + +export function parseCensusArguments(argv: readonly string[]): CensusOptions & { + readonly json: boolean; +} { + const { values, flags } = tokenizeCensusArguments(argv); + const since = values.get("since"); + const until = values.get("until"); + if ( + !(since && until && ISO_INSTANT.test(since) && ISO_INSTANT.test(until)) || + Number.isNaN(Date.parse(since)) || + Number.isNaN(Date.parse(until)) || + Date.parse(since) >= Date.parse(until) + ) { + throw new CensusError( + "--since and --until must be UTC instants (YYYY-MM-DDTHH:MM:SSZ) with since before until" + ); + } + const repository = values.get("repo") ?? "hack-dance/hack"; + const workflow = values.get("workflow") ?? "ci.yml"; + const job = values.get("job") ?? "test"; + if ( + !( + REPOSITORY.test(repository) && + WORKFLOW_FILE.test(workflow) && + JOB_NAME.test(job) + ) + ) { + throw new CensusError("Invalid argument"); + } + return { + repository, + workflow, + job, + since, + until, + scanPassing: flags.has("scan-passing"), + json: flags.has("json"), + }; +} + +async function gh( + args: readonly string[], + stdout: "pipe" | number +): Promise { + const proc = Bun.spawn(["gh", ...args], { + stdin: "ignore", + stdout, + stderr: "ignore", + }); + const text = + stdout === "pipe" && proc.stdout instanceof ReadableStream + ? await new Response(proc.stdout).text() + : ""; + if ((await proc.exited) !== 0) { + throw new CensusError("gh api request failed"); + } + return text; +} + +/** `gh api` reader; GET only, with the caller's existing authentication. */ +export const ghCliReader: GhReader = { + lines: async (path, jq) => + ( + await gh( + ["api", "--method", "GET", "--paginate", "--jq", jq, path], + "pipe" + ) + ) + .split("\n") + .filter((line) => line.length > 0), + json: async (path) => { + try { + return JSON.parse(await gh(["api", "--method", "GET", path], "pipe")); + } catch (error) { + throw error instanceof CensusError + ? error + : new CensusError("Unexpected GitHub response"); + } + }, + download: async (path, destination) => { + const fd = openSync(destination, "wx", 0o600); + try { + // Logs carry terminal colors; the body goes to the owned file, never the terminal. + await gh( + ["api", "--method", "GET", "--allow-escape-sequences", path], + fd + ); + } finally { + closeSync(fd); + } + }, +}; + +async function main(): Promise { + const { json, ...options } = parseCensusArguments(process.argv.slice(2)); + const workDir = await mkdtemp(join(tmpdir(), "hack-bun-census-")); + const cleanup = () => rmSync(workDir, { recursive: true, force: true }); + const interrupt = () => { + cleanup(); + process.exit(130); + }; + process.once("SIGINT", interrupt); + process.once("SIGTERM", interrupt); + try { + const result = await runCensus(options, ghCliReader, workDir); + process.stdout.write( + json + ? `${JSON.stringify({ options, runs: result.runs, totals: result.totals }, null, 2)}\n` + : renderCensus({ options, result }) + ); + } finally { + process.off("SIGINT", interrupt); + process.off("SIGTERM", interrupt); + await rm(workDir, { recursive: true, force: true }); + } +} + +if (import.meta.main) { + try { + await main(); + } catch (error) { + process.stderr.write( + `${error instanceof CensusError ? error.message : "Bun crash census failed"}\n` + ); + process.exitCode = 1; + } +} diff --git a/scripts/lib/bun-crash-classify.ts b/scripts/lib/bun-crash-classify.ts new file mode 100644 index 00000000..44317681 --- /dev/null +++ b/scripts/lib/bun-crash-classify.ts @@ -0,0 +1,135 @@ +/** + * Shared, pure classifier for one Bun process run or one hosted job log. + * + * The bounded crash-loop harness (`scripts/bun-crash-loop.ts`) and the read-only + * hosted census (`scripts/ci-bun-crash-census.ts`) both classify with this module, + * so a crash means the same thing in an experiment and in hosted history. + * + * Results are closed tokens only. Nothing from the output (paths, test names, + * crash-report links or values) is ever returned. + */ + +/** Exit statuses a shell reports for Bun's crash signals: SIGTRAP (133) and SIGSEGV (139). */ +export const BUN_CRASH_EXIT_CODES: readonly number[] = [133, 139]; + +export type BunRunOutcome = "crash" | "failure" | "pass"; + +export type BunRunReason = + /** The output carries Bun's own crash banner. */ + | "banner" + /** The process was ended by a signal it did not get from the harness. */ + | "signal" + /** The exit status encodes a crash signal (133 or 139). */ + | "signal_exit" + /** Exit 0 and no crash banner. */ + | "exit_zero" + /** A non-zero exit with `bun test` failure markers. */ + | "test_failure" + /** Any other non-zero exit status, for example a typecheck exit 2. */ + | "exit_code" + /** No exit status and no signal were observed. */ + | "missing_exit"; + +export interface BunRunClassification { + readonly outcome: BunRunOutcome; + readonly reason: BunRunReason; +} + +export interface BunRunObservation { + readonly exitCode: number | null; + readonly signal: string | null; + readonly output: string; +} + +const CRASH_BANNERS = [/Bun has crashed/, /panic\(main thread\)/]; +// CSI (ESC [ ... final), OSC (ESC ] ... BEL or ST) and two-byte ESC sequences. +const ANSI_ESCAPES = + /\u001b\[[0-?]*[ -/]*[@-~]|\u001b\][^\u0007\u001b]*(?:\u0007|\u001b\\)|\u001b[@-Z\\-_]/g; +const TEST_FAILURE_LINE = /(?:^|\s)\(fail\)\s/m; +const TEST_FAILURE_SUMMARY = /(?:^|\s)[1-9]\d* fail\s*$/m; +const HOSTED_EXIT = /##\[error\]Process completed with exit code (\d{1,3})\./g; +const SIGNAL_NAME = /^SIG[A-Z0-9]{2,10}$/; +const BUN_VERSION = /^\d{1,3}\.\d{1,3}\.\d{1,3}$/; +const HOSTED_BUN_VERSION_INPUT = + /^(?:\S+Z )?\s*bun-version: "?(\d{1,3}\.\d{1,3}\.\d{1,3})"?\s*$/m; + +/** Removes terminal escape sequences so banners and markers match colored output. */ +export function stripAnsi(text: string): string { + return text.replace(ANSI_ESCAPES, ""); +} + +/** True when the output carries Bun's crash banner, colored or not. */ +export function hasBunCrashBanner(output: string): boolean { + const plain = stripAnsi(output); + return CRASH_BANNERS.some((banner) => banner.test(plain)); +} + +/** + * Classifies one run. A crash banner, a signal, or a crash exit status is a crash + * whatever else the run printed; exit 0 is a pass; every other result is a + * non-crash failure. + */ +export function classifyBunRun( + observation: BunRunObservation +): BunRunClassification { + const plain = stripAnsi(observation.output); + if (CRASH_BANNERS.some((banner) => banner.test(plain))) { + return { outcome: "crash", reason: "banner" }; + } + if (observation.signal !== null) { + return { outcome: "crash", reason: "signal" }; + } + if (observation.exitCode === null) { + return { outcome: "failure", reason: "missing_exit" }; + } + if (BUN_CRASH_EXIT_CODES.includes(observation.exitCode)) { + return { outcome: "crash", reason: "signal_exit" }; + } + if (observation.exitCode === 0) { + return { outcome: "pass", reason: "exit_zero" }; + } + if (TEST_FAILURE_LINE.test(plain) || TEST_FAILURE_SUMMARY.test(plain)) { + return { outcome: "failure", reason: "test_failure" }; + } + return { outcome: "failure", reason: "exit_code" }; +} + +/** + * The exit status of the step that failed a hosted job, from the runner's + * `Process completed with exit code N.` annotation, or null when there is none + * (for example a cancelled or timed-out job). + */ +export function hostedJobExitCode(log: string): number | null { + let last: number | null = null; + for (const match of stripAnsi(log).matchAll(HOSTED_EXIT)) { + last = Number(match[1]); + } + return last; +} + +/** The Bun version the hosted `Setup Bun` step was given, or null. */ +export function hostedJobBunVersion(log: string): string | null { + const match = HOSTED_BUN_VERSION_INPUT.exec(stripAnsi(log)); + return match?.[1] ?? null; +} + +/** Classifies a hosted job log with the same rules as a local run. */ +export function classifyHostedJobLog(log: string): BunRunClassification { + return classifyBunRun({ + exitCode: hostedJobExitCode(log), + signal: null, + output: log, + }); +} + +/** A closed signal name, or null for anything that is not one. */ +export function closedSignalName(signal: unknown): string | null { + return typeof signal === "string" && SIGNAL_NAME.test(signal) ? signal : null; +} + +/** A closed `major.minor.patch` Bun version, or null for anything else. */ +export function closedBunVersion(version: unknown): string | null { + return typeof version === "string" && BUN_VERSION.test(version) + ? version + : null; +} diff --git a/tests/bun-crash-loop.test.ts b/tests/bun-crash-loop.test.ts new file mode 100644 index 00000000..2f575c7d --- /dev/null +++ b/tests/bun-crash-loop.test.ts @@ -0,0 +1,402 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { mkdtemp, readdir, rm, utimes, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + collectDiagnosticReports, + type ExecutedRun, + executeWithLog, + type LoopCommand, + loopCommands, + parseLoopArguments, + planNextCommand, + renderLoopSummary, + runBunCrashLoop, +} from "../scripts/bun-crash-loop.ts"; +import { + classifyBunRun, + hasBunCrashBanner, + stripAnsi, +} from "../scripts/lib/bun-crash-classify.ts"; + +const CRASH_TRANSCRIPT = [ + "tests/native-https-owner.test.ts:", + "(pass) owner starts [12.00ms]", + "Bun v1.3.9 (cf6cdbbb) macOS Silicon", + "panic(main thread): Segmentation fault at address 0x80", + "oh no: Bun has crashed. This indicates a bug in Bun, not your code.", + " https://bun.report/1.3.9/Mt1cf6cdbbAAAA", +].join("\n"); + +const roots: string[] = []; +afterEach(async () => { + await Promise.all( + roots.splice(0).map((root) => rm(root, { recursive: true, force: true })) + ); +}); + +async function tempRoot(): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-bun-crash-loop-")); + roots.push(root); + return root; +} + +describe("classifyBunRun", () => { + test("a crash-banner transcript is a crash even when the exit looks ordinary", () => { + expect( + classifyBunRun({ exitCode: 1, signal: null, output: CRASH_TRANSCRIPT }) + ).toEqual({ outcome: "crash", reason: "banner" }); + expect( + classifyBunRun({ exitCode: 0, signal: null, output: CRASH_TRANSCRIPT }) + ).toEqual({ outcome: "crash", reason: "banner" }); + }); + + test("exit 133 or 139 without a banner is a signal crash", () => { + for (const exitCode of [133, 139]) { + expect(classifyBunRun({ exitCode, signal: null, output: "" })).toEqual({ + outcome: "crash", + reason: "signal_exit", + }); + } + }); + + test("a signal is a crash", () => { + expect( + classifyBunRun({ exitCode: null, signal: "SIGSEGV", output: "" }) + ).toEqual({ outcome: "crash", reason: "signal" }); + }); + + test("(fail) with exit 1 is a non-crash test failure", () => { + expect( + classifyBunRun({ + exitCode: 1, + signal: null, + output: "(fail) owner starts [5.00ms]\n\n 3 pass\n 1 fail\n", + }) + ).toEqual({ outcome: "failure", reason: "test_failure" }); + }); + + test("a typecheck exit 2 is a non-crash failure", () => { + expect( + classifyBunRun({ + exitCode: 2, + signal: null, + output: "src/index.ts(1,1): error TS2304: Cannot find name 'x'.\n", + }) + ).toEqual({ outcome: "failure", reason: "exit_code" }); + }); + + test("exit 0 is a pass", () => { + expect( + classifyBunRun({ + exitCode: 0, + signal: null, + output: " 12 pass\n 0 fail\n", + }) + ).toEqual({ outcome: "pass", reason: "exit_zero" }); + }); + + test("no exit status and no signal is a failure, not a pass", () => { + expect( + classifyBunRun({ exitCode: null, signal: null, output: "" }) + ).toEqual({ outcome: "failure", reason: "missing_exit" }); + }); + + test("ANSI-escaped banners and failure markers still match", () => { + const colored = + "\u001b[31mpanic\u001b[0m(main thread): Segmentation fault\n\u001b]8;;https://example.invalid\u0007link\u001b]8;;\u0007\n"; + expect(stripAnsi(colored)).toContain("panic(main thread)"); + expect(hasBunCrashBanner(colored)).toBe(true); + expect( + classifyBunRun({ exitCode: 1, signal: null, output: colored }) + ).toEqual({ outcome: "crash", reason: "banner" }); + expect( + classifyBunRun({ + exitCode: 1, + signal: null, + output: "\u001b[0m\u001b[31m(fail)\u001b[0m owner starts\n", + }) + ).toEqual({ outcome: "failure", reason: "test_failure" }); + }); +}); + +const commands: readonly LoopCommand[] = [ + { id: "focused", argv: ["focused"], initialEstimateMs: 100_000 }, + { id: "full", argv: ["full"], initialEstimateMs: 300_000 }, +]; + +describe("planNextCommand deadline arithmetic", () => { + const estimatesMs = { focused: 100_000, full: 300_000 }; + + test("a command that ends exactly at the deadline may start", () => { + expect( + planNextCommand({ + nowMs: 700_000, + deadlineMs: 1_000_000, + commands, + cursor: 1, + estimatesMs, + })?.command.id + ).toBe("full"); + }); + + test("a command that would end past the deadline falls back to one that fits", () => { + const next = planNextCommand({ + nowMs: 700_001, + deadlineMs: 1_000_000, + commands, + cursor: 1, + estimatesMs, + }); + expect(next?.command.id).toBe("focused"); + expect(next?.cursor).toBe(2); + }); + + test("nothing starts when no command fits or the deadline has passed", () => { + expect( + planNextCommand({ + nowMs: 900_001, + deadlineMs: 1_000_000, + commands, + cursor: 0, + estimatesMs, + }) + ).toBeNull(); + expect( + planNextCommand({ + nowMs: 1_000_000, + deadlineMs: 1_000_000, + commands, + cursor: 0, + estimatesMs: { focused: 0, full: 0 }, + }) + ).toBeNull(); + }); +}); + +describe("runBunCrashLoop", () => { + test("no iteration starts past the budget and estimates grow to observed durations", async () => { + let clock = 0; + const durations = { focused: 90_000, full: 500_000 }; + const starts: { id: string; at: number }[] = []; + const result = await runBunCrashLoop({ + budgetMs: 1_000_000, + commands, + now: () => clock, + execute: async (command) => { + starts.push({ id: command.id, at: clock }); + clock += durations[command.id]; + return { + exitCode: 0, + signal: null, + output: "", + stoppedAtDeadline: false, + }; + }, + }); + // After the full suite takes 500 s, its estimate is 500 s and it no longer + // fits; focused runs (estimate 100 s) continue until 950 s + 100 s > 1000 s. + expect(starts).toEqual([ + { id: "focused", at: 0 }, + { id: "full", at: 90_000 }, + { id: "focused", at: 590_000 }, + { id: "focused", at: 680_000 }, + { id: "focused", at: 770_000 }, + { id: "focused", at: 860_000 }, + ]); + for (const start of starts) { + const estimate = start.id === "full" ? 300_000 : 100_000; + expect(start.at + estimate).toBeLessThanOrEqual(1_000_000); + } + expect(result.estimatesMs).toEqual({ focused: 100_000, full: 500_000 }); + expect(result.iterations.map((record) => record.outcome)).toEqual( + new Array(6).fill("pass") + ); + }); + + test("a run stopped at the deadline is an overrun unless it printed the crash banner", async () => { + let clock = 0; + const runs: ExecutedRun[] = [ + { + exitCode: null, + signal: "SIGTERM", + output: "", + stoppedAtDeadline: true, + }, + { + exitCode: null, + signal: "SIGTERM", + output: CRASH_TRANSCRIPT, + stoppedAtDeadline: true, + }, + ]; + const result = await runBunCrashLoop({ + budgetMs: 1_000_000, + commands: [commands[0] as LoopCommand], + now: () => clock, + execute: async () => { + clock += 100_000; + return ( + runs.shift() ?? { + exitCode: 0, + signal: null, + output: "", + stoppedAtDeadline: false, + } + ); + }, + }); + expect(result.iterations[0]).toMatchObject({ + outcome: "overrun", + reason: "deadline", + signal: null, + }); + expect(result.iterations[1]).toMatchObject({ + outcome: "crash", + reason: "banner", + }); + }); + + test("the summary is closed and value-free", async () => { + let clock = 0; + const result = await runBunCrashLoop({ + budgetMs: 250_000, + commands, + now: () => clock, + execute: async () => { + clock += 100_000; + return { + exitCode: 133, + signal: null, + output: `TOKEN=do-not-print ${CRASH_TRANSCRIPT}`, + stoppedAtDeadline: false, + }; + }, + }); + const summary = renderLoopSummary({ + result, + bunVersion: "1.3.9", + sourceRevision: "2b8057c1ae6f4311150b6e139a9a919bbc7788d4", + diagnosticReports: 1, + }); + expect(summary).toContain("| focused | 2 | 0 | 0 | 2 | 0 |"); + expect(summary).toContain("| 1 | focused | crash | banner | 133 | - |"); + expect(summary).not.toContain("do-not-print"); + expect(summary).not.toContain("bun.report"); + expect(summary).not.toContain("native-https-owner"); + }); +}); + +describe("executeWithLog", () => { + test("keeps each run's log, reports signals and stops a run at the deadline", async () => { + const root = await tempRoot(); + const execute = executeWithLog({ cwd: root, logDir: root }); + const far = Date.now() + 60_000; + const pass = await execute( + { + id: "focused", + argv: [process.execPath, "-e", "console.log('ok')"], + initialEstimateMs: 1000, + }, + { index: 1, deadlineMs: far } + ); + expect(pass).toMatchObject({ exitCode: 0, stoppedAtDeadline: false }); + expect(pass.output).toContain("ok"); + + const crashed = await execute( + { + id: "focused", + argv: [process.execPath, "-e", "process.kill(process.pid, 'SIGSEGV')"], + initialEstimateMs: 1000, + }, + { index: 2, deadlineMs: far } + ); + expect(crashed.signal).toBe("SIGSEGV"); + expect(classifyBunRun(crashed).outcome).toBe("crash"); + + const stopped = await execute( + { + id: "full", + argv: [process.execPath, "-e", "setTimeout(() => {}, 60_000)"], + initialEstimateMs: 1000, + }, + { index: 3, deadlineMs: Date.now() + 200 } + ); + expect(stopped.stoppedAtDeadline).toBe(true); + expect((await readdir(root)).sort()).toEqual([ + "iteration-001-focused.log", + "iteration-002-focused.log", + "iteration-003-full.log", + ]); + }); +}); + +test("loop commands run the focused owner test and then the full suite", () => { + expect( + loopCommands({ + bunExecutable: "/bin/bun", + focusedEstimateSeconds: 60, + fullEstimateSeconds: 600, + }).map((command) => [command.id, command.argv, command.initialEstimateMs]) + ).toEqual([ + [ + "focused", + ["/bin/bun", "test", "tests/native-https-owner.test.ts"], + 60_000, + ], + ["full", ["/bin/bun", "test"], 600_000], + ]); +}); + +test("loop arguments are validated and closed", () => { + expect( + parseLoopArguments([ + "--budget-seconds=1800", + "--out-dir=/tmp/x", + "--expect-bun-version=1.3.9", + "--source-revision=2b8057c1ae6f4311150b6e139a9a919bbc7788d4", + ]) + ).toMatchObject({ + budgetSeconds: 1800, + outDir: "/tmp/x", + expectBunVersion: "1.3.9", + focusedEstimateSeconds: 120, + fullEstimateSeconds: 600, + }); + for (const argv of [ + [], + ["--budget-seconds=10"], + ["--budget-seconds=abc"], + ["--budget-seconds=1800", "--budget-seconds=1800"], + ["--budget-seconds=1800", "--unknown=1"], + ["--budget-seconds=1800", "--expect-bun-version=latest"], + ["--budget-seconds=1800", "--source-revision=main"], + ]) { + expect(() => parseLoopArguments(argv)).toThrow(); + } +}); + +test("only bun crash reports written during the loop are collected", async () => { + const root = await tempRoot(); + const source = join(root, "DiagnosticReports"); + const destination = join(root, "out"); + await Bun.write(join(source, "bun-2026-10-12-old.ips"), "{}"); + await writeFile(join(source, "bun-2026-10-12-new.ips"), "{}"); + await writeFile(join(source, "node-2026-10-12.ips"), "{}"); + const old = new Date(Date.now() - 3_600_000); + await utimes(join(source, "bun-2026-10-12-old.ips"), old, old); + const copied = await collectDiagnosticReports({ + source, + destination, + sinceMs: Date.now() - 60_000, + }); + expect(copied).toBe(1); + expect(await readdir(destination)).toEqual(["bun-2026-10-12-new.ips"]); + expect( + await collectDiagnosticReports({ + source: join(root, "missing"), + destination, + sinceMs: 0, + }) + ).toBe(0); +}); diff --git a/tests/ci-bun-crash-census.test.ts b/tests/ci-bun-crash-census.test.ts new file mode 100644 index 00000000..ca037363 --- /dev/null +++ b/tests/ci-bun-crash-census.test.ts @@ -0,0 +1,446 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { mkdtemp, readdir, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + aggregateCensus, + type CensusOptions, + type GhReader, + type JobRecord, + parseCensusArguments, + renderCensus, + runCensus, + workflowJobBunVersion, +} from "../scripts/ci-bun-crash-census.ts"; +import { + classifyHostedJobLog, + hostedJobBunVersion, + hostedJobExitCode, +} from "../scripts/lib/bun-crash-classify.ts"; + +const stamp = (line: string) => `2026-09-30T21:01:25.1622680Z ${line}`; + +function hostedLog(opts: { + readonly bunVersion: string; + readonly body: readonly string[]; + readonly exitCode: number | null; +}): string { + return [ + stamp("##[group]Run oven-sh/setup-bun@v1"), + stamp("with:"), + stamp(` bun-version: ${opts.bunVersion}`), + stamp("##[endgroup]"), + ...opts.body.map(stamp), + ...(opts.exitCode === null + ? [] + : [stamp(`##[error]Process completed with exit code ${opts.exitCode}.`)]), + ].join("\n"); +} + +const CRASH_LOG = hostedLog({ + bunVersion: "1.3.9", + body: [ + "\u001b[0m\u001b[2mBun v1.3.9 (cf6cdbbb) macOS Silicon\u001b[0m", + "\u001b[31mpanic(main thread)\u001b[0m: Segmentation fault at address 0x80", + "oh no: Bun has crashed. This indicates a bug in Bun, not your code.", + " https://bun.report/1.3.9/Mt1cf6cdbbAAAA", + "SECRET_LOOKING_VALUE=do-not-print", + ], + exitCode: 133, +}); +const TEST_FAILURE_LOG = hostedLog({ + bunVersion: "1.4.2", + body: ["(fail) owner starts [5.00ms]", " 1 fail"], + exitCode: 1, +}); +const TYPECHECK_LOG = hostedLog({ + bunVersion: "1.4.2", + body: ["src/index.ts(1,1): error TS2304: Cannot find name 'x'."], + exitCode: 2, +}); +const SIGNAL_EXIT_LOG = hostedLog({ + bunVersion: "1.3.9", + body: ["/bin/bash: line 1: 1243 Trace/BPT trap: 5 bun test"], + exitCode: 133, +}); + +describe("hosted job logs", () => { + test("a hosted crash transcript with timestamps and ANSI escapes is a crash", () => { + expect(classifyHostedJobLog(CRASH_LOG)).toEqual({ + outcome: "crash", + reason: "banner", + }); + expect(hostedJobExitCode(CRASH_LOG)).toBe(133); + expect(hostedJobBunVersion(CRASH_LOG)).toBe("1.3.9"); + }); + + test("exit 133 without a banner is a signal crash", () => { + expect(classifyHostedJobLog(SIGNAL_EXIT_LOG)).toEqual({ + outcome: "crash", + reason: "signal_exit", + }); + }); + + test("test failures and typecheck exit 2 are non-crash failures", () => { + expect(classifyHostedJobLog(TEST_FAILURE_LOG)).toEqual({ + outcome: "failure", + reason: "test_failure", + }); + expect(classifyHostedJobLog(TYPECHECK_LOG)).toEqual({ + outcome: "failure", + reason: "exit_code", + }); + }); + + test("a log without an exit annotation is not a pass", () => { + expect( + classifyHostedJobLog( + hostedLog({ bunVersion: "1.4.2", body: [], exitCode: null }) + ) + ).toEqual({ outcome: "failure", reason: "missing_exit" }); + }); +}); + +test("the test job's Bun version is read from its own workflow block", () => { + const workflow = [ + "jobs:", + " state-models:", + " steps:", + " - uses: oven-sh/setup-bun@v1", + " with:", + ' bun-version: "1.0.0"', + " test:", + " runs-on: blacksmith-6vcpu-macos-15", + " steps:", + " - uses: oven-sh/setup-bun@v1", + " with:", + ' bun-version: "1.3.9"', + " linux-process-lifetime:", + ' bun-version: "1.4.2"', + ].join("\n"); + expect(workflowJobBunVersion(workflow, "test")).toBe("1.3.9"); + expect(workflowJobBunVersion(workflow, "missing")).toBeNull(); + expect( + workflowJobBunVersion( + "jobs:\n test:\n other:\n bun-version: 1.4.2", + "test" + ) + ).toBeNull(); +}); + +const record = (overrides: Partial): JobRecord => ({ + runId: 1, + attempt: 1, + jobId: 1, + conclusion: "success", + bunVersion: "1.4.2", + versionSource: "workflow", + scan: null, + reason: null, + ...overrides, +}); + +test("totals aggregate per Bun version with crashing run and job ids", () => { + const totals = aggregateCensus([ + record({ jobId: 1 }), + record({ + jobId: 2, + conclusion: "failure", + scan: "failure", + reason: "exit_code", + versionSource: "log", + }), + record({ jobId: 3, conclusion: "cancelled" }), + record({ + runId: 7, + attempt: 2, + jobId: 4, + bunVersion: "1.3.9", + conclusion: "failure", + scan: "crash", + reason: "banner", + versionSource: "log", + }), + record({ jobId: 5, bunVersion: "1.3.9" }), + record({ + jobId: 6, + bunVersion: "1.10.0", + conclusion: "timed_out", + scan: "unavailable", + }), + record({ jobId: 8, bunVersion: "unknown", versionSource: "unknown" }), + ]); + expect(totals.map((entry) => entry.bunVersion)).toEqual([ + "1.3.9", + "1.4.2", + "1.10.0", + "unknown", + ]); + expect(totals[0]).toMatchObject({ + executions: 2, + successes: 1, + failures: 1, + crashes: 1, + logsScanned: 1, + crashing: [{ runId: 7, attempt: 2, jobId: 4, reason: "banner" }], + }); + expect(totals[1]).toMatchObject({ + executions: 3, + successes: 1, + failures: 1, + cancelled: 1, + crashes: 0, + versionSources: { log: 1, workflow: 2, unknown: 0 }, + }); + expect(totals[2]).toMatchObject({ failures: 1, logsUnavailable: 1 }); +}); + +const SHA_OLD = "a".repeat(40); +const SHA_NEW = "b".repeat(40); +const workflowFor = (version: string) => + Buffer.from( + `jobs:\n test:\n steps:\n - with:\n bun-version: "${version}"\n` + ).toString("base64"); + +function fakeGh(opts: { readonly total?: number } = {}) { + const downloads: string[] = []; + const destinations: string[] = []; + const logs: Record = { + 11: CRASH_LOG, + 21: TEST_FAILURE_LOG, + 22: "passing log body", + }; + const reader: GhReader = { + json: async (path) => { + if (path.includes("/runs?created=")) { + return { total_count: opts.total ?? 2 }; + } + if (path.endsWith(`?ref=${SHA_OLD}`)) { + return { content: workflowFor("1.3.9") }; + } + if (path.endsWith(`?ref=${SHA_NEW}`)) { + return { content: workflowFor("1.4.2") }; + } + throw new Error("unexpected path"); + }, + lines: async (path) => { + if (path.includes("/runs?created=")) { + return [ + JSON.stringify({ id: 100, head_sha: SHA_OLD }), + JSON.stringify({ id: 200, head_sha: SHA_NEW }), + ]; + } + if (path.includes("/runs/100/jobs?filter=all")) { + return [ + { + id: 11, + name: "test", + run_attempt: 1, + status: "completed", + conclusion: "failure", + }, + { + id: 12, + name: "test", + run_attempt: 2, + status: "completed", + conclusion: "success", + }, + { + id: 13, + name: "runtime-images", + run_attempt: 1, + status: "completed", + conclusion: "failure", + }, + ].map((job) => JSON.stringify(job)); + } + if (path.includes("/runs/200/jobs?filter=all")) { + return [ + { + id: 21, + name: "test", + run_attempt: 1, + status: "completed", + conclusion: "failure", + }, + { + id: 21, + name: "test", + run_attempt: 1, + status: "completed", + conclusion: "failure", + }, + { + id: 22, + name: "test", + run_attempt: 2, + status: "completed", + conclusion: "success", + }, + { + id: 23, + name: "test", + run_attempt: 3, + status: "in_progress", + conclusion: null, + }, + { + id: 24, + name: "test", + run_attempt: 3, + status: "completed", + conclusion: "skipped", + }, + ].map((job) => JSON.stringify(job)); + } + throw new Error("unexpected path"); + }, + download: async (path, destination) => { + const jobId = Number(/\/jobs\/(\d+)\/logs$/.exec(path)?.[1]); + downloads.push(path); + destinations.push(destination); + await Bun.write(destination, logs[jobId] ?? ""); + }, + }; + return { reader, downloads, destinations }; +} + +const options: CensusOptions = { + repository: "hack-dance/hack", + workflow: "ci.yml", + job: "test", + since: "2026-09-29T00:00:00Z", + until: "2026-10-01T17:35:00Z", + scanPassing: false, + concurrency: 2, +}; + +const workDirs: string[] = []; +afterEach(async () => { + await Promise.all( + workDirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true })) + ); +}); + +async function workDir(): Promise { + const dir = await mkdtemp(join(tmpdir(), "hack-bun-census-test-")); + workDirs.push(dir); + return dir; +} + +describe("runCensus", () => { + test("reads every attempt's test job, downloads only failed logs and removes them", async () => { + const dir = await workDir(); + const gh = fakeGh(); + const result = await runCensus(options, gh.reader, dir); + expect(gh.downloads).toEqual([ + "repos/hack-dance/hack/actions/jobs/11/logs", + "repos/hack-dance/hack/actions/jobs/21/logs", + ]); + expect(gh.destinations.every((path) => path.startsWith(dir))).toBe(true); + expect(await readdir(dir)).toEqual([]); + expect(result.runs).toBe(2); + expect(result.records.map((entry) => [entry.jobId, entry.attempt])).toEqual( + [ + [11, 1], + [12, 2], + [21, 1], + [22, 2], + ] + ); + expect(result.totals).toMatchObject([ + { + bunVersion: "1.3.9", + executions: 2, + failures: 1, + crashes: 1, + versionSources: { log: 1, workflow: 1, unknown: 0 }, + crashing: [{ runId: 100, attempt: 1, jobId: 11, reason: "banner" }], + }, + { bunVersion: "1.4.2", executions: 2, failures: 1, crashes: 0 }, + ]); + + const text = renderCensus({ options, result }); + expect(text).toContain( + "bun=1.3.9 executions=2 successes=1 failures=1 cancelled=0 crashes=1" + ); + expect(text).toContain(" crash run=100 attempt=1 job=11 reason=banner"); + expect(text).not.toContain("do-not-print"); + expect(text).not.toContain("bun.report"); + expect(text).not.toContain("Segmentation"); + }); + + test("--scan-passing also classifies passing logs", async () => { + const dir = await workDir(); + const gh = fakeGh(); + const result = await runCensus( + { ...options, scanPassing: true }, + gh.reader, + dir + ); + expect(gh.downloads).toHaveLength(4); + expect(await readdir(dir)).toEqual([]); + expect(result.totals[1]).toMatchObject({ + bunVersion: "1.4.2", + logsScanned: 2, + }); + }); + + test("a window beyond the runs endpoint limit is refused before any job read", async () => { + const gh = fakeGh({ total: 1001 }); + await expect( + runCensus(options, gh.reader, await workDir()) + ).rejects.toThrow("narrow it"); + expect(gh.downloads).toEqual([]); + }); +}); + +test("census arguments are validated and closed", () => { + expect( + parseCensusArguments([ + "--since", + "2026-10-01T17:35:00Z", + "--until=2026-10-11T23:43:30Z", + "--scan-passing", + ]) + ).toEqual({ + repository: "hack-dance/hack", + workflow: "ci.yml", + job: "test", + since: "2026-10-01T17:35:00Z", + until: "2026-10-11T23:43:30Z", + scanPassing: true, + json: false, + }); + for (const argv of [ + [], + ["--since", "2026-10-01", "--until", "2026-10-11T23:43:30Z"], + ["--since", "2026-10-11T23:43:30Z", "--until", "2026-10-01T17:35:00Z"], + [ + "--since", + "2026-10-01T17:35:00Z", + "--until", + "2026-10-11T23:43:30Z", + "--repo", + "x/y/z", + ], + [ + "--since", + "2026-10-01T17:35:00Z", + "--until", + "2026-10-11T23:43:30Z", + "--rerun", + ], + [ + "--since", + "2026-10-01T17:35:00Z", + "--until", + "2026-10-11T23:43:30Z", + "--json", + "--json", + ], + ]) { + expect(() => parseCensusArguments(argv)).toThrow(); + } +}); From 134287129db41c88acca308646c08772be9af058 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:43:38 -0400 Subject: [PATCH 24/52] perf(storage): read engine identity directly for docker context selections Storage-witness engine-ID admissions now resolve the Docker CLI's endpoint selection the way the CLI does: a nonempty DOCKER_HOST, then DOCKER_CONTEXT, then the client config's currentContext, then the default context at unix:///var/run/docker.sock. A named context is read from its contexts/meta//meta.json through a no-follow binding; only a plain unix:/// docker endpoint with no TLS option, no unknown or duplicate field and no entry at the context's TLS path is eligible. TLS material is checked for absence by name and never opened. Missing, malformed, linked, TCP, SSH, TLS or header-bearing contexts, and DOCKER_HOST with DOCKER_CONTEXT, keep the CLI. Admission is unchanged: the first fence under a binding is the CLI read and must equal one direct GET /info on the resolved socket. The binding now also pins the context store entry and the absent TLS path, so a changed config, context entry, socket or executable is admitted through the CLI again. The ingress and VM-file observers, trusted without a CLI comparison, still accept only an explicit Unix DOCKER_HOST. Tests that run commands against a synthetic docker now pin an isolated client config naming a missing context, so they never read the contributor's Docker config or dial a real engine socket. A new opt-in test (HACK_TEST_DOCKER_CLI=1, run on docker-e2e shard 4) runs the real Docker CLI against two fake Unix engines for every selection. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 4 + docs/reference/native-compose-ownership.md | 22 +- .../native-compose-storage-xattr-carrier.md | 9 +- src/lib/docker-context.ts | 144 ++++++ src/lib/native-compose-engine-config.ts | 264 +++++++--- src/lib/native-compose-engine-identity.ts | 130 +++-- ...compose-storage-witness-docker-identity.ts | 12 +- tests/docker-context.test.ts | 461 ++++++++++++++++++ tests/helpers/docker-client.ts | 29 ++ tests/helpers/native-compose-command.ts | 8 +- tests/native-compose-engine-identity.test.ts | 239 ++++++++- ...compose-storage-activation-command.test.ts | 2 + .../native-compose-storage-fence-cost.test.ts | 65 ++- 13 files changed, 1267 insertions(+), 122 deletions(-) create mode 100644 src/lib/docker-context.ts create mode 100644 tests/docker-context.test.ts create mode 100644 tests/helpers/docker-client.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 457db89f..695b9a0d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -300,6 +300,10 @@ jobs: - name: Verify owned bridge policy and aliases through real compiler and Compose normalization if: matrix.shard == 4 run: HACK_TEST_COMPOSE_CONFIG=1 bun test tests/native-compose-renderer-config.test.ts --test-name-pattern '^one authored bridge keeps internal policy and static aliases through compiler and Compose normalization$' + # Real Docker CLI against two fake Unix engines (isolated HOME and config; no real engine). + - name: Verify Docker context endpoint selection against the real Docker CLI + if: matrix.shard == 4 + run: HACK_TEST_DOCKER_CLI=1 bun test tests/docker-context.test.ts - name: Run local and Docker E2E shard env: HACK_E2E_PROCESS_POLICY_REFUSAL_CAPSULE: "1" diff --git a/docs/reference/native-compose-ownership.md b/docs/reference/native-compose-ownership.md index 23aa7819..452a252e 100644 --- a/docs/reference/native-compose-ownership.md +++ b/docs/reference/native-compose-ownership.md @@ -137,7 +137,27 @@ binding for the rest of the command. A different or failed admission read keeps the CLI for that binding, a changed binding is admitted through the CLI again, and an admitted direct failure refuses without fallback. -Real Unix-server controls live in `tests/native-compose-engine-identity.test.ts`; +Because each admission is checked against the CLI's own read, storage fences +also accept Docker context selections, which is how Docker Desktop and OrbStack +select their engine. The endpoint is resolved in the Docker CLI's order: a +nonempty `DOCKER_HOST`, then `DOCKER_CONTEXT`, then the client config's +`currentContext` (field names matched without case, like the CLI's decoder), +then the `default` context at `unix:///var/run/docker.sock`. A named context is +read from `contexts/meta//meta.json` under the config +directory; its `docker` endpoint must be a plain `unix:///` host with no TLS +option, no unknown or duplicate field, and no entry at the context's +`contexts/tls/` path. That TLS path is only checked for absence by name; +key material is never opened or listed. Missing, malformed, linked, TCP, SSH, +TLS or header-bearing contexts, and `DOCKER_HOST` together with +`DOCKER_CONTEXT`, keep the CLI. The binding adds the context store entry's path +identities and content digest and the absent TLS path, so any change to the +config, that entry, the socket path or the CLI executable is admitted through +the CLI again. The ingress and VM-file observers, which are trusted without a +CLI comparison, still accept only an explicit Unix `DOCKER_HOST`. + +Real Unix-server controls live in `tests/native-compose-engine-identity.test.ts` +and `tests/docker-context.test.ts` (with `HACK_TEST_DOCKER_CLI=1`, it also runs +the installed Docker CLI against two fake Unix engines for every selection); existing CLI-transport and proxy ownership controls remain in `tests/native-compose-ingress.test.ts`. These tests establish the bounded transport and refusal behavior. Actual Docker compatibility and performance require separate diff --git a/docs/reference/native-compose-storage-xattr-carrier.md b/docs/reference/native-compose-storage-xattr-carrier.md index 059d03e7..e9a000ef 100644 --- a/docs/reference/native-compose-storage-xattr-carrier.md +++ b/docs/reference/native-compose-storage-xattr-carrier.md @@ -232,11 +232,14 @@ runs; three exact costs are removed. Each change is local to one CLI process, which runs exactly one native command. - Engine-ID fences (`docker info --format '{{json .ID}}'`) in the Docker - carrier use command-scoped admission for an explicit `DOCKER_HOST=unix:///...` - selection that the ingress transport also admits (see + carrier use command-scoped admission for a selection the Docker CLI resolves + to a plain Unix socket: an explicit `DOCKER_HOST=unix:///...`, + `DOCKER_CONTEXT`, the client config's current context, or the `default` + context (see [ownership](native-compose-ownership.md#ingress-daemon-identity-transport)). The first fence under one exact binding (selection values, `docker` - executable, client config and socket identities) is the Docker CLI read and + executable, client config, context store entry and socket identities) is the + Docker CLI read and must equal one direct `GET /info` on that socket. Only then does each later fence under the unchanged binding make one fresh direct request instead of a CLI process that also lists every client plugin. A different or failed diff --git a/src/lib/docker-context.ts b/src/lib/docker-context.ts new file mode 100644 index 00000000..fd51c3df --- /dev/null +++ b/src/lib/docker-context.ts @@ -0,0 +1,144 @@ +import { isAbsolute, join, normalize } from "node:path"; +import { isRecord } from "./guards.ts"; +import { + bindNativeComposeEngineAbsence, + bindNativeComposeEngineDocument, + foldNativeComposeEngineFields, + nativeComposeDockerConfigDirectory, +} from "./native-compose-engine-config.ts"; + +type Environment = Readonly>; + +/** The Docker CLI's built-in default endpoint on Unix (`client.DefaultDockerHost`). */ +export const DOCKER_DEFAULT_UNIX_SOCKET = "/var/run/docker.sock"; +/** The reserved name the CLI resolves from its flags and environment, never the store. */ +const DEFAULT_CONTEXT = "default"; +/** docker/cli `ValidateContextName`; other spellings keep the CLI. */ +const CONTEXT_NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.+-]+$/; +const AMBIGUOUS_PATH = /[%?#\0\r\n]/; + +/** + * The Unix socket the Docker CLI would dial for one selection, and the + * value-free facts that pin how it was selected. The facts are compared and + * digested only; they are never displayed. + */ +export type DockerContextEndpoint = { + readonly socketPath: string; + readonly binding: unknown; +}; + +/** + * Path of an unambiguous `unix:///absolute/path` host, else null. The CLI uses + * the text after `unix://` verbatim; spellings it would trim, decode or + * normalize are refused so that no other spelling can name a different socket. + */ +export function dockerUnixHostPath(host: string): string | null { + if (!host.startsWith("unix:///")) { + return null; + } + const path = host.slice("unix://".length); + return AMBIGUOUS_PATH.test(path) || + path.trim() !== path || + !isAbsolute(path) || + normalize(path) !== path + ? null + : path; +} + +/** sha256 of the context name: the store's directory name for that context. */ +function contextId(name: string): string { + return new Bun.CryptoHasher("sha256").update(name).digest("hex"); +} + +/** + * The docker endpoint of a context's `meta.json`, matched the way the CLI's Go + * decoder matches it: top-level and endpoint fields without case, endpoint map + * keys exactly. Unknown fields, duplicate spellings, TLS options and any host + * other than an unambiguous Unix socket keep the CLI. + */ +function inspectMeta(value: Record): string | null { + const fields = foldNativeComposeEngineFields(value, [ + "name", + "metadata", + "endpoints", + ]); + const name = fields?.get("name"); + const metadata = fields?.get("metadata"); + const endpoints = fields?.get("endpoints"); + if ( + fields === null || + (name !== undefined && typeof name !== "string") || + !(metadata === undefined || metadata === null || isRecord(metadata)) || + !isRecord(endpoints) || + !Object.hasOwn(endpoints, "docker") + ) { + return null; + } + const docker = endpoints.docker; + const endpoint = isRecord(docker) + ? foldNativeComposeEngineFields(docker, ["host", "skiptlsverify"]) + : null; + const host = endpoint?.get("host"); + const skip = endpoint?.get("skiptlsverify"); + return typeof host === "string" && (skip === undefined || skip === false) + ? dockerUnixHostPath(host) + : null; +} + +/** + * Resolve the endpoint exactly as the Docker CLI does for a flagless command, + * then return it only when it is a plain Unix socket this transport can read: + * + * 1. a non-empty DOCKER_HOST selects the default context with that host (with a + * non-empty DOCKER_CONTEXT too, the selection is treated as ambiguous); + * 2. else a non-empty DOCKER_CONTEXT names the context; + * 3. else the client config's `currentContext` names it; + * 4. else the `default` context, whose endpoint is `defaultSocketPath`. + * + * A named context is read from `/contexts/meta//meta.json` + * through a no-follow binding. Its TLS directory must be absent: it is proven + * absent by name and never opened, so no key material is read. A missing, + * malformed, TLS, TCP, SSH or otherwise ineligible context returns null, and + * the caller keeps the Docker CLI. The caller must have already refused TLS, + * certificate, header and API-version environment selections, and must + * re-resolve and compare the binding around every read. + */ +export function resolveDockerContextEndpoint(opts: { + readonly environment: Environment; + readonly currentContext: string | null; + readonly defaultSocketPath: string; +}): DockerContextEndpoint | null { + const { environment } = opts; + if (environment.DOCKER_HOST) { + const path = environment.DOCKER_CONTEXT + ? null + : dockerUnixHostPath(environment.DOCKER_HOST); + return path === null ? null : { socketPath: path, binding: ["host"] }; + } + const name = + environment.DOCKER_CONTEXT || opts.currentContext || DEFAULT_CONTEXT; + if (name === DEFAULT_CONTEXT) { + return { socketPath: opts.defaultSocketPath, binding: [DEFAULT_CONTEXT] }; + } + const directory = nativeComposeDockerConfigDirectory(environment); + if (directory === null || !CONTEXT_NAME.test(name)) { + return null; + } + const id = contextId(name); + const meta = bindNativeComposeEngineDocument({ + path: join(directory, "contexts", "meta", id, "meta.json"), + inspect: inspectMeta, + }); + if (meta === null || !meta.found) { + return null; + } + const tls = bindNativeComposeEngineAbsence( + join(directory, "contexts", "tls", id) + ); + return tls === null + ? null + : { + socketPath: meta.value, + binding: ["context", name, meta.binding, tls], + }; +} diff --git a/src/lib/native-compose-engine-config.ts b/src/lib/native-compose-engine-config.ts index 5a3e7f1d..58a0ba58 100644 --- a/src/lib/native-compose-engine-config.ts +++ b/src/lib/native-compose-engine-config.ts @@ -11,15 +11,21 @@ import { isAbsolute, join, normalize, parse } from "node:path"; import { isRecord } from "./guards.ts"; import { parseImportDocument } from "./native-config-import-parser.ts"; -const CONFIG_LIMIT = 1024 * 1024; +const DOCUMENT_LIMIT = 1024 * 1024; const ASCII_FIELD = /^[\x20-\x7e]+$/; +type Environment = Readonly>; type Entry = { readonly path: string; readonly identity: readonly number[] }; -type ConfigBinding = { +/** Named-path facts of one document: each component identity, then content. */ +type DocumentBinding = { readonly path: string; readonly entries: readonly Entry[]; readonly missing: string | null; readonly sha256: string | null; }; +type ConfigBinding = DocumentBinding & { + /** The client config's `currentContext`, or null when absent or empty. */ + readonly currentContext: string | null; +}; function identity(info: Stats): readonly number[] { return [ @@ -36,8 +42,60 @@ function identity(info: Stats): readonly number[] { function same(left: readonly number[], right: readonly number[]): boolean { return JSON.stringify(left) === JSON.stringify(right); } -function eligibleFile(path: string, named: Stats): string | null { - if (!(named.isFile() && named.size <= CONFIG_LIMIT)) { + +/** + * lstat every named component from the root without following any link. A + * missing component ends the walk with that component recorded; a link, a + * non-directory ancestor or any other error makes the path ineligible. + */ +function walk(path: string): { + readonly entries: readonly Entry[]; + readonly missing: string | null; + readonly leaf: Stats | null; +} | null { + if (!isAbsolute(path) || normalize(path) !== path) { + return null; + } + const root = parse(path).root; + let current = root; + const entries: Entry[] = []; + for (const component of ["", ...path.slice(root.length).split("/")]) { + current = component ? join(current, component) : current; + let info: Stats; + try { + info = lstatSync(current); + } catch (error: unknown) { + if (isRecord(error) && error.code === "ENOENT") { + return { entries, missing: current, leaf: null }; + } + return null; + } + if (info.isSymbolicLink()) { + return null; + } + entries.push({ path: current, identity: identity(info) }); + if (current === path) { + return { entries, missing: null, leaf: info }; + } + if (!info.isDirectory()) { + return null; + } + } + return null; +} + +/** + * Read one bounded strict JSON object through a no-follow descriptor whose + * identity equals the named entry before and after the read. The existing + * strict parser rejects decoded duplicate keys; every object key must be + * printable ASCII so that case-insensitive field matching is exact. + */ +function readDocument( + path: string, + named: Stats, + inspect: (value: Record) => T | null +): { readonly sha256: string; readonly value: T } | null { + if (!(named.isFile() && named.size <= DOCUMENT_LIMIT)) { return null; } const fd = openSync( @@ -66,31 +124,123 @@ function eligibleFile(path: string, named: Stats): string | null { } const data = bytes.subarray(0, size); const text = new TextDecoder("utf-8", { fatal: true }).decode(data); - // The existing bounded strict parser rejects decoded duplicate keys. Go's - // Docker decoder also matches field names without case, so inspect that set. const { value } = parseImportDocument({ text, document: "config" }); if (!value || Object.keys(value).some((key) => !ASCII_FIELD.test(key))) { return null; } - const headers = Object.entries(value).filter( - ([key]) => key.toLowerCase() === "httpheaders" - ); + const inspected = inspect(value); + return inspected === null + ? null + : { + sha256: new Bun.CryptoHasher("sha256").update(data).digest("hex"), + value: inspected, + }; + } finally { + closeSync(fd); + } +} + +/** + * Go's JSON decoder matches struct fields without case, and a later duplicate + * spelling overwrites (or merges into) an earlier one. Return the fields by + * folded name only when every key is printable ASCII and no folded name repeats. + * With `allowed`, any other folded name is ineligible too. + */ +export function foldNativeComposeEngineFields( + value: Record, + allowed?: readonly string[] +): ReadonlyMap | null { + const fields = new Map(); + for (const [key, field] of Object.entries(value)) { + const folded = key.toLowerCase(); if ( - headers.length > 1 || - headers.some( - ([, value]) => !isRecord(value) || Object.keys(value).length > 0 - ) + !ASCII_FIELD.test(key) || + fields.has(folded) || + (allowed !== undefined && !allowed.includes(folded)) ) { return null; } - return new Bun.CryptoHasher("sha256").update(data).digest("hex"); - } finally { - closeSync(fd); + fields.set(folded, field); + } + return fields; +} + +/** + * Bind one named JSON document without following links. Returns null when the + * path or document is ineligible (including when `inspect` returns null), + * `found: false` with the first missing component, or the inspected value. + * Never returns diagnostics or document text; the digest is compared only. + */ +export function bindNativeComposeEngineDocument(opts: { + readonly path: string; + readonly inspect: (value: Record) => T | null; +}): + | { readonly binding: DocumentBinding; readonly found: false } + | { + readonly binding: DocumentBinding; + readonly found: true; + readonly value: T; + } + | null { + try { + const named = walk(opts.path); + if (named === null) { + return null; + } + if (named.leaf === null) { + return { + binding: { + path: opts.path, + entries: named.entries, + missing: named.missing, + sha256: null, + }, + found: false, + }; + } + const document = readDocument(opts.path, named.leaf, opts.inspect); + return document === null + ? null + : { + binding: { + path: opts.path, + entries: named.entries, + missing: null, + sha256: document.sha256, + }, + found: true, + value: document.value, + }; + } catch { + return null; + } +} + +/** + * Prove a named path absent without following links or reading anything below + * it: the binding records each existing ancestor and the first missing + * component. Any existing entry at the path, a link or an error returns null. + */ +export function bindNativeComposeEngineAbsence(path: string): { + readonly entries: readonly Entry[]; + readonly missing: string; +} | null { + try { + const named = walk(path); + return named === null || named.missing === null + ? null + : { entries: named.entries, missing: named.missing }; + } catch { + return null; } } -function configPath( - environment: Readonly> +/** + * The Docker CLI's client config directory: DOCKER_CONFIG, else HOME/.docker. + * Relative or unnormalized spellings return null and keep the CLI. + */ +export function nativeComposeDockerConfigDirectory( + environment: Environment ): string | null { const directory = environment.DOCKER_CONFIG || @@ -98,54 +248,54 @@ function configPath( return directory && isAbsolute(directory) && normalize(directory) === directory - ? join(directory, "config.json") + ? directory : null; } +function inspectConfig( + value: Record +): { readonly currentContext: string | null } | null { + const fields = foldNativeComposeEngineFields(value); + if (fields === null) { + return null; + } + const headers = fields.get("httpheaders"); + if ( + headers !== undefined && + !(isRecord(headers) && Object.keys(headers).length === 0) + ) { + return null; + } + const current = fields.get("currentcontext"); + if (current !== undefined && typeof current !== "string") { + return null; + } + return { currentContext: current || null }; +} + /** * Eligibility only: configured headers, ambiguous paths and unreadable inputs - * keep Docker's CLI transport. Never return parsed configuration or diagnostics. - * Once admitted, compare this named-path/content binding before and after reads; - * a changed or newly ineligible config must refuse, never select another transport. + * keep Docker's CLI transport. Never return parsed configuration or diagnostics + * beyond the selected context name. Once admitted, compare this named-path and + * content binding before and after reads; a changed or newly ineligible config + * must refuse, never select another transport. */ export function bindNativeComposeEngineConfig( - environment: Readonly> + environment: Environment ): ConfigBinding | null { - try { - const path = configPath(environment); - if (path === null) { - return null; - } - const root = parse(path).root; - let current = root; - const entries: Entry[] = []; - for (const component of ["", ...path.slice(root.length).split("/")]) { - current = component ? join(current, component) : current; - let info: Stats; - try { - info = lstatSync(current); - } catch (error: unknown) { - if (isRecord(error) && error.code === "ENOENT") { - return { path, entries, missing: current, sha256: null }; - } - return null; - } - if (info.isSymbolicLink()) { - return null; - } - entries.push({ path: current, identity: identity(info) }); - if (current === path) { - const sha256 = eligibleFile(path, info); - return sha256 === null - ? null - : { path, entries, missing: null, sha256 }; - } - if (!info.isDirectory()) { - return null; - } - } + const directory = nativeComposeDockerConfigDirectory(environment); + if (directory === null) { return null; - } catch { + } + const bound = bindNativeComposeEngineDocument({ + path: join(directory, "config.json"), + inspect: inspectConfig, + }); + if (bound === null) { return null; } + return { + ...bound.binding, + currentContext: bound.found ? bound.value.currentContext : null, + }; } diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index 63618b2c..d9b31f73 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -1,6 +1,11 @@ import { lstatSync, readlinkSync, type Stats } from "node:fs"; import { request } from "node:http"; import { dirname, isAbsolute, join, normalize, parse } from "node:path"; +import { + DOCKER_DEFAULT_UNIX_SOCKET, + dockerUnixHostPath, + resolveDockerContextEndpoint, +} from "./docker-context.ts"; import { isRecord } from "./guards.ts"; import { bindNativeComposeEngineConfig } from "./native-compose-engine-config.ts"; @@ -150,29 +155,47 @@ export function bindNativeComposeEngineSocketPath(path: string): { function check(signal: AbortSignal | undefined, deadline: number): void { requireValue(!signal?.aborted && Date.now() < deadline); } -function localEndpoint(environment: Environment): string | null { - if ( +/** TLS, certificate, header and API-version selections always keep the CLI. */ +function transportSelected(environment: Environment): boolean { + return Boolean( process.platform === "win32" || - environment.DOCKER_CONTEXT || - environment.DOCKER_TLS || - environment.DOCKER_TLS_VERIFY || - environment.DOCKER_CUSTOM_HEADERS || - environment.DOCKER_CERT_PATH || - environment.DOCKER_API_VERSION - ) { - return null; - } + environment.DOCKER_TLS || + environment.DOCKER_TLS_VERIFY || + environment.DOCKER_CUSTOM_HEADERS || + environment.DOCKER_CERT_PATH || + environment.DOCKER_API_VERSION + ); +} +/** An explicit Unix DOCKER_HOST with no context selection, else null. */ +function explicitEndpoint(environment: Environment): string | null { const host = environment.DOCKER_HOST; - if (!host?.startsWith("unix:///")) { - return null; - } - const path = host.slice("unix://".length); // Ambiguous URL spelling keeps the existing Docker CLI selection semantics. - return AMBIGUOUS_PATH.test(path) || - !isAbsolute(path) || - normalize(path) !== path + return transportSelected(environment) || environment.DOCKER_CONTEXT || !host ? null - : path; + : dockerUnixHostPath(host); +} +type Contexts = { readonly defaultSocketPath: string } | null; +/** + * Endpoint selection under one config binding. Explicit observers accept only an + * explicit Unix DOCKER_HOST; admissions resolve the CLI's full context + * selection. Both return null for every other selection. + */ +function selectEndpoint(opts: { + readonly environment: Environment; + readonly currentContext: string | null; + readonly contexts: Contexts; +}): { readonly socketPath: string; readonly binding: unknown } | null { + if (opts.contexts === null) { + const path = explicitEndpoint(opts.environment); + return path === null ? null : { socketPath: path, binding: ["host"] }; + } + return transportSelected(opts.environment) + ? null + : resolveDockerContextEndpoint({ + environment: opts.environment, + currentContext: opts.currentContext, + defaultSocketPath: opts.contexts.defaultSocketPath, + }); } function readId(opts: { @@ -282,12 +305,16 @@ function bindObserver(opts: { readonly signal?: AbortSignal; readonly timeoutMs?: number; readonly environment?: Environment; + readonly contexts: Contexts; }): BoundObserver | null { const environment = opts.environment ?? process.env; const signal = opts.signal; const selected = selection(environment); - const path = localEndpoint(environment); - if (path === null) { + if ( + opts.contexts === null + ? explicitEndpoint(environment) === null + : transportSelected(environment) + ) { return null; } try { @@ -301,6 +328,17 @@ function bindObserver(opts: { if (config === null) { return null; } + const select = () => + selectEndpoint({ + environment, + currentContext: config.currentContext, + contexts: opts.contexts, + }); + const endpoint = select(); + if (endpoint === null) { + return null; + } + const path = endpoint.socketPath; const binary = Bun.which("docker", { PATH: environment.PATH ?? "" }); requireValue(binary); const socket = bindPath(path, "socket"); @@ -319,13 +357,18 @@ function bindObserver(opts: { JSON.stringify(config) === JSON.stringify(bindNativeComposeEngineConfig(environment)) ); + // An unchanged config can still name another socket through a changed + // context store entry or a newly created context TLS directory. + requireValue(JSON.stringify(endpoint) === JSON.stringify(select())); requireValue(sameBinding(socket, bindPath(path, "socket"))); requireValue(sameBinding(executable, bindPath(binary, "executable"))); check(signal, deadline); }; recheck(); const key = new Bun.CryptoHasher("sha256") - .update(JSON.stringify([selected, binary, config, socket, executable])) + .update( + JSON.stringify([selected, binary, config, endpoint, socket, executable]) + ) .digest("hex"); return Object.freeze({ key, @@ -360,9 +403,11 @@ function bindObserver(opts: { /** * Replace only `docker info` transport for an explicit local Unix selection. * Every call makes a fresh bounded daemon-ID request; no result is cached. Other - * selections return null before admission and retain their existing CLI path. - * Admitted errors never fall back. Socket, executable and selection bindings are - * checked before and after each request under the caller's existing time budget. + * selections, including every Docker context selection, return null before + * admission and retain their existing CLI path: this observer is trusted + * without a CLI comparison. Admitted errors never fall back. Socket, executable + * and selection bindings are checked before and after each request under the + * caller's existing time budget. */ export function createNativeComposeEngineIdentityObserver( opts: { @@ -371,31 +416,43 @@ export function createNativeComposeEngineIdentityObserver( readonly environment?: Environment; } = {} ): (() => Promise) | null { - return bindObserver(opts)?.read ?? null; + return bindObserver({ ...opts, contexts: null })?.read ?? null; } /** * Command-scoped transport for repeated `docker info --format '{{json .ID}}'` * fences. Each fence still makes exactly one fresh engine-ID read; no ID is cached. * + * Eligible selections are those the Docker CLI resolves to a plain Unix socket: + * an explicit Unix DOCKER_HOST, DOCKER_CONTEXT, the client config's + * `currentContext`, or the `default` context (`resolveDockerContextEndpoint`). * Admission is per exact binding: the selection values, `docker` executable path - * and identity, client-config path/content and socket path identities. The first - * fence under an eligible binding returns the caller's own Docker CLI read and - * compares it with one direct `GET /info` on the same explicit Unix socket. Only - * an equal ID admits that binding; a direct failure or a different ID keeps the - * CLI for that binding for the rest of the command. Later fences under an admitted - * binding use one fresh direct request each, and its failures refuse without CLI - * fallback. A changed binding is a new binding and is admitted through the CLI - * again. Ineligible selections, and bindings that cannot be established, always - * use the CLI. The returned text has the CLI's `{{json .ID}}` line shape. + * and identity, client-config path/content, the selected context store entry + * and absent TLS directory, and socket path identities. The first fence under + * an eligible binding returns the caller's own Docker CLI read and compares it + * with one direct `GET /info` on the socket that selection names. Only an equal + * ID admits that binding; a direct failure or a different ID keeps the CLI for + * that binding for the rest of the command. Later fences under an admitted + * binding use one fresh direct request each, and its failures refuse without + * CLI fallback. A changed binding is a new binding and is admitted through the + * CLI again. Ineligible selections, and bindings that cannot be established, + * always use the CLI. The returned text has the CLI's `{{json .ID}}` line shape. + * + * `defaultSocketPath` is the `default` context's endpoint; only tests replace + * the CLI's built-in default. */ -export function createNativeComposeEngineIdentityAdmissions(): (opts: { +export function createNativeComposeEngineIdentityAdmissions( + options: { readonly defaultSocketPath?: string } = {} +): (opts: { readonly cli: () => Promise; readonly signal?: AbortSignal; readonly timeoutMs: number; readonly environment: Environment; }) => Promise { const admitted = new Map(); + const contexts = { + defaultSocketPath: options.defaultSocketPath ?? DOCKER_DEFAULT_UNIX_SOCKET, + }; return async (opts) => { let bound: BoundObserver | null; try { @@ -403,6 +460,7 @@ export function createNativeComposeEngineIdentityAdmissions(): (opts: { signal: opts.signal, timeoutMs: opts.timeoutMs, environment: opts.environment, + contexts, }); } catch { bound = null; diff --git a/src/lib/native-compose-storage-witness-docker-identity.ts b/src/lib/native-compose-storage-witness-docker-identity.ts index b77d53f2..4280d8a5 100644 --- a/src/lib/native-compose-storage-witness-docker-identity.ts +++ b/src/lib/native-compose-storage-witness-docker-identity.ts @@ -17,11 +17,13 @@ function identityQuery(args: readonly string[]): boolean { /** * Storage-witness probe whose exact engine-ID query keeps one fresh read per * fence but may use the command's admitted direct Unix transport - * (`createNativeComposeEngineIdentityAdmissions`). Every other query, every - * ineligible selection and every unadmitted binding goes to the supplied CLI - * probe unchanged. The direct read shares this probe's signal and deadline, and - * the same environment snapshot the CLI probe captured. Recovery observation - * keeps its own CLI probes. + * (`createNativeComposeEngineIdentityAdmissions`): an explicit Unix DOCKER_HOST + * or a Docker context selection (DOCKER_CONTEXT, the client config's current + * context, or the default context) that the CLI resolves to a Unix socket. + * Every other query, every ineligible selection and every unadmitted binding + * goes to the supplied CLI probe unchanged. The direct read shares this probe's + * signal and deadline, and the same environment snapshot the CLI probe + * captured. Recovery observation keeps its own CLI probes. */ export function nativeComposeStorageIdentityProbe(opts: { readonly probe: Probe; diff --git a/tests/docker-context.test.ts b/tests/docker-context.test.ts new file mode 100644 index 00000000..c85a6f75 --- /dev/null +++ b/tests/docker-context.test.ts @@ -0,0 +1,461 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { createHash } from "node:crypto"; +import { + mkdir, + mkdtemp, + realpath, + rename, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { createServer, type Server } from "node:http"; +import type { Socket } from "node:net"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { + DOCKER_DEFAULT_UNIX_SOCKET, + resolveDockerContextEndpoint, +} from "../src/lib/docker-context.ts"; +import { bindNativeComposeEngineConfig } from "../src/lib/native-compose-engine-config.ts"; +import { createNativeComposeEngineIdentityAdmissions } from "../src/lib/native-compose-engine-identity.ts"; + +/** + * Two fake Unix engines answer both the Docker CLI (`HEAD /_ping`, then a + * versioned `GET /v1.N/info`) and the direct transport (unversioned + * `GET /info`). Each records which requests it served, so a test can prove + * which socket each transport dialed. + */ +type Engine = { readonly path: string; readonly served: string[] }; +type Environment = Record; +const CANARY = "docker-context-private-canary"; +let root: string; +let config: string; +let engines: Record<"a" | "b", Engine>; +let servers: Server[]; +let peers: Set; + +async function engine(name: string): Promise { + const path = join(root, `${name}.sock`); + const served: string[] = []; + const server = createServer((req, res) => { + served.push(`${req.method} ${req.url}`); + res.setHeader("Api-Version", "1.47"); + res.setHeader("Ostype", "linux"); + if (req.url?.endsWith("/_ping")) { + res.end("OK"); + } else if (req.url?.endsWith("/info")) { + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify({ ID: `engine-${name}`, Name: CANARY })); + } else { + res.statusCode = 404; + res.end("{}"); + } + }); + server.on("connection", (peer) => peers.add(peer)); + servers.push(server); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(path, resolve); + }); + return { path, served }; +} +beforeEach(async () => { + root = await realpath(await mkdtemp(join(tmpdir(), "docker-context-"))); + config = join(root, "client"); + await mkdir(config, { mode: 0o700 }); + servers = []; + peers = new Set(); + engines = { a: await engine("a"), b: await engine("b") }; + // The shape `docker context create --docker host=unix://...` writes. + await context("ctx-a", { + Host: `unix://${engines.a.path}`, + SkipTLSVerify: false, + }); + await context("ctx-b", { + Host: `unix://${engines.b.path}`, + SkipTLSVerify: false, + }); +}); +afterEach(async () => { + for (const peer of peers) { + peer.destroy(); + } + for (const server of servers) { + await new Promise((resolve) => server.close(() => resolve())); + } + await rm(root, { recursive: true, force: true }); +}); + +function id(name: string): string { + return createHash("sha256").update(name).digest("hex"); +} +/** Write a context store entry the way `docker context create` lays it out. */ +async function context( + name: string, + docker: Record | null, + raw?: string +) { + const directory = join(config, "contexts", "meta", id(name)); + await mkdir(directory, { recursive: true, mode: 0o755 }); + await writeFile( + join(directory, "meta.json"), + raw ?? + JSON.stringify({ + Name: name, + Metadata: { Description: "synthetic" }, + Endpoints: { docker }, + }), + { mode: 0o644 } + ); +} +async function current(name: string | null, raw?: string) { + await writeFile( + join(config, "config.json"), + raw ?? + JSON.stringify({ + auths: { synthetic: { auth: CANARY } }, + ...(name === null ? {} : { currentContext: name }), + }), + { mode: 0o600 } + ); +} +function base(extra: Environment = {}): Environment { + return { HOME: root, DOCKER_CONFIG: config, ...extra }; +} +/** The socket this transport would read, or null when the CLI keeps the selection. */ +function resolved(environment: Environment): string | null { + const bound = bindNativeComposeEngineConfig(environment); + if (bound === null) { + return null; + } + return ( + resolveDockerContextEndpoint({ + environment, + currentContext: bound.currentContext, + defaultSocketPath: DOCKER_DEFAULT_UNIX_SOCKET, + })?.socketPath ?? null + ); +} + +// Each row is one selection and the socket the Docker CLI dials for it, as +// docker/cli resolveContextName orders them: DOCKER_HOST, DOCKER_CONTEXT, the +// config's currentContext, then `default`. Empty variables count as unset. +const SELECTIONS: readonly { + readonly name: string; + readonly current: string | null; + readonly env: (engines: Record<"a" | "b", Engine>) => Environment; + readonly cli: "a" | "b" | "default"; + /** False only where the CLI resolves but this transport keeps the CLI. */ + readonly eligible: boolean; +}[] = [ + { + name: "no selection", + current: null, + env: () => ({}), + cli: "default", + eligible: true, + }, + { + name: "currentContext", + current: "ctx-b", + env: () => ({}), + cli: "b", + eligible: true, + }, + { + name: "DOCKER_CONTEXT over currentContext", + current: "ctx-b", + env: () => ({ DOCKER_CONTEXT: "ctx-a" }), + cli: "a", + eligible: true, + }, + { + name: "DOCKER_HOST over currentContext", + current: "ctx-b", + env: (e) => ({ DOCKER_HOST: `unix://${e.a.path}` }), + cli: "a", + eligible: true, + }, + { + name: "empty DOCKER_HOST and DOCKER_CONTEXT are unset", + current: "ctx-b", + env: () => ({ DOCKER_HOST: "", DOCKER_CONTEXT: "" }), + cli: "b", + eligible: true, + }, + { + name: "DOCKER_CONTEXT=default over currentContext", + current: "ctx-b", + env: () => ({ DOCKER_CONTEXT: "default" }), + cli: "default", + eligible: true, + }, + { + name: "currentContext=default", + current: "default", + env: () => ({}), + cli: "default", + eligible: true, + }, + { + name: "DOCKER_HOST with DOCKER_CONTEXT", + current: "ctx-b", + env: (e) => ({ + DOCKER_HOST: `unix://${e.a.path}`, + DOCKER_CONTEXT: "ctx-b", + }), + cli: "a", + eligible: false, + }, +]; +function socketFor(target: "a" | "b" | "default"): string { + return target === "default" + ? DOCKER_DEFAULT_UNIX_SOCKET + : engines[target].path; +} + +test("selection precedence names the same socket the Docker CLI dials", async () => { + for (const selection of SELECTIONS) { + await current(selection.current); + expect({ + name: selection.name, + socket: resolved(base(selection.env(engines))), + }).toEqual({ + name: selection.name, + socket: selection.eligible ? socketFor(selection.cli) : null, + }); + } + // A missing config file is the CLI's empty config: the default context. + await rm(join(config, "config.json")); + expect(resolved(base())).toBe(DOCKER_DEFAULT_UNIX_SOCKET); + expect([...engines.a.served, ...engines.b.served]).toEqual([]); +}); + +test("currentContext is matched without case, like the CLI's config decoder", async () => { + for (const spelling of [ + "currentContext", + "CurrentContext", + "currentcontext", + ]) { + await current(null, JSON.stringify({ [spelling]: "ctx-a" })); + expect(resolved(base())).toBe(engines.a.path); + } + for (const raw of [ + '{"currentContext":"ctx-a","CurrentContext":"ctx-b"}', + '{"currentContext":"ctx-a","currentContext":"ctx-b"}', + '{"currentContext":7}', + '{"currentContext":null}', + '{"currentContexſ":"ctx-a"}', + ]) { + await current(null, raw); + expect(resolved(base())).toBeNull(); + } +}); + +test("a context store entry is read only as an unambiguous plain Unix endpoint", async () => { + await current("ctx-x"); + const eligible = (docker: Record | null, raw?: string) => + context("ctx-x", docker, raw).then(() => resolved(base())); + expect(await eligible({ Host: `unix://${engines.a.path}` })).toBe( + engines.a.path + ); + expect( + await eligible({ host: `unix://${engines.a.path}`, skiptlsverify: false }) + ).toBe(engines.a.path); + for (const docker of [ + { Host: "tcp://127.0.0.1:2375" }, + { Host: "ssh://synthetic@example.invalid" }, + { Host: "npipe:////./pipe/docker_engine" }, + { Host: "unix://relative.sock" }, + { Host: `unix://${engines.a.path}?x=1` }, + { Host: `unix://${root}/./a.sock` }, + { Host: `unix://${engines.a.path} ` }, + { Host: `unix://${engines.a.path}%00` }, + { Host: "" }, + { Host: null }, + {}, + { Host: `unix://${engines.a.path}`, SkipTLSVerify: true }, + { + Host: `unix://${engines.a.path}`, + AdditionalHeaders: { Private: CANARY }, + }, + { Host: `unix://${engines.a.path}`, host: `unix://${engines.b.path}` }, + ]) { + expect(await eligible(docker)).toBeNull(); + } + const host = `unix://${engines.a.path}`; + for (const raw of [ + JSON.stringify({ Name: "ctx-x", Endpoints: { Docker: { Host: host } } }), + JSON.stringify({ Name: "ctx-x", Endpoints: { docker: null } }), + JSON.stringify({ Name: "ctx-x", Endpoints: null }), + JSON.stringify({ + Name: "ctx-x", + Endpoints: { docker: { Host: host } }, + endpoints: {}, + }), + JSON.stringify({ + Name: "ctx-x", + Extra: true, + Endpoints: { docker: { Host: host } }, + }), + JSON.stringify({ Name: 1, Endpoints: { docker: { Host: host } } }), + JSON.stringify({ + Name: "ctx-x", + Metadata: [], + Endpoints: { docker: { Host: host } }, + }), + `{"Endpoints":{"docker":{"Host":"${host}"},"docker":{"Host":"${host}"}}}`, + "[]", + CANARY, + ]) { + expect(await eligible(null, raw)).toBeNull(); + } + // Other endpoints in the same context are ignored by the CLI's docker client. + expect( + await eligible( + null, + JSON.stringify({ + Name: "ctx-x", + Endpoints: { docker: { Host: host }, kubernetes: { Host: CANARY } }, + }) + ) + ).toBe(engines.a.path); +}); + +test("missing, unnamed or invalid contexts keep the CLI", async () => { + for (const name of ["missing", "x", "-leading", "a/b", "a b", "ctx-a\n"]) { + expect(resolved(base({ DOCKER_CONTEXT: name }))).toBeNull(); + } + // "Default" is a store name, not the reserved default context. + expect(resolved(base({ DOCKER_CONTEXT: "Default" }))).toBeNull(); + for (const environment of [ + { HOME: undefined, DOCKER_CONFIG: undefined, DOCKER_CONTEXT: "ctx-a" }, + { HOME: root, DOCKER_CONFIG: "client", DOCKER_CONTEXT: "ctx-a" }, + { HOME: root, DOCKER_CONFIG: `${root}/./client`, DOCKER_CONTEXT: "ctx-a" }, + ]) { + expect(resolved(environment)).toBeNull(); + } +}); + +test("context TLS material keeps the CLI without opening or listing it", async () => { + const tls = join(config, "contexts", "tls", id("ctx-a")); + await current("ctx-a"); + expect(resolved(base())).toBe(engines.a.path); + await mkdir(join(tls, "docker"), { recursive: true, mode: 0o700 }); + // A FIFO blocks any reader until a writer opens it: a read would hang here. + const fifo = Bun.spawnSync(["mkfifo", join(tls, "docker", "key.pem")]); + expect(fifo.exitCode).toBe(0); + expect(resolved(base())).toBeNull(); + await rm(tls, { recursive: true }); + // Any entry at the TLS path, even an empty file, keeps the CLI. + await mkdir(dirname(tls), { recursive: true }); + await writeFile(tls, ""); + expect(resolved(base())).toBeNull(); + await rm(tls); + expect(resolved(base())).toBe(engines.a.path); +}); + +test("linked store paths keep the CLI", async () => { + await current("ctx-a"); + const meta = join(config, "contexts", "meta", id("ctx-a"), "meta.json"); + await rename(meta, join(root, "meta.json")); + await symlink(join(root, "meta.json"), meta); + expect(resolved(base())).toBeNull(); + await rm(meta); + await rename(join(root, "meta.json"), meta); + expect(resolved(base())).toBe(engines.a.path); + await rename(join(config, "contexts"), join(root, "contexts")); + await symlink(join(root, "contexts"), join(config, "contexts")); + expect(resolved(base())).toBeNull(); +}); + +// Real Docker CLI agreement. Opt-in (HACK_TEST_DOCKER_CLI=1): it runs the +// installed `docker` against the two fake engines only, with an isolated HOME +// and DOCKER_CONFIG, and never contacts a real engine. +const dockerCliTest = + process.env.HACK_TEST_DOCKER_CLI === "1" ? test : test.skip; +function dockerEnvironment(extra: Environment): Environment { + const docker = Bun.which("docker"); + if (!docker) { + throw new Error("HACK_TEST_DOCKER_CLI=1 requires the docker CLI on PATH"); + } + return base({ PATH: dirname(docker), ...extra }); +} +async function docker(environment: Environment, args: readonly string[]) { + const child = Bun.spawn(["docker", ...args], { + env: Object.fromEntries( + Object.entries(environment).filter( + (entry): entry is [string, string] => entry[1] !== undefined + ) + ), + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, code] = await Promise.all([ + new Response(child.stdout).text(), + child.exited, + ]); + if (code !== 0) { + throw new Error("docker CLI selection failed"); + } + return stdout; +} + +dockerCliTest( + "the direct read targets the same socket the real Docker CLI picks for each selection", + async () => { + for (const selection of SELECTIONS) { + await current(selection.current); + const environment = dockerEnvironment(selection.env(engines)); + // Client-only resolution: the CLI prints the endpoint without dialing it. + expect( + ( + await docker(environment, [ + "context", + "inspect", + "--format", + "{{.Endpoints.docker.Host}}", + ]) + ).trim() + ).toBe(`unix://${socketFor(selection.cli)}`); + if (selection.cli === "default") { + continue; + } + for (const fake of Object.values(engines)) { + fake.served.length = 0; + } + const admissions = createNativeComposeEngineIdentityAdmissions({ + defaultSocketPath: join(root, "unused-default.sock"), + }); + const fence = () => + admissions({ + cli: () => docker(environment, ["info", "--format", "{{json .ID}}"]), + timeoutMs: 30_000, + environment, + }); + const picked = engines[selection.cli]; + const other = engines[selection.cli === "a" ? "b" : "a"]; + // The first fence is the CLI's own read, on the CLI's own socket. + expect(await fence()).toBe(`"engine-${selection.cli}"\n`); + expect( + picked.served.some((line) => /^GET \/v[\d.]+\/info$/.test(line)) + ).toBe(true); + expect(await fence()).toBe(`"engine-${selection.cli}"\n`); + const direct = picked.served.filter((line) => line === "GET /info"); + expect({ name: selection.name, direct: direct.length }).toEqual({ + name: selection.name, + // Admission read plus one fresh read for the admitted second fence. + direct: selection.eligible ? 2 : 0, + }); + // The second fence never ran the CLI when admitted; the other engine was + // never dialed by either transport. + expect( + picked.served.filter((line) => line.endsWith("/_ping")).length + ).toBe(selection.eligible ? 1 : 2); + expect(other.served).toEqual([]); + } + }, + 120_000 +); diff --git a/tests/helpers/docker-client.ts b/tests/helpers/docker-client.ts new file mode 100644 index 00000000..0130ec58 --- /dev/null +++ b/tests/helpers/docker-client.ts @@ -0,0 +1,29 @@ +import { existsSync, mkdirSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; + +/** A context name with no store entry: the Docker CLI itself would refuse it. */ +export const ISOLATED_DOCKER_CONTEXT = "hack-test-unselected"; + +/** + * Docker client selection for commands that run against a synthetic `docker`: + * an isolated client config whose current context does not exist. The storage + * engine-ID admission then keeps the synthetic CLI. It never reads the + * contributor's Docker config or context store, and never dials the default + * engine socket or a real context's socket. An explicit `DOCKER_HOST` or + * `DOCKER_CONFIG` passed by a test still takes precedence. + */ +export function isolatedDockerClientEnv( + root: string +): Readonly> { + const directory = join(root, "docker-client"); + const config = join(directory, "config.json"); + if (!existsSync(config)) { + mkdirSync(directory, { recursive: true, mode: 0o700 }); + writeFileSync( + config, + `${JSON.stringify({ currentContext: ISOLATED_DOCKER_CONTEXT })}\n`, + { mode: 0o600 } + ); + } + return { DOCKER_CONFIG: directory }; +} diff --git a/tests/helpers/native-compose-command.ts b/tests/helpers/native-compose-command.ts index 33dc84cc..b928ec72 100644 --- a/tests/helpers/native-compose-command.ts +++ b/tests/helpers/native-compose-command.ts @@ -2,6 +2,7 @@ import { afterEach, expect } from "bun:test"; import { chmod, mkdir, mkdtemp, realpath, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; +import { isolatedDockerClientEnv } from "./docker-client.ts"; import { nativeComposeStorageDockerFixtureScript } from "./native-compose-storage-docker.ts"; const roots: string[] = []; @@ -227,7 +228,11 @@ export async function invoke( args = ["up", "--detach", "--json"], startupTimeoutMs = 1000, commandTimeoutMs = 15_000, - /** Docker client selection only (for example an explicit Unix DOCKER_HOST). */ + /** + * Docker client selection only (for example an explicit Unix DOCKER_HOST). + * Without one, the command uses an isolated client config that selects no + * real engine. + */ extraEnv: Readonly> = {} ) { const child = Bun.spawn( @@ -252,6 +257,7 @@ export async function invoke( HACK_COMPOSE_STARTUP_TIMEOUT_MS: String(startupTimeoutMs), CI: "1", HACK_EXECUTION_MODE: "non_interactive", + ...isolatedDockerClientEnv(root), ...extraEnv, }, stdin: "ignore", diff --git a/tests/native-compose-engine-identity.test.ts b/tests/native-compose-engine-identity.test.ts index 6b64461d..98d7b933 100644 --- a/tests/native-compose-engine-identity.test.ts +++ b/tests/native-compose-engine-identity.test.ts @@ -1,4 +1,5 @@ import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; +import { createHash } from "node:crypto"; import { chmod, mkdir, @@ -679,8 +680,15 @@ function fence( ) { return admissions({ cli: cli.read, timeoutMs: 5000, environment }); } +/** + * No test dials the CLI's built-in default socket: the `default` context's + * endpoint is a fixture path, absent unless a test listens on it. + */ +function createAdmissions(defaultSocketPath = join(root, "default.sock")) { + return createNativeComposeEngineIdentityAdmissions({ defaultSocketPath }); +} test("storage fences answer the first one with the CLI and admit only an equal direct ID", async () => { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); expect([cli.calls(), requests.length]).toEqual([1, 1]); @@ -691,7 +699,7 @@ test("storage fences answer the first one with the CLI and admit only an equal d expect([cli.calls(), requests.length, connections.size]).toEqual([1, 4, 4]); }); test("a changed daemon ID between two admitted fences is returned, never cached", async () => { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); await fence(admissions, cli); expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); @@ -713,7 +721,7 @@ test("a different or failed admission read keeps the CLI for that binding", asyn ]) { handler = reply; requests = []; - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); for (let index = 0; index < 3; index++) { expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); @@ -722,7 +730,7 @@ test("a different or failed admission read keeps the CLI for that binding", asyn } }); test("admitted direct failures refuse without CLI fallback", async () => { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); await fence(admissions, cli); handler = (_req, res) => { @@ -734,7 +742,7 @@ test("admitted direct failures refuse without CLI fallback", async () => { }); test("a moved config, socket or executable binding is admitted through the CLI again", async () => { const path = await config("{}"); - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); await fence(admissions, cli); await fence(admissions, cli); @@ -757,11 +765,14 @@ test("a moved config, socket or executable binding is admitted through the CLI a }); test("ineligible or unbindable selections and CLI admission failures stay on the CLI", async () => { for (const selection of [ + // The default context, whose fixture socket is absent. { DOCKER_HOST: undefined }, + // DOCKER_HOST with DOCKER_CONTEXT is treated as ambiguous. { DOCKER_CONTEXT: "selected-context" }, + { DOCKER_HOST: undefined, DOCKER_CONTEXT: "selected-context" }, { DOCKER_HOST: `unix://${join(root, "missing.sock")}` }, ]) { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const cli = cliStandIn(); for (let index = 0; index < 2; index++) { expect( @@ -775,7 +786,7 @@ test("ineligible or unbindable selections and CLI admission failures stay on the expect(cli.calls()).toBe(2); } expect(requests).toHaveLength(0); - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const failure = new Error("synthetic CLI fence failure"); await expect( admissions({ @@ -818,7 +829,7 @@ async function withSelectedProcessEnvironment(body: () => Promise) { } test("cancelling an admitted direct fence keeps the CLI probe's closed cancellation without a process", async () => { await withSelectedProcessEnvironment(async () => { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); await fence(admissions, cliStandIn()); const controller = new AbortController(); const probe = nativeComposeStorageIdentityProbe({ @@ -846,7 +857,7 @@ test("cancelling an admitted direct fence keeps the CLI probe's closed cancellat test("storage identity probe routes only the exact engine-ID query", async () => { const routed: string[] = []; await withSelectedProcessEnvironment(async () => { - const admissions = createNativeComposeEngineIdentityAdmissions(); + const admissions = createAdmissions(); const probe = nativeComposeStorageIdentityProbe({ probe: (args) => { routed.push(args.join(" ")); @@ -871,3 +882,213 @@ test("storage identity probe routes only the exact engine-ID query", async () => expect(requests).toHaveLength(2); }); }); + +// Docker context selections (DOCKER_HOST unset): the default context, the +// client config's currentContext and DOCKER_CONTEXT. Only admissions resolve +// them; the explicit observer, trusted without a CLI comparison, never does. +const CLIENT_CONTEXT = "fixture-context"; +function contextId(name: string): string { + return createHash("sha256").update(name).digest("hex"); +} +async function storeContext(name: string, host: string): Promise { + const directory = join(root, "client", "contexts", "meta", contextId(name)); + await mkdir(directory, { recursive: true }); + const path = join(directory, "meta.json"); + await writeFile( + path, + JSON.stringify({ + Name: name, + Metadata: {}, + Endpoints: { docker: { Host: host, SkipTLSVerify: false } }, + }) + ); + return path; +} +/** Select the fixture engine through the client config's currentContext. */ +async function contextSelection(): Promise<{ + readonly config: string; + readonly meta: string; +}> { + environment.DOCKER_HOST = undefined; + const meta = await storeContext(CLIENT_CONTEXT, `unix://${socket}`); + const path = await config(JSON.stringify({ currentContext: CLIENT_CONTEXT })); + return { config: path, meta }; +} +/** A second engine with its own request count. */ +async function otherEngine(id: string) { + const path = join(root, "other.sock"); + let served = 0; + const server = createServer((_req, res) => { + served++; + res.end(JSON.stringify({ ID: id })); + }); + server.on("connection", (peer) => connections.add(peer)); + servers.push(server); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(path, resolve); + }); + return { path, served: () => served }; +} + +test("context selections admit through the CLI, then read that context's socket", async () => { + await contextSelection(); + for (const selection of [ + {}, + // DOCKER_CONTEXT names the same store entry over an unrelated currentContext. + { DOCKER_CONTEXT: CLIENT_CONTEXT }, + ]) { + if ("DOCKER_CONTEXT" in selection) { + await config(JSON.stringify({ currentContext: "unrelated-context" })); + } + requests = []; + const admissions = createAdmissions(); + const cli = cliStandIn(); + for (let index = 0; index < 4; index++) { + expect( + await admissions({ + cli: cli.read, + timeoutMs: 5000, + environment: { ...environment, ...selection }, + }) + ).toBe(`"${ENGINE}"\n`); + } + expect([cli.calls(), requests.length]).toEqual([1, 4]); + } +}); +test("the default context admits its own endpoint, never a store entry named default", async () => { + environment.DOCKER_HOST = undefined; + await config("{}"); + const admissions = createAdmissions(socket); + const cli = cliStandIn(); + for (let index = 0; index < 3; index++) { + await fence(admissions, cli); + } + expect([cli.calls(), requests.length]).toEqual([1, 3]); + const other = await otherEngine(ENGINE); + await storeContext("default", `unix://${other.path}`); + environment.DOCKER_CONTEXT = "default"; + const named = createAdmissions(socket); + await fence(named, cli); + await fence(named, cli); + expect([cli.calls(), requests.length, other.served()]).toEqual([2, 5, 0]); +}); +test("DOCKER_HOST takes precedence: a current context never redirects the direct read", async () => { + const other = await otherEngine("other-engine"); + await storeContext(CLIENT_CONTEXT, `unix://${other.path}`); + await config(JSON.stringify({ currentContext: CLIENT_CONTEXT })); + const admissions = createAdmissions(); + const cli = cliStandIn(); + for (let index = 0; index < 3; index++) { + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + } + expect([cli.calls(), requests.length, other.served()]).toEqual([1, 3, 0]); +}); +test("a context naming another engine than the CLI's read is never admitted", async () => { + const { meta } = await contextSelection(); + const other = await otherEngine("other-engine"); + await writeFile( + meta, + JSON.stringify({ + Name: CLIENT_CONTEXT, + Endpoints: { docker: { Host: `unix://${other.path}` } }, + }) + ); + const admissions = createAdmissions(); + const cli = cliStandIn(); + for (let index = 0; index < 3; index++) { + expect(await fence(admissions, cli)).toBe(`"${ENGINE}"\n`); + } + expect([cli.calls(), requests.length, other.served()]).toEqual([3, 0, 1]); +}); +test("a changed config, context entry, TLS directory, socket or executable admits through the CLI again", async () => { + const selected = await contextSelection(); + const admissions = createAdmissions(); + const cli = cliStandIn(); + const twice = async (counts: readonly [number, number]) => { + await fence(admissions, cli); + await fence(admissions, cli); + expect([cli.calls(), requests.length]).toEqual([...counts]); + }; + await twice([1, 2]); + // Same bytes under a new inode: the config, then the context store entry. + for (const [index, path] of [selected.config, selected.meta].entries()) { + const bytes = await readFile(path); + await rename(path, join(root, `previous-${index}.json`)); + await writeFile(path, bytes); + await twice([2 + index, 4 + 2 * index]); + } + // Published TLS material keeps the CLI while present; it is never opened. + const tls = join( + root, + "client", + "contexts", + "tls", + contextId(CLIENT_CONTEXT) + ); + await mkdir(join(tls, "docker"), { recursive: true }); + await writeFile(join(tls, "docker", "key.pem"), CANARY, { mode: 0o000 }); + await twice([5, 6]); + await rm(tls, { recursive: true }); + await twice([6, 8]); + await rename(socket, join(root, "original.sock")); + await listen(socket); + await twice([7, 10]); + await rename(docker, join(root, "original-docker")); + await writeFile(docker, "#!/bin/sh\nexit 98\n", { mode: 0o700 }); + await twice([8, 12]); +}); +test("a context store change during an admitted read refuses without fallback", async () => { + const { meta } = await contextSelection(); + const admissions = createAdmissions(); + const cli = cliStandIn(); + await fence(admissions, cli); + handler = (_req, res) => { + void (async () => { + await writeFile( + meta, + JSON.stringify({ + Name: CLIENT_CONTEXT, + Endpoints: { docker: { Host: `unix://${join(root, "other.sock")}` } }, + }) + ); + res.end(JSON.stringify({ ID: ENGINE })); + })(); + }; + await refusal(() => fence(admissions, cli)); + expect([cli.calls(), requests.length]).toEqual([1, 2]); +}); +test("TLS, header and API-version environment keeps context selections on the CLI", async () => { + await contextSelection(); + for (const selection of [ + { DOCKER_TLS: "1" }, + { DOCKER_TLS_VERIFY: "1" }, + { DOCKER_CERT_PATH: join(root, "certificates") }, + { DOCKER_CUSTOM_HEADERS: `Private=${CANARY}` }, + { DOCKER_API_VERSION: "1.44" }, + ]) { + for (const context of [{}, { DOCKER_CONTEXT: "default" }]) { + const admissions = createAdmissions(socket); + const cli = cliStandIn(); + for (let index = 0; index < 2; index++) { + await admissions({ + cli: cli.read, + timeoutMs: 5000, + environment: { ...environment, ...selection, ...context }, + }); + } + expect(cli.calls()).toBe(2); + } + } + expect(requests).toHaveLength(0); +}); +test("the explicit observer never resolves a context or the default context", async () => { + await contextSelection(); + expect(createNativeComposeEngineIdentityObserver({ environment })).toBeNull(); + environment.DOCKER_CONTEXT = CLIENT_CONTEXT; + expect(createNativeComposeEngineIdentityObserver({ environment })).toBeNull(); + await config("{}"); + environment.DOCKER_CONTEXT = undefined; + expect(createNativeComposeEngineIdentityObserver({ environment })).toBeNull(); + expect(requests).toHaveLength(0); +}); diff --git a/tests/native-compose-storage-activation-command.test.ts b/tests/native-compose-storage-activation-command.test.ts index 93267d2d..e7c5fae6 100644 --- a/tests/native-compose-storage-activation-command.test.ts +++ b/tests/native-compose-storage-activation-command.test.ts @@ -2,6 +2,7 @@ import { afterEach, expect, test } from "bun:test"; import { join, resolve } from "node:path"; import { openNativeComposeGenerationStore } from "../src/lib/native-compose-generation.ts"; import { captureCompletedJobFixtureCommand } from "./e2e/scenarios/native-compose-adoption-job-worktrees.ts"; +import { isolatedDockerClientEnv } from "./helpers/docker-client.ts"; import { fixture, invoke } from "./helpers/native-compose-command.ts"; const storage = { data: { kind: "persistent", scope: "worktree" } } as const; @@ -82,6 +83,7 @@ async function ownedColdCase( HACK_COMPOSE_STARTUP_TIMEOUT_MS: "30000", CI: "1", HACK_EXECUTION_MODE: "non_interactive", + ...isolatedDockerClientEnv(root), }, captures: join(root, `storage-capture-${capture++}`), timeoutMs: 120_000, diff --git a/tests/native-compose-storage-fence-cost.test.ts b/tests/native-compose-storage-fence-cost.test.ts index 3f096511..c0322ce8 100644 --- a/tests/native-compose-storage-fence-cost.test.ts +++ b/tests/native-compose-storage-fence-cost.test.ts @@ -1,4 +1,5 @@ import { afterEach, expect, test } from "bun:test"; +import { createHash } from "node:crypto"; import { mkdir, writeFile } from "node:fs/promises"; import { createServer, type Server } from "node:http"; import type { Socket } from "node:net"; @@ -33,15 +34,46 @@ afterEach(async () => { await new Promise((resolve) => server.close(() => resolve())); } }); -/** A real Unix `GET /info` endpoint for the same engine the synthetic CLI reports. */ +/** + * A real Unix `GET /info` endpoint for the same engine the synthetic CLI reports, + * selected by an explicit DOCKER_HOST or, with `context`, by the client config's + * currentContext naming a context store entry for that socket. + */ async function engine( root: string, - id: (request: number) => string = () => "fixture-engine" + id: (request: number) => string = () => "fixture-engine", + selection: "host" | "context" = "host" ) { - const socket = join(root, "engine.sock"); - const client = join(root, "client"); + const socket = join(root, `${selection}-engine.sock`); + const client = join(root, `${selection}-client`); await mkdir(client, { mode: 0o700 }); - await writeFile(join(client, "config.json"), "{}\n", { mode: 0o600 }); + await writeFile( + join(client, "config.json"), + selection === "host" + ? "{}\n" + : `${JSON.stringify({ currentContext: "fixture-context" })}\n`, + { mode: 0o600 } + ); + if (selection === "context") { + const meta = join( + client, + "contexts", + "meta", + createHash("sha256").update("fixture-context").digest("hex") + ); + await mkdir(meta, { recursive: true, mode: 0o755 }); + await writeFile( + join(meta, "meta.json"), + JSON.stringify({ + Name: "fixture-context", + Metadata: {}, + Endpoints: { + docker: { Host: `unix://${socket}`, SkipTLSVerify: false }, + }, + }), + { mode: 0o644 } + ); + } let served = 0; const server = createServer((req, res) => { const request = served++; @@ -58,10 +90,11 @@ async function engine( server.once("error", reject); server.listen(socket, resolve); }); - return { - env: { DOCKER_HOST: `unix://${socket}`, DOCKER_CONFIG: client }, - served: () => served, - }; + const env: Readonly> = + selection === "host" + ? { DOCKER_HOST: `unix://${socket}`, DOCKER_CONFIG: client } + : { DOCKER_CONFIG: client }; + return { env, served: () => served }; } async function lines(root: string, name: string): Promise { const file = Bun.file(join(root, name)); @@ -111,7 +144,8 @@ async function enrolled() { test("storage fence count per command: every proof kept, exact duplicates gone, one CLI admission", async () => { const root = await enrolled(); - // CLI transport (no explicit Unix selection): every fence is a `docker info`. + // CLI transport (the isolated client selects a context with no store entry): + // every fence is a `docker info`. expect(await counted(root, ["exec", "web", "--", "true"])).toEqual({ code: 0, cli: FILE_ENGINE_FENCES + 2 * PROOF_FENCES, @@ -139,6 +173,17 @@ test("storage fence count per command: every proof kept, exact duplicates gone, direct: 6 * PROOF_FENCES, observations: 6 * PROOF_OBSERVATIONS, }); + // Admitted context selection (DOCKER_HOST unset; the client config's current + // context names the socket): the same counts as an explicit Unix host. + const contextual = await engine(root, undefined, "context"); + expect( + await counted(root, ["exec", "web", "--", "true"], contextual) + ).toEqual({ + code: 0, + cli: FILE_ENGINE_FENCES + 1, + direct: 2 * PROOF_FENCES, + observations: 2 * PROOF_OBSERVATIONS, + }); }, 390_000); test("a refused admission keeps every storage fence on the Docker CLI for the command", async () => { From c2ecf9cb321d3865d1d5d0ab059a225e5b5904b8 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:47:12 -0400 Subject: [PATCH 25/52] chore(x04): temporary witness probe trace for hosted measurement Temporary measurement only; reverted before the final commit. Re-applies the X03 closed-label trace (6a4e118e) without its identity benchmark, labels each engine-ID admission with its selection kind (host, default or context), counts direct reads as trace leaves, and prints the runner's Docker client selection. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 25 ++ scripts/x01-trace-summary.ts | 285 ++++++++++++++++++ src/lib/native-compose-command-storage.ts | 14 +- src/lib/native-compose-command.ts | 9 +- src/lib/native-compose-engine-identity.ts | 18 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- ...tive-compose-storage-command-executable.ts | 4 + ...ompose-storage-witness-docker-inventory.ts | 8 +- .../native-compose-storage-witness-docker.ts | 45 ++- src/lib/native-compose-storage-witness.ts | 34 ++- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 176 +++++++++++ tests/e2e/harness.ts | 9 + .../e2e/native-config-networks-acceptance.ts | 4 + 15 files changed, 629 insertions(+), 32 deletions(-) create mode 100644 scripts/x01-trace-summary.ts create mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 695b9a0d..419fb1a1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -304,13 +304,22 @@ jobs: - name: Verify Docker context endpoint selection against the real Docker CLI if: matrix.shard == 4 run: HACK_TEST_DOCKER_CLI=1 bun test tests/docker-context.test.ts + # TEMPORARY X04 measurement; reverted before the final commit. + - name: X04 runner Docker client selection + run: | + docker context inspect --format 'context={{.Name}} host={{.Endpoints.docker.Host}}' || true + if [ -f "$HOME/.docker/config.json" ]; then echo "client-config=present"; else echo "client-config=absent"; fi + echo "DOCKER_HOST set=${DOCKER_HOST:+yes} DOCKER_CONTEXT set=${DOCKER_CONTEXT:+yes} DOCKER_CONFIG set=${DOCKER_CONFIG:+yes}" - name: Run local and Docker E2E shard env: HACK_E2E_PROCESS_POLICY_REFUSAL_CAPSULE: "1" HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 + # TEMPORARY X01 measurement; reverted before the final commit. + HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | + mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -319,6 +328,22 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" + # TEMPORARY X01 measurement; reverted before the final commit. + - name: X01 witness cost summary + if: always() + run: | + for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do + [ -f "$f" ] || continue + bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true + done + - name: X01 witness cost trace artifact + if: always() + uses: actions/upload-artifact@v4 + with: + name: x01-trace-shard-${{ matrix.shard }} + path: ${{ runner.temp }}/x01-trace + if-no-files-found: ignore + retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts new file mode 100644 index 00000000..62ace753 --- /dev/null +++ b/scripts/x01-trace-summary.ts @@ -0,0 +1,285 @@ +// TEMPORARY X01 measurement summary. Reverted before the final commit. +// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. +type Event = { + readonly p: number; + readonly v: string; + readonly k: string; + readonly s?: string; + readonly l?: string; + readonly ms?: number; + readonly t?: number; + readonly c?: number; + readonly line?: { + readonly phase?: string; + readonly boundary?: string; + readonly durationMs?: number | null; + }; +}; +const WITNESS = new Set([ + "storage-prepare", + "storage-verify", + "carrier-new", + "inspect", + "provision", + "finish", + "witness-verify", + "witness-prepare", + "witness-enroll", +]); +const isWitness = (label: string) => + WITNESS.has(label) || label.startsWith("invoke:"); +// TEMPORARY X04: direct engine-ID reads (`ident:direct`, `ident:admission`) are leaves too. +const LEAF = new Set(["probe", "run", "exec", "hash", "ident"]); +const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; + +const path = process.argv[2]; +const title = process.argv[3] ?? path; +if (!path) { + throw new Error("usage: x01-trace-summary [title]"); +} +const text = await Bun.file(path).text(); +const events: Event[] = []; +for (const line of text.split("\n")) { + if (line.trim() === "") { + continue; + } + try { + events.push(JSON.parse(line) as Event); + } catch { + // skip partial line + } +} +const byProcess = new Map(); +for (const event of events) { + const list = byProcess.get(event.p) ?? []; + list.push(event); + byProcess.set(event.p, list); +} +type Proc = { + readonly pid: number; + readonly verb: string; + readonly totalMs: number; + readonly witnessMs: number; + readonly events: Event[]; + readonly kind: string; +}; +function topWitnessMs(list: Event[]): number { + let total = 0; + for (const event of list) { + if (event.k !== "scope" || !event.l || !isWitness(event.l)) { + continue; + } + const parts = (event.s ?? "").split(">"); + parts.pop(); + if (parts.some(isWitness)) { + continue; + } + total += event.ms ?? 0; + } + return total; +} +const procs: Proc[] = []; +for (const [pid, list] of byProcess) { + const exit = list.find((event) => event.k === "exit"); + const verb = list[0]?.v ?? "other"; + const labels = new Set(list.map((event) => event.l ?? "")); + let kind = verb; + if (verb === "up" || verb === "restart" || verb === "run") { + if (labels.has("invoke:seed")) { + kind = `${verb}(cold)`; + } else if (labels.has("invoke:verify")) { + kind = `${verb}(warm)`; + } else { + kind = `${verb}(no-witness)`; + } + } else if (verb === "exec") { + kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; + } + procs.push({ + pid, + verb, + totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), + witnessMs: topWitnessMs(list), + events: list, + kind, + }); +} +const fmt = (ms: number) => (ms / 1000).toFixed(1); +const out: string[] = []; +out.push(`#### X01 trace: ${title}`); +out.push(`events=${events.length} processes=${procs.length}`); +out.push(""); +out.push( + "kind | n | avg total s | sum total s | avg witness s | sum witness s" +); +const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); +for (const kind of kinds) { + const selected = procs.filter((proc) => proc.kind === kind); + const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); + const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); + out.push( + `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` + ); +} +const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); +const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); +out.push( + `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` +); +out.push(""); +// TEMPORARY X04: engine-ID admissions by selection kind and outcome. +const admissions = new Map(); +for (const event of events) { + if (event.k === "ident-admit" && event.l) { + admissions.set(event.l, (admissions.get(event.l) ?? 0) + 1); + } +} +out.push( + `ident admissions: ${ + [...admissions.entries()] + .sort() + .map(([label, count]) => `${label}=${count}`) + .join(" ") || "none" + }` +); +out.push(""); + +function leafTable(selected: Proc[], heading: string, limit = 24) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const key = `${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} (per process avg over n=${selected.length})`); + out.push("leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function leafByScope(selected: Proc[], heading: string, limit = 40) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (!(LEAF.has(event.k) && event.l)) { + continue; + } + const parts = (event.s ?? "") + .split(">") + .filter( + (part) => + isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) + ); + const inner = parts.slice(-2).join(">") || "(none)"; + const key = `${inner} :: ${event.k}:${event.l}`; + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push( + `${heading} leaves by scope (per process avg over n=${selected.length})` + ); + out.push("scope :: leaf | calls/proc | s/proc | ms/call"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function scopeTable(selected: Proc[], heading: string, limit = 30) { + const rows = new Map(); + for (const proc of selected) { + for (const event of proc.events) { + if (event.k !== "scope" || !event.l) { + continue; + } + const parts = (event.s ?? "").split(">"); + const key = parts + .filter( + (part) => + PHASE.test(part) || + isWitness(part) || + part.startsWith("xattr:") || + part.startsWith("cmd:") || + part.startsWith("exec-") + ) + .join(">"); + if (key === "") { + continue; + } + const row = rows.get(key) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.ms ?? 0; + rows.set(key, row); + } + } + out.push(`${heading} scopes (per process avg over n=${selected.length})`); + out.push("scope path | n/proc | s/proc | ms/each"); + const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); + for (const [key, row] of sorted.slice(0, limit)) { + out.push( + `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` + ); + } + out.push(""); +} +function phaseRows(proc: Proc) { + const rows = new Map(); + for (const event of proc.events) { + if ( + event.k === "phase" && + event.line?.boundary !== "begin" && + event.line?.phase + ) { + const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; + row.n += 1; + row.ms += event.line.durationMs ?? 0; + rows.set(event.line.phase, row); + } + } + return rows; +} +for (const kind of [ + "up(cold)", + "up(warm)", + "exec(storage)", + "restart(warm)", + "run(warm)", + "run(cold)", + "down", +]) { + const selected = procs.filter((proc) => proc.kind === kind); + if (selected.length === 0) { + continue; + } + const sample = selected[0] as Proc; + out.push( + `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` + ); + out.push("phase-trace phase | spans | s"); + for (const [phase, row] of phaseRows(sample)) { + out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); + } + out.push(""); + scopeTable([sample], `${kind} first sample`, 40); + leafTable([sample], `${kind} first sample leaves`); + scopeTable(selected, `${kind} all`, 30); + leafTable(selected, `${kind} all leaves`); + leafByScope(selected, `${kind} all`); +} +console.log(out.join("\n")); diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index c9384e64..d9b58c13 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,6 +23,7 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -134,7 +135,12 @@ export type NativeComposeCommandStorageVerifyOptions = { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export async function prepareNativeComposeCommandStorage(opts: { +export function prepareNativeComposeCommandStorage( + opts: Parameters[0] +): ReturnType { + return x01Scope("storage-prepare", () => x01Prepare(opts)); +} +async function x01Prepare(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -225,7 +231,11 @@ export async function prepareNativeComposeCommandStorage(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = async ( + const verify = ( + generation: NativeComposeGeneration, + options: NativeComposeCommandStorageVerifyOptions = {} + ) => x01Scope("storage-verify", () => x01Verify(generation, options)); + const x01Verify = async ( generation: NativeComposeGeneration, options: NativeComposeCommandStorageVerifyOptions = {} ) => { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 8c5dcaf8..0aa92b81 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -138,6 +138,7 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -943,7 +944,9 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = async ( + const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => + x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); + const x01AssertSaved = async ( proof: NativeComposeCommandStorageVerifyOptions ) => { const latest = await store.loadCurrent(); @@ -962,7 +965,9 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - runSavedProcess({ options, generation, document, base, signal }), + x01Scope("exec-process", () => + runSavedProcess({ options, generation, document, base, signal }) + ), }); }); } diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index d9b31f73..d19ec5ed 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -8,6 +8,7 @@ import { } from "./docker-context.ts"; import { isRecord } from "./guards.ts"; import { bindNativeComposeEngineConfig } from "./native-compose-engine-config.ts"; +import { x01Mark, x01Time } from "./x01-cost-probe.ts"; const ENGINE_ID = /^[A-Za-z0-9:-]{1,128}$/; const AMBIGUOUS_PATH = /[%?#\0\r\n]/; @@ -300,6 +301,8 @@ type BoundObserver = { /** Opaque digest of the exact selection, executable, config and socket binding. */ readonly key: string; readonly read: () => Promise; + /** TEMPORARY X04 measurement: closed selection kind label. */ + readonly kind: string; }; function bindObserver(opts: { readonly signal?: AbortSignal; @@ -370,8 +373,15 @@ function bindObserver(opts: { JSON.stringify([selected, binary, config, endpoint, socket, executable]) ) .digest("hex"); + const label = Array.isArray(endpoint.binding) + ? endpoint.binding[0] + : undefined; return Object.freeze({ key, + kind: + label === "host" || label === "default" || label === "context" + ? label + : "other", read: async () => { if (reading) { return refuse(); @@ -469,18 +479,22 @@ export function createNativeComposeEngineIdentityAdmissions( return await opts.cli(); } const state = admitted.get(bound.key); + const direct = bound.read; if (state === true) { - return `${JSON.stringify(await bound.read())}\n`; + return `${JSON.stringify(await x01Time("ident", "direct", direct))}\n`; } const text = await opts.cli(); if (state === undefined) { let equal = false; try { const value: unknown = JSON.parse(text); - equal = typeof value === "string" && value === (await bound.read()); + equal = + typeof value === "string" && + value === (await x01Time("ident", "admission", direct)); } catch { equal = false; } + x01Mark("ident-admit", `${bound.kind}:${equal ? "admitted" : "refused"}`); admitted.set(bound.key, equal); } return text; diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index 0019b3fc..d8bc9dc8 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,6 +3,7 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1232,7 +1233,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - return async (args) => { + const x01Inner = async (args: readonly string[]): Promise => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1310,6 +1311,8 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; + return (args) => + x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 540f833b..943b7932 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,5 +1,6 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; +import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -135,6 +136,8 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); + } else if (x01Enabled()) { + trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -147,5 +150,8 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return context.getStore()?.measure(phase, operation) ?? operation(); + return x01Scope( + phase, + () => context.getStore()?.measure(phase, operation) ?? operation() + ); } diff --git a/src/lib/native-compose-storage-command-executable.ts b/src/lib/native-compose-storage-command-executable.ts index a8817d68..780dae04 100644 --- a/src/lib/native-compose-storage-command-executable.ts +++ b/src/lib/native-compose-storage-command-executable.ts @@ -2,6 +2,7 @@ import { constants } from "node:fs"; import { type FileHandle, lstat, open, realpath } from "node:fs/promises"; import type { NativeComposeStorageCommandExecutable } from "./native-compose-storage-command-record.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Label, x01Mark } from "./x01-cost-probe.ts"; /** Coarser than any supported filesystem timestamp granularity (1 s on HFS+/ext3). */ const RACY_MS = 2000; @@ -83,6 +84,9 @@ export function createNativeComposeStorageCommandExecutableReader( const size = Number(before.size); const started = now(); const cached = settled.get(canonical); + if (cached?.identity === identity) { + x01Mark("hash-cached", x01Label([canonical])); + } const hash = cached?.identity === identity ? cached.hash : await digest(file, size); const after = await file.stat({ bigint: true }), diff --git a/src/lib/native-compose-storage-witness-docker-inventory.ts b/src/lib/native-compose-storage-witness-docker-inventory.ts index b5bdb741..f292021c 100644 --- a/src/lib/native-compose-storage-witness-docker-inventory.ts +++ b/src/lib/native-compose-storage-witness-docker-inventory.ts @@ -8,6 +8,7 @@ import { type NativeComposeStorageXattrTarget, } from "./native-compose-storage-witness-xattr-carrier.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,254}$/; @@ -164,7 +165,12 @@ export async function assertNativeComposeStorageDockerCarrierVolume(opts: { /** Complete holder reads include foreign containers. The two inventories and selected * volume reread are observed fences; they do not claim an atomic daemon transaction. */ -export async function observeNativeComposeStorageDockerTarget(opts: { +export function observeNativeComposeStorageDockerTarget( + opts: Parameters[0] +): ReturnType { + return x01Scope("target", () => x01ObserveTarget(opts)); +} +async function x01ObserveTarget(opts: { readonly probe: Probe; readonly selection: Selection; readonly current: NativeComposeMaterialBinding; diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index c8e06218..d955faf0 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -98,6 +98,7 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; +import { x01Label, x01Mark, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -817,7 +818,10 @@ async function groupAbsent( } /** Every identity check runs per call; only an unchanged stat identity skips the * content read for the rest of this command. */ -const commandExecutable = readNativeComposeStorageCommandExecutable; +const commandExecutable = (path: string) => + x01Time("hash", x01Label([path]), () => + readNativeComposeStorageCommandExecutable(path) + ); async function boundedHostFile(path: string): Promise { const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); try { @@ -903,7 +907,14 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export async function runNativeComposeStorageDockerCommand(supplied: { +export function runNativeComposeStorageDockerCommand( + supplied: Parameters[0] +): ReturnType { + return x01Scope(`cmd:${x01Label(supplied.args)}`, () => + x01RunCommandInner(supplied) + ); +} +async function x01RunCommandInner(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1248,7 +1259,12 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export async function createNativeComposeDockerStorageXattrCarrier( +export function createNativeComposeDockerStorageXattrCarrier( + opts: Context +): Promise { + return x01Scope("carrier-new", () => x01CreateCarrier(opts)); +} +async function x01CreateCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -1370,6 +1386,7 @@ export async function createNativeComposeDockerStorageXattrCarrier( const binding = await current(context, selection); if (same(binding, closed.binding)) { lifetime(context); + x01Mark("target-reused", "closing"); return closed.target; } } catch (error) { @@ -2034,17 +2051,21 @@ export async function createNativeComposeDockerStorageXattrCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect: inspectPort, - provision, - invoke, + inspect: (selection, options) => + x01Scope("inspect", () => inspectPort(selection, options)), + provision: (selection) => + x01Scope("provision", () => provision(selection)), + invoke: (input) => + x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), finish: context.originalCommandRecords === true - ? async ({ input, completion }) => { - closing = null; - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - } + ? ({ input, completion }) => + x01Scope("finish", async () => { + closing = null; + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + }) : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 7586eb73..1fc68abf 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,6 +78,7 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; +import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -498,7 +499,16 @@ function requireXattrAdmission( refuse(); } } -async function invokeXattr(opts: { +function invokeXattr( + opts: Parameters[0] +): ReturnType { + const operation = + isRecord(opts.request) && typeof opts.request.operation === "string" + ? opts.request.operation + : "unknown"; + return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); +} +async function x01InvokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1464,17 +1474,21 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }); + return await x01Scope("witness-prepare", () => + prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }) + ); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }); + return await x01Scope("witness-enroll", () => + enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }) + ); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1485,7 +1499,9 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); + await x01Scope("witness-verify", () => + verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) + ); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ef9faf1b..ff27e6b8 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,6 +4,7 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; +import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -40,7 +41,13 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export async function exec( +export function exec( + cmd: readonly string[], + opts: ExecOptions = {} +): Promise { + return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); +} +async function x01ExecInner( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -123,7 +130,13 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export async function run( +export function run( + cmd: readonly string[], + opts: RunOptions = {} +): Promise { + return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); +} +async function x01RunInner( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts new file mode 100644 index 00000000..17b2f555 --- /dev/null +++ b/src/lib/x01-cost-probe.ts @@ -0,0 +1,176 @@ +// TEMPORARY X01 measurement probe. Reverted before the final commit. +// Writes closed labels and durations only (no ids, paths or values). +import { AsyncLocalStorage } from "node:async_hooks"; +import { appendFileSync } from "node:fs"; + +const FILE = process.env.HACK_X01_TRACE_FILE; +const VERBS = new Set([ + "up", + "down", + "exec", + "run", + "restart", + "ps", + "logs", + "status", + "init", +]); +const SUBS = new Set([ + "info", + "image", + "container", + "volume", + "network", + "compose", + "create", + "start", + "rm", + "inspect", + "ls", + "ps", + "exec", + "run", + "logs", + "version", + "events", + "stop", + "kill", + "wait", + "up", + "down", + "config", + "pull", + "build", + "restart", + "port", + "cp", + "top", +]); +const verb = (() => { + for (const arg of process.argv.slice(2)) { + if (VERBS.has(arg)) { + return arg; + } + } + return "other"; +})(); +const scopes = new AsyncLocalStorage(); +let enabled = typeof FILE === "string" && FILE.startsWith("/"); +let wroteStart = false; + +function append(record: Record): void { + appendFileSync( + FILE as string, + `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` + ); +} + +function write(record: Record): void { + if (!enabled) { + return; + } + try { + if (!wroteStart) { + wroteStart = true; + append({ k: "start", t: Math.round(performance.now()) }); + process.on("exit", (code) => { + try { + append({ k: "exit", c: code, t: Math.round(performance.now()) }); + } catch { + // measurement only + } + }); + } + append({ + s: scopes.getStore() ?? "", + t: Math.round(performance.now()), + ...record, + }); + } catch { + enabled = false; + } +} + +export function x01Enabled(): boolean { + return enabled; +} + +/** Raw line from the phase trace, re-wrapped with process identity. */ +export function x01Phase(line: string): void { + try { + write({ k: "phase", line: JSON.parse(line) as unknown }); + } catch { + enabled = false; + } +} + +export function x01Label(argv: readonly string[], binary = true): string { + const parts: string[] = []; + let rest = argv; + if (binary) { + const name = (argv[0] ?? "").split("/").at(-1) ?? ""; + parts.push( + ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) + ? name + : "bin" + ); + rest = argv.slice(1); + } else { + parts.push("docker"); + } + for (const token of rest) { + if (parts.length >= 3) { + break; + } + if (token.startsWith("-")) { + continue; + } + if (SUBS.has(token)) { + parts.push(token); + } else { + break; + } + } + return parts.join(" "); +} + +export async function x01Time( + kind: string, + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const started = performance.now(); + let ok = false; + try { + const result = await operation(); + ok = true; + return result; + } finally { + write({ + k: kind, + l: label, + ms: Math.round((performance.now() - started) * 10) / 10, + ok, + }); + } +} + +/** TEMPORARY X03 measurement: a closed-label point event. */ +export function x01Mark(kind: string, label: string): void { + write({ k: kind, l: label }); +} + +export async function x01Scope( + label: string, + operation: () => Promise +): Promise { + if (!enabled) { + return await operation(); + } + const parent = scopes.getStore(); + const path = parent ? `${parent}>${label}` : label; + return await scopes.run(path, () => x01Time("scope", label, operation)); +} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index 2201457b..e7599c52 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,6 +43,8 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -768,6 +770,13 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); + // TEMPORARY X01 measurement: one trace file per scenario. + if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { + process.env.HACK_X01_TRACE_FILE = join( + process.env.HACK_X01_TRACE_DIR, + `${scenario.name}.jsonl` + ); + } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 633cb044..6c519470 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1154,6 +1154,10 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", + // TEMPORARY X01 measurement passthrough; reverted before the final commit. + ...(process.env.HACK_X01_TRACE_FILE + ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } + : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From 5cf9f0f3eef372541afac832ce4b30cada6c26fc Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 04:48:03 -0400 Subject: [PATCH 26/52] chore(x04): revert temporary witness probe trace Measurement is complete; the hosted trace run was 38225752202. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 25 -- scripts/x01-trace-summary.ts | 285 ------------------ src/lib/native-compose-command-storage.ts | 14 +- src/lib/native-compose-command.ts | 9 +- src/lib/native-compose-engine-identity.ts | 18 +- src/lib/native-compose-ownership.ts | 5 +- src/lib/native-compose-phase-trace.ts | 8 +- ...tive-compose-storage-command-executable.ts | 4 - ...ompose-storage-witness-docker-inventory.ts | 8 +- .../native-compose-storage-witness-docker.ts | 45 +-- src/lib/native-compose-storage-witness.ts | 34 +-- src/lib/shell.ts | 17 +- src/lib/x01-cost-probe.ts | 176 ----------- tests/e2e/harness.ts | 9 - .../e2e/native-config-networks-acceptance.ts | 4 - 15 files changed, 32 insertions(+), 629 deletions(-) delete mode 100644 scripts/x01-trace-summary.ts delete mode 100644 src/lib/x01-cost-probe.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 419fb1a1..695b9a0d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -304,22 +304,13 @@ jobs: - name: Verify Docker context endpoint selection against the real Docker CLI if: matrix.shard == 4 run: HACK_TEST_DOCKER_CLI=1 bun test tests/docker-context.test.ts - # TEMPORARY X04 measurement; reverted before the final commit. - - name: X04 runner Docker client selection - run: | - docker context inspect --format 'context={{.Name}} host={{.Endpoints.docker.Host}}' || true - if [ -f "$HOME/.docker/config.json" ]; then echo "client-config=present"; else echo "client-config=absent"; fi - echo "DOCKER_HOST set=${DOCKER_HOST:+yes} DOCKER_CONTEXT set=${DOCKER_CONTEXT:+yes} DOCKER_CONFIG set=${DOCKER_CONFIG:+yes}" - name: Run local and Docker E2E shard env: HACK_E2E_PROCESS_POLICY_REFUSAL_CAPSULE: "1" HACK_E2E_CAPSULE_DIR: ${{ runner.temp }}/process-policy-capsules HACK_E2E_CLI_BIN: ${{ github.workspace }}/dist/hack HACK_E2E_SHARD: ${{ matrix.shard }}/4 - # TEMPORARY X01 measurement; reverted before the final commit. - HACK_X01_TRACE_DIR: ${{ runner.temp }}/x01-trace run: | - mkdir -p "$HACK_X01_TRACE_DIR" HACK_E2E_DOCKER_HOST="${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')}" HACK_E2E_NETWORK_SOURCE_REVISION="$(git rev-parse HEAD)" HACK_E2E_NETWORK_CLI_SHA256="$(sha256sum dist/hack | cut -d ' ' -f1)" @@ -328,22 +319,6 @@ jobs: HACK_E2E_COMPOSE_PLUGIN_SHA256="$(sha256sum "$HACK_E2E_COMPOSE_PLUGIN_PATH" | cut -d ' ' -f1)" export HACK_E2E_DOCKER_HOST HACK_E2E_NETWORK_SOURCE_REVISION HACK_E2E_NETWORK_CLI_SHA256 HACK_E2E_NETWORK_COMPILER_SHA256 HACK_E2E_COMPOSE_PLUGIN_PATH HACK_E2E_COMPOSE_PLUGIN_SHA256 HACK_E2E_REQUIRE_DOCKER=1 HACK_E2E_REQUIRE_TMUX=1 bun run test:e2e:local:docker --shard="$HACK_E2E_SHARD" - # TEMPORARY X01 measurement; reverted before the final commit. - - name: X01 witness cost summary - if: always() - run: | - for f in "$RUNNER_TEMP"/x01-trace/*.jsonl; do - [ -f "$f" ] || continue - bun scripts/x01-trace-summary.ts "$f" "$(basename "$f" .jsonl)" || true - done - - name: X01 witness cost trace artifact - if: always() - uses: actions/upload-artifact@v4 - with: - name: x01-trace-shard-${{ matrix.shard }} - path: ${{ runner.temp }}/x01-trace - if-no-files-found: ignore - retention-days: 7 - name: Preserve process-policy capsule and startup diagnostic # Closed fixed-field JSON only (counts, enums, booleans); the gh log viewer truncates it inline. if: failure() diff --git a/scripts/x01-trace-summary.ts b/scripts/x01-trace-summary.ts deleted file mode 100644 index 62ace753..00000000 --- a/scripts/x01-trace-summary.ts +++ /dev/null @@ -1,285 +0,0 @@ -// TEMPORARY X01 measurement summary. Reverted before the final commit. -// Reads closed-label JSONL written by src/lib/x01-cost-probe.ts. -type Event = { - readonly p: number; - readonly v: string; - readonly k: string; - readonly s?: string; - readonly l?: string; - readonly ms?: number; - readonly t?: number; - readonly c?: number; - readonly line?: { - readonly phase?: string; - readonly boundary?: string; - readonly durationMs?: number | null; - }; -}; -const WITNESS = new Set([ - "storage-prepare", - "storage-verify", - "carrier-new", - "inspect", - "provision", - "finish", - "witness-verify", - "witness-prepare", - "witness-enroll", -]); -const isWitness = (label: string) => - WITNESS.has(label) || label.startsWith("invoke:"); -// TEMPORARY X04: direct engine-ID reads (`ident:direct`, `ident:admission`) are leaves too. -const LEAF = new Set(["probe", "run", "exec", "hash", "ident"]); -const PHASE = /^(oneoff|storage|guard|finish|finalize)\.[a-z-]+$/; - -const path = process.argv[2]; -const title = process.argv[3] ?? path; -if (!path) { - throw new Error("usage: x01-trace-summary [title]"); -} -const text = await Bun.file(path).text(); -const events: Event[] = []; -for (const line of text.split("\n")) { - if (line.trim() === "") { - continue; - } - try { - events.push(JSON.parse(line) as Event); - } catch { - // skip partial line - } -} -const byProcess = new Map(); -for (const event of events) { - const list = byProcess.get(event.p) ?? []; - list.push(event); - byProcess.set(event.p, list); -} -type Proc = { - readonly pid: number; - readonly verb: string; - readonly totalMs: number; - readonly witnessMs: number; - readonly events: Event[]; - readonly kind: string; -}; -function topWitnessMs(list: Event[]): number { - let total = 0; - for (const event of list) { - if (event.k !== "scope" || !event.l || !isWitness(event.l)) { - continue; - } - const parts = (event.s ?? "").split(">"); - parts.pop(); - if (parts.some(isWitness)) { - continue; - } - total += event.ms ?? 0; - } - return total; -} -const procs: Proc[] = []; -for (const [pid, list] of byProcess) { - const exit = list.find((event) => event.k === "exit"); - const verb = list[0]?.v ?? "other"; - const labels = new Set(list.map((event) => event.l ?? "")); - let kind = verb; - if (verb === "up" || verb === "restart" || verb === "run") { - if (labels.has("invoke:seed")) { - kind = `${verb}(cold)`; - } else if (labels.has("invoke:verify")) { - kind = `${verb}(warm)`; - } else { - kind = `${verb}(no-witness)`; - } - } else if (verb === "exec") { - kind = labels.has("invoke:verify") ? "exec(storage)" : "exec(image)"; - } - procs.push({ - pid, - verb, - totalMs: exit?.t ?? Math.max(...list.map((event) => event.t ?? 0)), - witnessMs: topWitnessMs(list), - events: list, - kind, - }); -} -const fmt = (ms: number) => (ms / 1000).toFixed(1); -const out: string[] = []; -out.push(`#### X01 trace: ${title}`); -out.push(`events=${events.length} processes=${procs.length}`); -out.push(""); -out.push( - "kind | n | avg total s | sum total s | avg witness s | sum witness s" -); -const kinds = [...new Set(procs.map((proc) => proc.kind))].sort(); -for (const kind of kinds) { - const selected = procs.filter((proc) => proc.kind === kind); - const total = selected.reduce((sum, proc) => sum + proc.totalMs, 0); - const witness = selected.reduce((sum, proc) => sum + proc.witnessMs, 0); - out.push( - `${kind} | ${selected.length} | ${fmt(total / selected.length)} | ${fmt(total)} | ${fmt(witness / selected.length)} | ${fmt(witness)}` - ); -} -const allTotal = procs.reduce((sum, proc) => sum + proc.totalMs, 0); -const allWitness = procs.reduce((sum, proc) => sum + proc.witnessMs, 0); -out.push( - `ALL | ${procs.length} | - | ${fmt(allTotal)} | - | ${fmt(allWitness)}` -); -out.push(""); -// TEMPORARY X04: engine-ID admissions by selection kind and outcome. -const admissions = new Map(); -for (const event of events) { - if (event.k === "ident-admit" && event.l) { - admissions.set(event.l, (admissions.get(event.l) ?? 0) + 1); - } -} -out.push( - `ident admissions: ${ - [...admissions.entries()] - .sort() - .map(([label, count]) => `${label}=${count}`) - .join(" ") || "none" - }` -); -out.push(""); - -function leafTable(selected: Proc[], heading: string, limit = 24) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const key = `${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} (per process avg over n=${selected.length})`); - out.push("leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function leafByScope(selected: Proc[], heading: string, limit = 40) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (!(LEAF.has(event.k) && event.l)) { - continue; - } - const parts = (event.s ?? "") - .split(">") - .filter( - (part) => - isWitness(part) || part.startsWith("cmd:") || PHASE.test(part) - ); - const inner = parts.slice(-2).join(">") || "(none)"; - const key = `${inner} :: ${event.k}:${event.l}`; - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push( - `${heading} leaves by scope (per process avg over n=${selected.length})` - ); - out.push("scope :: leaf | calls/proc | s/proc | ms/call"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function scopeTable(selected: Proc[], heading: string, limit = 30) { - const rows = new Map(); - for (const proc of selected) { - for (const event of proc.events) { - if (event.k !== "scope" || !event.l) { - continue; - } - const parts = (event.s ?? "").split(">"); - const key = parts - .filter( - (part) => - PHASE.test(part) || - isWitness(part) || - part.startsWith("xattr:") || - part.startsWith("cmd:") || - part.startsWith("exec-") - ) - .join(">"); - if (key === "") { - continue; - } - const row = rows.get(key) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.ms ?? 0; - rows.set(key, row); - } - } - out.push(`${heading} scopes (per process avg over n=${selected.length})`); - out.push("scope path | n/proc | s/proc | ms/each"); - const sorted = [...rows.entries()].sort((a, b) => b[1].ms - a[1].ms); - for (const [key, row] of sorted.slice(0, limit)) { - out.push( - `${key} | ${(row.n / selected.length).toFixed(1)} | ${fmt(row.ms / selected.length)} | ${(row.ms / row.n).toFixed(0)}` - ); - } - out.push(""); -} -function phaseRows(proc: Proc) { - const rows = new Map(); - for (const event of proc.events) { - if ( - event.k === "phase" && - event.line?.boundary !== "begin" && - event.line?.phase - ) { - const row = rows.get(event.line.phase) ?? { n: 0, ms: 0 }; - row.n += 1; - row.ms += event.line.durationMs ?? 0; - rows.set(event.line.phase, row); - } - } - return rows; -} -for (const kind of [ - "up(cold)", - "up(warm)", - "exec(storage)", - "restart(warm)", - "run(warm)", - "run(cold)", - "down", -]) { - const selected = procs.filter((proc) => proc.kind === kind); - if (selected.length === 0) { - continue; - } - const sample = selected[0] as Proc; - out.push( - `### ${kind}: first sample total ${fmt(sample.totalMs)} s, witness ${fmt(sample.witnessMs)} s` - ); - out.push("phase-trace phase | spans | s"); - for (const [phase, row] of phaseRows(sample)) { - out.push(`${phase} | ${row.n} | ${fmt(row.ms)}`); - } - out.push(""); - scopeTable([sample], `${kind} first sample`, 40); - leafTable([sample], `${kind} first sample leaves`); - scopeTable(selected, `${kind} all`, 30); - leafTable(selected, `${kind} all leaves`); - leafByScope(selected, `${kind} all`); -} -console.log(out.join("\n")); diff --git a/src/lib/native-compose-command-storage.ts b/src/lib/native-compose-command-storage.ts index d9b58c13..c9384e64 100644 --- a/src/lib/native-compose-command-storage.ts +++ b/src/lib/native-compose-command-storage.ts @@ -23,7 +23,6 @@ import { } from "./native-compose-storage-witness.ts"; import { createNativeComposeDockerStorageXattrCarrier } from "./native-compose-storage-witness-docker.ts"; import type { NativeComposeStorageXattrCarrier } from "./native-compose-storage-witness-xattr-carrier.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Selection = { readonly name: string; readonly storage: string }; type Document = Readonly>; @@ -135,12 +134,7 @@ export type NativeComposeCommandStorageVerifyOptions = { /** Command admission owns no helper process. Each proof gets a fresh finite carrier; * only the original startup effect can publish Expected and consume cold enrollment. */ -export function prepareNativeComposeCommandStorage( - opts: Parameters[0] -): ReturnType { - return x01Scope("storage-prepare", () => x01Prepare(opts)); -} -async function x01Prepare(opts: { +export async function prepareNativeComposeCommandStorage(opts: { readonly store: NativeComposeGenerationStore; readonly mutation: NativeComposeMutation; readonly operation: "up" | "restart" | "run" | "exec"; @@ -231,11 +225,7 @@ async function x01Prepare(opts: { await engine({ expected: engineId, signal }); await assertFresh?.(); }; - const verify = ( - generation: NativeComposeGeneration, - options: NativeComposeCommandStorageVerifyOptions = {} - ) => x01Scope("storage-verify", () => x01Verify(generation, options)); - const x01Verify = async ( + const verify = async ( generation: NativeComposeGeneration, options: NativeComposeCommandStorageVerifyOptions = {} ) => { diff --git a/src/lib/native-compose-command.ts b/src/lib/native-compose-command.ts index 0aa92b81..8c5dcaf8 100644 --- a/src/lib/native-compose-command.ts +++ b/src/lib/native-compose-command.ts @@ -138,7 +138,6 @@ import { NativeConfigCompilerError } from "./native-config-compiler.ts"; import { nativeFilePlanningRequired } from "./native-file-plan-protocol.ts"; import { parseEnvConfigSelection } from "./project.ts"; import { run } from "./shell.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; type Operation = "up" | "restart" | "down" | "ps" | "logs" | "exec" | "run"; export type NativeComposeCommandOptions = { @@ -944,9 +943,7 @@ async function savedCommand(opts: { selected, signal, }); - const assertSaved = (proof: NativeComposeCommandStorageVerifyOptions) => - x01Scope("exec-assert-saved", () => x01AssertSaved(proof)); - const x01AssertSaved = async ( + const assertSaved = async ( proof: NativeComposeCommandStorageVerifyOptions ) => { const latest = await store.loadCurrent(); @@ -965,9 +962,7 @@ async function savedCommand(opts: { signal, assertSaved, run: () => - x01Scope("exec-process", () => - runSavedProcess({ options, generation, document, base, signal }) - ), + runSavedProcess({ options, generation, document, base, signal }), }); }); } diff --git a/src/lib/native-compose-engine-identity.ts b/src/lib/native-compose-engine-identity.ts index d19ec5ed..d9b31f73 100644 --- a/src/lib/native-compose-engine-identity.ts +++ b/src/lib/native-compose-engine-identity.ts @@ -8,7 +8,6 @@ import { } from "./docker-context.ts"; import { isRecord } from "./guards.ts"; import { bindNativeComposeEngineConfig } from "./native-compose-engine-config.ts"; -import { x01Mark, x01Time } from "./x01-cost-probe.ts"; const ENGINE_ID = /^[A-Za-z0-9:-]{1,128}$/; const AMBIGUOUS_PATH = /[%?#\0\r\n]/; @@ -301,8 +300,6 @@ type BoundObserver = { /** Opaque digest of the exact selection, executable, config and socket binding. */ readonly key: string; readonly read: () => Promise; - /** TEMPORARY X04 measurement: closed selection kind label. */ - readonly kind: string; }; function bindObserver(opts: { readonly signal?: AbortSignal; @@ -373,15 +370,8 @@ function bindObserver(opts: { JSON.stringify([selected, binary, config, endpoint, socket, executable]) ) .digest("hex"); - const label = Array.isArray(endpoint.binding) - ? endpoint.binding[0] - : undefined; return Object.freeze({ key, - kind: - label === "host" || label === "default" || label === "context" - ? label - : "other", read: async () => { if (reading) { return refuse(); @@ -479,22 +469,18 @@ export function createNativeComposeEngineIdentityAdmissions( return await opts.cli(); } const state = admitted.get(bound.key); - const direct = bound.read; if (state === true) { - return `${JSON.stringify(await x01Time("ident", "direct", direct))}\n`; + return `${JSON.stringify(await bound.read())}\n`; } const text = await opts.cli(); if (state === undefined) { let equal = false; try { const value: unknown = JSON.parse(text); - equal = - typeof value === "string" && - value === (await x01Time("ident", "admission", direct)); + equal = typeof value === "string" && value === (await bound.read()); } catch { equal = false; } - x01Mark("ident-admit", `${bound.kind}:${equal ? "admitted" : "refused"}`); admitted.set(bound.key, equal); } return text; diff --git a/src/lib/native-compose-ownership.ts b/src/lib/native-compose-ownership.ts index d8bc9dc8..0019b3fc 100644 --- a/src/lib/native-compose-ownership.ts +++ b/src/lib/native-compose-ownership.ts @@ -3,7 +3,6 @@ import { isRecord } from "./guards.ts"; import { nativeComposeVolumeCreatedAt } from "./native-compose-retained-storage.ts"; import { beginNativeCpuChild } from "./native-cpu-diagnostics.ts"; import { findExecutableInPath } from "./shell.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; const PROJECT = /^[a-z0-9][a-z0-9_-]*$/; const SERVICE = /^[a-z0-9][a-z0-9._-]{0,62}$/; @@ -1233,7 +1232,7 @@ export function createNativeComposeProbe( probeRefused("E_NATIVE_COMPOSE_PROBE", "operation"); } const environment = { ...process.env }; - const x01Inner = async (args: readonly string[]): Promise => { + return async (args) => { checkInterruption(opts.signal, false); const remainingTime = deadline - Date.now(); if (remainingTime <= 0) { @@ -1311,8 +1310,6 @@ export function createNativeComposeProbe( observeCpu(probeExitCode(completion[2])); } }; - return (args) => - x01Time("probe", x01Label(args, false), () => x01Inner(args)); } function probeExitCode( diff --git a/src/lib/native-compose-phase-trace.ts b/src/lib/native-compose-phase-trace.ts index 943b7932..540f833b 100644 --- a/src/lib/native-compose-phase-trace.ts +++ b/src/lib/native-compose-phase-trace.ts @@ -1,6 +1,5 @@ import { AsyncLocalStorage } from "node:async_hooks"; import { writeSync } from "node:fs"; -import { x01Enabled, x01Phase, x01Scope } from "./x01-cost-probe.ts"; export const NATIVE_COMPOSE_PHASE_TRACE = "HACK_NATIVE_COMPOSE_PHASE_TRACE"; const PHASES = [ @@ -136,8 +135,6 @@ export function withNativeComposePhaseTrace(operation: () => Promise) { writeSync(2, line); }, }); - } else if (x01Enabled()) { - trace = createNativeComposePhaseTrace({ write: x01Phase }); } } catch { // Diagnostic setup never changes selection, results or effect authority. @@ -150,8 +147,5 @@ export function measureNativeComposePhase( phase: Phase, operation: () => Promise ): Promise { - return x01Scope( - phase, - () => context.getStore()?.measure(phase, operation) ?? operation() - ); + return context.getStore()?.measure(phase, operation) ?? operation(); } diff --git a/src/lib/native-compose-storage-command-executable.ts b/src/lib/native-compose-storage-command-executable.ts index 780dae04..a8817d68 100644 --- a/src/lib/native-compose-storage-command-executable.ts +++ b/src/lib/native-compose-storage-command-executable.ts @@ -2,7 +2,6 @@ import { constants } from "node:fs"; import { type FileHandle, lstat, open, realpath } from "node:fs/promises"; import type { NativeComposeStorageCommandExecutable } from "./native-compose-storage-command-record.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Label, x01Mark } from "./x01-cost-probe.ts"; /** Coarser than any supported filesystem timestamp granularity (1 s on HFS+/ext3). */ const RACY_MS = 2000; @@ -84,9 +83,6 @@ export function createNativeComposeStorageCommandExecutableReader( const size = Number(before.size); const started = now(); const cached = settled.get(canonical); - if (cached?.identity === identity) { - x01Mark("hash-cached", x01Label([canonical])); - } const hash = cached?.identity === identity ? cached.hash : await digest(file, size); const after = await file.stat({ bigint: true }), diff --git a/src/lib/native-compose-storage-witness-docker-inventory.ts b/src/lib/native-compose-storage-witness-docker-inventory.ts index f292021c..b5bdb741 100644 --- a/src/lib/native-compose-storage-witness-docker-inventory.ts +++ b/src/lib/native-compose-storage-witness-docker-inventory.ts @@ -8,7 +8,6 @@ import { type NativeComposeStorageXattrTarget, } from "./native-compose-storage-witness-xattr-carrier.ts"; import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const NAME = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,254}$/; @@ -165,12 +164,7 @@ export async function assertNativeComposeStorageDockerCarrierVolume(opts: { /** Complete holder reads include foreign containers. The two inventories and selected * volume reread are observed fences; they do not claim an atomic daemon transaction. */ -export function observeNativeComposeStorageDockerTarget( - opts: Parameters[0] -): ReturnType { - return x01Scope("target", () => x01ObserveTarget(opts)); -} -async function x01ObserveTarget(opts: { +export async function observeNativeComposeStorageDockerTarget(opts: { readonly probe: Probe; readonly selection: Selection; readonly current: NativeComposeMaterialBinding; diff --git a/src/lib/native-compose-storage-witness-docker.ts b/src/lib/native-compose-storage-witness-docker.ts index d955faf0..c8e06218 100644 --- a/src/lib/native-compose-storage-witness-docker.ts +++ b/src/lib/native-compose-storage-witness-docker.ts @@ -98,7 +98,6 @@ import { refuseNativeComposeStorageXattr as refuse, } from "./native-compose-storage-witness-xattr-codec.ts"; import { findExecutableInPath, type RunExitEvent, run } from "./shell.ts"; -import { x01Label, x01Mark, x01Scope, x01Time } from "./x01-cost-probe.ts"; const ID = /^[a-f0-9]{64}$/; const PROCESS = @@ -818,10 +817,7 @@ async function groupAbsent( } /** Every identity check runs per call; only an unchanged stat identity skips the * content read for the rest of this command. */ -const commandExecutable = (path: string) => - x01Time("hash", x01Label([path]), () => - readNativeComposeStorageCommandExecutable(path) - ); +const commandExecutable = readNativeComposeStorageCommandExecutable; async function boundedHostFile(path: string): Promise { const file = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); try { @@ -907,14 +903,7 @@ async function commandBoot( } /** No callback can reject before the shared owner awaits exit. A rejected owner or * unproven group retains the journal; daemon effects are never inferred from exit. */ -export function runNativeComposeStorageDockerCommand( - supplied: Parameters[0] -): ReturnType { - return x01Scope(`cmd:${x01Label(supplied.args)}`, () => - x01RunCommandInner(supplied) - ); -} -async function x01RunCommandInner(supplied: { +export async function runNativeComposeStorageDockerCommand(supplied: { readonly context: Pick; readonly files: Pick< Files, @@ -1259,12 +1248,7 @@ function matchesScope( * image; other daemons and uncached images refuse rather than pull or emulate. * PostgreSQL data roots and crash durability remain unqualified. No module * initialization executes Docker or the helper. */ -export function createNativeComposeDockerStorageXattrCarrier( - opts: Context -): Promise { - return x01Scope("carrier-new", () => x01CreateCarrier(opts)); -} -async function x01CreateCarrier( +export async function createNativeComposeDockerStorageXattrCarrier( opts: Context ): Promise { const context: Context = Object.freeze({ ...opts }); @@ -1386,7 +1370,6 @@ async function x01CreateCarrier( const binding = await current(context, selection); if (same(binding, closed.binding)) { lifetime(context); - x01Mark("target-reused", "closing"); return closed.target; } } catch (error) { @@ -2051,21 +2034,17 @@ async function x01CreateCarrier( return captureNativeComposeStorageXattrCarrier({ artifact: selectedArtifact, ports: { - inspect: (selection, options) => - x01Scope("inspect", () => inspectPort(selection, options)), - provision: (selection) => - x01Scope("provision", () => provision(selection)), - invoke: (input) => - x01Scope(`invoke:${input.request.operation}`, () => invoke(input)), + inspect: inspectPort, + provision, + invoke, finish: context.originalCommandRecords === true - ? ({ input, completion }) => - x01Scope("finish", async () => { - closing = null; - const finish = finalizers.get(input) ?? refuse(); - finalizers.delete(input); - await finish(completion); - }) + ? async ({ input, completion }) => { + closing = null; + const finish = finalizers.get(input) ?? refuse(); + finalizers.delete(input); + await finish(completion); + } : undefined, }, signal: context.signal, diff --git a/src/lib/native-compose-storage-witness.ts b/src/lib/native-compose-storage-witness.ts index 1fc68abf..7586eb73 100644 --- a/src/lib/native-compose-storage-witness.ts +++ b/src/lib/native-compose-storage-witness.ts @@ -78,7 +78,6 @@ import { nativeComposeStorageXattrValueMatches, sameNativeComposeStorageXattrRoot, } from "./native-compose-storage-witness-xattr-codec.ts"; -import { x01Scope } from "./x01-cost-probe.ts"; export type { NativeComposeStorageWitnessReference } from "./native-compose-storage-witness-state.ts"; @@ -499,16 +498,7 @@ function requireXattrAdmission( refuse(); } } -function invokeXattr( - opts: Parameters[0] -): ReturnType { - const operation = - isRecord(opts.request) && typeof opts.request.operation === "string" - ? opts.request.operation - : "unknown"; - return x01Scope(`xattr:${operation}`, () => x01InvokeXattr(opts)); -} -async function x01InvokeXattr(opts: { +async function invokeXattr(opts: { readonly carrier: NativeComposeStorageXattrCarrier; readonly saved: XattrExpectation; readonly volume: NativeComposeRetainedVolume; @@ -1474,21 +1464,17 @@ export async function prepareNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - return await x01Scope("witness-prepare", () => - prepareNativeComposeStorageWitness({ - ...input, - xattrCarrier: carrier, - }) - ); + return await prepareNativeComposeStorageWitness({ + ...input, + xattrCarrier: carrier, + }); } export async function enrollNativeComposeStorageXattrWitness(opts: { readonly enrollment: NativeComposeStorageWitnessEnrollment; }): Promise { - return await x01Scope("witness-enroll", () => - enrollNativeComposeStorageWitness({ - enrollment: opts.enrollment, - }) - ); + return await enrollNativeComposeStorageWitness({ + enrollment: opts.enrollment, + }); } export async function verifyNativeComposeStorageXattrWitness( opts: Omit< @@ -1499,9 +1485,7 @@ export async function verifyNativeComposeStorageXattrWitness( } ): Promise { const { carrier, ...input } = opts; - await x01Scope("witness-verify", () => - verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }) - ); + await verifyNativeComposeStorageWitness({ ...input, xattrCarrier: carrier }); } /** Private saved observation seam. It never performs a kernel read, repairs an diff --git a/src/lib/shell.ts b/src/lib/shell.ts index ff27e6b8..ef9faf1b 100644 --- a/src/lib/shell.ts +++ b/src/lib/shell.ts @@ -4,7 +4,6 @@ import { } from "./native-cpu-diagnostics.ts"; import { readSubprocessResourceUsage } from "./process-resource-usage.ts"; import { hasControllingTerminal } from "./tty-process-group.ts"; -import { x01Label, x01Time } from "./x01-cost-probe.ts"; export interface ExecResult { readonly exitCode: number; @@ -41,13 +40,7 @@ function buildSpawnEnv( return override ? { ...base, ...override } : base; } -export function exec( - cmd: readonly string[], - opts: ExecOptions = {} -): Promise { - return x01Time("exec", x01Label(cmd), () => x01ExecInner(cmd, opts)); -} -async function x01ExecInner( +export async function exec( cmd: readonly string[], opts: ExecOptions = {} ): Promise { @@ -130,13 +123,7 @@ export type RunExitEvent = { readonly maxRssBytes: number | null; }; -export function run( - cmd: readonly string[], - opts: RunOptions = {} -): Promise { - return x01Time("run", x01Label(cmd), () => x01RunInner(cmd, opts)); -} -async function x01RunInner( +export async function run( cmd: readonly string[], opts: RunOptions = {} ): Promise { diff --git a/src/lib/x01-cost-probe.ts b/src/lib/x01-cost-probe.ts deleted file mode 100644 index 17b2f555..00000000 --- a/src/lib/x01-cost-probe.ts +++ /dev/null @@ -1,176 +0,0 @@ -// TEMPORARY X01 measurement probe. Reverted before the final commit. -// Writes closed labels and durations only (no ids, paths or values). -import { AsyncLocalStorage } from "node:async_hooks"; -import { appendFileSync } from "node:fs"; - -const FILE = process.env.HACK_X01_TRACE_FILE; -const VERBS = new Set([ - "up", - "down", - "exec", - "run", - "restart", - "ps", - "logs", - "status", - "init", -]); -const SUBS = new Set([ - "info", - "image", - "container", - "volume", - "network", - "compose", - "create", - "start", - "rm", - "inspect", - "ls", - "ps", - "exec", - "run", - "logs", - "version", - "events", - "stop", - "kill", - "wait", - "up", - "down", - "config", - "pull", - "build", - "restart", - "port", - "cp", - "top", -]); -const verb = (() => { - for (const arg of process.argv.slice(2)) { - if (VERBS.has(arg)) { - return arg; - } - } - return "other"; -})(); -const scopes = new AsyncLocalStorage(); -let enabled = typeof FILE === "string" && FILE.startsWith("/"); -let wroteStart = false; - -function append(record: Record): void { - appendFileSync( - FILE as string, - `${JSON.stringify({ p: process.pid, v: verb, ...record })}\n` - ); -} - -function write(record: Record): void { - if (!enabled) { - return; - } - try { - if (!wroteStart) { - wroteStart = true; - append({ k: "start", t: Math.round(performance.now()) }); - process.on("exit", (code) => { - try { - append({ k: "exit", c: code, t: Math.round(performance.now()) }); - } catch { - // measurement only - } - }); - } - append({ - s: scopes.getStore() ?? "", - t: Math.round(performance.now()), - ...record, - }); - } catch { - enabled = false; - } -} - -export function x01Enabled(): boolean { - return enabled; -} - -/** Raw line from the phase trace, re-wrapped with process identity. */ -export function x01Phase(line: string): void { - try { - write({ k: "phase", line: JSON.parse(line) as unknown }); - } catch { - enabled = false; - } -} - -export function x01Label(argv: readonly string[], binary = true): string { - const parts: string[] = []; - let rest = argv; - if (binary) { - const name = (argv[0] ?? "").split("/").at(-1) ?? ""; - parts.push( - ["docker", "ps", "sysctl", "sh", "git", "bun", "tmux"].includes(name) - ? name - : "bin" - ); - rest = argv.slice(1); - } else { - parts.push("docker"); - } - for (const token of rest) { - if (parts.length >= 3) { - break; - } - if (token.startsWith("-")) { - continue; - } - if (SUBS.has(token)) { - parts.push(token); - } else { - break; - } - } - return parts.join(" "); -} - -export async function x01Time( - kind: string, - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const started = performance.now(); - let ok = false; - try { - const result = await operation(); - ok = true; - return result; - } finally { - write({ - k: kind, - l: label, - ms: Math.round((performance.now() - started) * 10) / 10, - ok, - }); - } -} - -/** TEMPORARY X03 measurement: a closed-label point event. */ -export function x01Mark(kind: string, label: string): void { - write({ k: kind, l: label }); -} - -export async function x01Scope( - label: string, - operation: () => Promise -): Promise { - if (!enabled) { - return await operation(); - } - const parent = scopes.getStore(); - const path = parent ? `${parent}>${label}` : label; - return await scopes.run(path, () => x01Time("scope", label, operation)); -} diff --git a/tests/e2e/harness.ts b/tests/e2e/harness.ts index e7599c52..2201457b 100644 --- a/tests/e2e/harness.ts +++ b/tests/e2e/harness.ts @@ -43,8 +43,6 @@ const PASSTHROUGH_ENV_KEYS = [ "LC_ALL", "DOCKER_HOST", "DOCKER_CONFIG", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - "HACK_X01_TRACE_FILE", ] as const; export type CliResult = { @@ -770,13 +768,6 @@ export async function runScenarios( }; process.stdout.write(`-- ${scenario.name} (${scenario.tier}) --\n`); - // TEMPORARY X01 measurement: one trace file per scenario. - if (process.env.HACK_X01_TRACE_DIR?.startsWith("/")) { - process.env.HACK_X01_TRACE_FILE = join( - process.env.HACK_X01_TRACE_DIR, - `${scenario.name}.jsonl` - ); - } let outcome: ScenarioOutcome; let preserveFailedFixture = false; try { diff --git a/tests/e2e/native-config-networks-acceptance.ts b/tests/e2e/native-config-networks-acceptance.ts index 6c519470..633cb044 100644 --- a/tests/e2e/native-config-networks-acceptance.ts +++ b/tests/e2e/native-config-networks-acceptance.ts @@ -1154,10 +1154,6 @@ export const nativeConfigNetworksScenario: Scenario = { NO_COLOR: "1", HACK_RUNTIME_BACKEND: "compose", HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", - // TEMPORARY X01 measurement passthrough; reverted before the final commit. - ...(process.env.HACK_X01_TRACE_FILE - ? { HACK_X01_TRACE_FILE: process.env.HACK_X01_TRACE_FILE } - : {}), }; const engine = Bun.which("docker"); const git = Bun.which("git"); From ebe05399e018a59f394930b0dc16d2bd8099694a Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:14:23 -0400 Subject: [PATCH 27/52] test(agents): cover reference generation and reference-style and HTML skill links Add a temp-root regression test proving generateAgentPlugins writes every registered references/*.md byte for byte and prunes stale reference markdown while keeping non-markdown and out-of-directory files. Extend the offline skill link contract to check Markdown reference-style definitions and HTML href/src attributes, with fixtures proving links into repository docs/ or external hosts are refused and in-plugin targets pass. Co-Authored-By: Claude Opus 5.5 --- tests/agent-plugin-generation.test.ts | 66 +++++++++++++++++--- tests/helpers/skill-links.ts | 88 ++++++++++++++++++++------- tests/skill-references.test.ts | 58 ++++++++++++++++++ 3 files changed, 181 insertions(+), 31 deletions(-) diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 891b7036..85b16ec6 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -26,15 +26,14 @@ afterEach(async () => { } }); -test("release generation applies the requested version to every client manifest", async () => { - tempDir = await mkdtemp(join(tmpdir(), "hack-agent-plugins-")); - const pluginRoot = join(tempDir, "plugins", "hack"); - const manifestDirectories = [ - ".codex-plugin", - ".claude-plugin", - ".cursor-plugin", - ] as const; +const manifestDirectories = [ + ".codex-plugin", + ".claude-plugin", + ".cursor-plugin", +] as const; +/** Seed the client manifests the generator rewrites into a temp plugin root. */ +async function seedPluginManifests(pluginRoot: string): Promise { for (const manifestDirectory of manifestDirectories) { const directory = join(pluginRoot, manifestDirectory); await mkdir(directory, { recursive: true }); @@ -55,6 +54,12 @@ test("release generation applies the requested version to every client manifest" mcpServers: { hack: { command: "hack", args: ["mcp", "serve"] } }, }) ); +} + +test("release generation applies the requested version to every client manifest", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-agent-plugins-")); + const pluginRoot = join(tempDir, "plugins", "hack"); + await seedPluginManifests(pluginRoot); await generateAgentPlugins({ repoRoot: tempDir, version: "9.8.7" }); @@ -69,6 +74,51 @@ test("release generation applies the requested version to every client manifest" ); }); +test("generation writes every registered reference and prunes only stale reference markdown", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-agent-plugin-references-")); + const pluginRoot = join(tempDir, "plugins", "hack"); + await seedPluginManifests(pluginRoot); + const references = renderPluginReferences(); + expect(references.size).toBeGreaterThan(0); + const [firstReference] = references.keys(); + if (!firstReference) { + throw new Error("expected at least one registered reference"); + } + // Generated references are rewritten in place, not left with stale bytes. + await Bun.write(join(pluginRoot, firstReference), "outdated\n"); + const stale = [ + "skills/hack-cli/references/retired.md", + "skills/hack-init/references/retired.md", + ] as const; + const kept = { + "skills/hack-cli/references/notes.txt": "not a reference\n", + "skills/hack-cli/references/diagram.svg": "\n", + "skills/hack-cli/notes.md": "outside references\n", + } as const; + for (const path of stale) { + await Bun.write(join(pluginRoot, path), "# Retired\n"); + } + for (const [path, content] of Object.entries(kept)) { + await Bun.write(join(pluginRoot, path), content); + } + + await generateAgentPlugins({ repoRoot: tempDir, version: "9.8.7" }); + + for (const [path, content] of references) { + const file = Bun.file(join(pluginRoot, path)); + expect(await file.exists()).toBe(true); + expect(new Uint8Array(await file.arrayBuffer())).toEqual( + new TextEncoder().encode(content) + ); + } + for (const path of stale) { + expect(await Bun.file(join(pluginRoot, path)).exists()).toBe(false); + } + for (const [path, content] of Object.entries(kept)) { + expect(await Bun.file(join(pluginRoot, path)).text()).toBe(content); + } +}); + test("semantic-release commits all generated client manifests", async () => { const releaseConfig = await Bun.file( join(import.meta.dir, "..", ".releaserc.json") diff --git a/tests/helpers/skill-links.ts b/tests/helpers/skill-links.ts index fcd41f66..d8ec4fa1 100644 --- a/tests/helpers/skill-links.ts +++ b/tests/helpers/skill-links.ts @@ -13,8 +13,11 @@ import { dirname, isAbsolute, relative, resolve, sep } from "node:path"; * - every `skills//references/*.md` file is linked from that skill's * SKILL.md. * - * Inline code and fenced blocks are literals (commands, schema ids), not - * links, and are ignored. Issues carry closed codes and plugin-relative paths. + * Links are inline Markdown links and images (`[x](target)`), reference-style + * definitions (`[x]: target`), HTML `href`/`src` attributes (``, + * ``), and bare URLs. Inline code and fenced blocks are literals + * (commands, schema ids), not links, and are ignored. Issues carry closed codes + * and plugin-relative paths. */ export type SkillLinkIssueCode = | "absolute-link" @@ -32,8 +35,14 @@ export type SkillLinkIssue = { const FENCED_BLOCK = /```[\s\S]*?```/g; const INLINE_CODE = /`[^`\n]*`/g; -const HEADING = /^#{1,6}\s+(.+?)\s*#*\s*$/; +const HEADING = /^#{1,6}[ \t]+(.+?)[ \t]*#*[ \t]*$/gm; const MARKDOWN_LINK = /\]\(\s*]+)>?(?:\s+"[^"]*")?\s*\)/g; +/** `[label]: target` or `[label]: `; `[^n]:` footnotes are not links. */ +const REFERENCE_DEFINITION = + /^[ \t>]*\[(?!\^)(?:[^\]\\\n]|\\.)+\]:[ \t]*\n?[ \t]*(?:<([^>\n]*)>|([^\s<]\S*))/gm; +const HTML_TAG = /<[a-z][a-z0-9-]*(?:\s[^<>]*)?>/gi; +const HTML_LINK_ATTRIBUTE = + /(?:^|\s)(?:href|src)\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'=<>`]+))/gi; const URL = /\b[a-z][a-z0-9+.-]*:\/\/[^\s)>\]]+/gi; const SCHEME = /^[a-z][a-z0-9+.-]*:/i; const FURTHER_READING = /^further reading$/i; @@ -143,30 +152,63 @@ function referenceLinkIssue(opts: { : "unlinked-reference"; } -/** Yield link targets and URLs with whether they sit under "Further reading". */ -function* scanLinks( - text: string -): Generator<{ readonly target: string; readonly furtherReading: boolean }> { +type ScannedLink = { + readonly target: string; + readonly furtherReading: boolean; +}; + +type Located = { readonly start: number; readonly end: number }; + +/** + * Yield link targets and URLs in document order, each with whether it sits + * under a "Further reading" heading. A bare URL inside an explicit link is not + * yielded twice. + */ +function* scanLinks(text: string): Generator { const prose = text .replace(FENCED_BLOCK, (block) => block.replace(/[^\n]/g, " ")) .replace(INLINE_CODE, (code) => " ".repeat(code.length)); - let furtherReading = false; - for (const line of prose.split("\n")) { - const heading = HEADING.exec(line); - if (heading) { - furtherReading = FURTHER_READING.test(heading[1] ?? ""); - continue; - } - const seen = new Set(); - for (const match of line.matchAll(MARKDOWN_LINK)) { - const target = match[1] ?? ""; - seen.add(target); - yield { target, furtherReading }; + const headings = [...prose.matchAll(HEADING)].map((match) => ({ + start: match.index, + furtherReading: FURTHER_READING.test(match[1] ?? ""), + })); + const furtherReadingAt = (offset: number): boolean => + headings.findLast((heading) => heading.start <= offset)?.furtherReading ?? + false; + + const explicit: (Located & { readonly target: string })[] = []; + const add = (match: RegExpMatchArray, target: string | undefined): void => { + const start = match.index ?? 0; + explicit.push({ + start, + end: start + match[0].length, + target: target ?? "", + }); + }; + for (const match of prose.matchAll(MARKDOWN_LINK)) { + add(match, match[1]); + } + for (const match of prose.matchAll(REFERENCE_DEFINITION)) { + add(match, match[1] ?? match[2]); + } + for (const tag of prose.matchAll(HTML_TAG)) { + for (const attribute of tag[0].matchAll(HTML_LINK_ATTRIBUTE)) { + add(tag, (attribute[1] ?? attribute[2] ?? attribute[3] ?? "").trim()); } - for (const match of line.matchAll(URL)) { - if (!seen.has(match[0])) { - yield { target: match[0], furtherReading }; - } + } + + const links: (Located & { readonly target: string })[] = [...explicit]; + for (const match of prose.matchAll(URL)) { + const start = match.index; + if (!explicit.some((link) => link.start <= start && start < link.end)) { + links.push({ start, end: start + match[0].length, target: match[0] }); } } + links.sort((a, b) => a.start - b.start); + for (const link of links) { + yield { + target: link.target, + furtherReading: furtherReadingAt(link.start), + }; + } } diff --git a/tests/skill-references.test.ts b/tests/skill-references.test.ts index c650199e..35622f3f 100644 --- a/tests/skill-references.test.ts +++ b/tests/skill-references.test.ts @@ -115,6 +115,64 @@ test("negative control: absolute, missing, external and unlinked references fail ]); }); +test("negative control: reference-style and HTML links into repository docs or external hosts fail", async () => { + const root = await fixturePlugin({ + "skills/x/SKILL.md": [ + "# X", + "", + "See [env][env-doc], [guide][guide] and [up][up].", + "", + "[env-doc]: ../../docs/env.md", + "[guide]: https://example.com/guide", + '[up]: <../../../../docs/env.md> "Repository env"', + "", + 'env', + "a", + 'logo', + "', + "", + "[^1]: A footnote definition is prose, not a link target.", + "", + ].join("\n"), + }); + expect(await findSkillLinkIssues({ pluginRoot: root })).toEqual([ + { file: "skills/x/SKILL.md", code: "repo-docs-link" }, + { file: "skills/x/SKILL.md", code: "external-outside-further-reading" }, + { file: "skills/x/SKILL.md", code: "escapes-plugin" }, + { file: "skills/x/SKILL.md", code: "repo-docs-link" }, + { file: "skills/x/SKILL.md", code: "external-outside-further-reading" }, + { file: "skills/x/SKILL.md", code: "escapes-plugin" }, + { file: "skills/x/SKILL.md", code: "external-outside-further-reading" }, + ]); +}); + +test("reference-style and HTML links to in-plugin targets pass and count as reference links", async () => { + const root = await fixturePlugin({ + "skills/y/SKILL.md": [ + "# Y", + "", + 'Read the [patterns][patterns] and the lifecycle notes.', + "", + "[patterns]: references/patterns.md", + "", + 'diagram', + "", + "## Further reading", + "", + "- [Site][site] or the site.", + "", + "[site]: ", + "", + ].join("\n"), + "skills/y/references/patterns.md": "# Patterns\n", + "skills/y/references/lifecycle.md": "# Lifecycle\n", + "skills/y/assets/diagram.svg": "\n", + }); + expect(await findSkillLinkIssues({ pluginRoot: root })).toEqual([]); +}); + test("reference file names are constrained to plain markdown names", () => { for (const file of ["../escape.md", "nested/x.md", "x.txt", "X.md"]) { expect(() => From 6b5498034642663a0eb89f2f03e8f063e9433c64 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:15:56 -0400 Subject: [PATCH 28/52] test: keep the Bun crash loop tests free of real crash reports The executeWithLog test made a real Bun child kill itself with SIGSEGV, so every run wrote a bun-*.ips into the user's DiagnosticReports. In labeled mode the experiment's own full-suite iterations would also have counted those self-induced reports. - The real child now ends itself with SIGTERM, a non-crash signal that still exercises the "signal the harness did not send" path. - A pure classifier test feeds the exact Bun.spawn signal shapes (SIGSEGV, SIGABRT, SIGBUS, SIGILL, SIGTRAP, SIGTERM, SIGKILL). - A fake-executor loop test checks that signal deaths are recorded as crash/signal with a closed signal name. - The real-child test records the bun*.ips listing before and after and asserts it is unchanged. The experiment workflow's concurrency group now contends only for experiment events: a dispatch or a bun-crash-experiment label shares a group, and any other label gets a group keyed by its run id, so it can no longer replace or cancel a pending experiment run. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/bun-crash-experiment.yml | 5 +- tests/bun-crash-loop.test.ts | 113 ++++++++++++++++++++- 2 files changed, 112 insertions(+), 6 deletions(-) diff --git a/.github/workflows/bun-crash-experiment.yml b/.github/workflows/bun-crash-experiment.yml index d7c60965..170bd2e1 100644 --- a/.github/workflows/bun-crash-experiment.yml +++ b/.github/workflows/bun-crash-experiment.yml @@ -31,8 +31,11 @@ on: permissions: contents: read +# Only experiment events contend. A label other than `bun-crash-experiment` gets a +# group of its own (keyed by run id), so its no-op run can never queue behind, +# replace or cancel a pending experiment run on the same pull request. concurrency: - group: bun-crash-experiment-${{ github.event.pull_request.number || github.ref }} + group: bun-crash-experiment-${{ github.event_name == 'workflow_dispatch' && format('dispatch-{0}', github.ref) || github.event.label.name == 'bun-crash-experiment' && format('pr-{0}', github.event.pull_request.number) || format('ignored-label-{0}', github.run_id) }} cancel-in-progress: false jobs: diff --git a/tests/bun-crash-loop.test.ts b/tests/bun-crash-loop.test.ts index 2f575c7d..e7a3ff68 100644 --- a/tests/bun-crash-loop.test.ts +++ b/tests/bun-crash-loop.test.ts @@ -1,6 +1,6 @@ import { afterEach, describe, expect, test } from "bun:test"; import { mkdtemp, readdir, rm, utimes, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; +import { homedir, tmpdir } from "node:os"; import { join } from "node:path"; import { collectDiagnosticReports, @@ -66,6 +66,29 @@ describe("classifyBunRun", () => { ).toEqual({ outcome: "crash", reason: "signal" }); }); + test("every signal shape Bun.spawn reports is a crash, whatever the signal", () => { + // Bun.spawn reports a signal death as exitCode null plus the signal name. + // These shapes are fed directly, so no test process ever receives a crash + // signal (which would make macOS write a real crash report). + for (const signal of [ + "SIGSEGV", + "SIGABRT", + "SIGBUS", + "SIGILL", + "SIGTRAP", + "SIGTERM", + "SIGKILL", + ]) { + expect(classifyBunRun({ exitCode: null, signal, output: "" })).toEqual({ + outcome: "crash", + reason: "signal", + }); + expect( + classifyBunRun({ exitCode: null, signal, output: " 3 pass\n 0 fail\n" }) + ).toEqual({ outcome: "crash", reason: "signal" }); + } + }); + test("(fail) with exit 1 is a non-crash test failure", () => { expect( classifyBunRun({ @@ -257,6 +280,58 @@ describe("runBunCrashLoop", () => { }); }); + test("a signal death from the executor is recorded as a crash with its closed signal name", async () => { + let clock = 0; + const runs: ExecutedRun[] = [ + { + exitCode: null, + signal: "SIGSEGV", + output: "", + stoppedAtDeadline: false, + }, + { + exitCode: null, + signal: "SIGABRT", + output: "", + stoppedAtDeadline: false, + }, + { + exitCode: null, + signal: "not a signal name", + output: "", + stoppedAtDeadline: false, + }, + ]; + const result = await runBunCrashLoop({ + budgetMs: 300_000, + commands: [commands[0] as LoopCommand], + now: () => clock, + execute: async () => { + clock += 100_000; + return ( + runs.shift() ?? { + exitCode: 0, + signal: null, + output: "", + stoppedAtDeadline: false, + } + ); + }, + }); + expect( + result.iterations.map((record) => [ + record.outcome, + record.reason, + record.exitCode, + record.signal, + ]) + ).toEqual([ + ["crash", "signal", null, "SIGSEGV"], + ["crash", "signal", null, "SIGABRT"], + ["crash", "signal", null, null], + ]); + }); + test("the summary is closed and value-free", async () => { let clock = 0; const result = await runBunCrashLoop({ @@ -287,8 +362,26 @@ describe("runBunCrashLoop", () => { }); }); +const BUN_CRASH_REPORT = /^bun.*\.ips$/i; + +/** The `bun*.ips` names in the user's crash-report directory (read-only; empty when absent). */ +async function bunCrashReportNames(): Promise { + try { + const names = await readdir( + join(homedir(), "Library", "Logs", "DiagnosticReports") + ); + return names.filter((name) => BUN_CRASH_REPORT.test(name)).sort(); + } catch { + return []; + } +} + describe("executeWithLog", () => { test("keeps each run's log, reports signals and stops a run at the deadline", async () => { + // Real children here must never die from a crash signal (SIGSEGV, SIGABRT, + // SIGBUS, SIGILL, SIGTRAP): macOS ReportCrash would write a bun-*.ips into + // the user's real DiagnosticReports, whatever HOME is set to. + const reportsBefore = await bunCrashReportNames(); const root = await tempRoot(); const execute = executeWithLog({ cwd: root, logDir: root }); const far = Date.now() + 60_000; @@ -303,16 +396,25 @@ describe("executeWithLog", () => { expect(pass).toMatchObject({ exitCode: 0, stoppedAtDeadline: false }); expect(pass.output).toContain("ok"); - const crashed = await execute( + // SIGTERM is a non-crash signal, so no crash report is written; it still + // exercises the path where a run dies from a signal the harness did not send. + const signaled = await execute( { id: "focused", - argv: [process.execPath, "-e", "process.kill(process.pid, 'SIGSEGV')"], + argv: [process.execPath, "-e", "process.kill(process.pid, 'SIGTERM')"], initialEstimateMs: 1000, }, { index: 2, deadlineMs: far } ); - expect(crashed.signal).toBe("SIGSEGV"); - expect(classifyBunRun(crashed).outcome).toBe("crash"); + expect(signaled).toMatchObject({ + exitCode: null, + signal: "SIGTERM", + stoppedAtDeadline: false, + }); + expect(classifyBunRun(signaled)).toEqual({ + outcome: "crash", + reason: "signal", + }); const stopped = await execute( { @@ -328,6 +430,7 @@ describe("executeWithLog", () => { "iteration-002-focused.log", "iteration-003-full.log", ]); + expect(await bunCrashReportNames()).toEqual(reportsBefore); }); }); From 6b22e00ec4386f0dce2b04cdee6e59b5c9db050b Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:39:52 -0400 Subject: [PATCH 29/52] test(patterns): add validated HACK-P01 and HACK-P02 example fixtures Add minimal example projects for the pattern catalogue's partial cards HACK-P01 (one-shot dependency and generated-artifact bootstrap, HACK-1145) and HACK-P02 (portable toolchain without an always-on app, HACK-1146) under examples/patterns/. Each card gains a contributor-only `example` reference; it is not rendered into the packaged hack-cli reference, so plugin output and render fingerprints are unchanged. tests/pattern-examples.test.ts checks, without starting containers, that: - every card example resolves and every fixture directory belongs to one card; - each hack.config.json passes the project config schema and runtime parser; - the P01 Compose file passes the locked-v1 dependency cache contract (resolved against a temporary copy) with separate generation and gated app; - the P02 toolchain is profiled, digest-pinned, bounded, uses named cache volumes and mounts no Docker socket, home, SSH agent or credentials; - every hack command in the fixture READMEs resolves in CLI_SPEC, names a declared service and profile, covers the card's commands and contains no banned recipe. Negative controls mutate each fixture's config, Compose and README in memory (and a corrupted on-disk copy) and assert the matching closed failure codes. The banned-recipe scan moves to tests/helpers/pattern-recipes.ts unchanged so both suites share it. Co-Authored-By: Claude Opus 5.5 --- docs/agent-guidance.md | 11 + .../p01-dependency-bootstrap/.gitignore | 1 + .../.hack/docker-compose.yml | 57 +++ .../.hack/hack.config.json | 10 + .../p01-dependency-bootstrap/README.md | 64 +++ .../p01-dependency-bootstrap/bun.lock | 15 + .../p01-dependency-bootstrap/package.json | 8 + .../packages/greeting/index.js | 4 + .../packages/greeting/package.json | 6 + .../scripts/generate.ts | 13 + .../p01-dependency-bootstrap/src/server.ts | 20 + .../p02-portable-toolchain/.gitignore | 1 + .../.hack/docker-compose.yml | 21 + .../.hack/hack.config.json | 5 + .../patterns/p02-portable-toolchain/README.md | 48 ++ .../p02-portable-toolchain/package.json | 8 + .../p02-portable-toolchain/src/sum.test.ts | 7 + .../p02-portable-toolchain/src/sum.ts | 4 + src/agents/pattern-catalogue.ts | 22 + tests/helpers/json-schema-subset.ts | 170 +++++++ tests/helpers/pattern-fixtures.ts | 302 ++++++++++++ tests/helpers/pattern-recipes.ts | 52 ++ tests/pattern-catalogue.test.ts | 43 +- tests/pattern-examples.test.ts | 462 ++++++++++++++++++ 24 files changed, 1312 insertions(+), 42 deletions(-) create mode 100644 examples/patterns/p01-dependency-bootstrap/.gitignore create mode 100644 examples/patterns/p01-dependency-bootstrap/.hack/docker-compose.yml create mode 100644 examples/patterns/p01-dependency-bootstrap/.hack/hack.config.json create mode 100644 examples/patterns/p01-dependency-bootstrap/README.md create mode 100644 examples/patterns/p01-dependency-bootstrap/bun.lock create mode 100644 examples/patterns/p01-dependency-bootstrap/package.json create mode 100644 examples/patterns/p01-dependency-bootstrap/packages/greeting/index.js create mode 100644 examples/patterns/p01-dependency-bootstrap/packages/greeting/package.json create mode 100644 examples/patterns/p01-dependency-bootstrap/scripts/generate.ts create mode 100644 examples/patterns/p01-dependency-bootstrap/src/server.ts create mode 100644 examples/patterns/p02-portable-toolchain/.gitignore create mode 100644 examples/patterns/p02-portable-toolchain/.hack/docker-compose.yml create mode 100644 examples/patterns/p02-portable-toolchain/.hack/hack.config.json create mode 100644 examples/patterns/p02-portable-toolchain/README.md create mode 100644 examples/patterns/p02-portable-toolchain/package.json create mode 100644 examples/patterns/p02-portable-toolchain/src/sum.test.ts create mode 100644 examples/patterns/p02-portable-toolchain/src/sum.ts create mode 100644 tests/helpers/json-schema-subset.ts create mode 100644 tests/helpers/pattern-fixtures.ts create mode 100644 tests/helpers/pattern-recipes.ts create mode 100644 tests/pattern-examples.test.ts diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 63df3f65..74d7c5f3 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -93,6 +93,17 @@ and domain owner; the disposition does not waive that owner's runtime gates. `tests/pattern-catalogue.test.ts` checks completeness, CLI_SPEC command resolution and the banned-recipe scan. +A card's `example` field names a static example project under +`examples/patterns/` (today HACK-P01 and HACK-P02). The field is contributor +reference only and is not rendered, because the plugin ships without the +repository's `examples/` tree. `tests/pattern-examples.test.ts` checks that each +reference resolves and every fixture belongs to one card. It also validates +each fixture's `hack.config.json` against the project schema and parser, the +P01 Compose file against the `locked-v1` dependency cache contract, and the P02 +toolchain service against its card. Every `hack` command in a fixture README +must resolve in CLI_SPEC and name a declared service and profile. These checks +are static: they do not start containers or qualify the runtime gates. + Skill candidates and where their patterns live now: | Candidate | Decision | Patterns live in | Promotion owner and missing contract | diff --git a/examples/patterns/p01-dependency-bootstrap/.gitignore b/examples/patterns/p01-dependency-bootstrap/.gitignore new file mode 100644 index 00000000..c2658d7d --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/.gitignore @@ -0,0 +1 @@ +node_modules/ diff --git a/examples/patterns/p01-dependency-bootstrap/.hack/docker-compose.yml b/examples/patterns/p01-dependency-bootstrap/.hack/docker-compose.yml new file mode 100644 index 00000000..7ab4a4cd --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/.hack/docker-compose.yml @@ -0,0 +1,57 @@ +name: pattern-p01 +services: + # One-shot initializer for the immutable dependency payload (HACK-P01). + install: + image: oven/bun:1.4.2-alpine@sha256:d888c0ae6c86d7866ff10c5aafdd9077b36aee6455b33dd270fb93c0dd5cef6f + platform: linux/arm64 + working_dir: /app + entrypoint: [] + command: ["bun", "install", "--frozen-lockfile"] + labels: + hack.dependencies.cache-volume: dependencies + hack.dependencies.lockfiles: bun.lock + hack.dependencies.runtime-files: package.json,packages/greeting/package.json,packages/greeting/index.js + hack.dependencies.cache-protocol: locked-v1 + hack.dependencies.cache-generation: "0" + hack.dependencies.cache-verify: '["bun", "-e", "require.resolve(\"greeting\")"]' + volumes: + - ..:/app + - dependencies:/app/node_modules + + # Worktree-specific generation: always runs, owns its own output volume. + generate: + image: oven/bun:1.4.2-alpine@sha256:d888c0ae6c86d7866ff10c5aafdd9077b36aee6455b33dd270fb93c0dd5cef6f + platform: linux/arm64 + working_dir: /app + command: ["bun", "scripts/generate.ts"] + depends_on: + install: + condition: service_completed_successfully + volumes: + - ..:/app + - dependencies:/app/node_modules:ro + - generated:/generated + + app: + image: oven/bun:1.4.2-alpine@sha256:d888c0ae6c86d7866ff10c5aafdd9077b36aee6455b33dd270fb93c0dd5cef6f + platform: linux/arm64 + working_dir: /app + command: ["bun", "src/server.ts"] + depends_on: + install: + condition: service_completed_successfully + generate: + condition: service_completed_successfully + volumes: + - ..:/app + - dependencies:/app/node_modules:ro + - generated:/generated:ro + healthcheck: + test: ["CMD", "bun", "-e", "const r = await fetch('http://127.0.0.1:3000/healthz'); process.exit(r.ok ? 0 : 1)"] + interval: 5s + timeout: 3s + retries: 12 + +volumes: + dependencies: {} + generated: {} diff --git a/examples/patterns/p01-dependency-bootstrap/.hack/hack.config.json b/examples/patterns/p01-dependency-bootstrap/.hack/hack.config.json new file mode 100644 index 00000000..f60f4a7c --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/.hack/hack.config.json @@ -0,0 +1,10 @@ +{ + "$schema": "https://schemas.hack/hack.config.schema.json", + "name": "pattern-p01", + "dev_host": "pattern-p01.hack.local", + "logs": { + "follow_backend": "compose", + "snapshot_backend": "compose", + "clear_on_down": false + } +} diff --git a/examples/patterns/p01-dependency-bootstrap/README.md b/examples/patterns/p01-dependency-bootstrap/README.md new file mode 100644 index 00000000..636ff42a --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/README.md @@ -0,0 +1,64 @@ +# HACK-P01 example: one-shot dependency and generated-artifact bootstrap + +Minimal project for catalogue card HACK-P01 (tracked in HACK-1145). It shows the +part of the pattern Hack supports today. The card's disposition is **partial**; +see "What this example does not prove" below. + +## Layout + +- `install` is a one-shot initializer. It opts into the + `hack.dependencies.cache-protocol: locked-v1` label, uses a digest-pinned + installer image with an explicit platform, keeps the default (empty) + environment, and writes only the `dependencies` volume. +- `generate` is the worktree-specific generation step. It always runs, owns its + own `generated` volume, and publishes its output by rename. +- `app` starts only after both one-shot services exit successfully. It mounts + `dependencies` and `generated` read-only and has its own health check. + +Installation, generation and application readiness are separate steps. The +verifier label (`require.resolve("greeting")`) checks the real dependency the +application imports, not just a populated directory. + +Set `platform` to the platform of your Docker daemon before you run it. + +## Commands + +Start the project and wait for the detached startup result: + +```sh +hack up --json +``` + +Inspect service state, including the one-shot exits: + +```sh +hack ps --json +``` + +Read the initializer's bounded log: + +```sh +hack logs --no-follow --tail 100 install +``` + +Stop the project. Ordinary `hack down` keeps the dependency volume: + +```sh +hack down +``` + +If an initializer attempt fails or is interrupted, find the cause, then increment +`hack.dependencies.cache-generation` in `.hack/docker-compose.yml` and run +`hack up --json` again. Do not delete protocol markers to force reuse. + +## What this example does not prove + +- Atomic publication of worktree-specific generated artifacts across concurrent + writers. +- Installers that need credentials or environment values. `locked-v1` refuses + explicit `environment` and `env_file` on the initializer. +- Two-worktree or interrupted-writer qualification. Those runtime gates belong to + WU07/WU11. + +The static checks in `tests/pattern-examples.test.ts` validate this config and +its commands. They do not start containers. diff --git a/examples/patterns/p01-dependency-bootstrap/bun.lock b/examples/patterns/p01-dependency-bootstrap/bun.lock new file mode 100644 index 00000000..b86aec31 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/bun.lock @@ -0,0 +1,15 @@ +{ + "lockfileVersion": 2, + "configVersion": 1, + "workspaces": { + "": { + "name": "pattern-p01", + "dependencies": { + "greeting": "file:./packages/greeting", + }, + }, + }, + "packages": { + "greeting": ["greeting@file:packages/greeting", {}], + } +} diff --git a/examples/patterns/p01-dependency-bootstrap/package.json b/examples/patterns/p01-dependency-bootstrap/package.json new file mode 100644 index 00000000..fdcf9570 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/package.json @@ -0,0 +1,8 @@ +{ + "name": "pattern-p01", + "private": true, + "type": "module", + "dependencies": { + "greeting": "file:./packages/greeting" + } +} diff --git a/examples/patterns/p01-dependency-bootstrap/packages/greeting/index.js b/examples/patterns/p01-dependency-bootstrap/packages/greeting/index.js new file mode 100644 index 00000000..5efec645 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/packages/greeting/index.js @@ -0,0 +1,4 @@ +/** Stand-in for an installed dependency that the application imports. */ +export function greet(name) { + return `hello, ${name}`; +} diff --git a/examples/patterns/p01-dependency-bootstrap/packages/greeting/package.json b/examples/patterns/p01-dependency-bootstrap/packages/greeting/package.json new file mode 100644 index 00000000..621d2ca4 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/packages/greeting/package.json @@ -0,0 +1,6 @@ +{ + "name": "greeting", + "version": "1.0.0", + "type": "module", + "main": "index.js" +} diff --git a/examples/patterns/p01-dependency-bootstrap/scripts/generate.ts b/examples/patterns/p01-dependency-bootstrap/scripts/generate.ts new file mode 100644 index 00000000..22671787 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/scripts/generate.ts @@ -0,0 +1,13 @@ +/** + * Worktree-specific generation step. It runs only after the shared dependency + * initializer succeeds and writes only to this instance's `generated` volume. + */ +import { rename, writeFile } from "node:fs/promises"; + +const target = "/generated/build-info.json"; +const pending = `${target}.pending`; +const payload = `${JSON.stringify({ generatedAt: new Date().toISOString() })}\n`; + +// Publish by rename so readers see a complete old or complete new file. +await writeFile(pending, payload); +await rename(pending, target); diff --git a/examples/patterns/p01-dependency-bootstrap/src/server.ts b/examples/patterns/p01-dependency-bootstrap/src/server.ts new file mode 100644 index 00000000..92810081 --- /dev/null +++ b/examples/patterns/p01-dependency-bootstrap/src/server.ts @@ -0,0 +1,20 @@ +import { readFile } from "node:fs/promises"; +import { greet } from "greeting"; + +const buildInfo = JSON.parse( + await readFile("/generated/build-info.json", "utf8") +) as { readonly generatedAt: string }; + +Bun.serve({ + port: 3000, + fetch(request) { + const { pathname } = new URL(request.url); + if (pathname === "/healthz") { + return new Response("ok"); + } + return Response.json({ + message: greet("pattern-p01"), + generatedAt: buildInfo.generatedAt, + }); + }, +}); diff --git a/examples/patterns/p02-portable-toolchain/.gitignore b/examples/patterns/p02-portable-toolchain/.gitignore new file mode 100644 index 00000000..c2658d7d --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/.gitignore @@ -0,0 +1 @@ +node_modules/ diff --git a/examples/patterns/p02-portable-toolchain/.hack/docker-compose.yml b/examples/patterns/p02-portable-toolchain/.hack/docker-compose.yml new file mode 100644 index 00000000..2012afec --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/.hack/docker-compose.yml @@ -0,0 +1,21 @@ +name: pattern-p02 +services: + # One-shot toolchain behind an opt-in profile (HACK-P02). There is no + # long-running application service in this project. + toolchain: + image: oven/bun:1.4.2-alpine@sha256:d888c0ae6c86d7866ff10c5aafdd9077b36aee6455b33dd270fb93c0dd5cef6f + profiles: [toolchain] + init: true + working_dir: /app + cpus: 2 + mem_limit: 2g + environment: + BUN_INSTALL_CACHE_DIR: /cache/bun + volumes: + - ..:/app + - toolchain_dependencies:/app/node_modules + - toolchain_bun_cache:/cache/bun + +volumes: + toolchain_dependencies: {} + toolchain_bun_cache: {} diff --git a/examples/patterns/p02-portable-toolchain/.hack/hack.config.json b/examples/patterns/p02-portable-toolchain/.hack/hack.config.json new file mode 100644 index 00000000..cbd68d06 --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/.hack/hack.config.json @@ -0,0 +1,5 @@ +{ + "$schema": "https://schemas.hack/hack.config.schema.json", + "name": "pattern-p02", + "dev_host": "pattern-p02.hack.local" +} diff --git a/examples/patterns/p02-portable-toolchain/README.md b/examples/patterns/p02-portable-toolchain/README.md new file mode 100644 index 00000000..f092d68d --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/README.md @@ -0,0 +1,48 @@ +# HACK-P02 example: portable toolchain without an always-on application + +Minimal project for catalogue card HACK-P02 (tracked in HACK-1146). It shows the +part of the pattern Hack supports today. The card's disposition is **partial**; +see "What this example does not prove" below. + +## Layout + +The project has no long-running application. `.hack/docker-compose.yml` declares +one one-shot `toolchain` service behind the opt-in `toolchain` profile: + +- a digest-pinned tool image, so every machine runs the same Bun version; +- explicit resource bounds (`cpus`, `mem_limit`); +- separate named volumes for installed dependencies and the Bun download cache; +- no Docker socket, user home, SSH agent or credential directory mounts, and no + `env_file`. Keep project secrets out of the image build context as well. + +## Commands + +Install dependencies into the toolchain's own volume: + +```sh +hack run --profile toolchain toolchain -- bun install --frozen-lockfile +``` + +Run the checks: + +```sh +hack run --profile toolchain toolchain -- bun test +``` + +The command's own exit status is the result. Check it directly; do not pipe the +output through a filter that can hide a failed check. + +The service is one-shot, so there is nothing to keep running. `hack down` keeps +the named cache volumes. + +## What this example does not prove + +- Cache reuse across worktrees with bounded eviction. +- A source receipt for linked worktrees, whose `.git` entry is a pointer file + rather than a directory. +- Owned cache accounting. Those runtime gates belong to WU11. + +A test-only source identity override must never reach release evidence. + +The static checks in `tests/pattern-examples.test.ts` validate this config and +its commands. They do not start containers. diff --git a/examples/patterns/p02-portable-toolchain/package.json b/examples/patterns/p02-portable-toolchain/package.json new file mode 100644 index 00000000..14159702 --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/package.json @@ -0,0 +1,8 @@ +{ + "name": "pattern-p02", + "private": true, + "type": "module", + "scripts": { + "check": "bun test" + } +} diff --git a/examples/patterns/p02-portable-toolchain/src/sum.test.ts b/examples/patterns/p02-portable-toolchain/src/sum.test.ts new file mode 100644 index 00000000..68391b96 --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/src/sum.test.ts @@ -0,0 +1,7 @@ +import { expect, test } from "bun:test"; +import { sum } from "./sum.ts"; + +test("sum adds every value", () => { + expect(sum([1, 2, 3])).toBe(6); + expect(sum([])).toBe(0); +}); diff --git a/examples/patterns/p02-portable-toolchain/src/sum.ts b/examples/patterns/p02-portable-toolchain/src/sum.ts new file mode 100644 index 00000000..c2b7f543 --- /dev/null +++ b/examples/patterns/p02-portable-toolchain/src/sum.ts @@ -0,0 +1,4 @@ +/** Tiny library code so the toolchain service has something to check. */ +export function sum(values: readonly number[]): number { + return values.reduce((total, value) => total + value, 0); +} diff --git a/src/agents/pattern-catalogue.ts b/src/agents/pattern-catalogue.ts index 82e36565..2aa0b783 100644 --- a/src/agents/pattern-catalogue.ts +++ b/src/agents/pattern-catalogue.ts @@ -10,6 +10,9 @@ * - Every `hack` invocation a card names resolves in CLI_SPEC on this branch. * - Unsafe recipes (broad process kills, kill-by-port, placeholder liveness, * secrets in argv) appear only inside negative-example blocks. + * - A card's `example` directory exists and is statically validated by + * tests/pattern-examples.test.ts (config schema, Compose contract, README + * commands); every directory under `examples/patterns/` belongs to one card. * * Consumer names stay in the Linear issues; the shipped reference records * only how many consumers a card generalizes from. @@ -55,6 +58,13 @@ export type PatternCard = { /** Contract Hack still lacks; `null` only for `supported` cards. */ readonly missingContract: string | null; readonly provenance: PatternProvenance; + /** + * Repository-relative directory of a static example project that realizes + * the supported part of the card, or `null` when none exists. Contributor + * reference only: it is not rendered into the packaged reference because the + * plugin ships without the repository's `examples/` tree. + */ + readonly example: string | null; }; /** The complete pattern register, in id order. */ @@ -85,6 +95,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "locked-v1 covers one immutable payload per volume. Atomic publication of worktree-specific generated artifacts, env-dependent installers, and two-worktree or interrupted-writer qualification remain open.", provenance: { consumers: 4, source: "HACK-P01 research register" }, + example: "examples/patterns/p01-dependency-bootstrap", }, { id: "HACK-P02", @@ -108,6 +119,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Cross-worktree cache reuse with bounded eviction, a source receipt for linked worktrees, and owned cache accounting remain open.", provenance: { consumers: 4, source: "HACK-P02 research register" }, + example: "examples/patterns/p02-portable-toolchain", }, { id: "HACK-P03", @@ -134,6 +146,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has no credential availability, expiry or scope check and no machine-identity path for headless runners; the WU27 env contract owns both.", provenance: { consumers: 3, source: "HACK-P03 research register" }, + example: null, }, { id: "HACK-P04", @@ -157,6 +170,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Today's adoption is port-only. Health- and upstream-checked adoption with a recorded adopted-versus-owned state is the WU29 lifecycle ownership contract.", provenance: { consumers: 1, source: "HACK-P04 research register" }, + example: null, }, { id: "HACK-P05", @@ -179,6 +193,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has no branch-scoped external claim/release contract with bounded retry, stop-during-retry and PID-reuse safety.", provenance: { consumers: 1, source: "HACK-P05 research register" }, + example: null, }, { id: "HACK-P06", @@ -202,6 +217,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has no opt-in tunnel lifecycle with external-route readiness and owned-child shutdown.", provenance: { consumers: 1, source: "HACK-P06 research register" }, + example: null, }, { id: "HACK-P07", @@ -229,6 +245,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has env overlays and value-free inspection but no laptop/headless capability contract or headless machine identity; WU27 and WU30 own them.", provenance: { consumers: 1, source: "HACK-P07 research register" }, + example: null, }, { id: "HACK-P08", @@ -251,6 +268,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has no semantic no-op detection or atomic machine-local import contract for env hydration.", provenance: { consumers: 1, source: "HACK-P08 research register" }, + example: null, }, { id: "HACK-P09", @@ -276,6 +294,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Generation/run ownership, active references, holds and interrupted-reclamation recovery are not implemented for run workspaces.", provenance: { consumers: 1, source: "HACK-P09 research register" }, + example: null, }, { id: "HACK-P10", @@ -302,6 +321,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack does not yet distinguish L1, L2 and cold misses, detect a tampered cache, or emit stage timing and provenance receipts.", provenance: { consumers: 3, source: "HACK-P10 research register" }, + example: null, }, { id: "HACK-P11", @@ -330,6 +350,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "A guidance freeze during execution and a typed terminal result for resumed background operations remain open.", provenance: { consumers: 2, source: "HACK-P11 research register" }, + example: null, }, { id: "HACK-P12", @@ -352,6 +373,7 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [ missingContract: "Hack has no multi-lane admission, lease or durable run-provenance contract.", provenance: { consumers: 2, source: "HACK-P12 research register" }, + example: null, }, ]; diff --git a/tests/helpers/json-schema-subset.ts b/tests/helpers/json-schema-subset.ts new file mode 100644 index 00000000..b19b7415 --- /dev/null +++ b/tests/helpers/json-schema-subset.ts @@ -0,0 +1,170 @@ +/** + * Minimal JSON Schema validator for the keyword subset that Hack's legacy + * project config schema (`renderProjectConfigSchemaJson`) uses. + * + * It is closed: a schema keyword outside the supported set is reported as + * `unsupported-keyword` instead of being ignored, so a schema change that this + * helper cannot evaluate fails the caller rather than passing silently. + * Issues carry a JSON pointer and a fixed code; instance values are never + * echoed. + */ + +export type SchemaIssue = { + readonly pointer: string; + readonly code: string; +}; + +const ANNOTATIONS = new Set(["$schema", "$id", "title", "description"]); +const SUPPORTED = new Set([ + "type", + "properties", + "additionalProperties", + "required", + "enum", + "items", + "anyOf", + "minItems", + "uniqueItems", + "minimum", +]); + +type Schema = Readonly>; + +function isSchema(value: unknown): value is Schema { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function isPlainObject( + value: unknown +): value is Readonly> { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function escapePointer(segment: string): string { + return segment.replaceAll("~", "~0").replaceAll("/", "~1"); +} + +function matchesType(value: unknown, type: unknown): boolean { + switch (type) { + case "string": + return typeof value === "string"; + case "boolean": + return typeof value === "boolean"; + case "number": + return typeof value === "number" && Number.isFinite(value); + case "integer": + return typeof value === "number" && Number.isInteger(value); + case "object": + return isPlainObject(value); + case "array": + return Array.isArray(value); + case "null": + return value === null; + default: + return false; + } +} + +function sameJson(left: unknown, right: unknown): boolean { + return JSON.stringify(left) === JSON.stringify(right); +} + +/** Validate `value` against `schema`; an empty result means valid. */ +export function validateJsonSchemaSubset(opts: { + readonly schema: unknown; + readonly value: unknown; +}): SchemaIssue[] { + return validateAt(opts.schema, opts.value, ""); +} + +// biome-ignore lint/complexity/noExcessiveCognitiveComplexity: One switch over the closed keyword set keeps the subset auditable. +function validateAt( + schema: unknown, + value: unknown, + pointer: string +): SchemaIssue[] { + if (schema === true) { + return []; + } + if (schema === false) { + return [{ pointer, code: "forbidden" }]; + } + if (!isSchema(schema)) { + return [{ pointer, code: "invalid-schema" }]; + } + const issues: SchemaIssue[] = []; + for (const keyword of Object.keys(schema)) { + if (!(ANNOTATIONS.has(keyword) || SUPPORTED.has(keyword))) { + issues.push({ pointer, code: "unsupported-keyword" }); + } + } + if (schema.type !== undefined) { + const types = Array.isArray(schema.type) ? schema.type : [schema.type]; + if (!types.some((type) => matchesType(value, type))) { + return [...issues, { pointer, code: "type" }]; + } + } + if ( + Array.isArray(schema.enum) && + !schema.enum.some((entry) => sameJson(entry, value)) + ) { + issues.push({ pointer, code: "enum" }); + } + if ( + typeof schema.minimum === "number" && + typeof value === "number" && + value < schema.minimum + ) { + issues.push({ pointer, code: "minimum" }); + } + if (Array.isArray(schema.anyOf)) { + const matched = schema.anyOf.some( + (option) => validateAt(option, value, pointer).length === 0 + ); + if (!matched) { + issues.push({ pointer, code: "any-of" }); + } + } + if (Array.isArray(value)) { + if (typeof schema.minItems === "number" && value.length < schema.minItems) { + issues.push({ pointer, code: "min-items" }); + } + if ( + schema.uniqueItems === true && + new Set(value.map((entry) => JSON.stringify(entry))).size !== value.length + ) { + issues.push({ pointer, code: "unique-items" }); + } + if (schema.items !== undefined) { + value.forEach((entry, index) => { + issues.push(...validateAt(schema.items, entry, `${pointer}/${index}`)); + }); + } + } + if (isPlainObject(value)) { + const properties = isSchema(schema.properties) ? schema.properties : {}; + if (Array.isArray(schema.required)) { + for (const key of schema.required) { + if (typeof key === "string" && !Object.hasOwn(value, key)) { + issues.push({ + pointer: `${pointer}/${escapePointer(key)}`, + code: "required", + }); + } + } + } + for (const [key, entry] of Object.entries(value)) { + const childPointer = `${pointer}/${escapePointer(key)}`; + if (Object.hasOwn(properties, key)) { + issues.push(...validateAt(properties[key], entry, childPointer)); + } else if (schema.additionalProperties === false) { + issues.push({ pointer: childPointer, code: "additional-property" }); + } else if (schema.additionalProperties !== undefined) { + issues.push( + ...validateAt(schema.additionalProperties, entry, childPointer) + ); + } + } + } + return issues; +} diff --git a/tests/helpers/pattern-fixtures.ts b/tests/helpers/pattern-fixtures.ts new file mode 100644 index 00000000..84113d05 --- /dev/null +++ b/tests/helpers/pattern-fixtures.ts @@ -0,0 +1,302 @@ +import { readdir } from "node:fs/promises"; +import { join, normalize } from "node:path"; +import type { PatternCard } from "../../src/agents/pattern-catalogue.ts"; +import { builtinOptions, type CliSpec } from "../../src/cli/command.ts"; +import { isRecord } from "../../src/lib/guards.ts"; +import { extractHackInvocations } from "./cli-guidance-commands.ts"; + +/** + * Static checks for the HACK-P example fixtures under `examples/patterns/`. + * They read files only: no container runtime, no HOME, no network. Every + * finding is a fixed code (plus a card id or Compose service name the fixture + * authored); config values are never echoed. + */ + +export const PATTERN_EXAMPLES_DIR = "examples/patterns"; + +/** Files every example fixture must provide. */ +export const PATTERN_EXAMPLE_FILES = [ + "README.md", + ".hack/hack.config.json", + ".hack/docker-compose.yml", +] as const; + +/** + * Check that card `example` references resolve to fixture directories and + * that every fixture directory belongs to exactly one card. + */ +export async function exampleReferenceIssues(opts: { + readonly repoRoot: string; + readonly cards: readonly PatternCard[]; +}): Promise { + const issues: string[] = []; + const referenced = new Map(); + for (const card of opts.cards) { + if (card.example === null) { + continue; + } + const dir = normalize(card.example); + const parent = `${PATTERN_EXAMPLES_DIR}/`; + const name = dir.slice(parent.length); + if (!dir.startsWith(parent) || name.length === 0 || name.includes("/")) { + issues.push(`${card.id}: example-outside-patterns-dir`); + continue; + } + if (referenced.has(name)) { + issues.push(`${card.id}: example-shared-with-${referenced.get(name)}`); + continue; + } + referenced.set(name, card.id); + for (const file of PATTERN_EXAMPLE_FILES) { + if (!(await Bun.file(join(opts.repoRoot, dir, file)).exists())) { + issues.push(`${card.id}: missing-${file}`); + } + } + const readme = Bun.file(join(opts.repoRoot, dir, "README.md")); + if ((await readme.exists()) && !(await readme.text()).includes(card.id)) { + issues.push(`${card.id}: readme-missing-card-id`); + } + } + const entries = await readdir(join(opts.repoRoot, PATTERN_EXAMPLES_DIR), { + withFileTypes: true, + }); + for (const entry of entries) { + if (entry.isDirectory() && !referenced.has(entry.name)) { + issues.push("unreferenced-example-directory"); + } + } + return issues; +} + +type ComposeService = Readonly>; + +function composeServices( + compose: unknown +): Readonly> { + if (!(isRecord(compose) && isRecord(compose.services))) { + return {}; + } + const services: Record = {}; + for (const [name, service] of Object.entries(compose.services)) { + if (isRecord(service)) { + services[name] = service; + } + } + return services; +} + +function serviceProfiles(service: ComposeService): readonly string[] { + return Array.isArray(service.profiles) + ? service.profiles.filter( + (profile): profile is string => typeof profile === "string" + ) + : []; +} + +type Mount = { readonly source: string; readonly target: string }; + +function serviceMounts(service: ComposeService): Mount[] | null { + if (service.volumes === undefined) { + return []; + } + if (!Array.isArray(service.volumes)) { + return null; + } + const mounts: Mount[] = []; + for (const mount of service.volumes) { + if (typeof mount === "string") { + const [source = "", target = ""] = mount.split(":"); + mounts.push({ source, target }); + } else if ( + isRecord(mount) && + typeof mount.source === "string" && + typeof mount.target === "string" + ) { + mounts.push({ source: mount.source, target: mount.target }); + } else { + return null; + } + } + return mounts; +} + +const DIGEST_PINNED_IMAGE = /@sha256:[a-f0-9]{64}$/; +const DOCKER_SOCKET = /docker\.sock|\/var\/run\/docker|orbstack/i; +const HOME_SOURCE = /^(?:~|\$\{?HOME\}?|\/Users\/|\/home\/|\/root(?:\/|$))/; +const SSH_SOURCE = /SSH_AUTH_SOCK|\.ssh(?:\/|$)|ssh-agent/i; +const CREDENTIAL_SOURCE = + /\.(?:aws|config\/gcloud|kube|docker|npmrc|netrc|gnupg)(?:\/|$)|credentials/i; + +function mountIssues(opts: { + readonly mount: Mount; + readonly namedVolumes: ReadonlySet; +}): string[] { + const { source } = opts.mount; + if (DOCKER_SOCKET.test(source)) { + return ["forbidden-mount-docker-socket"]; + } + if (HOME_SOURCE.test(source)) { + return ["forbidden-mount-home"]; + } + if (SSH_SOURCE.test(source)) { + return ["forbidden-mount-ssh"]; + } + if (CREDENTIAL_SOURCE.test(source)) { + return ["forbidden-mount-credentials"]; + } + // The project checkout is the only permitted bind mount. + if (source === "..") { + return []; + } + return opts.namedVolumes.has(source) ? [] : ["cache-not-named-volume"]; +} + +/** + * HACK-P02 static contract for a portable toolchain service: one-shot behind + * an opt-in profile, digest-pinned image, explicit resource bounds, caches in + * named volumes, and no Docker socket, home, SSH agent or credential mounts. + * The project must not declare an always-on service. + */ +// biome-ignore lint/complexity/noExcessiveCognitiveComplexity: A flat list of independent card checks reads best in one place. +export function toolchainServiceIssues(opts: { + readonly compose: unknown; + readonly service: string; +}): string[] { + const services = composeServices(opts.compose); + const service = services[opts.service]; + if (service === undefined) { + return ["missing-service"]; + } + const issues: string[] = []; + for (const [name, entry] of Object.entries(services)) { + if (serviceProfiles(entry).length === 0) { + issues.push(`always-on-service-${name}`); + } + } + if ( + !( + typeof service.image === "string" && + DIGEST_PINNED_IMAGE.test(service.image) + ) + ) { + issues.push("image-not-digest-pinned"); + } + if (service.cpus === undefined) { + issues.push("missing-cpu-bound"); + } + if (service.mem_limit === undefined) { + issues.push("missing-memory-bound"); + } + if (service.restart !== undefined && service.restart !== "no") { + issues.push("restart-policy"); + } + if (service.ports !== undefined) { + issues.push("published-ports"); + } + if (service.env_file !== undefined) { + issues.push("env-file"); + } + if (service.privileged === true) { + issues.push("privileged"); + } + const compose = opts.compose; + const namedVolumes = new Set( + isRecord(compose) && isRecord(compose.volumes) + ? Object.keys(compose.volumes) + : [] + ); + const mounts = serviceMounts(service); + if (mounts === null) { + issues.push("unreadable-mounts"); + } else { + for (const mount of mounts) { + issues.push(...mountIssues({ mount, namedVolumes })); + } + if (!mounts.some((mount) => namedVolumes.has(mount.source))) { + issues.push("no-named-cache-volume"); + } + } + return [...new Set(issues)]; +} + +const SERVICE_COMMANDS: ReadonlySet = new Set(["run", "exec", "logs"]); + +/** + * Check that every `hack run|exec|logs ` in a README names a Compose + * service, and that each `--profile` it passes is one of that service's + * profiles (a profiled service also needs its profile on the command line). + */ +// biome-ignore lint/complexity/noExcessiveCognitiveComplexity: Token walking keeps option arity and positional selection in one pass. +export function readmeServiceIssues(opts: { + readonly cli: CliSpec; + readonly readme: string; + readonly compose: unknown; +}): string[] { + const services = composeServices(opts.compose); + const issues: string[] = []; + for (const invocation of extractHackInvocations(opts.readme)) { + const tokens = invocation.split(/\s+/).slice(1); + const name = tokens[0]; + if (name === undefined || !SERVICE_COMMANDS.has(name)) { + continue; + } + const command = opts.cli.commands.find((entry) => entry.name === name); + if (!command) { + issues.push(`unknown-command-${name}`); + continue; + } + const options = [ + ...builtinOptions(), + ...opts.cli.globalOptions, + ...command.options, + ]; + const profiles: string[] = []; + const positionals: string[] = []; + for (let index = 1; index < tokens.length; index += 1) { + const token = tokens[index] ?? ""; + if (token === "--") { + break; + } + if (!token.startsWith("--")) { + positionals.push(token); + continue; + } + const [flag = "", inline] = token.split("=", 2); + const option = options.find((entry) => entry.long === flag); + if (option === undefined || option.type === "boolean") { + continue; + } + const value = inline ?? tokens[index + 1] ?? ""; + if (inline === undefined) { + index += 1; + } + if (option.long === "--profile") { + profiles.push(...value.split(",")); + } + } + const serviceName = positionals[0]; + if (serviceName === undefined) { + if (name !== "logs") { + issues.push(`${name}-missing-service`); + } + continue; + } + const service = services[serviceName]; + if (service === undefined) { + issues.push(`${name}-unknown-service`); + continue; + } + const declared = serviceProfiles(service); + if (profiles.some((profile) => !declared.includes(profile))) { + issues.push(`${name}-undeclared-profile`); + } + if ( + name !== "logs" && + declared.length > 0 && + !profiles.some((profile) => declared.includes(profile)) + ) { + issues.push(`${name}-missing-profile`); + } + } + return issues; +} diff --git a/tests/helpers/pattern-recipes.ts b/tests/helpers/pattern-recipes.ts new file mode 100644 index 00000000..fe447a02 --- /dev/null +++ b/tests/helpers/pattern-recipes.ts @@ -0,0 +1,52 @@ +import { + NEGATIVE_EXAMPLE_END, + NEGATIVE_EXAMPLE_START, +} from "../../src/agents/pattern-catalogue.ts"; + +/** + * Banned-recipe scan shared by the pattern catalogue and its example fixtures. + * Findings are closed codes; the scanned text is never echoed. + */ + +/** Recipes that must never appear outside a negative-example block. */ +export const BANNED_RECIPES: readonly (readonly [string, RegExp])[] = [ + ["broad-process-kill", /\b(?:pkill|killall)\b/], + ["kill-by-port", /\blsof\b[^\n|]*-t[^\n|]*\|\s*xargs\s+(?:-\S+\s+)*kill\b/], + ["placeholder-liveness-tail", /\btail\s+-f\s+\/dev\/null\b/], + ["placeholder-liveness-sleep", /\bsleep\s+infinity\b/], + [ + "argv-secret-flag", + /--(?:password|passwd|token|secret|api-key|apikey|access-key)(?:=|\s+)\S/i, + ], + [ + "argv-secret-assignment", + /\b[A-Z][A-Z0-9_]*(?:TOKEN|SECRET|PASSWORD|PASSWD|API_KEY)=\S/, + ], + ["argv-env-value", /\bhack\s+env\s+(?:set|add)\s+\S+\s+\S+/], +]; + +function stripNegativeExamples(text: string): string { + let result = ""; + let rest = text; + for (;;) { + const start = rest.indexOf(NEGATIVE_EXAMPLE_START); + if (start === -1) { + return result + rest; + } + const end = rest.indexOf(NEGATIVE_EXAMPLE_END, start); + if (end === -1) { + // An unterminated block exempts nothing. + return result + rest; + } + result += rest.slice(0, start); + rest = rest.slice(end + NEGATIVE_EXAMPLE_END.length); + } +} + +/** Codes of banned recipes found outside negative-example blocks. */ +export function bannedRecipeFindings(text: string): string[] { + const scanned = stripNegativeExamples(text); + return BANNED_RECIPES.filter(([, pattern]) => pattern.test(scanned)).map( + ([code]) => code + ); +} diff --git a/tests/pattern-catalogue.test.ts b/tests/pattern-catalogue.test.ts index 96e6df6b..4aca9e39 100644 --- a/tests/pattern-catalogue.test.ts +++ b/tests/pattern-catalogue.test.ts @@ -14,6 +14,7 @@ import { extractHackInvocations, findGuidanceCommandIssues, } from "./helpers/cli-guidance-commands.ts"; +import { bannedRecipeFindings } from "./helpers/pattern-recipes.ts"; const EXPECTED_IDS = Array.from( { length: 12 }, @@ -37,48 +38,6 @@ function catalogueIdIssues(cards: readonly PatternCard[]): string[] { ]; } -/** Recipes that must never appear outside a negative-example block. */ -const BANNED_RECIPES: readonly (readonly [string, RegExp])[] = [ - ["broad-process-kill", /\b(?:pkill|killall)\b/], - ["kill-by-port", /\blsof\b[^\n|]*-t[^\n|]*\|\s*xargs\s+(?:-\S+\s+)*kill\b/], - ["placeholder-liveness-tail", /\btail\s+-f\s+\/dev\/null\b/], - ["placeholder-liveness-sleep", /\bsleep\s+infinity\b/], - [ - "argv-secret-flag", - /--(?:password|passwd|token|secret|api-key|apikey|access-key)(?:=|\s+)\S/i, - ], - [ - "argv-secret-assignment", - /\b[A-Z][A-Z0-9_]*(?:TOKEN|SECRET|PASSWORD|PASSWD|API_KEY)=\S/, - ], - ["argv-env-value", /\bhack\s+env\s+(?:set|add)\s+\S+\s+\S+/], -]; - -function stripNegativeExamples(text: string): string { - let result = ""; - let rest = text; - for (;;) { - const start = rest.indexOf(NEGATIVE_EXAMPLE_START); - if (start === -1) { - return result + rest; - } - const end = rest.indexOf(NEGATIVE_EXAMPLE_END, start); - if (end === -1) { - // An unterminated block exempts nothing. - return result + rest; - } - result += rest.slice(0, start); - rest = rest.slice(end + NEGATIVE_EXAMPLE_END.length); - } -} - -function bannedRecipeFindings(text: string): string[] { - const scanned = stripNegativeExamples(text); - return BANNED_RECIPES.filter(([, pattern]) => pattern.test(scanned)).map( - ([code]) => code - ); -} - test("the catalogue holds exactly the twelve HACK-P cards in order", () => { expect(catalogueIdIssues(PATTERN_CATALOGUE)).toEqual([]); expect(PATTERN_CATALOGUE.map((card) => card.id)).toEqual(EXPECTED_IDS); diff --git a/tests/pattern-examples.test.ts b/tests/pattern-examples.test.ts new file mode 100644 index 00000000..a9ec9d6a --- /dev/null +++ b/tests/pattern-examples.test.ts @@ -0,0 +1,462 @@ +import { afterEach, expect, test } from "bun:test"; +import { cp, mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, relative } from "node:path"; +import { YAML } from "bun"; + +import { + PATTERN_CATALOGUE, + type PatternCard, +} from "../src/agents/pattern-catalogue.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { resolveDependencyCacheOverride } from "../src/lib/dependency-cache.ts"; +import { isRecord } from "../src/lib/guards.ts"; +import { + findProjectContextAtRoot, + readProjectConfig, +} from "../src/lib/project.ts"; +import { renderProjectConfigSchemaJson } from "../src/templates.ts"; +import { + extractHackInvocations, + findGuidanceCommandIssues, +} from "./helpers/cli-guidance-commands.ts"; +import { validateJsonSchemaSubset } from "./helpers/json-schema-subset.ts"; +import { + exampleReferenceIssues, + readmeServiceIssues, + toolchainServiceIssues, +} from "./helpers/pattern-fixtures.ts"; +import { bannedRecipeFindings } from "./helpers/pattern-recipes.ts"; + +/** + * Static validation of the HACK-P01/P02 example fixtures. Nothing here starts + * containers or contacts Docker; the dependency-cache resolution runs against + * a temporary copy so generated `.hack/.internal` files never land in the + * repository. + */ + +const REPO_ROOT = join(import.meta.dir, ".."); +const P01_DIR = "examples/patterns/p01-dependency-bootstrap"; +const P02_DIR = "examples/patterns/p02-portable-toolchain"; + +const roots: string[] = []; +afterEach(async () => { + for (const root of roots.splice(0)) { + await rm(root, { recursive: true, force: true }); + } +}); + +function card(id: string): PatternCard { + const found = PATTERN_CATALOGUE.find((entry) => entry.id === id); + if (!found) { + throw new Error("missing card"); + } + return found; +} + +function exampleDir(id: string): string { + const example = card(id).example; + if (example === null) { + throw new Error("card has no example"); + } + return join(REPO_ROOT, example); +} + +async function readText(dir: string, file: string): Promise { + return await Bun.file(join(dir, file)).text(); +} + +async function readCompose(dir: string): Promise> { + const parsed: unknown = YAML.parse( + await readText(dir, ".hack/docker-compose.yml") + ); + if (!isRecord(parsed)) { + throw new Error("compose root must be a mapping"); + } + return parsed; +} + +async function readConfigJson(dir: string): Promise> { + const parsed: unknown = JSON.parse( + await readText(dir, ".hack/hack.config.json") + ); + if (!isRecord(parsed)) { + throw new Error("config root must be an object"); + } + return parsed; +} + +function projectSchema(): unknown { + return JSON.parse(renderProjectConfigSchemaJson()); +} + +/** Structured clone so mutations never touch the parsed fixture. */ +function clone(value: T): T { + return structuredClone(value); +} + +function service( + compose: Record, + name: string +): Record { + const services = compose.services; + const entry = isRecord(services) ? services[name] : undefined; + if (!isRecord(entry)) { + throw new Error("missing service"); + } + return entry; +} + +async function copyExample(id: string): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-pattern-example-")); + roots.push(root); + const target = join(root, "project"); + await cp(exampleDir(id), target, { recursive: true }); + return target; +} + +async function resolveP01Cache(opts: { + readonly projectRoot: string; + readonly compose: unknown; +}) { + const projectDir = join(opts.projectRoot, ".hack"); + const composeFile = join(projectDir, "docker-compose.yml"); + await Bun.write(composeFile, YAML.stringify(opts.compose)); + return await resolveDependencyCacheOverride({ + projectRoot: opts.projectRoot, + projectDir, + composeFile, + projectName: "pattern-p01", + }); +} + +test("P01 and P02 reference their example fixtures and every fixture belongs to a card", async () => { + expect(card("HACK-P01").example).toBe(P01_DIR); + expect(card("HACK-P02").example).toBe(P02_DIR); + expect( + await exampleReferenceIssues({ + repoRoot: REPO_ROOT, + cards: PATTERN_CATALOGUE, + }) + ).toEqual([]); +}); + +test("negative control: a broken, shared or escaping example reference fails", async () => { + const withExample = (id: string, example: string | null) => + PATTERN_CATALOGUE.map((entry) => + entry.id === id ? { ...entry, example } : entry + ); + expect( + await exampleReferenceIssues({ + repoRoot: REPO_ROOT, + cards: withExample("HACK-P01", "examples/patterns/missing"), + }) + ).toEqual([ + "HACK-P01: missing-README.md", + "HACK-P01: missing-.hack/hack.config.json", + "HACK-P01: missing-.hack/docker-compose.yml", + "unreferenced-example-directory", + ]); + expect( + await exampleReferenceIssues({ + repoRoot: REPO_ROOT, + cards: withExample("HACK-P03", P01_DIR), + }) + ).toEqual(["HACK-P03: example-shared-with-HACK-P01"]); + expect( + await exampleReferenceIssues({ + repoRoot: REPO_ROOT, + cards: withExample("HACK-P02", "examples/basic"), + }) + ).toEqual([ + "HACK-P02: example-outside-patterns-dir", + "unreferenced-example-directory", + ]); + expect( + await exampleReferenceIssues({ + repoRoot: REPO_ROOT, + cards: withExample("HACK-P02", "examples/patterns/../basic"), + }) + ).toEqual([ + "HACK-P02: example-outside-patterns-dir", + "unreferenced-example-directory", + ]); +}); + +test("each fixture's hack.config.json validates against the project schema and parser", async () => { + for (const id of ["HACK-P01", "HACK-P02"]) { + const dir = exampleDir(id); + const config = await readConfigJson(dir); + expect( + validateJsonSchemaSubset({ schema: projectSchema(), value: config }), + id + ).toEqual([]); + + const context = await findProjectContextAtRoot({ projectRoot: dir }); + expect(context?.projectDirName, id).toBe(".hack"); + if (!context) { + throw new Error("fixture is not a legacy project root"); + } + const parsed = await readProjectConfig(context); + expect(parsed.parseError, id).toBeUndefined(); + expect(parsed.name, id).toBe(config.name as string); + expect((await readCompose(dir)).name, id).toBe(config.name); + } +}); + +test("negative control: breaking a fixture config fails schema or parser validation", async () => { + const schema = projectSchema(); + const p01 = await readConfigJson(exampleDir("HACK-P01")); + + const { dev_host: _removedHost, ...withoutHost } = clone(p01); + expect(validateJsonSchemaSubset({ schema, value: withoutHost })).toEqual([ + { pointer: "/dev_host", code: "required" }, + ]); + + const badBackend = clone(p01); + badBackend.logs = { follow_backend: "syslog" }; + expect(validateJsonSchemaSubset({ schema, value: badBackend })).toEqual([ + { pointer: "/logs/follow_backend", code: "enum" }, + ]); + + const badName = clone(p01); + badName.name = 42; + expect(validateJsonSchemaSubset({ schema, value: badName })).toEqual([ + { pointer: "/name", code: "type" }, + ]); + + // The validator refuses keywords it cannot evaluate instead of passing. + expect( + validateJsonSchemaSubset({ + schema: { type: "string", pattern: "^x$" }, + value: "x", + }) + ).toEqual([{ pointer: "", code: "unsupported-keyword" }]); + + // On disk: a corrupted copy is reported by the runtime parser. + const copy = await copyExample("HACK-P02"); + await Bun.write(join(copy, ".hack/hack.config.json"), "{ not json"); + const context = await findProjectContextAtRoot({ projectRoot: copy }); + if (!context) { + throw new Error("copied fixture is not a legacy project root"); + } + expect((await readProjectConfig(context)).parseError).toBeDefined(); +}); + +test("P01 compose passes the locked-v1 dependency cache contract", async () => { + const projectRoot = await copyExample("HACK-P01"); + const compose = await readCompose(projectRoot); + const resolution = await resolveP01Cache({ projectRoot, compose }); + + expect(resolution.progressServices).toEqual(["install"]); + expect(resolution.sharingDisabledReason).toBeUndefined(); + expect(resolution.fingerprint).toMatch(/^[a-f0-9]{16}$/); + expect( + resolution.volumes.map((volume) => ({ + logicalName: volume.logicalName, + services: volume.services, + })) + ).toEqual([{ logicalName: "dependencies", services: ["install"] }]); + expect( + resolution.inputs.map((input) => relative(projectRoot, input)) + ).toEqual([ + "bun.lock", + "package.json", + "packages/greeting/index.js", + "packages/greeting/package.json", + ]); + + // Generation is a separate always-run step with its own output volume, and + // the application waits for both one-shot steps. + const generate = service(compose, "generate"); + const app = service(compose, "app"); + expect(generate.labels).toBeUndefined(); + expect(generate.volumes).toContain("generated:/generated"); + expect(service(compose, "install").volumes).not.toContain( + "generated:/generated" + ); + expect(app.depends_on).toEqual({ + install: { condition: "service_completed_successfully" }, + generate: { condition: "service_completed_successfully" }, + }); + expect(app.volumes).toContain("generated:/generated:ro"); + expect(app.healthcheck).toBeDefined(); +}); + +test("negative control: breaking the P01 compose contract is refused", async () => { + const projectRoot = await copyExample("HACK-P01"); + const original = await readCompose(projectRoot); + const mutations: Readonly< + Record) => void> + > = { + "writable consumer mount": (compose) => { + service(compose, "app").volumes = [ + "..:/app", + "dependencies:/app/node_modules", + "generated:/generated:ro", + ]; + }, + "missing completion gate": (compose) => { + service(compose, "generate").depends_on = undefined; + }, + "initializer environment": (compose) => { + service(compose, "install").environment = { NODE_ENV: "production" }; + }, + "unresolved installer image": (compose) => { + service(compose, "install").image = "oven/bun:${BUN_TAG}"; + }, + "string initializer command": (compose) => { + service(compose, "install").command = "bun install --frozen-lockfile"; + }, + "named cache volume": (compose) => { + compose.volumes = { + dependencies: { name: "shared-dependencies" }, + generated: {}, + }; + }, + }; + // Baseline: the unmodified copy resolves, so each refusal is the mutation's. + expect( + (await resolveP01Cache({ projectRoot, compose: original })).progressServices + ).toEqual(["install"]); + for (const [name, mutate] of Object.entries(mutations)) { + const compose = clone(original); + mutate(compose); + let message = ""; + try { + await resolveP01Cache({ projectRoot, compose }); + } catch (error: unknown) { + message = error instanceof Error ? error.message : "non-error"; + } + expect(message, name).toMatch(/dependency cache/i); + } +}); + +test("P02 compose is a profiled, pinned, bounded one-shot toolchain", async () => { + const compose = await readCompose(exampleDir("HACK-P02")); + expect(toolchainServiceIssues({ compose, service: "toolchain" })).toEqual([]); +}); + +test("negative control: breaking the P02 toolchain contract fails", async () => { + const original = await readCompose(exampleDir("HACK-P02")); + const check = (mutate: (compose: Record) => void) => { + const compose = clone(original); + mutate(compose); + return toolchainServiceIssues({ compose, service: "toolchain" }); + }; + const addMount = (mount: string) => (compose: Record) => { + const toolchain = service(compose, "toolchain"); + toolchain.volumes = [...(toolchain.volumes as unknown[]), mount]; + }; + expect(check(addMount("/var/run/docker.sock:/var/run/docker.sock"))).toEqual([ + "forbidden-mount-docker-socket", + ]); + expect(check(addMount("~/.config:/root/.config"))).toEqual([ + "forbidden-mount-home", + ]); + expect(check(addMount("$SSH_AUTH_SOCK:/ssh-agent"))).toEqual([ + "forbidden-mount-ssh", + ]); + expect(check(addMount("/opt/cache:/cache/other"))).toEqual([ + "cache-not-named-volume", + ]); + expect( + check((compose) => { + service(compose, "toolchain").profiles = undefined; + }) + ).toEqual(["always-on-service-toolchain"]); + expect( + check((compose) => { + service(compose, "toolchain").image = "oven/bun:latest"; + }) + ).toEqual(["image-not-digest-pinned"]); + expect( + check((compose) => { + const toolchain = service(compose, "toolchain"); + toolchain.cpus = undefined; + toolchain.mem_limit = undefined; + }) + ).toEqual(["missing-cpu-bound", "missing-memory-bound"]); + expect( + check((compose) => { + service(compose, "toolchain").env_file = [".env"]; + }) + ).toEqual(["env-file"]); + expect( + check((compose) => { + (compose.services as Record).app = { + image: "nginx", + }; + }) + ).toEqual(["always-on-service-app"]); +}); + +test("every hack command in the fixture READMEs resolves in CLI_SPEC and names real services", async () => { + for (const id of ["HACK-P01", "HACK-P02"]) { + const dir = exampleDir(id); + const readme = await readText(dir, "README.md"); + const compose = await readCompose(dir); + const invocations = extractHackInvocations(readme); + expect(invocations.length, id).toBeGreaterThan(0); + expect( + findGuidanceCommandIssues({ cli: CLI_SPEC, rendered: readme }), + id + ).toEqual([]); + expect(readmeServiceIssues({ cli: CLI_SPEC, readme, compose }), id).toEqual( + [] + ); + // The fixture exercises each command family its card names. + for (const command of card(id).supportedCommands) { + const prefix = command.split(" <")[0] ?? command; + expect( + invocations.some((invocation) => invocation.startsWith(prefix)), + `${id}: ${prefix}` + ).toBe(true); + } + expect(bannedRecipeFindings(readme), id).toEqual([]); + expect( + bannedRecipeFindings(await readText(dir, ".hack/docker-compose.yml")), + id + ).toEqual([]); + } +}); + +test("negative control: an invalid README command or service reference fails", async () => { + const dir = exampleDir("HACK-P02"); + const readme = await readText(dir, "README.md"); + const compose = await readCompose(dir); + const withLine = (line: string) => `${readme}\n\`${line}\`\n`; + + expect( + findGuidanceCommandIssues({ + cli: CLI_SPEC, + rendered: withLine( + "hack run --profile toolchain --fast toolchain -- bun test" + ), + }).length + ).toBe(1); + expect( + readmeServiceIssues({ + cli: CLI_SPEC, + compose, + readme: withLine("hack run --profile other toolchain -- bun test"), + }) + ).toEqual(["run-undeclared-profile", "run-missing-profile"]); + expect( + readmeServiceIssues({ + cli: CLI_SPEC, + compose, + readme: withLine("hack run toolchain -- bun test"), + }) + ).toEqual(["run-missing-profile"]); + expect( + readmeServiceIssues({ + cli: CLI_SPEC, + compose, + readme: withLine("hack logs --no-follow --tail 100 nope"), + }) + ).toEqual(["logs-unknown-service"]); + expect( + bannedRecipeFindings(`${readme}\n command: sleep infinity\n`) + ).toEqual(["placeholder-liveness-sleep"]); +}); From b07121f8bd10c7835168fd25812e6803507e4c14 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:40:58 -0400 Subject: [PATCH 30/52] feat(exec): warn value-free when a running service's env changed since start `hack up` and `hack restart` now record, per Compose service, a salted HMAC-SHA256 digest of the env set delivered to it in the Hack-managed runtime state (.hack/.internal/runtime-state.json). `hack exec` resolves the same env set through the shared --env resolver and, when the digest differs, prints one closed stderr warning naming the service and suggesting `hack restart `. No key or value is stored or printed; exec still runs with the container's start-time env. Service-scoped up and targeted restart refresh only their services' digests; a failed targeted restart or a different env selection drops them so exec reports nothing rather than guessing. State without digests never warns. Adds the tier-2 docker e2e scenario exec-stale-env, registered in the default set so it runs only in the hosted docker-e2e shards. Refs HACK-1140 Co-Authored-By: Claude Opus 5.5 --- docs/env.md | 17 +- src/commands/project.ts | 55 +++- src/lib/compose-env-fingerprint.ts | 213 ++++++++++++ src/lib/project-runtime-state.ts | 87 +++++ tests/compose-env-staleness.test.ts | 438 +++++++++++++++++++++++++ tests/e2e/run.ts | 2 + tests/e2e/scenarios/exec-stale-env.ts | 183 +++++++++++ tests/e2e/shard.ts | 2 + tests/project-exec-command.test.ts | 123 ++++++- tests/project-up-startup-state.test.ts | 58 ++++ 10 files changed, 1175 insertions(+), 3 deletions(-) create mode 100644 src/lib/compose-env-fingerprint.ts create mode 100644 tests/compose-env-staleness.test.ts create mode 100644 tests/e2e/scenarios/exec-stale-env.ts diff --git a/docs/env.md b/docs/env.md index a9e1eed5..112dd3bd 100644 --- a/docs/env.md +++ b/docs/env.md @@ -234,7 +234,7 @@ into a host command, and `--target` on env and host commands chooses the `host` | `hack down` | `--env`, `--target` | At teardown, for lifecycle down hooks only; omitted reuses a named overlay recorded for the running stack, else the configured default. | None; nothing is delivered to containers. | The native runtime backend refuses `--env`. | | `hack restart` | `--env`, `--target` | When services are recreated; omitted reuses a named overlay recorded for the running stack, else the configured default, and an explicit `--env` (including `base`) wins. | This is the refresh path for the whole stack or the named services. | Authored native projects receive the same selection before dispatch. | | `hack run` | `--env` | Per invocation, for the new one-off container. | Each run resolves again. | The native runtime backend refuses `--env`; restart with the selection first. | -| `hack exec` | `--env` | Checked at invocation against the env recorded for the running stack; the command runs with the env the container was created with. | `hack restart --env ` | Compose refuses when the requested env differs from the env recorded for the running stack; the native runtime backend refuses `--env`. | +| `hack exec` | `--env` | Checked at invocation against the env recorded for the running stack; the command runs with the env the container was created with. Compose warns on stderr when the service's resolved env changed since it started. | `hack restart ` (or `hack restart --env `) | Compose refuses when the requested env differs from the env recorded for the running stack; the native runtime backend refuses `--env`. | | `hack env list` | `--env`, `--service` | Per invocation, read-only; secret values are masked unless `--show-secrets`. | Not needed. | Reads files; no runtime required. | | `hack env get` | `--env`, `--service` | Per invocation; prints exactly one plaintext value to stdout. | Not needed. | Reads files; refuses native project inputs. | | `hack env explain` | `--env`, `--service`, `--target` | Per invocation; reports source, precedence and delivery without the value. | Not needed. | Reads files; no runtime required. | @@ -252,6 +252,21 @@ into a host command, and `--target` on env and host commands chooses the `host` | `hack session start` | `--env`, `--service` | When the workspace is created; the name gains `.env-` (`.env-base` for base spellings), and `--up` runs `hack up -d` with the same selection. | Start a new workspace or pass `--env` to `hack session exec`. | Injects into the workspace shell, not into containers. | | `hack session exec` | `--env`, `--service` | Per command; omitted reuses the selection encoded in the workspace name. | Each command resolves again. | Injects into the workspace command, not into containers. | +### Stale env warning on `hack exec` + +`hack up` and `hack restart` record, for each Compose service they create, a keyed +digest of the env set delivered to it. The digest lives in Hack-managed runtime state +(`.hack/.internal/runtime-state.json`) and holds no key name or value. `hack exec` +resolves the same env set again and, when the digest differs, prints one fixed warning +to stderr that names the service and suggests `hack restart `. The command +still runs with the env the container was created with, and stdout stays the target +command's output. The warning never prints a key or value. + +The check is advisory. Identical env, state written before this check existed, and a +service whose last targeted restart failed produce no warning. `hack up `, +`hack restart ` and `hack env apply --service ` refresh only the +named services' digests. The native runtime backend does not run this check. + MCP tools accept no env selector. They run the CLI without `--env`, so each row's omitted-`--env` behavior applies. diff --git a/src/commands/project.ts b/src/commands/project.ts index d8d96763..4c2ecc4a 100644 --- a/src/commands/project.ts +++ b/src/commands/project.ts @@ -149,6 +149,7 @@ import { HackCliError, okResult, } from "../lib/cli-result.ts"; +import { withComposeServiceEnvFingerprints } from "../lib/compose-env-fingerprint.ts"; import { renderManagedComposeEnvOverride } from "../lib/compose-managed-env.ts"; import { DEFAULT_COMPOSE_STARTUP_TIMEOUT_MS, @@ -242,6 +243,8 @@ import { normalizeProjectName } from "../lib/project-name.ts"; import { readProjectRuntimeStateEntry, removeProjectRuntimeStateEntry, + resolveComposeExecStaleEnvWarning, + updateProjectRuntimeServiceEnvFingerprints, upsertProjectRuntimeStateEntry, } from "../lib/project-runtime-state.ts"; import { @@ -5259,11 +5262,22 @@ async function runUpCommand({ envByService: envOverrides.preflightEnvByService, }); - if (!serviceScoped) { + if (serviceScoped) { + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir: project.projectDir, + composeProject: lifecycleComposeProject, + effectiveEnvName: envOverrides.effectiveEnvName, + services: targetServices, + envByService: envOverrides.preflightEnvByService, + outcome: "applied", + }); + } else { await persistProjectRuntimeEnvSelection({ projectDir: project.projectDir, composeProject: lifecycleComposeProject, envName: envOverrides.effectiveEnvName, + services: preparedServices, + envByService: envOverrides.preflightEnvByService, }); } @@ -6162,6 +6176,8 @@ async function runRestartUpPhase(opts: { projectDir: opts.project.projectDir, composeProject: opts.lifecycleComposeProject, envName: envOverrides.effectiveEnvName, + services: targetServices, + envByService: envOverrides.preflightEnvByService, }); let lifecycleCleanup: LifecycleOperationCleanup | null = null; @@ -6481,6 +6497,19 @@ async function runTargetedServiceRestart(opts: { env: envOverrides.env, routeStdoutToStderr: opts.routeStdoutToStderr, }); + // Force-recreate applies the env resolved above; after a failure the + // services' delivered env is unknown, so their digests are dropped. + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir: opts.project.projectDir, + composeProject: resolveLifecycleComposeProjectName({ + projectName: opts.projectName, + branch: opts.branch, + }), + effectiveEnvName: envOverrides.effectiveEnvName, + services: opts.services, + envByService: envOverrides.preflightEnvByService, + outcome: code === 0 ? "applied" : "unknown", + }); if (code !== 0) { const failure = composeStartupFailure( code, @@ -6955,10 +6984,19 @@ export function resolveRestartTarget(opts: { }; } +/** + * Record the env selection of a full-stack up/restart together with fresh, + * value-free digests of the env set delivered to each prepared service, so + * `hack exec` can warn when the resolved env later changes. + */ async function persistProjectRuntimeEnvSelection(opts: { readonly projectDir: string; readonly composeProject: string; readonly envName: string | null; + readonly services: readonly string[]; + readonly envByService: Readonly< + Record>> + >; }): Promise { await upsertProjectRuntimeStateEntry({ projectDir: opts.projectDir, @@ -6966,6 +7004,11 @@ async function persistProjectRuntimeEnvSelection(opts: { composeProject: opts.composeProject, envName: opts.envName, updatedAt: new Date().toISOString(), + envFingerprint: withComposeServiceEnvFingerprints({ + existing: undefined, + services: opts.services, + envByService: opts.envByService, + }), }, }); } @@ -7551,6 +7594,16 @@ async function handleExec({ if (!execReady.ok) { throw new CliUsageError(execReady.message); } + // exec stdout is the target command's output; keep the warning on stderr. + const staleEnvWarning = await resolveComposeExecStaleEnvWarning({ + projectDir: project.projectDir, + composeProject: lifecycleComposeProject, + service, + currentEnv: envOverrides.preflightEnvByService[service] ?? {}, + }); + if (staleEnvWarning) { + process.stderr.write(`WARNING: ${staleEnvWarning}\n`); + } return await composeRuntimeBackend.exec({ composeFiles: composeFilesWithEnv, diff --git a/src/lib/compose-env-fingerprint.ts b/src/lib/compose-env-fingerprint.ts new file mode 100644 index 00000000..381d7664 --- /dev/null +++ b/src/lib/compose-env-fingerprint.ts @@ -0,0 +1,213 @@ +import { createHmac, randomBytes } from "node:crypto"; + +import { isRecord } from "./guards.ts"; + +/** + * Value-free fingerprints of the env set Hack delivered to each compose + * service when it last (re)created that service. + * + * `hack up` and `hack restart` record one keyed digest per service in the + * project runtime state (`.hack/.internal/runtime-state.json`). `hack exec` + * recomputes the digest for the env set that the same resolver produces now + * and, when it differs, prints a closed warning: the running container still + * has the env it started with until the service is recreated. + * + * Invariants: + * - Digests are HMAC-SHA256 over a canonical encoding of the resolved + * key/value pairs, keyed by a random per-record salt. The record never holds + * a key name or value, and the salt prevents precomputed or cross-project + * comparison of low-entropy values. + * - Comparison is advisory. A missing, malformed, or foreign record yields + * `unknown` and no warning; it never refuses exec. + * - Everything here is pure except salt generation; callers own all I/O. + */ + +/** Current record encoding; any other version is treated as absent. */ +export const COMPOSE_ENV_FINGERPRINT_VERSION = 1 as const; + +const FINGERPRINT_DOMAIN = "hack.compose-service-env.v1"; +const SALT_PATTERN = /^[0-9a-f]{32}$/; +const DIGEST_PATTERN = /^[0-9a-f]{64}$/; +/** + * Compose service-name grammar. Names outside it are never recorded or echoed. + */ +const SERVICE_NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$/; + +export type ComposeEnvFingerprintRecord = { + readonly version: typeof COMPOSE_ENV_FINGERPRINT_VERSION; + /** 16 random bytes, lowercase hex. */ + readonly salt: string; + /** Compose service name to lowercase-hex HMAC-SHA256 digest. */ + readonly services: Readonly>; +}; + +export type ComposeServiceEnvFreshness = "unchanged" | "changed" | "unknown"; + +/** Fresh random salt for a new fingerprint record. */ +export function createComposeEnvFingerprintSalt(): string { + return randomBytes(16).toString("hex"); +} + +/** + * Keyed digest of one resolved env set. Key order does not matter; any key, + * value, addition, or removal changes the digest. + */ +export function computeComposeEnvFingerprint(opts: { + readonly salt: string; + readonly env: Readonly>; +}): string { + if (!SALT_PATTERN.test(opts.salt)) { + throw new Error("Invalid compose env fingerprint salt."); + } + const entries = Object.entries(opts.env).sort(([left], [right]) => + compareCodeUnits(left, right) + ); + return createHmac("sha256", Buffer.from(opts.salt, "hex")) + .update(FINGERPRINT_DOMAIN) + .update("\0") + .update(JSON.stringify(entries)) + .digest("hex"); +} + +/** + * Record or refresh the digests of `services` from `envByService`, keeping + * every other service's digest and the existing salt. A service with no + * resolved entry is recorded as an empty env set. + */ +export function withComposeServiceEnvFingerprints(opts: { + readonly existing: ComposeEnvFingerprintRecord | undefined; + readonly services: readonly string[]; + readonly envByService: Readonly< + Record>> + >; + readonly createSalt?: () => string; +}): ComposeEnvFingerprintRecord { + const salt = + opts.existing?.salt ?? + (opts.createSalt ?? createComposeEnvFingerprintSalt)(); + const services: Record = { ...opts.existing?.services }; + for (const service of opts.services) { + if (!SERVICE_NAME_PATTERN.test(service)) { + continue; + } + const env = Object.hasOwn(opts.envByService, service) + ? (opts.envByService[service] ?? {}) + : {}; + services[service] = computeComposeEnvFingerprint({ salt, env }); + } + return { + version: COMPOSE_ENV_FINGERPRINT_VERSION, + salt, + services: sortRecord(services), + }; +} + +/** + * Forget the digests of `services` (their delivered env is no longer known). + * Returns undefined when nothing remains. + */ +export function withoutComposeServiceEnvFingerprints(opts: { + readonly existing: ComposeEnvFingerprintRecord | undefined; + readonly services: readonly string[]; +}): ComposeEnvFingerprintRecord | undefined { + if (!opts.existing) { + return; + } + const forgotten = new Set(opts.services); + const services = Object.fromEntries( + Object.entries(opts.existing.services).filter( + ([service]) => !forgotten.has(service) + ) + ); + if (Object.keys(services).length === 0) { + return; + } + return { ...opts.existing, services: sortRecord(services) }; +} + +/** Compare the recorded digest for `service` with the env resolved now. */ +export function evaluateComposeServiceEnvFreshness(opts: { + readonly record: ComposeEnvFingerprintRecord | undefined; + readonly service: string; + readonly currentEnv: Readonly>; +}): ComposeServiceEnvFreshness { + const record = opts.record; + if (!(record && Object.hasOwn(record.services, opts.service))) { + return "unknown"; + } + const recorded = record.services[opts.service]; + const current = computeComposeEnvFingerprint({ + salt: record.salt, + env: opts.currentEnv, + }); + return recorded === current ? "unchanged" : "changed"; +} + +/** + * Closed, value-free warning for a stale running service. Names the service + * only when it matches the compose service-name grammar. + */ +export function buildStaleComposeEnvWarning(opts: { + readonly service: string; +}): string { + const named = SERVICE_NAME_PATTERN.test(opts.service); + const subject = named ? `service "${opts.service}"` : "this service"; + const restart = named + ? `hack restart ${opts.service}` + : "hack restart "; + return `Env for ${subject} changed since it started; this exec uses the env it started with. Run \`${restart}\` to apply the current env.`; +} + +/** Strict parse of a stored record; anything unexpected is treated as absent. */ +export function parseComposeEnvFingerprintRecord( + value: unknown +): ComposeEnvFingerprintRecord | undefined { + if (!isRecord(value)) { + return; + } + if (value.version !== COMPOSE_ENV_FINGERPRINT_VERSION) { + return; + } + const salt = value.salt; + if (typeof salt !== "string" || !SALT_PATTERN.test(salt)) { + return; + } + const rawServices = value.services; + if (!isRecord(rawServices)) { + return; + } + const services: Record = {}; + for (const [service, digest] of Object.entries(rawServices)) { + if ( + !SERVICE_NAME_PATTERN.test(service) || + typeof digest !== "string" || + !DIGEST_PATTERN.test(digest) + ) { + return; + } + services[service] = digest; + } + return { + version: COMPOSE_ENV_FINGERPRINT_VERSION, + salt, + services: sortRecord(services), + }; +} + +function sortRecord( + input: Readonly> +): Readonly> { + return Object.fromEntries( + Object.entries(input).sort(([left], [right]) => + compareCodeUnits(left, right) + ) + ); +} + +/** Locale-independent ordering so digests are stable across hosts. */ +function compareCodeUnits(left: string, right: string): number { + if (left === right) { + return 0; + } + return left < right ? -1 : 1; +} diff --git a/src/lib/project-runtime-state.ts b/src/lib/project-runtime-state.ts index f999864d..798caecb 100644 --- a/src/lib/project-runtime-state.ts +++ b/src/lib/project-runtime-state.ts @@ -1,4 +1,12 @@ import { resolve } from "node:path"; +import { + buildStaleComposeEnvWarning, + type ComposeEnvFingerprintRecord, + evaluateComposeServiceEnvFreshness, + parseComposeEnvFingerprintRecord, + withComposeServiceEnvFingerprints, + withoutComposeServiceEnvFingerprints, +} from "./compose-env-fingerprint.ts"; import { ensureDir, pathExists, readTextFile, writeTextFile } from "./fs.ts"; import { getString, isRecord } from "./guards.ts"; @@ -6,6 +14,12 @@ export type ProjectRuntimeStateEntry = { readonly composeProject: string; readonly envName: string | null; readonly updatedAt: string; + /** + * Value-free digests of the env set each service was last created with. + * Absent for state written before fingerprints existed; exec then skips the + * staleness check. + */ + readonly envFingerprint?: ComposeEnvFingerprintRecord; }; type ProjectRuntimeStateFile = { @@ -107,6 +121,76 @@ export async function removeProjectRuntimeStateEntry(opts: { }); } +/** + * Refresh or forget the env digests of `services` after a service-scoped + * (re)create, preserving the rest of the stack's entry. + * + * - No entry: nothing is written. Service-scoped commands never create the + * stack entry (its env selection belongs to a full `up`/`restart`). + * - `applied` with the entry's env selection: digests are recorded. + * - Otherwise (`unknown` outcome, or a different env selection): the digests + * are dropped so exec reports nothing rather than a wrong comparison. + */ +export async function updateProjectRuntimeServiceEnvFingerprints(opts: { + readonly projectDir: string; + readonly composeProject: string; + readonly effectiveEnvName: string | null; + readonly services: readonly string[]; + readonly envByService: Readonly< + Record>> + >; + readonly outcome: "applied" | "unknown"; +}): Promise { + const entry = await readProjectRuntimeStateEntry({ + projectDir: opts.projectDir, + composeProject: opts.composeProject, + }); + if (!entry) { + return; + } + const envFingerprint = + opts.outcome === "applied" && entry.envName === opts.effectiveEnvName + ? withComposeServiceEnvFingerprints({ + existing: entry.envFingerprint, + services: opts.services, + envByService: opts.envByService, + }) + : withoutComposeServiceEnvFingerprints({ + existing: entry.envFingerprint, + services: opts.services, + }); + const { envFingerprint: _previous, ...rest } = entry; + await upsertProjectRuntimeStateEntry({ + projectDir: opts.projectDir, + entry: { ...rest, ...(envFingerprint ? { envFingerprint } : {}) }, + }); +} + +/** + * Closed warning for `hack exec` when the env resolved now differs from the + * env `service` was created with; null when unchanged or unknown. Never + * includes a key or value. + */ +export async function resolveComposeExecStaleEnvWarning(opts: { + readonly projectDir: string; + readonly composeProject: string; + readonly service: string; + readonly currentEnv: Readonly>; +}): Promise { + const entry = await readProjectRuntimeStateEntry({ + projectDir: opts.projectDir, + composeProject: opts.composeProject, + }); + const freshness = evaluateComposeServiceEnvFreshness({ + record: entry?.envFingerprint, + service: opts.service, + currentEnv: opts.currentEnv, + }); + return freshness === "changed" + ? buildStaleComposeEnvWarning({ service: opts.service }) + : null; +} + function parseProjectRuntimeStateEntry( value: unknown ): ProjectRuntimeStateEntry | null { @@ -124,10 +208,13 @@ function parseProjectRuntimeStateEntry( const envName = typeof envNameRaw === "string" ? envNameRaw.trim() || null : null; + const envFingerprint = parseComposeEnvFingerprintRecord(value.envFingerprint); + return { composeProject, envName, updatedAt, + ...(envFingerprint ? { envFingerprint } : {}), }; } diff --git a/tests/compose-env-staleness.test.ts b/tests/compose-env-staleness.test.ts new file mode 100644 index 00000000..7cbc3fdc --- /dev/null +++ b/tests/compose-env-staleness.test.ts @@ -0,0 +1,438 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { + mkdir, + mkdtemp, + readFile, + realpath, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { resolveRequestedEnvName } from "../src/cli/env-selection.ts"; +import { resolveModernComposeEnvOverrides } from "../src/commands/project.ts"; +import { + buildStaleComposeEnvWarning, + COMPOSE_ENV_FINGERPRINT_VERSION, + computeComposeEnvFingerprint, + evaluateComposeServiceEnvFreshness, + parseComposeEnvFingerprintRecord, + withComposeServiceEnvFingerprints, + withoutComposeServiceEnvFingerprints, +} from "../src/lib/compose-env-fingerprint.ts"; +import { findProjectContextAtRoot } from "../src/lib/project.ts"; +import { + readProjectRuntimeStateEntry, + resolveComposeExecStaleEnvWarning, + resolveProjectRuntimeStatePath, + updateProjectRuntimeServiceEnvFingerprints, + upsertProjectRuntimeStateEntry, +} from "../src/lib/project-runtime-state.ts"; +import { restoreEnv } from "./helpers/env.ts"; + +/** + * HACK-1140: `hack exec` warns, without printing any key or value, when the + * env resolved now differs from the env a running compose service was created + * with. All state lives in a temp project; no Docker, HOME, or client config + * is touched. + */ + +const SALT = "0123456789abcdef0123456789abcdef"; +const OTHER_SALT = "fedcba9876543210fedcba9876543210"; +const SECRET_ONE = "synthetic-stale-env-value-one"; +const SECRET_TWO = "synthetic-stale-env-value-two"; +const KEY = "SYNTHETIC_STALE_KEY"; +const COMPOSE_PROJECT = "stale-fixture"; +const ENV_NAMES = [ + "CI", + "HACK_ENV_SECRET_KEY", + "HACK_EXECUTION_MODE", + "HACK_GLOBAL_CONFIG_PATH", + "HACK_HOME", + "HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK", + "HOME", +] as const; + +const savedEnv = new Map(); +let root = ""; +let projectDir = ""; + +beforeEach(async () => { + root = await realpath(await mkdtemp(join(tmpdir(), "hack-stale-env-"))); + projectDir = join(root, "project", ".hack"); + for (const name of ENV_NAMES) { + savedEnv.set(name, process.env[name]); + Reflect.deleteProperty(process.env, name); + } + process.env.HOME = join(root, "home"); + process.env.HACK_HOME = join(root, "hack-home"); + process.env.HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK = "1"; + process.env.HACK_GLOBAL_CONFIG_PATH = join(root, "global.json"); + await writeFile(process.env.HACK_GLOBAL_CONFIG_PATH, "{}"); + await mkdir(projectDir, { recursive: true }); +}); + +afterEach(async () => { + for (const [name, value] of savedEnv) { + restoreEnv(name, value); + } + savedEnv.clear(); + await rm(root, { recursive: true, force: true }); +}); + +describe("compose env fingerprint", () => { + test("is order-independent and changes with any key, value, or salt", () => { + const base = computeComposeEnvFingerprint({ + salt: SALT, + env: { A: "1", B: "2" }, + }); + expect(base).toMatch(/^[0-9a-f]{64}$/); + expect( + computeComposeEnvFingerprint({ salt: SALT, env: { B: "2", A: "1" } }) + ).toBe(base); + const variants: readonly Readonly>[] = [ + { A: "1", B: "3" }, + { A: "1" }, + { A: "1", B: "2", C: "" }, + { A: "1", C: "2" }, + ]; + for (const env of variants) { + expect(computeComposeEnvFingerprint({ salt: SALT, env })).not.toBe(base); + } + expect( + computeComposeEnvFingerprint({ + salt: OTHER_SALT, + env: { A: "1", B: "2" }, + }) + ).not.toBe(base); + expect(() => + computeComposeEnvFingerprint({ salt: "not-hex", env: {} }) + ).toThrow("Invalid compose env fingerprint salt."); + }); + + test("records hold no key or value and round-trip through the strict parser", () => { + const record = withComposeServiceEnvFingerprints({ + existing: undefined, + services: ["web", "api", "bad name"], + envByService: { web: { [KEY]: SECRET_ONE } }, + createSalt: () => SALT, + }); + expect(record.version).toBe(COMPOSE_ENV_FINGERPRINT_VERSION); + expect(Object.keys(record.services)).toEqual(["api", "web"]); + const serialized = JSON.stringify(record); + expect(serialized).not.toContain(KEY); + expect(serialized).not.toContain(SECRET_ONE); + expect(parseComposeEnvFingerprintRecord(JSON.parse(serialized))).toEqual( + record + ); + for (const malformed of [ + null, + { ...record, version: 2 }, + { ...record, salt: "short" }, + { ...record, services: { web: "not-a-digest" } }, + { ...record, services: { "bad name": "0".repeat(64) } }, + ]) { + expect(parseComposeEnvFingerprintRecord(malformed)).toBeUndefined(); + } + }); + + test("freshness: identical env is unchanged, a changed value is changed, absence is unknown", () => { + const record = withComposeServiceEnvFingerprints({ + existing: undefined, + services: ["web"], + envByService: { web: { [KEY]: SECRET_ONE } }, + createSalt: () => SALT, + }); + expect( + evaluateComposeServiceEnvFreshness({ + record, + service: "web", + currentEnv: { [KEY]: SECRET_ONE }, + }) + ).toBe("unchanged"); + expect( + evaluateComposeServiceEnvFreshness({ + record, + service: "web", + currentEnv: { [KEY]: SECRET_TWO }, + }) + ).toBe("changed"); + expect( + evaluateComposeServiceEnvFreshness({ + record, + service: "api", + currentEnv: {}, + }) + ).toBe("unknown"); + expect( + evaluateComposeServiceEnvFreshness({ + record: undefined, + service: "web", + currentEnv: {}, + }) + ).toBe("unknown"); + expect( + evaluateComposeServiceEnvFreshness({ + record: withoutComposeServiceEnvFingerprints({ + existing: record, + services: ["web"], + }), + service: "web", + currentEnv: { [KEY]: SECRET_TWO }, + }) + ).toBe("unknown"); + }); + + test("the warning is closed: it names a valid service and never echoes other input", () => { + expect(buildStaleComposeEnvWarning({ service: "web" })).toBe( + 'Env for service "web" changed since it started; this exec uses the env it started with. Run `hack restart web` to apply the current env.' + ); + const hostile = buildStaleComposeEnvWarning({ + service: `web; echo ${SECRET_ONE}`, + }); + expect(hostile).not.toContain(SECRET_ONE); + expect(hostile).toContain("this service"); + expect(hostile).toContain("`hack restart `"); + }); +}); + +describe("runtime state staleness", () => { + async function recordUp(env: Readonly>) { + await upsertProjectRuntimeStateEntry({ + projectDir, + entry: { + composeProject: COMPOSE_PROJECT, + envName: null, + updatedAt: "2026-10-12T00:00:00.000Z", + envFingerprint: withComposeServiceEnvFingerprints({ + existing: undefined, + services: ["web", "api"], + envByService: { web: env, api: env }, + }), + }, + }); + } + + function warningFor(service: string, env: Readonly>) { + return resolveComposeExecStaleEnvWarning({ + projectDir, + composeProject: COMPOSE_PROJECT, + service, + currentEnv: env, + }); + } + + test("negative control: identical env produces no warning", async () => { + await recordUp({ [KEY]: SECRET_ONE }); + expect(await warningFor("web", { [KEY]: SECRET_ONE })).toBeNull(); + const stored = await readFile( + resolveProjectRuntimeStatePath({ projectDir }), + "utf8" + ); + expect(stored).not.toContain(KEY); + expect(stored).not.toContain(SECRET_ONE); + }); + + test("changed env warns with the service name and no value", async () => { + await recordUp({ [KEY]: SECRET_ONE }); + const warning = await warningFor("web", { [KEY]: SECRET_TWO }); + expect(warning).toBe(buildStaleComposeEnvWarning({ service: "web" })); + expect(warning).toContain("`hack restart web`"); + for (const text of [KEY, SECRET_ONE, SECRET_TWO]) { + expect(warning).not.toContain(text); + } + }); + + test("state without fingerprints (pre-upgrade) or a missing entry never warns", async () => { + expect(await warningFor("web", { [KEY]: SECRET_TWO })).toBeNull(); + await upsertProjectRuntimeStateEntry({ + projectDir, + entry: { + composeProject: COMPOSE_PROJECT, + envName: null, + updatedAt: "2026-10-12T00:00:00.000Z", + }, + }); + expect(await warningFor("web", { [KEY]: SECRET_TWO })).toBeNull(); + const path = resolveProjectRuntimeStatePath({ projectDir }); + const tampered = JSON.parse(await readFile(path, "utf8")) as { + entries: Record[]; + }; + tampered.entries[0] = { + ...tampered.entries[0], + envFingerprint: { version: 1, salt: SALT, services: { web: "x" } }, + }; + await writeFile(path, JSON.stringify(tampered)); + expect(await warningFor("web", { [KEY]: SECRET_TWO })).toBeNull(); + expect( + ( + await readProjectRuntimeStateEntry({ + projectDir, + composeProject: COMPOSE_PROJECT, + }) + )?.envName + ).toBeNull(); + }); + + test("a scoped restart refreshes only its services; other services stay stale", async () => { + await recordUp({ [KEY]: SECRET_ONE }); + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir, + composeProject: COMPOSE_PROJECT, + effectiveEnvName: null, + services: ["web"], + envByService: { web: { [KEY]: SECRET_TWO } }, + outcome: "applied", + }); + expect(await warningFor("web", { [KEY]: SECRET_TWO })).toBeNull(); + expect(await warningFor("api", { [KEY]: SECRET_TWO })).toBe( + buildStaleComposeEnvWarning({ service: "api" }) + ); + }); + + test("an unknown outcome or a different env selection forgets the service instead of guessing", async () => { + await recordUp({ [KEY]: SECRET_ONE }); + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir, + composeProject: COMPOSE_PROJECT, + effectiveEnvName: null, + services: ["web"], + envByService: { web: { [KEY]: SECRET_TWO } }, + outcome: "unknown", + }); + expect(await warningFor("web", { [KEY]: SECRET_TWO })).toBeNull(); + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir, + composeProject: COMPOSE_PROJECT, + effectiveEnvName: "qa", + services: ["api"], + envByService: { api: { [KEY]: SECRET_TWO } }, + outcome: "applied", + }); + expect(await warningFor("api", { [KEY]: SECRET_TWO })).toBeNull(); + const entry = await readProjectRuntimeStateEntry({ + projectDir, + composeProject: COMPOSE_PROJECT, + }); + expect(entry?.envName).toBeNull(); + expect(entry?.envFingerprint).toBeUndefined(); + }); + + test("a scoped update without a stack entry writes nothing", async () => { + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir, + composeProject: COMPOSE_PROJECT, + effectiveEnvName: null, + services: ["web"], + envByService: { web: { [KEY]: SECRET_ONE } }, + outcome: "applied", + }); + expect( + await Bun.file(resolveProjectRuntimeStatePath({ projectDir })).exists() + ).toBe(false); + }); +}); + +describe("resolver-backed refresh (W13 selection)", () => { + async function writeLayer(opts: { + readonly projectRoot: string; + readonly name: string; + readonly value: string; + }) { + await writeFile( + join(opts.projectRoot, `.hack/hack.env.${opts.name}.yaml`), + JSON.stringify({ + version: 1, + environment: opts.name, + secretsprovider: "project_key", + values: { global: { [KEY]: opts.value } }, + }) + ); + } + + test("an env edit after up is detected for the default overlay and cleared by a refresh", async () => { + const projectRoot = join(root, "project"); + await writeFile( + join(projectDir, "hack.config.json"), + JSON.stringify({ name: COMPOSE_PROJECT, env: { defaultOverlay: "qa" } }) + ); + await writeFile( + join(projectDir, "docker-compose.yml"), + "services:\n web:\n image: alpine:3.20\n" + ); + await writeLayer({ projectRoot, name: "default", value: "base-value" }); + await writeLayer({ projectRoot, name: "qa", value: SECRET_ONE }); + const project = await findProjectContextAtRoot({ + projectRoot, + projectDirName: ".hack", + }); + if (!project) { + throw new Error("Expected a compose project context"); + } + const resolveNow = async (envOption: string | undefined) => { + const resolved = await resolveModernComposeEnvOverrides({ + project, + targetServices: ["web"], + allServiceNames: ["web"], + envName: resolveRequestedEnvName({ envOption }), + }); + if (!resolved) { + throw new Error("Expected modern env config"); + } + return resolved; + }; + + const atUp = await resolveNow(undefined); + expect(atUp.effectiveEnvName).toBe("qa"); + await upsertProjectRuntimeStateEntry({ + projectDir, + entry: { + composeProject: COMPOSE_PROJECT, + envName: atUp.effectiveEnvName, + updatedAt: "2026-10-12T00:00:00.000Z", + envFingerprint: withComposeServiceEnvFingerprints({ + existing: undefined, + services: ["web"], + envByService: atUp.preflightEnvByService, + }), + }, + }); + const execWarning = async (envOption: string | undefined) => { + const now = await resolveNow(envOption); + return await resolveComposeExecStaleEnvWarning({ + projectDir, + composeProject: COMPOSE_PROJECT, + service: "web", + currentEnv: now.preflightEnvByService.web ?? {}, + }); + }; + + expect(await execWarning(undefined)).toBeNull(); + expect(await execWarning("qa")).toBeNull(); + + await writeLayer({ projectRoot, name: "qa", value: SECRET_TWO }); + const warning = await execWarning(undefined); + expect(warning).toBe(buildStaleComposeEnvWarning({ service: "web" })); + for (const text of [KEY, SECRET_ONE, SECRET_TWO, "base-value"]) { + expect(warning).not.toContain(text); + } + + const refreshed = await resolveNow(undefined); + await updateProjectRuntimeServiceEnvFingerprints({ + projectDir, + composeProject: COMPOSE_PROJECT, + effectiveEnvName: refreshed.effectiveEnvName, + services: ["web"], + envByService: refreshed.preflightEnvByService, + outcome: "applied", + }); + expect(await execWarning(undefined)).toBeNull(); + const stored = await readFile( + resolveProjectRuntimeStatePath({ projectDir }), + "utf8" + ); + for (const text of [KEY, SECRET_ONE, SECRET_TWO]) { + expect(stored).not.toContain(text); + } + }); +}); diff --git a/tests/e2e/run.ts b/tests/e2e/run.ts index e7151477..f2abd8e3 100644 --- a/tests/e2e/run.ts +++ b/tests/e2e/run.ts @@ -17,6 +17,7 @@ import { doctorScenario } from "./scenarios/doctor.ts"; import { domainMigrationScenario } from "./scenarios/domain-migration.ts"; import { domainMigrationFilesScenario } from "./scenarios/domain-migration-files.ts"; import { envSecretsScenario } from "./scenarios/env-secrets.ts"; +import { execStaleEnvScenario } from "./scenarios/exec-stale-env.ts"; import { initScenario } from "./scenarios/init.ts"; import { lifecycleHostProcessScenario } from "./scenarios/lifecycle-host-process.ts"; import { lifecycleSessionRecoveryScenario } from "./scenarios/lifecycle-session-recovery.ts"; @@ -93,6 +94,7 @@ export const ALL_SCENARIOS: readonly Scenario[] = [ dependencyCachePackageScenario, domainMigrationScenario, upDownScenario, + execStaleEnvScenario, portableMultiserviceScenario, nativeConfigComposeScenario, nativeConfigBuildScenario, diff --git a/tests/e2e/scenarios/exec-stale-env.ts b/tests/e2e/scenarios/exec-stale-env.ts new file mode 100644 index 00000000..6eba98cb --- /dev/null +++ b/tests/e2e/scenarios/exec-stale-env.ts @@ -0,0 +1,183 @@ +import { createMonorepoFixture } from "../fixture.ts"; +import { + type CliResult, + expect, + expectExit, + type Scenario, + type ScenarioContext, +} from "../harness.ts"; +import { downBestEffort, requireDockerPreconditions } from "./docker-shared.ts"; + +const UP_TIMEOUT_MS = 420_000; +const EXEC_TIMEOUT_MS = 120_000; +const PROBE_KEY = "E2E_STALE_PROBE"; +const VALUE_AT_UP = "synthetic-stale-probe-at-up"; +const VALUE_AFTER_EDIT = "synthetic-stale-probe-after-edit"; +const STALE_MARKER = "changed since it started"; + +/** + * HACK-1140: synthetic env delivery and refresh against real compose + * containers. `hack up` delivers a global env value to web and api; editing + * the value afterwards makes `hack exec` print a closed stderr warning that + * names the service and suggests `hack restart ` without printing any + * value, while the container keeps its start-time env. `hack restart web` + * recreates web with the new value and clears its warning; api, not + * restarted, still warns. Identical env never warns (negative control). + */ +export const execStaleEnvScenario: Scenario = { + name: "exec-stale-env", + tier: "docker", + summary: "hack exec warns value-free when delivered env changed since up", + run: async (ctx) => { + await requireDockerPreconditions({ ctx }); + + const fixture = await createMonorepoFixture({ + parentDir: ctx.tempRoot, + withHackConfig: true, + lifecycle: { standaloneContainers: true }, + }); + + const setProbe = async (value: string): Promise => { + const result = await ctx.cli({ + args: ["env", "add", PROBE_KEY, value], + cwd: fixture.root, + }); + expectExit({ + result, + codes: [0], + message: "hack env add should update the synthetic probe value", + }); + }; + + try { + await setProbe(VALUE_AT_UP); + const up = await ctx.cli({ + args: ["up", "--detach"], + cwd: fixture.root, + timeoutMs: UP_TIMEOUT_MS, + }); + expectExit({ + result: up, + codes: [0], + message: "hack up --detach should succeed", + }); + + const fresh = await execProbe({ + ctx, + root: fixture.root, + service: "web", + }); + expect({ + that: fresh.stdout.trim() === VALUE_AT_UP, + message: "web should receive the env value resolved at up", + result: fresh, + }); + expect({ + that: !fresh.stderr.includes(STALE_MARKER), + message: "negative control: identical env must not warn", + result: fresh, + }); + + await setProbe(VALUE_AFTER_EDIT); + for (const service of ["web", "api"]) { + const stale = await execProbe({ ctx, root: fixture.root, service }); + expect({ + that: stale.stdout.trim() === VALUE_AT_UP, + message: `${service} keeps its start-time env until recreated`, + result: stale, + }); + expectValueFreeWarning({ result: stale, service }); + } + + const restart = await ctx.cli({ + args: ["restart", "web"], + cwd: fixture.root, + timeoutMs: UP_TIMEOUT_MS, + }); + expectExit({ + result: restart, + codes: [0], + message: "hack restart web should recreate web", + }); + + const refreshed = await execProbe({ + ctx, + root: fixture.root, + service: "web", + }); + expect({ + that: refreshed.stdout.trim() === VALUE_AFTER_EDIT, + message: "restarted web should receive the edited env value", + result: refreshed, + }); + expect({ + that: !refreshed.stderr.includes(STALE_MARKER), + message: "restarted web must not warn", + result: refreshed, + }); + + const stillStale = await execProbe({ + ctx, + root: fixture.root, + service: "api", + }); + expectValueFreeWarning({ result: stillStale, service: "api" }); + + const down = await ctx.cli({ + args: ["down"], + cwd: fixture.root, + timeoutMs: UP_TIMEOUT_MS, + }); + expectExit({ + result: down, + codes: [0], + message: "hack down should succeed", + }); + } finally { + await downBestEffort({ ctx, fixture }); + } + }, +}; + +async function execProbe(opts: { + readonly ctx: ScenarioContext; + readonly root: string; + readonly service: string; +}): Promise { + const result = await opts.ctx.cli({ + args: ["exec", opts.service, "--", "printenv", PROBE_KEY], + cwd: opts.root, + timeoutMs: EXEC_TIMEOUT_MS, + }); + expectExit({ + result, + codes: [0], + message: `hack exec ${opts.service} printenv should succeed`, + }); + return result; +} + +function expectValueFreeWarning(opts: { + readonly result: CliResult; + readonly service: string; +}): void { + const { stderr } = opts.result; + const warning = + stderr.split("\n").find((line) => line.includes(STALE_MARKER)) ?? ""; + expect({ + that: + warning.includes(`service "${opts.service}" ${STALE_MARKER}`) && + warning.includes(`hack restart ${opts.service}`), + message: `${opts.service} exec should warn that its env changed and suggest a restart`, + result: opts.result, + }); + expect({ + that: !( + warning.includes(PROBE_KEY) || + stderr.includes(VALUE_AT_UP) || + stderr.includes(VALUE_AFTER_EDIT) + ), + message: "stale env warning must not print any key or value", + result: opts.result, + }); +} diff --git a/tests/e2e/shard.ts b/tests/e2e/shard.ts index 5e4e9855..45985cc4 100644 --- a/tests/e2e/shard.ts +++ b/tests/e2e/shard.ts @@ -59,6 +59,8 @@ export const E2E_DURATION_HINTS_SECONDS: Readonly> = { doctor: 18, "lifecycle-host-process": 13, "up-down": 12, + // Estimate until a hosted docker-e2e summary measures it. + "exec-stale-env": 20, "cache-prune": 12, "dependency-cache-protocol": 11, "worktree-branch-default": 2, diff --git a/tests/project-exec-command.test.ts b/tests/project-exec-command.test.ts index 14ea8e85..b106dae6 100644 --- a/tests/project-exec-command.test.ts +++ b/tests/project-exec-command.test.ts @@ -8,6 +8,9 @@ import { PROJECT_COMPOSE_FILENAME, PROJECT_CONFIG_FILENAME, } from "../src/constants.ts"; +import { withComposeServiceEnvFingerprints } from "../src/lib/compose-env-fingerprint.ts"; +import { findProjectContextAtRoot } from "../src/lib/project.ts"; +import { upsertProjectRuntimeStateEntry } from "../src/lib/project-runtime-state.ts"; import { registerScopedModuleMock } from "./helpers/scoped-module-mock.ts"; const execCalls: Array<{ @@ -53,7 +56,9 @@ const runtimeBackendMock = await registerScopedModuleMock({ }, }); -const { execCommand } = await import("../src/commands/project.ts"); +const { execCommand, resolveModernComposeEnvOverrides } = await import( + "../src/commands/project.ts" +); beforeAll(() => { runtimeBackendMock.activate(); @@ -256,6 +261,122 @@ test("exec rejects services that are not running", async () => { expect(execCalls).toHaveLength(0); }); +test("exec warns value-free on stderr only after the service env changed since start", async () => { + const projectRoot = await createProject({ + runningServices: ["api"], + runtimeEnvName: "qa", + }); + const project = await findProjectContextAtRoot({ + projectRoot, + projectDirName: ".hack", + }); + if (!project) { + throw new Error("Expected exec fixture project context"); + } + const atUp = await resolveModernComposeEnvOverrides({ + project, + targetServices: ["api"], + allServiceNames: ["api", "worker"], + envName: "qa", + }); + if (!atUp) { + throw new Error("Expected modern env config"); + } + await upsertProjectRuntimeStateEntry({ + projectDir: project.projectDir, + entry: { + composeProject: "project-exec-test", + envName: "qa", + updatedAt: "2026-04-01T00:00:00.000Z", + envFingerprint: withComposeServiceEnvFingerprints({ + existing: undefined, + services: ["api"], + envByService: atUp.preflightEnvByService, + }), + }, + }); + + const unchanged = await captureOutput(() => + execCommand.handler(execInput({ projectRoot, env: "qa" })) + ); + expect(unchanged.result).toBe(0); + expect(unchanged.stderr).not.toContain("changed since it started"); + expect(unchanged.stdout).toBe(""); + + const qaPath = resolve(project.projectDir, "hack.env.qa.yaml"); + const qaText = await readFile(qaPath, "utf8"); + await writeFile( + qaPath, + qaText.replace("overlay-secret", "rotated-overlay-secret") + ); + const stale = await captureOutput(() => + execCommand.handler(execInput({ projectRoot, env: "qa" })) + ); + expect(stale.result).toBe(0); + expect(stale.stdout).toBe(""); + // Other stderr diagnostics (logger backend dependent) may precede it. + expect( + stale.stderr.split("\n").filter((line) => line.startsWith("WARNING: ")) + ).toEqual([ + 'WARNING: Env for service "api" changed since it started; this exec uses the env it started with. Run `hack restart api` to apply the current env.', + ]); + for (const text of ["SERVICE_TOKEN", "overlay-secret", "rotated"]) { + expect(stale.stderr).not.toContain(text); + } + expect(execCalls).toHaveLength(2); +}); + +function execInput(opts: { + readonly projectRoot: string; + readonly env: string | undefined; +}): Parameters[0] { + const envArgs = opts.env ? ["--env", opts.env] : []; + return { + ctx: { cwd: opts.projectRoot, cli: CLI_SPEC }, + args: { + options: { + path: opts.projectRoot, + project: undefined, + env: opts.env, + branch: undefined, + workdir: undefined, + profile: undefined, + }, + positionals: { service: "api", cmd: ["true"] }, + raw: { + argv: ["--path", opts.projectRoot, ...envArgs, "api", "true"], + positionals: ["api", "true"], + }, + }, + } as unknown as Parameters[0]; +} + +async function captureOutput(run: () => Promise): Promise<{ + readonly result: T; + readonly stdout: string; + readonly stderr: string; +}> { + let stdout = ""; + let stderr = ""; + const originalOut = process.stdout.write; + const originalErr = process.stderr.write; + process.stdout.write = ((chunk: string | Uint8Array) => { + stdout += chunk.toString(); + return true; + }) as typeof process.stdout.write; + process.stderr.write = ((chunk: string | Uint8Array) => { + stderr += chunk.toString(); + return true; + }) as typeof process.stderr.write; + try { + const result = await run(); + return { result, stdout, stderr }; + } finally { + process.stdout.write = originalOut; + process.stderr.write = originalErr; + } +} + async function createProject(input?: { readonly composeName?: string; readonly runningServices?: readonly string[]; diff --git a/tests/project-up-startup-state.test.ts b/tests/project-up-startup-state.test.ts index 4a0ad93c..5f82989e 100644 --- a/tests/project-up-startup-state.test.ts +++ b/tests/project-up-startup-state.test.ts @@ -19,6 +19,8 @@ import { PROJECT_COMPOSE_FILENAME, PROJECT_CONFIG_FILENAME, } from "../src/constants.ts"; +import { evaluateComposeServiceEnvFreshness } from "../src/lib/compose-env-fingerprint.ts"; +import { readProjectRuntimeStateEntry } from "../src/lib/project-runtime-state.ts"; import { restoreEnv } from "./helpers/env.ts"; import { registerScopedModuleMock } from "./helpers/scoped-module-mock.ts"; @@ -384,6 +386,62 @@ test("targeted restart preserves delivered Compose environment", async () => { ]); }); +test("up records value-free env digests and a targeted restart refreshes only its service", async () => { + const projectRoot = await createProject(); + const projectDir = resolve(projectRoot, ".hack"); + psRows.push( + JSON.stringify({ Service: "api", State: "running", ExitCode: 0 }), + JSON.stringify({ Service: "migrate", State: "exited", ExitCode: 0 }) + ); + const readEntry = () => + readProjectRuntimeStateEntry({ + projectDir, + composeProject: "startup-state-test", + }); + + expect(await runDetachedUp({ projectRoot })).toBe(0); + const atUp = (await readEntry())?.envFingerprint; + expect(Object.keys(atUp?.services ?? {})).toEqual([ + "api", + "migrate", + "profiled-worker", + ]); + const freshness = ( + record: typeof atUp, + service: string, + mode: string + ): string => + evaluateComposeServiceEnvFreshness({ + record, + service, + currentEnv: { SHARED_MODE: mode }, + }); + expect(freshness(atUp, "api", "compose")).toBe("unchanged"); + expect(freshness(atUp, "api", "rotated")).toBe("changed"); + + const envPath = resolve(projectDir, "hack.env.default.yaml"); + await writeFile( + envPath, + (await readFile(envPath, "utf8")).replace( + ' SHARED_MODE: "compose"', + ' SHARED_MODE: "rotated"' + ) + ); + expect(await runRestart({ projectRoot, services: ["api"] })).toBe(0); + const afterRestart = (await readEntry())?.envFingerprint; + expect(afterRestart?.salt).toBe(atUp?.salt); + expect(freshness(afterRestart, "api", "rotated")).toBe("unchanged"); + expect(freshness(afterRestart, "migrate", "rotated")).toBe("changed"); + expect(afterRestart?.services.migrate).toBe(atUp?.services.migrate); + const stored = await readFile( + resolve(projectDir, ".internal/runtime-state.json"), + "utf8" + ); + for (const text of ["SHARED_MODE", "rotated", '"compose"']) { + expect(stored).not.toContain(text); + } +}); + test("up --json emits E_STARTUP_INCOMPLETE for a created service", async () => { const projectRoot = await createProject(); psRows.push( From 9f203664dfbaf4aa3e5fd486c6416a7079b92b3c Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:41:12 -0400 Subject: [PATCH 31/52] fix(env): write .hack/.env and the encrypted store atomically with 0600 .hack/.env and .hack/.env. (materialize, legacy env set/unset and the daemon env route) are now read without following a symlink and replaced atomically: a 0600 O_EXCL|O_NOFOLLOW sibling in .hack/ is written, fsynced and renamed over the target, then the directory is synced. A symlinked (including dangling) or non-regular env file, or a symlinked .hack directory, is refused with a fixed message that names no file, overlay or value. An unchanged file with group/other bits is tightened. The encrypted_file store (and the cloud shim that shares it) is replaced the same way with mode 0600; a symlinked store path is refused for writes. set/delete now serialize read-modify-write with a per-path in-process queue plus the cooperative hard-link owner lock used by the projects registry at .lock, so concurrent writers, including separate processes, no longer lose updates. Reads stay lock-free. Adds v1 AES-GCM context-binding characterization tests for the store envelope and legacy v1: values: SHA-256 key derivation, fixed field set, no authenticated associated data, no path/scope/variable binding beyond the scoped plaintext key. No envelope format change. Refs HACK-1127 Co-Authored-By: Claude Opus 5.5 --- docs/env.md | 18 +- src/lib/hack-env.ts | 24 +- src/lib/private-env-file.ts | 74 ++++ src/lib/private-file.ts | 166 +++++++- src/lib/project-env-config.ts | 6 +- src/lib/projects-registry-lock.ts | 5 +- src/lib/secret-store.ts | 133 +++++-- tests/aes-gcm-v1-context-binding.test.ts | 482 +++++++++++++++++++++++ tests/private-env-file.test.ts | 358 +++++++++++++++++ tests/secret-store-file-hygiene.test.ts | 248 ++++++++++++ 10 files changed, 1467 insertions(+), 47 deletions(-) create mode 100644 src/lib/private-env-file.ts create mode 100644 tests/aes-gcm-v1-context-binding.test.ts create mode 100644 tests/private-env-file.test.ts create mode 100644 tests/secret-store-file-hygiene.test.ts diff --git a/docs/env.md b/docs/env.md index a9e1eed5..ccd5e690 100644 --- a/docs/env.md +++ b/docs/env.md @@ -195,6 +195,15 @@ That means `.hack/.env` is no longer the primary runtime source of truth. `hack env materialize` is manual by design. Use it only when you need a compatibility file for an external tool that expects `.env` on disk. +Hack writes `.hack/.env` and `.hack/.env.` (from `materialize`, legacy +`hack env set`, and the daemon env route) with mode `0600` by atomic replace: a +private temporary file in `.hack/` is written, fsynced and renamed over the target, +so readers never see a partial file. An existing file with group or other access is +tightened to `0600` even when its contents are unchanged. Hack refuses, with a fixed +message that names no file or value, to read or write an env file that is a symlink +(including a dangling one) or not a regular file, or one whose `.hack` directory is a +symlink. Replace the link with a regular file or directory to continue. + For host execution, `hack host exec`, `hack host shell`, and lifecycle hooks/processes use a host-local view. That means Hack prefers host-usable values when it can: @@ -529,7 +538,14 @@ The modern YAML env model is the canonical repo format today. Some older surfaces are still compatibility-oriented: -- `hack env backend ...` remains relevant for legacy repos and older secret-store configurations +- `hack env backend ...` remains relevant for legacy repos and older secret-store configurations. + The `encrypted_file` store (also used by the `cloud` shim) is replaced atomically + with mode `0600`, and set/delete hold an in-process queue plus a cooperative + `.lock` so concurrent writers, including separate processes, do not lose + updates. A store path that is a symlink is refused for writes. The v1 store + envelope and legacy `v1:` values authenticate no associated data: they are not + bound to a path, project or variable name beyond the scoped key inside the + encrypted store plaintext - the daemon's env HTTP route (used by the macOS app) reads and writes the legacy dotenv-shaped contract (`source: 'plain_env'|'keychain'`, `resolvedFrom: 'dotenv'/'process'/'keychain'/'portable_backend'`), not the modern YAML `hack env list --json` shape; there is no web dashboard surface anymore diff --git a/src/lib/hack-env.ts b/src/lib/hack-env.ts index 6a0277ae..77ba543c 100644 --- a/src/lib/hack-env.ts +++ b/src/lib/hack-env.ts @@ -6,8 +6,12 @@ import { } from "../constants.ts"; import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { parseDotEnv, serializeDotEnv } from "./env.ts"; -import { readTextFile, writeTextFileIfChanged } from "./fs.ts"; +import { readTextFile } from "./fs.ts"; import { getString, isRecord } from "./guards.ts"; +import { + readPrivateEnvFile, + writePrivateEnvFileIfChanged, +} from "./private-env-file.ts"; import { readProjectDefaultEnvConfig } from "./project.ts"; import type { SecretStoreDescriptor } from "./secret-store.ts"; import { @@ -502,19 +506,23 @@ export async function upsertDotEnvValue(opts: { readonly key: string; readonly value: string; }): Promise<{ readonly changed: boolean }> { - const existingText = (await readTextFile(opts.envFile)) ?? ""; + const existingText = + (await readPrivateEnvFile({ envFile: opts.envFile })) ?? ""; const env = parseDotEnv(existingText); const nextEnv: Record = { ...env, [opts.key]: opts.value }; const nextText = serializeDotEnvStable(nextEnv); - const result = await writeTextFileIfChanged(opts.envFile, nextText); - return { changed: result.changed }; + return await writePrivateEnvFileIfChanged({ + envFile: opts.envFile, + text: nextText, + }); } export async function removeDotEnvKey(opts: { readonly envFile: string; readonly key: string; }): Promise<{ readonly changed: boolean }> { - const existingText = (await readTextFile(opts.envFile)) ?? ""; + const existingText = + (await readPrivateEnvFile({ envFile: opts.envFile })) ?? ""; const env = parseDotEnv(existingText); if (!(opts.key in env)) { return { changed: false }; @@ -527,8 +535,10 @@ export async function removeDotEnvKey(opts: { nextEnv[key] = value; } const nextText = serializeDotEnvStable(nextEnv); - const result = await writeTextFileIfChanged(opts.envFile, nextText); - return { changed: result.changed }; + return await writePrivateEnvFileIfChanged({ + envFile: opts.envFile, + text: nextText, + }); } export function resolveKeychainServiceName(opts: { diff --git a/src/lib/private-env-file.ts b/src/lib/private-env-file.ts new file mode 100644 index 00000000..48e58a46 --- /dev/null +++ b/src/lib/private-env-file.ts @@ -0,0 +1,74 @@ +import { lstat, mkdir } from "node:fs/promises"; +import { dirname } from "node:path"; +import { isRecord } from "./guards.ts"; +import { + type ReplaceablePrivateFileOptions, + readReplaceablePrivateFile, + replacePrivateFileAtomic, +} from "./private-file.ts"; + +/** + * Plaintext env files under `.hack/` (`.hack/.env` and `.hack/.env.`). + * + * Ownership: Hack owns these files. They are read without following a symlink + * and replaced atomically with mode 0600. A symlink or non-regular entry at the + * file, or a symlinked `.hack` directory, is refused with + * {@link PRIVATE_ENV_FILE_REFUSAL}; the refusal never names the file, overlay + * or any value. Ancestors above `.hack` are not inspected. + */ +export const PRIVATE_ENV_FILE_REFUSAL = + "Refused .hack env file access: the env file or its .hack directory is a symlink or not a regular file. Replace it with a regular file or directory and retry; values omitted."; + +function privateEnvFileRefusal(): Error { + return new Error(PRIVATE_ENV_FILE_REFUSAL); +} + +function envFileOptions(envFile: string): ReplaceablePrivateFileOptions { + return { + path: envFile, + directories: [dirname(envFile)], + refusal: privateEnvFileRefusal, + }; +} + +/** Current text, or null when the env file is absent. */ +export async function readPrivateEnvFile(opts: { + readonly envFile: string; +}): Promise { + const current = await readReplaceablePrivateFile( + envFileOptions(opts.envFile) + ); + return current?.text ?? null; +} + +/** + * Writes `text` when it differs from the current contents, or when the existing + * file grants any group/other permission. `changed` reports a content change. + */ +export async function writePrivateEnvFileIfChanged(opts: { + readonly envFile: string; + readonly text: string; +}): Promise<{ readonly changed: boolean }> { + await ensureEnvFileDirectory({ directory: dirname(opts.envFile) }); + const options = envFileOptions(opts.envFile); + const existing = await readReplaceablePrivateFile(options); + const changed = existing?.text !== opts.text; + if (changed || ((existing?.mode ?? 0) & 0o077) !== 0) { + await replacePrivateFileAtomic({ ...options, text: opts.text }); + } + return { changed }; +} + +/** Creates a missing `.hack` directory; an existing entry is left to the refusal checks. */ +async function ensureEnvFileDirectory(opts: { + readonly directory: string; +}): Promise { + try { + await lstat(opts.directory); + } catch (error: unknown) { + if (!(isRecord(error) && error.code === "ENOENT")) { + throw error; + } + await mkdir(opts.directory, { recursive: true }); + } +} diff --git a/src/lib/private-file.ts b/src/lib/private-file.ts index fd7f02d6..3776ef23 100644 --- a/src/lib/private-file.ts +++ b/src/lib/private-file.ts @@ -1,12 +1,14 @@ +import { randomUUID } from "node:crypto"; import { constants, type Stats } from "node:fs"; import { type FileHandle, lstat, open, realpath, + rename, unlink, } from "node:fs/promises"; -import { dirname } from "node:path"; +import { basename, dirname, join } from "node:path"; import { isRecord } from "./guards.ts"; /** @@ -199,6 +201,168 @@ export async function createPrivateFileExclusive(opts: { await syncDirectory({ path: dirname(opts.path) }); } +/** Options shared by the no-follow read and atomic replace of a private file. */ +export type ReplaceablePrivateFileOptions = { + readonly path: string; + /** + * Directories (normally the file's parent inside the project) that must be + * real directories rather than symlinks, with a stable identity across the + * operation. Ancestors outside this list are not inspected. + */ + readonly directories: readonly string[]; + /** Builds the caller's fixed, value-free refusal. */ + readonly refusal: () => Error; +}; + +/** Current contents and permission bits of a replaceable private file. */ +export type ReplaceablePrivateFile = { + readonly text: string; + readonly mode: number; +}; + +type DirectoryIdentity = { readonly path: string; readonly stats: Stats }; + +async function captureRealDirectories( + opts: ReplaceablePrivateFileOptions +): Promise { + return await Promise.all( + opts.directories.map(async (path) => { + const stats = await lstat(path).catch(() => null); + if (!stats?.isDirectory()) { + throw opts.refusal(); + } + return { path, stats }; + }) + ); +} + +async function recheckRealDirectories(opts: { + readonly directories: readonly DirectoryIdentity[]; + readonly refusal: () => Error; +}): Promise { + for (const { path, stats } of opts.directories) { + const current = await lstat(path).catch(() => null); + if ( + !current?.isDirectory() || + current.dev !== stats.dev || + current.ino !== stats.ino + ) { + throw opts.refusal(); + } + } +} + +/** Null when absent; refuses a symlink (dangling or not) or non-regular entry. */ +async function observeReplaceableTarget(opts: { + readonly path: string; + readonly refusal: () => Error; +}): Promise { + const observed = await lstat(opts.path).catch((error: unknown) => { + if (isMissingEntryError(error)) { + return null; + } + throw error; + }); + if (observed && !observed.isFile()) { + throw opts.refusal(); + } + return observed; +} + +/** + * Reads a private file that is later replaced with + * {@link replacePrivateFileAtomic}. Returns null when the entry is absent. A + * symlink at the path, a non-regular entry, a symlinked or replaced directory + * from `directories`, or a file that changes identity during the read throws + * `refusal()`; other filesystem errors (for example EACCES) propagate, so an + * unreadable file is never mistaken for an empty one and overwritten. + */ +export async function readReplaceablePrivateFile( + opts: ReplaceablePrivateFileOptions +): Promise { + const directories = await captureRealDirectories(opts); + const observed = await observeReplaceableTarget(opts); + if (!observed) { + await recheckRealDirectories({ directories, refusal: opts.refusal }); + return null; + } + let file: FileHandle; + try { + file = await open( + opts.path, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK + ); + } catch (error: unknown) { + if (isRecord(error) && error.code === "ELOOP") { + throw opts.refusal(); + } + throw error; + } + try { + const opened = await file.stat(); + if ( + !opened.isFile() || + opened.dev !== observed.dev || + opened.ino !== observed.ino + ) { + throw opts.refusal(); + } + const text = await file.readFile({ encoding: "utf8" }); + await recheckRealDirectories({ directories, refusal: opts.refusal }); + return { text, mode: opened.mode & 0o7777 }; + } finally { + await file.close(); + } +} + +/** + * Replaces `path` with `text` atomically and durably, leaving mode 0600. + * + * A uniquely named sibling is created with O_CREAT|O_EXCL|O_NOFOLLOW and mode + * 0600, fchmod 0600, written and fsynced, then renamed over `path`; readers see + * either the previous or the complete new contents, never a partial file. A + * symlink (dangling or not) or non-regular entry at `path`, or a symlinked or + * replaced directory from `directories`, throws `refusal()` before any write + * reaches the target. rename(2) never follows a final-component symlink, so + * even a link planted after the check is replaced rather than written through. + * The staging file is removed on every failure path. + */ +export async function replacePrivateFileAtomic( + opts: ReplaceablePrivateFileOptions & { readonly text: string } +): Promise { + const directories = await captureRealDirectories(opts); + await observeReplaceableTarget(opts); + const staging = join( + dirname(opts.path), + `.${basename(opts.path)}.${randomUUID()}.tmp` + ); + const file = await open( + staging, + constants.O_CREAT | + constants.O_EXCL | + constants.O_NOFOLLOW | + constants.O_WRONLY, + 0o600 + ); + let committed = false; + try { + // The umask can only remove bits; restore owner read/write explicitly. + await file.chmod(0o600); + await file.writeFile(opts.text, "utf8"); + await file.sync(); + await recheckRealDirectories({ directories, refusal: opts.refusal }); + await observeReplaceableTarget(opts); + await rename(staging, opts.path); + committed = true; + } finally { + await file.close(); + if (!committed) { + await unlink(staging).catch(() => undefined); + } + } + await syncDirectory({ path: dirname(opts.path) }); +} + /** Best effort: persist the directory entry where the platform allows it. */ async function syncDirectory(opts: { readonly path: string }): Promise { let directory: FileHandle | null = null; diff --git a/src/lib/project-env-config.ts b/src/lib/project-env-config.ts index 1d756962..b4d0166e 100644 --- a/src/lib/project-env-config.ts +++ b/src/lib/project-env-config.ts @@ -54,6 +54,7 @@ import { acquireNativeManagedEnvFile, assertNativeProjectInputRoot, } from "./native-project-inputs.ts"; +import { writePrivateEnvFileIfChanged } from "./private-env-file.ts"; import { createPrivateFileExclusive, decodeCanonicalBase64, @@ -2398,7 +2399,10 @@ export async function materializeProjectEnv(opts: { const envPath = resolve(opts.projectDir, PROJECT_ENV_FILENAME); const text = serializeDotEnv(selectedEnv); await assertLegacyProjectDirectory({ projectDir: opts.projectDir }); - const changed = (await writeTextFileIfChanged(envPath, text)).changed; + // Mode 0600, atomic replace; a symlinked .env or .hack is refused. + const changed = ( + await writePrivateEnvFileIfChanged({ envFile: envPath, text }) + ).changed; await ensureDir( dirname(resolveProjectEnvStatePath({ projectDir: opts.projectDir })) ); diff --git a/src/lib/projects-registry-lock.ts b/src/lib/projects-registry-lock.ts index 84f03bb4..4448879f 100644 --- a/src/lib/projects-registry-lock.ts +++ b/src/lib/projects-registry-lock.ts @@ -39,6 +39,8 @@ export async function withProjectsRegistryLock(opts: { readonly waitForLock?: boolean; readonly signal?: AbortSignal; readonly timeoutMs?: number; + /** Fixed resource name for the timeout message; defaults to the registry. */ + readonly label?: string; }): Promise { opts.signal?.throwIfAborted(); if (opts.waitForLock === false) { @@ -102,6 +104,7 @@ async function acquire(opts: { readonly waitForLock?: boolean; readonly signal?: AbortSignal; readonly timeoutMs?: number; + readonly label?: string; }): Promise { const started = performance.now(); const timeout = opts.timeoutMs ?? WAIT_MS; @@ -121,7 +124,7 @@ async function acquire(opts: { ? "; recovery guard is busy (interrupted recovery requires offline inspection)" : ""; throw new Error( - `Timed out waiting for projects registry lock${recovery}` + `Timed out waiting for ${opts.label ?? "projects registry"} lock${recovery}` ); } // Jitter prevents a burst of CLI clients from retrying in lockstep. diff --git a/src/lib/secret-store.ts b/src/lib/secret-store.ts index 84749224..5d56cdbe 100644 --- a/src/lib/secret-store.ts +++ b/src/lib/secret-store.ts @@ -15,7 +15,11 @@ import type { import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { resolveGlobalHackDir } from "./config-paths.ts"; import { ensureDir, readTextFile } from "./fs.ts"; -import { decodeCanonicalBase64 } from "./private-file.ts"; +import { + decodeCanonicalBase64, + replacePrivateFileAtomic, +} from "./private-file.ts"; +import { withProjectsRegistryLock } from "./projects-registry-lock.ts"; const ENCRYPTED_FILE_KEY_ENV = "HACK_SECRETS_FILE_KEY"; const DISABLE_ENCRYPTED_FILE_KEYCHAIN_FALLBACK_ENV = @@ -31,7 +35,11 @@ const ENCRYPTED_IV_BYTES = 12; const ENCRYPTED_TAG_BYTES = 16; /** Decoded ciphertext bound for the whole encrypted store. */ const ENCRYPTED_CIPHERTEXT_LIMIT = 16 * 1024 * 1024; +const ENCRYPTED_STORE_WRITE_REFUSAL = + "Refused encrypted secret store write: the store path is a symlink or not a regular file. Replace it with a regular file and retry; values omitted."; let cachedEncryptedFileKey: string | null = null; +/** Per-path tail of the in-process mutation queue; see withEncryptedStoreMutation. */ +const encryptedStoreMutationTails = new Map>(); let encryptedFileKeyFailureCooldownUntilMs = 0; type EncryptedStorePayload = { @@ -388,45 +396,91 @@ function createEncryptedScopedStore(input: { }); return entries[scopedKey] ?? null; }, - set: async ({ key, value }) => { - const entries = await readEncryptedEntries({ - filePath: input.filePath, - encryptionKey: input.encryptionKey, - }); - const scopedKey = buildScopedSecretKey({ - scope: input.scope, - key, - }); - entries[scopedKey] = value; - await writeEncryptedEntries({ - filePath: input.filePath, - encryptionKey: input.encryptionKey, - entries, - }); - }, - delete: async ({ key }) => { - const entries = await readEncryptedEntries({ + set: async ({ key, value }) => + await withEncryptedStoreMutation({ filePath: input.filePath, - encryptionKey: input.encryptionKey, - }); - const scopedKey = buildScopedSecretKey({ - scope: input.scope, - key, - }); - if (!(scopedKey in entries)) { - return false; - } - delete entries[scopedKey]; - await writeEncryptedEntries({ + run: async () => { + const entries = await readEncryptedEntries({ + filePath: input.filePath, + encryptionKey: input.encryptionKey, + }); + const scopedKey = buildScopedSecretKey({ + scope: input.scope, + key, + }); + entries[scopedKey] = value; + await writeEncryptedEntries({ + filePath: input.filePath, + encryptionKey: input.encryptionKey, + entries, + }); + }, + }), + delete: async ({ key }) => + await withEncryptedStoreMutation({ filePath: input.filePath, - encryptionKey: input.encryptionKey, - entries, - }); - return true; - }, + run: async () => { + const entries = await readEncryptedEntries({ + filePath: input.filePath, + encryptionKey: input.encryptionKey, + }); + const scopedKey = buildScopedSecretKey({ + scope: input.scope, + key, + }); + if (!(scopedKey in entries)) { + return false; + } + delete entries[scopedKey]; + await writeEncryptedEntries({ + filePath: input.filePath, + encryptionKey: input.encryptionKey, + entries, + }); + return true; + }, + }), }; } +/** + * Serializes read-modify-write of one encrypted store file. + * + * Writers in this process queue per configured path, so they never poll each + * other; across processes the cooperative `.lock` (the same exclusive + * hard-link owner protocol as the projects registry, with dead-owner recovery) + * admits one writer at a time. Every set/delete re-reads the store inside the + * lock, so a concurrent writer's update is never lost. Reads take no lock: + * writes replace the file atomically, so a reader sees a whole envelope. + */ +async function withEncryptedStoreMutation(input: { + readonly filePath: string; + readonly run: () => Promise; +}): Promise { + const previous = + encryptedStoreMutationTails.get(input.filePath) ?? Promise.resolve(); + let release = (): void => undefined; + const turn = new Promise((resolveTurn) => { + release = resolveTurn; + }); + const tail = previous.then(() => turn); + encryptedStoreMutationTails.set(input.filePath, tail); + await previous; + try { + await ensureDir(dirname(input.filePath)); + return await withProjectsRegistryLock({ + lockPath: `${input.filePath}.lock`, + label: "encrypted secret store", + run: input.run, + }); + } finally { + release(); + if (encryptedStoreMutationTails.get(input.filePath) === tail) { + encryptedStoreMutationTails.delete(input.filePath); + } + } +} + function buildScopedSecretKey(input: { readonly scope: string; readonly key: string; @@ -751,7 +805,14 @@ async function writeEncryptedEntries(input: { encryptionKey: input.encryptionKey, }); await ensureDir(dirname(input.filePath)); - await Bun.write(input.filePath, `${JSON.stringify(envelope, null, 2)}\n`); + // Atomic same-directory replace with mode 0600. The configured store + // location's ancestors are the user's choice and are not inspected. + await replacePrivateFileAtomic({ + path: input.filePath, + directories: [], + text: `${JSON.stringify(envelope, null, 2)}\n`, + refusal: () => new Error(ENCRYPTED_STORE_WRITE_REFUSAL), + }); } /** diff --git a/tests/aes-gcm-v1-context-binding.test.ts b/tests/aes-gcm-v1-context-binding.test.ts new file mode 100644 index 00000000..3f29993b --- /dev/null +++ b/tests/aes-gcm-v1-context-binding.test.ts @@ -0,0 +1,482 @@ +/** + * Characterization of v1 AES-GCM context binding (HACK-1127). + * + * These tests pin existing behavior; they do not change or endorse it. Both v1 + * formats (the encrypted_file store envelope and the legacy `v1:` secure env + * value) derive the AES-256 key as SHA-256(key text), use a 12-byte IV and a + * 16-byte tag, and authenticate NO associated data. Consequently a v1 envelope + * is not bound to its file path, project, scope, overlay or variable name: + * + * - the store binds an entry to its project only through the scoped key inside + * the authenticated plaintext (`encodeURIComponent(scope)::encodeURIComponent(key)`); + * - a legacy `v1:` value moved to another variable or scope still decrypts. + * + * Adding context binding requires a new envelope version; a v1 change that made + * any of these tests fail would break existing v1 stores and values. + */ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import * as nodeCrypto from "node:crypto"; +import { + copyFile, + mkdir, + mkdtemp, + readFile, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { YAML } from "bun"; + +import { + PROJECT_COMPOSE_FILENAME, + PROJECT_CONFIG_FILENAME, + PROJECT_ENV_KEY_FILENAME, +} from "../src/constants.ts"; +import { + resolveProjectEnvConfig, + setProjectEnvValue, +} from "../src/lib/project-env-config.ts"; +import { resolveSecretStore } from "../src/lib/secret-store.ts"; +import { restoreEnv } from "./helpers/env.ts"; + +const STORE_KEY = "synthetic-v1-binding-store-key"; +const ENV_KEY = "synthetic-v1-binding-env-key"; +const SENTINEL = "synthetic-v1-binding-value-never-in-errors"; +const OTHER_SENTINEL = "synthetic-v1-binding-other-value"; +/** Fixed test vector IV; production always draws a random IV. */ +const FIXED_IV = Buffer.from("000102030405060708090a0b", "hex"); +const ISOLATED_ENV_KEYS = [ + "HOME", + "HACK_HOME", + "HACK_GLOBAL_CONFIG_PATH", + "HACK_SECRETS_FILE_KEY", + "HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK", + "HACK_ENV_SECRET_KEY", +] as const; + +let root: string | null = null; +const previousEnv = new Map(); + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-v1-binding-")); + for (const key of ISOLATED_ENV_KEYS) { + previousEnv.set(key, process.env[key]); + } + process.env.HOME = join(root, "home"); + process.env.HACK_HOME = join(root, "home", ".hack"); + process.env.HACK_GLOBAL_CONFIG_PATH = join(root, "home", "hack.config.json"); + process.env.HACK_SECRETS_FILE_KEY = STORE_KEY; + process.env.HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK = "1"; + Reflect.deleteProperty(process.env, "HACK_ENV_SECRET_KEY"); + await mkdir(join(root, "home"), { recursive: true }); +}); + +afterEach(async () => { + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } + for (const key of ISOLATED_ENV_KEYS) { + restoreEnv(key, previousEnv.get(key)); + } + previousEnv.clear(); +}); + +function requireRoot(): string { + if (!root) { + throw new Error("Missing temp root"); + } + return root; +} + +function deriveKey(keyText: string): Buffer { + return nodeCrypto.createHash("sha256").update(keyText, "utf8").digest(); +} + +/** AES-256-GCM seal with an optional AAD, mirroring the v1 parameters. */ +function seal(opts: { + readonly keyText: string; + readonly plaintext: string; + readonly aad?: Buffer; +}): { readonly iv: Buffer; readonly tag: Buffer; readonly ciphertext: Buffer } { + const cipher = nodeCrypto.createCipheriv( + "aes-256-gcm", + deriveKey(opts.keyText), + FIXED_IV, + { authTagLength: 16 } + ); + if (opts.aad) { + cipher.setAAD(opts.aad); + } + const ciphertext = Buffer.concat([ + cipher.update(opts.plaintext, "utf8"), + cipher.final(), + ]); + return { iv: FIXED_IV, tag: cipher.getAuthTag(), ciphertext }; +} + +function open(opts: { + readonly keyText: string; + readonly iv: string; + readonly tag: string; + readonly ciphertext: string; +}): string { + const decipher = nodeCrypto.createDecipheriv( + "aes-256-gcm", + deriveKey(opts.keyText), + Buffer.from(opts.iv, "base64"), + { authTagLength: 16 } + ); + decipher.setAuthTag(Buffer.from(opts.tag, "base64")); + return Buffer.concat([ + decipher.update(Buffer.from(opts.ciphertext, "base64")), + decipher.final(), + ]).toString("utf8"); +} + +// --------------------------------------------------------------------------- +// encrypted_file store envelope +// --------------------------------------------------------------------------- + +async function configureStore(opts: { + readonly storePath: string; + readonly cloud?: boolean; +}): Promise { + const configPath = process.env.HACK_GLOBAL_CONFIG_PATH; + if (!configPath) { + throw new Error("Missing HACK_GLOBAL_CONFIG_PATH"); + } + const secrets = opts.cloud + ? { + backend: "cloud", + encryptedFile: { path: opts.storePath }, + cloud: { + provider: "aws", + project: "dev-account", + secretPrefix: "hack-cli", + }, + } + : { backend: "encrypted_file", encryptedFile: { path: opts.storePath } }; + await writeFile( + configPath, + `${JSON.stringify({ controlPlane: { secrets } })}\n` + ); +} + +function storeEnvelopeText(opts: { + readonly plaintext: string; + readonly aad?: Buffer; +}): string { + const sealed = seal({ keyText: STORE_KEY, ...opts }); + return `${JSON.stringify({ + version: 1, + algorithm: "aes-256-gcm", + iv: sealed.iv.toString("base64"), + tag: sealed.tag.toString("base64"), + ciphertext: sealed.ciphertext.toString("base64"), + })}\n`; +} + +const STORE_PLAINTEXT = JSON.stringify({ + version: 1, + entries: { "app::TOKEN": SENTINEL }, +}); + +test("v1 store: written envelope has the pinned fields and a scoped-key plaintext", async () => { + const storePath = resolve(requireRoot(), "secrets.enc.json"); + await configureStore({ storePath }); + const store = await resolveSecretStore({ projectName: "app" }); + await store.set({ key: "TOKEN", value: SENTINEL }); + + const envelope = JSON.parse(await readFile(storePath, "utf8")) as Record< + string, + string | number + >; + expect(Object.keys(envelope)).toEqual([ + "version", + "algorithm", + "iv", + "tag", + "ciphertext", + ]); + expect(envelope.version).toBe(1); + expect(envelope.algorithm).toBe("aes-256-gcm"); + // Opening with SHA-256(key), no AAD, reproduces the exact v1 plaintext. + expect( + open({ + keyText: STORE_KEY, + iv: String(envelope.iv), + tag: String(envelope.tag), + ciphertext: String(envelope.ciphertext), + }) + ).toBe(STORE_PLAINTEXT); +}); + +test("v1 store: a test-vector envelope sealed without AAD opens", async () => { + const storePath = resolve(requireRoot(), "secrets.enc.json"); + await configureStore({ storePath }); + await writeFile(storePath, storeEnvelopeText({ plaintext: STORE_PLAINTEXT })); + const store = await resolveSecretStore({ projectName: "app" }); + expect(await store.get({ key: "TOKEN" })).toBe(SENTINEL); +}); + +for (const [label, aad] of [ + ["the store path", (storePath: string) => Buffer.from(storePath, "utf8")], + ["the project scope", () => Buffer.from("app", "utf8")], + ["the envelope version", () => Buffer.from("v1", "utf8")], +] as const) { + test(`v1 store: an envelope sealed with ${label} as AAD is refused (no AAD is authenticated)`, async () => { + const storePath = resolve(requireRoot(), "secrets.enc.json"); + await configureStore({ storePath }); + await writeFile( + storePath, + storeEnvelopeText({ plaintext: STORE_PLAINTEXT, aad: aad(storePath) }) + ); + const store = await resolveSecretStore({ projectName: "app" }); + const error = await store.get({ key: "TOKEN" }).then( + () => null, + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Error); + expect(String(error)).toContain( + "Cannot decrypt encrypted secret store at " + ); + for (const privateText of [SENTINEL, STORE_KEY]) { + expect(String(error)).not.toContain(privateText); + } + }); +} + +test("v1 store: an envelope is not bound to its file path", async () => { + const firstPath = resolve(requireRoot(), "first.enc.json"); + await configureStore({ storePath: firstPath }); + const first = await resolveSecretStore({ projectName: "app" }); + await first.set({ key: "TOKEN", value: SENTINEL }); + + const movedPath = resolve(requireRoot(), "moved", "second.enc.json"); + await mkdir(resolve(requireRoot(), "moved")); + await copyFile(firstPath, movedPath); + await configureStore({ storePath: movedPath }); + const moved = await resolveSecretStore({ projectName: "app" }); + expect(moved.descriptor.location).toBe(movedPath); + expect(await moved.get({ key: "TOKEN" })).toBe(SENTINEL); +}); + +test("v1 store: project binding lives only in the scoped plaintext key", async () => { + const storePath = resolve(requireRoot(), "secrets.enc.json"); + await configureStore({ storePath }); + // A test-vector entry under another project's scoped key is visible to that + // project and invisible to this one; nothing outside the plaintext binds it. + await writeFile( + storePath, + storeEnvelopeText({ + plaintext: JSON.stringify({ + version: 1, + entries: { + "app::TOKEN": SENTINEL, + "my%20app::TOKEN": OTHER_SENTINEL, + }, + }), + }) + ); + expect( + await (await resolveSecretStore({ projectName: "app" })).get({ + key: "TOKEN", + }) + ).toBe(SENTINEL); + expect( + await (await resolveSecretStore({ projectName: "other" })).get({ + key: "TOKEN", + }) + ).toBeNull(); + expect( + await (await resolveSecretStore({ projectName: "my app" })).get({ + key: "TOKEN", + }) + ).toBe(OTHER_SENTINEL); +}); + +test("v1 store: the cloud shim scope is a plaintext key prefix in the same envelope", async () => { + const storePath = resolve(requireRoot(), "cloud.enc.json"); + await configureStore({ storePath, cloud: true }); + const store = await resolveSecretStore({ projectName: "app" }); + await store.set({ key: "API_TOKEN", value: SENTINEL }); + const envelope = JSON.parse(await readFile(storePath, "utf8")) as Record< + string, + string + >; + const plaintext = JSON.parse( + open({ + keyText: STORE_KEY, + iv: String(envelope.iv), + tag: String(envelope.tag), + ciphertext: String(envelope.ciphertext), + }) + ) as { readonly entries: Record }; + expect(plaintext.entries).toEqual({ + "cloud%3Aaws%3Adev-account%3Ahack-cli%3Aapp::API_TOKEN": SENTINEL, + }); +}); + +// --------------------------------------------------------------------------- +// legacy `v1:` secure env values +// --------------------------------------------------------------------------- + +async function createLegacyProject(): Promise<{ + readonly projectRoot: string; + readonly projectDir: string; + readonly configPath: string; +}> { + const projectRoot = resolve(requireRoot(), "repo"); + const projectDir = resolve(projectRoot, ".hack"); + await mkdir(projectDir, { recursive: true }); + await writeFile( + resolve(projectDir, PROJECT_COMPOSE_FILENAME), + "services:\n api: {}\n" + ); + await writeFile( + resolve(projectDir, PROJECT_CONFIG_FILENAME), + `${JSON.stringify({ name: "v1-binding", dev_host: "v1-binding.hack" })}\n` + ); + await writeFile( + resolve(projectRoot, PROJECT_ENV_KEY_FILENAME), + `${ENV_KEY}\n` + ); + return { + projectRoot, + projectDir, + configPath: resolve(projectDir, "hack.env.default.yaml"), + }; +} + +type LegacyConfig = { + readonly version: number; + readonly environment: string; + readonly secretsprovider: string; + readonly values: Record>; +}; + +async function readLegacyConfig(path: string): Promise { + return YAML.parse(await readFile(path, "utf8")) as LegacyConfig; +} + +function secureOf(config: LegacyConfig, scope: string, key: string): string { + const secure = config.values[scope]?.[key]?.secure; + if (typeof secure !== "string") { + throw new Error("missing secure value"); + } + return secure; +} + +async function resolveLegacy(project: { + readonly projectRoot: string; + readonly projectDir: string; +}) { + const resolved = await resolveProjectEnvConfig({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + serviceNames: ["api"], + }); + if (!resolved) { + throw new Error("missing resolved config"); + } + return resolved; +} + +test("legacy v1: a secure value swapped across variable and scope still decrypts", async () => { + const project = await createLegacyProject(); + for (const [scope, key, value] of [ + ["global", "TOKEN", SENTINEL], + ["api", "OTHER", OTHER_SENTINEL], + ] as const) { + await setProjectEnvValue({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + scope, + key, + value, + secret: true, + }); + } + const config = await readLegacyConfig(project.configPath); + const token = secureOf(config, "global", "TOKEN"); + const other = secureOf(config, "api", "OTHER"); + expect(token.split(":")[0]).toBe("v1"); + expect(Buffer.from(token.split(":")[1] ?? "", "base64").length).toBe(12); + + await writeFile( + project.configPath, + JSON.stringify({ + ...config, + values: { + global: { TOKEN: { secure: other } }, + api: { OTHER: { secure: token } }, + }, + }) + ); + const resolved = await resolveLegacy(project); + expect(resolved.globalEnv.TOKEN).toBe(OTHER_SENTINEL); + expect(resolved.serviceEnv.api?.OTHER).toBe(SENTINEL); +}); + +function legacyValue(aad?: Buffer): string { + const sealed = seal({ keyText: ENV_KEY, plaintext: SENTINEL, aad }); + return [ + "v1", + sealed.iv.toString("base64"), + sealed.tag.toString("base64"), + sealed.ciphertext.toString("base64"), + ].join(":"); +} + +async function writeLegacyValue(opts: { + readonly configPath: string; + readonly secure: string; +}): Promise { + await writeFile( + opts.configPath, + JSON.stringify({ + version: 1, + environment: "default", + secretsprovider: "project_key", + values: { global: { TOKEN: { secure: opts.secure } } }, + }) + ); +} + +test("legacy v1: a test-vector value sealed without AAD decrypts", async () => { + const project = await createLegacyProject(); + await writeLegacyValue({ + configPath: project.configPath, + secure: legacyValue(), + }); + expect((await resolveLegacy(project)).globalEnv.TOKEN).toBe(SENTINEL); +}); + +for (const [label, aad] of [ + ["the variable name", Buffer.from("TOKEN", "utf8")], + ["the scope", Buffer.from("global", "utf8")], + ["the version prefix", Buffer.from("v1", "utf8")], +] as const) { + test(`legacy v1: a value sealed with ${label} as AAD is refused (no AAD is authenticated)`, async () => { + const project = await createLegacyProject(); + await writeLegacyValue({ + configPath: project.configPath, + secure: legacyValue(aad), + }); + const error = await resolveLegacy(project).then( + () => null, + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Error); + expect(error instanceof Error ? error.message : "").toBe( + "Invalid secure env value." + ); + for (const privateText of [SENTINEL, ENV_KEY]) { + expect(String(error)).not.toContain(privateText); + } + }); +} diff --git a/tests/private-env-file.test.ts b/tests/private-env-file.test.ts new file mode 100644 index 00000000..6d099d27 --- /dev/null +++ b/tests/private-env-file.test.ts @@ -0,0 +1,358 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { + chmod, + link, + lstat, + mkdir, + mkdtemp, + readdir, + readFile, + rm, + stat, + symlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +import { + PROJECT_COMPOSE_FILENAME, + PROJECT_CONFIG_FILENAME, + PROJECT_ENV_FILENAME, +} from "../src/constants.ts"; +import { removeDotEnvKey, upsertDotEnvValue } from "../src/lib/hack-env.ts"; +import { PRIVATE_ENV_FILE_REFUSAL } from "../src/lib/private-env-file.ts"; +import { + materializeProjectEnv, + setProjectEnvValue, +} from "../src/lib/project-env-config.ts"; +import { restoreEnv } from "./helpers/env.ts"; + +// Synthetic values; none of them may appear in a refusal. +const SENTINEL_KEY = "SYNTHETIC_PRIVATE_ENV_KEY"; +const SENTINEL_VALUE = "synthetic-private-env-value-never-in-errors"; +const OUTSIDE_TEXT = "OUTSIDE_SENTINEL=synthetic-outside-file-never-read\n"; + +let root: string | null = null; +let previousHome: string | undefined; +let previousGlobalConfigPath: string | undefined; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-private-env-file-")); + previousHome = process.env.HOME; + previousGlobalConfigPath = process.env.HACK_GLOBAL_CONFIG_PATH; + // No real HOME or global config is read or written by these tests. + process.env.HOME = join(root, "home"); + process.env.HACK_GLOBAL_CONFIG_PATH = join(root, "home", "hack.config.json"); + await mkdir(join(root, "home"), { recursive: true }); +}); + +afterEach(async () => { + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } + restoreEnv("HOME", previousHome); + restoreEnv("HACK_GLOBAL_CONFIG_PATH", previousGlobalConfigPath); +}); + +async function createProject(): Promise<{ + readonly projectRoot: string; + readonly projectDir: string; + readonly envFile: string; + readonly outsideFile: string; +}> { + if (!root) { + throw new Error("Missing temp root"); + } + const projectRoot = resolve(root, "repo"); + const projectDir = resolve(projectRoot, ".hack"); + await mkdir(projectDir, { recursive: true }); + await writeFile( + resolve(projectDir, PROJECT_COMPOSE_FILENAME), + "services:\n api: {}\n" + ); + await writeFile( + resolve(projectDir, PROJECT_CONFIG_FILENAME), + `${JSON.stringify({ name: "private-env-test", dev_host: "private-env.hack" })}\n` + ); + const outsideFile = resolve(root, "outside.env"); + await writeFile(outsideFile, OUTSIDE_TEXT); + return { + projectRoot, + projectDir, + envFile: resolve(projectDir, PROJECT_ENV_FILENAME), + outsideFile, + }; +} + +function expectClosedRefusal(error: unknown, paths: readonly string[]): void { + expect(error).toBeInstanceOf(Error); + expect(error instanceof Error ? error.message : "").toBe( + PRIVATE_ENV_FILE_REFUSAL + ); + for (const privateText of [ + SENTINEL_KEY, + SENTINEL_VALUE, + "OUTSIDE_SENTINEL", + ...paths, + ]) { + expect(String(error)).not.toContain(privateText); + } +} + +async function caught(promise: Promise): Promise { + return await promise.then( + () => null, + (error: unknown) => error + ); +} + +async function stagingLeftovers(directory: string): Promise { + return (await readdir(directory)).filter((name) => name.endsWith(".tmp")); +} + +test("upsertDotEnvValue creates .hack/.env with mode 0600", async () => { + const project = await createProject(); + const result = await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }); + expect(result.changed).toBe(true); + expect((await stat(project.envFile)).mode & 0o777).toBe(0o600); + expect(await readFile(project.envFile, "utf8")).toContain( + `${SENTINEL_KEY}=${SENTINEL_VALUE}` + ); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); + +test("upsertDotEnvValue replaces .hack/.env atomically rather than writing in place", async () => { + const project = await createProject(); + await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: "first", + }); + // A second name for the old inode keeps the old bytes only if the writer + // renamed a new file over the path instead of truncating and rewriting it. + const previousName = resolve(project.projectDir, "previous-inode"); + await link(project.envFile, previousName); + const before = await stat(project.envFile); + + await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: "second", + }); + + const after = await stat(project.envFile); + expect(after.ino).not.toBe(before.ino); + expect(after.mode & 0o777).toBe(0o600); + expect(await readFile(previousName, "utf8")).toContain( + `${SENTINEL_KEY}=first` + ); + expect(await readFile(project.envFile, "utf8")).toContain( + `${SENTINEL_KEY}=second` + ); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); + +test("an unchanged .hack/.env with group or other bits is tightened to 0600", async () => { + const project = await createProject(); + await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }); + const text = await readFile(project.envFile, "utf8"); + await chmod(project.envFile, 0o644); + + const result = await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }); + expect(result.changed).toBe(false); + expect((await stat(project.envFile)).mode & 0o777).toBe(0o600); + expect(await readFile(project.envFile, "utf8")).toBe(text); +}); + +test("removeDotEnvKey keeps mode 0600 on the rewritten file", async () => { + const project = await createProject(); + await upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }); + await upsertDotEnvValue({ + envFile: project.envFile, + key: "KEEP", + value: "1", + }); + const result = await removeDotEnvKey({ + envFile: project.envFile, + key: SENTINEL_KEY, + }); + expect(result.changed).toBe(true); + expect((await stat(project.envFile)).mode & 0o777).toBe(0o600); + expect(await readFile(project.envFile, "utf8")).toBe("KEEP=1\n"); +}); + +test("negative control: a symlinked .hack/.env is refused without reading or writing its target", async () => { + const project = await createProject(); + await symlink(project.outsideFile, project.envFile); + + for (const attempt of [ + upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }), + removeDotEnvKey({ envFile: project.envFile, key: "OUTSIDE_SENTINEL" }), + ]) { + expectClosedRefusal(await caught(attempt), [ + project.envFile, + project.outsideFile, + ]); + } + + expect(await readFile(project.outsideFile, "utf8")).toBe(OUTSIDE_TEXT); + expect((await lstat(project.envFile)).isSymbolicLink()).toBe(true); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); + +test("negative control: a dangling .hack/.env symlink is refused and its target is never created", async () => { + const project = await createProject(); + const missingTarget = resolve(project.projectRoot, "..", "never-created.env"); + await symlink(missingTarget, project.envFile); + + expectClosedRefusal( + await caught( + upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }) + ), + [project.envFile, missingTarget] + ); + await expect(lstat(missingTarget)).rejects.toThrow(); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); + +test("negative control: an env overlay file that is a symlink is refused", async () => { + const project = await createProject(); + const overlayFile = `${project.envFile}.staging`; + await symlink(project.outsideFile, overlayFile); + + expectClosedRefusal( + await caught( + upsertDotEnvValue({ + envFile: overlayFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }) + ), + [overlayFile, project.outsideFile, "staging"] + ); + expect(await readFile(project.outsideFile, "utf8")).toBe(OUTSIDE_TEXT); +}); + +test("negative control: a symlinked .hack directory is refused and the linked directory is untouched", async () => { + if (!root) { + throw new Error("Missing temp root"); + } + const outsideDir = resolve(root, "outside-hack"); + await mkdir(outsideDir); + const projectRoot = resolve(root, "linked-repo"); + await mkdir(projectRoot); + const projectDir = resolve(projectRoot, ".hack"); + await symlink(outsideDir, projectDir); + const envFile = resolve(projectDir, PROJECT_ENV_FILENAME); + + expectClosedRefusal( + await caught( + upsertDotEnvValue({ envFile, key: SENTINEL_KEY, value: SENTINEL_VALUE }) + ), + [envFile, outsideDir] + ); + expect(await readdir(outsideDir)).toEqual([]); +}); + +test("a directory at the .hack/.env path is refused with the same closed diagnostic", async () => { + const project = await createProject(); + await mkdir(project.envFile); + expectClosedRefusal( + await caught( + upsertDotEnvValue({ + envFile: project.envFile, + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + }) + ), + [project.envFile] + ); +}); + +test("materializeProjectEnv writes .hack/.env with mode 0600 by atomic replace", async () => { + const project = await createProject(); + await setProjectEnvValue({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + scope: "global", + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + secret: false, + }); + await writeFile(project.envFile, "STALE=1\n", { mode: 0o644 }); + await chmod(project.envFile, 0o644); + const previousName = resolve(project.projectDir, "previous-inode"); + await link(project.envFile, previousName); + + const result = await materializeProjectEnv({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + serviceName: null, + serviceNames: ["api"], + }); + + expect(result.changed).toBe(true); + expect((await stat(project.envFile)).mode & 0o777).toBe(0o600); + expect(await readFile(project.envFile, "utf8")).toContain( + `${SENTINEL_KEY}=${SENTINEL_VALUE}` + ); + expect(await readFile(previousName, "utf8")).toBe("STALE=1\n"); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); + +test("negative control: materializeProjectEnv refuses a symlinked .hack/.env target", async () => { + const project = await createProject(); + await setProjectEnvValue({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + scope: "global", + key: SENTINEL_KEY, + value: SENTINEL_VALUE, + secret: false, + }); + await symlink(project.outsideFile, project.envFile); + + expectClosedRefusal( + await caught( + materializeProjectEnv({ + projectRoot: project.projectRoot, + projectDir: project.projectDir, + envName: null, + serviceName: null, + serviceNames: ["api"], + }) + ), + [project.envFile, project.outsideFile] + ); + expect(await readFile(project.outsideFile, "utf8")).toBe(OUTSIDE_TEXT); + expect((await lstat(project.envFile)).isSymbolicLink()).toBe(true); +}); diff --git a/tests/secret-store-file-hygiene.test.ts b/tests/secret-store-file-hygiene.test.ts new file mode 100644 index 00000000..3be32e5c --- /dev/null +++ b/tests/secret-store-file-hygiene.test.ts @@ -0,0 +1,248 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { + link, + lstat, + mkdir, + mkdtemp, + readdir, + readFile, + rm, + stat, + symlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +import { resolveSecretStore } from "../src/lib/secret-store.ts"; +import { restoreEnv } from "./helpers/env.ts"; + +const STORE_KEY = "synthetic-store-hygiene-key-for-tests-only"; +const STORE_SENTINEL = "synthetic-store-hygiene-value-never-in-errors"; +const SECRET_STORE_MODULE = resolve( + import.meta.dir, + "..", + "src", + "lib", + "secret-store.ts" +); +const WRITES_PER_PROCESS = 20; +const ISOLATED_ENV_KEYS = [ + "HOME", + "HACK_HOME", + "HACK_GLOBAL_CONFIG_PATH", + "HACK_SECRETS_FILE_KEY", + "HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK", +] as const; + +let root: string | null = null; +const previousEnv = new Map(); + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-secret-store-hygiene-")); + for (const key of ISOLATED_ENV_KEYS) { + previousEnv.set(key, process.env[key]); + } + // Temp HOME and global config; the env key wins, and the keychain fallback is + // disabled so no test can reach the native keychain. + process.env.HOME = join(root, "home"); + process.env.HACK_HOME = join(root, "home", ".hack"); + process.env.HACK_GLOBAL_CONFIG_PATH = join(root, "home", "hack.config.json"); + process.env.HACK_SECRETS_FILE_KEY = STORE_KEY; + process.env.HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK = "1"; + await mkdir(join(root, "home"), { recursive: true }); +}); + +afterEach(async () => { + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } + for (const key of ISOLATED_ENV_KEYS) { + restoreEnv(key, previousEnv.get(key)); + } + previousEnv.clear(); +}); + +async function configureEncryptedFileStore(): Promise<{ + readonly storeDir: string; + readonly storePath: string; +}> { + if (!root) { + throw new Error("Missing temp root"); + } + const storeDir = resolve(root, "store"); + await mkdir(storeDir); + const storePath = resolve(storeDir, "secrets.enc.json"); + const configPath = process.env.HACK_GLOBAL_CONFIG_PATH; + if (!configPath) { + throw new Error("Missing HACK_GLOBAL_CONFIG_PATH"); + } + await writeFile( + configPath, + `${JSON.stringify({ + controlPlane: { + secrets: { + backend: "encrypted_file", + encryptedFile: { path: storePath }, + }, + }, + })}\n` + ); + return { storeDir, storePath }; +} + +async function nonEnvelopeEntries(directory: string): Promise { + return (await readdir(directory)).filter( + (name) => name !== "secrets.enc.json" + ); +} + +test("encrypted_file store writes mode 0600 by atomic replace", async () => { + const { storeDir, storePath } = await configureEncryptedFileStore(); + const store = await resolveSecretStore({ projectName: "app" }); + await store.set({ key: "TOKEN", value: "first" }); + expect((await stat(storePath)).mode & 0o777).toBe(0o600); + + // The old inode keeps its bytes only when the writer renames a new file over + // the path instead of truncating and rewriting the existing one. + const previousName = resolve(storeDir, "previous-inode"); + await link(storePath, previousName); + const previousText = await readFile(storePath, "utf8"); + const before = await stat(storePath); + + await store.set({ key: "TOKEN", value: STORE_SENTINEL }); + + const after = await stat(storePath); + expect(after.ino).not.toBe(before.ino); + expect(after.mode & 0o777).toBe(0o600); + expect(await readFile(previousName, "utf8")).toBe(previousText); + expect(await store.get({ key: "TOKEN" })).toBe(STORE_SENTINEL); + expect(await readFile(storePath, "utf8")).not.toContain(STORE_SENTINEL); + + expect(await store.delete({ key: "TOKEN" })).toBe(true); + expect((await stat(storePath)).mode & 0o777).toBe(0o600); + // No staging file or lock receipt is left behind. + expect(await nonEnvelopeEntries(storeDir)).toEqual(["previous-inode"]); +}); + +test("encrypted_file store refuses to write through a symlinked store path", async () => { + if (!root) { + throw new Error("Missing temp root"); + } + const { storeDir, storePath } = await configureEncryptedFileStore(); + const outsideFile = resolve(root, "outside-store.json"); + // An empty target reads as an empty store, so only the write guard can stop + // the set below. + await writeFile(outsideFile, ""); + await symlink(outsideFile, storePath); + + const store = await resolveSecretStore({ projectName: "app" }); + const error = await store.set({ key: "TOKEN", value: STORE_SENTINEL }).then( + () => null, + (caught: unknown) => caught + ); + + expect(error).toBeInstanceOf(Error); + expect(error instanceof Error ? error.message : "").toBe( + "Refused encrypted secret store write: the store path is a symlink or not a regular file. Replace it with a regular file and retry; values omitted." + ); + for (const privateText of [STORE_SENTINEL, STORE_KEY, "TOKEN", outsideFile]) { + expect(String(error)).not.toContain(privateText); + } + expect(await readFile(outsideFile, "utf8")).toBe(""); + expect((await lstat(storePath)).isSymbolicLink()).toBe(true); + expect(await nonEnvelopeEntries(storeDir)).toEqual([]); +}); + +test("negative control: concurrent in-process writers do not lose an update", async () => { + const { storeDir } = await configureEncryptedFileStore(); + const writers = await Promise.all( + ["app", "app", "other", "other"].map((projectName) => + resolveSecretStore({ projectName }) + ) + ); + const writes: Promise[] = []; + for (let index = 0; index < 24; index += 1) { + const writer = writers[index % writers.length]; + if (!writer) { + throw new Error("missing writer"); + } + writes.push(writer.set({ key: `KEY_${index}`, value: `value-${index}` })); + } + await Promise.all(writes); + + for (const projectName of ["app", "other"]) { + const store = await resolveSecretStore({ projectName }); + for (let index = 0; index < 24; index += 1) { + const writtenBy = index % 4 < 2 ? "app" : "other"; + expect(await store.get({ key: `KEY_${index}` })).toBe( + writtenBy === projectName ? `value-${index}` : null + ); + } + } + expect(await nonEnvelopeEntries(storeDir)).toEqual([]); +}); + +test("negative control: concurrent writer processes do not lose an update", async () => { + if (!root) { + throw new Error("Missing temp root"); + } + const { storeDir } = await configureEncryptedFileStore(); + const script = resolve(root, "store-writer.ts"); + await writeFile( + script, + [ + `import { resolveSecretStore } from ${JSON.stringify(SECRET_STORE_MODULE)};`, + "const prefix = process.argv[2] ?? 'P';", + "const startAt = Number(process.argv[3]);", + "const store = await resolveSecretStore({ projectName: 'app' });", + "// Start together so the writers' read-modify-write cycles overlap.", + "await Bun.sleep(Math.max(0, startAt - Date.now()));", + `for (let index = 0; index < ${WRITES_PER_PROCESS}; index += 1) {`, + " await store.set({ key: `${prefix}_${index}`, value: `${prefix}-${index}` });", + "}", + "", + ].join("\n") + ); + const childEnv: Record = { + PATH: process.env.PATH ?? "", + TMPDIR: process.env.TMPDIR ?? tmpdir(), + }; + for (const key of ISOLATED_ENV_KEYS) { + const value = process.env[key]; + if (value !== undefined) { + childEnv[key] = value; + } + } + const prefixes = ["A", "B", "C"]; + const startAt = String(Date.now() + 750); + const children = prefixes.map((prefix) => + Bun.spawn({ + cmd: [process.execPath, script, prefix, startAt], + cwd: root ?? undefined, + env: childEnv, + stdout: "ignore", + stderr: "pipe", + }) + ); + const exits = await Promise.all( + children.map(async (child) => ({ + code: await child.exited, + stderr: await new Response(child.stderr).text(), + })) + ); + for (const exit of exits) { + expect(exit).toEqual({ code: 0, stderr: "" }); + } + + const store = await resolveSecretStore({ projectName: "app" }); + for (const prefix of prefixes) { + for (let index = 0; index < WRITES_PER_PROCESS; index += 1) { + expect(await store.get({ key: `${prefix}_${index}` })).toBe( + `${prefix}-${index}` + ); + } + } + expect(await nonEnvelopeEntries(storeDir)).toEqual([]); +}); From e2f063be6f12f804821da5dce13a8e63efa48097 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:41:31 -0400 Subject: [PATCH 32/52] feat(agents): add focused hack-branches, hack-env and hack-diagnose plugin skills Compose three focused plugin skills from canonical instruction section ids (src/agents/focused-skills.ts); composition fails on an unknown or repeated id with a closed error. New canonical sections (branch-aliases, env-commands, env-safety, diagnose) carry no surface tags, so they render only in the focused skills and leave the broad surfaces unchanged apart from the content revision. - hack-branches leads with --branch and hack branch add/list/open/ remove; linked worktrees are an optional entry. - hack-env renders the canonical env-composition section (the only precedence statement) plus env list/set --local/exec/apply and safety: no secret values in argv or prompts, get and list --show-secrets are plaintext sinks. - hack-diagnose gives the read-first order (ps, doctor, finite logs, env explain, global status) and ships references/error-codes.md rendered from the newly exported HACK_ERROR_CODES registry. - The plugin hack-cli description routes branch/env/diagnose intents to the focused skills; the standalone render keeps broad routing. - Replace the bare env-guide URL in env-composition with an offline pointer so packaged skills pass the offline link contract. Refs: HACK-1131 Co-Authored-By: Claude Opus 5.5 --- .codex/skills/hack-cli/SKILL.md | 4 +- .cursor/rules/hack.mdc | 4 +- docs/agent-guidance.md | 15 +- docs/integrations.md | 11 +- docs/reference/agent-client-capabilities.md | 4 +- examples/basic/AGENTS.md | 4 +- examples/basic/CLAUDE.md | 4 +- plugins/hack/README.md | 11 +- plugins/hack/rules/hack.mdc | 4 +- plugins/hack/skills/hack-branches/SKILL.md | 38 ++ plugins/hack/skills/hack-cli/SKILL.md | 12 +- plugins/hack/skills/hack-diagnose/SKILL.md | 44 ++ .../hack-diagnose/references/error-codes.md | 32 ++ plugins/hack/skills/hack-env/SKILL.md | 36 ++ scripts/generate-agent-plugins.ts | 37 +- src/agents/client-capabilities.ts | 2 +- src/agents/codex-skill.ts | 26 +- src/agents/error-code-reference.ts | 126 ++++++ src/agents/focused-skills.ts | 224 ++++++++++ src/agents/instruction-source.ts | 61 ++- src/agents/integration-ownership.ts | 2 + src/agents/integration-revision.ts | 2 +- src/agents/skill-references.ts | 12 + src/lib/cli-result.ts | 50 ++- tests/agent-plugin-generation.test.ts | 7 + tests/focused-skills.test.ts | 417 ++++++++++++++++++ tests/skill-references.test.ts | 8 +- 27 files changed, 1131 insertions(+), 66 deletions(-) create mode 100644 plugins/hack/skills/hack-branches/SKILL.md create mode 100644 plugins/hack/skills/hack-diagnose/SKILL.md create mode 100644 plugins/hack/skills/hack-diagnose/references/error-codes.md create mode 100644 plugins/hack/skills/hack-env/SKILL.md create mode 100644 src/agents/error-code-reference.ts create mode 100644 src/agents/focused-skills.ts create mode 100644 tests/focused-skills.test.ts diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index a46be0db..907da6fa 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). ## Product boundary @@ -152,7 +152,7 @@ Use `hack` as the primary interface for local-first development. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Branch instances (parallel envs) diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 1adb401c..60833f48 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -9,7 +9,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). ## Product boundary @@ -71,7 +71,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Agent integration maintenance diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 63df3f65..d6440098 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -21,7 +21,9 @@ run only `hack agent prime`. See [model-policy ownership](reference/agent-client | `src/agents/hack-init-skill.ts` | Thin Claude/Codex/plugin adapter that fetches onboarding guidance | | `src/agents/init-patterns.ts`, `init-assistant.ts` | Optional init inventory and pattern hints | | `src/agents/install-skill.ts`, `remote-bootstrap-skill.ts` | Plugin installation/channel selection and fresh-host/cloud setup skills | -| `scripts/generate-agent-plugins.ts` | Portable plugin/MCP manifests, three client compatibility manifests, four skills, and Cursor rule | +| `src/agents/focused-skills.ts` | Plugin `hack-branches`, `hack-env` and `hack-diagnose` skills, composed from canonical section ids, and the plugin `hack-cli` routing description | +| `src/agents/error-code-reference.ts` | `hack-diagnose` error-code reference, rendered from the exported `HACK_ERROR_CODES` registry in `src/lib/cli-result.ts` | +| `scripts/generate-agent-plugins.ts` | Portable plugin/MCP manifests, three client compatibility manifests, seven skills, and Cursor rule | The portable plugin manifest derives identity and effective OpenAI presentation from `.codex-plugin/plugin.json`; portable MCP derives from `.mcp.json` with an @@ -93,6 +95,17 @@ and domain owner; the disposition does not waive that owner's runtime gates. `tests/pattern-catalogue.test.ts` checks completeness, CLI_SPEC command resolution and the banned-recipe scan. +The focused plugin skills `hack-branches`, `hack-env` and `hack-diagnose` list +canonical section ids in `src/agents/focused-skills.ts` and carry no prose of +their own; composition fails on an unknown id. A section with no surface tags +renders only in the focused skills that name it. Each trigger phrase belongs to +one plugin skill, and the plugin `hack-cli` description hands branch, env and +diagnosis intents to them; the standalone `hack-cli` keeps broad routing because +those installs ship no focused skills. The `hack-diagnose` error-code reference +renders from `HACK_ERROR_CODES` in `src/lib/cli-result.ts`. +`tests/focused-skills.test.ts` checks section ids, CLI_SPEC resolution, +error-code parity and trigger overlap. + Skill candidates and where their patterns live now: | Candidate | Decision | Patterns live in | Promotion owner and missing contract | diff --git a/docs/integrations.md b/docs/integrations.md index 9519e9e7..5618bcce 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -68,9 +68,12 @@ trust and approval rules. ## Optional native plugins Hack also ships an optional native plugin for Codex, Claude Code, and Cursor in -`plugins/hack`. It bundles four skills (`hack-cli`, `hack-init`, `hack-install`, and -`hack-remote`), the `hack mcp serve` server, Claude primer hooks, and a Cursor rule -that the agent can select when relevant. The +`plugins/hack`. It bundles seven skills (`hack-cli`, the focused `hack-branches`, +`hack-env` and `hack-diagnose`, `hack-init`, `hack-install`, and `hack-remote`), the +`hack mcp serve` server, Claude primer hooks, and a Cursor rule that the agent can +select when relevant. The plugin's `hack-cli` routes branch-instance, env and +diagnosis requests to the focused skills; standalone `hack setup` installs keep the +broad `hack-cli` skill only. The [agent client capability matrix](reference/agent-client-capabilities.md) records which of these components have been observed loading in each client. Install the Hack CLI first and make `hack` available on the client's PATH. @@ -105,7 +108,7 @@ outside its local-plugin root; use a copy rather than weakening that boundary. S [Cursor's plugin documentation](https://cursor.com/docs/plugins) for marketplace and local development support in the installed client version. -After installing, verify in a fresh session that all four skills are discoverable, +After installing, verify in a fresh session that all seven skills are discoverable, `hack mcp serve` can initialize, and the client's hooks or rule load. A successful marketplace registration alone does not prove those components loaded. The plugin's MCP fallback is for clients without shell access; use the Hack CLI when shell is available. diff --git a/docs/reference/agent-client-capabilities.md b/docs/reference/agent-client-capabilities.md index 8c408b2c..d19bb313 100644 --- a/docs/reference/agent-client-capabilities.md +++ b/docs/reference/agent-client-capabilities.md @@ -56,7 +56,7 @@ Matrix version 1, observed 2026-10-11. | Capability | Meaning | | --- | --- | | Packaging | The client accepts, installs and enables the Hack plugin package in a session. | -| Skill discovery | A fresh session discovers the plugin's bundled skills (hack-cli, hack-init, hack-install, hack-remote). | +| Skill discovery | A fresh session discovers the plugin's bundled skills (hack-cli, hack-branches, hack-env, hack-diagnose, hack-init, hack-install, hack-remote). | | Hooks | The client runs plugin-shipped hooks; Hack hooks may only run `hack agent prime`. | | Plugin MCP | The client starts the plugin-declared `hack mcp serve` server and lists its tools. | | Standalone MCP | The client loads an MCP entry written by `hack mcp install` (stdio or a selected MCP bundle) without the plugin. | @@ -117,7 +117,7 @@ owner, and none of them replaces the others. Users choose one per client. | Layout | What it contains | Authoritative owner | Installed and updated by | | --- | --- | --- | --- | -| Native plugin (`plugins/hack`) | Client manifests, four skills, Claude primer hooks, Cursor rule, `hack mcp serve` declaration | `scripts/generate-agent-plugins.ts`, rendering from `src/agents/instruction-source.ts` and the skill renderers in `src/agents/` | The client's plugin manager. Hack commands never install or update it. | +| Native plugin (`plugins/hack`) | Client manifests, seven skills, Claude primer hooks, Cursor rule, `hack mcp serve` declaration | `scripts/generate-agent-plugins.ts`, rendering from `src/agents/instruction-source.ts` and the skill renderers in `src/agents/` | The client's plugin manager. Hack commands never install or update it. | | Standalone setup | Project AGENTS.md/CLAUDE.md snippets, Cursor rules, shared skills, Claude primer hooks, MCP entries | `src/agents/instruction-source.ts` and its renderers, written by `hack setup` and `hack mcp install` | Explicit `hack setup` and `hack mcp install` commands in a chosen scope | | MCP bundle | Native MCP adapter, owner and backend executables with a content-derived identity | `src/mcp/bundle.ts` (packaging and verification) | `hack mcp install --bundle ... --cli ...` writes the client entry; the bundle is never modified after selection | diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index 8c448dc1..458dbed6 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -129,7 +129,7 @@ Env composition: - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index 8c448dc1..458dbed6 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -129,7 +129,7 @@ Env composition: - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/plugins/hack/README.md b/plugins/hack/README.md index 7533f828..1d408c86 100644 --- a/plugins/hack/README.md +++ b/plugins/hack/README.md @@ -3,9 +3,12 @@ Optional Hack integration for Codex, Claude Code, and Cursor. Install the `hack` executable first; the plugin uses it from PATH. -The bundle contains four skills: +The bundle contains seven skills: -- `hack-cli`: services, logs, environments, worktrees and diagnostics. +- `hack-cli`: services, logs, routed URLs, lifecycle and workspaces. +- `hack-branches`: parallel branch instances (`--branch `) and branch aliases. +- `hack-env`: env values, overlays and worktree-local overrides without exposing secrets. +- `hack-diagnose`: read-first diagnosis and the `--json` error-code reference. - `hack-init`: configure a new or existing project for Hack. - `hack-install`: installation, stable/prerelease selection, upgrade and rollback. - `hack-remote`: prepare an existing project on another machine or in Codex Cloud. @@ -26,7 +29,7 @@ that selection and disable the bundle's MCP entry in the client configuration: enabled = false ``` -This keeps the four plugin skills available without launching a second bundled +This keeps the seven plugin skills available without launching a second bundled server. A standalone registration remains independently configured. Preserve existing customized skills; resolve overlapping installation paths deliberately. @@ -38,7 +41,7 @@ codex plugin add hack@hack-dance codex plugin list --json --marketplace hack-dance ``` -Start a fresh client, confirm all four plugin skills load, and call a harmless +Start a fresh client, confirm all seven plugin skills load, and call a harmless MCP tool such as `hack.projects.list` against an isolated registry. Inspect actual discovered tool names first. The server and Claude hooks still resolve `hack` from the client's PATH; installing diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index 77e495ec..9bae8cb9 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). ## Product boundary @@ -76,7 +76,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Agent integration maintenance diff --git a/plugins/hack/skills/hack-branches/SKILL.md b/plugins/hack/skills/hack-branches/SKILL.md new file mode 100644 index 00000000..8ba7b741 --- /dev/null +++ b/plugins/hack/skills/hack-branches/SKILL.md @@ -0,0 +1,38 @@ +--- +name: hack-branches +description: > + Run parallel branch instances of a Hack project with `--branch ` and manage their aliases + with `hack branch`. Trigger when asked to run two versions of a project side by side; start, + open, tail or stop a branch instance; review a pull request in its own instance; or add, list, + open or remove a branch alias. +--- + +# Hack branch instances + +## Branch instances (parallel envs) + +- Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. +- Target one with `--branch ` on up/open/logs/down (for example: `hack up --branch --detach`). +- Linked worktrees pick a branch instance automatically (see Linked git worktrees). +- Containers receive `HACK_RUNTIME_METADATA` plus `HACK_DEV_URL`, `HACK_ALIAS_URL`, and current-service URL fields derived from effective Caddy routes. Use Compose DNS for internal traffic and this metadata for browser-facing links, OAuth callbacks, and webhooks. + +## Branch aliases + +- Aliases are entries in `.hack/hack.branches.json`; `hack up --branch ` records or refreshes its entry automatically. +- `hack branch add --note ` registers one ahead of time, and `hack branch list` shows each alias with its note and last use. +- `hack branch open ` opens that branch's routed host; `hack branch open --prefer ` selects which host. +- `hack branch remove ` deletes only the alias entry and leaves any running instance alone; stop the instance with `hack down --branch `. + +## Linked git worktrees (optional) + +- Secret key inherits from the primary checkout automatically through the shared git common dir; set `HACK_ENV_SECRET_KEY` for CI or detached environments. +- `hack up` in a linked worktree defaults to a branch instance named after the worktree's git branch; a detached linked worktree requires an explicit `--branch`, unless config `worktree.auto_branch=false` explicitly opts into the base instance. +- Before `hack up` or `hack restart` auto-targets a new branch instance, Hack warns when the same worktree already owns a non-terminal instance; pass `--branch ` to make the target explicit. +- Implicit `hack down` retargets a uniquely owned same-checkout runtime after a Git branch rename, including Created and stopped containers; when multiple runtimes belong to the checkout, pass `--branch ` explicitly. +- `hack doctor` flags divergent secret keys and dev_host collisions across checkouts. + +## Integration freshness + +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. +- Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index 6a5159aa..de781f20 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -1,9 +1,11 @@ --- name: hack-cli description: > - Use the hack CLI for local runtime orchestration (compose, DNS/TLS, logs, env, persistent project workspaces) and agent setup. - Trigger when asked to run/start/stop services, inspect logs, manage lifecycle/workspace workflows, or update - agent integrations. Prefer CLI over MCP when shell access is available. + Use the hack CLI for local runtime orchestration (compose, DNS/TLS, routed URLs, persistent + project workspaces) and agent setup. Trigger when asked to run, start or stop services; tail + service logs; open a routed app URL; manage lifecycle or workspace workflows; or update agent + integrations. Use hack-branches for branch instances, hack-env for env values and hack-diagnose + for a failing project. Prefer CLI over MCP when shell access is available. --- # hack CLI @@ -14,7 +16,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). ## Product boundary @@ -152,7 +154,7 @@ Use `hack` as the primary interface for local-first development. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Branch instances (parallel envs) diff --git a/plugins/hack/skills/hack-diagnose/SKILL.md b/plugins/hack/skills/hack-diagnose/SKILL.md new file mode 100644 index 00000000..e19d89ec --- /dev/null +++ b/plugins/hack/skills/hack-diagnose/SKILL.md @@ -0,0 +1,44 @@ +--- +name: hack-diagnose +description: > + Diagnose a Hack-managed project by reading its state first and repairing only within the + request. Trigger when asked to find out why a service is unhealthy or did not start; interpret + an E_* error code from a JSON envelope; review hack doctor findings; or investigate routing or + global infrastructure failures. +--- + +# Diagnose a Hack project + +## Read-first diagnosis + +- Read before changing anything, in this order, and stop once the cause is clear: +- Step 1, `hack ps --json`: which services exist and their state. +- Step 2, `hack doctor --json`: findings and stable E_* codes. +- Step 3, `hack logs --json --no-follow --tail 200`: a finite snapshot; add a service name to narrow it. +- Step 4, `hack env explain --service `: only when a value is suspect; it reports provenance without the value. +- Step 5, `hack global status`: shared proxy, DNS and logging infrastructure. +- Branch on `error.code`, not on message text; the code is the stable contract and the message may change. +- Report what each read showed before proposing a repair. + +## Operating rules + +- Prefer `hack` over raw `docker` / `docker compose` for project workflows. +- Do not start/stop services from Docker Desktop UI for `hack`-managed projects. +- Treat `.hack/.internal` and `.hack/.branch` as hack-managed artifacts; do not hand-edit generated files there. +- Use `--json` for machine-readable output when available; `hack up/down/restart/doctor --json` emit an `{ok, data | error: {code, message}}` envelope with stable E_* error codes. +- When the exit status matters, do not pipe `hack` output through `| tail` or `| grep`: the pipeline reports the filter's status, and filtered text can hide failures. Check the `hack` exit status or `--json` envelope first, and use finite reads such as `hack logs --no-follow --tail 100`. +- Scripted/agent runs: pass `--no-interactive` (or set `HACK_NO_INTERACTIVE=1`) so commands never block on prompts — they apply documented defaults or fail fast with E_INTERACTIVE_REQUIRED. +- Use MCP only when shell access is unavailable. +- A task request authorizes its stated scope, not unrelated service, global config, or production changes. Preserve native credential/trust approvals and destructive-cleanup gates; do not ask again for the same authorized action. +- Cleanup must prove resource ownership. Successful one-shot containers may exit with code 0; adopted external processes must remain running when Hack stops its own services. +- If runtime state looks wrong, inspect with `hack doctor`; apply `hack doctor --fix` only for repairs covered by the request. Warnings do not authorize broader changes. + +## Integration freshness + +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. +- Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). + +## References + +- [Error codes](references/error-codes.md): every stable `--json` error code with its meaning and the first read-only step. diff --git a/plugins/hack/skills/hack-diagnose/references/error-codes.md b/plugins/hack/skills/hack-diagnose/references/error-codes.md new file mode 100644 index 00000000..041c17ec --- /dev/null +++ b/plugins/hack/skills/hack-diagnose/references/error-codes.md @@ -0,0 +1,32 @@ +# Hack error codes + +Commands run with `--json` print one envelope. A failure has the shape +`{ok: false, error: {code, message}}`; branch on `code`, not on `message`. +Each read-first step only inspects state. Repairs such as `hack doctor --fix` +stay within the request. + +| Code | Meaning | Read first | +| --- | --- | --- | +| `E_DOCKER_UNAVAILABLE` | The container runtime CLI or engine cannot be reached. | `hack doctor --json` | +| `E_GLOBAL_INFRA_DOWN` | Shared global infrastructure the command needs is offline. | `hack global status` | +| `E_CONFIG_PARSE` | A Hack configuration file could not be parsed. | `hack doctor --json` | +| `E_CONFIG_INVALID` | Project configuration or saved runtime input did not pass validation. | `hack doctor --json` | +| `E_PROJECT_NOT_FOUND` | No Hack project resolved from the working directory, `--project` or `--path`. | `hack projects --json` | +| `E_NATIVE_PROJECT_CONFLICT` | Native project input and legacy Hack inputs coexist; project commands refuse until one is reconciled. | `hack doctor --json` | +| `E_NATIVE_PROJECT_UNSUPPORTED` | The project or the requested operation needs native runtime support that this CLI does not provide. | `hack doctor --json` | +| `E_NATIVE_COMPOSE_PROXY_ACCESS` | Native Compose routing could not read the live proxy state. | `hack global status` | +| `E_SERVICE_NOT_FOUND` | The named service is not part of the selected project runtime. | `hack ps --json` | +| `E_COMPOSE_FAILED` | A Compose operation failed or left a stop incomplete. | `hack logs --json --no-follow --tail 200` | +| `E_STARTUP_INCOMPLETE` | After start or restart, a service is neither running nor a successful one-shot. | `hack ps --json` | +| `E_DEPENDENCY_BOOTSTRAP_FAILED` | A dependency installer service did not complete successfully. | `hack logs --json --no-follow --tail 200` | +| `E_STARTUP_TIMEOUT` | Bounded startup ran out of time before services settled. | `hack ps --json` | +| `E_LIFECYCLE_FAILED` | A lifecycle hook or host process failed. | `hack projects --details` | +| `E_ENV_KEY_MISSING` | The selected env overlay cannot supply keys the command requires; the detail lists names only. | `hack env explain --service ` | +| `E_PERMISSION` | A filesystem or runtime permission was denied. | `hack doctor --json` | +| `E_INTERACTIVE_REQUIRED` | The command needs an answer at a prompt but is running without one. | `hack --help` | +| `E_USAGE` | Arguments or flags were invalid for the command. | `hack --help` | +| `E_UNEXPECTED` | An unclassified failure; the message is the only detail. | `hack doctor --json` | + +--- + +Content revision: `33e546b5dc60`. Generated by Hack; edit the canonical source and regenerate the plugin. diff --git a/plugins/hack/skills/hack-env/SKILL.md b/plugins/hack/skills/hack-env/SKILL.md new file mode 100644 index 00000000..b6984ba6 --- /dev/null +++ b/plugins/hack/skills/hack-env/SKILL.md @@ -0,0 +1,36 @@ +--- +name: hack-env +description: > + Inspect and change Hack-managed env values, overlays and worktree-local overrides without + exposing secret values. Trigger when asked to explain where an env value comes from; list, get + or set an env value; select or compare env overlays; write a worktree-local env override; run a + host command with managed env; or refresh a service after an env change. +--- + +# Hack env + +## Env composition + +- Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. +- Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. +- Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). + +## Env commands + +- `hack env list --env --service ` shows the resolved values for one overlay and scope with secret values masked; add `--json` for a machine-readable view. +- `hack env set --local` writes a non-secret value to the worktree-local override file for the selected overlay; drop `--local` only when the request covers the shared, committed overlay. +- `hack env exec --env --service -- ` runs one host command with the resolved env injected; use `hack env exec --env --shell ''` when `$VAR` must expand inside the child shell, and add `--timeout ` to bound a non-TTY job. +- `hack env apply --env --service ` recreates that one service so it picks up changed values. + +## Env safety + +- A secret value never goes in command arguments or agent prompts. Store one with `hack env set --secret` and no value, which the user runs in their own terminal and answers at its hidden prompt. +- `hack env get` and `hack env list --show-secrets` are plaintext sinks: run them only when the user asks for the value itself, never to check that a key exists, and do not repeat their output in replies, logs or files. + +## Integration freshness + +- These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. +- Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. +- Content revision: `0201e6bfa037` (version alone is not a freshness guarantee). diff --git a/scripts/generate-agent-plugins.ts b/scripts/generate-agent-plugins.ts index 1a8e8520..83b952fb 100644 --- a/scripts/generate-agent-plugins.ts +++ b/scripts/generate-agent-plugins.ts @@ -1,8 +1,16 @@ import { readdir, rm } from "node:fs/promises"; import { dirname, resolve } from "node:path"; -import { renderCodexSkill } from "../src/agents/codex-skill.ts"; +import { renderHackCliSkill } from "../src/agents/codex-skill.ts"; import { renderCursorRules } from "../src/agents/cursor.ts"; +import { + FOCUSED_SKILLS, + type FocusedSkillName, + HACK_CLI_PLUGIN_ROUTING, + renderDescriptionFrontmatter, + renderFocusedSkill, + renderSkillDescription, +} from "../src/agents/focused-skills.ts"; import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; @@ -29,6 +37,10 @@ export async function generateAgentPlugins({ path: resolve(pluginRoot, "skills", "hack-cli", "SKILL.md"), content: renderPluginSkill("hack-cli"), }, + ...FOCUSED_SKILLS.map((skill) => ({ + path: resolve(pluginRoot, "skills", skill.name, "SKILL.md"), + content: renderPluginSkill(skill.name), + })), { path: resolve(pluginRoot, "skills", "hack-init", "SKILL.md"), content: renderHackInitSkill(), @@ -147,16 +159,29 @@ export function portableMcpManifest( } /** - * Render a plugin skill's SKILL.md. The plugin variant of `hack-cli` adds - * links to its packaged references; standalone installs keep - * {@link renderCodexSkill} because they ship no references. + * Render a plugin skill's SKILL.md with links to its packaged references. + * + * The plugin `hack-cli` routes branch, env and diagnosis intents to the + * focused skills; standalone installs keep `renderCodexSkill` (broad routing, + * no references) because they ship neither. */ -export function renderPluginSkill(skill: "hack-cli"): string { - return withSkillReferenceLinks({ skill, content: renderCodexSkill() }); +export function renderPluginSkill( + skill: "hack-cli" | FocusedSkillName +): string { + const content = + skill === "hack-cli" + ? renderHackCliSkill({ + descriptionLines: renderDescriptionFrontmatter( + renderSkillDescription(HACK_CLI_PLUGIN_ROUTING) + ), + }) + : renderFocusedSkill(skill); + return withSkillReferenceLinks({ skill, content }); } const PLUGIN_SKILLS: readonly PluginSkillName[] = [ "hack-cli", + ...FOCUSED_SKILLS.map((skill) => skill.name), "hack-init", "hack-install", "hack-remote", diff --git a/src/agents/client-capabilities.ts b/src/agents/client-capabilities.ts index 657ee65f..f1140a8a 100644 --- a/src/agents/client-capabilities.ts +++ b/src/agents/client-capabilities.ts @@ -124,7 +124,7 @@ export const AGENT_CLIENT_CAPABILITY_DEFINITIONS: readonly AgentClientCapability capability: "skillDiscovery", label: "Skill discovery", meaning: - "A fresh session discovers the plugin's bundled skills (hack-cli, hack-init, hack-install, hack-remote).", + "A fresh session discovers the plugin's bundled skills (hack-cli, hack-branches, hack-env, hack-diagnose, hack-init, hack-install, hack-remote).", }, { capability: "hooks", diff --git a/src/agents/codex-skill.ts b/src/agents/codex-skill.ts index 4621d83d..ff89c1ba 100644 --- a/src/agents/codex-skill.ts +++ b/src/agents/codex-skill.ts @@ -251,16 +251,32 @@ export async function removeCodexSkill(opts: { * Render the Codex skill template for hack CLI usage. * * Frontmatter and top-level framing live here; the instructional content is - * the `skill`-tagged sections from the shared instruction source. + * the `skill`-tagged sections from the shared instruction source. Standalone + * installs ship no focused skills, so this variant keeps broad routing. */ export function renderCodexSkill(): string { + return renderHackCliSkill({ + descriptionLines: [ + "description: >", + " Use the hack CLI for local runtime orchestration (compose, DNS/TLS, logs, env, persistent project workspaces) and agent setup.", + " Trigger when asked to run/start/stop services, inspect logs, manage lifecycle/workspace workflows, or update", + " agent integrations. Prefer CLI over MCP when shell access is available.", + ], + }); +} + +/** + * Render the `hack-cli` SKILL.md with the given frontmatter description + * lines. The standalone and plugin variants share the body; only routing + * differs. + */ +export function renderHackCliSkill(opts: { + readonly descriptionLines: readonly string[]; +}): string { const lines = [ "---", "name: hack-cli", - "description: >", - " Use the hack CLI for local runtime orchestration (compose, DNS/TLS, logs, env, persistent project workspaces) and agent setup.", - " Trigger when asked to run/start/stop services, inspect logs, manage lifecycle/workspace workflows, or update", - " agent integrations. Prefer CLI over MCP when shell access is available.", + ...opts.descriptionLines, "---", "", "# hack CLI", diff --git a/src/agents/error-code-reference.ts b/src/agents/error-code-reference.ts new file mode 100644 index 00000000..841c38f5 --- /dev/null +++ b/src/agents/error-code-reference.ts @@ -0,0 +1,126 @@ +import { HACK_ERROR_CODES, type HackErrorCode } from "../lib/cli-result.ts"; + +/** + * Agent-facing meaning of each stable `--json` error code. + * + * Keyed by {@link HackErrorCode}, so adding a code to the registry without a + * row here fails typecheck; the packaged reference renders in registry order + * from {@link HACK_ERROR_CODES}, and `tests/focused-skills.test.ts` checks the + * rendered code set against the registry. + */ +type ErrorCodeGuide = { + /** What the failure means, without caller values. */ + readonly meaning: string; + /** The first read-only step; never a repair. */ + readonly readFirst: string; +}; + +const ERROR_CODE_GUIDES: Readonly> = { + E_DOCKER_UNAVAILABLE: { + meaning: "The container runtime CLI or engine cannot be reached.", + readFirst: "`hack doctor --json`", + }, + E_GLOBAL_INFRA_DOWN: { + meaning: "Shared global infrastructure the command needs is offline.", + readFirst: "`hack global status`", + }, + E_CONFIG_PARSE: { + meaning: "A Hack configuration file could not be parsed.", + readFirst: "`hack doctor --json`", + }, + E_CONFIG_INVALID: { + meaning: + "Project configuration or saved runtime input did not pass validation.", + readFirst: "`hack doctor --json`", + }, + E_PROJECT_NOT_FOUND: { + meaning: + "No Hack project resolved from the working directory, `--project` or `--path`.", + readFirst: "`hack projects --json`", + }, + E_NATIVE_PROJECT_CONFLICT: { + meaning: + "Native project input and legacy Hack inputs coexist; project commands refuse until one is reconciled.", + readFirst: "`hack doctor --json`", + }, + E_NATIVE_PROJECT_UNSUPPORTED: { + meaning: + "The project or the requested operation needs native runtime support that this CLI does not provide.", + readFirst: "`hack doctor --json`", + }, + E_NATIVE_COMPOSE_PROXY_ACCESS: { + meaning: "Native Compose routing could not read the live proxy state.", + readFirst: "`hack global status`", + }, + E_SERVICE_NOT_FOUND: { + meaning: "The named service is not part of the selected project runtime.", + readFirst: "`hack ps --json`", + }, + E_COMPOSE_FAILED: { + meaning: "A Compose operation failed or left a stop incomplete.", + readFirst: "`hack logs --json --no-follow --tail 200`", + }, + E_STARTUP_INCOMPLETE: { + meaning: + "After start or restart, a service is neither running nor a successful one-shot.", + readFirst: "`hack ps --json`", + }, + E_DEPENDENCY_BOOTSTRAP_FAILED: { + meaning: "A dependency installer service did not complete successfully.", + readFirst: "`hack logs --json --no-follow --tail 200`", + }, + E_STARTUP_TIMEOUT: { + meaning: "Bounded startup ran out of time before services settled.", + readFirst: "`hack ps --json`", + }, + E_LIFECYCLE_FAILED: { + meaning: "A lifecycle hook or host process failed.", + readFirst: "`hack projects --details`", + }, + E_ENV_KEY_MISSING: { + meaning: + "The selected env overlay cannot supply keys the command requires; the detail lists names only.", + readFirst: "`hack env explain --service `", + }, + E_PERMISSION: { + meaning: "A filesystem or runtime permission was denied.", + readFirst: "`hack doctor --json`", + }, + E_INTERACTIVE_REQUIRED: { + meaning: + "The command needs an answer at a prompt but is running without one.", + readFirst: "`hack --help`", + }, + E_USAGE: { + meaning: "Arguments or flags were invalid for the command.", + readFirst: "`hack --help`", + }, + E_UNEXPECTED: { + meaning: "An unclassified failure; the message is the only detail.", + readFirst: "`hack doctor --json`", + }, +}; + +/** + * Render the `hack-diagnose` error-code reference from the exported registry. + * One table row per code, in registry order. + */ +export function renderErrorCodeReference(): string { + const rows = HACK_ERROR_CODES.map((code) => { + const guide = ERROR_CODE_GUIDES[code]; + return `| \`${code}\` | ${guide.meaning} | ${guide.readFirst} |`; + }); + return [ + "# Hack error codes", + "", + "Commands run with `--json` print one envelope. A failure has the shape", + "`{ok: false, error: {code, message}}`; branch on `code`, not on `message`.", + "Each read-first step only inspects state. Repairs such as `hack doctor --fix`", + "stay within the request.", + "", + "| Code | Meaning | Read first |", + "| --- | --- | --- |", + ...rows, + "", + ].join("\n"); +} diff --git a/src/agents/focused-skills.ts b/src/agents/focused-skills.ts new file mode 100644 index 00000000..39bd7a6b --- /dev/null +++ b/src/agents/focused-skills.ts @@ -0,0 +1,224 @@ +/** + * Focused plugin skills composed from canonical instruction sections. + * + * Each focused skill names the {@link INSTRUCTION_SECTIONS} ids it renders, in + * order; no guidance prose lives here. Only the frontmatter description (the + * routing contract) is owned by this module, built from a summary and a list + * of trigger phrases. A trigger phrase belongs to exactly one plugin skill, + * which `tests/focused-skills.test.ts` enforces across every description. + */ + +import { + INSTRUCTION_SECTIONS, + type InstructionSection, + renderInstructionSection, +} from "./instruction-source.ts"; + +export type FocusedSkillName = "hack-branches" | "hack-env" | "hack-diagnose"; + +/** One canonical section in a focused skill, optionally marked as optional. */ +export type FocusedSkillSection = { + readonly id: string; + /** A secondary entry, rendered with an "(optional)" title suffix. */ + readonly optional?: boolean; +}; + +/** The routing contract a skill's frontmatter description renders from. */ +export type SkillRouting = { + readonly summary: string; + /** Intents that select this skill; each phrase is unique to one skill. */ + readonly triggers: readonly string[]; + /** Optional closing sentences, e.g. where neighbouring intents go. */ + readonly closing?: string; +}; + +export type FocusedSkillSpec = { + readonly name: FocusedSkillName; + readonly heading: string; + readonly routing: SkillRouting; + readonly sections: readonly FocusedSkillSection[]; +}; + +/** + * Routing for the plugin's `hack-cli` skill. It keeps general runtime work + * and hands branch, env and diagnosis intents to the focused skills. The + * standalone `hack setup` render keeps its broad description because those + * installs ship no focused skills. + */ +export const HACK_CLI_PLUGIN_ROUTING: SkillRouting = { + summary: + "Use the hack CLI for local runtime orchestration (compose, DNS/TLS, routed URLs, persistent project workspaces) and agent setup.", + triggers: [ + "run, start or stop services", + "tail service logs", + "open a routed app URL", + "manage lifecycle or workspace workflows", + "update agent integrations", + ], + closing: + "Use hack-branches for branch instances, hack-env for env values and hack-diagnose for a failing project. Prefer CLI over MCP when shell access is available.", +}; + +/** Focused plugin skills, in generation order. */ +export const FOCUSED_SKILLS: readonly FocusedSkillSpec[] = [ + { + name: "hack-branches", + heading: "Hack branch instances", + routing: { + summary: + "Run parallel branch instances of a Hack project with `--branch ` and manage their aliases with `hack branch`.", + triggers: [ + "run two versions of a project side by side", + "start, open, tail or stop a branch instance", + "review a pull request in its own instance", + "add, list, open or remove a branch alias", + ], + }, + sections: [ + { id: "branch-instances" }, + { id: "branch-aliases" }, + { id: "worktrees", optional: true }, + { id: "freshness" }, + ], + }, + { + name: "hack-env", + heading: "Hack env", + routing: { + summary: + "Inspect and change Hack-managed env values, overlays and worktree-local overrides without exposing secret values.", + triggers: [ + "explain where an env value comes from", + "list, get or set an env value", + "select or compare env overlays", + "write a worktree-local env override", + "run a host command with managed env", + "refresh a service after an env change", + ], + }, + sections: [ + { id: "env-composition" }, + { id: "env-commands" }, + { id: "env-safety" }, + { id: "freshness" }, + ], + }, + { + name: "hack-diagnose", + heading: "Diagnose a Hack project", + routing: { + summary: + "Diagnose a Hack-managed project by reading its state first and repairing only within the request.", + triggers: [ + "find out why a service is unhealthy or did not start", + "interpret an E_* error code from a JSON envelope", + "review hack doctor findings", + "investigate routing or global infrastructure failures", + ], + }, + sections: [ + { id: "diagnose" }, + { id: "operating-rules" }, + { id: "freshness" }, + ], + }, +]; + +/** Look up a focused skill by name. */ +export function focusedSkill(name: FocusedSkillName): FocusedSkillSpec { + const spec = FOCUSED_SKILLS.find((candidate) => candidate.name === name); + if (!spec) { + throw new Error("Unknown focused skill"); + } + return spec; +} + +/** + * Render a skill description sentence: the summary, the trigger phrases, and + * the optional closing sentences. + */ +export function renderSkillDescription(routing: SkillRouting): string { + const triggers = routing.triggers; + const last = triggers.at(-1); + if (last === undefined) { + throw new Error("A skill description needs at least one trigger phrase"); + } + const joined = + triggers.length === 1 + ? last + : `${triggers.slice(0, -1).join("; ")}; or ${last}`; + return [ + routing.summary, + `Trigger when asked to ${joined}.`, + ...(routing.closing ? [routing.closing] : []), + ].join(" "); +} + +/** + * Render a description as a YAML folded block (`description: >`) wrapped at + * a readable width. Folding joins the lines back into one sentence. + */ +export function renderDescriptionFrontmatter(description: string): string[] { + const lines: string[] = []; + let current = ""; + for (const word of description.split(" ")) { + if (current.length > 0 && current.length + word.length + 1 > 96) { + lines.push(` ${current}`); + current = word; + } else { + current = current.length > 0 ? `${current} ${word}` : word; + } + } + if (current.length > 0) { + lines.push(` ${current}`); + } + return ["description: >", ...lines]; +} + +/** + * Compose canonical sections by id, in the given order. + * + * Throws on an unknown or repeated id so a renamed canonical section cannot + * silently drop out of a focused skill. Errors are closed and do not echo the + * id. + */ +export function composeCanonicalSections(opts: { + readonly sections: readonly FocusedSkillSection[]; + readonly source?: readonly InstructionSection[]; +}): string { + const source = opts.source ?? INSTRUCTION_SECTIONS; + const seen = new Set(); + const blocks = opts.sections.map((entry) => { + if (seen.has(entry.id)) { + throw new Error("Duplicate instruction section id in focused skill"); + } + seen.add(entry.id); + const section = source.find((candidate) => candidate.id === entry.id); + if (!section) { + throw new Error("Unknown instruction section id in focused skill"); + } + return renderInstructionSection({ + section: entry.optional + ? { ...section, title: `${section.title} (optional)` } + : section, + headingStyle: "markdown", + }); + }); + return blocks.join("\n\n"); +} + +/** Render a focused skill's SKILL.md (without packaged reference links). */ +export function renderFocusedSkill(name: FocusedSkillName): string { + const spec = focusedSkill(name); + return [ + "---", + `name: ${spec.name}`, + ...renderDescriptionFrontmatter(renderSkillDescription(spec.routing)), + "---", + "", + `# ${spec.heading}`, + "", + composeCanonicalSections({ sections: spec.sections }), + "", + ].join("\n"); +} diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 153f83eb..401a9893 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -35,6 +35,10 @@ export const HACK_AGENT_INTEGRATION_CLI_VERSION = ( * - `skill`: Codex SKILL.md (full content). * - `rules`: Cursor `.mdc` rules (lean operational subset). * - `primer`: `hack agent prime` session primer (concise subset). + * + * A section with no surface tags renders only where a focused plugin skill + * names its id (see `src/agents/focused-skills.ts`); the broad surfaces never + * include it, but it still counts toward the content revision. */ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ { @@ -196,6 +200,21 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Global infra logs: `hack global logs caddy --no-follow --tail 200`", ], }, + { + id: "diagnose", + title: "Read-first diagnosis", + surfaces: [], + bullets: [ + "Read before changing anything, in this order, and stop once the cause is clear:", + "Step 1, `hack ps --json`: which services exist and their state.", + "Step 2, `hack doctor --json`: findings and stable E_* codes.", + "Step 3, `hack logs --json --no-follow --tail 200`: a finite snapshot; add a service name to narrow it.", + "Step 4, `hack env explain --service `: only when a value is suspect; it reports provenance without the value.", + "Step 5, `hack global status`: shared proxy, DNS and logging infrastructure.", + "Branch on `error.code`, not on message text; the code is the stable contract and the message may change.", + "Report what each read showed before proposing a repair.", + ], + }, { id: "lifecycle", title: "Lifecycle + startup", @@ -241,7 +260,27 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation.", "Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret.", "Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`.", - "Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix", + "Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository).", + ], + }, + { + id: "env-commands", + title: "Env commands", + surfaces: [], + bullets: [ + "`hack env list --env --service ` shows the resolved values for one overlay and scope with secret values masked; add `--json` for a machine-readable view.", + "`hack env set --local` writes a non-secret value to the worktree-local override file for the selected overlay; drop `--local` only when the request covers the shared, committed overlay.", + "`hack env exec --env --service -- ` runs one host command with the resolved env injected; use `hack env exec --env --shell ''` when `$VAR` must expand inside the child shell, and add `--timeout ` to bound a non-TTY job.", + "`hack env apply --env --service ` recreates that one service so it picks up changed values.", + ], + }, + { + id: "env-safety", + title: "Env safety", + surfaces: [], + bullets: [ + "A secret value never goes in command arguments or agent prompts. Store one with `hack env set --secret` and no value, which the user runs in their own terminal and answers at its hidden prompt.", + "`hack env get` and `hack env list --show-secrets` are plaintext sinks: run them only when the user asks for the value itself, never to check that a key exists, and do not repeat their output in replies, logs or files.", ], }, @@ -256,6 +295,17 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Containers receive `HACK_RUNTIME_METADATA` plus `HACK_DEV_URL`, `HACK_ALIAS_URL`, and current-service URL fields derived from effective Caddy routes. Use Compose DNS for internal traffic and this metadata for browser-facing links, OAuth callbacks, and webhooks.", ], }, + { + id: "branch-aliases", + title: "Branch aliases", + surfaces: [], + bullets: [ + "Aliases are entries in `.hack/hack.branches.json`; `hack up --branch ` records or refreshes its entry automatically.", + "`hack branch add --note ` registers one ahead of time, and `hack branch list` shows each alias with its note and last use.", + "`hack branch open ` opens that branch's routed host; `hack branch open --prefer ` selects which host.", + "`hack branch remove ` deletes only the alias entry and leaves any running instance alone; stop the instance with `hack down --branch `.", + ], + }, { id: "targeting", @@ -340,7 +390,7 @@ export function renderInstructionSections(opts: { readonly headingStyle: InstructionHeadingStyle; }): string { const blocks = sectionsForSurface({ surface: opts.surface }).map((section) => - renderSection({ section, headingStyle: opts.headingStyle }) + renderInstructionSection({ section, headingStyle: opts.headingStyle }) ); return blocks.join("\n\n"); } @@ -361,7 +411,12 @@ export function normalizeInstructionText(opts: { .trim(); } -function renderSection(opts: { +/** + * Render one canonical section. The `freshness` section also carries the + * content revision line, so any render that includes it can be checked for + * drift against the canonical source. + */ +export function renderInstructionSection(opts: { readonly section: InstructionSection; readonly headingStyle: InstructionHeadingStyle; }): string { diff --git a/src/agents/integration-ownership.ts b/src/agents/integration-ownership.ts index 0e1cbb07..1446b31d 100644 --- a/src/agents/integration-ownership.ts +++ b/src/agents/integration-ownership.ts @@ -105,6 +105,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "b6465c18c6d4f41b30a600b8", // v5.0.0-next.1..v5.0.0-next.2 "121d8f26374d24f2a09890a3", // v5.0.0-next.3..next "6449442a8f230412d4e5f9e7", // v5.0.0-next.4..next + "e620e980660f0443df7732fe", // next: focused plugin skills (HACK-1131) ]), "hack-init-skill": new Set([ "cd15fb6d8ed8e25071d05e75", // v3.1.0..v4.0.3 @@ -131,6 +132,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "01d7431b9a5caaba3ffd3463", // v5.0.0-next.1..v5.0.0-next.2 "06cf6d0920afc1780b9946ba", // v5.0.0-next.3..next "cf0a31e197f9034485413674", // v5.0.0-next.4..next + "ea46c0d85989de79dba95fd4", // next: focused plugin skills (HACK-1131) ]), }; diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index 6f3a624e..d799e73f 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "3cb1b0eabdea"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "0201e6bfa037"; diff --git a/src/agents/skill-references.ts b/src/agents/skill-references.ts index 57792bea..d678ee71 100644 --- a/src/agents/skill-references.ts +++ b/src/agents/skill-references.ts @@ -1,10 +1,14 @@ import { createHash } from "node:crypto"; +import { renderErrorCodeReference } from "./error-code-reference.ts"; import { renderPatternCatalogue } from "./pattern-catalogue.ts"; /** Plugin skills that can carry packaged references. */ export type PluginSkillName = | "hack-cli" + | "hack-branches" + | "hack-env" + | "hack-diagnose" | "hack-init" | "hack-install" | "hack-remote"; @@ -39,6 +43,14 @@ export const SKILL_REFERENCES: readonly SkillReference[] = [ "reusable project patterns with what Hack supports today, the missing contract and its owner.", render: renderPatternCatalogue, }, + { + skill: "hack-diagnose", + file: "error-codes.md", + title: "Error codes", + summary: + "every stable `--json` error code with its meaning and the first read-only step.", + render: renderErrorCodeReference, + }, ]; const REFERENCE_FILE_PATTERN = /^[a-z0-9][a-z0-9-]*\.md$/; diff --git a/src/lib/cli-result.ts b/src/lib/cli-result.ts index be1d6630..799d88cb 100644 --- a/src/lib/cli-result.ts +++ b/src/lib/cli-result.ts @@ -8,31 +8,37 @@ */ /** - * Stable machine-readable error codes for `--json` output. + * Stable machine-readable error codes for `--json` output, in registry order. * * Codes are part of the CLI contract: add new ones as failure sites are - * wired up, never repurpose existing ones. + * wired up, never repurpose existing ones. Generated agent references (the + * `hack-diagnose` error-code reference) render from this list, and a drift + * test fails when the two diverge. */ -export type HackErrorCode = - | "E_DOCKER_UNAVAILABLE" - | "E_GLOBAL_INFRA_DOWN" - | "E_CONFIG_PARSE" - | "E_CONFIG_INVALID" - | "E_PROJECT_NOT_FOUND" - | "E_NATIVE_PROJECT_CONFLICT" - | "E_NATIVE_PROJECT_UNSUPPORTED" - | "E_NATIVE_COMPOSE_PROXY_ACCESS" - | "E_SERVICE_NOT_FOUND" - | "E_COMPOSE_FAILED" - | "E_STARTUP_INCOMPLETE" - | "E_DEPENDENCY_BOOTSTRAP_FAILED" - | "E_STARTUP_TIMEOUT" - | "E_LIFECYCLE_FAILED" - | "E_ENV_KEY_MISSING" - | "E_PERMISSION" - | "E_INTERACTIVE_REQUIRED" - | "E_USAGE" - | "E_UNEXPECTED"; +export const HACK_ERROR_CODES = [ + "E_DOCKER_UNAVAILABLE", + "E_GLOBAL_INFRA_DOWN", + "E_CONFIG_PARSE", + "E_CONFIG_INVALID", + "E_PROJECT_NOT_FOUND", + "E_NATIVE_PROJECT_CONFLICT", + "E_NATIVE_PROJECT_UNSUPPORTED", + "E_NATIVE_COMPOSE_PROXY_ACCESS", + "E_SERVICE_NOT_FOUND", + "E_COMPOSE_FAILED", + "E_STARTUP_INCOMPLETE", + "E_DEPENDENCY_BOOTSTRAP_FAILED", + "E_STARTUP_TIMEOUT", + "E_LIFECYCLE_FAILED", + "E_ENV_KEY_MISSING", + "E_PERMISSION", + "E_INTERACTIVE_REQUIRED", + "E_USAGE", + "E_UNEXPECTED", +] as const; + +/** One stable `--json` error code from {@link HACK_ERROR_CODES}. */ +export type HackErrorCode = (typeof HACK_ERROR_CODES)[number]; export type CliResultError = { readonly code: HackErrorCode; diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 8a20add7..ecaafc6b 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -12,6 +12,7 @@ import { renderPluginRule, renderPluginSkill, } from "../scripts/generate-agent-plugins.ts"; +import { FOCUSED_SKILLS } from "../src/agents/focused-skills.ts"; import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; @@ -139,6 +140,12 @@ test("bundled skills and rule match current canonical guidance", async () => { expect(await Bun.file(join(root, "skills/hack-cli/SKILL.md")).text()).toBe( renderPluginSkill("hack-cli") ); + expect(FOCUSED_SKILLS.length).toBeGreaterThan(0); + for (const skill of FOCUSED_SKILLS) { + expect( + await Bun.file(join(root, "skills", skill.name, "SKILL.md")).text() + ).toBe(renderPluginSkill(skill.name)); + } const references = renderPluginReferences(); expect(references.size).toBeGreaterThan(0); for (const [path, content] of references) { diff --git a/tests/focused-skills.test.ts b/tests/focused-skills.test.ts new file mode 100644 index 00000000..580929f4 --- /dev/null +++ b/tests/focused-skills.test.ts @@ -0,0 +1,417 @@ +import { expect, test } from "bun:test"; +import { join } from "node:path"; +import { parse as parseYaml } from "yaml"; + +import { renderPluginSkill } from "../scripts/generate-agent-plugins.ts"; +import { renderErrorCodeReference } from "../src/agents/error-code-reference.ts"; +import { + composeCanonicalSections, + FOCUSED_SKILLS, + type FocusedSkillName, + focusedSkill, + HACK_CLI_PLUGIN_ROUTING, + renderSkillDescription, +} from "../src/agents/focused-skills.ts"; +import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; +import { renderHackInstallSkill } from "../src/agents/install-skill.ts"; +import { + INSTRUCTION_SECTIONS, + type InstructionSection, +} from "../src/agents/instruction-source.ts"; +import { renderHackRemoteSkill } from "../src/agents/remote-bootstrap-skill.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { HACK_ERROR_CODES } from "../src/lib/cli-result.ts"; +import { isRecord } from "../src/lib/guards.ts"; +import { + extractHackInvocations, + findGuidanceCommandIssues, +} from "./helpers/cli-guidance-commands.ts"; + +const pluginRoot = join(import.meta.dir, "..", "plugins", "hack"); + +const FRONTMATTER = /^---\n([\s\S]*?)\n---\n/; + +/** The frontmatter `description` of a rendered skill, as YAML parses it. */ +function descriptionOf(rendered: string): string { + const parsed: unknown = parseYaml(FRONTMATTER.exec(rendered)?.[1] ?? ""); + if (!(isRecord(parsed) && typeof parsed.description === "string")) { + throw new Error("skill render lacks a string description"); + } + return parsed.description.trim(); +} + +function sectionById(id: string): InstructionSection { + const section = INSTRUCTION_SECTIONS.find((candidate) => candidate.id === id); + if (!section) { + throw new Error(`missing canonical section ${id}`); + } + return section; +} + +/** Rendered SKILL.md body of a focused skill, without its frontmatter. */ +function bodyOf(name: FocusedSkillName): string { + const rendered = renderPluginSkill(name); + return rendered.slice(rendered.indexOf("\n---\n") + 5); +} + +type RoutedSkill = { + readonly name: string; + readonly description: string; + readonly triggers: readonly string[]; +}; + +/** Every plugin skill's description, with the trigger phrases it owns. */ +function pluginRouting(): RoutedSkill[] { + return [ + { + name: "hack-cli", + description: descriptionOf(renderPluginSkill("hack-cli")), + triggers: HACK_CLI_PLUGIN_ROUTING.triggers, + }, + ...FOCUSED_SKILLS.map((skill) => ({ + name: skill.name, + description: descriptionOf(renderPluginSkill(skill.name)), + triggers: skill.routing.triggers, + })), + ...[ + ["hack-init", renderHackInitSkill()], + ["hack-install", renderHackInstallSkill()], + ["hack-remote", renderHackRemoteSkill()], + ].map(([name, rendered]) => ({ + name: name ?? "", + description: descriptionOf(rendered ?? ""), + triggers: [], + })), + ]; +} + +/** + * Closed overlap findings: a trigger phrase owned by one skill that also + * appears (case-insensitively) in another skill's description. + */ +function triggerOverlaps(skills: readonly RoutedSkill[]): string[] { + const findings: string[] = []; + for (const owner of skills) { + for (const trigger of owner.triggers) { + for (const other of skills) { + if ( + other !== owner && + other.description.toLowerCase().includes(trigger.toLowerCase()) + ) { + findings.push(`${owner.name} -> ${other.name}`); + } + } + } + } + return findings; +} + +/** Codes listed in the first column of the reference table, in order. */ +function referenceCodes(rendered: string): string[] { + return [...rendered.matchAll(/^\| `(E_[A-Z0-9_]+)` \|/gm)].map( + (match) => match[1] ?? "" + ); +} + +/** Parity between the reference rows and the exported registry. */ +function errorCodeParityIssues(opts: { + readonly rendered: string; + readonly registry: readonly string[]; +}): string[] { + const listed = referenceCodes(opts.rendered); + return [ + ...opts.registry + .filter((code) => !listed.includes(code)) + .map((code) => `missing ${code}`), + ...listed + .filter((code) => !opts.registry.includes(code)) + .map((code) => `unexpected ${code}`), + ...listed + .filter((code, index) => listed.indexOf(code) !== index) + .map((code) => `duplicate ${code}`), + ]; +} + +test("every focused skill section id exists in the canonical source", () => { + const ids = new Set(INSTRUCTION_SECTIONS.map((section) => section.id)); + expect(FOCUSED_SKILLS.map((skill) => skill.name)).toEqual([ + "hack-branches", + "hack-env", + "hack-diagnose", + ]); + for (const skill of FOCUSED_SKILLS) { + expect(skill.sections.length, skill.name).toBeGreaterThan(0); + for (const entry of skill.sections) { + expect(ids.has(entry.id), `${skill.name}: ${entry.id}`).toBe(true); + } + } +}); + +test("focused skills render canonical sections verbatim, in order", () => { + for (const skill of FOCUSED_SKILLS) { + const body = bodyOf(skill.name); + let previous = -1; + for (const entry of skill.sections) { + const section = sectionById(entry.id); + const title = entry.optional + ? `## ${section.title} (optional)` + : `## ${section.title}`; + const at = body.indexOf(title); + expect(at, `${skill.name}: ${entry.id} heading order`).toBeGreaterThan( + previous + ); + for (const bullet of section.bullets) { + expect(body, `${skill.name}: ${entry.id}`).toContain(`- ${bullet}`); + } + previous = at; + } + // Only canonical headings appear: no skill-local prose sections. + const headings = [...body.matchAll(/^## (.+)$/gm)].map( + (match) => match[1] ?? "" + ); + expect(headings.filter((heading) => heading !== "References")).toEqual( + skill.sections.map((entry) => { + const title = sectionById(entry.id).title; + return entry.optional ? `${title} (optional)` : title; + }) + ); + } +}); + +test("negative control: an unknown or repeated section id fails composition", () => { + expect(() => + composeCanonicalSections({ sections: [{ id: "no-such-section" }] }) + ).toThrow("Unknown instruction section id in focused skill"); + expect(() => + composeCanonicalSections({ + sections: [{ id: "env-composition" }, { id: "env-composition" }], + }) + ).toThrow("Duplicate instruction section id in focused skill"); + // The refusal is closed: it never echoes the caller's id. + let message = ""; + try { + composeCanonicalSections({ sections: [{ id: "secret-looking-id" }] }); + } catch (error) { + message = error instanceof Error ? error.message : String(error); + } + expect(message).toBe("Unknown instruction section id in focused skill"); +}); + +test("focused-only canonical sections stay out of the broad surfaces", () => { + for (const id of [ + "branch-aliases", + "env-commands", + "env-safety", + "diagnose", + ]) { + expect(sectionById(id).surfaces, id).toEqual([]); + } +}); + +test("hack-branches leads with --branch and the branch subcommands; worktrees are optional", () => { + const spec = focusedSkill("hack-branches"); + expect(spec.sections[0]?.id).toBe("branch-instances"); + expect(spec.routing.summary).toContain("`--branch `"); + const body = bodyOf("hack-branches"); + const first = extractHackInvocations(body)[0] ?? ""; + expect(first).toContain("--branch "); + for (const subcommand of ["add", "list", "open", "remove"]) { + expect(body).toContain(`\`hack branch ${subcommand}`); + } + const worktrees = spec.sections.find((entry) => entry.id === "worktrees"); + expect(worktrees?.optional).toBe(true); + expect(spec.sections.filter((entry) => entry.optional)).toEqual([ + { id: "worktrees", optional: true }, + ]); + expect(body.indexOf("## Linked git worktrees (optional)")).toBeGreaterThan( + body.indexOf("## Branch aliases") + ); +}); + +test("hack-env covers its commands and states precedence only from the canonical env section", () => { + const spec = focusedSkill("hack-env"); + expect(spec.sections[0]?.id).toBe("env-composition"); + const body = bodyOf("hack-env"); + for (const command of [ + "`hack env list", + "`hack env get", + "`hack env explain", + "`hack env apply", + "`hack env set --local`", + "`hack env set --secret`", + "`hack env exec", + ".hack/hack.env..yaml", + ".hack/hack.env.local.yaml", + ]) { + expect(body, command).toContain(command); + } + // Precedence wording lives in `env-composition` alone; skill-specific + // sections must not restate (and so cannot contradict) it. + const precedence = + /\b(?:layer order|precedence|override[sn]?\s+`global`|values last)\b/i; + for (const entry of spec.sections) { + if (entry.id === "env-composition") { + continue; + } + for (const bullet of sectionById(entry.id).bullets) { + expect(precedence.test(bullet), `${entry.id}: ${bullet}`).toBe(false); + } + } + expect( + sectionById("env-composition").bullets.some((bullet) => + precedence.test(bullet) + ) + ).toBe(true); +}); + +test("hack-env keeps secret values out of argv and treats get as a plaintext sink", () => { + const body = bodyOf("hack-env"); + expect(body).toContain( + "A secret value never goes in command arguments or agent prompts" + ); + expect(body).toContain("are plaintext sinks"); + expect(body).toContain("`hack env explain "); + // No rendered example passes a secret-looking value as an argument or + // assignment. + for (const pattern of [ + /--(?:password|passwd|token|secret|api-key|apikey)(?:=|\s+)(?!-)[^\s`]/i, + /\b[A-Z][A-Z0-9_]*(?:TOKEN|SECRET|PASSWORD|PASSWD|API_KEY)=\S/, + /`hack env (?:set|add) --secret/, + ]) { + expect(pattern.test(body), String(pattern)).toBe(false); + } +}); + +test("hack-diagnose reads first in the documented order and repairs only within the request", () => { + const spec = focusedSkill("hack-diagnose"); + expect(spec.sections[0]?.id).toBe("diagnose"); + const body = bodyOf("hack-diagnose"); + const order = [ + "`hack ps --json`", + "`hack doctor --json`", + "`hack logs --json --no-follow --tail 200`", + "`hack env explain --service `", + "`hack global status`", + ].map((step) => body.indexOf(step)); + expect(order.every((index) => index >= 0)).toBe(true); + expect([...order].sort((a, b) => a - b)).toEqual(order); + expect(body).toContain( + "apply `hack doctor --fix` only for repairs covered by the request" + ); + // Every `--fix` mention comes after the read-first steps. + expect(body.indexOf("--fix")).toBeGreaterThan(order.at(-1) ?? 0); + expect(body).toContain("](references/error-codes.md)"); +}); + +test("every hack command and flag in the focused skills resolves in CLI_SPEC", () => { + const rendered: Record = { + "hack-cli": renderPluginSkill("hack-cli"), + "error-codes": renderErrorCodeReference(), + }; + for (const skill of FOCUSED_SKILLS) { + rendered[skill.name] = renderPluginSkill(skill.name); + } + for (const [name, text] of Object.entries(rendered)) { + expect(extractHackInvocations(text).length, name).toBeGreaterThan(0); + expect( + findGuidanceCommandIssues({ cli: CLI_SPEC, rendered: text }) + ).toEqual([]); + } + // The walker still rejects a bad branch or env invocation. + for (const fixture of [ + "`hack branch rename `", + "`hack env set --shared`", + "`hack logs --json --no-follow --last 200`", + ]) { + expect( + findGuidanceCommandIssues({ cli: CLI_SPEC, rendered: fixture }), + fixture + ).not.toEqual([]); + } +}); + +test("the error-code reference lists exactly the exported registry", async () => { + const rendered = renderErrorCodeReference(); + expect(HACK_ERROR_CODES.length).toBeGreaterThan(0); + expect(referenceCodes(rendered)).toEqual([...HACK_ERROR_CODES]); + expect( + errorCodeParityIssues({ rendered, registry: HACK_ERROR_CODES }) + ).toEqual([]); + const packaged = await Bun.file( + join(pluginRoot, "skills/hack-diagnose/references/error-codes.md") + ).text(); + expect(packaged.startsWith(rendered.trimEnd())).toBe(true); + expect(referenceCodes(packaged)).toEqual([...HACK_ERROR_CODES]); +}); + +test("negative control: a fake or missing E_* code fails registry parity", () => { + const rendered = renderErrorCodeReference(); + expect( + errorCodeParityIssues({ + rendered: `${rendered}| \`E_FAKE_CODE\` | Not a real code. | \`hack doctor --json\` |\n`, + registry: HACK_ERROR_CODES, + }) + ).toEqual(["unexpected E_FAKE_CODE"]); + expect( + errorCodeParityIssues({ + rendered: rendered.replace(/^\| `E_USAGE` \|.*\n/m, ""), + registry: HACK_ERROR_CODES, + }) + ).toEqual(["missing E_USAGE"]); + expect( + errorCodeParityIssues({ + rendered, + registry: [...HACK_ERROR_CODES, "E_NOT_RENDERED"], + }) + ).toEqual(["missing E_NOT_RENDERED"]); +}); + +test("rendered descriptions are the routing sentences and parse as YAML", () => { + expect(descriptionOf(renderPluginSkill("hack-cli"))).toBe( + renderSkillDescription(HACK_CLI_PLUGIN_ROUTING) + ); + for (const skill of FOCUSED_SKILLS) { + expect(descriptionOf(renderPluginSkill(skill.name))).toBe( + renderSkillDescription(skill.routing) + ); + } + const cli = descriptionOf(renderPluginSkill("hack-cli")); + for (const name of ["hack-branches", "hack-env", "hack-diagnose"]) { + expect(cli).toContain(name); + } +}); + +test("no trigger phrase is shared between plugin skill descriptions", () => { + const skills = pluginRouting(); + expect(skills.map((skill) => skill.name)).toHaveLength(7); + const owned = skills.flatMap((skill) => skill.triggers); + expect(owned.length).toBeGreaterThan(0); + expect(new Set(owned.map((trigger) => trigger.toLowerCase())).size).toBe( + owned.length + ); + expect(triggerOverlaps(skills)).toEqual([]); +}); + +test("negative control: a duplicated trigger phrase fails the overlap test", () => { + const skills = pluginRouting(); + const env = skills.find((skill) => skill.name === "hack-env"); + const borrowed = env?.triggers[0]; + if (!borrowed) { + throw new Error("hack-env has no trigger phrases"); + } + const withDuplicate = skills.map((skill) => + skill.name === "hack-diagnose" + ? { + ...skill, + description: renderSkillDescription({ + ...focusedSkill("hack-diagnose").routing, + triggers: [ + ...focusedSkill("hack-diagnose").routing.triggers, + borrowed, + ], + }), + } + : skill + ); + expect(triggerOverlaps(withDuplicate)).toEqual(["hack-env -> hack-diagnose"]); +}); diff --git a/tests/skill-references.test.ts b/tests/skill-references.test.ts index 35622f3f..951fc459 100644 --- a/tests/skill-references.test.ts +++ b/tests/skill-references.test.ts @@ -63,8 +63,12 @@ test("every packaged reference is linked and carries a recomputable revision", a test("only the plugin skill links references; standalone renders stay unchanged", () => { expect(renderCodexSkill()).not.toContain("references/"); const plugin = renderPluginSkill("hack-cli"); - expect(plugin.startsWith(renderCodexSkill().trimEnd())).toBe(true); - expect(plugin.slice(renderCodexSkill().trimEnd().length)).toBe( + // The plugin variant differs only in its routing description and the + // appended references; the instruction body is the standalone body. + const body = (text: string) => text.slice(text.indexOf("\n---\n") + 5); + const standaloneBody = body(renderCodexSkill()).trimEnd(); + expect(body(plugin).startsWith(standaloneBody)).toBe(true); + expect(body(plugin).slice(standaloneBody.length)).toBe( "\n\n## References\n\n- [Pattern catalogue](references/patterns.md): reusable project patterns with what Hack supports today, the missing contract and its owner.\n" ); const unchanged = "---\nname: hack-init\n---\n"; From 42e7b3b7b31026a22058d6b6fbaa211c470ac430 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:41:54 -0400 Subject: [PATCH 33/52] fix(mcp): separate log cutoffs from backend failures hack.project.logs.tail and hack.project.logs.snapshot now report a server-side bound as a successful cutoff with data.cutoff ({bound: events|bytes|time, limit}) and a backend failure as the closed backend_error (E_UNEXPECTED, fixed value-free message). maxEvents counts type:"log" events only. The 8 MiB byte cutoff keeps the partial events and cuts captured output on a UTF-8 code point boundary. Failure evidence after a server-initiated stop does not turn a cutoff into an error. Adds no-replay regression tests for consecutive calls, cancellation and disconnect/reconnect against a scripted stub backend, and a decision doc (docs/reference/mcp-logs.md) recording no-go on handles, cursors and idempotency keys. Refs HACK-1134 Co-Authored-By: Claude Opus 5.5 --- docs/reference.md | 1 + docs/reference/mcp-logs.md | 96 +++++ src/mcp/logs-contract.ts | 136 ++++++ src/mcp/output-budget.ts | 113 ++++- src/mcp/server.ts | 196 +++++---- tests/mcp-lifetime.test.ts | 27 +- tests/mcp-logs-contract.test.ts | 712 ++++++++++++++++++++++++++++++++ 7 files changed, 1199 insertions(+), 82 deletions(-) create mode 100644 docs/reference/mcp-logs.md create mode 100644 src/mcp/logs-contract.ts create mode 100644 tests/mcp-logs-contract.test.ts diff --git a/docs/reference.md b/docs/reference.md index 99645b04..64c7e50a 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -27,3 +27,4 @@ - [Extensions](./extensions.md) - [Creating an extension](./guides/create-extension.md) - [Integrations](./integrations.md) +- [MCP logs contract](./reference/mcp-logs.md) — cutoff versus backend failure for the MCP log tools, and the cursor/handle decision diff --git a/docs/reference/mcp-logs.md b/docs/reference/mcp-logs.md new file mode 100644 index 00000000..e56d1353 --- /dev/null +++ b/docs/reference/mcp-logs.md @@ -0,0 +1,96 @@ +# MCP logs contract + +This page covers the `hack.project.logs.tail` and `hack.project.logs.snapshot` +MCP tools. Both tools run `hack logs --json` once per call and return the stream +events they collected. When a shell is available, use `hack logs` directly. + +## Outcomes + +Each call ends in exactly one outcome. + +| Outcome | `ok` | `isError` | `data.cutoff` | `error` | +| --- | --- | --- | --- | --- | +| Complete: the backend ended the stream and exited 0 | `true` | absent | `null` | absent | +| Cutoff: the server stopped at one of its bounds | `true` | absent | set | absent | +| Backend failure | `false` | `true` | `null` | `backend_error` | + +A cutoff is a success. The collected events are valid. The stream simply +continued past the bound. `data.cutoff` is `{ "bound": ..., "limit": ... }`: + +- `events`: `maxEvents` log events were collected (tail only). Only + `type: "log"` events count. `start`, `heartbeat`, `error` and `end` events + are returned in `data.events` but do not use up the limit. +- `time`: the collection window ended. For tail this is `maxMs`. For snapshot + it is the fixed 120 second command timeout. +- `bytes`: stdout plus stderr reached the 8 MiB capture limit. + `outputTruncated` is `true`, and `data` still carries every event parsed + before the cut. The cut never splits a UTF-8 code point. A trailing partial + line can be present in `stdout`, but it is never parsed as an event. + +After a cutoff, the server stopped the child process, so `exitCode` is often +non-zero. Use `ok` and `data.cutoff` to read the result, not `exitCode`. +`data.stop_reason` (tail only) keeps its earlier values: `eof`, `max_events`, +`timeout`, `output_limit`, `cancelled` or `interrupted`. + +A backend failure is not a truncated success. It is reported when the stream +contains an `error` event or an `end` event with a reason other than `eof` or +`closed`, or when the child exits non-zero at the end of the stream. The error +is closed: `code` is `E_UNEXPECTED`, `reason` is `backend_error`, and `message` +is fixed text that never repeats backend output, caller input or the query. +Raw `stdout` and `stderr` are still returned, as they are for other tools. + +Failure evidence counts only before the server stops the child. A child that +writes `error` or `end` events, or exits non-zero, in response to the server's +own stop signal still produces a cutoff. + +## Reconnects and repeated calls + +The server keeps no log state between calls. Every call starts a new backend +stream: + +- A repeated call does not replay events from an earlier call, and it does not + continue after the earlier call's last event. +- A cancelled request, or a session that closes mid-call, stops the backend + process group. The interrupted call's events are not delivered later, to this + session or to a reconnected one. + +To continue a stream approximately, call again with `since` set to the last +event's `ts`. `since` selects the Loki backend and cannot be combined with +`compose`. Expect overlap at the boundary and remove duplicates on the client. + +## Decision: no handles, cursors or idempotency keys (no-go) + +Status: no-go for now (HACK-1134). This decision adds no cursor implementation. + +We considered three additions: + +- A stream handle, so a later call can attach to a running tail. +- A resume cursor, so a later call can continue after the last returned event. +- An idempotency key, so a retried call returns the same result. + +Reasons for no-go: + +1. **The backends have no exact cursor.** Hack's compose path accepts only + `tail`, and `docker compose logs` itself offers only timestamp filters that + can repeat or skip lines with the same timestamp. The lifecycle log + companion is a file tail. Loki selects by time range, and its timestamps + are not unique. A cursor would promise gapless, duplicate-free resume that + Hack cannot deliver, least of all for the default compose backend. +2. **A handle needs state that does not survive the common reconnect.** A + stdio MCP client reconnects by starting a new `hack mcp serve` process, so + in-memory handles are lost. Keeping them on disk under `HACK_HOME` would add + retained log data, plus ownership, expiry and cleanup rules. Logs can hold + secrets, so that retention is a risk. +3. **Idempotency adds nothing to a read.** Both tools are read-only and + annotated `readOnlyHint`. A retry is already safe. It returns current data + rather than a replay, and that is the documented behavior above. +4. **Bounded calls already cover the need.** Agents call tail with a small + window, read `data.cutoff`, and call again with `since` when they need more. + +Revisit this decision (go) only when all of these are true: + +- A backend exposes a stable, monotonic position, such as a daemon-owned log + index. +- Hack has a daemon-owned buffer with defined retention, ownership and secret + handling. +- A real agent workflow shows that `since` overlap is not good enough. diff --git a/src/mcp/logs-contract.ts b/src/mcp/logs-contract.ts new file mode 100644 index 00000000..2902f09f --- /dev/null +++ b/src/mcp/logs-contract.ts @@ -0,0 +1,136 @@ +/** + * Result contract for the MCP `hack.project.logs.tail` and + * `hack.project.logs.snapshot` tools. + * + * A collection ends in exactly one outcome: + * - `complete`: the backend reached end of stream and exited 0. + * - `cutoff`: the MCP server stopped collection at one of its own bounds. This + * is a success; `data.cutoff` names the bound and its effective limit. + * - `backend_error`: the log backend reported failure (a stream `error` event, + * a failing `end` event, or a non-zero exit the server did not cause). This + * is a closed error, never a truncated success. + * - `cancelled`: the request was aborted or the session closed; nothing is + * kept for a later request (see docs/reference/mcp-logs.md). + * + * Failure evidence counts only while collection is still running. Once the + * server stops the command for a bound, a later `error`/`end` event or a + * signal exit status is a consequence of that stop, not a backend failure. + */ + +/** Why collection stopped, as reported in `data.stop_reason`. */ +export type McpLogStopReason = + | "eof" + | "max_events" + | "timeout" + | "output_limit" + | "cancelled" + | "interrupted"; + +/** The bound that ended a successful, incomplete collection. */ +export type McpLogCutoff = { + /** `events` counts `type: "log"` events only; `bytes` is the shared stdout + * plus stderr capture budget; `time` is the collection window. */ + readonly bound: "events" | "bytes" | "time"; + readonly limit: number; +}; + +export type McpLogOutcome = + | { readonly kind: "complete" } + | { readonly kind: "cutoff"; readonly cutoff: McpLogCutoff } + | { readonly kind: "backend_error" } + | { readonly kind: "cancelled" }; + +/** Closed post-spawn failure. The message is fixed and never repeats backend + * output, caller input or query text. */ +export const MCP_LOG_BACKEND_ERROR = Object.freeze({ + code: "E_UNEXPECTED", + reason: "backend_error", + message: "Log backend failed. Run 'hack logs' for details.", +} as const); + +/** End reasons written by the CLI's JSON stream for a normal end of stream. */ +const CLEAN_END_REASONS: ReadonlySet = new Set(["eof", "closed"]); + +/** Classify one parsed `hack logs --json` stream event. */ +export function observeLogStreamEvent(event: Record): { + readonly isLog: boolean; + readonly backendFailure: boolean; +} { + const type = event.type; + if (type === "log") { + return { isLog: true, backendFailure: false }; + } + if (type === "error") { + return { isLog: false, backendFailure: true }; + } + if (type === "end") { + const reason = event.reason; + return { + isLog: false, + backendFailure: + typeof reason === "string" && !CLEAN_END_REASONS.has(reason), + }; + } + return { isLog: false, backendFailure: false }; +} + +/** + * Decide the outcome of a finished collection. Backend failure evidence wins + * over a cutoff; a non-zero exit after a server-initiated stop is ignored. + */ +export function classifyLogCollection(opts: { + readonly stopReason: McpLogStopReason; + readonly exitCode: number; + readonly backendFailureObserved: boolean; + readonly limits: { + readonly maxEvents?: number; + readonly timeMs: number; + readonly bytes: number; + }; +}): McpLogOutcome { + if (opts.backendFailureObserved) { + return { kind: "backend_error" }; + } + switch (opts.stopReason) { + case "cancelled": + case "interrupted": + return { kind: "cancelled" }; + case "max_events": + return { + kind: "cutoff", + cutoff: { bound: "events", limit: opts.limits.maxEvents ?? 0 }, + }; + case "timeout": + return { + kind: "cutoff", + cutoff: { bound: "time", limit: opts.limits.timeMs }, + }; + case "output_limit": + return { + kind: "cutoff", + cutoff: { bound: "bytes", limit: opts.limits.bytes }, + }; + case "eof": + return opts.exitCode === 0 + ? { kind: "complete" } + : { kind: "backend_error" }; + default: { + const unreachable: never = opts.stopReason; + return unreachable; + } + } +} + +/** Narrow a lifetime stop reason to the closed set the logs contract reports. */ +export function toLogStopReason(reason: string): McpLogStopReason { + switch (reason) { + case "max_events": + case "timeout": + case "output_limit": + case "cancelled": + case "interrupted": + return reason; + default: + return "interrupted"; + } +} diff --git a/src/mcp/output-budget.ts b/src/mcp/output-budget.ts index 2c706f7f..61bab29e 100644 --- a/src/mcp/output-budget.ts +++ b/src/mcp/output-budget.ts @@ -3,10 +3,18 @@ export const MCP_OUTPUT_LIMIT_BYTES = 8 * 1024 * 1024; /** Share one byte budget across command output streams, before decoding or line * buffering. Over-budget data is drained without retention while the owner stops * the command. Exact-limit output is valid; an additional byte triggers the limit. + * + * With `codePointSafe`, a stream never ends on a cut through a UTF-8 code point: + * an incomplete trailing sequence is held until its next chunk arrives, and the + * limit cut backs off to the preceding code point boundary. Bytes are charged + * to the budget when they arrive, so the limit decision is unchanged; only the + * retained prefix can be up to three bytes shorter. Source bytes that are + * themselves invalid UTF-8 are passed through untouched at stream end. */ export function createMcpOutputBudget(opts: { readonly onLimit: () => void; readonly maxBytes?: number; + readonly codePointSafe?: boolean; }): { readonly exceeded: () => boolean; readonly wrap: ( @@ -18,28 +26,40 @@ export function createMcpOutputBudget(opts: { throw new RangeError("MCP output budget must be a positive integer"); } let exceeded = false; + /** Charge a chunk to the shared budget; returns the accepted byte count. */ + const charge = (size: number): number => { + const accepted = Math.min(remaining, size); + remaining -= accepted; + if (accepted < size && !exceeded) { + exceeded = true; + opts.onLimit(); + } + return accepted; + }; return { exceeded: () => exceeded, wrap: (stream) => { const reader = stream.getReader(); + const retain = + opts.codePointSafe === true + ? createCodePointRetainer() + : BYTE_EXACT_RETAINER; return new ReadableStream({ async pull(controller) { while (true) { const { done, value } = await reader.read(); if (done) { reader.releaseLock(); + const rest = retain.flush(); + if (rest.byteLength > 0) { + controller.enqueue(rest); + } controller.close(); return; } - const accepted = Math.min(remaining, value.byteLength); - remaining -= accepted; - if (accepted < value.byteLength && !exceeded) { - exceeded = true; - opts.onLimit(); - } - if (accepted > 0) { - // Do not keep a reference to a larger discarded backing buffer. - controller.enqueue(value.slice(0, accepted)); + const kept = retain.take(value, charge(value.byteLength)); + if (kept.byteLength > 0) { + controller.enqueue(kept); return; } } @@ -52,3 +72,78 @@ export function createMcpOutputBudget(opts: { }, }; } + +/** Decides which accepted bytes of one stream are emitted now. */ +type Retainer = { + /** `accepted` is the prefix of `chunk` charged to the budget. */ + readonly take: (chunk: Uint8Array, accepted: number) => Uint8Array; + readonly flush: () => Uint8Array; +}; + +const EMPTY = new Uint8Array(0); + +const BYTE_EXACT_RETAINER: Retainer = { + // Copy so a larger discarded backing buffer is not kept alive. + take: (chunk, accepted) => (accepted > 0 ? chunk.slice(0, accepted) : EMPTY), + flush: () => EMPTY, +}; + +function createCodePointRetainer(): Retainer { + /** Accepted bytes of an incomplete trailing code point. */ + let held = EMPTY; + return { + take: (chunk, accepted) => { + const candidate = concatBytes(held, chunk.subarray(0, accepted)); + const keep = candidate.byteLength - incompleteUtf8TailLength(candidate); + // After a cut this stream retains nothing more, so a partial code point + // is dropped instead of being held for a continuation. + held = accepted < chunk.byteLength ? EMPTY : candidate.slice(keep); + return keep > 0 ? candidate.slice(0, keep) : EMPTY; + }, + flush: () => { + const rest = held; + held = EMPTY; + return rest; + }, + }; +} + +/** + * Number of trailing bytes that start a UTF-8 sequence the buffer does not + * complete (0-3). An invalid lead byte or an over-long continuation run counts + * as complete, so malformed input is never held back indefinitely. + */ +export function incompleteUtf8TailLength(bytes: Uint8Array): number { + const end = bytes.byteLength; + for (let back = 1; back <= Math.min(4, end); back += 1) { + const byte = bytes[end - back] ?? 0; + if ((byte & 0xc0) === 0x80) { + continue; + } + return back < utf8SequenceLength(byte) ? back : 0; + } + return 0; +} + +function utf8SequenceLength(lead: number): number { + if ((lead & 0xe0) === 0xc0) { + return 2; + } + if ((lead & 0xf0) === 0xe0) { + return 3; + } + if ((lead & 0xf8) === 0xf0) { + return 4; + } + return 1; +} + +function concatBytes(head: Uint8Array, tail: Uint8Array): Uint8Array { + if (head.byteLength === 0) { + return tail; + } + const joined = new Uint8Array(head.byteLength + tail.byteLength); + joined.set(head, 0); + joined.set(tail, head.byteLength); + return joined; +} diff --git a/src/mcp/server.ts b/src/mcp/server.ts index b3dcefcc..317233cc 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -34,7 +34,19 @@ import { import { readLinesFromStream } from "../ui/lines.ts"; import type { McpCommandAdmission } from "./command-admission.ts"; import { superviseMcpCommand } from "./command-lifetime.ts"; -import { createMcpOutputBudget } from "./output-budget.ts"; +import { + classifyLogCollection, + MCP_LOG_BACKEND_ERROR, + type McpLogCutoff, + type McpLogOutcome, + type McpLogStopReason, + observeLogStreamEvent, + toLogStopReason, +} from "./logs-contract.ts"; +import { + createMcpOutputBudget, + MCP_OUTPUT_LIMIT_BYTES, +} from "./output-budget.ts"; import { captureMcpSessionContext, type McpSessionContext, @@ -136,11 +148,13 @@ type ToolResultPayload = HackCommandResult & { readonly data?: unknown; /** Present (and false) only on refusals returned before any child spawned. */ readonly spawned?: false; - readonly error?: { - readonly code: McpRefusalCode; - readonly reason: McpRefusalReason; - readonly message: string; - }; + readonly error?: + | { + readonly code: McpRefusalCode; + readonly reason: McpRefusalReason; + readonly message: string; + } + | typeof MCP_LOG_BACKEND_ERROR; }; type ProjectArgsResolution = @@ -205,8 +219,8 @@ const toolOutputSchema = { spawned: z.literal(false).optional(), error: z .object({ - code: z.enum(["E_USAGE", "E_PROJECT_NOT_FOUND"]), - reason: z.enum(MCP_REFUSAL_REASONS), + code: z.enum(["E_USAGE", "E_PROJECT_NOT_FOUND", "E_UNEXPECTED"]), + reason: z.enum([...MCP_REFUSAL_REASONS, MCP_LOG_BACKEND_ERROR.reason]), message: z.string(), }) .optional(), @@ -700,18 +714,20 @@ function registerTools(opts: { ...buildLogSelectionArgs(input), ]; - const result = await runHackCommand({ + const result = await runHackLogCollect({ context: opts.context, signal: extra.signal, tool: "hack.project.logs.snapshot", args, + timeoutMs: DEFAULT_CMD_TIMEOUT_MS, }); - return buildToolResult({ + return buildLogToolResult({ result, data: { - events: parseJsonLines(result.stdout), - count: countJsonLines(result.stdout), + events: result.events, + count: result.events.length, + cutoff: cutoffOf(result.outcome), limits: { tail: input.tail ?? null }, }, }); @@ -758,26 +774,21 @@ function registerTools(opts: { const maxEvents = input.maxEvents ?? DEFAULT_LOG_TAIL_EVENTS; const maxMs = input.maxMs ?? DEFAULT_LOG_TAIL_MS; - const result = await runHackLogTail({ + const result = await runHackLogCollect({ context: opts.context, signal: extra.signal, tool: "hack.project.logs.tail", args, maxEvents, - maxMs, + timeoutMs: maxMs, }); - return buildToolResult({ - result: { - ...(result.outputTruncated ? { outputTruncated: true } : {}), - command: result.command, - exitCode: result.exitCode, - stdout: result.stdout, - stderr: result.stderr, - }, + return buildLogToolResult({ + result, data: { events: result.events, count: result.events.length, + cutoff: cutoffOf(result.outcome), stop_reason: result.stopReason, duration_ms: result.durationMs, limits: { tail: input.tail ?? null, maxEvents, maxMs }, @@ -1057,10 +1068,48 @@ function buildToolResult(opts: { }; } +/** + * Logs tools result. Unlike {@link buildToolResult}, success follows the + * collection outcome rather than the child's exit status: a cutoff is `ok` + * with `data.cutoff` set (the server stopped the child, so `exitCode` may be + * non-zero), and `data` is kept after the byte bound so partial events + * survive. A backend failure is `ok: false` with the closed `backend_error`. + */ +function buildLogToolResult(opts: { + readonly result: HackLogCollection; + readonly data: Record; +}): CallToolResult { + const { result } = opts; + const ok = + result.outcome.kind === "complete" || result.outcome.kind === "cutoff"; + const payload: ToolResultPayload = { + ok, + command: result.command, + exitCode: result.exitCode, + stdout: result.stdout, + stderr: result.stderr, + ...(result.outputTruncated ? { outputTruncated: true } : {}), + data: opts.data, + ...(result.outcome.kind === "backend_error" + ? { error: MCP_LOG_BACKEND_ERROR } + : {}), + }; + + return { + content: [{ type: "text", text: JSON.stringify(payload, null, 2) }], + structuredContent: payload, + ...(ok ? {} : { isError: true }), + }; +} + +function cutoffOf(outcome: McpLogOutcome): McpLogCutoff | null { + return outcome.kind === "cutoff" ? outcome.cutoff : null; +} + /** * Typed refusal for a request rejected before any child process exists. * `spawned: false` is a guarantee: every caller returns before - * `runHackCommand`/`runHackLogTail`, so no argv, audit line or mutation occurs. + * `runHackCommand`/`runHackLogCollect`, so no argv, audit line or mutation occurs. * `stderr` repeats the message for clients reading the earlier contract. * The message is the reason's fixed text from {@link MCP_REFUSALS}; only * `invalid_arguments` may pass its schema-key-only summary instead. @@ -1360,24 +1409,34 @@ async function runHackCommand(opts: { : await work(); } -async function runHackLogTail(opts: { - readonly context: McpSessionContext; - readonly signal: AbortSignal; - readonly tool: string; - readonly args: readonly string[]; - readonly maxEvents: number; - readonly maxMs: number; -}): Promise<{ +type HackLogCollection = { readonly outputTruncated?: boolean; readonly command: string; readonly exitCode: number; readonly stdout: string; readonly stderr: string; readonly events: readonly Record[]; - readonly stopReason: string; + readonly stopReason: McpLogStopReason; readonly durationMs: number; -}> { - const work = async () => { + readonly outcome: McpLogOutcome; +}; + +/** + * Run `hack logs --json` and collect its stream events under the contract in + * `logs-contract.ts`. `maxEvents` counts `type: "log"` events only; `timeoutMs` + * bounds the whole collection; the shared byte budget cuts on a UTF-8 code + * point boundary. Nothing outlives the call: a later call, in this session or + * a reconnected one, starts a new backend stream (no cursor, no replay). + */ +async function runHackLogCollect(opts: { + readonly context: McpSessionContext; + readonly signal: AbortSignal; + readonly tool: string; + readonly args: readonly string[]; + readonly maxEvents?: number; + readonly timeoutMs: number; +}): Promise { + const work = async (): Promise => { const invocation = await resolveHackInvocation(opts.context); opts.signal.throwIfAborted(); const cmd = [invocation.bin, ...invocation.args, ...opts.args]; @@ -1390,39 +1449,50 @@ async function runHackLogTail(opts: { stderr: "pipe", }); - const { maxEvents, maxMs } = opts; + const { maxEvents } = opts; const start = Date.now(); const events: Record[] = []; const stdoutLines: string[] = []; const stderrLines: string[] = []; - let stopReason = "eof"; + let stopReason: McpLogStopReason = "eof"; + let stopped = false; + let logCount = 0; + let backendFailureObserved = false; const lifetime = superviseMcpCommand({ proc, signal: opts.signal, - timeoutMs: maxMs, + timeoutMs: opts.timeoutMs, onStop: (reason) => { - stopReason = reason; + stopped = true; + stopReason = toLogStopReason(reason); }, }); const output = createMcpOutputBudget({ + codePointSafe: true, onLimit: () => lifetime.stop("output_limit"), }); + /** Record one stream event; true once the log-event bound is reached. */ + const recordEvent = (event: Record): boolean => { + events.push(event); + const observed = observeLogStreamEvent(event); + // Events after a server-initiated stop describe that stop. + if (observed.backendFailure && !stopped) { + backendFailureObserved = true; + } + if (observed.isLog) { + logCount += 1; + } + return maxEvents !== undefined && logCount >= maxEvents; + }; const stdoutTask = (async () => { for await (const line of readLinesFromStream(output.wrap(proc.stdout))) { stdoutLines.push(line); - const trimmed = line.trim(); - if (trimmed.length === 0) { - continue; - } - const parsed = parseJsonLine(trimmed); - if (parsed) { - events.push(parsed); - } - if (events.length >= maxEvents) { + const parsed = parseJsonLine(line.trim()); + if (parsed && recordEvent(parsed)) { lifetime.stop("max_events"); break; } @@ -1443,7 +1513,6 @@ async function runHackLogTail(opts: { } if (output.exceeded()) { - exitCode = 1; stderrLines.push( "MCP output exceeded the 8 MiB capture limit; use the CLI for full output." ); @@ -1465,6 +1534,16 @@ async function runHackLogTail(opts: { ...(output.exceeded() ? { outputTruncated: true } : {}), stopReason, durationMs: Date.now() - start, + outcome: classifyLogCollection({ + stopReason, + exitCode, + backendFailureObserved, + limits: { + ...(maxEvents === undefined ? {} : { maxEvents }), + timeMs: opts.timeoutMs, + bytes: MCP_OUTPUT_LIMIT_BYTES, + }, + }), }; }; return opts.context.admission @@ -1491,17 +1570,6 @@ function parseJson(text: string): Record | null { } } -function parseJsonLines(text: string): Record[] { - const events: Record[] = []; - for (const line of text.split("\n")) { - const parsed = parseJsonLine(line.trim()); - if (parsed) { - events.push(parsed); - } - } - return events; -} - function parseJsonLine(line: string): Record | null { if (line.length === 0) { return null; @@ -1514,16 +1582,6 @@ function parseJsonLine(line: string): Record | null { } } -function countJsonLines(text: string): number { - let count = 0; - for (const line of text.split("\n")) { - if (parseJsonLine(line.trim())) { - count += 1; - } - } - return count; -} - function joinLines(lines: readonly string[]): string { if (lines.length === 0) { return ""; diff --git a/tests/mcp-lifetime.test.ts b/tests/mcp-lifetime.test.ts index 8109ff3c..4a7787f6 100644 --- a/tests/mcp-lifetime.test.ts +++ b/tests/mcp-lifetime.test.ts @@ -197,7 +197,15 @@ test("MCP log-tail timeout stops a command that ignores SIGTERM and returns a re }); await waitFor(() => messages.some((item) => item.id === 2)); expect(messages.find((item) => item.id === 2)).toMatchObject({ - result: { structuredContent: { data: { stop_reason: "timeout" } } }, + result: { + structuredContent: { + ok: true, + data: { + stop_reason: "timeout", + cutoff: { bound: "time", limit: 1000 }, + }, + }, + }, }); expect(alive(fixture.childPid ?? 0)).toBe(false); expect(fixture.proc.exitCode).toBeNull(); @@ -217,10 +225,21 @@ for (const tool of ["hack.projects.list", "hack.project.logs.tail"]) { const { fixture, send, messages } = await startFixture({ tool, output }); await waitFor(() => messages.some((item) => item.id === 2)); const result = messages.find((item) => item.id === 2)?.result; - expect(result?.isError).toBe(true); expect(result?.structuredContent?.outputTruncated).toBe(true); - expect(result?.structuredContent?.ok).toBe(false); - expect(result?.structuredContent?.data).toBeUndefined(); + if (tool === "hack.project.logs.tail") { + // HACK-1134: the byte bound is a cutoff (success with partial data), + // not a failure; see docs/reference/mcp-logs.md. + expect(result?.isError).toBeUndefined(); + expect(result?.structuredContent?.ok).toBe(true); + expect(result?.structuredContent?.data).toMatchObject({ + cutoff: { bound: "bytes", limit: 8 * 1024 * 1024 }, + stop_reason: "output_limit", + }); + } else { + expect(result?.isError).toBe(true); + expect(result?.structuredContent?.ok).toBe(false); + expect(result?.structuredContent?.data).toBeUndefined(); + } expect(result?.structuredContent?.stderr).toContain( "8 MiB capture limit" ); diff --git a/tests/mcp-logs-contract.test.ts b/tests/mcp-logs-contract.test.ts new file mode 100644 index 00000000..833a17fd --- /dev/null +++ b/tests/mcp-logs-contract.test.ts @@ -0,0 +1,712 @@ +import { afterEach, expect, test } from "bun:test"; +import { + chmod, + mkdir, + mkdtemp, + readFile, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; + +import { + classifyLogCollection, + MCP_LOG_BACKEND_ERROR, + type McpLogStopReason, + observeLogStreamEvent, +} from "../src/mcp/logs-contract.ts"; +import { + createMcpOutputBudget, + incompleteUtf8TailLength, + MCP_OUTPUT_LIMIT_BYTES, +} from "../src/mcp/output-budget.ts"; +import { createMcpServer } from "../src/mcp/server.ts"; + +// --------------------------------------------------------------------------- +// HACK-1134: logs.tail / logs.snapshot separate a cutoff (success, data.cutoff) +// from a backend failure (closed backend_error), cut output on a UTF-8 code +// point boundary, and never replay events across calls or reconnects. +// +// Every fixture is a temp root with its own HOME, HACK_HOME and registry; the +// hack binary is a stub selected by HACK_MCP_COMMAND, so no Docker/OrbStack +// socket, agent client config or real home directory is touched. +// --------------------------------------------------------------------------- + +const LIMITS = { maxEvents: 2, timeMs: 1000, bytes: 64 } as const; + +test("only type:log events count toward maxEvents; error and failing end events are failures", () => { + expect(observeLogStreamEvent({ type: "log", entry: {} })).toEqual({ + isLog: true, + backendFailure: false, + }); + for (const type of ["start", "heartbeat", undefined]) { + expect(observeLogStreamEvent({ type })).toEqual({ + isLog: false, + backendFailure: false, + }); + } + expect(observeLogStreamEvent({ type: "error", message: "x" })).toEqual({ + isLog: false, + backendFailure: true, + }); + for (const reason of ["eof", "closed", undefined]) { + expect(observeLogStreamEvent({ type: "end", reason }).backendFailure).toBe( + false + ); + } + for (const reason of ["error", "exit:1", "exit:125"]) { + expect(observeLogStreamEvent({ type: "end", reason }).backendFailure).toBe( + true + ); + } +}); + +test("each stop reason maps to exactly one outcome and failure evidence wins", () => { + const classify = ( + stopReason: McpLogStopReason, + exitCode: number, + backendFailureObserved = false + ) => + classifyLogCollection({ + stopReason, + exitCode, + backendFailureObserved, + limits: LIMITS, + }); + expect(classify("eof", 0)).toEqual({ kind: "complete" }); + expect(classify("eof", 1)).toEqual({ kind: "backend_error" }); + // A server-initiated stop is a cutoff even though the child exits non-zero. + expect(classify("max_events", 143)).toEqual({ + kind: "cutoff", + cutoff: { bound: "events", limit: 2 }, + }); + expect(classify("timeout", 137)).toEqual({ + kind: "cutoff", + cutoff: { bound: "time", limit: 1000 }, + }); + expect(classify("output_limit", 143)).toEqual({ + kind: "cutoff", + cutoff: { bound: "bytes", limit: 64 }, + }); + expect(classify("cancelled", 143)).toEqual({ kind: "cancelled" }); + expect(classify("interrupted", 143)).toEqual({ kind: "cancelled" }); + for (const stopReason of [ + "eof", + "max_events", + "timeout", + "output_limit", + ] as const) { + expect(classify(stopReason, 0, true)).toEqual({ kind: "backend_error" }); + } + expect(Object.isFrozen(MCP_LOG_BACKEND_ERROR)).toBe(true); +}); + +test("incomplete UTF-8 tails are measured for 2-, 3- and 4-byte sequences", () => { + const bytes = (text: string) => new TextEncoder().encode(text); + expect(incompleteUtf8TailLength(bytes(""))).toBe(0); + expect(incompleteUtf8TailLength(bytes("ab"))).toBe(0); + for (const char of ["é", "€", "😀"]) { + const encoded = bytes(`a${char}`); + expect(incompleteUtf8TailLength(encoded)).toBe(0); + for (let drop = 1; drop < encoded.byteLength - 1; drop += 1) { + expect( + incompleteUtf8TailLength(encoded.subarray(0, encoded.byteLength - drop)) + ).toBe(encoded.byteLength - 1 - drop); + } + } + // Malformed input is never held back: stray continuations, bad leads. + expect(incompleteUtf8TailLength(new Uint8Array([0x80, 0x80, 0x80]))).toBe(0); + expect(incompleteUtf8TailLength(new Uint8Array([0x61, 0xff]))).toBe(0); +}); + +function chunkedStream(chunks: readonly Uint8Array[]) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) { + controller.enqueue(chunk); + } + controller.close(); + }, + }); +} + +async function readAll(stream: ReadableStream) { + return new Uint8Array(await new Response(stream).arrayBuffer()); +} + +test("code-point-safe capture never ends on a split code point at the byte limit", async () => { + const text = "a😀😀😀"; + const encoded = new TextEncoder().encode(text); + for (let maxBytes = 1; maxBytes < encoded.byteLength; maxBytes += 1) { + const safe = createMcpOutputBudget({ + maxBytes, + codePointSafe: true, + onLimit: () => undefined, + }); + const kept = await readAll(safe.wrap(chunkedStream([encoded]))); + expect(safe.exceeded()).toBe(true); + expect(kept.byteLength).toBeLessThanOrEqual(maxBytes); + expect(new TextDecoder("utf-8", { fatal: true }).decode(kept)).toBe( + text.slice(0, 1 + 2 * Math.floor((maxBytes - 1) / 4)) + ); + } +}); + +test("negative control: the byte-exact default capture does split a code point", async () => { + const encoded = new TextEncoder().encode("a😀"); + const exact = createMcpOutputBudget({ + maxBytes: 3, + onLimit: () => undefined, + }); + const kept = await readAll(exact.wrap(chunkedStream([encoded]))); + expect(kept.byteLength).toBe(3); + expect(new TextDecoder().decode(kept)).toContain("�"); +}); + +test("code-point-safe capture joins sequences split across chunks and keeps exact-limit output", async () => { + const encoded = new TextEncoder().encode("é€😀"); + const chunks = [...encoded].map((byte) => new Uint8Array([byte])); + let stops = 0; + const budget = createMcpOutputBudget({ + maxBytes: encoded.byteLength, + codePointSafe: true, + onLimit: () => { + stops += 1; + }, + }); + const kept = await readAll(budget.wrap(chunkedStream(chunks))); + expect(new TextDecoder("utf-8", { fatal: true }).decode(kept)).toBe("é€😀"); + expect(budget.exceeded()).toBe(false); + expect(stops).toBe(0); +}); + +test("code-point-safe capture shares one budget and drops a held partial sequence once cut", async () => { + const budget = createMcpOutputBudget({ + maxBytes: 4, + codePointSafe: true, + onLimit: () => undefined, + }); + const euro = new TextEncoder().encode("€"); + // First stream ends after two of the euro's three bytes, consuming 3 bytes. + const first = await readAll( + budget.wrap(chunkedStream([new Uint8Array([0x61]), euro.subarray(0, 2)])) + ); + // Source bytes that are themselves incomplete pass through at stream end. + expect([...first]).toEqual([0x61, ...euro.subarray(0, 2)]); + const second = await readAll(budget.wrap(chunkedStream([euro]))); + expect(second.byteLength).toBe(0); + expect(budget.exceeded()).toBe(true); +}); + +// --------------------------------------------------------------------------- +// In-process MCP sessions against a scripted log backend. +// --------------------------------------------------------------------------- + +type Scenario = + | "clean" + | "heartbeats" + | "follow" + | "backend-error" + | "exit-end" + | "nonzero-exit" + | "error-after-stop" + | "bytes-utf8" + | "sequenced"; + +type LogsFixture = { + readonly root: string; + readonly stubDir: string; + readonly connect: () => Promise; +}; + +const CANARY = "LOGS-CANARY-QUERY-ECHO"; +const fixtureRoots: string[] = []; +const fixtureClients: Client[] = []; + +afterEach(async () => { + await Promise.all( + fixtureClients.splice(0).map(async (mcp) => { + try { + await mcp.close(); + } catch { + /* already closed */ + } + }) + ); + for (const root of fixtureRoots.splice(0)) { + for (const pid of await readStubPids(join(root, "stub"))) { + try { + process.kill(pid, "SIGKILL"); + } catch { + /* already stopped */ + } + } + await rm(root, { recursive: true, force: true }); + } +}); + +const STUB_SOURCE = String.raw` +import { appendFileSync, existsSync, readFileSync, writeFileSync } from "node:fs"; +const dir = process.env.HACK_TEST_STUB_DIR; +const scenario = process.env.HACK_TEST_LOG_SCENARIO; +const counter = dir + "/invocations"; +const n = (existsSync(counter) ? Number(readFileSync(counter, "utf8")) : 0) + 1; +writeFileSync(counter, String(n)); +appendFileSync(dir + "/pids", process.pid + "\n"); +const line = (event) => + JSON.stringify({ ts: "2026-01-01T00:00:00.000Z", backend: "compose", ...event }) + "\n"; +const emit = (event) => process.stdout.write(line(event)); +const log = (message) => emit({ type: "log", entry: { message } }); +const hang = () => setInterval(() => {}, 1000); +emit({ type: "start", follow: true }); +switch (scenario) { + case "clean": + log("one"); + log("two"); + emit({ type: "end", reason: "eof" }); + break; + case "heartbeats": + emit({ type: "heartbeat" }); + log("one"); + emit({ type: "heartbeat" }); + emit({ type: "heartbeat" }); + log("two"); + log("three"); + hang(); + break; + case "follow": + log("one"); + hang(); + break; + case "backend-error": + log("one"); + emit({ type: "error", message: "Loki error: " + process.env.HACK_TEST_CANARY }); + emit({ type: "end", reason: "error" }); + process.stderr.write("Loki error: " + process.env.HACK_TEST_CANARY + "\n"); + process.exitCode = 1; + break; + case "exit-end": + log("one"); + emit({ type: "end", reason: "exit:1" }); + break; + case "nonzero-exit": + process.stderr.write("compose failed\n"); + process.exitCode = 1; + break; + case "error-after-stop": { + log("one"); + const timer = hang(); + process.on("SIGTERM", () => { + clearInterval(timer); + emit({ type: "error", message: "terminated" }); + emit({ type: "end", reason: "error" }); + process.exitCode = 1; + }); + break; + } + case "bytes-utf8": { + log("one"); + log("two"); + const written = Buffer.byteLength(line({ type: "start", follow: true })) + + Buffer.byteLength(line({ type: "log", entry: { message: "one" } })) + + Buffer.byteLength(line({ type: "log", entry: { message: "two" } })); + // Pad so the 8 MiB cut lands three bytes into a four-byte code point. + const pad = (5 - (written % 4)) % 4; + process.stdout.write("a".repeat(pad) + "😀".repeat(2_400_000)); + hang(); + break; + } + case "sequenced": { + let seq = 0; + setInterval(() => { + log("inv-" + n + "-seq-" + seq); + appendFileSync(dir + "/emitted-" + n, seq + "\n"); + seq += 1; + }, 10); + break; + } + default: + process.exitCode = 2; +} +`; + +async function startLogsFixture(scenario: Scenario): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-mcp-logs-")); + fixtureRoots.push(root); + const project = join(root, "repo"); + const projectDir = join(project, ".hack"); + const registryDir = join(root, "registry"); + const stubDir = join(root, "stub"); + await mkdir(projectDir, { recursive: true }); + await mkdir(registryDir); + await mkdir(stubDir); + await writeFile(join(projectDir, "docker-compose.yml"), "services: {}\n"); + await writeFile( + join(projectDir, "hack.config.json"), + JSON.stringify({ name: "demo" }) + ); + await writeFile( + join(registryDir, "projects.json"), + JSON.stringify({ + version: 1, + projects: [ + { + id: "demo", + createdAt: "2026-01-01T00:00:00Z", + name: "demo", + repoRoot: project, + projectDirName: ".hack", + projectDir, + }, + ], + }) + ); + const stub = join(root, "hack-stub"); + await writeFile(stub, `#!${process.execPath}\n${STUB_SOURCE}`); + await chmod(stub, 0o755); + const env: Record = { + ...process.env, + HOME: root, + HACK_HOME: join(root, "state"), + HACK_GLOBAL_CONFIG_PATH: join(registryDir, "config.json"), + HACK_MCP_COMMAND: stub, + HACK_TEST_STUB_DIR: stubDir, + HACK_TEST_LOG_SCENARIO: scenario, + HACK_TEST_CANARY: CANARY, + }; + // Each connect is a fresh session, as a client reconnecting to Hack gets. + const connect = async (): Promise => { + const server = createMcpServer({ cwd: project, env }); + const client = new Client({ name: "hack-mcp-logs", version: "1" }); + fixtureClients.push(client); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + await client.connect(clientTransport); + return client; + }; + return { root, stubDir, connect }; +} + +type LogsPayload = { + readonly ok: boolean; + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; + readonly outputTruncated?: boolean; + readonly error?: { code: string; reason: string; message: string }; + readonly data: { + readonly events: readonly Record[]; + readonly count: number; + readonly cutoff: { bound: string; limit: number } | null; + readonly stop_reason?: string; + }; +}; + +async function callLogs(opts: { + readonly client: Client; + readonly tool: "hack.project.logs.tail" | "hack.project.logs.snapshot"; + readonly args?: Record; + readonly signal?: AbortSignal; +}): Promise<{ isError: boolean; payload: LogsPayload; text: string }> { + const result = await opts.client.callTool( + { + name: opts.tool, + arguments: { projectName: "demo", ...opts.args }, + }, + undefined, + opts.signal ? { signal: opts.signal } : undefined + ); + const content = result.content as Array<{ type: string; text?: string }>; + return { + isError: result.isError === true, + payload: result.structuredContent as LogsPayload, + text: content.map((item) => item.text ?? "").join(""), + }; +} + +function logMessages(events: readonly Record[]): string[] { + return events + .filter((event) => event.type === "log") + .map((event) => { + const entry = event.entry as { message?: unknown } | undefined; + return String(entry?.message); + }); +} + +async function readStubPids(stubDir: string): Promise { + const text = await readFile(join(stubDir, "pids"), "utf8").catch(() => ""); + return text + .split("\n") + .filter((value) => value.length > 0) + .map(Number); +} + +async function readEmitted(stubDir: string, invocation: number) { + const text = await readFile( + join(stubDir, `emitted-${invocation}`), + "utf8" + ).catch(() => ""); + return text.split("\n").filter((value) => value.length > 0).length; +} + +function alive(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch { + return false; + } +} + +async function waitFor(check: () => boolean | Promise): Promise { + const end = Date.now() + 8000; + while (!(await check())) { + if (Date.now() >= end) { + throw new Error("Timed out waiting for logs fixture"); + } + await Bun.sleep(20); + } +} + +test("a clean end of stream is complete: ok with a null cutoff for tail and snapshot", async () => { + const fixture = await startLogsFixture("clean"); + const client = await fixture.connect(); + for (const tool of [ + "hack.project.logs.tail", + "hack.project.logs.snapshot", + ] as const) { + const { isError, payload } = await callLogs({ client, tool }); + expect(isError).toBe(false); + expect(payload.ok).toBe(true); + expect(payload.exitCode).toBe(0); + expect(payload.error).toBeUndefined(); + expect(payload.data.cutoff).toBeNull(); + expect(logMessages(payload.data.events)).toEqual(["one", "two"]); + expect(payload.data.count).toBe(payload.data.events.length); + } +}); + +test("maxEvents counts log events only and its cutoff is a success", async () => { + const fixture = await startLogsFixture("heartbeats"); + const client = await fixture.connect(); + const { isError, payload } = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxEvents: 2, maxMs: 10_000 }, + }); + expect(isError).toBe(false); + expect(payload.ok).toBe(true); + expect(payload.data.cutoff).toEqual({ bound: "events", limit: 2 }); + expect(payload.data.stop_reason).toBe("max_events"); + expect(logMessages(payload.data.events)).toEqual(["one", "two"]); + // start + 3 heartbeats are returned but did not consume the event budget. + expect(payload.data.events.map((event) => event.type)).toEqual([ + "start", + "heartbeat", + "log", + "heartbeat", + "heartbeat", + "log", + ]); + for (const pid of await readStubPids(fixture.stubDir)) { + await waitFor(() => !alive(pid)); + } +}); + +test("the collection window cutoff is a success even though the child is signalled", async () => { + const fixture = await startLogsFixture("follow"); + const client = await fixture.connect(); + const { isError, payload } = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxMs: 300 }, + }); + expect(isError).toBe(false); + expect(payload.ok).toBe(true); + expect(payload.exitCode).not.toBe(0); + expect(payload.data.cutoff).toEqual({ bound: "time", limit: 300 }); + expect(logMessages(payload.data.events)).toEqual(["one"]); +}); + +test("failure events after a server-initiated stop do not turn a cutoff into an error", async () => { + const fixture = await startLogsFixture("error-after-stop"); + const client = await fixture.connect(); + const { isError, payload } = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxMs: 300 }, + }); + // The stub did report error/end(error) in response to SIGTERM ... + expect(payload.data.events.map((event) => event.type)).toEqual([ + "start", + "log", + "error", + "end", + ]); + // ... yet the outcome is the time cutoff the server caused. + expect(isError).toBe(false); + expect(payload.ok).toBe(true); + expect(payload.data.cutoff).toEqual({ bound: "time", limit: 300 }); +}); + +for (const [scenario, label] of [ + ["backend-error", "an error event"], + ["exit-end", "a failing end event"], + ["nonzero-exit", "a non-zero exit at end of stream"], +] as const) { + test(`${label} is a closed backend_error, not a truncated success`, async () => { + const fixture = await startLogsFixture(scenario); + const client = await fixture.connect(); + for (const tool of [ + "hack.project.logs.tail", + "hack.project.logs.snapshot", + ] as const) { + const { isError, payload } = await callLogs({ client, tool }); + expect(isError).toBe(true); + expect(payload.ok).toBe(false); + expect(payload.error).toEqual({ + code: "E_UNEXPECTED", + reason: "backend_error", + message: MCP_LOG_BACKEND_ERROR.message, + }); + expect(payload.data.cutoff).toBeNull(); + expect(payload.outputTruncated).toBeUndefined(); + // The closed diagnostic never repeats backend text or caller input. + expect(payload.error?.message).not.toContain(CANARY); + } + }); +} + +test("the byte cutoff returns earlier events and never splits a code point", async () => { + const fixture = await startLogsFixture("bytes-utf8"); + const client = await fixture.connect(); + const { isError, payload } = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxMs: 30_000 }, + }); + expect(isError).toBe(false); + expect(payload.ok).toBe(true); + expect(payload.outputTruncated).toBe(true); + expect(payload.data.cutoff).toEqual({ + bound: "bytes", + limit: MCP_OUTPUT_LIMIT_BYTES, + }); + expect(payload.data.stop_reason).toBe("output_limit"); + expect(logMessages(payload.data.events)).toEqual(["one", "two"]); + expect(payload.stdout).not.toContain("�"); + const lastLine = payload.stdout.trimEnd().split("\n").at(-1) ?? ""; + expect(lastLine.endsWith("😀")).toBe(true); + // The cut backed off the three bytes of the split emoji, and no more. + expect(Buffer.byteLength(payload.stdout)).toBe( + MCP_OUTPUT_LIMIT_BYTES - 3 + 1 + ); + expect(payload.stderr).toContain("8 MiB capture limit"); + for (const pid of await readStubPids(fixture.stubDir)) { + await waitFor(() => !alive(pid)); + } +}, 30_000); + +test("consecutive tail calls each start a new stream: no cursor, no replay", async () => { + const fixture = await startLogsFixture("sequenced"); + const client = await fixture.connect(); + const first = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxEvents: 3, maxMs: 10_000 }, + }); + const second = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxEvents: 3, maxMs: 10_000 }, + }); + expect(logMessages(first.payload.data.events)).toEqual([ + "inv-1-seq-0", + "inv-1-seq-1", + "inv-1-seq-2", + ]); + // The second call neither repeats the first call's events nor resumes + // after them; it is an independent read of the backend. + expect(logMessages(second.payload.data.events)).toEqual([ + "inv-2-seq-0", + "inv-2-seq-1", + "inv-2-seq-2", + ]); +}); + +test("a disconnect mid-tail stops the backend and a reconnected session never sees its events", async () => { + const fixture = await startLogsFixture("sequenced"); + const disconnected = await fixture.connect(); + const pending = callLogs({ + client: disconnected, + tool: "hack.project.logs.tail", + args: { maxEvents: 10_000, maxMs: 30_000 }, + }); + const settled = pending.then( + () => "resolved" as const, + () => "rejected" as const + ); + await waitFor(async () => (await readEmitted(fixture.stubDir, 1)) >= 3); + const [firstPid] = await readStubPids(fixture.stubDir); + expect(alive(firstPid ?? 0)).toBe(true); + + await disconnected.close(); + expect(await settled).toBe("rejected"); + await waitFor(() => !alive(firstPid ?? 0)); + const emittedBeforeClose = await readEmitted(fixture.stubDir, 1); + expect(emittedBeforeClose).toBeGreaterThanOrEqual(3); + + const reconnected = await fixture.connect(); + const { isError, payload } = await callLogs({ + client: reconnected, + tool: "hack.project.logs.tail", + args: { maxEvents: 3, maxMs: 10_000 }, + }); + expect(isError).toBe(false); + expect(payload.data.cutoff).toEqual({ bound: "events", limit: 3 }); + expect(logMessages(payload.data.events)).toEqual([ + "inv-2-seq-0", + "inv-2-seq-1", + "inv-2-seq-2", + ]); + expect(payload.stdout).not.toContain("inv-1-"); + // The interrupted stream produced nothing after its session closed. + expect(await readEmitted(fixture.stubDir, 1)).toBe(emittedBeforeClose); +}, 30_000); + +test("a cancelled tail stops its backend and the same session's retry starts fresh", async () => { + const fixture = await startLogsFixture("sequenced"); + const client = await fixture.connect(); + const controller = new AbortController(); + const pending = callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxEvents: 10_000, maxMs: 30_000 }, + signal: controller.signal, + }); + const settled = pending.then( + () => "resolved" as const, + () => "rejected" as const + ); + await waitFor(async () => (await readEmitted(fixture.stubDir, 1)) >= 3); + const [firstPid] = await readStubPids(fixture.stubDir); + controller.abort(); + expect(await settled).toBe("rejected"); + await waitFor(() => !alive(firstPid ?? 0)); + + const retry = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxEvents: 2, maxMs: 10_000 }, + }); + expect(retry.payload.ok).toBe(true); + expect(logMessages(retry.payload.data.events)).toEqual([ + "inv-2-seq-0", + "inv-2-seq-1", + ]); +}, 30_000); From e32b528621de1f5137fc8162d0ee46351607afff Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:43:58 -0400 Subject: [PATCH 34/52] feat(agents): ship a skills-only hack plugin and an opt-in hack-mcp plugin The default `hack` plugin now carries skills, Claude primer hooks and the Cursor rule but no MCP: its client manifests declare no `mcpServers` and the package ships no `.mcp.json` or `mcp.json`. The generator refuses a skills manifest that declares MCP and removes stale MCP files from it. The generator emits a complete `plugins/hack-mcp` package (Codex, Claude Code, Cursor and portable manifests, `.mcp.json`, portable `mcp.json`, README) from one source, `HACK_MCP_SERVER` (`hack mcp serve` over stdio). All three marketplaces list `hack` and `hack-mcp` separately, and release preparation versions and commits both packages. Agent-setup guidance is plugin-first: MCP only by explicit selection for no-shell clients, `hack setup cursor|claude|codex|agents` as opt-in export, and setup/onboarding never write AGENTS/CLAUDE/rules or repo skill copies unless asked. The `/hack-init` MCP fallback applies only when a Hack MCP server is installed. Docs, the capability matrix (v2) and render fingerprints are updated. Migration: existing `hack` plugin installs lose the bundled MCP server on update; add `hack-mcp` (or keep a `hack mcp install` entry) to keep MCP. Refs: HACK-1132 Co-Authored-By: Claude Opus 5.5 --- .agents/plugins/marketplace.json | 12 + .claude-plugin/marketplace.json | 7 + .claude/skills/hack-init/SKILL.md | 2 +- .codex/skills/hack-cli/SKILL.md | 13 +- .codex/skills/hack-init/SKILL.md | 2 +- .cursor-plugin/marketplace.json | 5 + .cursor/rules/hack.mdc | 2 +- .releaserc.json | 3 +- README.md | 23 +- docs/agent-guidance.md | 13 +- docs/guides/agent-first-setup.md | 21 +- docs/integrations.md | 90 +++-- docs/reference/agent-client-capabilities.md | 41 ++- examples/basic/AGENTS.md | 13 +- examples/basic/CLAUDE.md | 13 +- plugins/hack-mcp/.claude-plugin/plugin.json | 14 + plugins/hack-mcp/.codex-plugin/plugin.json | 25 ++ plugins/hack-mcp/.cursor-plugin/plugin.json | 13 + plugins/{hack => hack-mcp}/.mcp.json | 0 plugins/hack-mcp/README.md | 20 ++ plugins/{hack => hack-mcp}/mcp.json | 0 plugins/hack-mcp/plugin.json | 29 ++ plugins/hack/.claude-plugin/plugin.json | 3 +- plugins/hack/.codex-plugin/plugin.json | 5 +- plugins/hack/.cursor-plugin/plugin.json | 3 +- plugins/hack/README.md | 69 ++-- plugins/hack/plugin.json | 6 +- plugins/hack/rules/hack.mdc | 2 +- plugins/hack/skills/hack-cli/SKILL.md | 13 +- plugins/hack/skills/hack-init/SKILL.md | 2 +- plugins/hack/skills/hack-install/SKILL.md | 7 +- scripts/generate-agent-client-capabilities.ts | 6 +- scripts/generate-agent-plugins.ts | 220 ++++++++++-- src/agents/client-capabilities.ts | 46 ++- src/agents/hack-init-skill.ts | 7 +- src/agents/install-skill.ts | 7 +- src/agents/instruction-source.ts | 11 +- src/agents/integration-ownership.ts | 3 + src/agents/integration-revision.ts | 2 +- tests/agent-client-capabilities.test.ts | 98 +++++- tests/agent-plugin-generation.test.ts | 328 ++++++++++++++++-- tests/hack-init-skill.test.ts | 5 + 42 files changed, 960 insertions(+), 244 deletions(-) create mode 100644 plugins/hack-mcp/.claude-plugin/plugin.json create mode 100644 plugins/hack-mcp/.codex-plugin/plugin.json create mode 100644 plugins/hack-mcp/.cursor-plugin/plugin.json rename plugins/{hack => hack-mcp}/.mcp.json (100%) create mode 100644 plugins/hack-mcp/README.md rename plugins/{hack => hack-mcp}/mcp.json (100%) create mode 100644 plugins/hack-mcp/plugin.json diff --git a/.agents/plugins/marketplace.json b/.agents/plugins/marketplace.json index eaa1539a..c67f4560 100644 --- a/.agents/plugins/marketplace.json +++ b/.agents/plugins/marketplace.json @@ -15,6 +15,18 @@ "authentication": "ON_INSTALL" }, "category": "Developer Tools" + }, + { + "name": "hack-mcp", + "source": { + "source": "local", + "path": "./plugins/hack-mcp" + }, + "policy": { + "installation": "AVAILABLE", + "authentication": "ON_INSTALL" + }, + "category": "Developer Tools" } ] } diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index d87b0fbb..a9a0a711 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -11,6 +11,13 @@ "description": "Operate and onboard Hack-managed local development environments.", "category": "development", "tags": ["local-development", "docker", "developer-tools", "agents"] + }, + { + "name": "hack-mcp", + "source": "./plugins/hack-mcp", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access.", + "category": "development", + "tags": ["local-development", "developer-tools", "agents", "mcp"] } ] } diff --git a/.claude/skills/hack-init/SKILL.md b/.claude/skills/hack-init/SKILL.md index 0453fac6..1b186c46 100644 --- a/.claude/skills/hack-init/SKILL.md +++ b/.claude/skills/hack-init/SKILL.md @@ -12,6 +12,6 @@ Fetch the current onboarding guidance from the installed Hack CLI. 1. Fetch the canonical onboarding prompt: - With shell access: run `hack agent onboard` (works for new and existing projects). - - Without shell access: fetch the `hack-init` MCP prompt from the hack MCP server. + - Without shell access: only if the `hack-mcp` plugin or another Hack MCP server is installed, fetch the `hack-init` MCP prompt from it; otherwise report that onboarding needs shell access or an explicitly selected Hack MCP server. 2. Apply the relevant phases to the authorized workflow; preserve native approvals, credential safety, and runtime ownership. 3. Verify the requested services and host workflows. Report remaining blockers; unrelated optional integration warnings do not prevent completion. diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index a46be0db..42d81524 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). ## Product boundary @@ -185,13 +185,12 @@ Use `hack` as the primary interface for local-first development. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. -## Agent setup (CLI-first) +## Agent setup (plugin-first) -- Cursor rules: `hack setup cursor` -- Claude hooks: `hack setup claude` -- Codex skill: `hack setup codex` +- Default integration: the optional `hack` plugin, installed through the client's plugin manager, ships skills, Claude primer hooks and a Cursor rule. It ships no MCP server, and Hack commands never install plugins. +- MCP is opt-in, only for clients without shell access and only when explicitly selected: add the separate `hack-mcp` plugin, or use `hack setup mcp` (client and scope via `hack mcp install --help`). Do not configure other clients as a side effect. +- Standalone export is an opt-in alternative to the plugin: `hack setup cursor` (rules), `hack setup claude` (hooks and skill), `hack setup codex` (skills) or `hack setup agents` (AGENTS.md/CLAUDE.md snippets). Use one integration path per client. +- Setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks for them. - Agent-assisted onboarding: `hack init --with claude|codex` (new repos) or `hack agent onboard` (existing projects) print/hand off the full setup prompt; the `/hack-init` skill and the `hack-init` MCP prompt return the same content. - Init prompt: `hack agent init` (use --client cursor|claude|codex to open) - Init patterns: `hack agent patterns` -- MCP (no-shell only): `hack setup mcp` -- MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect. diff --git a/.codex/skills/hack-init/SKILL.md b/.codex/skills/hack-init/SKILL.md index 0453fac6..1b186c46 100644 --- a/.codex/skills/hack-init/SKILL.md +++ b/.codex/skills/hack-init/SKILL.md @@ -12,6 +12,6 @@ Fetch the current onboarding guidance from the installed Hack CLI. 1. Fetch the canonical onboarding prompt: - With shell access: run `hack agent onboard` (works for new and existing projects). - - Without shell access: fetch the `hack-init` MCP prompt from the hack MCP server. + - Without shell access: only if the `hack-mcp` plugin or another Hack MCP server is installed, fetch the `hack-init` MCP prompt from it; otherwise report that onboarding needs shell access or an explicitly selected Hack MCP server. 2. Apply the relevant phases to the authorized workflow; preserve native approvals, credential safety, and runtime ownership. 3. Verify the requested services and host workflows. Report remaining blockers; unrelated optional integration warnings do not prevent completion. diff --git a/.cursor-plugin/marketplace.json b/.cursor-plugin/marketplace.json index 384d2e1d..46f3c3e7 100644 --- a/.cursor-plugin/marketplace.json +++ b/.cursor-plugin/marketplace.json @@ -11,6 +11,11 @@ "name": "hack", "source": "plugins/hack", "description": "Operate and onboard Hack-managed local development environments." + }, + { + "name": "hack-mcp", + "source": "plugins/hack-mcp", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access." } ] } diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 1adb401c..bdaaceb7 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -9,7 +9,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). ## Product boundary diff --git a/.releaserc.json b/.releaserc.json index d33cab3c..d249575c 100644 --- a/.releaserc.json +++ b/.releaserc.json @@ -24,7 +24,8 @@ "packages/db/package.json", "services/auth-broker/package.json", "bun.lock", - "plugins/hack" + "plugins/hack", + "plugins/hack-mcp" ], "message": "chore(release): ${nextRelease.version}\n\n${nextRelease.notes}" } diff --git a/README.md b/README.md index 15628d63..ddca2524 100644 --- a/README.md +++ b/README.md @@ -46,9 +46,11 @@ linked git worktree, `hack up` automatically becomes a separate instance named a the branch — review a PR while your main checkout keeps running. **Built for agents.** `hack init --with claude|codex` hands setup to an agent. -Machine surfaces are first-class: `--json` envelopes with stable error codes, -`--no-interactive`, and agent instructions that sync themselves into -AGENTS.md/CLAUDE.md, Cursor rules, and Codex skills. +Machine surfaces are first-class: `--json` envelopes with stable error codes and +`--no-interactive`. The optional `hack` plugin for Codex, Claude Code, and Cursor +gives agents Hack skills through the client's plugin manager, without writing into +your repository. Clients without shell access can add the separate `hack-mcp` +plugin. See [agent plugin setup](docs/integrations.md#optional-native-plugins). ## More than ports @@ -114,6 +116,18 @@ hack up --detach hack open ``` +Using a coding agent? Install the optional `hack` plugin for its skills: + +```bash +codex plugin marketplace add hack-dance/hack && codex plugin add hack@hack-dance +# or, in Claude Code: +claude plugin marketplace add hack-dance/hack --sparse .claude-plugin plugins/hack plugins/hack-mcp +claude plugin install hack@hack-dance +``` + +In Cursor, install Hack from Customize. Add `hack-mcp` only for a client without +shell access. + When something looks wrong: ```bash @@ -151,6 +165,3 @@ experimental — hidden behind `hack help --all`. ## License See [LICENSE](./LICENSE). - -Optional native plugins for Codex, Claude Code, and Cursor bundle Hack skills and MCP. -See [agent plugin setup](docs/integrations.md#optional-native-plugins). Existing standalone setup remains supported. diff --git a/docs/agent-guidance.md b/docs/agent-guidance.md index 63df3f65..cd13812b 100644 --- a/docs/agent-guidance.md +++ b/docs/agent-guidance.md @@ -4,7 +4,7 @@ Hack owns Hack-specific instructions. User and project policy continues to own m selection, general coding standards, work-unit planning, evolution, and writeback. Do not ship provider model pins or copy global instruction packs into this plugin. `tests/agent-plugin-generation.test.ts` enforces this for the generated guidance -surfaces (`plugins/hack`, the client marketplace manifests, `.agents`, and the +surfaces (`plugins/hack`, `plugins/hack-mcp`, the client marketplace manifests, `.agents`, and the example agent configuration): no model, model-override or reasoning-effort keys, no permission, approval or sandbox overrides, no output-style settings, and hooks that run only `hack agent prime`. See [model-policy ownership](reference/agent-client-capabilities.md#model-policy-ownership). @@ -21,11 +21,14 @@ run only `hack agent prime`. See [model-policy ownership](reference/agent-client | `src/agents/hack-init-skill.ts` | Thin Claude/Codex/plugin adapter that fetches onboarding guidance | | `src/agents/init-patterns.ts`, `init-assistant.ts` | Optional init inventory and pattern hints | | `src/agents/install-skill.ts`, `remote-bootstrap-skill.ts` | Plugin installation/channel selection and fresh-host/cloud setup skills | -| `scripts/generate-agent-plugins.ts` | Portable plugin/MCP manifests, three client compatibility manifests, four skills, and Cursor rule | +| `scripts/generate-agent-plugins.ts` | Skills-only `hack` plugin (portable manifest, three client compatibility manifests, four skills, Cursor rule) and the complete opt-in `hack-mcp` plugin | -The portable plugin manifest derives identity and effective OpenAI presentation -from `.codex-plugin/plugin.json`; portable MCP derives from `.mcp.json` with an -explicit transport. Keep launch selection identical across both formats. These +The `hack` plugin is skills-only: the generator refuses a client manifest that +declares `mcpServers` and removes stale `.mcp.json`/`mcp.json` files from it. Each +portable plugin manifest derives identity and effective OpenAI presentation from +its package's `.codex-plugin/plugin.json`. Every `hack-mcp` file renders from +code; its `.mcp.json` and portable `mcp.json` (with an explicit transport) both +come from `HACK_MCP_SERVER`, so launch selection stays identical. These plugin workflow skills complement the existing standalone setup; personal local plugin installation does not synchronize skills into a cloud repository. diff --git a/docs/guides/agent-first-setup.md b/docs/guides/agent-first-setup.md index 38153a50..8726d2de 100644 --- a/docs/guides/agent-first-setup.md +++ b/docs/guides/agent-first-setup.md @@ -11,8 +11,8 @@ your installed version. | --- | --- | | A fresh repo + an agent CLI installed | `hack init --with claude\|codex\|both` | | An existing hack project | `hack agent onboard` (prints the prompt) | -| An agent session with the hack skills installed | `/hack-init` | -| A no-shell MCP client | the `hack-init` MCP prompt (`hack setup mcp`) | +| An agent session with the `hack` plugin installed | `/hack-init` | +| A no-shell client with Hack MCP explicitly installed (`hack-mcp` plugin or `hack setup mcp`) | the `hack-init` MCP prompt | ## New repo: `hack init --with` @@ -42,11 +42,18 @@ when present. ## Agent-side skill: `/hack-init` -`hack setup claude` and `hack setup codex` install a thin `hack-init` skill -(`.claude/skills/hack-init/SKILL.md` / `.codex/skills/hack-init/SKILL.md`). -The skill tells the agent to run `hack agent onboard` (or fetch the `hack-init` -MCP prompt) and follow it — the content stays in the CLI, so installed skills -fetch guidance matching the installed CLI. +The optional `hack` plugin ships a thin `hack-init` skill; install the plugin +through your client's plugin manager (see +[Optional native plugins](../integrations.md#optional-native-plugins)). The skill +tells the agent to run `hack agent onboard` and follow the printed prompt. Without +shell access it fetches the `hack-init` MCP prompt only when a Hack MCP server, such +as the opt-in `hack-mcp` plugin, is installed; otherwise it reports that blocker. +The content stays in the CLI, so the skill fetches guidance matching the installed +CLI. + +A client that cannot use the plugin can export the same skill into a chosen scope as +an explicit opt-in (`hack setup claude` or `hack setup codex`). Onboarding itself +never installs it. ## MCP request lifetime diff --git a/docs/integrations.md b/docs/integrations.md index 9519e9e7..300efb61 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -6,8 +6,10 @@ What remains: - local env management and host/container injection - local sessions and runtime orchestration -- optional coding-agent setup helpers: `hack init --with claude|codex`, `hack agent onboard` / - `hack agent init` / `hack agent prime`, and scoped `hack setup` commands +- optional coding-agent integration: the `hack` plugin first (see + [Optional native plugins](#optional-native-plugins)), plus `hack init --with claude|codex`, + `hack agent onboard` / `hack agent init` / `hack agent prime`, and opt-in scoped `hack setup` + export commands `hack setup sync` keeps interactive output compact: it summarizes each scope and only expands the path and reason for stale, missing, or failed artifacts. Exit status remains the automation contract, @@ -67,15 +69,41 @@ trust and approval rules. ## Optional native plugins -Hack also ships an optional native plugin for Codex, Claude Code, and Cursor in -`plugins/hack`. It bundles four skills (`hack-cli`, `hack-init`, `hack-install`, and -`hack-remote`), the `hack mcp serve` server, Claude primer hooks, and a Cursor rule -that the agent can select when relevant. The +The plugin is the default way to give a coding agent Hack guidance. Hack ships two +plugins for Codex, Claude Code, and Cursor from this repository's marketplace: + +| Plugin | Contents | Install when | +| --- | --- | --- | +| `hack` (`plugins/hack`) | Four skills (`hack-cli`, `hack-init`, `hack-install`, and `hack-remote`), Claude primer hooks, and a Cursor rule that the agent can select when relevant. No MCP server. | You want Hack guidance in a client. This is the default. | +| `hack-mcp` (`plugins/hack-mcp`) | Only the stdio `hack mcp serve` server. No skills, hooks, or rules. | You explicitly want MCP, usually for a client without shell access. | + +With shell access, the `hack` plugin and the Hack CLI cover every workflow without a +server process. Install the Hack CLI first and make `hack` available on the client's +PATH; both plugins run it from there. The [agent client capability matrix](reference/agent-client-capabilities.md) records -which of these components have been observed loading in each client. -Install the Hack CLI first and make `hack` available on the client's PATH. +which components have been observed loading in each client. + +### One integration path per client + +Choose one way to deliver each component to each client: + +- Guidance: the `hack` plugin, or standalone export with `hack setup cursor`, + `hack setup claude`, `hack setup codex`, or `hack setup agents`. Standalone export + is opt-in; use it when a client cannot use the plugin. +- MCP: the `hack-mcp` plugin, or one `hack mcp install` entry (see + [Standalone MCP configuration](#standalone-mcp-configuration)). Never both for the + same client. + +Setup and onboarding never write `AGENTS.md`, `CLAUDE.md`, rule files, or repository +skill copies unless you ask for them. + +Clients discover MCP servers by convention, not only through a manifest key. Claude +Code loads a `.mcp.json` at a plugin's root even when the plugin manifest omits +`mcpServers`, and it also discovers a project's root `.mcp.json` (the file +`hack mcp install --claude --scope project` writes) as a project-scoped server that +needs approval. This is why the `hack` plugin ships no `.mcp.json` or `mcp.json` at +all, and why a repository `.mcp.json` counts as that client's MCP path. -Choose either the native plugin or the existing standalone setup for each client. Installing a plugin does not migrate or remove existing rules, skills, hooks, or MCP configuration. If switching an existing installation, inspect the chosen client's project and user scopes, verify the plugin loads in a fresh session, then explicitly @@ -83,40 +111,58 @@ remove only duplicate Hack-owned entries in that scope. Preserve customized cont Do not use a broad sync or removal command as a migration step. If validation fails, disable the plugin and keep using the standalone integration. +### Upgrading from a plugin that bundled MCP + +Earlier `hack` plugin releases also declared `hack mcp serve`. The `hack` plugin is +now skills-only, so an existing install loses its MCP server on update. If you used +that server, add `hack-mcp` (or keep an existing `hack mcp install` entry). If you +had disabled the bundled server in client configuration, you can remove that +override; there is no server left to disable. + +### Install + Codex: ```bash codex plugin marketplace add hack-dance/hack codex plugin add hack@hack-dance +# Only when MCP is wanted: +codex plugin add hack-mcp@hack-dance ``` Claude Code: ```bash -claude plugin marketplace add hack-dance/hack --sparse .claude-plugin plugins/hack +claude plugin marketplace add hack-dance/hack --sparse .claude-plugin plugins/hack plugins/hack-mcp claude plugin install hack@hack-dance +# Only when MCP is wanted: +claude plugin install hack-mcp@hack-dance ``` Cursor: open Customize, find Hack in the configured marketplace, and choose the -project or user installation scope. For local development, copy `plugins/hack` into -`~/.cursor/plugins/local/hack` when that destination is unused, then reload Cursor -and inspect the Hack plugin in Customize. Cursor 3.18.25 rejects symlinks to targets -outside its local-plugin root; use a copy rather than weakening that boundary. See +project or user installation scope. Add Hack MCP the same way only when MCP is +wanted. For local development, copy `plugins/hack` (and, if needed, +`plugins/hack-mcp`) into `~/.cursor/plugins/local/` when those destinations are +unused, then reload Cursor and inspect the plugins in Customize. Cursor 3.18.25 +rejects symlinks to targets outside its local-plugin root; use a copy rather than +weakening that boundary. See [Cursor's plugin documentation](https://cursor.com/docs/plugins) for marketplace and local development support in the installed client version. -After installing, verify in a fresh session that all four skills are discoverable, -`hack mcp serve` can initialize, and the client's hooks or rule load. A successful -marketplace registration alone does not prove those components loaded. The plugin's -MCP fallback is for clients without shell access; use the Hack CLI when shell is available. +After installing, verify in a fresh session that all four skills are discoverable +and the client's hooks or rule load. With `hack-mcp`, also verify that +`hack mcp serve` initializes and lists Hack tools. A successful marketplace +registration alone does not prove those components loaded. Plugin updates are managed by the client. Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or clean up integrations. A missing optional plugin is not a broken Hack installation. Model selection and global instruction, evolution, and writeback policy remain with the user's existing -provider configuration; the Hack plugin adds only Hack-specific guidance. +provider configuration; the Hack plugins add only Hack-specific guidance and tools. Contributors: change shared behavior in `src/agents/instruction-source.ts`, then run -`bun run generate:agent-plugins`. The generation tests check bundled guidance against -the canonical renderers. Release preparation regenerates all plugin content after the -version bump and commits the complete bundle with the CLI release. +`bun run generate:agent-plugins`. The generator writes both packages; the `hack-mcp` +launch comes from `HACK_MCP_SERVER` in `scripts/generate-agent-plugins.ts`. The +generation tests check bundled guidance against the canonical renderers and fail if +the `hack` package declares or ships MCP. Release preparation regenerates both +packages after the version bump and commits them with the CLI release. diff --git a/docs/reference/agent-client-capabilities.md b/docs/reference/agent-client-capabilities.md index 8c408b2c..c22ec76b 100644 --- a/docs/reference/agent-client-capabilities.md +++ b/docs/reference/agent-client-capabilities.md @@ -40,25 +40,25 @@ named test passes against the recorded client version. -Matrix version 1, observed 2026-10-11. +Matrix version 2, observed 2026-10-11. ### Clients -| Client | Manifest | -| --- | --- | -| Codex | `plugins/hack/.codex-plugin/plugin.json` | -| Claude Code | `plugins/hack/.claude-plugin/plugin.json` | -| Cursor | `plugins/hack/.cursor-plugin/plugin.json` | -| Standalone (portable manifest or generic stdio MCP client) | `plugins/hack/plugin.json` | +| Client | `hack` plugin manifest | `hack-mcp` plugin manifest | +| --- | --- | --- | +| Codex | `plugins/hack/.codex-plugin/plugin.json` | `plugins/hack-mcp/.codex-plugin/plugin.json` | +| Claude Code | `plugins/hack/.claude-plugin/plugin.json` | `plugins/hack-mcp/.claude-plugin/plugin.json` | +| Cursor | `plugins/hack/.cursor-plugin/plugin.json` | `plugins/hack-mcp/.cursor-plugin/plugin.json` | +| Standalone (portable manifest or generic stdio MCP client) | `plugins/hack/plugin.json` | `plugins/hack-mcp/plugin.json` | ### Capabilities | Capability | Meaning | | --- | --- | -| Packaging | The client accepts, installs and enables the Hack plugin package in a session. | +| Packaging | The client accepts, installs and enables the default `hack` plugin package in a session. | | Skill discovery | A fresh session discovers the plugin's bundled skills (hack-cli, hack-init, hack-install, hack-remote). | | Hooks | The client runs plugin-shipped hooks; Hack hooks may only run `hack agent prime`. | -| Plugin MCP | The client starts the plugin-declared `hack mcp serve` server and lists its tools. | +| Plugin MCP | The client starts the `hack mcp serve` server declared by the opt-in `hack-mcp` plugin and lists its tools. The default `hack` plugin ships no MCP. | | Standalone MCP | The client loads an MCP entry written by `hack mcp install` (stdio or a selected MCP bundle) without the plugin. | | Rules | The client loads a plugin-shipped rule file. | | UI | The client renders plugin presentation metadata (display name, default prompts, brand color). | @@ -82,28 +82,28 @@ Matrix version 1, observed 2026-10-11. | codex | Packaging | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | codex | Skill discovery | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | codex | Hooks | unsupported | source-read | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Codex plugin hooks; the manifest declares no `hooks` key. This says nothing about Codex's own hook support. | none | -| codex | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| codex | Plugin MCP | untested | schema | `plugins/hack-mcp/.mcp.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `3e261d517ed7` | The opt-in `hack-mcp` manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); the default `hack` plugin declares no MCP. No plugin session started the server. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | codex | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | In the opt-in fixture, `codex mcp get hack --json` resolved the generated bundle adapter from an isolated CODEX_HOME. The fixture then stopped at the Claude step, so no tool call was observed in that run. Configuration discovery alone does not prove a Codex model session loaded the server or accepted project trust. | Pass tests/mcp-bundle-client.test.ts end to end at this client version; a Codex model session tool call remains a separate authenticated-session gate. | | codex | Rules | unsupported | source-read | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Codex plugin rule file. Project AGENTS.md snippets are standalone setup, not a plugin component. | none | | codex | UI | untested | schema | `plugins/hack/.codex-plugin/plugin.json` | codex-cli 0.162.1 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares an `interface` block (display name, default prompts, brand color); rendering was not observed. | Observe the installed plugin's presentation in the Codex UI for this client version. | | claude | Packaging | untested | schema | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts; no client installed it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | claude | Skill discovery | untested | schema | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | claude | Hooks | untested | schema | `plugins/hack/hooks/claude-hooks.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | SessionStart and PreCompact hooks run only `hack agent prime`; the generation test enforces that command allowlist. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | -| claude | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| claude | Plugin MCP | untested | schema | `plugins/hack-mcp/.mcp.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `3e261d517ed7` | The opt-in `hack-mcp` manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); the default `hack` plugin declares no MCP. No plugin session started the server. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | claude | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | The opt-in fixture failed at this version: in an isolated HOME, `claude mcp get hack` reports the project-scoped server as pending approval instead of connected, so no connection or tool call was observed. Project approval is a native trust gate and must not be bypassed. | Qualify Claude standalone MCP without bypassing project approval (for example, user scope in an isolated HOME) and pass the fixture end to end. | | claude | Rules | unsupported | source-read | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Claude plugin rule file. Project CLAUDE.md snippets are standalone setup, not a plugin component. | none | | claude | UI | unsupported | source-read | `plugins/hack/.claude-plugin/plugin.json` | Claude Code 2.1.296 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares no presentation metadata. Codex UI metadata implies nothing for Claude Code. | none | | cursor | Packaging | untested | schema | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest is generated and parsed by tests/agent-plugin-generation.test.ts. docs/integrations.md records an earlier manual symlink-rejection observation (Cursor 3.18.25), not a fixture. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | cursor | Skill discovery | untested | schema | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares `skills: ./skills/`; skill content matches the canonical renderers. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | cursor | Hooks | unsupported | source-read | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Hack ships no Cursor plugin hooks; the manifest declares no `hooks` key. | none | -| cursor | Plugin MCP | untested | schema | `plugins/hack/.mcp.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | +| cursor | Plugin MCP | untested | schema | `plugins/hack-mcp/.mcp.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `3e261d517ed7` | The opt-in `hack-mcp` manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); the default `hack` plugin declares no MCP. No plugin session started the server. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | cursor | Standalone MCP | untested | schema | `tests/mcp-install.test.ts` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Tests check the `.cursor/mcp.json` entry Hack writes; Cursor itself never loaded it in a fixture. | Add an isolated Cursor fixture that loads the written entry and lists Hack tools. | | cursor | Rules | untested | schema | `plugins/hack/rules/hack.mdc` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Agent-selectable rule (`alwaysApply: false`) matches the canonical renderer; loading was not observed. | Install the plugin into an isolated client home, start a fresh session and observe the component load; a marketplace listing or manifest parse does not count. | | cursor | UI | unsupported | source-read | `plugins/hack/.cursor-plugin/plugin.json` | Cursor 3.24.12 | `macos-arm64-contributor` | `ec56769bdc04` | Manifest declares no presentation metadata. Codex UI metadata implies nothing for Cursor. | none | | standalone | Packaging | untested | schema | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Portable agent-plugins manifest is derived from the Codex manifest; no portable consumer was observed. | Name a portable consumer and observe it install the package. | | standalone | Skill discovery | untested | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | The portable manifest declares no component paths; discovery of `skills/` by convention is unverified. | Name a portable consumer and observe skill discovery in a fresh session. | | standalone | Hooks | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | The portable manifest carries hooks only through the OpenAI extension, and Hack ships none there. | none | -| standalone | Plugin MCP | untested | schema | `plugins/hack/mcp.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Portable MCP manifest adds an explicit stdio transport to the same launch as `.mcp.json`. | Name a portable consumer and observe it start the declared server. | +| standalone | Plugin MCP | untested | schema | `plugins/hack-mcp/mcp.json` | none | `macos-arm64-contributor` | `3e261d517ed7` | The opt-in `hack-mcp` portable MCP manifest adds an explicit stdio transport to the same launch as its `.mcp.json`; the default `hack` package ships no MCP file. | Name a portable consumer and observe it start the declared server. | | standalone | Standalone MCP | untested | isolated-fixture | `tests/mcp-bundle-client.test.ts` | none | `macos-arm64-contributor` | `ec56769bdc04` | The fixture's reference MCP SDK client calls a real tool through the generated entry, but that step was not reached in the recorded run. The SDK client is not an agent. | Pass tests/mcp-bundle-client.test.ts end to end, or add a client-independent stdio fixture. | | standalone | Rules | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | No portable rule component is shipped. | none | | standalone | UI | unsupported | source-read | `plugins/hack/plugin.json` | none | `macos-arm64-contributor` | `ec56769bdc04` | Presentation metadata exists only in the OpenAI extension; no portable UI is claimed. | none | @@ -112,12 +112,15 @@ Matrix version 1, observed 2026-10-11. ## Companion-package layout decision -Hack keeps three separate companion layouts. Each layout has one authoritative -owner, and none of them replaces the others. Users choose one per client. +Hack keeps four separate companion layouts. Each layout has one authoritative +owner, and none of them replaces the others. The default is the skills-only `hack` +plugin; users choose one integration path per client and add MCP only by explicit +selection. | Layout | What it contains | Authoritative owner | Installed and updated by | | --- | --- | --- | --- | -| Native plugin (`plugins/hack`) | Client manifests, four skills, Claude primer hooks, Cursor rule, `hack mcp serve` declaration | `scripts/generate-agent-plugins.ts`, rendering from `src/agents/instruction-source.ts` and the skill renderers in `src/agents/` | The client's plugin manager. Hack commands never install or update it. | +| Native plugin (`plugins/hack`, default) | Client manifests, four skills, Claude primer hooks, Cursor rule; no MCP | `scripts/generate-agent-plugins.ts`, rendering from `src/agents/instruction-source.ts` and the skill renderers in `src/agents/` | The client's plugin manager. Hack commands never install or update it. | +| MCP plugin (`plugins/hack-mcp`, opt-in) | Client manifests and the stdio `hack mcp serve` declaration only | `scripts/generate-agent-plugins.ts` (`HACK_MCP_SERVER` is the single launch source) | The client's plugin manager, only when MCP is selected. Hack commands never install or update it. | | Standalone setup | Project AGENTS.md/CLAUDE.md snippets, Cursor rules, shared skills, Claude primer hooks, MCP entries | `src/agents/instruction-source.ts` and its renderers, written by `hack setup` and `hack mcp install` | Explicit `hack setup` and `hack mcp install` commands in a chosen scope | | MCP bundle | Native MCP adapter, owner and backend executables with a content-derived identity | `src/mcp/bundle.ts` (packaging and verification) | `hack mcp install --bundle ... --cli ...` writes the client entry; the bundle is never modified after selection | @@ -125,7 +128,9 @@ Decisions: - Shared guidance has one canonical source, `src/agents/instruction-source.ts`. The plugin and the standalone setup are renderings of it, not separate sources. -- Installing the plugin does not migrate or remove standalone guidance. See +- The `hack` plugin never declares MCP. Plugin MCP ships only in `hack-mcp`, so a + client gets a Hack MCP server only when someone selects it. +- Installing a plugin does not migrate or remove standalone guidance. See [integration setup](../integrations.md). - The MCP bundle is an alternative launch for MCP only. It does not carry skills, hooks or rules. @@ -144,7 +149,7 @@ artifacts must not set them: Hack-shipped hooks may run only `hack agent prime`. `tests/agent-plugin-generation.test.ts` enforces this for the generated guidance -surfaces: `plugins/hack`, `.claude-plugin`, `.cursor-plugin`, `.agents` (including +surfaces: `plugins/hack`, `plugins/hack-mcp`, `.claude-plugin`, `.cursor-plugin`, `.agents` (including the skills it links to), and the agent configuration in `examples/basic/.claude`, `.codex` and `.cursor`. It parses JSON, JSON Lines, TOML, YAML and markdown frontmatter. It fails on structured-config formats it cannot parse, such as INI, diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index 8c448dc1..cf68a626 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -157,13 +157,12 @@ Agent integration maintenance: - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. -Agent setup (CLI-first): -- Cursor rules: `hack setup cursor` -- Claude hooks: `hack setup claude` -- Codex skill: `hack setup codex` +Agent setup (plugin-first): +- Default integration: the optional `hack` plugin, installed through the client's plugin manager, ships skills, Claude primer hooks and a Cursor rule. It ships no MCP server, and Hack commands never install plugins. +- MCP is opt-in, only for clients without shell access and only when explicitly selected: add the separate `hack-mcp` plugin, or use `hack setup mcp` (client and scope via `hack mcp install --help`). Do not configure other clients as a side effect. +- Standalone export is an opt-in alternative to the plugin: `hack setup cursor` (rules), `hack setup claude` (hooks and skill), `hack setup codex` (skills) or `hack setup agents` (AGENTS.md/CLAUDE.md snippets). Use one integration path per client. +- Setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks for them. - Agent-assisted onboarding: `hack init --with claude|codex` (new repos) or `hack agent onboard` (existing projects) print/hand off the full setup prompt; the `/hack-init` skill and the `hack-init` MCP prompt return the same content. - Init prompt: `hack agent init` (use --client cursor|claude|codex to open) - Init patterns: `hack agent patterns` -- MCP (no-shell only): `hack setup mcp` -- MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect. diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index 8c448dc1..cf68a626 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -157,13 +157,12 @@ Agent integration maintenance: - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. -Agent setup (CLI-first): -- Cursor rules: `hack setup cursor` -- Claude hooks: `hack setup claude` -- Codex skill: `hack setup codex` +Agent setup (plugin-first): +- Default integration: the optional `hack` plugin, installed through the client's plugin manager, ships skills, Claude primer hooks and a Cursor rule. It ships no MCP server, and Hack commands never install plugins. +- MCP is opt-in, only for clients without shell access and only when explicitly selected: add the separate `hack-mcp` plugin, or use `hack setup mcp` (client and scope via `hack mcp install --help`). Do not configure other clients as a side effect. +- Standalone export is an opt-in alternative to the plugin: `hack setup cursor` (rules), `hack setup claude` (hooks and skill), `hack setup codex` (skills) or `hack setup agents` (AGENTS.md/CLAUDE.md snippets). Use one integration path per client. +- Setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks for them. - Agent-assisted onboarding: `hack init --with claude|codex` (new repos) or `hack agent onboard` (existing projects) print/hand off the full setup prompt; the `/hack-init` skill and the `hack-init` MCP prompt return the same content. - Init prompt: `hack agent init` (use --client cursor|claude|codex to open) - Init patterns: `hack agent patterns` -- MCP (no-shell only): `hack setup mcp` -- MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect. diff --git a/plugins/hack-mcp/.claude-plugin/plugin.json b/plugins/hack-mcp/.claude-plugin/plugin.json new file mode 100644 index 00000000..38ec252d --- /dev/null +++ b/plugins/hack-mcp/.claude-plugin/plugin.json @@ -0,0 +1,14 @@ +{ + "name": "hack-mcp", + "version": "4.2.1", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access.", + "author": { + "name": "Hack Dance LLC", + "url": "https://github.com/hack-dance" + }, + "homepage": "https://github.com/hack-dance/hack", + "repository": "https://github.com/hack-dance/hack", + "license": "MIT", + "keywords": ["local-development", "developer-tools", "agents", "mcp"], + "mcpServers": "./.mcp.json" +} diff --git a/plugins/hack-mcp/.codex-plugin/plugin.json b/plugins/hack-mcp/.codex-plugin/plugin.json new file mode 100644 index 00000000..cd5f3ebd --- /dev/null +++ b/plugins/hack-mcp/.codex-plugin/plugin.json @@ -0,0 +1,25 @@ +{ + "name": "hack-mcp", + "version": "4.2.1", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access.", + "author": { + "name": "Hack Dance LLC", + "url": "https://github.com/hack-dance" + }, + "homepage": "https://github.com/hack-dance/hack", + "repository": "https://github.com/hack-dance/hack", + "license": "MIT", + "keywords": ["local-development", "developer-tools", "agents", "mcp"], + "interface": { + "displayName": "Hack MCP", + "shortDescription": "Hack tools over MCP", + "longDescription": "Opt-in MCP server for Codex sessions without shell access. Install the hack plugin for skills; prefer the Hack CLI whenever shell access is available.", + "developerName": "Hack Dance LLC", + "category": "Developer Tools", + "capabilities": ["MCP tools"], + "websiteURL": "https://github.com/hack-dance/hack", + "defaultPrompt": ["List my Hack projects and their status."], + "brandColor": "#FF5C35" + }, + "mcpServers": "./.mcp.json" +} diff --git a/plugins/hack-mcp/.cursor-plugin/plugin.json b/plugins/hack-mcp/.cursor-plugin/plugin.json new file mode 100644 index 00000000..99e6c4d7 --- /dev/null +++ b/plugins/hack-mcp/.cursor-plugin/plugin.json @@ -0,0 +1,13 @@ +{ + "name": "hack-mcp", + "version": "4.2.1", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access.", + "author": { + "name": "Hack Dance LLC" + }, + "homepage": "https://github.com/hack-dance/hack", + "repository": "https://github.com/hack-dance/hack", + "license": "MIT", + "keywords": ["local-development", "developer-tools", "agents", "mcp"], + "mcpServers": "./.mcp.json" +} diff --git a/plugins/hack/.mcp.json b/plugins/hack-mcp/.mcp.json similarity index 100% rename from plugins/hack/.mcp.json rename to plugins/hack-mcp/.mcp.json diff --git a/plugins/hack-mcp/README.md b/plugins/hack-mcp/README.md new file mode 100644 index 00000000..0ee90296 --- /dev/null +++ b/plugins/hack-mcp/README.md @@ -0,0 +1,20 @@ +# Hack MCP plugin + +Opt-in companion to the `hack` plugin. It declares one stdio MCP server, +`hack mcp serve`, for Codex, Claude Code and Cursor sessions without shell +access. It ships no skills, hooks or rules; those stay in the `hack` plugin. + +Install it only when MCP is wanted. With shell access, the `hack` plugin and the +Hack CLI cover the same workflows without a server process. Install the `hack` +executable first; the server resolves `hack` from the client's PATH. Keep one +Hack MCP registration per client: do not add this plugin next to an existing +`hack mcp install` entry for the same client and scope. + +Portable clients read `plugin.json` and `mcp.json`; other clients read their +`.-plugin/plugin.json` and `.mcp.json`. Verify in a fresh session that +the server initializes and lists Hack tools. A marketplace listing alone does not +prove that it loaded. + +Generated by `bun run generate:agent-plugins` from +`scripts/generate-agent-plugins.ts`; do not edit these files directly. See +[integration setup](../../docs/integrations.md#optional-native-plugins). diff --git a/plugins/hack/mcp.json b/plugins/hack-mcp/mcp.json similarity index 100% rename from plugins/hack/mcp.json rename to plugins/hack-mcp/mcp.json diff --git a/plugins/hack-mcp/plugin.json b/plugins/hack-mcp/plugin.json new file mode 100644 index 00000000..9fdecf4f --- /dev/null +++ b/plugins/hack-mcp/plugin.json @@ -0,0 +1,29 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "hack-mcp", + "version": "4.2.1", + "description": "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access.", + "author": { + "name": "Hack Dance LLC", + "url": "https://github.com/hack-dance" + }, + "homepage": "https://github.com/hack-dance/hack", + "repository": "https://github.com/hack-dance/hack", + "license": "MIT", + "keywords": ["local-development", "developer-tools", "agents", "mcp"], + "extensions": { + "com.openai": { + "interface": { + "displayName": "Hack MCP", + "shortDescription": "Hack tools over MCP", + "longDescription": "Opt-in MCP server for Codex sessions without shell access. Install the hack plugin for skills; prefer the Hack CLI whenever shell access is available.", + "developerName": "Hack Dance LLC", + "category": "Developer Tools", + "capabilities": ["MCP tools"], + "websiteURL": "https://github.com/hack-dance/hack", + "defaultPrompt": ["List my Hack projects and their status."], + "brandColor": "#FF5C35" + } + } + } +} diff --git a/plugins/hack/.claude-plugin/plugin.json b/plugins/hack/.claude-plugin/plugin.json index 0705b80c..ddc74a4e 100644 --- a/plugins/hack/.claude-plugin/plugin.json +++ b/plugins/hack/.claude-plugin/plugin.json @@ -11,6 +11,5 @@ "license": "MIT", "keywords": ["local-development", "docker", "developer-tools", "agents"], "skills": "./skills/", - "hooks": "./hooks/claude-hooks.json", - "mcpServers": "./.mcp.json" + "hooks": "./hooks/claude-hooks.json" } diff --git a/plugins/hack/.codex-plugin/plugin.json b/plugins/hack/.codex-plugin/plugin.json index 52214e5b..9345f06b 100644 --- a/plugins/hack/.codex-plugin/plugin.json +++ b/plugins/hack/.codex-plugin/plugin.json @@ -17,7 +17,7 @@ "longDescription": "Operate Hack-managed services, logs, environments, workspaces, and agent-assisted project onboarding from Codex.", "developerName": "Hack Dance LLC", "category": "Developer Tools", - "capabilities": ["Local development", "Project onboarding", "MCP tools"], + "capabilities": ["Local development", "Project onboarding"], "websiteURL": "https://github.com/hack-dance/hack", "defaultPrompt": [ "Start this Hack project and verify its services.", @@ -25,6 +25,5 @@ "Diagnose this Hack project's local runtime." ], "brandColor": "#FF5C35" - }, - "mcpServers": "./.mcp.json" + } } diff --git a/plugins/hack/.cursor-plugin/plugin.json b/plugins/hack/.cursor-plugin/plugin.json index 08493917..2e01315c 100644 --- a/plugins/hack/.cursor-plugin/plugin.json +++ b/plugins/hack/.cursor-plugin/plugin.json @@ -10,6 +10,5 @@ "license": "MIT", "keywords": ["local-development", "docker", "developer-tools", "agents"], "skills": "./skills/", - "rules": "./rules/", - "mcpServers": "./.mcp.json" + "rules": "./rules/" } diff --git a/plugins/hack/README.md b/plugins/hack/README.md index 7533f828..2b117beb 100644 --- a/plugins/hack/README.md +++ b/plugins/hack/README.md @@ -1,7 +1,8 @@ # Hack agent plugin -Optional Hack integration for Codex, Claude Code, and Cursor. Install the `hack` -executable first; the plugin uses it from PATH. +Optional Hack integration for Codex, Claude Code, and Cursor, and the default way to +give an agent Hack guidance. Install the `hack` executable first; the plugin uses it +from PATH. The bundle contains four skills: @@ -10,25 +11,24 @@ The bundle contains four skills: - `hack-install`: installation, stable/prerelease selection, upgrade and rollback. - `hack-remote`: prepare an existing project on another machine or in Codex Cloud. -It also contains `hack mcp serve`, Claude primer hooks, and a Cursor rule selected -when relevant. It is generated from Hack's canonical guidance and versioned with -the CLI. Portable clients use root `plugin.json` and `mcp.json`; older clients -retain their compatibility manifests. The repository marketplace is not evidence -of an OpenAI directory listing. +It also contains Claude primer hooks and a Cursor rule selected when relevant. It +ships no MCP server: no `mcpServers` declaration and no `.mcp.json` or `mcp.json` +file. It is generated from Hack's canonical guidance and versioned with the CLI. +Portable clients use root `plugin.json`; older clients retain their compatibility +manifests. The repository marketplace is not evidence of an OpenAI directory listing. -## Private Codex installation +## MCP is a separate plugin -If the client already has a Hack MCP registration, or only needs skills, retain -that selection and disable the bundle's MCP entry in the client configuration: +Install the companion [`hack-mcp`](../hack-mcp/README.md) plugin only when MCP is +wanted, usually for a client without shell access. With shell access, these skills +and the Hack CLI cover the same workflows. Keep one Hack MCP registration per client: +either `hack-mcp` or one `hack mcp install` entry, not both. -```toml -[plugins."hack@hack-dance".mcp_servers.hack] -enabled = false -``` +Earlier releases of this plugin bundled `hack mcp serve`. Updating removes that +server. Add `hack-mcp` if you relied on it. A client setting that disabled the old +bundled server has nothing left to disable and can be removed. -This keeps the four plugin skills available without launching a second bundled -server. A standalone registration remains independently configured. Preserve -existing customized skills; resolve overlapping installation paths deliberately. +## Private Codex installation From a reviewed checkout: @@ -38,13 +38,13 @@ codex plugin add hack@hack-dance codex plugin list --json --marketplace hack-dance ``` -Start a fresh client, confirm all four plugin skills load, and call a harmless -MCP tool such as `hack.projects.list` against an isolated registry. Inspect -actual discovered tool names first. -The server and Claude hooks still resolve `hack` from the client's PATH; installing -the plugin does not switch them to `hack-next`. Use the explicit candidate CLI -for candidate work. A separately configured MCP integration must pin the intended -executable and home before it can operate on that channel. +Start a fresh client and confirm all four plugin skills load. If you also installed +`hack-mcp`, call a harmless MCP tool such as `hack.projects.list` against an isolated +registry, after inspecting the actual discovered tool names. +The Claude hooks and the `hack-mcp` server still resolve `hack` from the client's +PATH; installing a plugin does not switch them to `hack-next`. Use the explicit +candidate CLI for candidate work. A separately configured MCP integration must pin +the intended executable and home before it can operate on that channel. Personal local plugins do not automatically appear in Codex Cloud. Make the needed skills available in the cloud repository/environment and verify a new @@ -54,13 +54,13 @@ Full runtime availability and app acceptance must be verified on that host. ## Distribution boundary -This bundle uses local stdio MCP. Public OpenAI submission currently requires an -HTTPS MCP endpoint or coordination with OpenAI for local MCP support; see the +This bundle has no MCP server. The separate `hack-mcp` bundle uses local stdio MCP; +public OpenAI submission of a local MCP server currently requires an HTTPS MCP +endpoint or coordination with OpenAI; see the [official packaging guidance](https://developers.openai.com/plugins/build/plugins). -Do not expose a workstation runtime publicly or remove MCP just to submit a -skills-only listing. Confirm any existing account registration before creating -another. Private installation, account registration and public publication are -separate steps. +Do not expose a workstation runtime publicly. Confirm any existing account +registration before creating another. Private installation, account registration +and public publication are separate steps. See [installation and migration guidance](../../docs/integrations.md#optional-native-plugins). Existing standalone setup remains supported. Installation never deletes legacy @@ -68,7 +68,8 @@ content, changes global policy, or triggers broad integration sync. Use one inte path per client to avoid duplicate skills, hooks, and MCP registrations. Contributors: run `bun run generate:agent-plugins` after changing canonical guidance. -Root portable manifests are derived from `.codex-plugin/plugin.json` and -`.mcp.json`; edit those sources and regenerate. Do not edit generated skills, -portable manifests or rules directly. Validate actual component loading in -a fresh client session before describing an installation as working. +The root portable manifest is derived from `.codex-plugin/plugin.json`; edit that +source and regenerate. The generator refuses a client manifest that declares +`mcpServers`; MCP belongs in `hack-mcp`. Do not edit generated skills, portable +manifests or rules directly. Validate actual component loading in a fresh client +session before describing an installation as working. diff --git a/plugins/hack/plugin.json b/plugins/hack/plugin.json index 3a6867ed..b1d7e4df 100644 --- a/plugins/hack/plugin.json +++ b/plugins/hack/plugin.json @@ -19,11 +19,7 @@ "longDescription": "Operate Hack-managed services, logs, environments, workspaces, and agent-assisted project onboarding from Codex.", "developerName": "Hack Dance LLC", "category": "Developer Tools", - "capabilities": [ - "Local development", - "Project onboarding", - "MCP tools" - ], + "capabilities": ["Local development", "Project onboarding"], "websiteURL": "https://github.com/hack-dance/hack", "defaultPrompt": [ "Start this Hack project and verify its services.", diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index 77e495ec..4b13fbed 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). ## Product boundary diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index 6a5159aa..633f93ef 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `444f2cb158b2` (version alone is not a freshness guarantee). ## Product boundary @@ -185,16 +185,15 @@ Use `hack` as the primary interface for local-first development. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. -## Agent setup (CLI-first) +## Agent setup (plugin-first) -- Cursor rules: `hack setup cursor` -- Claude hooks: `hack setup claude` -- Codex skill: `hack setup codex` +- Default integration: the optional `hack` plugin, installed through the client's plugin manager, ships skills, Claude primer hooks and a Cursor rule. It ships no MCP server, and Hack commands never install plugins. +- MCP is opt-in, only for clients without shell access and only when explicitly selected: add the separate `hack-mcp` plugin, or use `hack setup mcp` (client and scope via `hack mcp install --help`). Do not configure other clients as a side effect. +- Standalone export is an opt-in alternative to the plugin: `hack setup cursor` (rules), `hack setup claude` (hooks and skill), `hack setup codex` (skills) or `hack setup agents` (AGENTS.md/CLAUDE.md snippets). Use one integration path per client. +- Setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks for them. - Agent-assisted onboarding: `hack init --with claude|codex` (new repos) or `hack agent onboard` (existing projects) print/hand off the full setup prompt; the `/hack-init` skill and the `hack-init` MCP prompt return the same content. - Init prompt: `hack agent init` (use --client cursor|claude|codex to open) - Init patterns: `hack agent patterns` -- MCP (no-shell only): `hack setup mcp` -- MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect. ## References diff --git a/plugins/hack/skills/hack-init/SKILL.md b/plugins/hack/skills/hack-init/SKILL.md index 0453fac6..1b186c46 100644 --- a/plugins/hack/skills/hack-init/SKILL.md +++ b/plugins/hack/skills/hack-init/SKILL.md @@ -12,6 +12,6 @@ Fetch the current onboarding guidance from the installed Hack CLI. 1. Fetch the canonical onboarding prompt: - With shell access: run `hack agent onboard` (works for new and existing projects). - - Without shell access: fetch the `hack-init` MCP prompt from the hack MCP server. + - Without shell access: only if the `hack-mcp` plugin or another Hack MCP server is installed, fetch the `hack-init` MCP prompt from it; otherwise report that onboarding needs shell access or an explicitly selected Hack MCP server. 2. Apply the relevant phases to the authorized workflow; preserve native approvals, credential safety, and runtime ownership. 3. Verify the requested services and host workflows. Report remaining blockers; unrelated optional integration warnings do not prevent completion. diff --git a/plugins/hack/skills/hack-install/SKILL.md b/plugins/hack/skills/hack-install/SKILL.md index 02f401b7..f8fb8f7f 100644 --- a/plugins/hack/skills/hack-install/SKILL.md +++ b/plugins/hack/skills/hack-install/SKILL.md @@ -6,8 +6,9 @@ description: Install, upgrade, or roll back Hack; diagnose a missing CLI or plug # Install and select Hack Complete the requested installation and verify the exact executable. The plugin -provides instructions and a local MCP connection; installing it does not install -Hack, Docker, a native provider, DNS, or certificate trust. +provides instructions; the separate, opt-in `hack-mcp` plugin provides a local MCP +connection. Installing either does not install Hack, Docker, a native provider, +DNS, or certificate trust. ## Identify the target @@ -58,7 +59,7 @@ After rollback, verify the application's retained data through the restored runt ## Plugin and MCP selection -The bundled MCP server and Claude hook run `hack` from the client process's PATH. +The `hack-mcp` server and the Claude hook run `hack` from the client process's PATH. They do not automatically follow `hack-next`. Prefer the explicit selected CLI when shell access is available. Never use a stable MCP tool to mutate a candidate project, or infer channel selection from the skill/plugin version. diff --git a/scripts/generate-agent-client-capabilities.ts b/scripts/generate-agent-client-capabilities.ts index 2ad6a7af..aa52ed7c 100644 --- a/scripts/generate-agent-client-capabilities.ts +++ b/scripts/generate-agent-client-capabilities.ts @@ -107,11 +107,11 @@ export function renderAgentClientCapabilitySection({ "", "### Clients", "", - "| Client | Manifest |", - "| --- | --- |", + "| Client | `hack` plugin manifest | `hack-mcp` plugin manifest |", + "| --- | --- | --- |", ...AGENT_CLIENT_DEFINITIONS.map( (definition) => - `| ${escapeCell(definition.label)} | \`${definition.manifestPath}\` |` + `| ${escapeCell(definition.label)} | \`${definition.manifestPath}\` | \`${definition.mcpManifestPath}\` |` ), "", "### Capabilities", diff --git a/scripts/generate-agent-plugins.ts b/scripts/generate-agent-plugins.ts index 1a8e8520..a2d4cd21 100644 --- a/scripts/generate-agent-plugins.ts +++ b/scripts/generate-agent-plugins.ts @@ -53,35 +53,207 @@ export async function generateAgentPlugins({ } await writeSkillReferences({ pluginRoot }); - const manifestPaths = [ - resolve(pluginRoot, ".codex-plugin", "plugin.json"), - resolve(pluginRoot, ".claude-plugin", "plugin.json"), - resolve(pluginRoot, ".cursor-plugin", "plugin.json"), - ] as const; - - for (const manifestPath of manifestPaths) { - const manifest = (await Bun.file(manifestPath).json()) as Record< - string, - unknown - >; + for (const relative of SKILLS_PACKAGE_MANIFESTS) { + const manifestPath = resolve(pluginRoot, relative); + const manifest: unknown = await Bun.file(manifestPath).json(); + if (!isRecord(manifest)) { + throw new Error("Expected a hack plugin client manifest object"); + } + if ("mcpServers" in manifest) { + throw new Error( + "The hack plugin is skills-only; declare MCP in the hack-mcp package" + ); + } manifest.version = version; - await Bun.write(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); - await Bun.$`${process.execPath} x --no-install biome format --write ${manifestPath}`.quiet(); + await writeFormattedJson({ path: manifestPath, value: manifest }); } - // Keep legacy clients working while deriving portable files from the same source. - const codexManifest: unknown = await Bun.file(manifestPaths[0]).json(); - const mcpManifest: unknown = await Bun.file( - resolve(pluginRoot, ".mcp.json") + // The portable manifest derives from the Codex overlay. MCP is not part of + // the skills package; remove copies a pre-hack-mcp checkout still carries. + const codexManifest: unknown = await Bun.file( + resolve(pluginRoot, SKILLS_PACKAGE_MANIFESTS[0]) ).json(); - for (const [name, manifest] of [ - ["plugin.json", portablePluginManifest(codexManifest)], - ["mcp.json", portableMcpManifest(mcpManifest)], - ] as const) { - const path = resolve(pluginRoot, name); - await Bun.write(path, `${JSON.stringify(manifest, null, 2)}\n`); - await Bun.$`${process.execPath} x --no-install biome format --write ${path}`.quiet(); + await writeFormattedJson({ + path: resolve(pluginRoot, "plugin.json"), + value: portablePluginManifest(codexManifest), + }); + for (const retired of RETIRED_SKILLS_PACKAGE_FILES) { + await rm(resolve(pluginRoot, retired), { force: true }); + } + + await generateMcpPlugin({ + pluginRoot: resolve(repoRoot, "plugins", HACK_MCP_PLUGIN_NAME), + version, + }); +} + +/** Default plugin: skills, Claude primer hooks and a Cursor rule; no MCP. */ +export const HACK_PLUGIN_NAME = "hack"; + +/** Opt-in companion plugin that only declares the Hack MCP server. */ +export const HACK_MCP_PLUGIN_NAME = "hack-mcp"; + +/** Client manifests of the skills package; Codex first as the portable source. */ +const SKILLS_PACKAGE_MANIFESTS = [ + ".codex-plugin/plugin.json", + ".claude-plugin/plugin.json", + ".cursor-plugin/plugin.json", +] as const; + +/** MCP declarations the skills package shipped before hack-mcp existed. */ +const RETIRED_SKILLS_PACKAGE_FILES = [".mcp.json", "mcp.json"] as const; + +/** + * The single source of truth for the MCP launch Hack packages as a plugin: + * the `hack` executable from the client's PATH, serving stdio. Every hack-mcp + * client manifest and both MCP files render from this value. + */ +export const HACK_MCP_SERVER = { + name: "hack", + command: "hack", + args: ["mcp", "serve"], +} as const; + +/** Every file the generator owns in plugins/hack-mcp, relative to its root. */ +export const HACK_MCP_PLUGIN_FILES = [ + ".claude-plugin/plugin.json", + ".codex-plugin/plugin.json", + ".cursor-plugin/plugin.json", + ".mcp.json", + "README.md", + "mcp.json", + "plugin.json", +] as const; + +const HACK_MCP_DESCRIPTION = + "Opt-in Hack MCP server (`hack mcp serve`) for agent clients without shell access."; +const HACK_MCP_KEYWORDS = [ + "local-development", + "developer-tools", + "agents", + "mcp", +] as const; +const HACK_HOMEPAGE = "https://github.com/hack-dance/hack"; +const HACK_AUTHOR = { + name: "Hack Dance LLC", + url: "https://github.com/hack-dance", +} as const; +const MCP_DECLARATION = "./.mcp.json"; + +/** Client-compatible `.mcp.json` rendered from {@link HACK_MCP_SERVER}. */ +export function renderMcpServers(): Record { + return { + mcpServers: { + [HACK_MCP_SERVER.name]: { + command: HACK_MCP_SERVER.command, + args: [...HACK_MCP_SERVER.args], + }, + }, + }; +} + +/** Client manifests of the hack-mcp package for one release version. */ +export function renderMcpPluginManifests({ + version, +}: { + readonly version: string; +}): Readonly>> { + const identity = { + name: HACK_MCP_PLUGIN_NAME, + version, + description: HACK_MCP_DESCRIPTION, + author: { ...HACK_AUTHOR }, + homepage: HACK_HOMEPAGE, + repository: HACK_HOMEPAGE, + license: "MIT", + keywords: [...HACK_MCP_KEYWORDS], + }; + return { + codex: { + ...identity, + interface: { + displayName: "Hack MCP", + shortDescription: "Hack tools over MCP", + longDescription: + "Opt-in MCP server for Codex sessions without shell access. Install the hack plugin for skills; prefer the Hack CLI whenever shell access is available.", + developerName: HACK_AUTHOR.name, + category: "Developer Tools", + capabilities: ["MCP tools"], + websiteURL: HACK_HOMEPAGE, + defaultPrompt: ["List my Hack projects and their status."], + brandColor: "#FF5C35", + }, + mcpServers: MCP_DECLARATION, + }, + claude: { ...identity, mcpServers: MCP_DECLARATION }, + cursor: { + ...identity, + author: { name: HACK_AUTHOR.name }, + mcpServers: MCP_DECLARATION, + }, + }; +} + +/** README of the hack-mcp package. */ +export function renderMcpPluginReadme(): string { + const launch = [HACK_MCP_SERVER.command, ...HACK_MCP_SERVER.args].join(" "); + return `# Hack MCP plugin + +Opt-in companion to the \`hack\` plugin. It declares one stdio MCP server, +\`${launch}\`, for Codex, Claude Code and Cursor sessions without shell +access. It ships no skills, hooks or rules; those stay in the \`hack\` plugin. + +Install it only when MCP is wanted. With shell access, the \`hack\` plugin and the +Hack CLI cover the same workflows without a server process. Install the \`hack\` +executable first; the server resolves \`hack\` from the client's PATH. Keep one +Hack MCP registration per client: do not add this plugin next to an existing +\`hack mcp install\` entry for the same client and scope. + +Portable clients read \`plugin.json\` and \`mcp.json\`; other clients read their +\`.-plugin/plugin.json\` and \`.mcp.json\`. Verify in a fresh session that +the server initializes and lists Hack tools. A marketplace listing alone does not +prove that it loaded. + +Generated by \`bun run generate:agent-plugins\` from +\`scripts/generate-agent-plugins.ts\`; do not edit these files directly. See +[integration setup](../../docs/integrations.md#optional-native-plugins). +`; +} + +/** Write the complete hack-mcp package; every file renders from code. */ +async function generateMcpPlugin({ + pluginRoot, + version, +}: { + readonly pluginRoot: string; + readonly version: string; +}): Promise { + const manifests = renderMcpPluginManifests({ version }); + const servers = renderMcpServers(); + const files: readonly (readonly [string, unknown])[] = [ + [".codex-plugin/plugin.json", manifests.codex], + [".claude-plugin/plugin.json", manifests.claude], + [".cursor-plugin/plugin.json", manifests.cursor], + [".mcp.json", servers], + ["plugin.json", portablePluginManifest(manifests.codex)], + ["mcp.json", portableMcpManifest(servers)], + ]; + for (const [relative, value] of files) { + await writeFormattedJson({ path: resolve(pluginRoot, relative), value }); } + await Bun.write(resolve(pluginRoot, "README.md"), renderMcpPluginReadme()); +} + +async function writeFormattedJson({ + path, + value, +}: { + readonly path: string; + readonly value: unknown; +}): Promise { + await Bun.$`mkdir -p ${dirname(path)}`; + await Bun.write(path, `${JSON.stringify(value, null, 2)}\n`); + await Bun.$`${process.execPath} x --no-install biome format --write ${path}`.quiet(); } /** Preserve effective OpenAI settings: an inline extension replaces the overlay. */ diff --git a/src/agents/client-capabilities.ts b/src/agents/client-capabilities.ts index 657ee65f..3c7aaa34 100644 --- a/src/agents/client-capabilities.ts +++ b/src/agents/client-capabilities.ts @@ -8,7 +8,9 @@ * `isolated-fixture` or stronger. A manifest parse, schema check or * marketplace listing is never component-loading evidence. * - For the named native-plugin clients, a component is `unsupported` exactly - * when that client's plugin manifest does not declare it. + * when that client's plugin manifest does not declare it. Plugin MCP is read + * from the opt-in `hack-mcp` manifest; the default `hack` skills manifest + * never declares MCP. * - docs/reference/agent-client-capabilities.md matches this data; rewrite its * generated section with `bun run docs:agent-client-capabilities`. * @@ -19,7 +21,7 @@ */ /** Increment when cells are reclassified or the cell shape changes. */ -export const AGENT_CLIENT_CAPABILITY_MATRIX_VERSION = 1; +export const AGENT_CLIENT_CAPABILITY_MATRIX_VERSION = 2; /** Date (UTC) of the observations recorded below. */ export const AGENT_CLIENT_CAPABILITY_OBSERVED_ON = "2026-10-11"; @@ -61,8 +63,10 @@ export type EvidenceClass = (typeof EVIDENCE_CLASSES)[number]; export interface AgentClientDefinition { readonly client: AgentClient; readonly label: string; - /** Repository-relative plugin manifest Hack ships for this client. */ + /** Repository-relative manifest of the default `hack` (skills) plugin. */ readonly manifestPath: string; + /** Repository-relative manifest of the opt-in `hack-mcp` plugin. */ + readonly mcpManifestPath: string; } export interface AgentClientCapabilityDefinition { @@ -94,21 +98,25 @@ export const AGENT_CLIENT_DEFINITIONS: readonly AgentClientDefinition[] = [ client: "codex", label: "Codex", manifestPath: "plugins/hack/.codex-plugin/plugin.json", + mcpManifestPath: "plugins/hack-mcp/.codex-plugin/plugin.json", }, { client: "claude", label: "Claude Code", manifestPath: "plugins/hack/.claude-plugin/plugin.json", + mcpManifestPath: "plugins/hack-mcp/.claude-plugin/plugin.json", }, { client: "cursor", label: "Cursor", manifestPath: "plugins/hack/.cursor-plugin/plugin.json", + mcpManifestPath: "plugins/hack-mcp/.cursor-plugin/plugin.json", }, { client: "standalone", label: "Standalone (portable manifest or generic stdio MCP client)", manifestPath: "plugins/hack/plugin.json", + mcpManifestPath: "plugins/hack-mcp/plugin.json", }, ]; @@ -118,7 +126,7 @@ export const AGENT_CLIENT_CAPABILITY_DEFINITIONS: readonly AgentClientCapability capability: "packaging", label: "Packaging", meaning: - "The client accepts, installs and enables the Hack plugin package in a session.", + "The client accepts, installs and enables the default `hack` plugin package in a session.", }, { capability: "skillDiscovery", @@ -136,7 +144,7 @@ export const AGENT_CLIENT_CAPABILITY_DEFINITIONS: readonly AgentClientCapability capability: "pluginMcp", label: "Plugin MCP", meaning: - "The client starts the plugin-declared `hack mcp serve` server and lists its tools.", + "The client starts the `hack mcp serve` server declared by the opt-in `hack-mcp` plugin and lists its tools. The default `hack` plugin ships no MCP.", }, { capability: "standaloneMcp", @@ -158,6 +166,10 @@ export const AGENT_CLIENT_CAPABILITY_DEFINITIONS: readonly AgentClientCapability ]; const SOURCE_SHA = "ec56769bdc046f5b64e621eb04613cfb00a65ced"; +/** Base of the change that moved plugin MCP into the `hack-mcp` package. */ +const MCP_PACKAGE_SOURCE_SHA = "3e261d517ed78fbf55c786682ee95291021cba63"; +const MCP_PACKAGE_NOTE = + "The opt-in `hack-mcp` manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); the default `hack` plugin declares no MCP. No plugin session started the server."; const HOST = "macos-arm64-contributor"; const CODEX_VERSION = "codex-cli 0.162.1"; const CLAUDE_VERSION = "Claude Code 2.1.296"; @@ -210,11 +222,11 @@ export const AGENT_CLIENT_CAPABILITY_MATRIX: readonly ClientCapabilityCell[] = [ capability: "pluginMcp", status: "untested", evidenceClass: "schema", - evidenceRef: "plugins/hack/.mcp.json", + evidenceRef: "plugins/hack-mcp/.mcp.json", observedClientVersion: CODEX_VERSION, - sourceSha: SOURCE_SHA, + sourceSha: MCP_PACKAGE_SOURCE_SHA, hostLabel: HOST, - note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + note: MCP_PACKAGE_NOTE, openGate: PLUGIN_LOAD_GATE, }, { @@ -297,11 +309,11 @@ export const AGENT_CLIENT_CAPABILITY_MATRIX: readonly ClientCapabilityCell[] = [ capability: "pluginMcp", status: "untested", evidenceClass: "schema", - evidenceRef: "plugins/hack/.mcp.json", + evidenceRef: "plugins/hack-mcp/.mcp.json", observedClientVersion: CLAUDE_VERSION, - sourceSha: SOURCE_SHA, + sourceSha: MCP_PACKAGE_SOURCE_SHA, hostLabel: HOST, - note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + note: MCP_PACKAGE_NOTE, openGate: PLUGIN_LOAD_GATE, }, { @@ -383,11 +395,11 @@ export const AGENT_CLIENT_CAPABILITY_MATRIX: readonly ClientCapabilityCell[] = [ capability: "pluginMcp", status: "untested", evidenceClass: "schema", - evidenceRef: "plugins/hack/.mcp.json", + evidenceRef: "plugins/hack-mcp/.mcp.json", observedClientVersion: CURSOR_VERSION, - sourceSha: SOURCE_SHA, + sourceSha: MCP_PACKAGE_SOURCE_SHA, hostLabel: HOST, - note: "Manifest points `mcpServers` at `.mcp.json` (`hack mcp serve`); no plugin session started it.", + note: MCP_PACKAGE_NOTE, openGate: PLUGIN_LOAD_GATE, }, { @@ -470,11 +482,11 @@ export const AGENT_CLIENT_CAPABILITY_MATRIX: readonly ClientCapabilityCell[] = [ capability: "pluginMcp", status: "untested", evidenceClass: "schema", - evidenceRef: "plugins/hack/mcp.json", + evidenceRef: "plugins/hack-mcp/mcp.json", observedClientVersion: null, - sourceSha: SOURCE_SHA, + sourceSha: MCP_PACKAGE_SOURCE_SHA, hostLabel: HOST, - note: "Portable MCP manifest adds an explicit stdio transport to the same launch as `.mcp.json`.", + note: "The opt-in `hack-mcp` portable MCP manifest adds an explicit stdio transport to the same launch as its `.mcp.json`; the default `hack` package ships no MCP file.", openGate: "Name a portable consumer and observe it start the declared server.", }, diff --git a/src/agents/hack-init-skill.ts b/src/agents/hack-init-skill.ts index 795b74b8..f6906281 100644 --- a/src/agents/hack-init-skill.ts +++ b/src/agents/hack-init-skill.ts @@ -10,8 +10,9 @@ import { /** * Agent-side `/hack-init` skill: a thin pointer that tells the agent to fetch - * the canonical onboarding prompt from the CLI (`hack agent onboard`) or the - * `hack-init` MCP prompt and follow it. The onboarding content itself lives in + * the canonical onboarding prompt from the CLI (`hack agent onboard`) or, only + * when a Hack MCP server such as the opt-in `hack-mcp` plugin is installed, the + * `hack-init` MCP prompt, and follow it. The onboarding content itself lives in * `src/agents/onboarding-prompt.ts` — never in the installed skill file. * * Installed for both Claude Code (`.claude/skills/hack-init/SKILL.md`) and @@ -73,7 +74,7 @@ export function renderHackInitSkill(): string { "", "1. Fetch the canonical onboarding prompt:", " - With shell access: run `hack agent onboard` (works for new and existing projects).", - " - Without shell access: fetch the `hack-init` MCP prompt from the hack MCP server.", + " - Without shell access: only if the `hack-mcp` plugin or another Hack MCP server is installed, fetch the `hack-init` MCP prompt from it; otherwise report that onboarding needs shell access or an explicitly selected Hack MCP server.", "2. Apply the relevant phases to the authorized workflow; preserve native approvals, credential safety, and runtime ownership.", "3. Verify the requested services and host workflows. Report remaining blockers; unrelated optional integration warnings do not prevent completion.", "", diff --git a/src/agents/install-skill.ts b/src/agents/install-skill.ts index 0ea2c097..cc8f934e 100644 --- a/src/agents/install-skill.ts +++ b/src/agents/install-skill.ts @@ -8,8 +8,9 @@ description: Install, upgrade, or roll back Hack; diagnose a missing CLI or plug # Install and select Hack Complete the requested installation and verify the exact executable. The plugin -provides instructions and a local MCP connection; installing it does not install -Hack, Docker, a native provider, DNS, or certificate trust. +provides instructions; the separate, opt-in \`hack-mcp\` plugin provides a local MCP +connection. Installing either does not install Hack, Docker, a native provider, +DNS, or certificate trust. ## Identify the target @@ -60,7 +61,7 @@ After rollback, verify the application's retained data through the restored runt ## Plugin and MCP selection -The bundled MCP server and Claude hook run \`hack\` from the client process's PATH. +The \`hack-mcp\` server and the Claude hook run \`hack\` from the client process's PATH. They do not automatically follow \`hack-next\`. Prefer the explicit selected CLI when shell access is available. Never use a stable MCP tool to mutate a candidate project, or infer channel selection from the skill/plugin version. diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 153f83eb..5733eb86 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -300,17 +300,16 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ }, { id: "agent-setup", - title: "Agent setup (CLI-first)", + title: "Agent setup (plugin-first)", surfaces: ["docs", "skill", "primer"], bullets: [ - "Cursor rules: `hack setup cursor`", - "Claude hooks: `hack setup claude`", - "Codex skill: `hack setup codex`", + "Default integration: the optional `hack` plugin, installed through the client's plugin manager, ships skills, Claude primer hooks and a Cursor rule. It ships no MCP server, and Hack commands never install plugins.", + "MCP is opt-in, only for clients without shell access and only when explicitly selected: add the separate `hack-mcp` plugin, or use `hack setup mcp` (client and scope via `hack mcp install --help`). Do not configure other clients as a side effect.", + "Standalone export is an opt-in alternative to the plugin: `hack setup cursor` (rules), `hack setup claude` (hooks and skill), `hack setup codex` (skills) or `hack setup agents` (AGENTS.md/CLAUDE.md snippets). Use one integration path per client.", + "Setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks for them.", "Agent-assisted onboarding: `hack init --with claude|codex` (new repos) or `hack agent onboard` (existing projects) print/hand off the full setup prompt; the `/hack-init` skill and the `hack-init` MCP prompt return the same content.", "Init prompt: `hack agent init` (use --client cursor|claude|codex to open)", "Init patterns: `hack agent patterns`", - "MCP (no-shell only): `hack setup mcp`", - "MCP install: choose the requested client and scope with `hack mcp install --help`; do not configure other clients as a side effect.", ], }, ]; diff --git a/src/agents/integration-ownership.ts b/src/agents/integration-ownership.ts index 0e1cbb07..e12f7b8f 100644 --- a/src/agents/integration-ownership.ts +++ b/src/agents/integration-ownership.ts @@ -105,10 +105,12 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "b6465c18c6d4f41b30a600b8", // v5.0.0-next.1..v5.0.0-next.2 "121d8f26374d24f2a09890a3", // v5.0.0-next.3..next "6449442a8f230412d4e5f9e7", // v5.0.0-next.4..next + "c60f1fc7b1f6579fac892a54", // next: plugin-first agent setup (hack-mcp split) ]), "hack-init-skill": new Set([ "cd15fb6d8ed8e25071d05e75", // v3.1.0..v4.0.3 "f8e3018bcb8546e802dfa19c", // v4.1.0..next + "dde7ffb2975e3257395f9506", // next: conditional hack-mcp fallback ]), "cursor-rules": new Set([ "6acabe0f41a9785374970d6c", // v1.1.0..v1.10.2 @@ -131,6 +133,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "01d7431b9a5caaba3ffd3463", // v5.0.0-next.1..v5.0.0-next.2 "06cf6d0920afc1780b9946ba", // v5.0.0-next.3..next "cf0a31e197f9034485413674", // v5.0.0-next.4..next + "77f4258cf2570443aee021fd", // next: plugin-first agent setup (hack-mcp split) ]), }; diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index 6f3a624e..2f76dcaa 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "3cb1b0eabdea"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "444f2cb158b2"; diff --git a/tests/agent-client-capabilities.test.ts b/tests/agent-client-capabilities.test.ts index 8b1653ea..8510e688 100644 --- a/tests/agent-client-capabilities.test.ts +++ b/tests/agent-client-capabilities.test.ts @@ -39,7 +39,14 @@ type IssueCode = | "verified-below-fixture" | "verified-on-manifest-or-listing" | "verified-with-open-gate" - | "manifest-disagreement"; + | "manifest-disagreement" + | "skills-package-declares-mcp"; + +/** A client's default `hack` (skills) and opt-in `hack-mcp` manifests. */ +interface ClientManifests { + readonly skills: Record; + readonly mcp: Record; +} interface ClassificationIssue { readonly client: AgentClient; @@ -88,10 +95,19 @@ function classificationIssues({ exists, }: { readonly cells: readonly ClientCapabilityCell[]; - readonly manifests: Readonly>>; + readonly manifests: Readonly>; readonly exists: (repoRelativePath: string) => boolean; }): ClassificationIssue[] { const issues: ClassificationIssue[] = []; + for (const client of NATIVE_PLUGIN_CLIENTS) { + if ("mcpServers" in (manifests[client]?.skills ?? {})) { + issues.push({ + client, + capability: "pluginMcp", + code: "skills-package-declares-mcp", + }); + } + } for (const client of AGENT_CLIENTS) { for (const capability of AGENT_CLIENT_CAPABILITIES) { const matches = cells.filter( @@ -148,7 +164,11 @@ function classificationIssues({ } } if (NATIVE_PLUGIN_CLIENTS.includes(cell.client)) { - const manifest = manifests[cell.client] ?? {}; + const packages = manifests[cell.client]; + // Plugin MCP ships only in the opt-in hack-mcp package. + const manifest = + (cell.capability === "pluginMcp" ? packages?.mcp : packages?.skills) ?? + {}; const key = MANIFEST_COMPONENT_KEYS[cell.capability]; const declared = cell.capability === "packaging" @@ -165,18 +185,21 @@ function classificationIssues({ return issues; } -async function loadManifests(): Promise< - Record> -> { - const manifests: Record> = {}; +async function readManifest(path: string): Promise> { + const parsed: unknown = await Bun.file(join(repoRoot, path)).json(); + if (!isRecord(parsed)) { + throw new Error(`Expected a manifest object: ${path}`); + } + return parsed; +} + +async function loadManifests(): Promise> { + const manifests: Record = {}; for (const definition of AGENT_CLIENT_DEFINITIONS) { - const parsed: unknown = await Bun.file( - join(repoRoot, definition.manifestPath) - ).json(); - if (!isRecord(parsed)) { - throw new Error(`Expected a manifest object: ${definition.manifestPath}`); - } - manifests[definition.client] = parsed; + manifests[definition.client] = { + skills: await readManifest(definition.manifestPath), + mcp: await readManifest(definition.mcpManifestPath), + }; } return manifests; } @@ -301,7 +324,7 @@ test("negative control: plugin skills verified on manifest-only evidence fail", test("manifest-declared components agree with the matrix", async () => { const manifests = await loadManifests(); - expect("hooks" in (manifests.codex ?? {})).toBe(false); + expect("hooks" in (manifests.codex?.skills ?? {})).toBe(false); const codexHooks = replaceCell("codex", "hooks", { status: "verified", evidenceClass: "isolated-fixture", @@ -357,6 +380,51 @@ test("manifest-declared components agree with the matrix", async () => { ).toEqual(["open-gate-missing"]); }); +test("negative control: plugin MCP is classified from hack-mcp, never the skills package", async () => { + const manifests = await loadManifests(); + for (const client of NATIVE_PLUGIN_CLIENTS) { + expect("mcpServers" in (manifests[client]?.skills ?? {})).toBe(false); + expect(manifests[client]?.mcp.mcpServers).toBe("./.mcp.json"); + } + + const claude = manifests.claude; + if (!claude) { + throw new Error("expected Claude manifests"); + } + const readded = { + ...manifests, + claude: { + ...claude, + skills: { ...claude.skills, mcpServers: "./.mcp.json" }, + }, + }; + expect( + codesFor( + classificationIssues({ + cells: AGENT_CLIENT_CAPABILITY_MATRIX, + manifests: readded, + exists: repoExists, + }), + "claude", + "pluginMcp" + ) + ).toEqual(["skills-package-declares-mcp"]); + + const { mcpServers: _removed, ...withoutServer } = claude.mcp; + const emptied = { ...manifests, claude: { ...claude, mcp: withoutServer } }; + expect( + codesFor( + classificationIssues({ + cells: AGENT_CLIENT_CAPABILITY_MATRIX, + manifests: emptied, + exists: repoExists, + }), + "claude", + "pluginMcp" + ) + ).toEqual(["manifest-disagreement"]); +}); + test("committed reference section matches the generator output", async () => { const doc = await Bun.file(referencePath).text(); expect(extractGeneratedSection(doc)).toBe( diff --git a/tests/agent-plugin-generation.test.ts b/tests/agent-plugin-generation.test.ts index 8a20add7..0f6b6465 100644 --- a/tests/agent-plugin-generation.test.ts +++ b/tests/agent-plugin-generation.test.ts @@ -6,8 +6,14 @@ import { parse as parseYaml } from "yaml"; import { generateAgentPlugins, + HACK_MCP_PLUGIN_FILES, + HACK_MCP_PLUGIN_NAME, + HACK_MCP_SERVER, + HACK_PLUGIN_NAME, portableMcpManifest, portablePluginManifest, + renderMcpPluginManifests, + renderMcpServers, renderPluginReferences, renderPluginRule, renderPluginSkill, @@ -50,30 +56,276 @@ async function seedPluginManifests(pluginRoot: string): Promise { }) ); } +} + +/** Every file under a directory, keyed by its relative path. */ +async function snapshotTree(root: string): Promise> { + const files: Record = {}; + const glob = new Bun.Glob("**/*"); + for await (const relative of glob.scan({ + cwd: root, + dot: true, + onlyFiles: true, + })) { + files[relative] = await Bun.file(join(root, relative)).text(); + } + return files; +} + +async function readJsonObject(path: string): Promise> { + const parsed: unknown = await Bun.file(path).json(); + if (!isRecord(parsed)) { + throw new Error("Expected a JSON object"); + } + return parsed; +} + +const sameJson = (left: unknown, right: unknown): boolean => + JSON.stringify(left) === JSON.stringify(right); + +/** Shape findings for the skills package: no MCP declaration or MCP file. */ +async function skillsPackageShapeIssues(root: string): Promise { + const issues: string[] = []; + for (const relative of [ + ...manifestDirectories.map((directory) => `${directory}/plugin.json`), + "plugin.json", + ]) { + const manifest = await readJsonObject(join(root, relative)); + if (manifest.name !== HACK_PLUGIN_NAME) { + issues.push(`hack:${relative}:name`); + } + if ("mcpServers" in manifest) { + issues.push(`hack:${relative}:declares-mcp`); + } + } + for (const retired of [".mcp.json", "mcp.json"]) { + if (await Bun.file(join(root, retired)).exists()) { + issues.push(`hack:${retired}:ships-mcp`); + } + } + return issues; +} + +/** Shape findings for hack-mcp: exactly the generated MCP declaration. */ +async function mcpPackageShapeIssues(root: string): Promise { + const issues: string[] = []; + const files = Object.keys(await snapshotTree(root)).sort(); + if (!sameJson(files, [...HACK_MCP_PLUGIN_FILES].sort())) { + issues.push("hack-mcp:files"); + } + const codex = await readJsonObject(join(root, ".codex-plugin/plugin.json")); + for (const directory of manifestDirectories) { + const relative = `${directory}/plugin.json`; + const manifest = await readJsonObject(join(root, relative)); + if (manifest.name !== HACK_MCP_PLUGIN_NAME) { + issues.push(`hack-mcp:${relative}:name`); + } + if (manifest.mcpServers !== "./.mcp.json") { + issues.push(`hack-mcp:${relative}:mcp`); + } + for (const key of ["skills", "hooks", "rules"]) { + if (key in manifest) { + issues.push(`hack-mcp:${relative}:${key}`); + } + } + } + const servers = await readJsonObject(join(root, ".mcp.json")); + if (!sameJson(servers, renderMcpServers())) { + issues.push("hack-mcp:.mcp.json:server"); + } + if ( + !sameJson( + await readJsonObject(join(root, "mcp.json")), + portableMcpManifest(servers) + ) + ) { + issues.push("hack-mcp:mcp.json:derived"); + } + if ( + !sameJson( + await readJsonObject(join(root, "plugin.json")), + portablePluginManifest(codex) + ) + ) { + issues.push("hack-mcp:plugin.json:derived"); + } + return issues; +} + +/** + * Closed findings for both packages: `hack` ships skills, hooks and rules but + * no MCP; `hack-mcp` ships exactly the generated MCP declaration. Findings + * name a package-relative file and a fixed code, never file content. + */ +async function pluginPackageShapeIssues( + pluginsRoot: string +): Promise { + return [ + ...(await skillsPackageShapeIssues(join(pluginsRoot, HACK_PLUGIN_NAME))), + ...(await mcpPackageShapeIssues(join(pluginsRoot, HACK_MCP_PLUGIN_NAME))), + ]; +} + +test("the hack package is skills-only and hack-mcp carries the only MCP declaration", async () => { + expect(await pluginPackageShapeIssues(join(repoRoot, "plugins"))).toEqual([]); + expect(renderMcpServers()).toEqual({ + mcpServers: { + [HACK_MCP_SERVER.name]: { command: "hack", args: ["mcp", "serve"] }, + }, + }); + const claude = await readJsonObject( + join(repoRoot, "plugins/hack/.claude-plugin/plugin.json") + ); + expect(claude.skills).toBe("./skills/"); + expect(claude.hooks).toBe("./hooks/claude-hooks.json"); + const cursor = await readJsonObject( + join(repoRoot, "plugins/hack/.cursor-plugin/plugin.json") + ); + expect(cursor.rules).toBe("./rules/"); +}); + +test("negative control: re-adding MCP to the skills package fails the shape check and generation", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-plugin-shape-")); + const pluginsRoot = join(tempDir, "plugins"); + await cp(join(repoRoot, "plugins"), pluginsRoot, { recursive: true }); + const manifestPath = join(pluginsRoot, "hack/.claude-plugin/plugin.json"); await Bun.write( - join(pluginRoot, ".mcp.json"), + manifestPath, JSON.stringify({ - mcpServers: { hack: { command: "hack", args: ["mcp", "serve"] } }, + ...(await readJsonObject(manifestPath)), + mcpServers: "./.mcp.json", }) ); -} + await Bun.write( + join(pluginsRoot, "hack/.mcp.json"), + JSON.stringify(renderMcpServers()) + ); + expect(await pluginPackageShapeIssues(pluginsRoot)).toEqual([ + "hack:.claude-plugin/plugin.json:declares-mcp", + "hack:.mcp.json:ships-mcp", + ]); + await expect( + generateAgentPlugins({ repoRoot: tempDir, version: "9.8.7" }) + ).rejects.toThrow("skills-only"); + + await Bun.write( + join(pluginsRoot, "hack-mcp/.cursor-plugin/plugin.json"), + JSON.stringify({ name: HACK_MCP_PLUGIN_NAME, skills: "./skills/" }) + ); + expect( + await mcpPackageShapeIssues(join(pluginsRoot, HACK_MCP_PLUGIN_NAME)) + ).toEqual([ + "hack-mcp:.cursor-plugin/plugin.json:mcp", + "hack-mcp:.cursor-plugin/plugin.json:skills", + ]); +}); -test("release generation applies the requested version to every client manifest", async () => { +test("release generation applies the requested version to both packages", async () => { tempDir = await mkdtemp(join(tmpdir(), "hack-agent-plugins-")); - const pluginRoot = join(tempDir, "plugins", "hack"); + const pluginRoot = join(tempDir, "plugins", HACK_PLUGIN_NAME); await seedPluginManifests(pluginRoot); + // A checkout from before hack-mcp still carries the skills package's MCP files. + await Bun.write(join(pluginRoot, ".mcp.json"), "{}\n"); + await Bun.write(join(pluginRoot, "mcp.json"), "{}\n"); await generateAgentPlugins({ repoRoot: tempDir, version: "9.8.7" }); - for (const manifestDirectory of manifestDirectories) { - const manifest = await Bun.file( - join(pluginRoot, manifestDirectory, "plugin.json") - ).json(); - expect(manifest.version).toBe("9.8.7"); + for (const name of [HACK_PLUGIN_NAME, HACK_MCP_PLUGIN_NAME]) { + for (const manifestDirectory of manifestDirectories) { + const manifest = await Bun.file( + join(tempDir, "plugins", name, manifestDirectory, "plugin.json") + ).json(); + expect(manifest.version).toBe("9.8.7"); + } + expect( + (await Bun.file(join(tempDir, "plugins", name, "plugin.json")).json()) + .version + ).toBe("9.8.7"); } - expect((await Bun.file(join(pluginRoot, "plugin.json")).json()).version).toBe( - "9.8.7" + expect(await Bun.file(join(pluginRoot, ".mcp.json")).exists()).toBe(false); + expect(await Bun.file(join(pluginRoot, "mcp.json")).exists()).toBe(false); + expect( + await mcpPackageShapeIssues(join(tempDir, "plugins", HACK_MCP_PLUGIN_NAME)) + ).toEqual([]); + expect( + await Bun.file( + join(tempDir, "plugins/hack-mcp/.claude-plugin/plugin.json") + ).json() + ).toEqual(renderMcpPluginManifests({ version: "9.8.7" }).claude); +}); + +test("generating twice is a no-op and reproduces the committed packages", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-plugin-idempotent-")); + const pluginsRoot = join(tempDir, "plugins"); + await cp(join(repoRoot, "plugins"), pluginsRoot, { recursive: true }); + const committed = await snapshotTree(join(repoRoot, "plugins")); + const pkg = await readJsonObject(join(repoRoot, "package.json")); + const version = String(pkg.version); + + await generateAgentPlugins({ repoRoot: tempDir, version }); + const first = await snapshotTree(pluginsRoot); + await generateAgentPlugins({ repoRoot: tempDir, version }); + const second = await snapshotTree(pluginsRoot); + + expect(first).toEqual(committed); + expect(second).toEqual(first); +}, 15_000); + +test("docs present the skills-only plugin as default and hack-mcp as opt-in", async () => { + const read = (path: string) => Bun.file(join(repoRoot, path)).text(); + const readme = await read("README.md"); + expect(readme).not.toContain("sync themselves into"); + expect(readme).toContain("claude plugin install hack@hack-dance"); + expect(readme).toContain("`hack-mcp`"); + + const integrations = await read("docs/integrations.md"); + for (const phrase of [ + "`hack-mcp` (`plugins/hack-mcp`)", + "### One integration path per client", + "even when the plugin manifest omits", + "### Upgrading from a plugin that bundled MCP", + "--sparse .claude-plugin plugins/hack plugins/hack-mcp", + ]) { + expect(integrations).toContain(phrase); + } + + const guide = await read("docs/guides/agent-first-setup.md"); + expect(guide).not.toContain( + "`hack setup claude` and `hack setup codex` install a thin `hack-init` skill" ); + expect(guide).toContain( + "The optional `hack` plugin ships a thin `hack-init`" + ); + + const pluginReadme = await read("plugins/hack/README.md"); + expect(pluginReadme).not.toContain("mcp_servers.hack"); + expect(pluginReadme).toContain("only when MCP is\nwanted"); +}); + +test("each marketplace lists the hack and hack-mcp packages separately", async () => { + const sourcePath = (entry: Record): string => + String(isRecord(entry.source) ? entry.source.path : entry.source).replace( + /^\.\//, + "" + ); + for (const path of [ + ".claude-plugin/marketplace.json", + ".agents/plugins/marketplace.json", + ".cursor-plugin/marketplace.json", + ]) { + const marketplace = await readJsonObject(join(repoRoot, path)); + const entries = Array.isArray(marketplace.plugins) + ? marketplace.plugins.filter(isRecord) + : []; + expect(entries.map((entry) => entry.name)).toEqual([ + HACK_PLUGIN_NAME, + HACK_MCP_PLUGIN_NAME, + ]); + expect(entries.map(sourcePath)).toEqual([ + "plugins/hack", + "plugins/hack-mcp", + ]); + } }); test("generation writes every registered reference and prunes only stale reference markdown", async () => { @@ -132,6 +384,7 @@ test("semantic-release commits all generated client manifests", async () => { expect(gitPlugin).toBeDefined(); const assets = gitPlugin?.[1]?.assets as string[]; expect(assets).toContain("plugins/hack"); + expect(assets).toContain("plugins/hack-mcp"); }); test("bundled skills and rule match current canonical guidance", async () => { @@ -170,8 +423,15 @@ test("bundled skills and rule match current canonical guidance", async () => { ).json(); expect(portable.version).toBe(pkg.version); expect(portable).toEqual(portablePluginManifest(overlay)); - expect(await Bun.file(join(root, "mcp.json")).json()).toEqual( - portableMcpManifest(await Bun.file(join(root, ".mcp.json")).json()) + const mcpRoot = join(repoRoot, "plugins", HACK_MCP_PLUGIN_NAME); + const mcpManifests = renderMcpPluginManifests({ version: pkg.version }); + for (const client of ["codex", "claude", "cursor"] as const) { + expect( + await Bun.file(join(mcpRoot, `.${client}-plugin/plugin.json`)).json() + ).toEqual(mcpManifests[client]); + } + expect(await Bun.file(join(mcpRoot, "mcp.json")).json()).toEqual( + portableMcpManifest(await Bun.file(join(mcpRoot, ".mcp.json")).json()) ); }); @@ -260,15 +520,18 @@ test("release preparation updates both manifests and rendered guidance after bum new Response(child.stderr).text(), ]); expect(exitCode, stdout + stderr).toBe(0); - for (const client of ["codex", "claude", "cursor"]) { - const manifest = await Bun.file( - join(tempDir, "plugins/hack", `.${client}-plugin/plugin.json`) - ).json(); - expect(manifest.version).toBe("9.8.7"); + for (const name of [HACK_PLUGIN_NAME, HACK_MCP_PLUGIN_NAME]) { + for (const client of ["codex", "claude", "cursor"]) { + const manifest = await Bun.file( + join(tempDir, "plugins", name, `.${client}-plugin/plugin.json`) + ).json(); + expect(manifest.version).toBe("9.8.7"); + } + expect( + (await Bun.file(join(tempDir, "plugins", name, "plugin.json")).json()) + .version + ).toBe("9.8.7"); } - expect( - (await Bun.file(join(tempDir, "plugins/hack/plugin.json")).json()).version - ).toBe("9.8.7"); const skill = await Bun.file( join(tempDir, "plugins/hack/skills/hack-cli/SKILL.md") ).text(); @@ -298,13 +561,15 @@ interface PolicyScan { const HOOK_COMMAND = "hack agent prime"; /** - * Generated agent guidance surfaces: the native plugin bundle, the client - * marketplace manifests, the shared `.agents` tree (skills reach it through - * symlinks) and the agent configuration in the basic example. Example compose - * and runtime files are application configuration, not agent guidance. + * Generated agent guidance surfaces: the native skills and MCP plugin + * packages, the client marketplace manifests, the shared `.agents` tree + * (skills reach it through symlinks) and the agent configuration in the basic + * example. Example compose and runtime files are application configuration, + * not agent guidance. */ const GUIDANCE_SURFACES = [ "plugins/hack", + "plugins/hack-mcp", ".claude-plugin", ".cursor-plugin", ".agents", @@ -481,7 +746,12 @@ test("generated guidance surfaces never pin models, override permissions or run // YAML and markdown frontmatter reached through the `.agents` symlinks. for (const file of [ "plugins/hack/plugin.json", - "plugins/hack/.mcp.json", + "plugins/hack-mcp/plugin.json", + "plugins/hack-mcp/.mcp.json", + "plugins/hack-mcp/mcp.json", + "plugins/hack-mcp/.codex-plugin/plugin.json", + "plugins/hack-mcp/.claude-plugin/plugin.json", + "plugins/hack-mcp/.cursor-plugin/plugin.json", "plugins/hack/.codex-plugin/plugin.json", "plugins/hack/.claude-plugin/plugin.json", "plugins/hack/.cursor-plugin/plugin.json", @@ -668,10 +938,6 @@ test("shipped skill inventory matches the generator and the integration doc", as join(pluginRoot, directory, "plugin.json") ); } - await cp( - join(repoRoot, "plugins/hack/.mcp.json"), - join(pluginRoot, ".mcp.json") - ); await generateAgentPlugins({ repoRoot: tempDir, version: "0.0.0" }); const skillNames = async (root: string) => (await readdir(join(root, "skills"), { withFileTypes: true })) diff --git a/tests/hack-init-skill.test.ts b/tests/hack-init-skill.test.ts index 50eb6f3c..10173d55 100644 --- a/tests/hack-init-skill.test.ts +++ b/tests/hack-init-skill.test.ts @@ -72,6 +72,11 @@ test("renderHackInitSkill is a thin pointer to the CLI prompt source", () => { expect(skill).toContain("Stand up or adopt hack in a project"); expect(skill).toContain("hack agent onboard"); expect(skill).toContain("`hack-init` MCP prompt"); + // The default plugin ships no MCP, so the MCP fallback is conditional. + expect(skill).toContain( + "only if the `hack-mcp` plugin or another Hack MCP server is installed" + ); + expect(skill).not.toContain("MCP prompt from the hack MCP server."); // Thin: the full onboarding content must NOT be duplicated into the skill. expect(skill).not.toContain("node_modules:/app/node_modules"); expect(skill).not.toContain("## Phase 1"); From e2c41a980cb6770ec4855383b5dd644e99dd6096 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 06:55:01 -0400 Subject: [PATCH 35/52] fix(mcp): align budget docs with log cutoffs and keep byte-cut failures - Scope the 8 MiB budget paragraph in the agent-first setup guide to non-log tools; log tools report the bytes bound as an ok cutoff (link to the MCP logs contract). - An error or failing end event inside the retained prefix of a byte cut is real failure evidence (it was written before the child was signalled), so it is now a backend_error instead of a bytes cutoff. Events/time stops keep ignoring post-stop failure events; the docs state the remaining ambiguity. - The maxEvents input description says only type:log events count. - Widen the time-window tests to 2 s so Bun stub startup cannot flake them. HACK-1134 Co-Authored-By: Claude Opus 5.5 --- docs/guides/agent-first-setup.md | 14 +++++--- docs/reference/mcp-logs.md | 8 ++++- src/mcp/logs-contract.ts | 3 ++ src/mcp/server.ts | 16 ++++++--- tests/mcp-logs-contract.test.ts | 56 +++++++++++++++++++++++++++++--- 5 files changed, 82 insertions(+), 15 deletions(-) diff --git a/docs/guides/agent-first-setup.md b/docs/guides/agent-first-setup.md index 38153a50..1d515937 100644 --- a/docs/guides/agent-first-setup.md +++ b/docs/guides/agent-first-setup.md @@ -59,11 +59,15 @@ completed operations or delete persistent project data. These controls do not re the number of servers for clients that remain connected. MCP command results capture at most 8 MiB of combined stdout/stderr bytes before -text decoding. This also bounds log tails with oversized or newline-free messages. -Exceeding the budget stops the command and returns an error with -`outputTruncated: true`; parsed `data` is omitted so a prefix cannot masquerade as -a complete result. Run the equivalent CLI command when full output is needed. -The budget bounds captured bytes, not total process memory or JSON response size. +text decoding. For tools other than the log tools, exceeding the budget stops the +command and returns an error with `outputTruncated: true`; parsed `data` is omitted +so a prefix cannot masquerade as a complete result. The log tools +(`hack.project.logs.tail` and `hack.project.logs.snapshot`) use the same budget, +but reaching it is a cutoff: the result is `ok: true` with `outputTruncated: true`, +`data.cutoff` set to the `bytes` bound, and the events parsed before the cut. See +[MCP logs contract](../reference/mcp-logs.md). Run the equivalent CLI command when +full output is needed. The budget bounds captured bytes, not total process memory +or JSON response size. Each MCP session captures its launch working directory and environment. Project registry selection, command execution and audit-file locations use that snapshot; diff --git a/docs/reference/mcp-logs.md b/docs/reference/mcp-logs.md index e56d1353..a734b927 100644 --- a/docs/reference/mcp-logs.md +++ b/docs/reference/mcp-logs.md @@ -41,7 +41,13 @@ Raw `stdout` and `stderr` are still returned, as they are for other tools. Failure evidence counts only before the server stops the child. A child that writes `error` or `end` events, or exits non-zero, in response to the server's -own stop signal still produces a cutoff. +own stop signal still produces a cutoff. The byte bound is an exception for +events: every retained byte was written before the child was signalled, and +nothing after the cut is parsed, so an `error` or failing `end` event in the +returned prefix is still a backend failure. For the `events` and `time` bounds, +an `error` event that the backend wrote just before the stop but that the server +read after it is reported as a cutoff. The server cannot tell it apart from an +event written in response to the stop. ## Reconnects and repeated calls diff --git a/src/mcp/logs-contract.ts b/src/mcp/logs-contract.ts index 2902f09f..e2145cd3 100644 --- a/src/mcp/logs-contract.ts +++ b/src/mcp/logs-contract.ts @@ -15,6 +15,9 @@ * Failure evidence counts only while collection is still running. Once the * server stops the command for a bound, a later `error`/`end` event or a * signal exit status is a consequence of that stop, not a backend failure. + * The byte bound is the exception for events: the retained prefix was written + * before the child was signalled and nothing after the cut is parsed, so an + * `error` or failing `end` event inside that prefix is still a failure. */ /** Why collection stopped, as reported in `data.stop_reason`. */ diff --git a/src/mcp/server.ts b/src/mcp/server.ts index 317233cc..e9376c87 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -746,7 +746,8 @@ function registerTools(opts: { ...logSelectionInput, maxEvents: boundedLimitInput({ ceiling: MCP_INPUT_CEILINGS.maxEvents, - description: "Max events to collect", + description: + "Max type:log events to collect; start, heartbeat, error and end events do not count", }).optional(), maxMs: boundedLimitInput({ ceiling: MCP_INPUT_CEILINGS.maxMs, @@ -1457,7 +1458,11 @@ async function runHackLogCollect(opts: { const stderrLines: string[] = []; let stopReason: McpLogStopReason = "eof"; - let stopped = false; + // Set once a server stop can have reached the child. Later failure events + // may describe that stop. A byte cut is the exception: the retained prefix + // was written before the child was signalled, and nothing is retained after + // it, so failure evidence in that prefix still counts. + let failureEvidenceClosed = false; let logCount = 0; let backendFailureObserved = false; @@ -1466,8 +1471,10 @@ async function runHackLogCollect(opts: { signal: opts.signal, timeoutMs: opts.timeoutMs, onStop: (reason) => { - stopped = true; stopReason = toLogStopReason(reason); + if (stopReason !== "output_limit") { + failureEvidenceClosed = true; + } }, }); @@ -1479,8 +1486,7 @@ async function runHackLogCollect(opts: { const recordEvent = (event: Record): boolean => { events.push(event); const observed = observeLogStreamEvent(event); - // Events after a server-initiated stop describe that stop. - if (observed.backendFailure && !stopped) { + if (observed.backendFailure && !failureEvidenceClosed) { backendFailureObserved = true; } if (observed.isLog) { diff --git a/tests/mcp-logs-contract.test.ts b/tests/mcp-logs-contract.test.ts index 833a17fd..9902603f 100644 --- a/tests/mcp-logs-contract.test.ts +++ b/tests/mcp-logs-contract.test.ts @@ -215,6 +215,7 @@ type Scenario = | "nonzero-exit" | "error-after-stop" | "bytes-utf8" + | "error-in-byte-cut" | "sequenced"; type LogsFixture = { @@ -320,6 +321,19 @@ switch (scenario) { hang(); break; } + case "error-in-byte-cut": { + log("one"); + const limit = 8 * 1024 * 1024; + const written = Buffer.byteLength(line({ type: "start", follow: true })) + + Buffer.byteLength(line({ type: "log", entry: { message: "one" } })); + const failure = line({ type: "error", message: "backend failed" }); + // One write: filler, an error event ending 8 bytes before the limit, and + // bytes past the limit. The error is read in the same chunk as the cut. + const filler = "x".repeat(limit - written - Buffer.byteLength(failure) - 9); + process.stdout.write(filler + "\n" + failure + "z".repeat(1024 * 1024)); + hang(); + break; + } case "sequenced": { let seq = 0; setInterval(() => { @@ -518,18 +532,22 @@ test("maxEvents counts log events only and its cutoff is a success", async () => } }); +// The stub must start and emit its first log before the window closes. Bun +// startup is far below this even on a loaded host; a shorter window flakes. +const TIME_WINDOW_MS = 2000; + test("the collection window cutoff is a success even though the child is signalled", async () => { const fixture = await startLogsFixture("follow"); const client = await fixture.connect(); const { isError, payload } = await callLogs({ client, tool: "hack.project.logs.tail", - args: { maxMs: 300 }, + args: { maxMs: TIME_WINDOW_MS }, }); expect(isError).toBe(false); expect(payload.ok).toBe(true); expect(payload.exitCode).not.toBe(0); - expect(payload.data.cutoff).toEqual({ bound: "time", limit: 300 }); + expect(payload.data.cutoff).toEqual({ bound: "time", limit: TIME_WINDOW_MS }); expect(logMessages(payload.data.events)).toEqual(["one"]); }); @@ -539,7 +557,7 @@ test("failure events after a server-initiated stop do not turn a cutoff into an const { isError, payload } = await callLogs({ client, tool: "hack.project.logs.tail", - args: { maxMs: 300 }, + args: { maxMs: TIME_WINDOW_MS }, }); // The stub did report error/end(error) in response to SIGTERM ... expect(payload.data.events.map((event) => event.type)).toEqual([ @@ -551,7 +569,7 @@ test("failure events after a server-initiated stop do not turn a cutoff into an // ... yet the outcome is the time cutoff the server caused. expect(isError).toBe(false); expect(payload.ok).toBe(true); - expect(payload.data.cutoff).toEqual({ bound: "time", limit: 300 }); + expect(payload.data.cutoff).toEqual({ bound: "time", limit: TIME_WINDOW_MS }); }); for (const [scenario, label] of [ @@ -612,6 +630,36 @@ test("the byte cutoff returns earlier events and never splits a code point", asy } }, 30_000); +test("an error event read in the same chunk as the byte cut is still a backend_error", async () => { + const fixture = await startLogsFixture("error-in-byte-cut"); + const client = await fixture.connect(); + const { isError, payload } = await callLogs({ + client, + tool: "hack.project.logs.tail", + args: { maxMs: 30_000 }, + }); + // The error event sits inside the retained prefix, which the child wrote + // before it was signalled, so it is real failure evidence, not a cutoff. + expect(payload.outputTruncated).toBe(true); + expect(payload.data.stop_reason).toBe("output_limit"); + expect(payload.data.events.map((event) => event.type)).toEqual([ + "start", + "log", + "error", + ]); + expect(isError).toBe(true); + expect(payload.ok).toBe(false); + expect(payload.error).toEqual({ + code: "E_UNEXPECTED", + reason: "backend_error", + message: MCP_LOG_BACKEND_ERROR.message, + }); + expect(payload.data.cutoff).toBeNull(); + for (const pid of await readStubPids(fixture.stubDir)) { + await waitFor(() => !alive(pid)); + } +}, 30_000); + test("consecutive tail calls each start a new stream: no cursor, no replay", async () => { const fixture = await startLogsFixture("sequenced"); const client = await fixture.connect(); From cea44f8e21737d57fe593bb8c7de5168cabffb6b Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:11:24 -0400 Subject: [PATCH 36/52] feat(mcp): add exec, doctor, env explain/keys and instances tools Add five MCP tools on top of the hardened tool surface (HACK-1133): - hack.project.exec runs `hack exec` in an already-running service. It is separate from hack.project.run, validates service/branch/profile identifiers, ends hack options with `--`, requires a non-empty cmd, and echoes timeoutMs plus the shared 8 MiB output bound. - hack.project.doctor runs `hack doctor --json` with a closed argv (path, branch, --json). Repair, migration and recovery modes are unreachable. - hack.project.env.explain and hack.project.env.keys run `hack env explain --json` and `hack env list --json` (never --show-secrets). Child stdout and stderr are withheld; data is an allowlisted, value-free projection. - hack.project.instances lists a registered project's default and branch runtime instances from `hack projects --json --project=`. Inspection tools are annotated readOnlyHint; exec is destructive. Add docs/reference/mcp-tools.md as the MCP tool contract, with a test that it lists every registered tool and its annotation class. Co-Authored-By: Claude Opus 5.5 --- docs/guides/agent-first-setup.md | 3 + docs/reference.md | 1 + docs/reference/mcp-tools.md | 199 +++++++ src/mcp/inspect-projections.ts | 252 ++++++++ src/mcp/server.ts | 468 ++++++++++++++- tests/mcp-inspect-tools.test.ts | 992 +++++++++++++++++++++++++++++++ tests/mcp.test.ts | 28 + 7 files changed, 1942 insertions(+), 1 deletion(-) create mode 100644 docs/reference/mcp-tools.md create mode 100644 src/mcp/inspect-projections.ts create mode 100644 tests/mcp-inspect-tools.test.ts diff --git a/docs/guides/agent-first-setup.md b/docs/guides/agent-first-setup.md index 1d515937..c26c99e7 100644 --- a/docs/guides/agent-first-setup.md +++ b/docs/guides/agent-first-setup.md @@ -69,6 +69,9 @@ but reaching it is a cutoff: the result is `ok: true` with `outputTruncated: tru full output is needed. The budget bounds captured bytes, not total process memory or JSON response size. +The [MCP tool contract](../reference/mcp-tools.md) lists every tool with its +annotations, inputs, the CLI command it runs and its result shape. + Each MCP session captures its launch working directory and environment. Project registry selection, command execution and audit-file locations use that snapshot; later sessions do not replace the state of an existing session. The standard stdio diff --git a/docs/reference.md b/docs/reference.md index 64c7e50a..ee490977 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -27,4 +27,5 @@ - [Extensions](./extensions.md) - [Creating an extension](./guides/create-extension.md) - [Integrations](./integrations.md) +- [MCP tool contract](./reference/mcp-tools.md) — every MCP tool: annotations, inputs, argv, results, and the value-free env tools - [MCP logs contract](./reference/mcp-logs.md) — cutoff versus backend failure for the MCP log tools, and the cursor/handle decision diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md new file mode 100644 index 00000000..7d387fbb --- /dev/null +++ b/docs/reference/mcp-tools.md @@ -0,0 +1,199 @@ +# MCP tool contract + +`hack mcp serve` exposes the tools below to MCP clients that have no shell. Each +tool runs one `hack` CLI command as a child process, with the session's captured +working directory and environment, and `HACK_NO_INTERACTIVE=1`. When a shell is +available, run the CLI command directly instead. + +This page is the contract for tool names, annotations, inputs, the argv each tool +runs, and result shapes. The log tools have their own page: +[MCP logs contract](./mcp-logs.md). + +## Tools + +The annotation class is published in `tools/list`: + +- `read-only`: `readOnlyHint: true`. The tool only inspects state. +- `additive`: `readOnlyHint: false`, `destructiveHint: false`. The tool starts + or restarts local runtime state but does not remove it. +- `destructive`: `readOnlyHint: false`, `destructiveHint: true`. The tool can + stop, replace or overwrite local state, or run an arbitrary command. + +Every tool sets `openWorldHint: false`. Mutating tools set `idempotentHint: false`. + +| Tool | Class | CLI command | +| --- | --- | --- | +| `hack.projects.list` | read-only | `hack projects --json` | +| `hack.project.status` | read-only | `hack ps --json` | +| `hack.project.up` | additive | `hack up --detach` | +| `hack.project.down` | destructive | `hack down` | +| `hack.project.restart` | additive | `hack restart` | +| `hack.project.run` | destructive | `hack run -- [cmd...]` | +| `hack.project.logs.snapshot` | read-only | `hack logs --json --no-follow` | +| `hack.project.logs.tail` | read-only | `hack logs --json` | +| `hack.project.open` | read-only | `hack open --json` (never launches a browser) | +| `hack.project.init` | destructive | `hack init --auto` | +| `hack.project.exec` | destructive | `hack exec -- ` | +| `hack.project.doctor` | read-only | `hack doctor --json` | +| `hack.project.env.explain` | read-only | `hack env explain --json -- ` | +| `hack.project.env.keys` | read-only | `hack env list --json` | +| `hack.project.instances` | read-only | `hack projects --json --project=` | + +## Project selection + +Project-scoped tools accept at most one of `projectName`, `repoRoot` or `path`. +With none, the session's working directory selects the project. + +- `projectName` must name a registered project (`~/.hack/projects.json`). +- `repoRoot` and `path` must exist and lie inside a registered project's repo + root or project directory. +- With no selection, the working directory must contain a hack project or lie + inside a registered one. + +`hack.project.doctor` passes a named project to the CLI as `--path=`, +because `hack doctor` has no `--project` option. + +`hack.project.instances` needs a registered project name, because +`hack projects` selects by name only. A path, or the working directory when +nothing is selected, maps to the registered project whose repo root or project +directory contains it. When registrations are nested, the deepest one wins. Two +different registrations at the same root are refused as `ambiguous_project`. A +project that has a `.hack/` directory but no registration is refused; run +`hack projects` or any project command in it to register it. + +## Inputs and identifiers + +Inputs are validated before any child process starts. Unknown input fields are +dropped, so they never reach the CLI argv. + +- `service`, `services`, `profiles`, open `target`, and the env tools' `env` and + `service` must match `^[A-Za-z0-9][A-Za-z0-9_.-]*$`. +- `branch` must match `^[A-Za-z0-9][A-Za-z0-9_./-]*$`. +- Env `key` must match `^[A-Za-z_][A-Za-z0-9_]*$`. Scoped `service.KEY` syntax is + not accepted. Use the `service` input for the scope. +- Identifiers are at most 253 characters. None of these patterns can start with + `-`, so no identifier can be read as an option. +- Numeric limits are positive integers. A value above the ceiling is refused, + not clamped: `tail` and `maxEvents` up to 10,000, `maxMs` up to 60,000, and + `timeoutMs` up to 600,000. + +Option values are always fused (`--branch=`). Positional values that come +from the caller follow `--`, so the CLI parser ends option parsing before them. + +## Results + +Every result has the same structured content: + +| Field | Meaning | +| --- | --- | +| `ok` | The child exited 0 (for the log tools, see the logs contract) | +| `command` | The argv that ran, or `""` when refused before spawn | +| `exitCode` | The child's exit code (1 for a refusal) | +| `stdout`, `stderr` | Captured child output (withheld for the env tools; `stdout` withheld for instances) | +| `data` | Parsed or projected result, when the tool defines one | +| `outputTruncated` | `true` when output reached the 8 MiB capture limit | +| `spawned` | `false` only on a refusal: no child, no audit line | +| `error` | Closed refusal or backend error: `code`, `reason`, `message` | + +When `ok` is `false`, the result also has `isError: true`. Output above the +8 MiB capture limit stops the command, and `data` is omitted so that a prefix is +never presented as a complete result. The log tools are the exception, as their +contract describes. + +### Refusals + +A refusal happens before any child process exists. Its `error.reason` is one of +`invalid_arguments`, `conflicting_project_selection`, `invalid_project_name`, +`ambiguous_project`, `project_not_found`, `path_not_found`, +`path_not_registered`, `no_project_context` and `init_path_required`. The code is +`E_USAGE` or `E_PROJECT_NOT_FOUND`. The message is fixed text. It never repeats +caller input or registry values. For `invalid_arguments`, the message names the +rejected input fields, using only the tool's own schema keys. + +## `hack.project.exec` + +Runs a command in a service container that is already running (`docker compose +exec`). `hack.project.run` starts a new one-off container instead. Exec never +starts a container, and the CLI fails when the service is not running. + +- Required: `service` and `cmd`, with at least one entry. +- Optional: project selection, `branch`, `profiles`, `workdir` and `timeoutMs` + (default 120,000). +- Argv: `hack exec [--project=…|--path=…] [--branch=…] [--profile=…] + [--workdir=…] -- `. Entries in `cmd` such as `--help` or + `--branch` reach the container command and are never parsed by hack. +- `data.limits` echoes `{ timeoutMs, outputBytes }`. `outputBytes` is the shared + 8 MiB capture limit. + +## `hack.project.doctor` + +Runs read-only setup diagnostics: `hack doctor [--path=…] [--branch=…] --json`. +The argv set is closed. `--fix`, `--migrate-env-config`, `--domain-migration`, +`--native-cleanup`, `--native-run-mapping`, browser observations and legacy +rebind acceptance are never passed. The CLI also refuses `--json` together with +`--fix`. + +`data` is the CLI's JSON result envelope, with `data.checks`, `data.summary` and +`data.counts`. When any check reports an error, the CLI exits 1, so `ok` is +`false` while `data` still carries the full report. + +## `hack.project.env.explain` + +Explains one env key without its value: +`hack env explain [--project=…|--path=…] [--env=…] [--service=…] [--target=…] +--json -- `. + +- Required: `key`. Optional: project selection, `env` (overlay; `base` or + `default` selects base layers only), `service` (scope; default `global`) and + `target` (`host` or `compose`). +- `data` is an allowlisted projection: `key`, `available`, `secret`, `format`, + `environment`, `requested_scope`, `effective_scope`, `target`, + `delivered_to`, `source` and `precedence`. Any other field in the CLI output is + dropped. + +## `hack.project.env.keys` + +Lists env key names with metadata: `hack env list [--project=…|--path=…] +[--env=…] [--service=…] --json`. `--show-secrets` is never passed, and there is +no input for it. + +`data` is `{ format, environment, selected_scope, scopes, keys, +missing_required }`. Each entry in `keys` is `{ key, secret, required, source, +resolved_from, services }`. `secret` is `true` when the CLI reports the key as +`keychain`-backed. Row values, materialized values, storage locations and +warning text are dropped. A row whose key does not match the env key pattern is +dropped whole. + +### Value-free output + +`hack env list --json` prints plaintext values, and CLI errors can quote env +files. Both env tools therefore withhold the child's output: + +- `stdout` is always `""`. +- `stderr` is `""` on success. On failure it is a fixed message that tells you + to run the same `hack env` command in a terminal. +- `data` carries only the allowlisted fields above. It is `null` when the CLI + printed no parseable JSON object. + +`command` still shows the argv, which contains the key name but no values. The +MCP audit log records the same argv. + +## `hack.project.instances` + +Lists a registered project's runtime instances: `hack projects --json +--project=`. + +`data` is `{ project, found, runtime_ok, instances }`. `instances` lists the +default instance first, when it is running, as `branch: null`. Each branch +instance follows, in CLI order. Each instance is `{ branch, compose_project, +working_dir, services }`, and each service is `{ service, containers, running }` +with container counts. `stdout` is always `""`: the raw listing can cover other +projects and carries container labels, mounts and networks, which `data` omits. +`stderr` is returned as captured. `found` is `false` when the project is +registered but the CLI listing does not include it. + +## Audit log + +Every spawned command appends one JSON line to `$HACK_HOME/mcp-audit.log` (by +default `~/.hack/mcp-audit.log`): `ts`, `tool`, `command` and `exit_code`. +Refusals write nothing. diff --git a/src/mcp/inspect-projections.ts b/src/mcp/inspect-projections.ts new file mode 100644 index 00000000..fa2d27a8 --- /dev/null +++ b/src/mcp/inspect-projections.ts @@ -0,0 +1,252 @@ +/** + * Value-free projections for the MCP inspection tools (HACK-1133). + * + * `hack.project.env.explain` and `hack.project.env.keys` run `hack env explain + * --json` and `hack env list --json`. The CLI payloads are not guaranteed to be + * value-free (`env list --json` carries plaintext values), so the MCP server + * never forwards their raw stdout or stderr. Each projection below copies only + * an allowlisted set of metadata fields with narrowed types; a field that is + * missing, unexpected or of the wrong type is dropped, never passed through. + * + * `hack.project.instances` projects one project's runtime instances out of + * `hack projects --json`. It is not a secret boundary, but it keeps the result + * small: container inspect data (labels, mounts, networks) is not returned. + */ +import { isRecord } from "../lib/guards.ts"; +import { normalizeProjectName } from "../lib/project-name.ts"; + +/** + * Env key accepted by the env tools. It starts with a letter or underscore, so + * it can never be read as an option, and it carries no scope separator. The + * CLI applies its own stricter key rule after this. + */ +export const MCP_ENV_KEY_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/; + +/** Fixed stderr text for a failed env tool call; child output is withheld. */ +export const MCP_ENV_OUTPUT_WITHHELD_MESSAGE = + "hack env command failed. Its output is withheld because it may contain env values; run the same hack env command in a terminal for details."; + +function optionalString(value: unknown): string | null { + return typeof value === "string" ? value : null; +} + +function optionalBoolean(value: unknown): boolean | null { + return typeof value === "boolean" ? value : null; +} + +function stringList(value: unknown): string[] | null { + if (!Array.isArray(value)) { + return null; + } + return value.filter((entry): entry is string => typeof entry === "string"); +} + +function envKeyOrNull(value: unknown): string | null { + return typeof value === "string" && MCP_ENV_KEY_PATTERN.test(value) + ? value + : null; +} + +/** Env explain metadata: where a key comes from, never its value. */ +export type McpEnvExplainData = { + readonly key: string | null; + readonly available: boolean | null; + readonly secret: boolean | null; + readonly format: string | null; + readonly environment: string | null; + readonly requested_scope: string | null; + readonly effective_scope: string | null; + readonly target: string | null; + readonly delivered_to: readonly string[] | null; + readonly source: string | null; + readonly precedence: readonly string[] | null; +}; + +/** Project `hack env explain --json` stdout onto {@link McpEnvExplainData}. */ +export function projectEnvExplain(payload: unknown): McpEnvExplainData | null { + if (!isRecord(payload)) { + return null; + } + return { + key: envKeyOrNull(payload.key), + available: optionalBoolean(payload.available), + secret: optionalBoolean(payload.secret), + format: optionalString(payload.format), + environment: optionalString(payload.environment), + requested_scope: optionalString(payload.requested_scope), + effective_scope: optionalString(payload.effective_scope), + target: optionalString(payload.target), + delivered_to: stringList(payload.delivered_to), + source: optionalString(payload.source), + precedence: stringList(payload.precedence), + }; +} + +/** One inventory row: key metadata only. */ +export type McpEnvKeyEntry = { + readonly key: string; + readonly secret: boolean; + readonly required: boolean | null; + readonly source: string | null; + readonly resolved_from: string | null; + readonly services: readonly string[] | null; +}; + +export type McpEnvKeysData = { + readonly format: string | null; + readonly environment: string | null; + readonly selected_scope: string | null; + readonly scopes: readonly string[] | null; + readonly keys: readonly McpEnvKeyEntry[]; + readonly missing_required: readonly string[]; +}; + +/** + * Project `hack env list --json` stdout onto a key inventory. Row `value`, + * `materialized`, `storage`, `warnings` and every other field are dropped. A + * row whose key does not match {@link MCP_ENV_KEY_PATTERN} is dropped whole. + */ +export function projectEnvKeys(payload: unknown): McpEnvKeysData | null { + if (!isRecord(payload)) { + return null; + } + const vars = Array.isArray(payload.vars) ? payload.vars : []; + const keys: McpEnvKeyEntry[] = []; + for (const row of vars) { + if (!isRecord(row)) { + continue; + } + const key = envKeyOrNull(row.key); + if (key === null) { + continue; + } + const source = optionalString(row.source); + keys.push({ + key, + secret: source === "keychain", + required: optionalBoolean(row.required), + source, + resolved_from: optionalString(row.resolved_from), + services: stringList(row.services), + }); + } + const selection = isRecord(payload.env_selection) + ? payload.env_selection + : {}; + return { + format: optionalString(payload.format), + environment: optionalString(selection.effective), + selected_scope: optionalString(payload.selected_scope), + scopes: stringList(payload.scopes), + keys, + missing_required: (stringList(payload.missing_required) ?? []).filter( + (key) => MCP_ENV_KEY_PATTERN.test(key) + ), + }; +} + +export type McpInstanceService = { + readonly service: string; + readonly containers: number; + readonly running: number; +}; + +/** One runtime instance: the default instance (`branch: null`) or a branch. */ +export type McpProjectInstance = { + readonly branch: string | null; + readonly compose_project: string | null; + readonly working_dir: string | null; + readonly services: readonly McpInstanceService[]; +}; + +export type McpInstancesData = { + readonly project: string; + readonly found: boolean; + readonly runtime_ok: boolean | null; + readonly instances: readonly McpProjectInstance[]; +}; + +function projectRuntime(opts: { + readonly branch: string | null; + readonly runtime: unknown; +}): McpProjectInstance | null { + if (!isRecord(opts.runtime)) { + return null; + } + const services: McpInstanceService[] = []; + for (const entry of Array.isArray(opts.runtime.services) + ? opts.runtime.services + : []) { + if (!(isRecord(entry) && typeof entry.service === "string")) { + continue; + } + const containers = Array.isArray(entry.containers) + ? entry.containers.filter(isRecord) + : []; + services.push({ + service: entry.service, + containers: containers.length, + running: containers.filter((container) => container.state === "running") + .length, + }); + } + return { + branch: opts.branch, + compose_project: optionalString(opts.runtime.project), + working_dir: optionalString(opts.runtime.working_dir), + services, + }; +} + +/** + * Select `project` from a `hack projects --json` payload and list its runtime + * instances: the default instance first (when running), then each branch + * instance in the order the CLI reported them. + */ +export function projectInstances(opts: { + readonly payload: unknown; + readonly project: string; +}): McpInstancesData { + const payload = isRecord(opts.payload) ? opts.payload : {}; + const projects = Array.isArray(payload.projects) ? payload.projects : []; + const view = projects.find( + (entry): entry is Record => + isRecord(entry) && + typeof entry.name === "string" && + normalizeProjectName(entry.name) === opts.project + ); + const runtimeOk = optionalBoolean(payload.runtime_ok); + if (!view) { + return { + project: opts.project, + found: false, + runtime_ok: runtimeOk, + instances: [], + }; + } + const instances: McpProjectInstance[] = []; + const base = projectRuntime({ branch: null, runtime: view.runtime }); + if (base) { + instances.push(base); + } + for (const entry of Array.isArray(view.branch_runtime) + ? view.branch_runtime + : []) { + if (!(isRecord(entry) && typeof entry.branch === "string")) { + continue; + } + const instance = projectRuntime({ + branch: entry.branch, + runtime: entry.runtime, + }); + if (instance) { + instances.push(instance); + } + } + return { + project: opts.project, + found: true, + runtime_ok: runtimeOk, + instances, + }; +} diff --git a/src/mcp/server.ts b/src/mcp/server.ts index e9376c87..b6d595cd 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -28,12 +28,21 @@ import { } from "../lib/project-name.ts"; import { AmbiguousProjectIdError, + type RegisteredProject, readProjectsRegistry, resolveRegisteredProjectByName, + selectRegisteredProjectByName, } from "../lib/projects-registry.ts"; import { readLinesFromStream } from "../ui/lines.ts"; import type { McpCommandAdmission } from "./command-admission.ts"; import { superviseMcpCommand } from "./command-lifetime.ts"; +import { + MCP_ENV_KEY_PATTERN, + MCP_ENV_OUTPUT_WITHHELD_MESSAGE, + projectEnvExplain, + projectEnvKeys, + projectInstances, +} from "./inspect-projections.ts"; import { classifyLogCollection, MCP_LOG_BACKEND_ERROR, @@ -161,6 +170,10 @@ type ProjectArgsResolution = | { readonly ok: true; readonly args: string[] } | { readonly ok: false; readonly reason: FixedRefusalReason }; +type ProjectNameResolution = + | { readonly ok: true; readonly name: string } + | { readonly ok: false; readonly reason: FixedRefusalReason }; + const DEFAULT_CMD_TIMEOUT_MS = 120_000; const DEFAULT_LOG_TAIL_EVENTS = 200; const DEFAULT_LOG_TAIL_MS = 5000; @@ -234,6 +247,14 @@ function composeIdentifierInput(description: string) { .describe(description); } +function envKeyInput(description: string) { + return z + .string() + .max(IDENTIFIER_MAX_LENGTH) + .regex(MCP_ENV_KEY_PATTERN) + .describe(description); +} + function branchIdentifierInput(description: string) { return z .string() @@ -433,6 +454,15 @@ function registerTools(opts: { until: z.string().describe("End time (RFC3339 or duration)").optional(), }; + const envSelectionInput = { + env: composeIdentifierInput( + "Env overlay name; 'base' or 'default' selects base layers only" + ).optional(), + service: composeIdentifierInput( + "Env scope: 'global' or a compose service name" + ).optional(), + }; + register({ name: "hack.projects.list", title: "List hack projects", @@ -900,6 +930,221 @@ function registerTools(opts: { }, }); + register({ + name: "hack.project.exec", + title: "Exec command in running service", + description: + "Run a command inside an already-running service container (hack exec). Unlike hack.project.run it never starts a new container. Command arguments are passed to the child after `--` and are never parsed as hack options. Output is bounded by the shared 8 MiB capture limit and timeoutMs.", + annotations: DESTRUCTIVE_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + ...profileInput, + service: composeIdentifierInput("Running compose service name"), + cmd: z + .array(z.string()) + .min(1) + .describe("Command and arguments (at least one entry)"), + workdir: z + .string() + .describe("Working directory inside container") + .optional(), + timeoutMs: boundedLimitInput({ + ceiling: MCP_INPUT_CEILINGS.timeoutMs, + description: "Command timeout in ms", + }).optional(), + }, + handler: async (input, extra) => { + const resolved = await resolveProjectArgs({ + selection: toProjectSelection(input), + cwd: opts.context.cwd, + registryPath: opts.context.registryPath, + }); + if (!resolved.ok) { + return buildToolError(resolved); + } + + const timeoutMs = input.timeoutMs ?? DEFAULT_CMD_TIMEOUT_MS; + const result = await runHackCommand({ + context: opts.context, + signal: extra.signal, + tool: "hack.project.exec", + args: buildExecArgs({ + projectArgs: resolved.args, + branch: input.branch, + profiles: input.profiles, + workdir: input.workdir, + service: input.service, + cmd: input.cmd, + }), + timeoutMs, + }); + + return buildToolResult({ + result, + data: { + limits: { timeoutMs, outputBytes: MCP_OUTPUT_LIMIT_BYTES }, + }, + }); + }, + }); + + register({ + name: "hack.project.doctor", + title: "Diagnose project setup", + description: + "Run read-only setup diagnostics (hack doctor --json). Never applies repairs: --fix and the migration or recovery modes are not reachable from MCP.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...branchInput, + }, + handler: async (input, extra) => { + const resolved = await resolveProjectArgs({ + selection: toProjectSelection(input), + cwd: opts.context.cwd, + registryPath: opts.context.registryPath, + selectByPath: true, + }); + if (!resolved.ok) { + return buildToolError(resolved); + } + + const result = await runHackCommand({ + context: opts.context, + signal: extra.signal, + tool: "hack.project.doctor", + args: buildDoctorArgs({ + projectArgs: resolved.args, + branch: input.branch, + }), + }); + + return buildToolResult({ result, data: parseJson(result.stdout) }); + }, + }); + + register({ + name: "hack.project.env.explain", + title: "Explain env key", + description: + "Explain where one env key comes from: availability, secret flag, scope, source and precedence. Never returns the value; child stdout and stderr are withheld.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...envSelectionInput, + key: envKeyInput("Env key name (letters, digits and _)"), + target: z + .enum(["host", "compose"]) + .describe("Env view: host (default) or compose") + .optional(), + }, + handler: async (input, extra) => { + const resolved = await resolveProjectArgs({ + selection: toProjectSelection(input), + cwd: opts.context.cwd, + registryPath: opts.context.registryPath, + }); + if (!resolved.ok) { + return buildToolError(resolved); + } + + const result = await runHackCommand({ + context: opts.context, + signal: extra.signal, + tool: "hack.project.env.explain", + args: buildEnvExplainArgs({ + projectArgs: resolved.args, + env: input.env, + service: input.service, + target: input.target, + key: input.key, + }), + }); + + return buildValueFreeToolResult({ + result, + data: projectEnvExplain(parseJson(result.stdout)), + }); + }, + }); + + register({ + name: "hack.project.env.keys", + title: "List env keys", + description: + "List env key names with secret flag, source and scope metadata. Never returns values (never --show-secrets); child stdout and stderr are withheld.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + ...envSelectionInput, + }, + handler: async (input, extra) => { + const resolved = await resolveProjectArgs({ + selection: toProjectSelection(input), + cwd: opts.context.cwd, + registryPath: opts.context.registryPath, + }); + if (!resolved.ok) { + return buildToolError(resolved); + } + + const result = await runHackCommand({ + context: opts.context, + signal: extra.signal, + tool: "hack.project.env.keys", + args: buildEnvKeysArgs({ + projectArgs: resolved.args, + env: input.env, + service: input.service, + }), + }); + + return buildValueFreeToolResult({ + result, + data: projectEnvKeys(parseJson(result.stdout)), + }); + }, + }); + + register({ + name: "hack.project.instances", + title: "List project instances", + description: + "List a registered project's runtime instances: the default instance and each branch instance, with per-service container counts.", + annotations: READ_ONLY_TOOL_ANNOTATIONS, + inputSchema: { + ...projectSelectorInput, + }, + handler: async (input, extra) => { + const resolved = await resolveRegisteredProjectName({ + selection: toProjectSelection(input), + cwd: opts.context.cwd, + registryPath: opts.context.registryPath, + }); + if (!resolved.ok) { + return buildToolError(resolved); + } + + const result = await runHackCommand({ + context: opts.context, + signal: extra.signal, + tool: "hack.project.instances", + args: ["projects", "--json", `--project=${resolved.name}`], + }); + + // `data` replaces stdout: the raw listing covers every matching + // project and carries container inspect data the projection omits. + return buildToolResult({ + result: { ...result, stdout: "" }, + data: projectInstances({ + payload: parseJson(result.stdout), + project: resolved.name, + }), + }); + }, + }); + installTypedToolDispatch({ server: opts.server, tools }); } @@ -1069,6 +1314,30 @@ function buildToolResult(opts: { }; } +/** + * Result for the env tools. The child's stdout and stderr are never returned: + * `stdout` is always empty, and `stderr` is empty on success or the fixed + * {@link MCP_ENV_OUTPUT_WITHHELD_MESSAGE} on failure. Only `data`, already + * projected onto allowlisted metadata, carries the child's answer. Exit status, + * `outputTruncated` and the argv in `command` follow {@link buildToolResult}. + */ +function buildValueFreeToolResult(opts: { + readonly result: HackCommandResult; + readonly data: unknown; +}): CallToolResult { + const ok = opts.result.exitCode === 0; + return buildToolResult({ + result: { + command: opts.result.command, + exitCode: opts.result.exitCode, + stdout: "", + stderr: ok ? "" : MCP_ENV_OUTPUT_WITHHELD_MESSAGE, + ...(opts.result.outputTruncated ? { outputTruncated: true } : {}), + }, + data: opts.data, + }); +} + /** * Logs tools result. Unlike {@link buildToolResult}, success follows the * collection outcome rather than the child's exit status: a cutoff is `ok` @@ -1167,6 +1436,93 @@ export function buildRunArgs(opts: { ]; } +/** + * `hack exec` argv: same boundary as {@link buildRunArgs}. Every option value + * is fused and `--` precedes the service, so `cmd` entries such as `--branch` + * or `--help` reach the container command, never hack's parser. + */ +export function buildExecArgs(opts: { + readonly projectArgs: readonly string[]; + readonly branch?: string; + readonly profiles?: readonly string[]; + readonly workdir?: string; + readonly service: string; + readonly cmd: readonly string[]; +}): string[] { + return [ + "exec", + ...opts.projectArgs, + ...buildBranchArgs({ branch: opts.branch }), + ...buildProfileArgs({ profiles: opts.profiles }), + ...(opts.workdir ? [`--workdir=${opts.workdir}`] : []), + "--", + opts.service, + ...opts.cmd, + ]; +} + +/** + * `hack doctor` argv. The set is closed: project path, branch and `--json`. + * `--fix`, `--migrate-env-config` and the migration and recovery modes are + * never emitted, and the CLI also refuses `--json` with `--fix`. + */ +export function buildDoctorArgs(opts: { + readonly projectArgs: readonly string[]; + readonly branch?: string; +}): string[] { + return [ + "doctor", + ...opts.projectArgs, + ...buildBranchArgs({ branch: opts.branch }), + "--json", + ]; +} + +/** `hack env explain --json` argv; the validated key follows `--`. */ +export function buildEnvExplainArgs(opts: { + readonly projectArgs: readonly string[]; + readonly env?: string; + readonly service?: string; + readonly target?: "host" | "compose"; + readonly key: string; +}): string[] { + return [ + "env", + "explain", + ...opts.projectArgs, + ...buildEnvScopeArgs(opts), + ...(opts.target ? [`--target=${opts.target}`] : []), + "--json", + "--", + opts.key, + ]; +} + +/** `hack env list --json` argv. `--show-secrets` is never emitted. */ +export function buildEnvKeysArgs(opts: { + readonly projectArgs: readonly string[]; + readonly env?: string; + readonly service?: string; +}): string[] { + return [ + "env", + "list", + ...opts.projectArgs, + ...buildEnvScopeArgs(opts), + "--json", + ]; +} + +function buildEnvScopeArgs(opts: { + readonly env?: string; + readonly service?: string; +}): string[] { + return [ + ...(opts.env ? [`--env=${opts.env}`] : []), + ...(opts.service ? [`--service=${opts.service}`] : []), + ]; +} + function buildLogSelectionArgs(input: { readonly loki?: boolean; readonly compose?: boolean; @@ -1225,6 +1581,11 @@ async function resolveProjectArgs(opts: { readonly selection: ProjectSelection; readonly cwd: string; readonly allowUnregistered?: boolean; + /** + * Emit `--path=` for a projectName selection, for commands + * (doctor) that accept `--path` but not `--project`. + */ + readonly selectByPath?: boolean; }): Promise { const selection = opts.selection; const picked = [ @@ -1240,6 +1601,7 @@ async function resolveProjectArgs(opts: { return await resolveProjectNameArgs({ projectName: selection.projectName, registryPath: opts.registryPath, + selectByPath: opts.selectByPath === true, }); } @@ -1282,6 +1644,7 @@ async function resolveProjectArgs(opts: { async function resolveProjectNameArgs(opts: { readonly projectName: string; readonly registryPath: string; + readonly selectByPath: boolean; }): Promise { const name = normalizeProjectName(opts.projectName); if (!name) { @@ -1295,7 +1658,14 @@ async function resolveProjectNameArgs(opts: { if (!project) { return { ok: false, reason: "project_not_found" }; } - return { ok: true, args: [`--project=${name}`] }; + return { + ok: true, + args: [ + opts.selectByPath + ? `--path=${project.projectRoot}` + : `--project=${name}`, + ], + }; } catch (error) { if ( error instanceof AmbiguousProjectNameError || @@ -1307,6 +1677,102 @@ async function resolveProjectNameArgs(opts: { } } +/** + * Resolve a selection to one registered project name, for commands that select + * by name only (`hack projects --project`). A path (or, with no selection, the + * session cwd) selects the registered project whose repo root or project dir + * contains it; the deepest root wins, and two distinct registrations at that + * depth are ambiguous. Unregistered paths are refused rather than guessed. + */ +async function resolveRegisteredProjectName(opts: { + readonly registryPath: string; + readonly selection: ProjectSelection; + readonly cwd: string; +}): Promise { + const selection = opts.selection; + const picked = [ + selection.projectName, + selection.repoRoot, + selection.path, + ].filter((value) => value !== undefined); + if (picked.length > 1) { + return { ok: false, reason: "conflicting_project_selection" }; + } + + const registry = await readProjectsRegistry({ + registryPath: opts.registryPath, + }); + if (selection.projectName !== undefined) { + const name = normalizeProjectName(selection.projectName); + if (!name) { + return { ok: false, reason: "invalid_project_name" }; + } + try { + const match = selectRegisteredProjectByName({ + projects: registry.projects, + name, + }); + return match + ? { ok: true, name } + : { ok: false, reason: "project_not_found" }; + } catch (error) { + if ( + error instanceof AmbiguousProjectNameError || + error instanceof AmbiguousProjectIdError + ) { + return { ok: false, reason: "ambiguous_project" }; + } + throw error; + } + } + + const pathLike = selection.repoRoot ?? selection.path; + const absPath = resolve(opts.cwd, pathLike ?? "."); + if (pathLike && !(await pathExists(absPath))) { + return { ok: false, reason: "path_not_found" }; + } + const notFound: FixedRefusalReason = pathLike + ? "path_not_registered" + : "no_project_context"; + const containing = selectContainingRegistrations({ + projects: registry.projects, + absPath, + }); + const names = new Set( + containing.map((project) => normalizeProjectName(project.name)) + ); + if (names.size > 1) { + return { ok: false, reason: "ambiguous_project" }; + } + const [name] = [...names]; + return name ? { ok: true, name } : { ok: false, reason: notFound }; +} + +/** Registrations at the deepest root that contains `absPath`. */ +function selectContainingRegistrations(opts: { + readonly projects: readonly RegisteredProject[]; + readonly absPath: string; +}): RegisteredProject[] { + let best: RegisteredProject[] = []; + let bestDepth = -1; + for (const project of opts.projects) { + for (const root of [project.repoRoot, project.projectDir]) { + const resolvedRoot = resolve(root); + if (!isPathWithin({ parent: resolvedRoot, child: opts.absPath })) { + continue; + } + const depth = resolvedRoot.length; + if (depth > bestDepth) { + best = [project]; + bestDepth = depth; + } else if (depth === bestDepth && !best.includes(project)) { + best.push(project); + } + } + } + return best; +} + async function isPathAllowed(opts: { readonly registryPath: string; readonly absPath: string; diff --git a/tests/mcp-inspect-tools.test.ts b/tests/mcp-inspect-tools.test.ts new file mode 100644 index 00000000..b7fa5631 --- /dev/null +++ b/tests/mcp-inspect-tools.test.ts @@ -0,0 +1,992 @@ +import { afterEach, expect, test } from "bun:test"; +import { + chmod, + mkdir, + mkdtemp, + readFile, + realpath, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; + +import { parseCliArgv, resolveCommand } from "../src/cli/command.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { + MCP_ENV_OUTPUT_WITHHELD_MESSAGE, + projectEnvKeys, +} from "../src/mcp/inspect-projections.ts"; +import { MCP_OUTPUT_LIMIT_BYTES } from "../src/mcp/output-budget.ts"; +import { + buildEnvExplainArgs, + buildExecArgs, + createMcpServer, + MCP_INPUT_CEILINGS, +} from "../src/mcp/server.ts"; + +// --------------------------------------------------------------------------- +// HACK-1133: exec, doctor, env.explain, env.keys and instances MCP tools. +// +// Every fixture is a temp root with its own HOME, HACK_HOME and registry. The +// hack binary is a stub selected by HACK_MCP_COMMAND: it records its argv and +// replays a scripted response per command, so no Docker/OrbStack socket, agent +// client config, keychain or real home directory is touched. +// --------------------------------------------------------------------------- + +type ScriptedResponse = { + readonly stdout?: string; + readonly stderr?: string; + readonly exitCode?: number; +}; + +/** Keyed by the command word, or `env `. */ +type ScriptedResponses = Readonly>; + +type StubInvocation = { + readonly args: readonly string[]; + readonly noInteractive: string | null; +}; + +type InspectFixture = { + readonly root: string; + readonly repo: string; + readonly client: Client; + readonly script: (responses: ScriptedResponses) => Promise; + readonly invocations: () => Promise; + readonly auditLog: string; +}; + +type FixtureOptions = { + /** Session cwd relative to the fixture root (default: the demo repo). */ + readonly cwdName?: string; + /** Extra registrations, repo roots relative to the fixture root. */ + readonly extraProjects?: readonly { name: string; repoRoot: string }[]; +}; + +const fixtureClients: Client[] = []; +const fixtureRoots: string[] = []; + +afterEach(async () => { + await Promise.all(fixtureClients.splice(0).map((mcp) => mcp.close())); + await Promise.all( + fixtureRoots + .splice(0) + .map((root) => rm(root, { recursive: true, force: true })) + ); +}); + +function registryRow(opts: { readonly name: string; readonly repo: string }) { + return { + id: opts.name, + createdAt: "2026-01-01T00:00:00Z", + name: opts.name, + repoRoot: opts.repo, + projectDirName: ".hack", + projectDir: join(opts.repo, ".hack"), + }; +} + +async function writeProject(opts: { + readonly repo: string; + readonly name: string; +}): Promise { + await mkdir(join(opts.repo, ".hack"), { recursive: true }); + await writeFile( + join(opts.repo, ".hack", "docker-compose.yml"), + "services: {}\n" + ); + await writeFile( + join(opts.repo, ".hack", "hack.config.json"), + JSON.stringify({ name: opts.name }) + ); +} + +async function startFixture( + opts: FixtureOptions = {} +): Promise { + // realpath: macOS tmpdir is a symlink; the registry stores canonical roots. + const root = await realpath( + await mkdtemp(join(tmpdir(), "hack-mcp-inspect-")) + ); + fixtureRoots.push(root); + const repo = join(root, "repo"); + const state = join(root, "state"); + const registryDir = join(root, "registry"); + await writeProject({ repo, name: "demo" }); + await mkdir(registryDir); + const extra = opts.extraProjects ?? []; + for (const entry of extra) { + await writeProject({ repo: join(root, entry.repoRoot), name: entry.name }); + } + await writeFile( + join(registryDir, "projects.json"), + JSON.stringify({ + version: 1, + projects: [ + registryRow({ name: "demo", repo }), + ...extra.map((entry) => + registryRow({ name: entry.name, repo: join(root, entry.repoRoot) }) + ), + ], + }) + ); + let cwd = repo; + if (opts.cwdName !== undefined) { + cwd = join(root, opts.cwdName); + await mkdir(cwd, { recursive: true }); + } + const argvLog = join(root, "argv.jsonl"); + const responsesPath = join(root, "responses.json"); + await writeFile(responsesPath, "{}"); + const stub = join(root, "hack-stub"); + await writeFile( + stub, + [ + `#!${process.execPath}`, + 'import { appendFileSync, readFileSync } from "node:fs";', + "const args = Bun.argv.slice(2);", + `appendFileSync(${JSON.stringify(argvLog)}, \`\${JSON.stringify({ args, noInteractive: process.env.HACK_NO_INTERACTIVE ?? null })}\\n\`);`, + `const responses = JSON.parse(readFileSync(${JSON.stringify(responsesPath)}, "utf8"));`, + 'const key = args[0] === "env" ? `env ${args[1]}` : args[0];', + 'const response = responses[key] ?? { stdout: "{\\"ok\\":true}\\n" };', + 'process.stdout.write(response.stdout ?? "");', + 'process.stderr.write(response.stderr ?? "");', + "process.exitCode = response.exitCode ?? 0;", + ].join("\n") + ); + await chmod(stub, 0o755); + const env: Record = { + ...process.env, + HOME: root, + HACK_HOME: state, + HACK_GLOBAL_CONFIG_PATH: join(registryDir, "config.json"), + HACK_MCP_COMMAND: stub, + }; + Reflect.deleteProperty(env, "HACK_NO_INTERACTIVE"); + const server = createMcpServer({ cwd, env }); + const client = new Client({ name: "hack-mcp-inspect", version: "1" }); + fixtureClients.push(client); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + await client.connect(clientTransport); + return { + root, + repo, + client, + auditLog: join(state, "mcp-audit.log"), + script: async (responses) => { + await writeFile(responsesPath, JSON.stringify(responses)); + }, + invocations: async () => { + if (!(await Bun.file(argvLog).exists())) { + return []; + } + return (await readFile(argvLog, "utf8")) + .split("\n") + .filter((line) => line.length > 0) + .map((line) => JSON.parse(line) as StubInvocation); + }, + }; +} + +type ToolPayload = { + readonly ok: boolean; + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; + readonly command: string; + readonly data?: unknown; + readonly spawned?: false; + readonly error?: { code: string; reason: string; message: string }; +}; + +async function call(opts: { + readonly fixture: InspectFixture; + readonly name: string; + readonly arguments: Record; +}): Promise<{ + readonly isError: unknown; + readonly payload: ToolPayload; + readonly serialized: string; +}> { + const result = await opts.fixture.client.callTool({ + name: opts.name, + arguments: opts.arguments, + }); + return { + isError: result.isError, + payload: result.structuredContent as ToolPayload, + serialized: JSON.stringify(result), + }; +} + +// --------------------------------------------------------------------------- +// hack.project.exec +// --------------------------------------------------------------------------- + +test("exec targets the running container with hack exec and ends hack options with --", async () => { + const fixture = await startFixture(); + const exec = await call({ + fixture, + name: "hack.project.exec", + arguments: { + projectName: "demo", + branch: "feature/x", + profiles: ["tools"], + workdir: "/srv/app dir", + service: "api", + cmd: ["sh", "-c", "--help", "--branch", "other", "--version"], + timeoutMs: 5000, + }, + }); + expect(exec.isError).toBeUndefined(); + expect(exec.payload).toMatchObject({ + ok: true, + data: { limits: { timeoutMs: 5000, outputBytes: MCP_OUTPUT_LIMIT_BYTES } }, + }); + const run = await call({ + fixture, + name: "hack.project.run", + arguments: { projectName: "demo", service: "api", cmd: ["true"] }, + }); + expect(run.payload.ok).toBe(true); + expect(await fixture.invocations()).toEqual([ + { + args: [ + "exec", + "--project=demo", + "--branch=feature/x", + "--profile=tools", + "--workdir=/srv/app dir", + "--", + "api", + "sh", + "-c", + "--help", + "--branch", + "other", + "--version", + ], + noInteractive: "1", + }, + // run stays a separate tool and a separate hack command. + { + args: ["run", "--project=demo", "--", "api", "true"], + noInteractive: "1", + }, + ]); +}); + +test("exec echoes the default timeout and the shared output bound", async () => { + const fixture = await startFixture(); + const exec = await call({ + fixture, + name: "hack.project.exec", + arguments: { service: "api", cmd: ["true"] }, + }); + expect(exec.payload).toMatchObject({ + ok: true, + data: { + limits: { timeoutMs: 120_000, outputBytes: MCP_OUTPUT_LIMIT_BYTES }, + }, + }); + // No selection: the session cwd is the demo project, so no project flag. + expect((await fixture.invocations()).map((entry) => entry.args)).toEqual([ + ["exec", "--", "api", "true"], + ]); +}); + +test("hack's own parser reads exec child flags as container argv", () => { + const argv = buildExecArgs({ + projectArgs: ["--project=demo"], + service: "api", + cmd: ["node", "--version", "--branch", "other", "--help"], + }); + const parsed = parseCliArgv(CLI_SPEC, argv); + expect(parsed.values.version).toBeUndefined(); + expect(parsed.values.help).toBeUndefined(); + expect(parsed.values.branch).toBeUndefined(); + expect(parsed.values.project).toBe("demo"); + const resolved = resolveCommand(CLI_SPEC, parsed.positionals); + expect(resolved.command?.name).toBe("exec"); + expect(resolved.remainingPositionals).toEqual([ + "api", + "node", + "--version", + "--branch", + "other", + "--help", + ]); +}); + +const MARKER = "-REFUSED-VALUE-MARKER"; + +type RefusedCall = { + readonly label: string; + readonly name: string; + readonly arguments: Record; +}; + +const refusedCalls: readonly RefusedCall[] = [ + { + label: "exec missing cmd", + name: "hack.project.exec", + arguments: { projectName: "demo", service: "api" }, + }, + { + label: "exec empty cmd", + name: "hack.project.exec", + arguments: { projectName: "demo", service: "api", cmd: [] }, + }, + { + label: "exec leading-dash service", + name: "hack.project.exec", + arguments: { projectName: "demo", service: MARKER, cmd: [MARKER] }, + }, + { + label: "exec empty service", + name: "hack.project.exec", + arguments: { projectName: "demo", service: "", cmd: ["true"] }, + }, + { + label: "exec leading-dash branch", + name: "hack.project.exec", + arguments: { service: "api", cmd: ["true"], branch: MARKER }, + }, + { + label: "exec leading-dash profile", + name: "hack.project.exec", + arguments: { service: "api", cmd: ["true"], profiles: [MARKER] }, + }, + ...[0, -1, 1.5, MCP_INPUT_CEILINGS.timeoutMs + 1].map((timeoutMs) => ({ + label: `exec timeoutMs=${timeoutMs}`, + name: "hack.project.exec", + arguments: { service: "api", cmd: ["true"], timeoutMs }, + })), + { + label: "doctor leading-dash branch", + name: "hack.project.doctor", + arguments: { branch: MARKER }, + }, + { + label: "explain missing key", + name: "hack.project.env.explain", + arguments: {}, + }, + { + label: "explain leading-dash key", + name: "hack.project.env.explain", + arguments: { key: MARKER }, + }, + { + label: "explain key with assignment", + name: "hack.project.env.explain", + arguments: { key: "TOKEN=ZQXMARKERVALUE" }, + }, + { + label: "explain dotted key", + name: "hack.project.env.explain", + arguments: { key: "api.ZQXMARKERKEY" }, + }, + { + label: "explain unknown target", + name: "hack.project.env.explain", + arguments: { key: "PORT", target: "ZQXMARKERTARGET" }, + }, + { + label: "explain leading-dash env", + name: "hack.project.env.explain", + arguments: { key: "PORT", env: MARKER }, + }, + { + label: "keys leading-dash service", + name: "hack.project.env.keys", + arguments: { service: MARKER }, + }, +]; + +test("invalid exec, doctor and env inputs are typed refusals that never spawn or audit", async () => { + const fixture = await startFixture(); + for (const refused of refusedCalls) { + const result = await call({ + fixture, + name: refused.name, + arguments: refused.arguments, + }); + expect({ label: refused.label, isError: result.isError }).toEqual({ + label: refused.label, + isError: true, + }); + expect(result.payload).toMatchObject({ + ok: false, + spawned: false, + command: "", + exitCode: 1, + stdout: "", + error: { code: "E_USAGE", reason: "invalid_arguments" }, + }); + // Closed diagnostics: the rejected value is never echoed back. + expect(result.serialized).not.toContain(MARKER); + expect(result.serialized).not.toContain("ZQXMARKER"); + } + expect(await fixture.invocations()).toEqual([]); + expect(await Bun.file(fixture.auditLog).exists()).toBe(false); +}); + +// --------------------------------------------------------------------------- +// hack.project.doctor +// --------------------------------------------------------------------------- + +const DOCTOR_REPORT = { + ok: true, + data: { + checks: [{ id: "bun", status: "ok", detail: "found", durationMs: 1 }], + summary: [], + counts: { ok: 1, warn: 0, error: 0 }, + }, +}; + +test("doctor runs doctor --json only, selecting a named project by path", async () => { + const fixture = await startFixture(); + await fixture.script({ + doctor: { stdout: `${JSON.stringify(DOCTOR_REPORT)}\n` }, + }); + const named = await call({ + fixture, + name: "hack.project.doctor", + arguments: { projectName: "demo", branch: "feature/x" }, + }); + expect(named.isError).toBeUndefined(); + expect(named.payload).toMatchObject({ ok: true, data: DOCTOR_REPORT }); + // Repair and migration inputs are not in the schema; they are dropped + // before argv construction and can never reach the child. + const repair = await call({ + fixture, + name: "hack.project.doctor", + arguments: { + fix: true, + migrateEnvConfig: true, + domainMigration: "apply", + nativeRunMapping: "repair", + }, + }); + expect(repair.payload.ok).toBe(true); + const invocations = await fixture.invocations(); + expect(invocations.map((entry) => entry.args)).toEqual([ + ["doctor", `--path=${fixture.repo}`, "--branch=feature/x", "--json"], + ["doctor", "--json"], + ]); + for (const entry of invocations) { + expect(entry.noInteractive).toBe("1"); + expect(entry.args.join(" ")).not.toMatch( + /--fix|--migrate|--domain-migration|--native-|--accept-legacy/ + ); + } +}); + +test("doctor reports failing checks as ok:false while keeping the parsed report", async () => { + const fixture = await startFixture(); + const failing = { + ok: true, + data: { ...DOCTOR_REPORT.data, counts: { ok: 0, warn: 0, error: 1 } }, + }; + await fixture.script({ + doctor: { stdout: `${JSON.stringify(failing)}\n`, exitCode: 1 }, + }); + const result = await call({ + fixture, + name: "hack.project.doctor", + arguments: { path: "." }, + }); + expect(result.isError).toBe(true); + expect(result.payload).toMatchObject({ + ok: false, + exitCode: 1, + data: failing, + }); + expect((await fixture.invocations()).map((entry) => entry.args)).toEqual([ + ["doctor", `--path=${fixture.repo}`, "--json"], + ]); +}); + +// --------------------------------------------------------------------------- +// hack.project.env.explain / hack.project.env.keys +// --------------------------------------------------------------------------- + +const SECRET_CANARY = "zqx-secret-canary-7731"; + +const EXPLAIN_PAYLOAD = { + key: "DATABASE_URL", + available: true, + secret: true, + format: "project_env_config_v1", + environment: "dev", + requested_scope: "api", + effective_scope: "api", + target: "compose", + delivered_to: ["compose"], + source: "project_env_config", + precedence: [".hack/hack.env.default.yaml", ".hack/hack.env.dev.yaml"], +}; + +test("env.explain forwards only allowlisted metadata and withholds child output", async () => { + const fixture = await startFixture(); + await fixture.script({ + "env explain": { + stdout: `${JSON.stringify({ + ...EXPLAIN_PAYLOAD, + value: SECRET_CANARY, + debug: { resolved: SECRET_CANARY }, + })}\n`, + stderr: `resolved ${SECRET_CANARY}\n`, + }, + }); + const result = await call({ + fixture, + name: "hack.project.env.explain", + arguments: { + projectName: "demo", + env: "dev", + service: "api", + target: "compose", + key: "DATABASE_URL", + }, + }); + expect(result.isError).toBeUndefined(); + expect(result.payload).toMatchObject({ + ok: true, + stdout: "", + stderr: "", + data: EXPLAIN_PAYLOAD, + }); + expect(result.serialized).not.toContain(SECRET_CANARY); + expect((await fixture.invocations()).map((entry) => entry.args)).toEqual([ + [ + "env", + "explain", + "--project=demo", + "--env=dev", + "--service=api", + "--target=compose", + "--json", + "--", + "DATABASE_URL", + ], + ]); +}); + +test("hack's own parser reads the env.explain key as the explain positional", () => { + const parsed = parseCliArgv( + CLI_SPEC, + buildEnvExplainArgs({ projectArgs: [], key: "_PORT" }) + ); + expect(parsed.values.json).toBe(true); + const resolved = resolveCommand(CLI_SPEC, parsed.positionals); + expect(resolved.command?.name).toBe("explain"); + expect(resolved.remainingPositionals).toEqual(["_PORT"]); +}); + +test("a failed env call returns fixed stderr and never the child's output", async () => { + const fixture = await startFixture(); + const failure = { + stdout: `partial ${SECRET_CANARY}\n`, + stderr: `YAML parse error near: TOKEN: ${SECRET_CANARY}\n`, + exitCode: 1, + }; + await fixture.script({ "env explain": failure, "env list": failure }); + for (const [name, args] of [ + ["hack.project.env.explain", { key: "TOKEN" }], + ["hack.project.env.keys", {}], + ] as const) { + const result = await call({ fixture, name, arguments: args }); + expect(result.isError).toBe(true); + expect(result.payload).toMatchObject({ + ok: false, + exitCode: 1, + stdout: "", + stderr: MCP_ENV_OUTPUT_WITHHELD_MESSAGE, + data: null, + }); + expect(result.serialized).not.toContain(SECRET_CANARY); + } + expect(await fixture.invocations()).toHaveLength(2); +}); + +const ENV_LIST_PAYLOAD = { + project: "demo", + env_selection: { + requested: null, + default: null, + effective: "dev", + overlay_path: "/repo/.hack/hack.env.dev.yaml", + overlay_exists: true, + }, + format: "project_env_config_v1", + scopes: ["global", "api"], + selected_scope: "global", + storage: { local_plaintext: { path: "/repo/.hack/.env", exists: true } }, + warnings: [{ kind: "shadowed", key: "PORT", message: SECRET_CANARY }], + vars: [ + { + key: "PORT", + required: false, + source: "plain_env", + services: null, + resolved_from: "project_env_config", + storage: { kind: "plaintext", location: SECRET_CANARY }, + value: SECRET_CANARY, + }, + { + key: "API_TOKEN", + required: true, + source: "keychain", + services: ["api"], + resolved_from: "project_env_config", + value: SECRET_CANARY, + }, + { key: `bad key ${SECRET_CANARY}`, source: "plain_env", value: "x" }, + "not-a-row", + ], + missing_required: ["API_TOKEN"], + materialized: { PORT: SECRET_CANARY }, + materialized_keys: ["PORT"], +}; + +const ENV_KEYS_DATA = { + format: "project_env_config_v1", + environment: "dev", + selected_scope: "global", + scopes: ["global", "api"], + keys: [ + { + key: "PORT", + secret: false, + required: false, + source: "plain_env", + resolved_from: "project_env_config", + services: null, + }, + { + key: "API_TOKEN", + secret: true, + required: true, + source: "keychain", + resolved_from: "project_env_config", + services: ["api"], + }, + ], + missing_required: ["API_TOKEN"], +}; + +test("env.keys lists key metadata only and never requests or returns values", async () => { + const fixture = await startFixture(); + await fixture.script({ + "env list": { + stdout: `${JSON.stringify(ENV_LIST_PAYLOAD)}\n`, + stderr: `warning: ${SECRET_CANARY}\n`, + }, + }); + const result = await call({ + fixture, + name: "hack.project.env.keys", + arguments: { path: ".", env: "dev", showSecrets: true }, + }); + expect(result.isError).toBeUndefined(); + expect(result.payload).toMatchObject({ + ok: true, + stdout: "", + stderr: "", + }); + expect(result.payload.data).toEqual(ENV_KEYS_DATA); + expect(result.serialized).not.toContain(SECRET_CANARY); + const invocations = await fixture.invocations(); + expect(invocations.map((entry) => entry.args)).toEqual([ + ["env", "list", `--path=${fixture.repo}`, "--env=dev", "--json"], + ]); + expect(invocations[0]?.args).not.toContain("--show-secrets"); +}); + +test("negative control: the scripted env payload does carry values that a raw tool returns", async () => { + // The projection itself is what removes values from the payload ... + expect(JSON.stringify(ENV_LIST_PAYLOAD)).toContain(SECRET_CANARY); + expect(JSON.stringify(projectEnvKeys(ENV_LIST_PAYLOAD))).not.toContain( + SECRET_CANARY + ); + // ... and the same stub output reaches a raw-forwarding tool unchanged, so + // the env tools' absence of the canary is not a stub or transport artifact. + const fixture = await startFixture(); + await fixture.script({ + projects: { stdout: `${JSON.stringify(ENV_LIST_PAYLOAD)}\n` }, + }); + const raw = await call({ + fixture, + name: "hack.projects.list", + arguments: {}, + }); + expect(raw.serialized).toContain(SECRET_CANARY); +}); + +// --------------------------------------------------------------------------- +// hack.project.instances +// --------------------------------------------------------------------------- + +function runtime(opts: { + readonly project: string; + readonly services: Record; +}) { + return { + project: opts.project, + working_dir: "/repo", + services: Object.entries(opts.services).map(([service, states]) => ({ + service, + containers: states.map((state, index) => ({ + id: `${service}-${index}`, + state, + labels: { secretish: "label" }, + mounts: [{ source: "/host/path" }], + })), + })), + }; +} + +const PROJECTS_PAYLOAD = { + runtime_ok: true, + projects: [ + { + name: "other", + runtime: runtime({ project: "other", services: { web: ["running"] } }), + branch_runtime: [], + }, + { + name: "demo", + runtime: runtime({ + project: "demo", + services: { api: ["running"], db: ["running", "exited"] }, + }), + branch_runtime: [ + { + branch: "feature-x", + runtime: runtime({ + project: "demo--feature-x", + services: { api: ["exited"] }, + }), + }, + { branch: "broken", runtime: null }, + ], + }, + ], +}; + +test("instances lists the default and branch instances of the selected project", async () => { + const fixture = await startFixture(); + await fixture.script({ + projects: { stdout: `${JSON.stringify(PROJECTS_PAYLOAD)}\n` }, + }); + const result = await call({ + fixture, + name: "hack.project.instances", + arguments: { projectName: "Demo" }, + }); + expect(result.isError).toBeUndefined(); + expect(result.payload.data).toEqual({ + project: "demo", + found: true, + runtime_ok: true, + instances: [ + { + branch: null, + compose_project: "demo", + working_dir: "/repo", + services: [ + { service: "api", containers: 1, running: 1 }, + { service: "db", containers: 2, running: 1 }, + ], + }, + { + branch: "feature-x", + compose_project: "demo--feature-x", + working_dir: "/repo", + services: [{ service: "api", containers: 1, running: 0 }], + }, + ], + }); + expect(result.serialized).not.toContain("/host/path"); + expect((await fixture.invocations()).map((entry) => entry.args)).toEqual([ + ["projects", "--json", "--project=demo"], + ]); +}); + +test("instances resolves paths and the session cwd to the deepest registration", async () => { + const fixture = await startFixture({ + extraProjects: [{ name: "inner", repoRoot: "repo/packages/inner" }], + }); + await mkdir(join(fixture.repo, "src"), { recursive: true }); + await fixture.script({ + projects: { stdout: `${JSON.stringify(PROJECTS_PAYLOAD)}\n` }, + }); + const fromCwd = await call({ + fixture, + name: "hack.project.instances", + arguments: {}, + }); + const fromSubdir = await call({ + fixture, + name: "hack.project.instances", + arguments: { path: "src" }, + }); + const nested = await call({ + fixture, + name: "hack.project.instances", + arguments: { repoRoot: resolve(fixture.repo, "packages/inner") }, + }); + expect(fromCwd.payload.data).toMatchObject({ project: "demo", found: true }); + expect(fromSubdir.payload.data).toMatchObject({ + project: "demo", + found: true, + }); + // "inner" is registered but absent from the runtime listing. + expect(nested.payload.data).toEqual({ + project: "inner", + found: false, + runtime_ok: true, + instances: [], + }); + expect((await fixture.invocations()).map((entry) => entry.args)).toEqual([ + ["projects", "--json", "--project=demo"], + ["projects", "--json", "--project=demo"], + ["projects", "--json", "--project=inner"], + ]); +}); + +test("instances refuses unregistered or ambiguous selections before spawning", async () => { + const fixture = await startFixture({ + cwdName: "outside", + extraProjects: [ + { name: "twin-a", repoRoot: "shared" }, + { name: "twin-b", repoRoot: "shared" }, + ], + }); + const cases = [ + { arguments: {}, reason: "no_project_context" }, + { arguments: { path: "." }, reason: "path_not_registered" }, + { arguments: { path: "missing-dir" }, reason: "path_not_found" }, + { + arguments: { path: join(fixture.root, "shared") }, + reason: "ambiguous_project", + }, + { arguments: { projectName: "absent" }, reason: "project_not_found" }, + { arguments: { projectName: "%%%" }, reason: "invalid_project_name" }, + { + arguments: { projectName: "demo", path: "." }, + reason: "conflicting_project_selection", + }, + ] as const; + for (const entry of cases) { + const result = await call({ + fixture, + name: "hack.project.instances", + arguments: entry.arguments, + }); + expect({ reason: entry.reason, isError: result.isError }).toEqual({ + reason: entry.reason, + isError: true, + }); + expect(result.payload).toMatchObject({ + ok: false, + spawned: false, + error: { reason: entry.reason }, + }); + expect(result.serialized).not.toContain(fixture.root); + } + expect(await fixture.invocations()).toEqual([]); + expect(await Bun.file(fixture.auditLog).exists()).toBe(false); +}); + +// --------------------------------------------------------------------------- +// Discovery and documentation contract +// --------------------------------------------------------------------------- + +const READ_ONLY = { readOnlyHint: true, openWorldHint: false }; +const DESTRUCTIVE = { + readOnlyHint: false, + destructiveHint: true, + idempotentHint: false, + openWorldHint: false, +}; + +test("tools/list annotates inspection tools read-only and exec destructive", async () => { + const fixture = await startFixture(); + const listed = await fixture.client.listTools(); + const byName = new Map(listed.tools.map((tool) => [tool.name, tool])); + const expectations = [ + ["hack.project.exec", DESTRUCTIVE, ["service", "cmd"]], + ["hack.project.doctor", READ_ONLY, []], + ["hack.project.env.explain", READ_ONLY, ["key"]], + ["hack.project.env.keys", READ_ONLY, []], + ["hack.project.instances", READ_ONLY, []], + ] as const; + for (const [name, annotations, required] of expectations) { + const tool = byName.get(name); + expect({ name, annotations: tool?.annotations }).toMatchObject({ + name, + annotations, + }); + expect(tool?.inputSchema.required ?? []).toEqual([...required]); + } + const doctor = byName.get("hack.project.doctor"); + expect(Object.keys(doctor?.inputSchema.properties ?? {}).sort()).toEqual([ + "branch", + "path", + "projectName", + "repoRoot", + ]); + const keys = byName.get("hack.project.env.keys"); + expect(Object.keys(keys?.inputSchema.properties ?? {})).not.toContain( + "showSecrets" + ); + expect( + byName.get("hack.project.env.explain")?.inputSchema.properties?.key + ).toMatchObject({ type: "string", pattern: "^[A-Za-z_][A-Za-z0-9_]*$" }); +}); + +test("docs/reference/mcp-tools.md documents every listed tool and its annotation class", async () => { + const fixture = await startFixture(); + const listed = await fixture.client.listTools(); + const doc = await readFile( + resolve(import.meta.dir, "../docs/reference/mcp-tools.md"), + "utf8" + ); + const rows = new Map( + doc + .split("\n") + .filter((line) => line.startsWith("| `hack.")) + .map((line) => { + const cells = line.split("|").map((cell) => cell.trim()); + return [cells[1]?.replaceAll("`", "") ?? "", cells[2] ?? ""] as const; + }) + ); + const classOf = (annotations: unknown): string => { + const value = annotations as { + readOnlyHint?: boolean; + destructiveHint?: boolean; + }; + if (value.readOnlyHint) { + return "read-only"; + } + return value.destructiveHint ? "destructive" : "additive"; + }; + expect([...rows.keys()].sort()).toEqual( + listed.tools.map((tool) => tool.name).sort() + ); + for (const tool of listed.tools) { + expect({ name: tool.name, class: rows.get(tool.name) }).toEqual({ + name: tool.name, + class: classOf(tool.annotations), + }); + } +}); diff --git a/tests/mcp.test.ts b/tests/mcp.test.ts index e6570aa6..1dfce8c0 100644 --- a/tests/mcp.test.ts +++ b/tests/mcp.test.ts @@ -783,6 +783,34 @@ test("tools/list publishes names, annotations and required inputs for every tool annotations: { title: "Initialize project", ...DESTRUCTIVE }, required: [], }, + { + name: "hack.project.exec", + annotations: { + title: "Exec command in running service", + ...DESTRUCTIVE, + }, + required: ["service", "cmd"], + }, + { + name: "hack.project.doctor", + annotations: { title: "Diagnose project setup", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.env.explain", + annotations: { title: "Explain env key", ...READ_ONLY }, + required: ["key"], + }, + { + name: "hack.project.env.keys", + annotations: { title: "List env keys", ...READ_ONLY }, + required: [], + }, + { + name: "hack.project.instances", + annotations: { title: "List project instances", ...READ_ONLY }, + required: [], + }, ]); const run = listed.tools.find((tool) => tool.name === "hack.project.run"); expect(run?.inputSchema.properties?.timeoutMs).toMatchObject({ From 85c67b0c701e7ca79e07eb45699a65e852187d74 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:21:30 -0400 Subject: [PATCH 37/52] fix(ci): allowlist the synthetic MCP inspect-tool canary for gitleaks The secret-scan job flagged the fixed test canary in tests/mcp-inspect-tools.test.ts as a generic-api-key (entropy 3.79). It is a synthetic marker the HACK-1133 tests use to prove env values are never echoed, not a credential. Add a narrow AND allowlist: the exact test path plus the exact canary line. The line regex uses (?m) because gitleaks' git mode passes the finding line with surrounding newlines. A scratch-repo control shows a token-like value on the next line of the same file, and the same canary line in another path, are still reported. Co-Authored-By: Claude Opus 5.5 --- .gitleaks.toml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.gitleaks.toml b/.gitleaks.toml index b3f4602d..488141f1 100644 --- a/.gitleaks.toml +++ b/.gitleaks.toml @@ -22,3 +22,10 @@ condition = "AND" paths = ['''^packages/runtime-core/tests/stream_relay_contract\.rs$'''] regexTarget = "line" regexes = ['''^\s*let token = "0123456789abcdef0123456789abcdef";\s*$'''] + +[[rules.allowlists]] +description = "MCP inspect-tool tests use a fixed synthetic canary to prove values are never echoed" +condition = "AND" +paths = ['''^tests/mcp-inspect-tools\.test\.ts$'''] +regexTarget = "line" +regexes = ['''(?m)^\s*const SECRET_CANARY = "zqx-secret-canary-7731";\s*$'''] From 10b6d73cfa8fab7460bff1299a9c328e43f4bf35 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:30:32 -0400 Subject: [PATCH 38/52] fix(agents): point the env matrix bullet at the offline env guide The env-composition section linked the command and delivery matrix by a bare GitHub URL. Packaged skills must work offline (#295), and that test fails once both changes land. Point at docs/env.md in words instead, bump the content revision, re-render the checked-in surfaces, and append the new hack-cli-skill and cursor-rules render fingerprints. Co-Authored-By: Claude Opus 5.5 --- .codex/skills/hack-cli/SKILL.md | 4 ++-- .cursor/rules/hack.mdc | 4 ++-- examples/basic/AGENTS.md | 4 ++-- examples/basic/CLAUDE.md | 4 ++-- plugins/hack/rules/hack.mdc | 4 ++-- plugins/hack/skills/hack-cli/SKILL.md | 4 ++-- src/agents/instruction-source.ts | 2 +- src/agents/integration-ownership.ts | 2 ++ src/agents/integration-revision.ts | 2 +- 9 files changed, 16 insertions(+), 14 deletions(-) diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index a46be0db..28776a4d 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). ## Product boundary @@ -152,7 +152,7 @@ Use `hack` as the primary interface for local-first development. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Branch instances (parallel envs) diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 1adb401c..2dcc4e8f 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -9,7 +9,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). ## Product boundary @@ -71,7 +71,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Agent integration maintenance diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index 8c448dc1..c5fb011c 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -129,7 +129,7 @@ Env composition: - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index 8c448dc1..c5fb011c 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -129,7 +129,7 @@ Env composition: - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). Branch instances (parallel envs): - Use a branch instance when you need two versions running at once (PR review, experiments, migrations) or want to keep a stable environment while testing another branch. diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index 77e495ec..e184f0ef 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). ## Product boundary @@ -76,7 +76,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Agent integration maintenance diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index a46be0db..28776a4d 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `3cb1b0eabdea` (version alone is not a freshness guarantee). +- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). ## Product boundary @@ -152,7 +152,7 @@ Use `hack` as the primary interface for local-first development. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. -- Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix +- Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). ## Branch instances (parallel envs) diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 153f83eb..25a1cb56 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -241,7 +241,7 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation.", "Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret.", "Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`.", - "Accepted flags, injection timing and refresh path per command: https://github.com/hack-dance/hack/blob/next/docs/env.md#command-and-delivery-matrix", + "Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository).", ], }, diff --git a/src/agents/integration-ownership.ts b/src/agents/integration-ownership.ts index 0e1cbb07..4156a625 100644 --- a/src/agents/integration-ownership.ts +++ b/src/agents/integration-ownership.ts @@ -105,6 +105,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "b6465c18c6d4f41b30a600b8", // v5.0.0-next.1..v5.0.0-next.2 "121d8f26374d24f2a09890a3", // v5.0.0-next.3..next "6449442a8f230412d4e5f9e7", // v5.0.0-next.4..next + "d83da046cf7541ba30950880", // v5.0.0-next.4..next (offline env-matrix pointer) ]), "hack-init-skill": new Set([ "cd15fb6d8ed8e25071d05e75", // v3.1.0..v4.0.3 @@ -131,6 +132,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "01d7431b9a5caaba3ffd3463", // v5.0.0-next.1..v5.0.0-next.2 "06cf6d0920afc1780b9946ba", // v5.0.0-next.3..next "cf0a31e197f9034485413674", // v5.0.0-next.4..next + "3a99c9097855f93808e482cb", // v5.0.0-next.4..next (offline env-matrix pointer) ]), }; diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index 6f3a624e..d418edef 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "3cb1b0eabdea"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "51163ae32e56"; From 4e3acee9f4d03e8b97a18a9c5375b49cb296918d Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:32:33 -0400 Subject: [PATCH 39/52] test(exec): cover failed targeted restart env digest handling A failed targeted restart must not record the newly resolved env as applied. The test drives restartCommand with a failing compose up and asserts the service's digest is dropped (freshness unknown, not unchanged) while the salt and other services' digests are kept. Refs HACK-1140 Co-Authored-By: Claude Opus 5.5 --- tests/project-up-startup-state.test.ts | 41 ++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/tests/project-up-startup-state.test.ts b/tests/project-up-startup-state.test.ts index 5f82989e..f2524eaf 100644 --- a/tests/project-up-startup-state.test.ts +++ b/tests/project-up-startup-state.test.ts @@ -442,6 +442,47 @@ test("up records value-free env digests and a targeted restart refreshes only it } }); +test("failed targeted restart does not record its service's env as applied", async () => { + const projectRoot = await createProject(); + const projectDir = resolve(projectRoot, ".hack"); + psRows.push( + JSON.stringify({ Service: "api", State: "running", ExitCode: 0 }), + JSON.stringify({ Service: "migrate", State: "exited", ExitCode: 0 }) + ); + const readEntry = () => + readProjectRuntimeStateEntry({ + projectDir, + composeProject: "startup-state-test", + }); + expect(await runDetachedUp({ projectRoot })).toBe(0); + const atUp = (await readEntry())?.envFingerprint; + expect(atUp?.services.api).toBeDefined(); + + const envPath = resolve(projectDir, "hack.env.default.yaml"); + await writeFile( + envPath, + (await readFile(envPath, "utf8")).replace( + ' SHARED_MODE: "compose"', + ' SHARED_MODE: "rotated"' + ) + ); + upExitCode = 1; + expect(await runRestart({ projectRoot, services: ["api"] })).not.toBe(0); + + const afterFailure = (await readEntry())?.envFingerprint; + // The delivered env is unknown, so api's digest is dropped rather than + // refreshed to the rotated env; exec then reports nothing for api. + expect( + evaluateComposeServiceEnvFreshness({ + record: afterFailure, + service: "api", + currentEnv: { SHARED_MODE: "rotated" }, + }) + ).toBe("unknown"); + expect(afterFailure?.salt).toBe(atUp?.salt); + expect(afterFailure?.services.migrate).toBe(atUp?.services.migrate); +}); + test("up --json emits E_STARTUP_INCOMPLETE for a created service", async () => { const projectRoot = await createProject(); psRows.push( From cb665d5efc66c96560d748ba6f1afcff8028d7d1 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:32:42 -0400 Subject: [PATCH 40/52] fix(env): scope the env-file symlink refusal doc to write paths docs/env.md said Hack refuses to read or write a symlinked env file, but only the write and update paths (env set/unset, materialize, the daemon env route) use the no-follow reader; code that only reads an env file still follows a symlink. Narrow the sentence accordingly. Add a test that calls replacePrivateFileAtomic directly with a live and a dangling symlink, so its own target check is covered instead of being masked by the earlier O_NOFOLLOW read in the env-file callers. Refs HACK-1127 Co-Authored-By: Claude Opus 5.5 --- docs/env.md | 7 ++++--- tests/private-env-file.test.ts | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/docs/env.md b/docs/env.md index ccd5e690..339205d0 100644 --- a/docs/env.md +++ b/docs/env.md @@ -199,10 +199,11 @@ Hack writes `.hack/.env` and `.hack/.env.` (from `materialize`, legacy `hack env set`, and the daemon env route) with mode `0600` by atomic replace: a private temporary file in `.hack/` is written, fsynced and renamed over the target, so readers never see a partial file. An existing file with group or other access is -tightened to `0600` even when its contents are unchanged. Hack refuses, with a fixed -message that names no file or value, to read or write an env file that is a symlink +tightened to `0600` even when its contents are unchanged. These write and update paths +refuse, with a fixed message that names no file or value, an env file that is a symlink (including a dangling one) or not a regular file, or one whose `.hack` directory is a -symlink. Replace the link with a regular file or directory to continue. +symlink. Replace the link with a regular file or directory to continue. Code paths +that only read an env file do not apply this check and still follow a symlink. For host execution, `hack host exec`, `hack host shell`, and lifecycle hooks/processes use a host-local view. That means Hack prefers host-usable values when it can: diff --git a/tests/private-env-file.test.ts b/tests/private-env-file.test.ts index 6d099d27..798792ca 100644 --- a/tests/private-env-file.test.ts +++ b/tests/private-env-file.test.ts @@ -22,6 +22,7 @@ import { } from "../src/constants.ts"; import { removeDotEnvKey, upsertDotEnvValue } from "../src/lib/hack-env.ts"; import { PRIVATE_ENV_FILE_REFUSAL } from "../src/lib/private-env-file.ts"; +import { replacePrivateFileAtomic } from "../src/lib/private-file.ts"; import { materializeProjectEnv, setProjectEnvValue, @@ -356,3 +357,35 @@ test("negative control: materializeProjectEnv refuses a symlinked .hack/.env tar expect(await readFile(project.outsideFile, "utf8")).toBe(OUTSIDE_TEXT); expect((await lstat(project.envFile)).isSymbolicLink()).toBe(true); }); + +test("negative control: replacePrivateFileAtomic itself refuses a symlinked target without the read guard", async () => { + const project = await createProject(); + const danglingTarget = resolve( + project.projectRoot, + "..", + "never-created.env" + ); + const danglingLink = `${project.envFile}.dangling`; + await symlink(project.outsideFile, project.envFile); + await symlink(danglingTarget, danglingLink); + + // Called directly, so the earlier O_NOFOLLOW read in the env-file callers + // cannot mask a missing check in the replace step. + for (const path of [project.envFile, danglingLink]) { + expectClosedRefusal( + await caught( + replacePrivateFileAtomic({ + path, + directories: [project.projectDir], + text: `${SENTINEL_KEY}=${SENTINEL_VALUE}\n`, + refusal: () => new Error(PRIVATE_ENV_FILE_REFUSAL), + }) + ), + [path, project.outsideFile, danglingTarget] + ); + expect((await lstat(path)).isSymbolicLink()).toBe(true); + } + expect(await readFile(project.outsideFile, "utf8")).toBe(OUTSIDE_TEXT); + await expect(lstat(danglingTarget)).rejects.toThrow(); + expect(await stagingLeftovers(project.projectDir)).toEqual([]); +}); From 3a6f73cd3f880ac82d756a5caab74aca12b9ac70 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:32:44 -0400 Subject: [PATCH 41/52] docs(mcp): clarify which instances hack.project.instances lists The tool reads runtime and branch_runtime from `hack projects --json`, so it lists only instances that currently have containers (running or stopped), not every configured branch. Per-service counts include lifecycle-process containers. Align the tool description, reference doc and projection comment. Co-Authored-By: Claude Opus 5.5 --- docs/reference/mcp-tools.md | 9 ++++++--- src/mcp/inspect-projections.ts | 4 ++-- src/mcp/server.ts | 2 +- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md index 7d387fbb..237bf124 100644 --- a/docs/reference/mcp-tools.md +++ b/docs/reference/mcp-tools.md @@ -181,13 +181,16 @@ MCP audit log records the same argv. ## `hack.project.instances` Lists a registered project's runtime instances: `hack projects --json ---project=`. +--project=`. It reads the project's `runtime` and `branch_runtime` +entries, so it lists only instances that currently have containers, running or +stopped. A configured branch that was never started is not listed. `data` is `{ project, found, runtime_ok, instances }`. `instances` lists the -default instance first, when it is running, as `branch: null`. Each branch +default instance first, when it has containers, as `branch: null`. Each branch instance follows, in CLI order. Each instance is `{ branch, compose_project, working_dir, services }`, and each service is `{ service, containers, running }` -with container counts. `stdout` is always `""`: the raw listing can cover other +with container counts. Both counts include lifecycle-process containers, which +the CLI adds to the service with the process name. `stdout` is always `""`: the raw listing can cover other projects and carries container labels, mounts and networks, which `data` omits. `stderr` is returned as captured. `found` is `false` when the project is registered but the CLI listing does not include it. diff --git a/src/mcp/inspect-projections.ts b/src/mcp/inspect-projections.ts index fa2d27a8..59661330 100644 --- a/src/mcp/inspect-projections.ts +++ b/src/mcp/inspect-projections.ts @@ -200,8 +200,8 @@ function projectRuntime(opts: { /** * Select `project` from a `hack projects --json` payload and list its runtime - * instances: the default instance first (when running), then each branch - * instance in the order the CLI reported them. + * instances: the default instance first (when it has containers), then each + * branch instance in the order the CLI reported them. */ export function projectInstances(opts: { readonly payload: unknown; diff --git a/src/mcp/server.ts b/src/mcp/server.ts index b6d595cd..cc6c00ce 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -1111,7 +1111,7 @@ function registerTools(opts: { name: "hack.project.instances", title: "List project instances", description: - "List a registered project's runtime instances: the default instance and each branch instance, with per-service container counts.", + "List a registered project's runtime instances from `hack projects --json`: only the default and branch instances that currently have containers (running or stopped); a branch that was never started is not listed. Per-service `containers` and `running` counts include lifecycle-process containers.", annotations: READ_ONLY_TOOL_ANNOTATIONS, inputSchema: { ...projectSelectorInput, From 60acda42737083b7951bab882a0681dedd38cda4 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 07:32:48 -0400 Subject: [PATCH 42/52] test(agents): assert plugin-default and no-write agent setup rules Add a semantic test that the agent-setup section names the hack plugin as the default integration and states that setup and onboarding never write AGENTS.md, CLAUDE.md, rule files, or repository skill copies unless the user asks, on every surface the section renders to. Correct the onboarding-prompt consumer comment: /hack-init ships in the default hack plugin, with standalone hack setup claude|codex as opt-in. Co-Authored-By: Claude Opus 5.5 --- src/agents/onboarding-prompt.ts | 4 +-- tests/agent-instruction-source.test.ts | 37 ++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/src/agents/onboarding-prompt.ts b/src/agents/onboarding-prompt.ts index 7636acbf..b67d7e67 100644 --- a/src/agents/onboarding-prompt.ts +++ b/src/agents/onboarding-prompt.ts @@ -6,8 +6,8 @@ import { INSTRUCTION_SECTIONS } from "./instruction-source.ts"; * Consumers (keep this list current — the content must never fork): * - `hack init --with claude|codex` (CLI handoff for new repos), * - `hack agent onboard` (CLI print for existing projects), - * - the `/hack-init` agent-side skill (thin pointer installed by - * `hack setup claude|codex`), + * - the `/hack-init` agent-side skill (thin pointer shipped by the default + * `hack` plugin, or exported by opt-in standalone `hack setup claude|codex`), * - the `hack-init` MCP prompt (no-shell clients), * - the copy-paste bootstrap block in `docs/guides/agent-first-setup.md`. */ diff --git a/tests/agent-instruction-source.test.ts b/tests/agent-instruction-source.test.ts index 8d65f973..ae125d9e 100644 --- a/tests/agent-instruction-source.test.ts +++ b/tests/agent-instruction-source.test.ts @@ -257,6 +257,43 @@ test("all surfaces contain the canonical bullets for their tagged sections", () expect(misses).toEqual([]); }); +test("agent setup guidance keeps the plugin default and the no-write rule", () => { + const agentSetup = INSTRUCTION_SECTIONS.find( + (section) => section.id === "agent-setup" + ); + expect(agentSetup).toBeDefined(); + const bullets = agentSetup?.bullets ?? []; + const pluginDefault = bullets.find((bullet) => + /^Default integration: the optional `hack` plugin\b/.test(bullet) + ); + expect(pluginDefault, "agent-setup lacks the plugin default").toBeDefined(); + expect(pluginDefault).toContain("ships no MCP server"); + expect(pluginDefault).toContain("Hack commands never install plugins"); + + const noWrite = bullets.find((bullet) => + /^Setup and onboarding never write\b/.test(bullet) + ); + expect(noWrite, "agent-setup lacks the no-write rule").toBeDefined(); + for (const target of [ + "AGENTS.md", + "CLAUDE.md", + "rule files", + "repository skill copies", + ]) { + expect(noWrite).toContain(target); + } + expect(noWrite).toMatch(/unless the user asks for them\.$/); + + for (const surface of agentSetup?.surfaces ?? []) { + const rendered = RENDERED_SURFACES[surface]; + expect(rendered, `surface "${surface}"`).toContain(pluginDefault ?? ""); + expect(rendered, `surface "${surface}"`).toContain(noWrite ?? ""); + } + expect(agentSetup?.surfaces).toEqual( + expect.arrayContaining(["docs", "skill", "primer"]) + ); +}); + test("linked worktree guidance is present in every surface", () => { for (const rendered of Object.values(RENDERED_SURFACES)) { expect(rendered).toContain("Linked git worktrees"); From 36cd27201fb420bce63f1e74c6212c5e4ea9aaef Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 08:40:06 -0400 Subject: [PATCH 43/52] test(e2e): keep Hack diagnostics out of exec-stale-env probe output Hosted CI failed "web should receive the env value resolved at up" because `hack exec web -- printenv` stdout began with Clack warnings (CoreDNS/Caddy not reachable) before the probe value. Every exec probe now runs with HACK_LOGGER=console so Hack's logger writes to stderr, and compares the last non-empty stdout line exactly. A wrong or missing value still fails (missing output yields null). Co-Authored-By: Claude Opus 5.5 --- tests/e2e/exec-stale-env-output.test.ts | 26 +++++++++++++++++++++++++ tests/e2e/scenarios/exec-stale-env.ts | 25 +++++++++++++++++++++--- 2 files changed, 48 insertions(+), 3 deletions(-) create mode 100644 tests/e2e/exec-stale-env-output.test.ts diff --git a/tests/e2e/exec-stale-env-output.test.ts b/tests/e2e/exec-stale-env-output.test.ts new file mode 100644 index 00000000..5404d73a --- /dev/null +++ b/tests/e2e/exec-stale-env-output.test.ts @@ -0,0 +1,26 @@ +import { describe, expect, test } from "bun:test"; +import { execProbeValue } from "./scenarios/exec-stale-env.ts"; + +describe("execProbeValue", () => { + test("returns the child's value after Hack diagnostic lines", () => { + const stdout = [ + "▲ CoreDNS is not reachable; internal DNS for *.hack is disabled.", + "▲ Caddy is not reachable; internal *.hack host mappings are disabled.", + "synthetic-stale-probe-at-up", + "", + ].join("\n"); + expect(execProbeValue({ stdout })).toBe("synthetic-stale-probe-at-up"); + }); + + test("trims CRLF and surrounding whitespace", () => { + expect(execProbeValue({ stdout: " value\r\n\r\n" })).toBe("value"); + }); + + test("does not match when the value is wrong or missing", () => { + expect( + execProbeValue({ stdout: "synthetic-stale-probe-at-up\nother\n" }) + ).toBe("other"); + expect(execProbeValue({ stdout: "" })).toBeNull(); + expect(execProbeValue({ stdout: "\n \n" })).toBeNull(); + }); +}); diff --git a/tests/e2e/scenarios/exec-stale-env.ts b/tests/e2e/scenarios/exec-stale-env.ts index 6eba98cb..f6bf5bb1 100644 --- a/tests/e2e/scenarios/exec-stale-env.ts +++ b/tests/e2e/scenarios/exec-stale-env.ts @@ -68,7 +68,7 @@ export const execStaleEnvScenario: Scenario = { service: "web", }); expect({ - that: fresh.stdout.trim() === VALUE_AT_UP, + that: execProbeValue({ stdout: fresh.stdout }) === VALUE_AT_UP, message: "web should receive the env value resolved at up", result: fresh, }); @@ -82,7 +82,7 @@ export const execStaleEnvScenario: Scenario = { for (const service of ["web", "api"]) { const stale = await execProbe({ ctx, root: fixture.root, service }); expect({ - that: stale.stdout.trim() === VALUE_AT_UP, + that: execProbeValue({ stdout: stale.stdout }) === VALUE_AT_UP, message: `${service} keeps its start-time env until recreated`, result: stale, }); @@ -106,7 +106,7 @@ export const execStaleEnvScenario: Scenario = { service: "web", }); expect({ - that: refreshed.stdout.trim() === VALUE_AFTER_EDIT, + that: execProbeValue({ stdout: refreshed.stdout }) === VALUE_AFTER_EDIT, message: "restarted web should receive the edited env value", result: refreshed, }); @@ -144,9 +144,13 @@ async function execProbe(opts: { readonly root: string; readonly service: string; }): Promise { + // Hack's own diagnostics (for example CoreDNS/Caddy reachability warnings) + // go through the logger; the console backend writes them to stderr so + // stdout carries only the child command's output. const result = await opts.ctx.cli({ args: ["exec", opts.service, "--", "printenv", PROBE_KEY], cwd: opts.root, + env: { HACK_LOGGER: "console" }, timeoutMs: EXEC_TIMEOUT_MS, }); expectExit({ @@ -157,6 +161,21 @@ async function execProbe(opts: { return result; } +/** + * The value printed by the exec'd `printenv`: the last non-empty stdout line, + * trimmed. Returns null when stdout has no non-empty line, so a missing value + * never equals an expected one. + */ +export function execProbeValue(opts: { + readonly stdout: string; +}): string | null { + const lines = opts.stdout + .split(/\r?\n/) + .map((line) => line.trim()) + .filter((line) => line.length > 0); + return lines.at(-1) ?? null; +} + function expectValueFreeWarning(opts: { readonly result: CliResult; readonly service: string; From 919de8ba4c7a48f6c8b1a72e5d7188a4b6b35f0d Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 08:45:15 -0400 Subject: [PATCH 44/52] test(e2e): reproduce dependency config-hash drift for hack run/exec Add the docker-tier compose-config-hash-drift scenario (HACK-1124, F03). It starts app and dep, drifts only dep's Compose config hash, and requires hack run and hack exec to keep every running container's full ID, StartedAt, RestartCount and config-hash label, with no leftover one-off. Positive controls: the run one-off sees the drifted value, and a final hack up recreates dep. Reproduce-only: the legacy Compose path already passes --no-deps when the target is running and exec never converges, so no source change. Co-Authored-By: Claude Opus 5.5 --- tests/e2e/README.md | 26 ++ tests/e2e/run.ts | 2 + .../scenarios/compose-config-hash-drift.ts | 402 ++++++++++++++++++ tests/e2e/shard.ts | 2 + 4 files changed, 432 insertions(+) create mode 100644 tests/e2e/scenarios/compose-config-hash-drift.ts diff --git a/tests/e2e/README.md b/tests/e2e/README.md index 6e20ee84..d3773098 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -205,6 +205,32 @@ data survive that cleanup. The canary is then removed separately. PostgreSQL use fixture-only trust on this unexposed network. This qualifies the actual execution host, not Codex Cloud capabilities, browser access or native VM support elsewhere. +## Compose config-hash drift (F03) + +`compose-config-hash-drift` (HACK-1124) checks that `hack run` and `hack exec` +leave running containers alone after a dependency's Compose config changes. +It uses cached `alpine:3.20` with `pull_policy: never`, a private internal +network, no published ports, and Hack internal DNS/TLS turned off. `app` +depends on `dep`. After `hack up --detach`, the scenario changes only `dep`'s +environment, so only `dep`'s config hash drifts. It then runs `hack run app`, +`hack run dep`, `hack exec app` and `hack exec dep`. After each command, both +services must keep the same full container ID, `State.StartedAt`, +`RestartCount` and config-hash label, and no one-off container may remain. + +`hack run` has no user `--no-deps` flag. The legacy Compose path adds +`--no-deps` when the target service is already running, and `hack exec` maps +to `docker compose exec`, which never converges services. Two positive +controls keep the check from passing vacuously. The `hack run dep` one-off +must print the drifted value while the running `dep` still prints the old +one. A final `hack up --detach` must recreate `dep` with a new ID and config +hash. Whether that converging `up` also restarts `app` is logged, not +asserted. The scenario covers the legacy Compose backend only, not the +authored native-config path or `HACK_RUNTIME_BACKEND=native`. + +```sh +HACK_E2E_CLI_BIN=./dist/hack HACK_E2E_DOCKER=1 HACK_E2E_REQUIRE_DOCKER=1 bun tests/e2e/run.ts --only=compose-config-hash-drift +``` + For a managed cloud environment, repeat acceptance in a fresh task restored from the published environment. Setup-session access does not prove task access: local listeners and the Docker socket may be restricted even when the tools restore. diff --git a/tests/e2e/run.ts b/tests/e2e/run.ts index e7151477..af6150f2 100644 --- a/tests/e2e/run.ts +++ b/tests/e2e/run.ts @@ -9,6 +9,7 @@ import { agentDocsSyncScenario } from "./scenarios/agent-docs-sync.ts"; import { agentSurfaceFreezeScenario } from "./scenarios/agent-surface-freeze.ts"; import { automationCheckScenario } from "./scenarios/automation-check.ts"; import { cachePruneScenario } from "./scenarios/cache-prune.ts"; +import { composeConfigHashDriftScenario } from "./scenarios/compose-config-hash-drift.ts"; import { dependencyCacheScenario } from "./scenarios/dependency-cache.ts"; import { dependencyCacheBootstrapScenario } from "./scenarios/dependency-cache-bootstrap.ts"; import { dependencyCachePackageScenario } from "./scenarios/dependency-cache-package.ts"; @@ -94,6 +95,7 @@ export const ALL_SCENARIOS: readonly Scenario[] = [ domainMigrationScenario, upDownScenario, portableMultiserviceScenario, + composeConfigHashDriftScenario, nativeConfigComposeScenario, nativeConfigBuildScenario, nativeConfigDownHooksScenario, diff --git a/tests/e2e/scenarios/compose-config-hash-drift.ts b/tests/e2e/scenarios/compose-config-hash-drift.ts new file mode 100644 index 00000000..e1ea2f4d --- /dev/null +++ b/tests/e2e/scenarios/compose-config-hash-drift.ts @@ -0,0 +1,402 @@ +import { join } from "node:path"; +import { createMonorepoFixture } from "../fixture.ts"; +import { + type CliResult, + expect, + expectExit, + runCommand, + type Scenario, +} from "../harness.ts"; + +const OWNER = "hack.e2e.compose-config-hash-drift-owner"; +const PROJECT = "com.docker.compose.project"; +const SERVICE = "com.docker.compose.service"; +const ONEOFF = "com.docker.compose.oneoff"; +const CONFIG_HASH = "com.docker.compose.config-hash"; +const TIMEOUT = 180_000; +const IMAGE = "alpine:3.20"; +const IMAGE_ID = /^sha256:[a-f0-9]{64}$/; +const CONTAINER_ID = /^[a-f0-9]{64}$/; +const SERVICES = ["app", "dep"] as const; +const BEFORE = "before"; +const AFTER = "after"; +// Exits promptly on SIGTERM so `hack down` does not wait for the grace period. +const IDLE = ["sh", "-c", "trap 'exit 0' TERM; while :; do sleep 1; done"]; + +type ServiceName = (typeof SERVICES)[number]; + +/** Identity of one long-running (non-one-off) Compose service container. */ +type ContainerIdentity = { + readonly id: string; + readonly startedAt: string; + readonly restartCount: string; + readonly running: string; + readonly configHash: string; +}; + +type Snapshot = Readonly>; + +function object(value: unknown): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error("Expected an object from Docker inspect"); + } + return value as Record; +} + +function composeDocument(opts: { + readonly name: string; + readonly image: string; + readonly marker: string; +}): string { + const labels = { [OWNER]: opts.name }; + const service = { + image: opts.image, + pull_policy: "never", + labels, + command: IDLE, + }; + return JSON.stringify({ + name: opts.name, + services: { + app: { + ...service, + environment: { DRIFT_ROLE: "target" }, + depends_on: { dep: { condition: "service_started" } }, + }, + // Only this service's config changes between the two documents. + dep: { ...service, environment: { DRIFT_MARKER: opts.marker } }, + }, + networks: { default: { internal: true, labels } }, + }); +} + +function sameIdentity(a: ContainerIdentity, b: ContainerIdentity): boolean { + return ( + a.id === b.id && + a.startedAt === b.startedAt && + a.restartCount === b.restartCount && + a.configHash === b.configHash && + b.running === "true" + ); +} + +/** + * F03 / HACK-1124 reproduction: Compose config-hash drift on a dependency must + * not let `hack run` or `hack exec` recreate or restart any running container. + * + * `hack run` has no user `--no-deps` flag; the legacy Compose path adds + * `--no-deps` itself when the target service is already running + * (`resolveCanSkipRunDependencies`), and `hack exec` maps to + * `docker compose exec`, which never converges services. Without `--no-deps`, + * `docker compose run` would start dependencies and recreate one whose config + * hash diverged. The scenario therefore: + * + * 1. starts `app` (depends on `dep`) and records each container's full ID, + * `State.StartedAt`, `RestartCount` and config-hash label; + * 2. changes only `dep`'s environment, so only `dep`'s config hash drifts; + * 3. runs `hack run app`, `hack run dep`, `hack exec app` and `hack exec dep` + * and requires identical identities after each, no leftover one-off + * containers, and that the running `dep` still has the pre-drift value; + * 4. positive controls: the `hack run dep` one-off must see the drifted value + * (so Compose received the drifted config through Hack's real file set), + * and a final `hack up --detach` must recreate `dep` (so the identity + * comparison detects recreation instead of passing vacuously). + */ +export const composeConfigHashDriftScenario: Scenario = { + name: "compose-config-hash-drift", + tier: "docker", + summary: + "dependency config-hash drift: hack run/exec keep running containers (F03)", + run: async (ctx) => { + const docker = async (args: readonly string[]) => { + const result = await runCommand({ + argv: ["docker", ...args], + cwd: ctx.tempRoot, + timeoutMs: TIMEOUT, + }); + expectExit({ + result, + codes: [0], + message: `Docker ${args[0]} must succeed`, + }); + return result.stdout.trim(); + }; + expect({ + that: (await docker(["info", "--format", "{{.OSType}}"])) === "linux", + message: "A Linux Docker daemon is required", + }); + await docker(["compose", "version"]); + const image = await docker([ + "image", + "inspect", + IMAGE, + "--format", + "{{.Id}}", + ]); + expect({ + that: IMAGE_ID.test(image), + message: `${IMAGE} must already be cached with an immutable ID`, + }); + + const fixture = await createMonorepoFixture({ + parentDir: ctx.tempRoot, + withHackConfig: true, + lifecycle: { disableInternal: true }, + }); + const name = fixture.name; + const composePath = join(fixture.hackDir, "docker-compose.yml"); + await Bun.write( + join(fixture.hackDir, "hack.config.json"), + JSON.stringify({ + name, + internal: { dns: false, tls: false }, + logs: { + follow_backend: "compose", + snapshot_backend: "compose", + clear_on_down: false, + }, + }) + ); + await Bun.write( + composePath, + composeDocument({ name, image, marker: BEFORE }) + ); + + const cli = async (args: readonly string[]): Promise => { + const result = await ctx.cli({ + args, + cwd: fixture.root, + timeoutMs: TIMEOUT, + env: { + HACK_EXECUTION_MODE: "slim", + HACK_DAEMON_DISABLE_DOCKER_EVENTS: "1", + }, + }); + expectExit({ + result, + codes: [0], + message: `hack ${args[0]} must succeed`, + }); + return result; + }; + const list = async ( + kind: "container" | "network" | "volume", + filters: readonly string[] + ) => { + const args = kind === "container" ? ["ps", "-aq"] : [kind, "ls", "-q"]; + return ( + await docker([ + ...args, + "--filter", + `label=${OWNER}=${name}`, + ...filters.flatMap((filter) => ["--filter", `label=${filter}`]), + ]) + ) + .split(/\s+/) + .filter(Boolean); + }; + const owned = async ( + kind: "container" | "network" | "volume", + id: string + ) => { + const expression = + kind === "container" ? "{{json .Config.Labels}}" : "{{json .Labels}}"; + const actual = object( + JSON.parse(await docker([kind, "inspect", id, "--format", expression])) + ); + expect({ + that: actual[OWNER] === name && actual[PROJECT] === name, + message: `Refuse ${kind} mutation outside exact fixture ownership`, + }); + }; + const clean = async () => { + for (const kind of ["container", "network", "volume"] as const) { + for (const id of await list(kind, [])) { + await owned(kind, id); + await docker([ + kind, + "rm", + ...(kind === "container" ? ["--force"] : []), + id, + ]); + } + expect({ + that: (await list(kind, [])).length === 0, + message: `Owned ${kind} cleanup must leave no resources`, + }); + } + }; + const identity = async (service: ServiceName) => { + const ids = await list("container", [ + `${SERVICE}=${service}`, + `${ONEOFF}=False`, + ]); + const id = ids[0]; + if (ids.length !== 1 || !id) { + throw new Error(`Expected exactly one owned ${service} container`); + } + await owned("container", id); + const fields = object( + JSON.parse( + await docker([ + "inspect", + id, + "--format", + `{"id":{{json .Id}},"startedAt":{{json .State.StartedAt}},"restartCount":"{{.RestartCount}}","running":"{{.State.Running}}","configHash":{{json (index .Config.Labels "${CONFIG_HASH}")}}}`, + ]) + ) + ); + const value = { + id: String(fields.id), + startedAt: String(fields.startedAt), + restartCount: String(fields.restartCount), + running: String(fields.running), + configHash: String(fields.configHash), + }; + expect({ + that: + CONTAINER_ID.test(value.id) && + value.startedAt.length > 0 && + /^[a-f0-9]{64}$/.test(value.configHash), + message: `${service} must expose a full ID, start time and config hash`, + }); + return value; + }; + const snapshot = async (): Promise => ({ + app: await identity("app"), + dep: await identity("dep"), + }); + const record = (label: string, value: Snapshot) => { + for (const service of SERVICES) { + const entry = value[service]; + ctx.log( + `${label} ${service}: id=${entry.id.slice(0, 12)} startedAt=${entry.startedAt} restarts=${entry.restartCount} running=${entry.running} hash=${entry.configHash.slice(0, 12)}` + ); + } + }; + const requireUnchanged = async (opts: { + readonly step: string; + readonly baseline: Snapshot; + }) => { + const current = await snapshot(); + record(opts.step, current); + for (const service of SERVICES) { + expect({ + that: sameIdentity(opts.baseline[service], current[service]), + message: `${opts.step} must not recreate or restart ${service}`, + }); + } + expect({ + that: (await list("container", [`${ONEOFF}=True`])).length === 0, + message: `${opts.step} must not leave a one-off container behind`, + }); + }; + const printed = (result: CliResult, value: string, step: string) => { + expect({ + that: result.stdout.trim() === value, + message: `${step} must print the expected marker`, + result, + }); + }; + + try { + await cli(["up", "--detach"]); + const baseline = await snapshot(); + record("up", baseline); + expect({ + that: SERVICES.every((service) => baseline[service].running === "true"), + message: "Both services must run after hack up --detach", + }); + + // Drift only the dependency's config hash; the target stays identical. + await Bun.write( + composePath, + composeDocument({ name, image, marker: AFTER }) + ); + + const runApp = await cli([ + "run", + "app", + "--", + "sh", + "-c", + "printf run-ok", + ]); + printed(runApp, "run-ok", "hack run app"); + await requireUnchanged({ step: "hack run app", baseline }); + + // Positive control: the one-off sees the drifted dependency config, so + // Compose received the changed document through Hack's file set. + const runDep = await cli([ + "run", + "dep", + "--", + "sh", + "-c", + 'printf %s "$DRIFT_MARKER"', + ]); + printed(runDep, AFTER, "hack run dep"); + await requireUnchanged({ step: "hack run dep", baseline }); + + const execApp = await cli([ + "exec", + "app", + "--", + "sh", + "-c", + "printf exec-ok", + ]); + printed(execApp, "exec-ok", "hack exec app"); + await requireUnchanged({ step: "hack exec app", baseline }); + + // The running dependency must still be the pre-drift container. + const execDep = await cli([ + "exec", + "dep", + "--", + "sh", + "-c", + 'printf %s "$DRIFT_MARKER"', + ]); + printed(execDep, BEFORE, "hack exec dep"); + await requireUnchanged({ step: "hack exec dep", baseline }); + + // Positive control: converging the stack does recreate the drifted + // dependency, so the identity comparison above can observe recreation. + await cli(["up", "--detach"]); + const converged = await snapshot(); + record("converged up", converged); + expect({ + that: + converged.dep.id !== baseline.dep.id && + converged.dep.configHash !== baseline.dep.configHash && + converged.dep.running === "true", + message: + "hack up --detach must recreate the drifted dependency (positive control)", + }); + ctx.log( + `converged up target: ${sameIdentity(baseline.app, converged.app) ? "retained" : "recreated or restarted"} (observed only)` + ); + const convergedDep = await cli([ + "exec", + "dep", + "--", + "sh", + "-c", + 'printf %s "$DRIFT_MARKER"', + ]); + printed(convergedDep, AFTER, "hack exec dep after converge"); + + await cli(["down"]); + expect({ + that: (await list("container", [])).length === 0, + message: "hack down must remove all fixture containers", + }); + ctx.log( + "dependency config-hash drift: hack run/exec retained every container ID, start time and restart count" + ); + } finally { + await clean(); + } + }, +}; diff --git a/tests/e2e/shard.ts b/tests/e2e/shard.ts index 5e4e9855..af28bcad 100644 --- a/tests/e2e/shard.ts +++ b/tests/e2e/shard.ts @@ -52,6 +52,8 @@ export const E2E_DURATION_HINTS_SECONDS: Readonly> = { "native-compose-adoption-plural-network-worktrees": 33, "native-compose-adoption-branch-worktrees": 33, "portable-multiservice": 30, + // Estimate (no measured run yet): two cached-alpine ups plus four run/exec probes. + "compose-config-hash-drift": 25, "dependency-cache-package": 30, "native-compose-adoption-worktrees": 28, "native-compose-adoption-string-worktrees": 28, From de552362c486d755edc62dcc36e9afb3bdf5c3f5 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 08:51:00 -0400 Subject: [PATCH 45/52] feat(env): accept secret values on stdin with --value-stdin Add `--value-stdin` to `hack env add` and `hack env set` (HACK-1158, HACK-1127) so a value can be piped from a secret manager or file without appearing in argv, shell history, or process listings. - Reads all of stdin up to 64 KiB, cancels the stream once the bound is exceeded, and strips exactly one trailing newline (\n or \r\n). - Refuses, with fixed value-free messages and before any project file is migrated or written: an interactive TTY stdin, a positional value or KEY=value alongside the flag, a missing key, empty input, oversize input, invalid UTF-8, and NUL bytes. - The non-TTY E_INTERACTIVE_REQUIRED hints now point at --value-stdin or the hidden prompt instead of suggesting the secret as an argument. - Canonical agent guidance and the onboarding prompt describe piping secrets via --value-stdin; plugins, checked-in examples, the content revision, and render fingerprints are regenerated. docs/env.md documents the contract and docs/reference/cli.md is regenerated. Tests run the CLI in fresh subprocesses with a temp HOME, a preload that records argv/env of every child process, and either the project-key YAML store or an injected encrypted_file backend (no keychain or 1Password). Co-Authored-By: Claude Opus 5.5 --- .codex/skills/hack-cli/SKILL.md | 3 +- .cursor/rules/hack.mdc | 3 +- docs/env.md | 30 +- docs/reference/cli.md | 6 + examples/basic/AGENTS.md | 3 +- examples/basic/CLAUDE.md | 3 +- plugins/hack/rules/hack.mdc | 3 +- plugins/hack/skills/hack-cli/SKILL.md | 3 +- src/agents/instruction-source.ts | 1 + src/agents/integration-ownership.ts | 2 + src/agents/integration-revision.ts | 2 +- src/agents/onboarding-prompt.ts | 2 +- src/commands/env.ts | 103 +++++-- src/lib/env-value-stdin.ts | 202 +++++++++++++ tests/env-value-stdin-cli.test.ts | 406 ++++++++++++++++++++++++++ tests/env-value-stdin.test.ts | 261 +++++++++++++++++ tests/fixtures/spawn-argv-recorder.ts | 85 ++++++ 17 files changed, 1084 insertions(+), 34 deletions(-) create mode 100644 src/lib/env-value-stdin.ts create mode 100644 tests/env-value-stdin-cli.test.ts create mode 100644 tests/env-value-stdin.test.ts create mode 100644 tests/fixtures/spawn-argv-recorder.ts diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index 28776a4d..232d368e 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). ## Product boundary @@ -151,6 +151,7 @@ Use `hack` as the primary interface for local-first development. - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 2dcc4e8f..c682bd51 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -9,7 +9,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). ## Product boundary @@ -70,6 +70,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/docs/env.md b/docs/env.md index a9e1eed5..75822d81 100644 --- a/docs/env.md +++ b/docs/env.md @@ -325,13 +325,37 @@ Add or update values: ```bash hack env add API_BASE_URL https://api.example.com -hack env add SERVICE_TOKEN abc123 --service api --secret hack env add --env qa API_BASE_URL https://qa.example.com hack env add API_BASE_URL https://api.example.com --local ``` `hack env set` still works as a compatibility alias for `hack env add`, but `add` is the primary UX now. +### Entering secret values + +Never pass a secret value as a command argument: arguments land in shell history and are +visible to other local users and tools through process listings. Use one of these instead: + +```bash +# Pipe the value from a secret manager or file; it never appears in argv. + | hack env add SERVICE_TOKEN --service api --secret --value-stdin +hack env add SERVICE_TOKEN --service api --secret --value-stdin < token.txt + +# In an interactive terminal, omit the value to use the hidden prompt. +hack env add SERVICE_TOKEN --service api --secret +``` + +`--value-stdin` works on `hack env add` and `hack env set`, with or without `--secret`. +It reads all of stdin, up to 64 KiB, and strips exactly one trailing newline (`\n` or +`\r\n`), so `printf '%s\n' "$value"` and `printf '%s' "$value"` store the same value; any +further trailing newlines are kept. It refuses, with a fixed message that never repeats the +input and without writing anything: + +- an interactive terminal on stdin (pipe the value, or drop `--value-stdin` for the hidden prompt) +- a value argument alongside the flag, including `KEY=value` +- a missing key argument +- empty input, input over 64 KiB, invalid UTF-8, or a NUL byte + `--local` writes to the worktree-local override file (`.hack/hack.env.local.yaml` or `.hack/hack.env..local.yaml`) instead of the shared repo env file — use it for per-checkout values you don't want to commit. @@ -339,7 +363,8 @@ per-checkout values you don't want to commit. If you omit `key`, `value`, or `--service`, `hack env add` prompts for them interactively. Pass `--no-interactive` (or set `HACK_NO_INTERACTIVE=1`) for scripted/agent usage: the command applies documented defaults or fails fast with `E_INTERACTIVE_REQUIRED` instead of -blocking on a prompt. `hack env unset` and `hack env set` accept `--local` and +blocking on a prompt. In scripts, supply secret values with `--value-stdin` rather than as +arguments. `hack env unset` and `hack env set` accept `--local` and `--no-interactive` too. Remove a value: @@ -542,6 +567,7 @@ If you are writing new docs or new project setup flows, document the YAML overla - Commit `.hack/hack.env..yaml` when the overlay is team-shared - Never commit `.hack.secret.key` - Prefer `hack env add` over hand-editing encrypted values +- Enter secrets with `--value-stdin` or the hidden prompt, never as command arguments - Prefer direct runtime injection over materializing `.hack/.env` - Use `hack host exec` or env-aware sessions for host-side scripts like migrations, generators, and admin tasks diff --git a/docs/reference/cli.md b/docs/reference/cli.md index dddd00ec..5fe0d631 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -2334,6 +2334,8 @@ Add or update an env value hack env add [key] [value] [options] ``` +Non-secret values may be passed as arguments. For secrets, pipe the value with --value-stdin (for example ` \| hack env add API_TOKEN --secret --value-stdin`) or omit the value in a terminal to use the hidden prompt, so the value never appears in argv, shell history, or process listings. + ### Arguments | Arg | Description | @@ -2349,6 +2351,7 @@ hack env add [key] [value] [options] | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | | `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--secret` | Store value as an encrypted secure entry | +| `--value-stdin` | Read the value from piped stdin (max 64 KiB, one trailing newline stripped) so it never appears in argv or shell history | | `--local` | Write to the worktree-local env override file instead of the shared repo env file | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | @@ -2365,6 +2368,8 @@ Alias for env add hack env set [key] [value] [options] ``` +Same value rules as `hack env add`: pipe secrets with --value-stdin or use the hidden prompt instead of passing them as arguments. + ### Arguments | Arg | Description | @@ -2380,6 +2385,7 @@ hack env set [key] [value] [options] | `--project ` | Target a registered project by name (from ~/.hack/projects.json) | | `--env ` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only | | `--secret` | Store value as an encrypted secure entry | +| `--value-stdin` | Read the value from piped stdin (max 64 KiB, one trailing newline stripped) so it never appears in argv or shell history | | `--local` | Write to the worktree-local env override file instead of the shared repo env file | | `--service ` | Target scope (global or a discovered service name) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index c5fb011c..c430447b 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -128,6 +128,7 @@ Env composition: - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index c5fb011c..c430447b 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -128,6 +128,7 @@ Env composition: - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index e184f0ef..7e8faaf7 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). ## Product boundary @@ -75,6 +75,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index bbf2ff1b..315545a3 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `c3c646a036aa` (version alone is not a freshness guarantee). ## Product boundary @@ -151,6 +151,7 @@ Use `hack` as the primary interface for local-first development. - Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last. - Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation. - Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret. +- Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them. - Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`. - Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository). diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 25a1cb56..13f09d3e 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -240,6 +240,7 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ "Layer order: `.hack/hack.env.default.yaml`, the selected `.hack/hack.env..yaml`, then worktree-local `.hack/hack.env.local.yaml` and `.hack/hack.env..local.yaml`; service values override `global`, and host commands apply `host` values last.", "Omit `--env` to use the configured default overlay; `--env base` (or `default`/`none`) selects base layers only, the same on every command. An overlay name grants no production authority and no network or instance isolation.", "Use `hack env explain --env --service ` for provenance without the value; use `hack env get ` only when a plaintext value is explicitly needed, and treat its output as secret.", + "Store a secret by piping it: ` | hack env add --service --secret --value-stdin` (or the hidden prompt in a terminal); never pass secret values as command arguments, where shell history and process listings expose them.", "Running containers keep the env they were created with; after an env change, refresh with `hack env apply --service ` or `hack restart`.", "Accepted flags, injection timing and refresh path per command: the command and delivery matrix in Hack's env guide (`docs/env.md` in the Hack repository).", ], diff --git a/src/agents/integration-ownership.ts b/src/agents/integration-ownership.ts index 4156a625..cc5da86a 100644 --- a/src/agents/integration-ownership.ts +++ b/src/agents/integration-ownership.ts @@ -106,6 +106,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "121d8f26374d24f2a09890a3", // v5.0.0-next.3..next "6449442a8f230412d4e5f9e7", // v5.0.0-next.4..next "d83da046cf7541ba30950880", // v5.0.0-next.4..next (offline env-matrix pointer) + "c75c5f5eb9fdcf6842ed1eda", // v5.0.0-next.4..next (--value-stdin secret entry) ]), "hack-init-skill": new Set([ "cd15fb6d8ed8e25071d05e75", // v3.1.0..v4.0.3 @@ -133,6 +134,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "06cf6d0920afc1780b9946ba", // v5.0.0-next.3..next "cf0a31e197f9034485413674", // v5.0.0-next.4..next "3a99c9097855f93808e482cb", // v5.0.0-next.4..next (offline env-matrix pointer) + "29499060cad876931810b458", // v5.0.0-next.4..next (--value-stdin secret entry) ]), }; diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index d418edef..2b49586e 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "51163ae32e56"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "c3c646a036aa"; diff --git a/src/agents/onboarding-prompt.ts b/src/agents/onboarding-prompt.ts index 7636acbf..93217cef 100644 --- a/src/agents/onboarding-prompt.ts +++ b/src/agents/onboarding-prompt.ts @@ -143,7 +143,7 @@ function renderSetupPhase(opts: { "- Check for reuse before authoring new services: if the repo already containerizes for prod (existing Dockerfiles/compose with build contexts, runtime-config injection points like an nginx-served runtime-config.js), prefer reusing those images/contexts with minimal glue over hand-authoring new dev-mode services from scratch.", "- Define one compose service per process chosen for containerization in `.hack/docker-compose.yml`. HTTP services need the Caddy labels (`caddy`, `caddy.reverse_proxy`, `caddy.tls=internal`) and the `hack-dev` network to be routable.", `- Design hostnames: the primary app answers on dev_host (${devHostExample}); every other routable service gets a subdomain (\`api.${devHostExample}\`, \`grafana.${devHostExample}\`, ...). Set dev_host in \`.hack/hack.config.json\`.`, - "- Migrate only the env entries required for the authorized setup. Use `hack env add ` for non-secret values. For secrets, use an authorized secret-manager/injection path or let the user enter them through `hack env add --secret`; never put secret values in an agent prompt, command argument, log, or source control.", + "- Migrate only the env entries required for the authorized setup. Use `hack env add ` for non-secret values. For secrets, pipe the value from an authorized secret manager with ` | hack env add --secret --value-stdin`, or let the user enter it at the hidden prompt of `hack env add --secret`; never put secret values in an agent prompt, command argument, log, or source control.", "- Verify container env injection and host commands through `hack host exec` before treating any old env file as superseded. Preserve files still needed by other workflows; deletion or broader migration requires authorization.", "- Encode required host-side setup (tunnels, proxies, credential bootstrap) in `.hack/hack.config.json` under `startup`/`lifecycle` — not in ad-hoc terminal instructions.", ]; diff --git a/src/commands/env.ts b/src/commands/env.ts index 3e9e3a23..a988205b 100644 --- a/src/commands/env.ts +++ b/src/commands/env.ts @@ -14,6 +14,10 @@ import { optEnv, optJson, optPath, optProject } from "../cli/options.ts"; import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { updateGlobalConfig } from "../lib/config.ts"; import { resolveGlobalConfigPath } from "../lib/config-paths.ts"; +import { + assertEnvValueStdinUsage, + readEnvValueFromStdin, +} from "../lib/env-value-stdin.ts"; import { readTextFile } from "../lib/fs.ts"; import { isRecord } from "../lib/guards.ts"; import { resolveHackInvocation } from "../lib/hack-cli.ts"; @@ -95,6 +99,14 @@ const optSecret = defineOption({ description: "Store value as an encrypted secure entry", } as const); +const optValueStdin = defineOption({ + name: "valueStdin", + type: "boolean", + long: "--value-stdin", + description: + "Read the value from piped stdin (max 64 KiB, one trailing newline stripped) so it never appears in argv or shell history", +} as const); + const optLocal = defineOption({ name: "local", type: "boolean", @@ -230,8 +242,18 @@ const applySpec = defineCommand({ const addSpec = defineCommand({ name: "add", summary: "Add or update an env value", + description: + "Non-secret values may be passed as arguments. For secrets, pipe the value with --value-stdin (for example ` | hack env add API_TOKEN --secret --value-stdin`) or omit the value in a terminal to use the hidden prompt, so the value never appears in argv, shell history, or process listings.", group: "Project", - options: [optPath, optProject, optEnv, optSecret, optLocal, optService], + options: [ + optPath, + optProject, + optEnv, + optSecret, + optValueStdin, + optLocal, + optService, + ], positionals: [ { name: "key", required: false }, { name: "value", required: false }, @@ -242,8 +264,18 @@ const addSpec = defineCommand({ const setSpec = defineCommand({ name: "set", summary: "Alias for env add", + description: + "Same value rules as `hack env add`: pipe secrets with --value-stdin or use the hidden prompt instead of passing them as arguments.", group: "Project", - options: [optPath, optProject, optEnv, optSecret, optLocal, optService], + options: [ + optPath, + optProject, + optEnv, + optSecret, + optValueStdin, + optLocal, + optService, + ], positionals: [ { name: "key", required: false }, { name: "value", required: false }, @@ -901,7 +933,7 @@ async function promptForEnvAddInput(input: { if (!canPrompt()) { requireInteractive({ what: "hack env add needs a key and value", - hint: "Pass them as arguments: `hack env add / ` (or `hack env add KEY VALUE --service `, with --secret for secrets).", + hint: "Pass the key as an argument and pipe the value on stdin: ` | hack env add / --value-stdin` (add --secret for secrets). Non-secret values may also be passed as `hack env add / `; never pass secret values as arguments.", }); } @@ -1699,6 +1731,20 @@ const handleEnvAdd: CommandHandlerFor = async ({ ctx, args, }): Promise => { + const keyArg = args.positionals.key?.trim(); + const valueArg = args.positionals.value; + // --value-stdin: refuse bad combinations and read the bounded value before + // any project file is migrated or written, so a refusal stores nothing. + let stdinValue: string | null = null; + if (args.options.valueStdin === true) { + assertEnvValueStdinUsage({ + hasKey: Boolean(keyArg), + hasPositionalValue: + valueArg !== undefined || keyArg?.includes("=") === true, + }); + stdinValue = await readEnvValueFromStdin(); + } + const project = await resolveProjectForEnv({ ctx, pathOpt: args.options.path, @@ -1714,9 +1760,7 @@ const handleEnvAdd: CommandHandlerFor = async ({ reason: "mutation", }); - const keyArg = args.positionals.key?.trim(); - const valueArg = args.positionals.value; - const interactive = !(keyArg || valueArg); + const interactive = stdinValue === null && !(keyArg || valueArg); const prompted = interactive ? await promptForEnvAddInput({ project, @@ -1730,13 +1774,14 @@ const handleEnvAdd: CommandHandlerFor = async ({ keyOrPath: keyArg ?? "", scopeOverride: args.options.service, }); - const value = prompted - ? prompted.value - : await resolveEnvValue({ - key: parsedTarget.key, - value: valueArg ?? null, - secret: args.options.secret === true, - }); + const value = + prompted?.value ?? + stdinValue ?? + (await resolveEnvValue({ + key: parsedTarget.key, + value: valueArg ?? null, + secret: args.options.secret === true, + })); const secret = prompted ? prompted.secret : args.options.secret === true; const result = await setProjectEnvValue({ @@ -1769,6 +1814,19 @@ const handleEnvSet: CommandHandlerFor = async ({ ctx, args, }): Promise => { + const rawKey = (args.positionals.key ?? "").trim(); + const rawValue = args.positionals.value; + const [keyFromSpec, valueFromSpec] = parseKeyValueSpec( + rawValue === undefined ? rawKey : `${rawKey}=${rawValue}` + ); + const valueFromStdin = args.options.valueStdin === true; + if (valueFromStdin) { + assertEnvValueStdinUsage({ + hasKey: keyFromSpec.length > 0, + hasPositionalValue: valueFromSpec !== null, + }); + } + const project = await resolveProjectForEnv({ ctx, pathOpt: args.options.path, @@ -1799,11 +1857,6 @@ const handleEnvSet: CommandHandlerFor = async ({ const runtimeConfig = await readHackEnvRuntimeConfig({ projectDir: project.projectDir, }); - const rawKey = (args.positionals.key ?? "").trim(); - const rawValue = args.positionals.value; - const [keyFromSpec, valueFromSpec] = parseKeyValueSpec( - rawValue === undefined ? rawKey : `${rawKey}=${rawValue}` - ); const key = await resolveEnvKey({ key: keyFromSpec }); const mutationSource = storeSecret ? "keychain" : "plain_env"; const sourceValidation = await validateHackEnvMutationSource({ @@ -1815,11 +1868,13 @@ const handleEnvSet: CommandHandlerFor = async ({ logger.error({ message: sourceValidation.message }); return 1; } - const value = await resolveEnvValue({ - key, - value: valueFromSpec, - secret: storeSecret, - }); + const value = valueFromStdin + ? await readEnvValueFromStdin() + : await resolveEnvValue({ + key, + value: valueFromSpec, + secret: storeSecret, + }); if (storeSecret) { const store = await resolveConfiguredSecretStore({ project, projectName }); @@ -2422,7 +2477,7 @@ async function resolveEnvValue(opts: { if (!canPrompt()) { requireInteractive({ what: `a value for "${opts.key}" is required`, - hint: "Pass the value as an argument (e.g. `hack env set KEY value` or `hack env add KEY value --secret`).", + hint: "Pipe the value on stdin with --value-stdin (e.g. ` | hack env add KEY --secret --value-stdin`), or run in a terminal to use the hidden prompt. Never pass secret values as arguments.", }); } diff --git a/src/lib/env-value-stdin.ts b/src/lib/env-value-stdin.ts new file mode 100644 index 00000000..b43a8939 --- /dev/null +++ b/src/lib/env-value-stdin.ts @@ -0,0 +1,202 @@ +import { CliUsageError } from "../cli/command.ts"; + +/** + * Non-argv value input for `hack env add|set --value-stdin`. + * + * Contract: + * - The value is read only from a non-TTY stdin (a pipe, file, or here-doc), + * so it never appears in argv, shell history, or process listings. A TTY + * stdin is refused before any byte is read; the hidden prompt is the + * interactive path. + * - Input is bounded by {@link ENV_VALUE_STDIN_MAX_BYTES}. Reading stops and + * the stream is canceled as soon as the bound is exceeded, so an endless + * producer cannot exhaust memory. + * - Exactly one trailing newline (`\n`, or `\r\n` treated as one newline) is + * stripped, so `printf '%s\n' "$v"` and `printf '%s' "$v"` store the same + * value. Any further trailing newlines are preserved as part of the value. + * - Empty input (after that strip), invalid UTF-8, and NUL bytes are refused. + * + * Every refusal is a closed, value-free {@link EnvValueStdinError}: messages + * are fixed strings and never include caller input, byte content, or the + * underlying read error. + */ +export const ENV_VALUE_STDIN_MAX_BYTES = 64 * 1024; + +export const ENV_VALUE_STDIN_REFUSAL_REASONS = [ + "interactive_tty", + "oversize", + "empty", + "invalid_utf8", + "nul_byte", + "read_failed", + "positional_value", + "missing_key", +] as const; + +export type EnvValueStdinRefusalReason = + (typeof ENV_VALUE_STDIN_REFUSAL_REASONS)[number]; + +const REFUSAL_MESSAGES: Readonly> = { + interactive_tty: + "--value-stdin needs piped input, but stdin is an interactive terminal. Pipe the value from a secret manager or file, or omit --value-stdin to use the hidden prompt. Nothing was stored.", + oversize: `--value-stdin input exceeds the ${ENV_VALUE_STDIN_MAX_BYTES}-byte limit. Nothing was stored.`, + empty: "--value-stdin received an empty value. Nothing was stored.", + invalid_utf8: "--value-stdin input is not valid UTF-8. Nothing was stored.", + nul_byte: + "--value-stdin input contains a NUL byte, which env values cannot carry. Nothing was stored.", + read_failed: "--value-stdin could not read stdin. Nothing was stored.", + positional_value: + "--value-stdin cannot be combined with a value argument (including KEY=value). Pass only the key and pipe the value. Nothing was stored.", + missing_key: + "--value-stdin requires the key as an argument, for example `hack env add API_TOKEN --secret --value-stdin`. Nothing was stored.", +}; + +/** Closed refusal for `--value-stdin`; the message never carries input. */ +export class EnvValueStdinError extends CliUsageError { + readonly reason: EnvValueStdinRefusalReason; + + constructor(reason: EnvValueStdinRefusalReason) { + super(REFUSAL_MESSAGES[reason]); + this.name = "EnvValueStdinError"; + this.reason = reason; + } +} + +/** Where `--value-stdin` reads from. Injected by tests; defaults to process stdin. */ +export type EnvValueStdinSource = { + readonly isTTY: boolean; + readonly open: () => ReadableStream; +}; + +function processStdinSource(): EnvValueStdinSource { + return { + isTTY: process.stdin.isTTY === true, + open: () => Bun.stdin.stream(), + }; +} + +/** + * Refuse `--value-stdin` argument combinations before reading anything. + * + * @param input.hasKey - Whether a key argument was supplied. + * @param input.hasPositionalValue - Whether a value argument (or a + * `KEY=value` spec) was supplied alongside the flag. + */ +export function assertEnvValueStdinUsage(input: { + readonly hasKey: boolean; + readonly hasPositionalValue: boolean; +}): void { + if (input.hasPositionalValue) { + throw new EnvValueStdinError("positional_value"); + } + if (!input.hasKey) { + throw new EnvValueStdinError("missing_key"); + } +} + +/** + * Read one env value from a non-TTY stdin under the module contract. + * + * @param input.source - Stdin source; defaults to the process stdin. + * @param input.maxBytes - Byte bound; defaults to {@link ENV_VALUE_STDIN_MAX_BYTES}. + * @returns The decoded value with exactly one trailing newline removed. + * @throws EnvValueStdinError for every refusal (closed, value-free). + */ +export async function readEnvValueFromStdin( + input: { + readonly source?: EnvValueStdinSource; + readonly maxBytes?: number; + } = {} +): Promise { + const source = input.source ?? processStdinSource(); + const maxBytes = input.maxBytes ?? ENV_VALUE_STDIN_MAX_BYTES; + if (source.isTTY) { + throw new EnvValueStdinError("interactive_tty"); + } + + const bytes = await readBounded({ source, maxBytes }); + const decoded = decodeUtf8(bytes); + bytes.fill(0); + if (decoded.includes("\0")) { + throw new EnvValueStdinError("nul_byte"); + } + const value = stripOneTrailingNewline(decoded); + if (value.length === 0) { + throw new EnvValueStdinError("empty"); + } + return value; +} + +async function readBounded(input: { + readonly source: EnvValueStdinSource; + readonly maxBytes: number; +}): Promise { + const chunks: Uint8Array[] = []; + let total = 0; + let reader: ReadableStreamDefaultReader; + try { + reader = input.source.open().getReader(); + } catch { + throw new EnvValueStdinError("read_failed"); + } + + const wipeChunks = (): void => { + for (const chunk of chunks) { + chunk.fill(0); + } + }; + + try { + while (true) { + let result: Awaited>; + try { + result = await reader.read(); + } catch { + wipeChunks(); + throw new EnvValueStdinError("read_failed"); + } + if (result.done) { + break; + } + total += result.value.byteLength; + if (total > input.maxBytes) { + result.value.fill(0); + wipeChunks(); + await reader.cancel().catch(() => undefined); + throw new EnvValueStdinError("oversize"); + } + chunks.push(result.value); + } + } finally { + reader.releaseLock(); + } + + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.byteLength; + } + wipeChunks(); + return out; +} + +function decodeUtf8(bytes: Uint8Array): string { + try { + return new TextDecoder("utf-8", { fatal: true }).decode(bytes); + } catch { + bytes.fill(0); + throw new EnvValueStdinError("invalid_utf8"); + } +} + +/** Remove exactly one trailing `\n` or `\r\n`; leave everything else intact. */ +export function stripOneTrailingNewline(value: string): string { + if (value.endsWith("\r\n")) { + return value.slice(0, -2); + } + if (value.endsWith("\n")) { + return value.slice(0, -1); + } + return value; +} diff --git a/tests/env-value-stdin-cli.test.ts b/tests/env-value-stdin-cli.test.ts new file mode 100644 index 00000000..b84f1e6a --- /dev/null +++ b/tests/env-value-stdin-cli.test.ts @@ -0,0 +1,406 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +/** + * End-to-end `--value-stdin` checks in fresh CLI subprocesses. + * + * Isolation: every run gets a temp HOME and global config, a temp project, + * and either the project-key YAML store or an injected `encrypted_file` + * backend with an env-provided key. No keychain, 1Password, Docker, or agent + * client config is reached. A preload records argv/env of every child process + * the CLI spawns so the sentinel can be proven absent from them. + */ + +const REPO_ROOT = resolve(import.meta.dir, ".."); +const ENTRYPOINT = resolve(REPO_ROOT, "index.ts"); +const RECORDER = resolve(import.meta.dir, "fixtures/spawn-argv-recorder.ts"); +const SENTINEL = "stdin-sentinel-c1-5b8e2d"; +const SECRETS_FILE_KEY = "env-value-stdin-test-key"; + +type RunResult = { + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; + readonly spawnLog: string; +}; + +let root: string | null = null; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-env-value-stdin-")); + await mkdir(join(root, "home"), { recursive: true }); +}); + +afterEach(async () => { + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } +}); + +function requireRoot(): string { + if (!root) { + throw new Error("Missing temp root"); + } + return root; +} + +function isolatedEnv(spawnLog: string): Record { + const base = requireRoot(); + return { + PATH: process.env.PATH ?? "/usr/bin:/bin", + HOME: join(base, "home"), + TMPDIR: join(base, "home"), + HACK_GLOBAL_CONFIG_PATH: join(base, "home", "hack-config.json"), + HACK_LOGGER: "console", + HACK_SECRETS_FILE_KEY: SECRETS_FILE_KEY, + HACK_TEST_SPAWN_ARGV_LOG: spawnLog, + }; +} + +async function runHack(input: { + readonly args: readonly string[]; + readonly stdin?: string | Uint8Array | "ignore"; + readonly cwd?: string; +}): Promise { + const base = requireRoot(); + const spawnLog = join(base, `spawn-${crypto.randomUUID()}.jsonl`); + const stdinInput = input.stdin ?? "ignore"; + const proc = Bun.spawn( + [process.execPath, "--preload", RECORDER, ENTRYPOINT, ...input.args], + { + cwd: input.cwd ?? REPO_ROOT, + env: isolatedEnv(spawnLog), + stdin: stdinInput === "ignore" ? "ignore" : "pipe", + stdout: "pipe", + stderr: "pipe", + } + ); + if (stdinInput !== "ignore" && proc.stdin) { + proc.stdin.write(stdinInput); + await proc.stdin.end(); + } + const [stdout, stderr, exitCode] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + const spawnLogText = await readFile(spawnLog, "utf8").catch(() => ""); + return { exitCode, stdout, stderr, spawnLog: spawnLogText }; +} + +function expectNoSentinel(result: RunResult): void { + expect(result.stdout).not.toContain(SENTINEL); + expect(result.stderr).not.toContain(SENTINEL); + expect(result.spawnLog).not.toContain(SENTINEL); +} + +async function createModernProject(): Promise { + const projectRoot = join(requireRoot(), "modern"); + const projectDir = join(projectRoot, ".hack"); + await mkdir(projectDir, { recursive: true }); + await writeFile( + join(projectDir, "docker-compose.yml"), + "services:\n api:\n image: alpine:3.20\n" + ); + await writeFile( + join(projectDir, "hack.config.json"), + `${JSON.stringify({ name: "value-stdin-modern", dev_host: "value-stdin-modern.hack" }, null, 2)}\n` + ); + await writeFile( + join(projectDir, "hack.env.default.yaml"), + 'version: 1\nenvironment: default\nsecretsprovider: project_key\nvalues:\n global:\n GLOBAL_FLAG: "1"\n' + ); + return projectRoot; +} + +async function readProjectEnvFiles(projectRoot: string): Promise { + const files = ["hack.env.default.yaml", "hack.env.local.yaml", ".env"]; + const contents = await Promise.all( + files.map((file) => + readFile(join(projectRoot, ".hack", file), "utf8").catch(() => "") + ) + ); + return contents.join("\n"); +} + +async function envGet(input: { + readonly projectRoot: string; + readonly key: string; + readonly service?: string; +}): Promise { + return await runHack({ + args: [ + "env", + "get", + input.key, + "--path", + input.projectRoot, + ...(input.service ? ["--service", input.service] : []), + ], + }); +} + +test( + "env add --secret --value-stdin stores the piped value without exposing it", + async () => { + const projectRoot = await createModernProject(); + const result = await runHack({ + args: [ + "env", + "add", + "API_TOKEN", + "--service", + "api", + "--secret", + "--value-stdin", + "--path", + projectRoot, + ], + stdin: `${SENTINEL}\n`, + }); + expect(result.exitCode).toBe(0); + expectNoSentinel(result); + // The recorder is live: the write path's git probe was captured. + expect(result.spawnLog).toContain('"git"'); + + // Stored encrypted, never as plaintext in project files. + expect(await readProjectEnvFiles(projectRoot)).not.toContain(SENTINEL); + + // Positive control for the output scan: an explicit disclosure does + // contain the value, with exactly one trailing newline stripped. + const read = await envGet({ + projectRoot, + key: "API_TOKEN", + service: "api", + }); + expect(read.exitCode).toBe(0); + expect(read.stdout).toBe(SENTINEL); + }, + { timeout: 30_000 } +); + +test( + "env set --value-stdin keeps every newline beyond the single stripped one", + async () => { + const projectRoot = await createModernProject(); + const result = await runHack({ + args: ["env", "set", "MULTILINE", "--value-stdin", "--path", projectRoot], + stdin: `${SENTINEL}\nsecond\n\n`, + }); + expect(result.exitCode).toBe(0); + expectNoSentinel(result); + const read = await envGet({ projectRoot, key: "MULTILINE" }); + expect(read.exitCode).toBe(0); + expect(read.stdout).toBe(`${SENTINEL}\nsecond\n`); + }, + { timeout: 30_000 } +); + +test( + "the value never appears in the CLI process listing while it waits on stdin", + async () => { + const projectRoot = await createModernProject(); + const base = requireRoot(); + const spawnLog = join(base, "spawn-ps.jsonl"); + const proc = Bun.spawn( + [ + process.execPath, + "--preload", + RECORDER, + ENTRYPOINT, + "env", + "add", + "PS_TOKEN", + "--secret", + "--value-stdin", + "--path", + projectRoot, + ], + { + cwd: REPO_ROOT, + env: isolatedEnv(spawnLog), + stdin: "pipe", + stdout: "pipe", + stderr: "pipe", + } + ); + const ps = Bun.spawnSync(["ps", "-ww", "-o", "args=", "-p", `${proc.pid}`]); + const listing = ps.stdout.toString(); + proc.stdin.write(`${SENTINEL}\n`); + await proc.stdin.end(); + const [exitCode, stderr] = await Promise.all([ + proc.exited, + new Response(proc.stderr).text(), + ]); + expect(listing).toContain("--value-stdin"); + expect(listing).not.toContain(SENTINEL); + expect(exitCode).toBe(0); + expect(stderr).not.toContain(SENTINEL); + }, + { timeout: 30_000 } +); + +test( + "positional value plus --value-stdin is refused without echoing the value", + async () => { + const projectRoot = await createModernProject(); + const before = await readProjectEnvFiles(projectRoot); + const cases: readonly (readonly string[])[] = [ + ["env", "add", "API_TOKEN", SENTINEL, "--secret", "--value-stdin"], + ["env", "set", "API_TOKEN", SENTINEL, "--value-stdin"], + ["env", "set", `API_TOKEN=${SENTINEL}`, "--value-stdin"], + ["env", "add", `API_TOKEN=${SENTINEL}`, "--value-stdin"], + ]; + for (const args of cases) { + const result = await runHack({ + args: [...args, "--path", projectRoot], + stdin: "piped-should-not-be-read\n", + }); + expect(result.exitCode).toBe(1); + expect(result.stderr).toContain( + "--value-stdin cannot be combined with a value argument" + ); + expectNoSentinel(result); + } + expect(await readProjectEnvFiles(projectRoot)).toBe(before); + }, + { timeout: 60_000 } +); + +test( + "oversize and empty stdin are refused and nothing is stored", + async () => { + const projectRoot = await createModernProject(); + const before = await readProjectEnvFiles(projectRoot); + + const oversize = await runHack({ + args: [ + "env", + "add", + "BIG_TOKEN", + "--secret", + "--value-stdin", + "--path", + projectRoot, + ], + stdin: SENTINEL.repeat(Math.ceil((64 * 1024 + 1) / SENTINEL.length)), + }); + expect(oversize.exitCode).toBe(1); + expect(oversize.stderr).toContain("exceeds the 65536-byte limit"); + expectNoSentinel(oversize); + + for (const stdin of ["", "\n"]) { + const empty = await runHack({ + args: [ + "env", + "add", + "EMPTY_TOKEN", + "--value-stdin", + "--path", + projectRoot, + ], + stdin, + }); + expect(empty.exitCode).toBe(1); + expect(empty.stderr).toContain("--value-stdin received an empty value"); + } + + const missingKey = await runHack({ + args: ["env", "add", "--value-stdin", "--path", projectRoot], + stdin: `${SENTINEL}\n`, + }); + expect(missingKey.exitCode).toBe(1); + expect(missingKey.stderr).toContain("--value-stdin requires the key"); + expectNoSentinel(missingKey); + + expect(await readProjectEnvFiles(projectRoot)).toBe(before); + }, + { timeout: 60_000 } +); + +test( + "non-TTY refusal suggests --value-stdin instead of a value argument", + async () => { + const projectRoot = await createModernProject(); + const result = await runHack({ + args: ["env", "add", "API_TOKEN", "--secret", "--path", projectRoot], + }); + expect(result.exitCode).toBe(1); + expect(result.stderr).toContain("E_INTERACTIVE_REQUIRED"); + expect(result.stderr).toContain("--value-stdin"); + expect(result.stderr).not.toContain("Pass the value as an argument"); + expect(result.stderr).not.toContain("hack env add KEY value --secret"); + }, + { timeout: 30_000 } +); + +test( + "legacy env set --secret --value-stdin writes through an injected encrypted_file store", + async () => { + const base = requireRoot(); + const projectRoot = join(base, "legacy"); + const projectDir = join(projectRoot, ".hack"); + const storePath = join(base, "secrets.enc.json"); + await mkdir(projectDir, { recursive: true }); + await writeFile(join(projectDir, "docker-compose.yml"), "services: {}\n"); + await writeFile( + join(projectDir, "hack.config.json"), + `${JSON.stringify( + { + name: "value-stdin-legacy", + controlPlane: { + secrets: { + backend: "encrypted_file", + encryptedFile: { path: storePath }, + }, + }, + }, + null, + 2 + )}\n` + ); + await writeFile( + join(projectDir, "hack.env.json"), + `${JSON.stringify( + { + version: 1, + vars: [{ key: "API_KEY", required: false, source: "keychain" }], + }, + null, + 2 + )}\n` + ); + + const result = await runHack({ + args: ["env", "set", "API_KEY", "--secret", "--value-stdin"], + stdin: `${SENTINEL}\r\n`, + cwd: projectRoot, + }); + expect(result.exitCode).toBe(0); + expect(result.stderr).toContain("encrypted_file"); + expectNoSentinel(result); + expect(await readFile(storePath, "utf8")).not.toContain(SENTINEL); + expect( + await readFile(join(projectDir, ".env"), "utf8").catch(() => "") + ).not.toContain(SENTINEL); + + const list = await runHack({ + args: ["env", "list", "--show-secrets", "--json"], + cwd: projectRoot, + }); + expect(list.exitCode).toBe(0); + const payload = JSON.parse(list.stdout) as { + readonly vars: ReadonlyArray<{ + readonly key: string; + readonly value: string | null; + }>; + }; + expect(payload.vars.find((entry) => entry.key === "API_KEY")?.value).toBe( + SENTINEL + ); + }, + { timeout: 60_000 } +); diff --git a/tests/env-value-stdin.test.ts b/tests/env-value-stdin.test.ts new file mode 100644 index 00000000..34a7dfd7 --- /dev/null +++ b/tests/env-value-stdin.test.ts @@ -0,0 +1,261 @@ +import { describe, expect, test } from "bun:test"; + +import { CliUsageError } from "../src/cli/command.ts"; +import { + assertEnvValueStdinUsage, + ENV_VALUE_STDIN_MAX_BYTES, + ENV_VALUE_STDIN_REFUSAL_REASONS, + EnvValueStdinError, + type EnvValueStdinSource, + readEnvValueFromStdin, + stripOneTrailingNewline, +} from "../src/lib/env-value-stdin.ts"; + +const SENTINEL = "stdin-sentinel-7f3a9c"; + +function sourceFrom(input: { + readonly chunks: readonly (string | Uint8Array)[]; + readonly isTTY?: boolean; +}): EnvValueStdinSource & { opened: number } { + const encoder = new TextEncoder(); + const state = { + opened: 0, + isTTY: input.isTTY ?? false, + open: () => { + state.opened += 1; + return new ReadableStream({ + start(controller) { + for (const chunk of input.chunks) { + controller.enqueue( + typeof chunk === "string" ? encoder.encode(chunk) : chunk + ); + } + controller.close(); + }, + }); + }, + }; + return state; +} + +/** Never-ending producer: proves the reader stops at the bound. */ +function endlessSource(): EnvValueStdinSource & { + pulled: number; + canceled: boolean; +} { + const chunk = new TextEncoder().encode(SENTINEL.repeat(64)); + const state = { + pulled: 0, + canceled: false, + isTTY: false, + open: () => + new ReadableStream({ + pull(controller) { + state.pulled += 1; + controller.enqueue(chunk.slice()); + }, + cancel() { + state.canceled = true; + }, + }), + }; + return state; +} + +async function refusal(promise: Promise): Promise { + try { + await promise; + } catch (error: unknown) { + if (error instanceof EnvValueStdinError) { + return error; + } + throw error; + } + throw new Error("expected a --value-stdin refusal"); +} + +/** + * Leak detector shared by every refusal assertion: the caller's value (or any + * fragment of it) must never appear in a diagnostic. + */ +function expectValueFree(error: unknown, value: string): void { + const rendered = + error instanceof Error + ? `${error.name}\n${error.message}\n${error.stack ?? ""}\n${JSON.stringify(error)}` + : String(error); + for (const fragment of [value, value.slice(0, 8), value.slice(-8)]) { + if (rendered.includes(fragment)) { + throw new Error("diagnostic leaked caller input"); + } + } +} + +describe("readEnvValueFromStdin", () => { + test("returns piped input unchanged when it has no trailing newline", async () => { + const value = await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [SENTINEL] }), + }); + expect(value).toBe(SENTINEL); + }); + + test("strips exactly one trailing newline", async () => { + expect( + await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [`${SENTINEL}\n`] }), + }) + ).toBe(SENTINEL); + expect( + await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [`${SENTINEL}\n\n`] }), + }) + ).toBe(`${SENTINEL}\n`); + expect( + await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [`${SENTINEL}\r\n`] }), + }) + ).toBe(SENTINEL); + expect( + await readEnvValueFromStdin({ + source: sourceFrom({ chunks: ["line1\nline2\n"] }), + }) + ).toBe("line1\nline2"); + expect( + await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [` ${SENTINEL} \n`] }), + }) + ).toBe(` ${SENTINEL} `); + }); + + test("reassembles multi-byte characters split across chunks", async () => { + const bytes = new TextEncoder().encode("päss☃\n"); + const value = await readEnvValueFromStdin({ + source: sourceFrom({ chunks: [bytes.slice(0, 2), bytes.slice(2)] }), + }); + expect(value).toBe("päss☃"); + }); + + test("accepts input exactly at the byte bound", async () => { + const value = await readEnvValueFromStdin({ + source: sourceFrom({ chunks: ["a".repeat(ENV_VALUE_STDIN_MAX_BYTES)] }), + }); + expect(value.length).toBe(ENV_VALUE_STDIN_MAX_BYTES); + }); + + test("refuses oversize input with a closed error and stops reading", async () => { + const endless = endlessSource(); + const error = await refusal( + readEnvValueFromStdin({ source: endless, maxBytes: 4096 }) + ); + expect(error.reason).toBe("oversize"); + expect(error).toBeInstanceOf(CliUsageError); + expect(endless.canceled).toBe(true); + expect(endless.pulled).toBeLessThan(16); + expectValueFree(error, SENTINEL); + + const atDefault = await refusal( + readEnvValueFromStdin({ + source: sourceFrom({ + chunks: ["a".repeat(ENV_VALUE_STDIN_MAX_BYTES), "b"], + }), + }) + ); + expect(atDefault.reason).toBe("oversize"); + }); + + test("refuses empty and newline-only input", async () => { + for (const chunks of [[], [""], ["\n"], ["\r\n"]]) { + const error = await refusal( + readEnvValueFromStdin({ source: sourceFrom({ chunks }) }) + ); + expect(error.reason).toBe("empty"); + } + }); + + test("refuses an interactive TTY stdin before reading any byte", async () => { + const source = sourceFrom({ chunks: [SENTINEL], isTTY: true }); + const error = await refusal(readEnvValueFromStdin({ source })); + expect(error.reason).toBe("interactive_tty"); + expect(error.message).toContain("hidden prompt"); + expect(source.opened).toBe(0); + }); + + test("refuses invalid UTF-8 and NUL bytes without echoing input", async () => { + const invalid = new Uint8Array([ + ...new TextEncoder().encode(SENTINEL), + 0xff, + 0xfe, + ]); + const utf8 = await refusal( + readEnvValueFromStdin({ source: sourceFrom({ chunks: [invalid] }) }) + ); + expect(utf8.reason).toBe("invalid_utf8"); + expectValueFree(utf8, SENTINEL); + + const nul = await refusal( + readEnvValueFromStdin({ + source: sourceFrom({ chunks: [`${SENTINEL}\0tail`] }), + }) + ); + expect(nul.reason).toBe("nul_byte"); + expectValueFree(nul, SENTINEL); + }); + + test("maps read failures to a closed error that drops the cause", async () => { + const source: EnvValueStdinSource = { + isTTY: false, + open: () => + new ReadableStream({ + pull(controller) { + controller.error(new Error(`backend said ${SENTINEL}`)); + }, + }), + }; + const error = await refusal(readEnvValueFromStdin({ source })); + expect(error.reason).toBe("read_failed"); + expect(error.cause).toBeUndefined(); + expectValueFree(error, SENTINEL); + }); +}); + +describe("assertEnvValueStdinUsage", () => { + test("refuses a positional value and a missing key", () => { + expect(() => + assertEnvValueStdinUsage({ hasKey: true, hasPositionalValue: true }) + ).toThrow(EnvValueStdinError); + expect(() => + assertEnvValueStdinUsage({ hasKey: false, hasPositionalValue: false }) + ).toThrow(EnvValueStdinError); + expect(() => + assertEnvValueStdinUsage({ hasKey: true, hasPositionalValue: false }) + ).not.toThrow(); + }); +}); + +describe("refusal diagnostics", () => { + test("every refusal message is fixed and value-free", () => { + for (const reason of ENV_VALUE_STDIN_REFUSAL_REASONS) { + const error = new EnvValueStdinError(reason); + expect(error.message).toStartWith("--value-stdin"); + expect(error.message).toContain("Nothing was stored."); + expectValueFree(error, SENTINEL); + } + }); + + test("negative control: the leak detector catches a value in an error message", () => { + const leaky = new CliUsageError(`Invalid value: ${SENTINEL}`); + expect(() => expectValueFree(leaky, SENTINEL)).toThrow( + "diagnostic leaked caller input" + ); + const partial = new Error(`value starts with ${SENTINEL.slice(0, 8)}`); + expect(() => expectValueFree(partial, SENTINEL)).toThrow( + "diagnostic leaked caller input" + ); + }); +}); + +test("stripOneTrailingNewline removes at most one newline", () => { + expect(stripOneTrailingNewline("a\n\n")).toBe("a\n"); + expect(stripOneTrailingNewline("a\r\n")).toBe("a"); + expect(stripOneTrailingNewline("a\r")).toBe("a\r"); + expect(stripOneTrailingNewline("a")).toBe("a"); +}); diff --git a/tests/fixtures/spawn-argv-recorder.ts b/tests/fixtures/spawn-argv-recorder.ts new file mode 100644 index 00000000..8f2c166d --- /dev/null +++ b/tests/fixtures/spawn-argv-recorder.ts @@ -0,0 +1,85 @@ +/** + * Test preload: records argv and env of every child process the CLI spawns. + * + * Loaded with `bun --preload` in a fresh subprocess only. Each spawn appends + * one JSON line `{ via, argv, env }` to the file named by + * `HACK_TEST_SPAWN_ARGV_LOG`, then delegates to the original implementation, + * so behavior is unchanged. Tests scan the log for a secret sentinel. + */ +import { appendFileSync } from "node:fs"; +import { createRequire } from "node:module"; + +const logPath = process.env.HACK_TEST_SPAWN_ARGV_LOG; + +function record(input: { + readonly via: string; + readonly argv: unknown; + readonly env: unknown; +}): void { + if (!logPath) { + return; + } + appendFileSync( + logPath, + `${JSON.stringify({ via: input.via, argv: input.argv, env: input.env ?? null })}\n` + ); +} + +function bunSpawnArgv(args: readonly unknown[]): { + readonly argv: unknown; + readonly env: unknown; +} { + const [first, second] = args; + if (Array.isArray(first)) { + const options = (second ?? {}) as { readonly env?: unknown }; + return { argv: first, env: options.env }; + } + const options = (first ?? {}) as { + readonly cmd?: unknown; + readonly env?: unknown; + }; + return { argv: options.cmd, env: options.env }; +} + +const originalSpawn = Bun.spawn; +const originalSpawnSync = Bun.spawnSync; + +Bun.spawn = ((...args: unknown[]) => { + record({ via: "Bun.spawn", ...bunSpawnArgv(args) }); + return Reflect.apply(originalSpawn, Bun, args); +}) as typeof Bun.spawn; + +Bun.spawnSync = ((...args: unknown[]) => { + record({ via: "Bun.spawnSync", ...bunSpawnArgv(args) }); + return Reflect.apply(originalSpawnSync, Bun, args); +}) as typeof Bun.spawnSync; + +const require = createRequire(import.meta.url); +const childProcess = require("node:child_process") as Record; +for (const name of [ + "spawn", + "spawnSync", + "exec", + "execSync", + "execFile", + "execFileSync", + "fork", +]) { + const original = childProcess[name]; + if (typeof original !== "function") { + continue; + } + childProcess[name] = (...args: unknown[]) => { + const [command, maybeArgs, maybeOptions] = args; + const argList = Array.isArray(maybeArgs) ? maybeArgs : []; + const options = (Array.isArray(maybeArgs) ? maybeOptions : maybeArgs) as + | { readonly env?: unknown } + | undefined; + record({ + via: `child_process.${name}`, + argv: [command, ...argList], + env: options?.env, + }); + return Reflect.apply(original, childProcess, args); + }; +} From 507885528a76c51a1eb6e91c4c02eef96bbe894f Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 08:51:06 -0400 Subject: [PATCH 46/52] feat(doctor): add value-free env key strength diagnostic `hack doctor` and `hack doctor --json` now run a read-only "env key" check (HACK-1127). It audits the checkout-local, shared (git common dir) and primary-checkout key files, or HACK_ENV_SECRET_KEY when no file exists, and reports closed codes: env-key-ok, env-key-absent, env-key-weak-permissions, env-key-wrong-owner, env-key-wrong-length, env-key-low-entropy, env-key-symlink, env-key-not-regular, env-key-unreadable, env-key-tracked and env-key-public-path. Findings carry project-relative subjects and the exact command to run; key bytes and absolute paths are never emitted, symlinks are never followed and --fix does not act on the check. Also add scripts/dependabot-inventory.ts, a GET-only `gh api` inventory of open Dependabot alerts (package, severity, manifest, fixed-in) that writes nothing, documented in the docs contributor section. Co-Authored-By: Claude Opus 5.5 --- docs/README.md | 11 + docs/env.md | 29 ++ scripts/dependabot-inventory.ts | 293 ++++++++++++++++ src/commands/doctor.ts | 58 ++++ src/lib/doctor-env-key.ts | 518 +++++++++++++++++++++++++++++ tests/dependabot-inventory.test.ts | 228 +++++++++++++ tests/doctor-env-key.test.ts | 488 +++++++++++++++++++++++++++ 7 files changed, 1625 insertions(+) create mode 100644 scripts/dependabot-inventory.ts create mode 100644 src/lib/doctor-env-key.ts create mode 100644 tests/dependabot-inventory.test.ts create mode 100644 tests/doctor-env-key.test.ts diff --git a/docs/README.md b/docs/README.md index 14086450..d8d2dbb7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -32,3 +32,14 @@ That local archive is not included in a fresh clone. The command reference is generated with `bun run docs:cli-reference`. Agent-facing setup content is generated from `src/agents/instruction-source.ts` and `src/agents/onboarding-prompt.ts`. End-to-end checks live in `tests/e2e/`. + +### Dependabot alert inventory + +To plan dependency updates, list open Dependabot alerts with +`bun scripts/dependabot-inventory.ts` (add `--repo owner/name` for another repository, +or `--json` for machine-readable output). The script prints each alert's package, +ecosystem, severity, manifest path and first patched version ("fixed-in"), sorted by +severity. It sends only `GET` requests through your existing `gh` authentication, +writes no files and changes no alerts or dependencies. Make the dependency updates in +separate, reviewed changes. You need read access to the repository's Dependabot +alerts. diff --git a/docs/env.md b/docs/env.md index a9e1eed5..cc06f92a 100644 --- a/docs/env.md +++ b/docs/env.md @@ -482,6 +482,35 @@ Write-path guarantees in a linked worktree (first secret added from any checkout - if the shared location cannot be resolved (git unavailable) or written, Hack falls back to a checkout-local key and prints a loud divergence warning - `hack doctor` flags divergent `.hack.secret.key` contents across checkouts with the exact paths +Key strength and exposure (`hack doctor` "env key" check): + +`hack doctor` and `hack doctor --json` audit the checkout-local key, the shared key +under the git common dir and, in a linked worktree, the primary checkout's key. When +no key file exists, the check classifies `HACK_ENV_SECRET_KEY` instead. The check is +read-only. It never prints key bytes or absolute paths, and `--fix` does not act on it. +Each finding names a project-relative subject and the exact command to run from the +project root. `--json` adds an `envKey` object to the check with `status` and +`findings` (`code`, `status`, `subject`, `remedy`). + +| Code | Status | Meaning | +| --- | --- | --- | +| `env-key-ok` | ok | key material is a regular, private, 256-bit-class file | +| `env-key-absent` | ok | no key file and no `HACK_ENV_SECRET_KEY` | +| `env-key-weak-permissions` | warn | group or others have access; suggests `chmod 600` | +| `env-key-wrong-owner` | warn | the file is not owned by the current user | +| `env-key-wrong-length` | warn | shorter than the 43-character base64url form of a 256-bit key, or empty | +| `env-key-low-entropy` | warn | long enough but repetitive (estimated under 128 bits) | +| `env-key-symlink` | error | the key path is a symlink; Hack refuses symlinked keys | +| `env-key-not-regular` | error | the key path is a directory or another non-file entry | +| `env-key-unreadable` | error | unreadable, oversized or not UTF-8 | +| `env-key-tracked` | error | a file named `.hack.secret.key` is tracked by git at any depth | +| `env-key-public-path` | error | a key copy sits in `public/`, `static/`, `dist/`, `build/`, `out/`, `www/`, `_site/`, `.next/` or `.output/` | + +Hack derives the AES-256-GCM key by hashing the key text with SHA-256, so key +strength is the entropy of that text. Generated keys are 32 random bytes in base64url. +For a weak, tracked or published key, the remedy is rotation: move the key aside, then +re-add each secret with `hack env add --secret KEY VALUE` so Hack generates a new key. + CI and managed container fallback: - if `.hack.secret.key` is missing, Hack falls back to `HACK_ENV_SECRET_KEY` diff --git a/scripts/dependabot-inventory.ts b/scripts/dependabot-inventory.ts new file mode 100644 index 00000000..3bda062b --- /dev/null +++ b/scripts/dependabot-inventory.ts @@ -0,0 +1,293 @@ +#!/usr/bin/env bun +/** + * Read-only inventory of open Dependabot alerts for a GitHub repository. + * + * Lists each open alert's package, ecosystem, severity, manifest path, + * vulnerable range and first patched version through `gh api` with the + * caller's existing authentication. It never handles a token, sends only GET + * requests, writes no files and changes no alert, branch or dependency: use the + * output to plan dependency bumps in separate, reviewed changes. + * + * bun scripts/dependabot-inventory.ts [--repo owner/name] [--json] + * + * Diagnostics are closed and value-free: a refusal never echoes arguments or + * `gh` output. + */ + +/** A closed, value-free diagnostic the inventory may print. */ +export class InventoryError extends Error {} + +/** Runs `gh` with the given arguments and resolves its stdout; rejects on failure. */ +export type GhRunner = (args: readonly string[]) => Promise; + +export type AlertSeverity = "critical" | "high" | "medium" | "low" | "unknown"; + +export type DependabotAlert = { + readonly number: number; + readonly ecosystem: string; + readonly package: string; + readonly severity: AlertSeverity; + readonly manifest: string; + readonly vulnerableRange: string | null; + /** First patched version, or null when no fix is published. */ + readonly fixedIn: string | null; + readonly advisory: string | null; +}; + +export type InventoryOptions = { + readonly repository: string; + readonly json: boolean; +}; + +export const DEFAULT_REPOSITORY = "hack-dance/hack"; + +const REPOSITORY_PATTERN = + /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})\/[A-Za-z0-9._-]{1,100}$/; +const SEVERITY_ORDER: readonly AlertSeverity[] = [ + "critical", + "high", + "medium", + "low", + "unknown", +]; +/** Printable, bounded text from GitHub; anything else becomes a fixed placeholder. */ +const PRINTABLE = /^[\x20-\x7e]{1,200}$/; + +export const USAGE = + "Usage: bun scripts/dependabot-inventory.ts [--repo owner/name] [--json]\n"; + +export function parseInventoryArguments( + argv: readonly string[] +): InventoryOptions | "help" { + let repository = DEFAULT_REPOSITORY; + let json = false; + for (let index = 0; index < argv.length; index += 1) { + const arg = argv[index]; + if (arg === "--help" || arg === "-h") { + return "help"; + } + if (arg === "--json") { + json = true; + continue; + } + if (arg === "--repo") { + const value = argv[index + 1]; + if (value === undefined || !REPOSITORY_PATTERN.test(value)) { + throw new InventoryError("Invalid --repo value (expected owner/name)"); + } + repository = value; + index += 1; + continue; + } + throw new InventoryError("Unknown argument"); + } + return { repository, json }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function field(value: unknown, ...path: readonly string[]): unknown { + let current = value; + for (const key of path) { + if (!isRecord(current)) { + return undefined; + } + current = current[key]; + } + return current; +} + +function printable(value: unknown): string | null { + return typeof value === "string" && PRINTABLE.test(value) ? value : null; +} + +function severityOf(value: unknown): AlertSeverity { + return value === "critical" || + value === "high" || + value === "medium" || + value === "low" + ? value + : "unknown"; +} + +/** Narrows one alert from the GitHub REST response; null when it is not an open alert. */ +export function normalizeAlert(raw: unknown): DependabotAlert | null { + const number = field(raw, "number"); + if ( + typeof number !== "number" || + !Number.isSafeInteger(number) || + field(raw, "state") !== "open" + ) { + return null; + } + return { + number, + ecosystem: + printable(field(raw, "dependency", "package", "ecosystem")) ?? "unknown", + package: + printable(field(raw, "dependency", "package", "name")) ?? "", + severity: severityOf( + field(raw, "security_vulnerability", "severity") ?? + field(raw, "security_advisory", "severity") + ), + manifest: + printable(field(raw, "dependency", "manifest_path")) ?? "", + vulnerableRange: printable( + field(raw, "security_vulnerability", "vulnerable_version_range") + ), + fixedIn: printable( + field( + raw, + "security_vulnerability", + "first_patched_version", + "identifier" + ) + ), + advisory: printable(field(raw, "security_advisory", "ghsa_id")), + }; +} + +export function sortAlerts( + alerts: readonly DependabotAlert[] +): DependabotAlert[] { + return [...alerts].sort( + (left, right) => + SEVERITY_ORDER.indexOf(left.severity) - + SEVERITY_ORDER.indexOf(right.severity) || + left.package.localeCompare(right.package) || + left.manifest.localeCompare(right.manifest) || + left.number - right.number + ); +} + +/** Lists open alerts with GET-only, paginated `gh api` calls. */ +export async function listOpenAlerts(opts: { + readonly repository: string; + readonly gh: GhRunner; +}): Promise { + if (!REPOSITORY_PATTERN.test(opts.repository)) { + throw new InventoryError("Invalid repository"); + } + let stdout: string; + try { + stdout = await opts.gh([ + "api", + "--method", + "GET", + "--paginate", + "--jq", + ".[]", + "-H", + "Accept: application/vnd.github+json", + `repos/${opts.repository}/dependabot/alerts?state=open&per_page=100`, + ]); + } catch { + throw new InventoryError( + "gh api request failed (check `gh auth status` and Dependabot alert access)" + ); + } + const alerts: DependabotAlert[] = []; + for (const line of stdout.split("\n")) { + if (line.trim().length === 0) { + continue; + } + let raw: unknown; + try { + raw = JSON.parse(line); + } catch { + throw new InventoryError("Unexpected GitHub response"); + } + const alert = normalizeAlert(raw); + if (alert) { + alerts.push(alert); + } + } + return sortAlerts(alerts); +} + +export function renderInventory(opts: { + readonly repository: string; + readonly alerts: readonly DependabotAlert[]; +}): string { + const counts = SEVERITY_ORDER.map( + (severity) => + `${severity}=${opts.alerts.filter((alert) => alert.severity === severity).length}` + ).join(" "); + const lines = [ + `Open Dependabot alerts for ${opts.repository}: ${opts.alerts.length} (${counts})`, + ]; + if (opts.alerts.length === 0) { + return `${lines.join("\n")}\n`; + } + const header = ["#", "severity", "package", "manifest", "fixed-in"]; + const rows = opts.alerts.map((alert) => [ + String(alert.number), + alert.severity, + `${alert.package} (${alert.ecosystem})`, + alert.manifest, + alert.fixedIn ?? "no fix", + ]); + const widths = header.map((title, column) => + Math.max(title.length, ...rows.map((row) => row[column]?.length ?? 0)) + ); + const format = (row: readonly string[]) => + row + .map((cell, column) => cell.padEnd(widths[column] ?? 0)) + .join(" ") + .trimEnd(); + lines.push("", format(header), ...rows.map(format)); + return `${lines.join("\n")}\n`; +} + +/** Default runner: `gh` with the caller's authentication; stderr is discarded. */ +export const ghCliRunner: GhRunner = async (args) => { + const proc = Bun.spawn(["gh", ...args], { + stdin: "ignore", + stdout: "pipe", + stderr: "ignore", + }); + const text = await new Response(proc.stdout).text(); + if ((await proc.exited) !== 0) { + throw new InventoryError("gh failed"); + } + return text; +}; + +export async function runInventory(opts: { + readonly argv: readonly string[]; + readonly gh: GhRunner; + readonly write: (text: string) => void; +}): Promise { + const options = parseInventoryArguments(opts.argv); + if (options === "help") { + opts.write(USAGE); + return 0; + } + const alerts = await listOpenAlerts({ + repository: options.repository, + gh: opts.gh, + }); + opts.write( + options.json + ? `${JSON.stringify({ repository: options.repository, alerts }, null, 2)}\n` + : renderInventory({ repository: options.repository, alerts }) + ); + return 0; +} + +if (import.meta.main) { + try { + process.exitCode = await runInventory({ + argv: process.argv.slice(2), + gh: ghCliRunner, + write: (text) => process.stdout.write(text), + }); + } catch (error) { + process.stderr.write( + `${error instanceof InventoryError ? error.message : "Dependabot inventory failed"}\n` + ); + process.exitCode = 1; + } +} diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index ea3123c2..5d84c1fe 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -66,6 +66,13 @@ import { checkBrowserLocalNetwork, parseBrowserNetworkOptions, } from "../lib/doctor-browser-network.ts"; +import { + describeEnvKeyInspection, + type EnvKeyInspection, + gitListTrackedKeyPaths, + inspectProjectEnvKey, + resolveEnvKeyCandidates, +} from "../lib/doctor-env-key.ts"; import { inspectTrackedGeneratedFiles, untrackGeneratedFiles, @@ -170,6 +177,8 @@ interface CheckResult { readonly message: string; /** Counts and public fingerprints only; never paths or keychain contents. */ readonly supersededRoots?: SupersededRootInventory; + /** Closed, value-free env key findings; project-relative subjects only. */ + readonly envKey?: EnvKeyInspection; } const SUPERSEDED_ROOTS_CHECK = "superseded caddy roots"; @@ -846,6 +855,11 @@ const handleDoctor: CommandHandlerFor = async ({ } ) ); + results.push( + await runCheck(s, ENV_KEY_CHECK, () => checkProjectEnvKey({ startDir }), { + timeoutMs: 5000, + }) + ); results.push( await runCheck( s, @@ -1092,6 +1106,8 @@ export type DoctorJsonCheck = { readonly durationMs: number; /** Present only on the superseded Caddy roots check. */ readonly supersededRoots?: SupersededRootInventory; + /** Present only on the env key check: closed codes, no key bytes or absolute paths. */ + readonly envKey?: EnvKeyInspection; }; export type DoctorJsonData = { @@ -1123,6 +1139,7 @@ export function buildDoctorJsonData(opts: { ...(result.supersededRoots ? { supersededRoots: result.supersededRoots } : {}), + ...(result.envKey ? { envKey: result.envKey } : {}), })), summary: buildDoctorSummaryLines({ results: opts.results }), counts, @@ -2704,6 +2721,47 @@ async function checkWorktreeSecretKeys({ }; } +const ENV_KEY_CHECK = "env key"; + +/** Doctor check result for an env key inspection; message and data stay value-free. */ +export function envKeyCheckResult(inspection: EnvKeyInspection): CheckResult { + return { + name: ENV_KEY_CHECK, + status: inspection.status, + message: describeEnvKeyInspection(inspection), + envKey: inspection, + }; +} + +/** + * Read-only key-strength and exposure audit (HACK-1127). `--fix` never acts on + * it; each finding suggests the exact command instead. + */ +async function checkProjectEnvKey({ + startDir, +}: { + readonly startDir: string; +}): Promise { + const ctx = await findProjectContext(startDir); + if (!ctx) { + return { + name: ENV_KEY_CHECK, + status: "warn", + message: `Skipped (no ${HACK_PROJECT_DIR_PRIMARY}/ found)`, + }; + } + const inspection = await inspectProjectEnvKey({ + projectRoot: ctx.projectRoot, + candidates: await resolveEnvKeyCandidates({ + projectRoot: ctx.projectRoot, + }), + env: process.env, + uid: typeof process.getuid === "function" ? process.getuid() : null, + listTrackedKeyPaths: gitListTrackedKeyPaths, + }); + return envKeyCheckResult(inspection); +} + /** * Warns when hack-generated machine-local files (`.hack/.internal/`, * `.hack/.branch/`, `.hack/.env`, `.hack/.env.state.json`, diff --git a/src/lib/doctor-env-key.ts b/src/lib/doctor-env-key.ts new file mode 100644 index 00000000..7f3544a4 --- /dev/null +++ b/src/lib/doctor-env-key.ts @@ -0,0 +1,518 @@ +import type { Stats } from "node:fs"; +import { lstat, realpath } from "node:fs/promises"; +import { join } from "node:path"; +import { PROJECT_ENV_KEY_FILENAME } from "../constants.ts"; +import { resolveGitPrimaryWorktreeRoot } from "./git-worktree.ts"; +import { isRecord } from "./guards.ts"; +import { readBoundedPrivateFile } from "./private-file.ts"; +import { + resolveProjectEnvKeyPath, + resolveProjectEnvSharedKeyLocation, +} from "./project-env-config.ts"; +import { exec } from "./shell.ts"; + +/** + * Value-free strength and exposure audit of the project env secret key. + * + * Ownership: Doctor owns this read-only audit. It never writes, chmods, + * untracks or rotates anything, and `hack doctor --fix` does not act on it; + * every finding carries the exact command a person can run instead. + * + * Privacy invariant: key bytes are read only to classify their length and + * entropy class and never leave this module. Findings carry closed codes, + * fixed subject labels and project-relative paths only; absolute paths, key + * contents and environment values are never included. + */ + +/** Closed set of env key diagnostic codes. */ +export type EnvKeyDiagnosticCode = + | "env-key-ok" + | "env-key-absent" + | "env-key-symlink" + | "env-key-not-regular" + | "env-key-unreadable" + | "env-key-weak-permissions" + | "env-key-wrong-owner" + | "env-key-tracked" + | "env-key-public-path" + | "env-key-wrong-length" + | "env-key-low-entropy"; + +export type EnvKeyFindingStatus = "ok" | "warn" | "error"; + +export type EnvKeyFinding = { + readonly code: EnvKeyDiagnosticCode; + readonly status: EnvKeyFindingStatus; + /** Fixed label or project-relative path; never absolute, never key bytes. */ + readonly subject: string; + /** Exact suggested command or action, run from the project root; Doctor never runs it. */ + readonly remedy: string | null; +}; + +export type EnvKeyInspection = { + readonly status: EnvKeyFindingStatus; + /** Ordered most severe first; a clean result is one `env-key-ok` or `env-key-absent` finding. */ + readonly findings: readonly EnvKeyFinding[]; +}; + +/** Where a candidate key file lives relative to the project. */ +export type EnvKeyCandidateRole = "local" | "shared" | "primary"; + +export type EnvKeyCandidate = { + readonly role: EnvKeyCandidateRole; + /** Absolute path; used for filesystem reads only and never reported. */ + readonly path: string; +}; + +/** Lists project-relative paths of tracked files named like the key; null when unknown. */ +export type ListTrackedKeyPaths = (opts: { + readonly projectRoot: string; +}) => Promise; + +export type InspectEnvKeyOptions = { + readonly projectRoot: string; + /** Candidate key files in resolution order (local, shared, primary). */ + readonly candidates: readonly EnvKeyCandidate[]; + /** Environment consulted for the `HACK_ENV_SECRET_KEY` fallback. */ + readonly env: Readonly>; + /** Current user id; null skips the ownership and permission checks (non-POSIX). */ + readonly uid: number | null; + readonly listTrackedKeyPaths: ListTrackedKeyPaths; +}; + +/** Generated keys are 32 random bytes rendered as unpadded base64url. */ +const GENERATED_KEY_PATTERN = /^[A-Za-z0-9_-]{43}$/; +/** Shortest text that can carry the 256 bits AES-256-GCM expects in base64url. */ +export const ENV_KEY_MIN_TEXT_LENGTH = 43; +/** Entropy floor for non-generated key text (empirical Shannon estimate). */ +export const ENV_KEY_MIN_ESTIMATED_BITS = 128; +const ENV_KEY_MIN_DISTINCT_CHARACTERS = 8; +const ENV_KEY_MAX_BYTES = 1024 * 1024; +const ENV_KEY_ENV_NAME = "HACK_ENV_SECRET_KEY"; + +/** + * Build-output and static-hosting directories whose contents are commonly + * published. A key file directly inside one is reported as exposed. + */ +export const ENV_KEY_PUBLIC_ARTIFACT_DIRS = [ + "public", + "static", + "dist", + "build", + "out", + "www", + "_site", + ".next", + ".output", +] as const; + +const STATUS_RANK: Record = { + ok: 0, + warn: 1, + error: 2, +}; + +const SUBJECT_LABEL: Record = { + local: PROJECT_ENV_KEY_FILENAME, + shared: `/${PROJECT_ENV_KEY_FILENAME}`, + primary: `/${PROJECT_ENV_KEY_FILENAME}`, +}; + +const PRIMARY_REMEDY = + "Run 'hack doctor' in the primary checkout and follow its env key guidance."; + +/** Shell spelling of a candidate (plus optional suffix), run from the project root. */ +function shellPathFor( + role: Exclude, + suffix = "" +): string { + return role === "local" + ? `${PROJECT_ENV_KEY_FILENAME}${suffix}` + : `"$(git rev-parse --git-common-dir)/${PROJECT_ENV_KEY_FILENAME}${suffix}"`; +} + +function remedyFor( + role: EnvKeyCandidateRole, + build: (path: (suffix?: string) => string) => string +): string { + if (role === "primary") { + return PRIMARY_REMEDY; + } + return build((suffix) => shellPathFor(role, suffix)); +} + +/** + * Classifies key text into the AES-256-GCM key-material classes. The text is + * hashed with SHA-256 into the cipher key, so its strength is the entropy of + * the text itself. Returns null when the text is acceptable. + */ +export function classifyEnvKeyText( + keyText: string +): "env-key-wrong-length" | "env-key-low-entropy" | null { + if (GENERATED_KEY_PATTERN.test(keyText)) { + return null; + } + const characters = Array.from(keyText); + if (characters.length < ENV_KEY_MIN_TEXT_LENGTH) { + return "env-key-wrong-length"; + } + const counts = new Map(); + for (const character of characters) { + counts.set(character, (counts.get(character) ?? 0) + 1); + } + let bitsPerCharacter = 0; + for (const count of counts.values()) { + const probability = count / characters.length; + bitsPerCharacter -= probability * Math.log2(probability); + } + if ( + counts.size < ENV_KEY_MIN_DISTINCT_CHARACTERS || + bitsPerCharacter * characters.length < ENV_KEY_MIN_ESTIMATED_BITS + ) { + return "env-key-low-entropy"; + } + return null; +} + +class EnvKeyUnreadableError extends Error {} + +type EntryStats = Stats; + +/** lstat that maps absence to null and any other failure to EnvKeyUnreadableError. */ +async function lstatOrNull(path: string): Promise { + try { + return await lstat(path); + } catch (error: unknown) { + if ( + isRecord(error) && + (error.code === "ENOENT" || error.code === "ENOTDIR") + ) { + return null; + } + throw new EnvKeyUnreadableError(); + } +} + +function rotateRemedy(path: (suffix?: string) => string): string { + return `mv ${path()} ${path(".old")}, then re-add each secret with 'hack env add --secret KEY VALUE' (Hack generates a new 256-bit key).`; +} + +async function inspectCandidate(opts: { + readonly candidate: EnvKeyCandidate; + readonly uid: number | null; +}): Promise<{ + readonly present: boolean; + readonly findings: readonly EnvKeyFinding[]; +}> { + const { role, path } = opts.candidate; + const subject = SUBJECT_LABEL[role]; + const unreadable: EnvKeyFinding = { + code: "env-key-unreadable", + status: "error", + subject, + remedy: remedyFor(role, (p) => `ls -l ${p()}`), + }; + let stats: EntryStats | null; + try { + stats = await lstatOrNull(path); + } catch { + return { present: true, findings: [unreadable] }; + } + if (!stats) { + return { present: false, findings: [] }; + } + if (stats.isSymbolicLink()) { + return { + present: true, + findings: [ + { + code: "env-key-symlink", + status: "error", + subject, + remedy: remedyFor( + role, + (p) => + `after confirming the link target is your key: cp -L ${p()} ${p(".new")} && mv -f ${p(".new")} ${p()} && chmod 600 ${p()}` + ), + }, + ], + }; + } + if (!stats.isFile()) { + return { + present: true, + findings: [ + { + code: "env-key-not-regular", + status: "error", + subject, + remedy: remedyFor(role, (p) => `ls -ld ${p()}`), + }, + ], + }; + } + + const findings: EnvKeyFinding[] = []; + if (opts.uid !== null && stats.uid !== opts.uid) { + findings.push({ + code: "env-key-wrong-owner", + status: "warn", + subject, + remedy: remedyFor(role, (p) => `sudo chown "$(id -u)" ${p()}`), + }); + } + if (opts.uid !== null && (stats.mode & 0o077) !== 0) { + findings.push({ + code: "env-key-weak-permissions", + status: "warn", + subject, + remedy: remedyFor(role, (p) => `chmod 600 ${p()}`), + }); + } + + let keyText: string | null; + try { + keyText = await readBoundedPrivateFile({ + path, + directories: [], + canonical: false, + maxBytes: ENV_KEY_MAX_BYTES, + refusal: () => new EnvKeyUnreadableError(), + }); + } catch { + findings.push(unreadable); + return { present: true, findings }; + } + // Empty or whitespace-only files are skipped by key resolution. + const strength = + keyText === null ? "env-key-wrong-length" : classifyEnvKeyText(keyText); + if (strength) { + findings.push({ + code: strength, + status: "warn", + subject, + remedy: remedyFor(role, rotateRemedy), + }); + } + return { present: true, findings }; +} + +const PORTABLE_RELATIVE_PATH = /^[A-Za-z0-9._/-]+$/; + +/** Repo-relative path spelled only from portable characters, or null. */ +function portableRelativePath(path: string): string | null { + return PORTABLE_RELATIVE_PATH.test(path) && + !path.startsWith("/") && + !path.startsWith("-") && + !path.split("/").includes("..") + ? path + : null; +} + +async function inspectPublicArtifactCopies(opts: { + readonly projectRoot: string; +}): Promise { + const findings: EnvKeyFinding[] = []; + for (const dir of ENV_KEY_PUBLIC_ARTIFACT_DIRS) { + const relative = `${dir}/${PROJECT_ENV_KEY_FILENAME}`; + // An entry that exists but cannot be inspected still counts as present. + const present = await lstatOrNull(join(opts.projectRoot, relative)).then( + (stats) => stats !== null, + () => true + ); + if (present) { + findings.push({ + code: "env-key-public-path", + status: "error", + subject: relative, + remedy: `rm -- ${relative}, then rotate the key: a published copy is not secret.`, + }); + } + } + return findings; +} + +function sortFindings(findings: readonly EnvKeyFinding[]): EnvKeyFinding[] { + return findings + .map((finding, index) => ({ finding, index })) + .sort( + (left, right) => + STATUS_RANK[right.finding.status] - STATUS_RANK[left.finding.status] || + left.index - right.index + ) + .map(({ finding }) => finding); +} + +/** + * Inspects every present candidate key file, any git-tracked or published + * copy, and the `HACK_ENV_SECRET_KEY` fallback when no key file exists. + */ +export async function inspectProjectEnvKey( + opts: InspectEnvKeyOptions +): Promise { + const findings: EnvKeyFinding[] = []; + let anyPresent = false; + const seen = new Set(); + for (const candidate of opts.candidates) { + if (seen.has(candidate.path)) { + continue; + } + seen.add(candidate.path); + const inspected = await inspectCandidate({ candidate, uid: opts.uid }); + anyPresent ||= inspected.present; + findings.push(...inspected.findings); + } + + const tracked = await opts + .listTrackedKeyPaths({ projectRoot: opts.projectRoot }) + .catch(() => null); + for (const relative of [...new Set(tracked ?? [])].sort()) { + const portable = portableRelativePath(relative); + findings.push({ + code: "env-key-tracked", + status: "error", + subject: portable ?? `/${PROJECT_ENV_KEY_FILENAME}`, + remedy: + portable === null + ? "Untrack the committed key copy with 'git rm --cached', then rotate the key: a committed key is not secret." + : `git rm --cached -- ${portable}, then rotate the key: a committed key is not secret.`, + }); + } + + findings.push( + ...(await inspectPublicArtifactCopies({ projectRoot: opts.projectRoot })) + ); + + if (!anyPresent) { + const envKey = opts.env[ENV_KEY_ENV_NAME]?.trim() ?? ""; + if (envKey.length > 0) { + anyPresent = true; + const strength = classifyEnvKeyText(envKey); + if (strength) { + findings.push({ + code: strength, + status: "warn", + subject: `$${ENV_KEY_ENV_NAME}`, + remedy: `set ${ENV_KEY_ENV_NAME} to a 256-bit key such as the output of 'openssl rand -base64 32', then re-add each secret with 'hack env add --secret KEY VALUE'.`, + }); + } + } + } + + if (findings.length === 0) { + return { + status: "ok", + findings: [ + { + code: anyPresent ? "env-key-ok" : "env-key-absent", + status: "ok", + subject: PROJECT_ENV_KEY_FILENAME, + remedy: null, + }, + ], + }; + } + const sorted = sortFindings(findings); + return { status: sorted[0]?.status ?? "ok", findings: sorted }; +} + +async function canonicalPath(path: string): Promise { + try { + return await realpath(path); + } catch { + return path; + } +} + +/** + * Candidate key files in key-resolution order: checkout-local, shared under + * the git common dir, then the primary checkout's key for a linked worktree. + * Resolution never reads key contents. + */ +export async function resolveEnvKeyCandidates(opts: { + readonly projectRoot: string; +}): Promise { + const candidates: EnvKeyCandidate[] = [ + { role: "local", path: resolveProjectEnvKeyPath(opts) }, + ]; + const shared = await resolveProjectEnvSharedKeyLocation(opts); + if (shared) { + candidates.push({ role: "shared", path: shared.path }); + } + const primaryRoot = await resolveGitPrimaryWorktreeRoot({ + repoRoot: opts.projectRoot, + }); + if ( + primaryRoot && + (await canonicalPath(primaryRoot)) !== + (await canonicalPath(opts.projectRoot)) + ) { + candidates.push({ + role: "primary", + path: resolveProjectEnvKeyPath({ projectRoot: primaryRoot }), + }); + } + return candidates; +} + +/** + * Default tracked-key lister: `git ls-files` for every tracked file named like + * the key anywhere in the checkout. Null when git is unavailable or the root + * is not a work tree. + */ +export const gitListTrackedKeyPaths: ListTrackedKeyPaths = async (opts) => { + let result: Awaited>; + try { + result = await exec( + [ + "git", + "-C", + opts.projectRoot, + "ls-files", + "-z", + "--", + `:(glob)**/${PROJECT_ENV_KEY_FILENAME}`, + ], + { stdin: "ignore", timeoutMs: 5000 } + ); + } catch { + return null; + } + if (result.exitCode !== 0) { + return null; + } + return result.stdout + .split("\0") + .filter( + (path) => + path === PROJECT_ENV_KEY_FILENAME || + path.endsWith(`/${PROJECT_ENV_KEY_FILENAME}`) + ); +}; + +const SUMMARY: Record = { + "env-key-ok": "key material is private and in the 256-bit class", + "env-key-absent": "no project env key provisioned", + "env-key-symlink": "is a symlink; Hack refuses symlinked keys", + "env-key-not-regular": "is not a regular file", + "env-key-unreadable": "is unreadable, oversized or not UTF-8", + "env-key-weak-permissions": "is accessible by group or others (want 0600)", + "env-key-wrong-owner": "is not owned by the current user", + "env-key-tracked": "is tracked by git", + "env-key-public-path": "is present in a public artifact directory", + "env-key-wrong-length": "is shorter than a 256-bit key", + "env-key-low-entropy": "has too little entropy for a 256-bit key", +}; + +/** Value-free human summary of an inspection; Doctor never applies the suggestions. */ +export function describeEnvKeyInspection(inspection: EnvKeyInspection): string { + return inspection.findings + .map((finding) => { + const text = + finding.code === "env-key-ok" || finding.code === "env-key-absent" + ? `${finding.code}: ${SUMMARY[finding.code]}` + : `${finding.code}: ${finding.subject} ${SUMMARY[finding.code]}`; + return finding.remedy ? `${text}. Suggested: ${finding.remedy}` : text; + }) + .join("; "); +} diff --git a/tests/dependabot-inventory.test.ts b/tests/dependabot-inventory.test.ts new file mode 100644 index 00000000..84ec3a1b --- /dev/null +++ b/tests/dependabot-inventory.test.ts @@ -0,0 +1,228 @@ +import { describe, expect, test } from "bun:test"; +import { + DEFAULT_REPOSITORY, + type GhRunner, + InventoryError, + listOpenAlerts, + normalizeAlert, + parseInventoryArguments, + renderInventory, + runInventory, +} from "../scripts/dependabot-inventory.ts"; + +function alert(opts: { + readonly number: number; + readonly name: string; + readonly severity: string; + readonly manifest: string; + readonly fixedIn: string | null; + readonly state?: string; +}): Record { + return { + number: opts.number, + state: opts.state ?? "open", + dependency: { + manifest_path: opts.manifest, + package: { ecosystem: "npm", name: opts.name }, + }, + security_advisory: { + ghsa_id: `GHSA-test-${opts.number}`, + severity: opts.severity, + }, + security_vulnerability: { + severity: opts.severity, + vulnerable_version_range: "< 9.9.9", + first_patched_version: + opts.fixedIn === null ? null : { identifier: opts.fixedIn }, + }, + }; +} + +function stubGh(lines: readonly unknown[]): { + readonly gh: GhRunner; + readonly calls: string[][]; +} { + const calls: string[][] = []; + return { + calls, + gh: (args) => { + calls.push([...args]); + return Promise.resolve( + `${lines.map((line) => JSON.stringify(line)).join("\n")}\n` + ); + }, + }; +} + +describe("dependabot inventory", () => { + test("lists open alerts with GET-only paginated gh api calls", async () => { + const { gh, calls } = stubGh([ + alert({ + number: 2, + name: "drizzle-orm", + severity: "high", + manifest: "packages/db/package.json", + fixedIn: "0.45.2", + }), + alert({ + number: 1, + name: "next", + severity: "critical", + manifest: "examples/next-app/package.json", + fixedIn: "16.3.3", + }), + alert({ + number: 3, + name: "left-pad", + severity: "low", + manifest: "package.json", + fixedIn: null, + }), + alert({ + number: 4, + name: "fixed-already", + severity: "high", + manifest: "package.json", + fixedIn: "1.0.0", + state: "fixed", + }), + ]); + + const alerts = await listOpenAlerts({ repository: DEFAULT_REPOSITORY, gh }); + + expect(calls).toHaveLength(1); + const args = calls[0] ?? []; + expect(args.slice(0, 4)).toEqual(["api", "--method", "GET", "--paginate"]); + expect(args).not.toContain("POST"); + expect(args).not.toContain("PATCH"); + expect(args.at(-1)).toBe( + "repos/hack-dance/hack/dependabot/alerts?state=open&per_page=100" + ); + expect( + alerts.map((entry) => [ + entry.number, + entry.severity, + entry.package, + entry.manifest, + entry.fixedIn, + ]) + ).toEqual([ + [1, "critical", "next", "examples/next-app/package.json", "16.3.3"], + [2, "high", "drizzle-orm", "packages/db/package.json", "0.45.2"], + [3, "low", "left-pad", "package.json", null], + ]); + + const text = renderInventory({ repository: DEFAULT_REPOSITORY, alerts }); + expect(text).toContain( + "Open Dependabot alerts for hack-dance/hack: 3 (critical=1 high=1 medium=0 low=1 unknown=0)" + ); + expect(text).toContain("fixed-in"); + expect(text).toContain("no fix"); + }); + + test("renders JSON through the injected writer without touching files", async () => { + const { gh } = stubGh([ + alert({ + number: 7, + name: "next", + severity: "medium", + manifest: "examples/next-app/package.json", + fixedIn: "16.2.11", + }), + ]); + const output: string[] = []; + + const code = await runInventory({ + argv: ["--json", "--repo", "octo-org/example.repo"], + gh, + write: (text) => output.push(text), + }); + + expect(code).toBe(0); + const parsed: unknown = JSON.parse(output.join("")); + expect(parsed).toEqual({ + repository: "octo-org/example.repo", + alerts: [ + { + number: 7, + ecosystem: "npm", + package: "next", + severity: "medium", + manifest: "examples/next-app/package.json", + vulnerableRange: "< 9.9.9", + fixedIn: "16.2.11", + advisory: "GHSA-test-7", + }, + ], + }); + }); + + test("an empty inventory reports zero alerts", () => { + expect(renderInventory({ repository: "a/b", alerts: [] })).toBe( + "Open Dependabot alerts for a/b: 0 (critical=0 high=0 medium=0 low=0 unknown=0)\n" + ); + }); + + test("unprintable GitHub text is replaced, never echoed", () => { + const normalized = normalizeAlert( + alert({ + number: 9, + name: "evil\u001b[2Jpkg", + severity: "bogus", + manifest: "a\nb", + fixedIn: "1.0.0\u0007", + }) + ); + expect(normalized).toEqual({ + number: 9, + ecosystem: "npm", + package: "", + severity: "unknown", + manifest: "", + vulnerableRange: "< 9.9.9", + fixedIn: null, + advisory: "GHSA-test-9", + }); + expect(normalizeAlert({ state: "open" })).toBeNull(); + expect(normalizeAlert("not an object")).toBeNull(); + }); + + test("refusals are closed and never echo input or gh output", async () => { + const hostile = "owner/repo;$(touch /tmp/pwned)"; + expect(() => parseInventoryArguments(["--repo", hostile])).toThrow( + new InventoryError("Invalid --repo value (expected owner/name)") + ); + expect(() => parseInventoryArguments(["--token", "ghp_secret"])).toThrow( + new InventoryError("Unknown argument") + ); + expect(() => parseInventoryArguments(["--repo"])).toThrow(InventoryError); + + const failing: GhRunner = () => + Promise.reject(new Error("HTTP 403: secret-bearing gh stderr")); + const failure = await listOpenAlerts({ + repository: DEFAULT_REPOSITORY, + gh: failing, + }).catch((error: unknown) => error); + expect(failure).toBeInstanceOf(InventoryError); + expect((failure as Error).message).not.toContain("secret-bearing"); + + const garbage: GhRunner = () => Promise.resolve("{not json\n"); + await expect( + listOpenAlerts({ repository: DEFAULT_REPOSITORY, gh: garbage }) + ).rejects.toThrow(new InventoryError("Unexpected GitHub response")); + }); + + test("help prints usage without calling gh", async () => { + const { gh, calls } = stubGh([]); + const output: string[] = []; + expect( + await runInventory({ + argv: ["--help"], + gh, + write: (text) => output.push(text), + }) + ).toBe(0); + expect(calls).toHaveLength(0); + expect(output.join("")).toContain("Usage:"); + }); +}); diff --git a/tests/doctor-env-key.test.ts b/tests/doctor-env-key.test.ts new file mode 100644 index 00000000..5ff02d38 --- /dev/null +++ b/tests/doctor-env-key.test.ts @@ -0,0 +1,488 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { randomBytes } from "node:crypto"; +import { + chmod, + mkdir, + mkdtemp, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + buildDoctorJsonData, + envKeyCheckResult, +} from "../src/commands/doctor.ts"; +import { + classifyEnvKeyText, + type EnvKeyCandidate, + type EnvKeyInspection, + gitListTrackedKeyPaths, + type InspectEnvKeyOptions, + inspectProjectEnvKey, +} from "../src/lib/doctor-env-key.ts"; + +const KEY_FILE = ".hack.secret.key"; +const uid = typeof process.getuid === "function" ? process.getuid() : null; +const roots: string[] = []; + +afterEach(async () => { + await Promise.all( + roots.splice(0).map((root) => rm(root, { recursive: true, force: true })) + ); +}); + +async function tempProject(): Promise { + const root = await mkdtemp(join(tmpdir(), "hack-env-key-")); + roots.push(root); + return root; +} + +function generatedKey(): string { + return randomBytes(32).toString("base64url"); +} + +async function writeKey(opts: { + readonly path: string; + readonly text: string; + readonly mode: number; +}): Promise { + await writeFile(opts.path, `${opts.text}\n`, { mode: opts.mode }); + await chmod(opts.path, opts.mode); +} + +function localCandidate(root: string): EnvKeyCandidate { + return { role: "local", path: join(root, KEY_FILE) }; +} + +async function inspect( + root: string, + overrides: Partial = {} +): Promise { + return await inspectProjectEnvKey({ + projectRoot: root, + candidates: [localCandidate(root)], + env: {}, + uid, + listTrackedKeyPaths: async () => [], + ...overrides, + }); +} + +function codes(inspection: EnvKeyInspection): readonly string[] { + return inspection.findings.map((finding) => finding.code); +} + +/** Serialized doctor output (human message plus JSON) for leak assertions. */ +function serialized(inspection: EnvKeyInspection): string { + const check = envKeyCheckResult(inspection); + return `${check.message}\n${JSON.stringify( + buildDoctorJsonData({ results: [{ ...check, durationMs: 0 }] }) + )}`; +} + +function expectValueFree(opts: { + readonly output: string; + readonly root: string; + readonly secrets: readonly string[]; +}): void { + expect(opts.output).not.toContain(opts.root); + for (const secret of opts.secrets) { + expect(opts.output).not.toContain(secret); + } +} + +describe("env key doctor diagnostic", () => { + test("a private generated key reports env-key-ok without key bytes", async () => { + const root = await tempProject(); + const key = generatedKey(); + await writeKey({ path: join(root, KEY_FILE), text: key, mode: 0o600 }); + + const inspection = await inspect(root); + + expect(inspection.status).toBe("ok"); + expect(codes(inspection)).toEqual(["env-key-ok"]); + expectValueFree({ + output: serialized(inspection), + root, + secrets: [key], + }); + }); + + test("negative control: a 0644 key reports weak permissions", async () => { + const root = await tempProject(); + const key = generatedKey(); + await writeKey({ path: join(root, KEY_FILE), text: key, mode: 0o644 }); + + const inspection = await inspect(root); + + expect(inspection.status).toBe("warn"); + expect(codes(inspection)).toEqual(["env-key-weak-permissions"]); + expect(inspection.findings[0]).toEqual({ + code: "env-key-weak-permissions", + status: "warn", + subject: KEY_FILE, + remedy: `chmod 600 ${KEY_FILE}`, + }); + expectValueFree({ output: serialized(inspection), root, secrets: [key] }); + }); + + test("negative control: a symlinked key reports symlink and is not followed", async () => { + const root = await tempProject(); + const target = join(root, "elsewhere.key"); + // A weak target would add a strength finding if the link were followed. + await writeKey({ path: target, text: "short", mode: 0o600 }); + await symlink(target, join(root, KEY_FILE)); + + const inspection = await inspect(root); + + expect(inspection.status).toBe("error"); + expect(codes(inspection)).toEqual(["env-key-symlink"]); + expect(inspection.findings[0]?.remedy).toContain( + `cp -L ${KEY_FILE} ${KEY_FILE}.new && mv -f ${KEY_FILE}.new ${KEY_FILE} && chmod 600 ${KEY_FILE}` + ); + expectValueFree({ + output: serialized(inspection), + root, + secrets: ["elsewhere.key"], + }); + }); + + test("a dangling symlink still reports symlink", async () => { + const root = await tempProject(); + await symlink(join(root, "missing.key"), join(root, KEY_FILE)); + + expect(codes(await inspect(root))).toEqual(["env-key-symlink"]); + }); + + test("a directory at the key path reports not-regular", async () => { + const root = await tempProject(); + await mkdir(join(root, KEY_FILE)); + + const inspection = await inspect(root); + expect(inspection.status).toBe("error"); + expect(codes(inspection)).toEqual(["env-key-not-regular"]); + }); + + test("a short key reports wrong length and suggests rotation", async () => { + const root = await tempProject(); + const key = "hunter2-hunter2"; + await writeKey({ path: join(root, KEY_FILE), text: key, mode: 0o600 }); + + const inspection = await inspect(root); + + expect(codes(inspection)).toEqual(["env-key-wrong-length"]); + expect(inspection.findings[0]?.remedy).toContain( + `mv ${KEY_FILE} ${KEY_FILE}.old` + ); + expectValueFree({ output: serialized(inspection), root, secrets: [key] }); + }); + + test("a whitespace-only key reports wrong length", async () => { + const root = await tempProject(); + await writeKey({ path: join(root, KEY_FILE), text: " ", mode: 0o600 }); + + expect(codes(await inspect(root))).toEqual(["env-key-wrong-length"]); + }); + + test("a long repetitive key reports low entropy", async () => { + const root = await tempProject(); + const key = "password".repeat(8); + await writeKey({ path: join(root, KEY_FILE), text: key, mode: 0o600 }); + + const inspection = await inspect(root); + + expect(codes(inspection)).toEqual(["env-key-low-entropy"]); + expectValueFree({ output: serialized(inspection), root, secrets: [key] }); + }); + + test("invalid UTF-8 key material reports unreadable", async () => { + const root = await tempProject(); + const path = join(root, KEY_FILE); + await writeFile(path, Buffer.from([0xff, 0xfe, 0xfd]), { mode: 0o600 }); + + const inspection = await inspect(root); + expect(inspection.status).toBe("error"); + expect(codes(inspection)).toEqual(["env-key-unreadable"]); + }); + + test.skipIf(uid === null)( + "a key owned by another user reports wrong owner", + async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o600, + }); + + const inspection = await inspect(root, { uid: (uid ?? 0) + 1 }); + + expect(codes(inspection)).toEqual(["env-key-wrong-owner"]); + expect(inspection.findings[0]?.remedy).toBe( + `sudo chown "$(id -u)" ${KEY_FILE}` + ); + } + ); + + test("tracked key copies are errors with project-relative subjects only", async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o600, + }); + + const inspection = await inspect(root, { + listTrackedKeyPaths: async () => [ + KEY_FILE, + `config/${KEY_FILE}`, + `weird\u001b[31mdir/${KEY_FILE}`, + ], + }); + + expect(inspection.status).toBe("error"); + expect( + inspection.findings.map((finding) => [finding.code, finding.subject]) + ).toEqual([ + ["env-key-tracked", KEY_FILE], + ["env-key-tracked", `config/${KEY_FILE}`], + ["env-key-tracked", `/${KEY_FILE}`], + ]); + expect(inspection.findings[0]?.remedy).toContain( + `git rm --cached -- ${KEY_FILE}` + ); + expect(serialized(inspection)).not.toContain("\u001b"); + }); + + test("an unavailable git listing is not reported as tracked", async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o600, + }); + + const inspection = await inspect(root, { + listTrackedKeyPaths: () => Promise.reject(new Error("git exploded")), + }); + expect(codes(inspection)).toEqual(["env-key-ok"]); + }); + + test("a key copy in a public artifact directory is an error", async () => { + const root = await tempProject(); + const key = generatedKey(); + await writeKey({ path: join(root, KEY_FILE), text: key, mode: 0o600 }); + await mkdir(join(root, "public")); + await writeKey({ + path: join(root, "public", KEY_FILE), + text: key, + mode: 0o600, + }); + + const inspection = await inspect(root); + + expect(inspection.status).toBe("error"); + expect(inspection.findings).toEqual([ + { + code: "env-key-public-path", + status: "error", + subject: `public/${KEY_FILE}`, + remedy: `rm -- public/${KEY_FILE}, then rotate the key: a published copy is not secret.`, + }, + ]); + }); + + test("findings are ordered most severe first", async () => { + const root = await tempProject(); + await writeKey({ path: join(root, KEY_FILE), text: "short", mode: 0o644 }); + + const inspection = await inspect(root, { + listTrackedKeyPaths: async () => [KEY_FILE], + }); + + expect(codes(inspection)).toEqual([ + "env-key-tracked", + "env-key-weak-permissions", + "env-key-wrong-length", + ]); + }); + + test("no key anywhere reports env-key-absent", async () => { + const root = await tempProject(); + + const inspection = await inspect(root); + expect(inspection.status).toBe("ok"); + expect(codes(inspection)).toEqual(["env-key-absent"]); + }); + + test("the HACK_ENV_SECRET_KEY fallback is classified without echoing it", async () => { + const root = await tempProject(); + const weak = "not-a-real-key"; + + const weakInspection = await inspect(root, { + env: { HACK_ENV_SECRET_KEY: weak }, + }); + expect(weakInspection.findings.map((f) => [f.code, f.subject])).toEqual([ + ["env-key-wrong-length", "$HACK_ENV_SECRET_KEY"], + ]); + expectValueFree({ + output: serialized(weakInspection), + root, + secrets: [weak], + }); + + const strong = await inspect(root, { + env: { HACK_ENV_SECRET_KEY: generatedKey() }, + }); + expect(codes(strong)).toEqual(["env-key-ok"]); + }); + + test("a key file takes precedence over the environment fallback", async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o600, + }); + + const inspection = await inspect(root, { + env: { HACK_ENV_SECRET_KEY: "weak" }, + }); + expect(codes(inspection)).toEqual(["env-key-ok"]); + }); + + test("shared and primary keys use fixed labels and safe remedies", async () => { + const root = await tempProject(); + const sharedDir = join(root, "common"); + const primaryDir = join(root, "primary"); + await mkdir(sharedDir); + await mkdir(primaryDir); + await writeKey({ + path: join(sharedDir, KEY_FILE), + text: generatedKey(), + mode: 0o640, + }); + await writeKey({ + path: join(primaryDir, KEY_FILE), + text: generatedKey(), + mode: 0o604, + }); + + const inspection = await inspect(root, { + candidates: [ + localCandidate(root), + { role: "shared", path: join(sharedDir, KEY_FILE) }, + { role: "primary", path: join(primaryDir, KEY_FILE) }, + ], + }); + + expect(inspection.findings).toEqual([ + { + code: "env-key-weak-permissions", + status: "warn", + subject: `/${KEY_FILE}`, + remedy: `chmod 600 "$(git rev-parse --git-common-dir)/${KEY_FILE}"`, + }, + { + code: "env-key-weak-permissions", + status: "warn", + subject: `/${KEY_FILE}`, + remedy: + "Run 'hack doctor' in the primary checkout and follow its env key guidance.", + }, + ]); + expectValueFree({ output: serialized(inspection), root, secrets: [] }); + }); + + test("permission and ownership checks are skipped without a POSIX uid", async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o644, + }); + + expect(codes(await inspect(root, { uid: null }))).toEqual(["env-key-ok"]); + }); + + test("doctor JSON carries the closed envKey payload", async () => { + const root = await tempProject(); + await writeKey({ + path: join(root, KEY_FILE), + text: generatedKey(), + mode: 0o644, + }); + const check = envKeyCheckResult(await inspect(root)); + + const data = buildDoctorJsonData({ + results: [{ ...check, durationMs: 3 }], + }); + + expect(data.checks[0]?.id).toBe("env key"); + expect(data.checks[0]?.status).toBe("warn"); + expect(data.checks[0]?.detail).toContain("env-key-weak-permissions"); + expect(data.checks[0]?.envKey?.findings.map((f) => f.code)).toEqual([ + "env-key-weak-permissions", + ]); + }); +}); + +describe("classifyEnvKeyText", () => { + test("accepts generated keys and other 256-bit encodings", () => { + for (let index = 0; index < 200; index += 1) { + expect(classifyEnvKeyText(generatedKey())).toBeNull(); + expect(classifyEnvKeyText(randomBytes(32).toString("base64"))).toBeNull(); + expect(classifyEnvKeyText(randomBytes(32).toString("hex"))).toBeNull(); + } + }); + + test("rejects short and repetitive text", () => { + expect(classifyEnvKeyText("")).toBe("env-key-wrong-length"); + expect(classifyEnvKeyText(generatedKey().slice(0, 42))).toBe( + "env-key-wrong-length" + ); + expect(classifyEnvKeyText("a".repeat(64))).toBe("env-key-low-entropy"); + expect(classifyEnvKeyText("abcdefg".repeat(10))).toBe( + "env-key-low-entropy" + ); + }); +}); + +describe("gitListTrackedKeyPaths", () => { + test("lists tracked key copies at any depth in a temp repository", async () => { + const root = await tempProject(); + const gitEnv = { + PATH: process.env.PATH ?? "", + HOME: root, + GIT_CONFIG_NOSYSTEM: "1", + GIT_CONFIG_GLOBAL: "/dev/null", + }; + const git = async (args: readonly string[]) => { + const proc = Bun.spawn(["git", "-C", root, ...args], { + env: gitEnv, + stdin: "ignore", + stdout: "ignore", + stderr: "ignore", + }); + expect(await proc.exited).toBe(0); + }; + await git(["init", "-q"]); + await mkdir(join(root, "nested", "deeper"), { recursive: true }); + await writeFile(join(root, KEY_FILE), "x"); + await writeFile(join(root, "nested", "deeper", KEY_FILE), "x"); + await writeFile(join(root, "nested", `not${KEY_FILE}`), "x"); + await git(["add", "-f", "--", "."]); + + expect(await gitListTrackedKeyPaths({ projectRoot: root })).toEqual([ + KEY_FILE, + `nested/deeper/${KEY_FILE}`, + ]); + }); + + test("returns null outside a git work tree", async () => { + const root = await tempProject(); + expect(await gitListTrackedKeyPaths({ projectRoot: root })).toBeNull(); + }); +}); From 19726d319d0beb8917d19a60a9b8dedde2425bb3 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 08:54:12 -0400 Subject: [PATCH 47/52] feat(setup): make setup sync refresh-only with --install-missing opt-in `hack setup sync` now refreshes only integrations that are already installed and Hack-owned (current or known prior standalone renders, the marker-delimited AGENTS.md/CLAUDE.md section, the Claude primer hook entries, the MCP `hack` entry). Missing targets are reported as `missing` and never created; customized or unrecognized files are `preserved` with a closed reason and left byte-identical. `--install-missing` opts in to creating missing targets in the selected scope. `--check` stays read-only and reports what sync would do with the same flags; `--json` lists every target with its status. Paths derive from an explicit project root and home, so tests inject temp roots instead of the process HOME. Refs: HACK-1132 Co-Authored-By: Claude Opus 5.5 --- .codex/skills/hack-cli/SKILL.md | 4 +- .cursor/rules/hack.mdc | 4 +- docs/cli.md | 4 +- docs/integrations.md | 40 +- docs/reference/cli.md | 6 +- examples/basic/AGENTS.md | 4 +- examples/basic/CLAUDE.md | 4 +- plugins/hack/rules/hack.mdc | 4 +- plugins/hack/skills/hack-cli/SKILL.md | 4 +- src/agents/claude.ts | 40 + src/agents/instruction-source.ts | 2 +- src/agents/integration-ownership.ts | 15 +- src/agents/integration-revision.ts | 2 +- src/agents/setup-sync.ts | 762 ++++++++++++++++++++ src/commands/setup.ts | 444 ++++++------ src/mcp/agent-docs.ts | 7 + src/mcp/install.ts | 12 +- tests/e2e/README.md | 4 +- tests/e2e/scenarios/agent-docs-sync.ts | 39 +- tests/e2e/scenarios/agent-surface-freeze.ts | 12 +- tests/setup-sync.test.ts | 588 +++++++++++++++ tests/setup.test.ts | 173 +++-- 22 files changed, 1872 insertions(+), 302 deletions(-) create mode 100644 src/agents/setup-sync.ts create mode 100644 tests/setup-sync.test.ts diff --git a/.codex/skills/hack-cli/SKILL.md b/.codex/skills/hack-cli/SKILL.md index 28776a4d..9d6d0a6e 100644 --- a/.codex/skills/hack-cli/SKILL.md +++ b/.codex/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). ## Product boundary @@ -181,7 +181,7 @@ Use `hack` as the primary interface for local-first development. ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/.cursor/rules/hack.mdc b/.cursor/rules/hack.mdc index 2dcc4e8f..03d913b1 100644 --- a/.cursor/rules/hack.mdc +++ b/.cursor/rules/hack.mdc @@ -9,7 +9,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). ## Product boundary @@ -76,7 +76,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/docs/cli.md b/docs/cli.md index 67846089..f085919a 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -131,7 +131,9 @@ the Hack CLI version that generated them. Plain `hack agent prime` prints curren an integration scan. Use `hack agent prime --check` for an explicit project/global inventory or a targeted `hack setup ... --check` when diagnosing drift. Missing optional integrations do not require installation. Repair only affected targets within the authorized scope; use -`hack setup sync --all-scopes` for an explicitly requested full refresh. Read updated guidance after +`hack setup sync --all-scopes` for an explicitly requested full refresh. Sync refreshes only installed +Hack-owned integrations; add `--install-missing` to create missing ones (see +[integration setup](integrations.md#refresh-only-hack-setup-sync)). Read updated guidance after repair; restart only if the client cannot reload changed hooks or skills. Ordinary commands, `hack update`, and `hack doctor --fix` never render, repair, remove, or otherwise mutate integration files. CLI upgrades do not trigger integration updates. diff --git a/docs/integrations.md b/docs/integrations.md index 9519e9e7..ce1bfeaf 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -10,9 +10,10 @@ What remains: `hack agent init` / `hack agent prime`, and scoped `hack setup` commands `hack setup sync` keeps interactive output compact: it summarizes each scope and only expands the -path and reason for stale, missing, or failed artifacts. Exit status remains the automation contract, -and the individual `hack setup cursor|claude|codex|agents|mcp --check` commands remain available when -you need per-artifact detail. +path and reason for preserved or failed artifacts (and, with `--check`, the ones sync would change). +Exit status remains the automation contract, `--json` lists every target, and the individual +`hack setup cursor|claude|codex|agents|mcp --check` commands remain available when you need +per-client detail. What does not ship: Hack Tickets, built-in GitHub or Linear integrations, hosted auth/account/org/team surfaces, and the web dashboard control plane. @@ -46,7 +47,38 @@ files Hack owns: the current render or an unedited render from an earlier releas unrecognized `SKILL.md` or `hack.mdc` is reported as `preserved` with a reason and left unchanged; move or delete it yourself to reinstall. Removal deletes only the owned `SKILL.md` and removes its skill directory only when nothing else is in it. Claude hook removal drops only the -`hack agent prime` entries and keeps other commands. Preserved files do not change the exit status. +`hack agent prime` entries and keeps other commands. Preserved files do not change the exit status +of an install, sync, or remove. + +### Refresh-only `hack setup sync` + +`hack setup sync` refreshes only integrations that are already installed and Hack-owned. It never +creates a missing one. Per target, in the selected scope (project by default, `--global`, or +`--all-scopes`): + +| Status | Meaning | +| --- | --- | +| `refreshed` | An installed Hack-owned target was rewritten to the current render | +| `unchanged` | The target already matches the current render | +| `preserved` | The file is customized or unrecognized; it is left byte-identical and the closed reason is reported | +| `missing` | The target is not installed; sync skipped it | +| `installed` | Created only with `--install-missing` | +| `removed` | A recognized retired Hack artifact was deleted (or, with `--remove`, a Hack-owned target) | +| `error` | The target could not be read or parsed; it is left unchanged | + +Hack-owned means: a current or known earlier render of a standalone skill or rule file; the +marker-delimited section of `AGENTS.md`/`CLAUDE.md`; the `hack agent prime` entries of the Claude +settings file; and the `hack` entry of an MCP config. A doc, settings file, or MCP config without +the Hack section counts as `missing`, so the file is not modified. Existing MCP entries are +never replaced. + +Add `--install-missing` to also create missing integrations in the selected scope; customized +files stay `preserved`. `--check` is read-only: it reports what sync would do with the same flags +and exits non-zero when sync would refresh, install, or remove something, a file is preserved, +or a target errors. A missing target fails `--check` only together with `--install-missing`. +`--json` prints `{ok, data: {action, dryRun, installMissing, counts, targets}}`, where each +target carries `scope`, `target`, `path`, `status`, and, when preserved, `reason`. Messages are +fixed text and never include file contents. ## Standalone MCP configuration diff --git a/docs/reference/cli.md b/docs/reference/cli.md index dddd00ec..a03c4f77 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1801,7 +1801,7 @@ hack setup [options] | `hack setup claude` | Install Claude Code hooks for hack CLI usage | | `hack setup codex` | Install Codex skill for hack CLI usage | | `hack setup agents` | Install AGENTS.md / CLAUDE.md snippets for hack CLI usage | -| `hack setup sync` | Refresh project/global agent guidance | +| `hack setup sync` | Refresh installed Hack-owned agent integrations | | `hack setup mcp` | Install MCP configs for hack CLI usage (no-shell only) | ### Options @@ -1924,7 +1924,7 @@ hack setup agents [options] ## `hack setup sync` -Refresh project/global agent guidance +Refresh installed Hack-owned agent integrations ### Usage @@ -1939,8 +1939,10 @@ hack setup sync [options] | `--path, -p ` | Run a project command against a repo path (overrides cwd search) | | `--global` | Use global (user) scope instead of project scope | | `--all-scopes` | Target both project and global (user) scopes | +| `--install-missing` | Also create missing integrations in the selected scope (default: refresh only installed Hack-owned files) | | `--check` | Check whether integration is installed | | `--remove` | Remove integration files/config | +| `--json` | Output JSON (machine-readable) | | `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) | | `--help, -h` | Show help | | `--version, -v` | Show version | diff --git a/examples/basic/AGENTS.md b/examples/basic/AGENTS.md index c5fb011c..a3f20631 100644 --- a/examples/basic/AGENTS.md +++ b/examples/basic/AGENTS.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -153,7 +153,7 @@ Daemon (optional): Agent integration maintenance: - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/examples/basic/CLAUDE.md b/examples/basic/CLAUDE.md index c5fb011c..a3f20631 100644 --- a/examples/basic/CLAUDE.md +++ b/examples/basic/CLAUDE.md @@ -6,7 +6,7 @@ Use `hack` as the single interface for local-first runtime orchestration (compos Integration freshness: - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). Product boundary: - Supported CLI surface: project init, up/down/restart, open, logs, env, host exec/shell, sessions, doctor, and daemon. @@ -153,7 +153,7 @@ Daemon (optional): Agent integration maintenance: - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/plugins/hack/rules/hack.mdc b/plugins/hack/rules/hack.mdc index e184f0ef..411c48d4 100644 --- a/plugins/hack/rules/hack.mdc +++ b/plugins/hack/rules/hack.mdc @@ -14,7 +14,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). ## Product boundary @@ -81,7 +81,7 @@ Prefer `hack` when shell access is available. Use MCP only when shell access is ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/plugins/hack/skills/hack-cli/SKILL.md b/plugins/hack/skills/hack-cli/SKILL.md index bbf2ff1b..218dc42d 100644 --- a/plugins/hack/skills/hack-cli/SKILL.md +++ b/plugins/hack/skills/hack-cli/SKILL.md @@ -14,7 +14,7 @@ Use `hack` as the primary interface for local-first development. - These instructions were generated by hack CLI v4.2.1; use the content revision and command behavior to assess freshness. - Check guidance only when Hack is relevant and there is evidence of drift; use targeted `hack setup` checks for a read-only inventory. Routine tasks do not require a startup audit. -- Content revision: `51163ae32e56` (version alone is not a freshness guarantee). +- Content revision: `868831c8b339` (version alone is not a freshness guarantee). ## Product boundary @@ -181,7 +181,7 @@ Use `hack` as the primary interface for local-first development. ## Agent integration maintenance - Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files. -- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh. +- Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`. - Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations. - CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills. diff --git a/src/agents/claude.ts b/src/agents/claude.ts index ca401883..96813ea0 100644 --- a/src/agents/claude.ts +++ b/src/agents/claude.ts @@ -241,6 +241,46 @@ export async function removeClaudeHooks(opts: { return preserved ?? { scope: opts.scope, status: "removed", path }; } +/** + * Coverage of the Hack primer hooks in a parsed Claude settings object: + * `none` (no Hack primer entry), `partial` (some events), or `complete`. + */ +export function inspectClaudePrimerHooks({ + settings, +}: { + readonly settings: Record; +}): "none" | "partial" | "complete" { + const hooks = resolveHooksMap(settings); + const present = HOOK_EVENTS.filter((event) => + eventHasCommand({ + eventHooks: resolveEventHooks({ hooks, event }), + command: HOOK_COMMAND, + }) + ).length; + if (present === 0) { + return "none"; + } + return present === HOOK_EVENTS.length ? "complete" : "partial"; +} + +/** Add any missing Hack primer hook entries in place; returns whether it changed. */ +export function ensureClaudePrimerHooks({ + settings, +}: { + readonly settings: Record; +}): boolean { + return upsertHooks({ settings, command: HOOK_COMMAND }); +} + +/** Remove only the Hack primer hook entries in place; returns whether it changed. */ +export function stripClaudePrimerHooks({ + settings, +}: { + readonly settings: Record; +}): boolean { + return removeHooks({ settings, command: HOOK_COMMAND }); +} + function resolveClaudeSettingsPath(opts: { readonly scope: ClaudeScope; readonly projectRoot?: string; diff --git a/src/agents/instruction-source.ts b/src/agents/instruction-source.ts index 25a1cb56..df087169 100644 --- a/src/agents/instruction-source.ts +++ b/src/agents/instruction-source.ts @@ -293,7 +293,7 @@ export const INSTRUCTION_SECTIONS: readonly InstructionSection[] = [ surfaces: ALL_SURFACES, bullets: [ "Ordinary Hack commands, `hack update`, `hack doctor --fix`, and primer hooks never install plugins or mutate agent integration files.", - "Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh.", + "Use the matching `hack setup` command only for the affected client and authorized project scope; add `--global` only when user-scope changes are authorized. Use `--check` for read-only inspection. Reserve `hack setup sync --all-scopes` for an explicitly requested full refresh; it refreshes only installed Hack-owned files and creates missing integrations only with `--install-missing`.", "Standalone setup and optional native plugins are alternative integration paths. A missing optional integration does not require installation. Verify the replacement loads before explicitly removing duplicate Hack-owned entries; preserve customizations.", "CLI upgrades do not authorize integration changes. Edit guidance at its generator source and render the affected targets; restart only if the client cannot reload the changed hooks or skills.", ], diff --git a/src/agents/integration-ownership.ts b/src/agents/integration-ownership.ts index 4156a625..12a76cb8 100644 --- a/src/agents/integration-ownership.ts +++ b/src/agents/integration-ownership.ts @@ -106,6 +106,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "121d8f26374d24f2a09890a3", // v5.0.0-next.3..next "6449442a8f230412d4e5f9e7", // v5.0.0-next.4..next "d83da046cf7541ba30950880", // v5.0.0-next.4..next (offline env-matrix pointer) + "08f0db2d0a1889e5b3ef707f", // v5.0.0-next.4..next (refresh-only setup sync) ]), "hack-init-skill": new Set([ "cd15fb6d8ed8e25071d05e75", // v3.1.0..v4.0.3 @@ -133,6 +134,7 @@ const KNOWN_RENDER_FINGERPRINTS: Readonly< "06cf6d0920afc1780b9946ba", // v5.0.0-next.3..next "cf0a31e197f9034485413674", // v5.0.0-next.4..next "3a99c9097855f93808e482cb", // v5.0.0-next.4..next (offline env-matrix pointer) + "19099625646d196aa0e8a0c0", // v5.0.0-next.4..next (refresh-only setup sync) ]), }; @@ -232,7 +234,7 @@ export function describePreservedReason({ action, }: { readonly reason: IntegrationPreservedReason; - readonly action: "install" | "remove"; + readonly action: "install" | "remove" | "sync"; }): string { const cause: Readonly> = { "customized-content": @@ -241,10 +243,13 @@ export function describePreservedReason({ "not-a-regular-file": "it is not a regular file", unreadable: "it could not be read", }; - const next = - action === "install" - ? "Move or delete it, then rerun setup to install the current version." - : "Delete it yourself if it is no longer needed."; + const nextSteps: Readonly> = { + install: + "Move or delete it, then rerun setup to install the current version.", + sync: "Move or delete it, then run `hack setup sync --install-missing` or the client's `hack setup` command to install the current version.", + remove: "Delete it yourself if it is no longer needed.", + }; + const next = nextSteps[action]; return `Hack left this path unchanged because ${cause[reason]}. ${next}`; } diff --git a/src/agents/integration-revision.ts b/src/agents/integration-revision.ts index d418edef..64e19e42 100644 --- a/src/agents/integration-revision.ts +++ b/src/agents/integration-revision.ts @@ -3,4 +3,4 @@ * source test recomputes this value and fails whenever guidance changes * without a revision update. */ -export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "51163ae32e56"; +export const HACK_AGENT_INTEGRATION_CONTENT_REVISION = "868831c8b339"; diff --git a/src/agents/setup-sync.ts b/src/agents/setup-sync.ts new file mode 100644 index 00000000..5de2430b --- /dev/null +++ b/src/agents/setup-sync.ts @@ -0,0 +1,762 @@ +import { dirname, resolve } from "node:path"; + +import { + ensureDir, + readOptionalTextFile, + writeTextFileIfChanged, +} from "../lib/fs.ts"; +import { isRecord } from "../lib/guards.ts"; +import { + checkAgentDocs, + removeAgentDocs, + upsertAgentDocs, +} from "../mcp/agent-docs.ts"; +import { + checkMcpConfig, + installMcpConfig, + type McpTarget, + removeMcpConfig, +} from "../mcp/install.ts"; +import { + ensureClaudePrimerHooks, + inspectClaudePrimerHooks, + stripClaudePrimerHooks, +} from "./claude.ts"; +import { renderCodexSkill } from "./codex-skill.ts"; +import { renderCursorRules } from "./cursor.ts"; +import { renderHackInitSkill } from "./hack-init-skill.ts"; +import { normalizeInstructionText } from "./instruction-source.ts"; +import { + classifyIntegrationFile, + describePreservedReason, + type IntegrationPreservedReason, + type IntegrationSurface, + installOwnedIntegrationFile, + removeOwnedIntegrationFile, +} from "./integration-ownership.ts"; +import { + checkLegacyProjectAgentArtifacts, + checkLegacyUserAgentArtifacts, + type LegacyAgentArtifactResult, + removeLegacyProjectAgentArtifacts, + removeLegacyUserAgentArtifacts, +} from "./legacy-artifacts.ts"; + +/** + * Target-level engine for `hack setup sync` (HACK-1132). + * + * Contract: + * - `sync` is refresh-only by default. It rewrites a target only when the + * target already exists and Hack owns it (a current or known prior render + * for standalone files, the marker-delimited section for agent docs, the + * Hack primer entries for Claude hooks). A missing target is reported as + * `missing` and is never created. + * - `installMissing` opts in to creating missing targets in the selected + * scope. It never overrides the ownership gate. + * - A customized or unrecognized standalone file is `preserved` with a + * closed, value-free reason and left byte-identical. + * - `check` computes the outcome `sync` would report and writes nothing. + * - `remove` deletes only Hack-owned content. + * + * Every path derives from the explicit `projectRoot` and `home` inputs; this + * module never reads `HOME` itself. Diagnostics are closed strings that carry + * no file content or caller input. Classification and the following write + * are not atomic (see `integration-ownership.ts`). + */ + +export type SetupSyncAction = "sync" | "check" | "remove"; + +export type SetupSyncScope = "project" | "global"; + +export type SetupSyncTarget = + | "cursor-rules" + | "claude-hooks" + | "claude-hack-init-skill" + | "codex-hack-cli-skill" + | "codex-hack-init-skill" + | "shared-hack-cli-skill" + | "cursor-mcp" + | "claude-mcp" + | "codex-mcp" + | "agents-md" + | "claude-md" + | "retired-artifact" + | "scope-root"; + +/** + * Per-target outcome. With `check`, the status is what `sync` would report. + * `missing` means skipped: the target does not exist and was not created. + */ +export type SetupSyncStatus = + | "refreshed" + | "unchanged" + | "installed" + | "preserved" + | "missing" + | "removed" + | "error"; + +export type SetupSyncTargetResult = { + readonly scope: SetupSyncScope; + readonly target: SetupSyncTarget; + readonly path: string; + readonly status: SetupSyncStatus; + /** Set with `preserved`: why Hack left the file unchanged. */ + readonly reason?: IntegrationPreservedReason; + readonly message?: string; +}; + +type Outcome = Omit; + +type OwnedFileTarget = { + readonly target: SetupSyncTarget; + readonly surface: IntegrationSurface; + readonly relativePath: string; + readonly render: () => string; + readonly removeEmptyDirectory: boolean; +}; + +const PROJECT_OWNED_FILES: readonly OwnedFileTarget[] = [ + { + target: "cursor-rules", + surface: "cursor-rules", + relativePath: ".cursor/rules/hack.mdc", + render: renderCursorRules, + removeEmptyDirectory: false, + }, + { + target: "claude-hack-init-skill", + surface: "hack-init-skill", + relativePath: ".claude/skills/hack-init/SKILL.md", + render: renderHackInitSkill, + removeEmptyDirectory: true, + }, + { + target: "codex-hack-cli-skill", + surface: "hack-cli-skill", + relativePath: ".codex/skills/hack-cli/SKILL.md", + render: renderCodexSkill, + removeEmptyDirectory: true, + }, + { + target: "codex-hack-init-skill", + surface: "hack-init-skill", + relativePath: ".codex/skills/hack-init/SKILL.md", + render: renderHackInitSkill, + removeEmptyDirectory: true, + }, +]; + +const GLOBAL_OWNED_FILES: readonly OwnedFileTarget[] = [ + ...PROJECT_OWNED_FILES, + { + target: "shared-hack-cli-skill", + surface: "hack-cli-skill", + relativePath: ".ai/skills/hack-cli/SKILL.md", + render: renderCodexSkill, + removeEmptyDirectory: true, + }, +]; + +const CLAUDE_SETTINGS_PATHS: Readonly> = { + project: ".claude/settings.local.json", + global: ".claude/settings.json", +}; + +const MCP_TARGETS: readonly { + readonly target: SetupSyncTarget; + readonly client: McpTarget; +}[] = [ + { target: "cursor-mcp", client: "cursor" }, + { target: "claude-mcp", client: "claude" }, + { target: "codex-mcp", client: "codex" }, +]; + +const DOC_TARGETS: readonly { + readonly target: SetupSyncTarget; + readonly doc: "agents" | "claude"; +}[] = [ + { target: "agents-md", doc: "agents" }, + { target: "claude-md", doc: "claude" }, +]; + +/** + * Run one sync action for one scope. `root` is the project root for the + * project scope and the home directory for the global scope; an empty root + * yields one `error` result instead of touching any path. + */ +export async function runSetupSyncScope(opts: { + readonly action: SetupSyncAction; + readonly scope: SetupSyncScope; + readonly root: string; + readonly installMissing: boolean; +}): Promise { + const root = opts.root.trim(); + if (!root) { + return [ + { + scope: opts.scope, + target: "scope-root", + path: "", + status: "error", + message: + opts.scope === "global" + ? "HOME is not set; cannot resolve global integration paths." + : "Project root could not be resolved.", + }, + ]; + } + + const context: ScopeContext = { + action: opts.action, + scope: opts.scope, + root, + installMissing: opts.action === "sync" ? opts.installMissing : false, + planInstall: opts.action === "check" && opts.installMissing, + }; + const ownedFiles = + opts.scope === "global" ? GLOBAL_OWNED_FILES : PROJECT_OWNED_FILES; + + const results: SetupSyncTargetResult[] = []; + for (const file of ownedFiles) { + results.push(await runOwnedFileTarget({ context, file })); + } + results.push(await runClaudeHooksTarget({ context })); + results.push(...(await runRetiredArtifacts({ context }))); + for (const mcp of MCP_TARGETS) { + results.push(await runMcpTarget({ context, ...mcp })); + } + if (opts.scope === "project") { + for (const doc of DOC_TARGETS) { + results.push(await runAgentDocTarget({ context, ...doc })); + } + } + return results; +} + +/** + * Exit status for a sync action. `sync` and `remove` fail only on errors. + * `check` fails when `sync` with the same flags would change something, a + * file is preserved (its content is not Hack's current render), or a target + * could not be checked. A missing target is not a check failure unless + * `--install-missing` was requested, in which case it reports `installed`. + */ +export function setupSyncExitCode(opts: { + readonly action: SetupSyncAction; + readonly results: readonly SetupSyncTargetResult[]; +}): number { + return opts.results.some((result) => + isSetupSyncFailure({ action: opts.action, status: result.status }) + ) + ? 1 + : 0; +} + +/** Whether one target status fails the given action. */ +export function isSetupSyncFailure(opts: { + readonly action: SetupSyncAction; + readonly status: SetupSyncStatus; +}): boolean { + if (opts.status === "error") { + return true; + } + if (opts.action !== "check") { + return false; + } + return ( + opts.status === "refreshed" || + opts.status === "installed" || + opts.status === "removed" || + opts.status === "preserved" + ); +} + +type ScopeContext = { + readonly action: SetupSyncAction; + readonly scope: SetupSyncScope; + readonly root: string; + /** Write missing targets (sync with `--install-missing`). */ + readonly installMissing: boolean; + /** Report missing targets as `installed` (check with `--install-missing`). */ + readonly planInstall: boolean; +}; + +function withTarget(opts: { + readonly context: ScopeContext; + readonly target: SetupSyncTarget; + readonly path: string; + readonly outcome: Outcome; +}): SetupSyncTargetResult { + return { + scope: opts.context.scope, + target: opts.target, + path: opts.path, + ...opts.outcome, + }; +} + +/** Outcome for a target that does not exist yet. */ +function missingOutcome(context: ScopeContext): Outcome | null { + if (context.planInstall) { + return { status: "installed" }; + } + if (!context.installMissing) { + return { status: "missing" }; + } + return null; +} + +function preservedOutcome(reason: IntegrationPreservedReason): Outcome { + return { + status: "preserved", + reason, + message: describePreservedReason({ reason, action: "sync" }), + }; +} + +async function runOwnedFileTarget(opts: { + readonly context: ScopeContext; + readonly file: OwnedFileTarget; +}): Promise { + const { context, file } = opts; + const path = resolve(context.root, file.relativePath); + const outcome = + context.action === "remove" + ? await removeOwnedFile({ file, path }) + : await syncOwnedFile({ context, file, path }); + return withTarget({ context, target: file.target, path, outcome }); +} + +async function syncOwnedFile(opts: { + readonly context: ScopeContext; + readonly file: OwnedFileTarget; + readonly path: string; +}): Promise { + const { context, file, path } = opts; + const render = file.render(); + const ownership = await classifyIntegrationFile({ + surface: file.surface, + path, + currentRender: render, + }); + if (ownership.kind === "absent") { + const planned = missingOutcome(context); + if (planned) { + return planned; + } + const installed = await installOwnedIntegrationFile({ + surface: file.surface, + path, + render, + }); + return installed.status === "preserved" + ? preservedOutcome(installed.reason) + : { status: "installed" }; + } + if (ownership.kind === "customized" || ownership.kind === "unknown") { + return preservedOutcome(ownership.reason); + } + + const existing = await readOptionalTextFile(path); + if (existing.kind !== "text") { + return preservedOutcome("unreadable"); + } + if ( + normalizeInstructionText({ text: existing.text }) === + normalizeInstructionText({ text: render }) + ) { + return { status: "unchanged" }; + } + if (context.action === "check") { + return { status: "refreshed" }; + } + const refreshed = await installOwnedIntegrationFile({ + surface: file.surface, + path, + render, + }); + if (refreshed.status === "preserved") { + return preservedOutcome(refreshed.reason); + } + return { status: refreshed.status === "noop" ? "unchanged" : "refreshed" }; +} + +async function removeOwnedFile(opts: { + readonly file: OwnedFileTarget; + readonly path: string; +}): Promise { + const outcome = await removeOwnedIntegrationFile({ + surface: opts.file.surface, + path: opts.path, + currentRender: opts.file.render(), + removeEmptyDirectory: opts.file.removeEmptyDirectory, + }); + if (outcome.status === "preserved") { + return { + status: "preserved", + reason: outcome.reason, + message: outcome.message, + }; + } + if (outcome.status === "removed" && outcome.message) { + return { status: "removed", message: outcome.message }; + } + return { status: outcome.status }; +} + +type ParsedSettings = + | { readonly kind: "absent" } + | { readonly kind: "error"; readonly message: string } + | { readonly kind: "ok"; readonly value: Record }; + +async function readClaudeSettings(path: string): Promise { + const file = await readOptionalTextFile(path); + if (file.kind === "absent") { + return { kind: "absent" }; + } + if (file.kind === "unreadable") { + return { + kind: "error", + message: "Claude settings exist but could not be read; left unchanged.", + }; + } + if (file.text.trim().length === 0) { + return { kind: "ok", value: {} }; + } + try { + const parsed: unknown = JSON.parse(file.text); + if (isRecord(parsed)) { + return { kind: "ok", value: parsed }; + } + } catch { + // Fall through to the closed diagnostic; never echo parser output. + } + return { + kind: "error", + message: "Claude settings are not a JSON object; left unchanged.", + }; +} + +async function writeClaudeSettings(opts: { + readonly path: string; + readonly settings: Record; +}): Promise { + await ensureDir(dirname(opts.path)); + const result = await writeTextFileIfChanged( + opts.path, + `${JSON.stringify(opts.settings, null, 2)}\n` + ); + return result.changed; +} + +async function runClaudeHooksTarget(opts: { + readonly context: ScopeContext; +}): Promise { + const { context } = opts; + const path = resolve(context.root, CLAUDE_SETTINGS_PATHS[context.scope]); + const settings = await readClaudeSettings(path); + let outcome: Outcome; + if (settings.kind === "error") { + outcome = { status: "error", message: settings.message }; + } else if (context.action === "remove") { + outcome = await removeClaudeHooks({ path, settings }); + } else { + outcome = await syncClaudeHooks({ + context, + path, + value: settings.kind === "ok" ? settings.value : {}, + }); + } + return withTarget({ context, target: "claude-hooks", path, outcome }); +} + +async function removeClaudeHooks(opts: { + readonly path: string; + readonly settings: Exclude; +}): Promise { + if (opts.settings.kind === "absent") { + return { status: "missing" }; + } + if (!stripClaudePrimerHooks({ settings: opts.settings.value })) { + return { status: "missing" }; + } + await writeClaudeSettings({ path: opts.path, settings: opts.settings.value }); + return { status: "removed" }; +} + +/** + * Settings with no Hack primer entry are a missing integration; settings + * with some entries are Hack-owned and are completed on refresh. Other + * settings and hooks are kept. + */ +async function syncClaudeHooks(opts: { + readonly context: ScopeContext; + readonly path: string; + readonly value: Record; +}): Promise { + const { context, path, value } = opts; + const coverage = inspectClaudePrimerHooks({ settings: value }); + if (coverage === "complete") { + return { status: "unchanged" }; + } + if (coverage === "none") { + const planned = missingOutcome(context); + if (planned) { + return planned; + } + ensureClaudePrimerHooks({ settings: value }); + await writeClaudeSettings({ path, settings: value }); + return { status: "installed" }; + } + if (context.action === "check") { + return { status: "refreshed" }; + } + ensureClaudePrimerHooks({ settings: value }); + const changed = await writeClaudeSettings({ path, settings: value }); + return { status: changed ? "refreshed" : "unchanged" }; +} + +async function runMcpTarget(opts: { + readonly context: ScopeContext; + readonly target: SetupSyncTarget; + readonly client: McpTarget; +}): Promise { + const { context, target, client } = opts; + const scope = context.scope === "global" ? "user" : "project"; + const location = + context.scope === "global" + ? { home: context.root } + : { projectRoot: context.root }; + const outcome = await (async (): Promise< + Outcome & { readonly path?: string } + > => { + try { + if (context.action === "remove") { + const [removed] = await removeMcpConfig({ + targets: [client], + scope, + ...location, + }); + return mapMcpResult(removed); + } + const [checked] = await checkMcpConfig({ + targets: [client], + scope, + ...location, + }); + if (checked?.status !== "missing") { + return mapMcpResult(checked); + } + const planned = missingOutcome(context); + if (planned) { + return { ...planned, path: checked.path }; + } + // Ordinary install only adds a missing entry; it never replaces one. + const [installed] = await installMcpConfig({ + targets: [client], + scope, + ...location, + }); + if (installed?.status === "error") { + return mapMcpResult(installed); + } + return { status: "installed", path: installed?.path }; + } catch { + return { + status: "error", + message: "MCP configuration could not be read; left unchanged.", + }; + } + })(); + const { path, ...rest } = outcome; + return withTarget({ context, target, path: path ?? "", outcome: rest }); +} + +function mapMcpResult( + result: + | { + readonly status: string; + readonly path?: string; + readonly message?: string; + } + | undefined +): Outcome & { readonly path?: string } { + if (!result) { + return { status: "error", message: "MCP configuration was not checked." }; + } + const path = result.path; + switch (result.status) { + case "present": + return { status: "unchanged", path }; + case "removed": + return { status: "removed", path }; + case "missing": + case "noop": + return { status: "missing", path }; + default: + return { + status: "error", + path, + message: "MCP configuration is invalid or unreadable; left unchanged.", + }; + } +} + +async function runAgentDocTarget(opts: { + readonly context: ScopeContext; + readonly target: SetupSyncTarget; + readonly doc: "agents" | "claude"; +}): Promise { + const { context, target, doc } = opts; + const outcome = await (async (): Promise< + Outcome & { readonly path: string } + > => { + if (context.action === "remove") { + const [removed] = await removeAgentDocs({ + projectRoot: context.root, + targets: [doc], + }); + return mapAgentDocWrite({ + action: "remove", + result: removed, + root: context.root, + doc, + }); + } + const [checked] = await checkAgentDocs({ + projectRoot: context.root, + targets: [doc], + }); + const path = checked?.path ?? resolve(context.root, docFilename(doc)); + if (!checked || checked.status === "error") { + if (checked?.unmanaged) { + // A user-owned doc without a Hack section: the section is missing. + return await installAgentDocOrSkip({ context, doc, path }); + } + if (checked?.malformed) { + return { ...preservedOutcome("customized-content"), path }; + } + return { + status: "error", + path, + message: "Agent doc could not be read; left unchanged.", + }; + } + if (checked.status === "missing") { + return await installAgentDocOrSkip({ context, doc, path }); + } + if (checked.status === "present") { + return { status: "unchanged", path }; + } + if (context.action === "check") { + return { status: "refreshed", path }; + } + const [written] = await upsertAgentDocs({ + projectRoot: context.root, + targets: [doc], + }); + return mapAgentDocWrite({ + action: "sync", + result: written, + root: context.root, + doc, + }); + })(); + const { path, ...rest } = outcome; + return withTarget({ context, target, path, outcome: rest }); +} + +async function installAgentDocOrSkip(opts: { + readonly context: ScopeContext; + readonly doc: "agents" | "claude"; + readonly path: string; +}): Promise { + const planned = missingOutcome(opts.context); + if (planned) { + return { ...planned, path: opts.path }; + } + const [written] = await upsertAgentDocs({ + projectRoot: opts.context.root, + targets: [opts.doc], + }); + if (written?.status === "error") { + return { + status: "error", + path: opts.path, + message: "Agent doc could not be written; left unchanged.", + }; + } + return { status: "installed", path: opts.path }; +} + +function mapAgentDocWrite(opts: { + readonly action: "sync" | "remove"; + readonly result: + | { readonly status: string; readonly path: string } + | undefined; + readonly root: string; + readonly doc: "agents" | "claude"; +}): Outcome & { readonly path: string } { + const path = opts.result?.path ?? resolve(opts.root, docFilename(opts.doc)); + switch (opts.result?.status) { + case "removed": + return { status: "removed", path }; + case "noop": + // Remove found no Hack section; refresh found the section current. + return { + status: opts.action === "remove" ? "missing" : "unchanged", + path, + }; + case "updated": + case "created": + return { status: "refreshed", path }; + default: + return { + status: "error", + path, + message: "Agent doc could not be updated; left unchanged.", + }; + } +} + +function docFilename(doc: "agents" | "claude"): string { + return doc === "agents" ? "AGENTS.md" : "CLAUDE.md"; +} + +/** + * Retired Hack artifacts are reported only when present. `sync` and `remove` + * delete recognized ones; `check` reports them as `removed` (would remove). + */ +async function runRetiredArtifacts(opts: { + readonly context: ScopeContext; +}): Promise { + const { context } = opts; + const checkOnly = context.action === "check"; + let results: LegacyAgentArtifactResult[]; + if (context.scope === "project") { + results = checkOnly + ? await checkLegacyProjectAgentArtifacts({ projectRoot: context.root }) + : await removeLegacyProjectAgentArtifacts({ projectRoot: context.root }); + } else { + results = checkOnly + ? await checkLegacyUserAgentArtifacts({ home: context.root }) + : await removeLegacyUserAgentArtifacts({ home: context.root }); + } + return results + .filter((result) => result.status !== "absent") + .map((result) => + withTarget({ + context, + target: "retired-artifact", + path: result.path, + outcome: + result.status === "error" + ? { + status: "error", + message: + "Retired artifact path holds unrecognized or malformed content; left unchanged.", + } + : { status: "removed" }, + }) + ); +} diff --git a/src/commands/setup.ts b/src/commands/setup.ts index f8e1ab11..2711f94f 100644 --- a/src/commands/setup.ts +++ b/src/commands/setup.ts @@ -22,16 +22,14 @@ import { removeCursorRules, } from "../agents/cursor.ts"; import { - checkLegacyProjectAgentArtifacts, - checkLegacyUserAgentArtifacts, - removeLegacyProjectAgentArtifacts, - removeLegacyUserAgentArtifacts, -} from "../agents/legacy-artifacts.ts"; -import { - checkSharedHackSkill, - installSharedHackSkill, - removeSharedHackSkill, -} from "../agents/shared-skill.ts"; + isSetupSyncFailure, + runSetupSyncScope, + type SetupSyncAction, + type SetupSyncStatus, + type SetupSyncTarget, + type SetupSyncTargetResult, + setupSyncExitCode, +} from "../agents/setup-sync.ts"; import type { CliContext, CommandArgs } from "../cli/command.ts"; import { CliUsageError, @@ -39,7 +37,8 @@ import { defineOption, withHandler, } from "../cli/command.ts"; -import { optPath } from "../cli/options.ts"; +import { optJson, optPath } from "../cli/options.ts"; +import { emitCliResult, okResult } from "../lib/cli-result.ts"; import { pathExists, readTextFile, writeTextFile } from "../lib/fs.ts"; import { canPrompt } from "../lib/interactivity.ts"; import { findRepoRootForInit } from "../lib/project.ts"; @@ -115,6 +114,14 @@ const optCodex = defineOption({ description: "Target Codex integration", } as const); +const optInstallMissing = defineOption({ + name: "installMissing", + type: "boolean", + long: "--install-missing", + description: + "Also create missing integrations in the selected scope (default: refresh only installed Hack-owned files)", +} as const); + const optAgentsMd = defineOption({ name: "agentsMd", type: "boolean", @@ -145,8 +152,10 @@ const setupSyncOptions = [ optPath, optGlobal, optAllScopes, + optInstallMissing, optCheck, optRemove, + optJson, ] as const; const setupMcpOptions = [ optPath, @@ -167,12 +176,6 @@ type SetupAgentsArgs = CommandArgs; type SetupSyncArgs = CommandArgs; type SetupMcpArgs = CommandArgs; -type SetupMultiLogResult = { - readonly status: string; - readonly path?: string; - readonly message?: string; -}; - const tmuxSpec = defineCommand({ name: "tmux", summary: "Install the recommended tmux binding for hack workspaces", @@ -220,7 +223,7 @@ const agentsSpec = defineCommand({ const syncSpec = defineCommand({ name: "sync", - summary: "Refresh project/global agent guidance", + summary: "Refresh installed Hack-owned agent integrations", group: "Agents", options: setupSyncOptions, positionals: [], @@ -656,92 +659,157 @@ async function handleSetupAgents({ }); } -type SetupSyncAction = "install" | "check" | "remove"; - -type SetupSyncGroup = { - readonly label: string; - readonly results: readonly SetupMultiLogResult[]; -}; - type SetupSyncScopeResult = { readonly exitCode: number; readonly item: DisplayStatusItem; }; +const SETUP_SYNC_TARGET_LABELS: Readonly> = { + "cursor-rules": "Cursor rules", + "claude-hooks": "Claude hooks", + "claude-hack-init-skill": "Claude hack-init skill", + "codex-hack-cli-skill": "Codex hack-cli skill", + "codex-hack-init-skill": "Codex hack-init skill", + "shared-hack-cli-skill": "Shared Hack skill", + "cursor-mcp": "Cursor MCP config", + "claude-mcp": "Claude MCP config", + "codex-mcp": "Codex MCP config", + "agents-md": "AGENTS.md", + "claude-md": "CLAUDE.md", + "retired-artifact": "Retired agent artifact", + "scope-root": "Scope", +}; + +/** How `--check` words a status that `sync` would produce. */ +const SETUP_SYNC_CHECK_WORDS: Readonly< + Partial> +> = { + refreshed: "stale", + installed: "not installed (would install)", + removed: "retired (would remove)", +}; + +/** + * Collapse one scope's per-target results into a compact status row. Paths + * and reasons are expanded only for errors, preserved files, and (with + * `--check`) targets that sync would change. + */ export function buildSetupSyncScopeResult(input: { readonly action: SetupSyncAction; readonly scope: "Project" | "Global"; - readonly groups: readonly SetupSyncGroup[]; + readonly results: readonly SetupSyncTargetResult[]; }): SetupSyncScopeResult { - const entries = input.groups.flatMap((group) => - group.results.map((result) => ({ ...result, label: group.label })) + const { action, results } = input; + const failures = results.filter((result) => + isSetupSyncFailure({ action, status: result.status }) ); - const failures = entries.filter((entry) => { - if (entry.status === "error") { - return true; - } - return ( - input.action === "check" && - ["missing", "stale", "deprecated"].includes(entry.status) - ); - }); - const errorCount = failures.filter( - (entry) => entry.status === "error" - ).length; - // Preserved files are reported but do not fail an install or remove. - const preserved = entries.filter((entry) => entry.status === "preserved"); + const errorCount = countSetupSyncStatus({ results, status: "error" }); + const preserved = countSetupSyncStatus({ results, status: "preserved" }); + let status: DisplayStatusItem["status"] = "ok"; if (errorCount > 0) { status = "error"; - } else if (failures.length > 0 || preserved.length > 0) { + } else if (failures.length > 0 || preserved > 0) { status = "warn"; } - const meta = (() => { - if (input.action === "check") { - return failures.length === 0 - ? `${entries.length} current` - : `${entries.length - failures.length}/${entries.length} current`; - } - const preservedSuffix = - preserved.length > 0 ? `, ${preserved.length} preserved` : ""; - if (input.action === "remove") { - const removed = entries.filter( - (entry) => entry.status === "removed" - ).length; - return `${removed} removed${preservedSuffix}`; - } - const changed = entries.filter((entry) => - ["created", "updated", "removed"].includes(entry.status) - ).length; - return changed === 0 - ? `already current${preservedSuffix}` - : `${changed} updated${preservedSuffix}`; - })(); - const detail = [...failures, ...preserved] - .map((entry) => { - const location = entry.path ? ` at ${entry.path}` : ""; - if (entry.status === "preserved") { - const reason = entry.message ? ` ${entry.message}` : ""; - return `${entry.label}: preserved${location}.${reason}`; - } - if (entry.message) { - return `${entry.label}: ${entry.message}`; - } - return `${entry.label}: ${entry.status}${location}`; - }) - .join("\n"); + + const detailLines = results + .filter( + (result) => + result.status === "error" || + result.status === "preserved" || + failures.includes(result) + ) + .map((result) => describeSetupSyncResult({ action, result })); + if ( + action === "sync" && + countSetupSyncStatus({ results, status: "missing" }) > 0 + ) { + detailLines.push( + "Missing integrations were skipped; add --install-missing to create them." + ); + } return { - exitCode: failures.length > 0 ? 1 : 0, + exitCode: setupSyncExitCode({ action, results }), item: { label: input.scope, status, - meta, - detail: detail.length > 0 ? detail : undefined, + meta: buildSetupSyncMeta({ action, results, failures }), + detail: detailLines.length > 0 ? detailLines.join("\n") : undefined, }, }; } +function countSetupSyncStatus(opts: { + readonly results: readonly SetupSyncTargetResult[]; + readonly status: SetupSyncStatus; +}): number { + return opts.results.filter((result) => result.status === opts.status).length; +} + +function buildSetupSyncMeta(opts: { + readonly action: SetupSyncAction; + readonly results: readonly SetupSyncTargetResult[]; + readonly failures: readonly SetupSyncTargetResult[]; +}): string { + const count = (status: SetupSyncStatus): number => + countSetupSyncStatus({ results: opts.results, status }); + const preserved = count("preserved"); + const preservedSuffix = preserved > 0 ? `, ${preserved} preserved` : ""; + const missing = count("missing"); + + if (opts.action === "remove") { + return `${count("removed")} removed${preservedSuffix}`; + } + if (opts.action === "check") { + const considered = opts.results.length - missing; + const current = + opts.failures.length === 0 + ? `${count("unchanged")} current` + : `${count("unchanged")}/${considered} current`; + return missing > 0 ? `${current}, ${missing} not installed` : current; + } + + const changed = [ + { status: "refreshed", word: "refreshed" }, + { status: "installed", word: "installed" }, + { status: "removed", word: "retired removed" }, + ] as const; + const parts = changed + .filter((entry) => count(entry.status) > 0) + .map((entry) => `${count(entry.status)} ${entry.word}`); + let head = parts.join(", "); + if (parts.length === 0) { + head = count("unchanged") > 0 ? "already current" : "nothing to refresh"; + } + const missingSuffix = missing > 0 ? `, ${missing} missing (skipped)` : ""; + return `${head}${preservedSuffix}${missingSuffix}`; +} + +function describeSetupSyncResult(opts: { + readonly action: SetupSyncAction; + readonly result: SetupSyncTargetResult; +}): string { + const { action, result } = opts; + const label = SETUP_SYNC_TARGET_LABELS[result.target]; + const location = result.path ? ` at ${result.path}` : ""; + if (result.status === "preserved") { + const reason = result.message ? ` ${result.message}` : ""; + return `${label}: preserved${location}.${reason}`; + } + if (result.status === "error") { + return result.message + ? `${label}: ${result.message}` + : `${label}: error${location}`; + } + const word = + action === "check" + ? (SETUP_SYNC_CHECK_WORDS[result.status] ?? result.status) + : result.status; + return `${label}: ${word}${location}`; +} + async function handleSetupSync({ ctx, args, @@ -749,174 +817,108 @@ async function handleSetupSync({ readonly ctx: CliContext; readonly args: SetupSyncArgs; }): Promise { - const action = resolveAction(args.options); + const resolved = resolveAction(args.options); + const installMissing = args.options.installMissing === true; + if (resolved === "remove" && installMissing) { + throw new CliUsageError( + "Use either --install-missing or --remove, not both." + ); + } + const action: SetupSyncAction = resolved === "install" ? "sync" : resolved; const scopes = resolveSyncScopes({ global: args.options.global === true, allScopes: args.options.allScopes === true, }); - const includesProject = scopes.includes("project"); - const includesUser = scopes.includes("user"); - const projectRoot = includesProject - ? await resolveSetupRoot({ ctx, pathOpt: args.options.path }) - : undefined; - const scopeResults: SetupSyncScopeResult[] = []; - if (includesProject && projectRoot) { + const results: SetupSyncTargetResult[] = []; + const scopeResults: SetupSyncScopeResult[] = []; + if (scopes.includes("project")) { + const projectRoot = await resolveSetupRoot({ + ctx, + pathOpt: args.options.path, + }); + const projectResults = await runSetupSyncScope({ + action, + scope: "project", + root: projectRoot, + installMissing, + }); + results.push(...projectResults); scopeResults.push( - await runProjectScopeSync({ + buildSetupSyncScopeResult({ action, - projectRoot, + scope: "Project", + results: projectResults, + }) + ); + } + if (scopes.includes("user")) { + const globalResults = await runSetupSyncScope({ + action, + scope: "global", + root: process.env.HOME ?? "", + installMissing, + }); + results.push(...globalResults); + scopeResults.push( + buildSetupSyncScopeResult({ + action, + scope: "Global", + results: globalResults, }) ); } - if (includesUser) { - scopeResults.push(await runUserScopeSync({ action })); + const exitCode = setupSyncExitCode({ action, results }); + if (args.options.json === true) { + emitCliResult({ + result: okResult({ + data: buildSetupSyncJsonData({ action, installMissing, results }), + }), + }); + return exitCode; } const titles: Readonly> = { check: "Agent integrations", - install: "Agent integrations updated", + sync: "Agent integrations refreshed", remove: "Agent integrations removed", }; await display.statusList({ title: titles[action], items: scopeResults.map((result) => result.item), }); - return Math.max(0, ...scopeResults.map((result) => result.exitCode)); -} - -/** Run one explicit sync action across all project-scope integrations. */ -async function runProjectScopeSync(opts: { - readonly action: SetupSyncAction; - readonly projectRoot: string; -}): Promise { - const { action, projectRoot } = opts; - let cursorResult: Awaited>; - let claudeResult: Awaited>; - let codexResult: Awaited>; - let legacyResults: SetupMultiLogResult[]; - let mcpResults: SetupMultiLogResult[]; - let docsResults: SetupMultiLogResult[]; - - if (action === "check") { - cursorResult = await checkCursorRules({ scope: "project", projectRoot }); - claudeResult = await checkClaudeHooks({ scope: "project", projectRoot }); - codexResult = await checkCodexSkill({ scope: "project", projectRoot }); - legacyResults = await checkLegacyProjectAgentArtifacts({ projectRoot }); - mcpResults = await checkMcpConfig({ - scope: "project", - targets: ["cursor", "claude", "codex"], - projectRoot, - }); - docsResults = await checkAgentDocs({ - projectRoot, - targets: ["agents", "claude"], - }); - } else if (action === "remove") { - cursorResult = await removeCursorRules({ scope: "project", projectRoot }); - claudeResult = await removeClaudeHooks({ scope: "project", projectRoot }); - codexResult = await removeCodexSkill({ scope: "project", projectRoot }); - legacyResults = await removeLegacyProjectAgentArtifacts({ projectRoot }); - mcpResults = await removeMcpConfig({ - scope: "project", - targets: ["cursor", "claude", "codex"], - projectRoot, - }); - docsResults = await removeAgentDocs({ - projectRoot, - targets: ["agents", "claude"], - }); - } else { - cursorResult = await installCursorRules({ scope: "project", projectRoot }); - claudeResult = await installClaudeHooks({ scope: "project", projectRoot }); - codexResult = await installCodexSkill({ scope: "project", projectRoot }); - legacyResults = await removeLegacyProjectAgentArtifacts({ projectRoot }); - mcpResults = await installMcpConfig({ - scope: "project", - targets: ["cursor", "claude", "codex"], - projectRoot, - }); - docsResults = await upsertAgentDocs({ - projectRoot, - targets: ["agents", "claude"], - }); - } - - return buildSetupSyncScopeResult({ - action, - scope: "Project", - groups: [ - { label: "Cursor", results: [cursorResult] }, - { label: "Claude", results: [claudeResult] }, - { label: "Codex", results: [codexResult] }, - { label: "Retired agent artifacts", results: legacyResults }, - { label: "MCP config", results: mcpResults }, - { label: "Agent docs", results: docsResults }, - ], - }); + return exitCode; } /** - * Run one sync action across all global (user) scope integrations and log - * results. Shared `~/.ai/skills` guidance is managed alongside client-specific - * integrations. Known retired Hack-owned skills are cleaned up safely. + * Machine-readable `hack setup sync --json` payload: one entry per target. + * With `--check`, each status is what `sync` would report (`dryRun: true`). + * The envelope is `ok` whenever the action ran; the exit code carries + * failure, as with `hack doctor --json`. */ -async function runUserScopeSync(opts: { +export function buildSetupSyncJsonData(opts: { readonly action: SetupSyncAction; -}): Promise { - const { action } = opts; - let cursorResult: Awaited>; - let claudeResult: Awaited>; - let codexResult: Awaited>; - let sharedSkillResult: SetupMultiLogResult & { readonly path: string }; - let legacyResults: SetupMultiLogResult[]; - let mcpResults: SetupMultiLogResult[]; - - if (action === "check") { - cursorResult = await checkCursorRules({ scope: "user" }); - claudeResult = await checkClaudeHooks({ scope: "user" }); - codexResult = await checkCodexSkill({ scope: "user" }); - sharedSkillResult = await checkSharedHackSkill(); - legacyResults = await checkLegacyUserAgentArtifacts(); - mcpResults = await checkMcpConfig({ - scope: "user", - targets: ["cursor", "claude", "codex"], - }); - } else if (action === "remove") { - cursorResult = await removeCursorRules({ scope: "user" }); - claudeResult = await removeClaudeHooks({ scope: "user" }); - codexResult = await removeCodexSkill({ scope: "user" }); - sharedSkillResult = await removeSharedHackSkill(); - legacyResults = await removeLegacyUserAgentArtifacts(); - mcpResults = await removeMcpConfig({ - scope: "user", - targets: ["cursor", "claude", "codex"], - }); - } else { - cursorResult = await installCursorRules({ scope: "user" }); - claudeResult = await installClaudeHooks({ scope: "user" }); - codexResult = await installCodexSkill({ scope: "user" }); - sharedSkillResult = await installSharedHackSkill(); - legacyResults = await removeLegacyUserAgentArtifacts(); - mcpResults = await installMcpConfig({ - scope: "user", - targets: ["cursor", "claude", "codex"], - }); + readonly installMissing: boolean; + readonly results: readonly SetupSyncTargetResult[]; +}): { + readonly action: SetupSyncAction; + readonly dryRun: boolean; + readonly installMissing: boolean; + readonly counts: Readonly>>; + readonly targets: readonly SetupSyncTargetResult[]; +} { + const counts: Partial> = {}; + for (const result of opts.results) { + counts[result.status] = (counts[result.status] ?? 0) + 1; } - - return buildSetupSyncScopeResult({ - action, - scope: "Global", - groups: [ - { label: "Cursor", results: [cursorResult] }, - { label: "Claude", results: [claudeResult] }, - { label: "Codex", results: [codexResult] }, - { label: "Shared Hack skill", results: [sharedSkillResult] }, - { label: "Retired agent artifacts", results: legacyResults }, - { label: "MCP config", results: mcpResults }, - ], - }); + return { + action: opts.action, + dryRun: opts.action === "check", + installMissing: opts.installMissing, + counts, + targets: opts.results, + }; } async function handleSetupMcp({ diff --git a/src/mcp/agent-docs.ts b/src/mcp/agent-docs.ts index ee529022..8664b42f 100644 --- a/src/mcp/agent-docs.ts +++ b/src/mcp/agent-docs.ts @@ -31,6 +31,11 @@ export type AgentDocCheckResult = { * marker is a corrupted managed section and is never `unmanaged`. */ readonly unmanaged?: true; + /** + * Set only when the Hack markers are present but do not form one + * start-before-end pair, so the managed section cannot be located safely. + */ + readonly malformed?: true; }; export type AgentDocRemoveResult = { @@ -171,6 +176,7 @@ async function checkAgentDocTarget(opts: { path, message: "Malformed hack agent-docs markers: only one of the start/end markers is present.", + malformed: true, }; } @@ -181,6 +187,7 @@ async function checkAgentDocTarget(opts: { status: "error", path, message: "Malformed hack agent-docs markers.", + malformed: true, }; } if (normalizeInstructionText({ text: region }) !== opts.expected) { diff --git a/src/mcp/install.ts b/src/mcp/install.ts index 27715812..5ee8edb0 100644 --- a/src/mcp/install.ts +++ b/src/mcp/install.ts @@ -84,6 +84,8 @@ export async function installMcpConfig(opts: { readonly targets: readonly McpTarget[]; readonly scope: McpInstallScope; readonly projectRoot?: string; + /** User-scope root; defaults to `HOME`. */ + readonly home?: string; readonly bundle?: McpBundleSelection; }): Promise { const launch = opts.bundle @@ -99,6 +101,7 @@ export async function installMcpConfig(opts: { target, scope: opts.scope, projectRoot: opts.projectRoot, + home: opts.home, }); if (!resolved.ok) { results.push({ @@ -132,6 +135,8 @@ export async function checkMcpConfig(opts: { readonly targets: readonly McpTarget[]; readonly scope: McpInstallScope; readonly projectRoot?: string; + /** User-scope root; defaults to `HOME`. */ + readonly home?: string; }): Promise { const results: McpCheckResult[] = []; @@ -140,6 +145,7 @@ export async function checkMcpConfig(opts: { target, scope: opts.scope, projectRoot: opts.projectRoot, + home: opts.home, }); if (!resolved.ok) { results.push({ @@ -211,6 +217,8 @@ export async function removeMcpConfig(opts: { readonly targets: readonly McpTarget[]; readonly scope: McpInstallScope; readonly projectRoot?: string; + /** User-scope root; defaults to `HOME`. */ + readonly home?: string; }): Promise { const results: McpRemoveResult[] = []; @@ -219,6 +227,7 @@ export async function removeMcpConfig(opts: { target, scope: opts.scope, projectRoot: opts.projectRoot, + home: opts.home, }); if (!resolved.ok) { results.push({ @@ -471,6 +480,7 @@ function resolveConfigPath(opts: { readonly target: McpTarget; readonly scope: McpInstallScope; readonly projectRoot?: string; + readonly home?: string; }): | { readonly ok: true; readonly path: string } | { readonly ok: false; readonly message: string } { @@ -481,7 +491,7 @@ function resolveConfigPath(opts: { }; } - const home = resolveHomeDir(); + const home = opts.home === undefined ? resolveHomeDir() : opts.home.trim(); if (opts.scope === "user" && !home) { return { ok: false, diff --git a/tests/e2e/README.md b/tests/e2e/README.md index 6e20ee84..2c72d85b 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -468,7 +468,9 @@ sentinel client config (`~/.claude/settings.json`, `~/.claude.json`, `.claude/settings.local.json` and `.mcp.json` into two fixtures. It then runs `init --auto`, `init --auto --with codex`, `agent prime`, `agent prime --check`, `agent onboard`, `agent patterns`, `doctor --json`, -`doctor --fix --no-interactive` and `setup sync --check` twice each. After +`doctor --fix --no-interactive`, `setup sync --check`, and refresh-only +`setup sync --all-scopes` twice each (`setup sync --check` exits 1 because the +customized files are reported as preserved). After every run it requires byte-identical hashes for those trees, no new entries under them, and `git status` changes only under `.hack/`. Each command must also exit with its expected code, so a usage error cannot pass as "no write". diff --git a/tests/e2e/scenarios/agent-docs-sync.ts b/tests/e2e/scenarios/agent-docs-sync.ts index afdf47f6..7d692aea 100644 --- a/tests/e2e/scenarios/agent-docs-sync.ts +++ b/tests/e2e/scenarios/agent-docs-sync.ts @@ -147,15 +147,43 @@ export const agentDocsSyncScenario: Scenario = { message: "check after repair should report clean (exit 0)", }); - const fullSync = await ctx.cli({ + const globalSkillPath = join( + ctx.hackHome, + ".codex", + "skills", + "hack-cli", + "SKILL.md" + ); + // Refresh-only by default (HACK-1132): sync must not create missing + // integrations in either scope. + const refreshOnly = await ctx.cli({ args: ["setup", "sync", "--all-scopes"], cwd: fixture.root, env: isolatedUserEnv, }); + expectExit({ + result: refreshOnly, + codes: [0], + message: "refresh-only hack setup sync should succeed", + }); + expect({ + that: !( + (await Bun.file(globalSkillPath).exists()) || + (await Bun.file(join(fixture.root, "CLAUDE.md")).exists()) + ), + message: "refresh-only hack setup sync must not create missing targets", + result: refreshOnly, + }); + + const fullSync = await ctx.cli({ + args: ["setup", "sync", "--all-scopes", "--install-missing"], + cwd: fixture.root, + env: isolatedUserEnv, + }); expectExit({ result: fullSync, codes: [0], - message: "hack setup sync (project scope) should succeed", + message: "hack setup sync --install-missing should succeed", }); const fullSyncCheck = await ctx.cli({ args: ["setup", "sync", "--all-scopes", "--check"], @@ -187,13 +215,6 @@ export const agentDocsSyncScenario: Scenario = { "outside-project inventory must inspect installed global integrations", result: globalPrime, }); - const globalSkillPath = join( - ctx.hackHome, - ".codex", - "skills", - "hack-cli", - "SKILL.md" - ); const globalSkill = await Bun.file(globalSkillPath).text(); await Bun.write(globalSkillPath, `${globalSkill}\nSTALE-GLOBAL-PROBE\n`); const staleGlobalPrime = await ctx.cli({ diff --git a/tests/e2e/scenarios/agent-surface-freeze.ts b/tests/e2e/scenarios/agent-surface-freeze.ts index e9ae98c9..06057ed5 100644 --- a/tests/e2e/scenarios/agent-surface-freeze.ts +++ b/tests/e2e/scenarios/agent-surface-freeze.ts @@ -42,8 +42,10 @@ type FreezeCommand = { * onboarding prompt for the existing setup (exit 0). `doctor` exits 1 for the * unreachable Docker and missing global install, and `--fix` runs only its * Docker-free remediations (see `freezeEnv`). `setup sync --check` exits 1 - * because the customized guidance differs from Hack's render; it reports the - * drift and must not repair it. + * because the customized guidance is reported as preserved; it must not + * repair it. Refresh-only `setup sync --all-scopes` (HACK-1132) exits 0: every + * seeded file is customized or lacks a Hack section, so nothing is refreshed + * and nothing missing is created. */ const FREEZE_COMMANDS: readonly FreezeCommand[] = [ { @@ -106,6 +108,12 @@ const FREEZE_COMMANDS: readonly FreezeCommand[] = [ args: ["setup", "sync", "--check"], codes: [[1], [1]], }, + { + label: "setup sync --all-scopes", + fixture: "project", + args: ["setup", "sync", "--all-scopes"], + codes: [[0], [0]], + }, ]; type FreezeState = { diff --git a/tests/setup-sync.test.ts b/tests/setup-sync.test.ts new file mode 100644 index 00000000..d96e224f --- /dev/null +++ b/tests/setup-sync.test.ts @@ -0,0 +1,588 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, readdir, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; + +import { checkClaudeHooks } from "../src/agents/claude.ts"; +import { + checkCodexSkill, + renderCodexSkill, +} from "../src/agents/codex-skill.ts"; +import { checkCursorRules, renderCursorRules } from "../src/agents/cursor.ts"; +import { renderHackInitSkill } from "../src/agents/hack-init-skill.ts"; +import { + runSetupSyncScope, + type SetupSyncAction, + type SetupSyncScope, + type SetupSyncTarget, + type SetupSyncTargetResult, + setupSyncExitCode, +} from "../src/agents/setup-sync.ts"; +import { checkSharedHackSkill } from "../src/agents/shared-skill.ts"; +import { buildSetupSyncJsonData } from "../src/commands/setup.ts"; +import { + checkAgentDocs, + renderAgentDocsSnippet, +} from "../src/mcp/agent-docs.ts"; +import { checkMcpConfig } from "../src/mcp/install.ts"; + +/** + * `hack setup sync` engine tests (HACK-1132). Every call passes a temp + * project root or temp home explicitly. `HOME` is pointed at an empty decoy + * directory for each test and asserted untouched afterwards, so a fallback + * to the process `HOME` fails the test instead of writing real client config. + */ + +/** The `/hack-init` render shipped from v3.1.0 through v4.0.3. */ +const HACK_INIT_V4_0_3 = [ + "---", + "name: hack-init", + "description: >", + " Stand up or adopt hack in a project (agent-assisted onboarding).", + " Trigger when asked to set a repo up with hack, migrate a project onto", + " hack, or finish/repair a partial hack setup.", + "---", + "", + "# hack init (agent-assisted onboarding)", + "", + "The full onboarding prompt is generated by the hack CLI — do not improvise the steps.", + "", + "1. Fetch the canonical onboarding prompt:", + " - With shell access: run `hack agent onboard` (works for new and existing projects).", + " - Without shell access: fetch the `hack-init` MCP prompt from the hack MCP server.", + "2. Follow the printed prompt phase by phase (inventory, setup, platform nuances, running things, verification).", + "3. Finish only when `hack doctor` is clean and every routable service responds.", + "", +].join("\n"); + +let tempRoot: string | null = null; +let decoyHome = ""; +const originalHome = process.env.HOME; + +beforeEach(async () => { + tempRoot = await mkdtemp(join(tmpdir(), "hack-setup-sync-")); + decoyHome = join(tempRoot, "decoy-home"); + await mkdir(decoyHome, { recursive: true }); + process.env.HOME = decoyHome; +}); + +afterEach(async () => { + try { + expect( + await listFiles(decoyHome), + "setup sync must not resolve paths from the process HOME" + ).toEqual([]); + } finally { + if (originalHome === undefined) { + Reflect.deleteProperty(process.env, "HOME"); + } else { + process.env.HOME = originalHome; + } + if (tempRoot) { + await rm(tempRoot, { recursive: true, force: true }); + tempRoot = null; + } + } +}); + +async function makeRoot(name: string): Promise { + if (!tempRoot) { + throw new Error("temp root not initialized"); + } + const root = join(tempRoot, name); + await mkdir(root, { recursive: true }); + return root; +} + +async function writeFile(path: string, content: string): Promise { + await mkdir(dirname(path), { recursive: true }); + await Bun.write(path, content); +} + +async function listFiles(root: string): Promise { + const entries = await readdir(root, { recursive: true, withFileTypes: true }); + return entries + .filter((entry) => entry.isFile()) + .map((entry) => join(entry.parentPath, entry.name).slice(root.length + 1)) + .sort(); +} + +async function snapshot(root: string): Promise> { + const files = await listFiles(root); + const contents = new Map(); + for (const file of files) { + contents.set(file, await Bun.file(join(root, file)).text()); + } + return contents; +} + +async function run(opts: { + readonly action: SetupSyncAction; + readonly scope: SetupSyncScope; + readonly root: string; + readonly installMissing?: boolean; +}): Promise { + return await runSetupSyncScope({ + action: opts.action, + scope: opts.scope, + root: opts.root, + installMissing: opts.installMissing ?? false, + }); +} + +function byTarget( + results: readonly SetupSyncTargetResult[], + target: SetupSyncTarget +): SetupSyncTargetResult { + const found = results.find((result) => result.target === target); + if (!found) { + throw new Error(`no result for ${target}`); + } + return found; +} + +function statuses( + results: readonly SetupSyncTargetResult[] +): Record { + return Object.fromEntries( + results.map((result) => [result.target, result.status]) + ); +} + +const PROJECT_TARGETS: readonly SetupSyncTarget[] = [ + "cursor-rules", + "claude-hack-init-skill", + "codex-hack-cli-skill", + "codex-hack-init-skill", + "claude-hooks", + "cursor-mcp", + "claude-mcp", + "codex-mcp", + "agents-md", + "claude-md", +]; + +const GLOBAL_TARGETS: readonly SetupSyncTarget[] = [ + "cursor-rules", + "claude-hack-init-skill", + "codex-hack-cli-skill", + "codex-hack-init-skill", + "shared-hack-cli-skill", + "claude-hooks", + "cursor-mcp", + "claude-mcp", + "codex-mcp", +]; + +test("default sync skips missing targets in both scopes and creates nothing", async () => { + const project = await makeRoot("project"); + const home = await makeRoot("home"); + + const projectResults = await run({ + action: "sync", + scope: "project", + root: project, + }); + const globalResults = await run({ + action: "sync", + scope: "global", + root: home, + }); + + expect(projectResults.map((result) => result.target)).toEqual([ + ...PROJECT_TARGETS, + ]); + expect(globalResults.map((result) => result.target)).toEqual([ + ...GLOBAL_TARGETS, + ]); + for (const result of [...projectResults, ...globalResults]) { + expect(result.status, result.target).toBe("missing"); + } + expect(await listFiles(project)).toEqual([]); + expect(await listFiles(home)).toEqual([]); + expect( + setupSyncExitCode({ + action: "sync", + results: [...projectResults, ...globalResults], + }) + ).toBe(0); +}); + +test("--install-missing creates missing targets that the client checks accept", async () => { + const project = await makeRoot("project"); + const home = await makeRoot("home"); + + const planned = await run({ + action: "check", + scope: "project", + root: project, + installMissing: true, + }); + expect(new Set(Object.values(statuses(planned)))).toEqual( + new Set(["installed"]) + ); + expect(setupSyncExitCode({ action: "check", results: planned })).toBe(1); + expect(await listFiles(project)).toEqual([]); + + const projectResults = await run({ + action: "sync", + scope: "project", + root: project, + installMissing: true, + }); + const globalResults = await run({ + action: "sync", + scope: "global", + root: home, + installMissing: true, + }); + for (const result of [...projectResults, ...globalResults]) { + expect(result.status, result.target).toBe("installed"); + expect(await Bun.file(result.path).exists(), result.target).toBe(true); + } + + // Path and content parity with the per-client checks. + expect( + (await checkCursorRules({ scope: "project", projectRoot: project })).status + ).toBe("noop"); + expect( + (await checkClaudeHooks({ scope: "project", projectRoot: project })).status + ).toBe("noop"); + expect( + (await checkCodexSkill({ scope: "project", projectRoot: project })).status + ).toBe("noop"); + for (const check of await checkMcpConfig({ + scope: "project", + projectRoot: project, + targets: ["cursor", "claude", "codex"], + })) { + expect(check.status, check.target).toBe("present"); + } + for (const check of await checkMcpConfig({ + scope: "user", + home, + targets: ["cursor", "claude", "codex"], + })) { + expect(check.status, check.target).toBe("present"); + } + for (const check of await checkAgentDocs({ + projectRoot: project, + targets: ["agents", "claude"], + })) { + expect(check.status, check.target).toBe("present"); + } + process.env.HOME = home; + try { + expect((await checkCursorRules({ scope: "user" })).status).toBe("noop"); + expect((await checkClaudeHooks({ scope: "user" })).status).toBe("noop"); + expect((await checkCodexSkill({ scope: "user" })).status).toBe("noop"); + expect((await checkSharedHackSkill()).status).toBe("noop"); + } finally { + process.env.HOME = decoyHome; + } + + const again = await run({ action: "sync", scope: "global", root: home }); + expect(new Set(Object.values(statuses(again)))).toEqual( + new Set(["unchanged"]) + ); + const checked = await run({ + action: "check", + scope: "project", + root: project, + }); + expect(setupSyncExitCode({ action: "check", results: checked })).toBe(0); +}); + +test("customized and unowned files are preserved byte-identical", async () => { + const project = await makeRoot("project"); + const rules = join(project, ".cursor/rules/hack.mdc"); + const skill = join(project, ".codex/skills/hack-cli/SKILL.md"); + const customizedRules = renderCursorRules().replace( + "# END HACK INTEGRATION", + "- Team rule: run checks before pushing.\n# END HACK INTEGRATION" + ); + const foreignSkill = "---\nname: team-skill\n---\nOur own skill.\n"; + await writeFile(rules, customizedRules); + await writeFile(skill, foreignSkill); + const before = await snapshot(project); + + for (const action of ["check", "sync"] as const) { + for (const installMissing of [false, true]) { + const results = await run({ + action, + scope: "project", + root: project, + installMissing, + }); + expect(byTarget(results, "cursor-rules")).toMatchObject({ + status: "preserved", + reason: "customized-content", + }); + expect(byTarget(results, "codex-hack-cli-skill")).toMatchObject({ + status: "preserved", + reason: "missing-ownership-marker", + }); + const message = byTarget(results, "cursor-rules").message ?? ""; + expect(message).toContain("--install-missing"); + expect(message).not.toContain("Team rule"); + if (action === "check") { + expect(setupSyncExitCode({ action, results })).toBe(1); + } + expect(await Bun.file(rules).text()).toBe(customizedRules); + expect(await Bun.file(skill).text()).toBe(foreignSkill); + } + } + // The refresh-only runs left every other path absent. + const after = await snapshot(project); + for (const [path, content] of before) { + expect(after.get(path)).toBe(content); + } +}); + +test("an old known Hack render is refreshed; check reports it without writing", async () => { + const project = await makeRoot("project"); + const initSkill = join(project, ".claude/skills/hack-init/SKILL.md"); + await writeFile(initSkill, HACK_INIT_V4_0_3); + + const checked = await run({ + action: "check", + scope: "project", + root: project, + }); + expect(byTarget(checked, "claude-hack-init-skill").status).toBe("refreshed"); + expect(byTarget(checked, "codex-hack-init-skill").status).toBe("missing"); + expect(setupSyncExitCode({ action: "check", results: checked })).toBe(1); + expect(await Bun.file(initSkill).text()).toBe(HACK_INIT_V4_0_3); + + const synced = await run({ action: "sync", scope: "project", root: project }); + expect(byTarget(synced, "claude-hack-init-skill").status).toBe("refreshed"); + expect(await Bun.file(initSkill).text()).toBe(renderHackInitSkill()); + expect(await listFiles(project)).toEqual([ + ".claude/skills/hack-init/SKILL.md", + ]); + + const recheck = await run({ + action: "check", + scope: "project", + root: project, + }); + expect(byTarget(recheck, "claude-hack-init-skill").status).toBe("unchanged"); + expect(setupSyncExitCode({ action: "check", results: recheck })).toBe(0); +}); + +test("a current render stamped by another CLI version is refreshed", async () => { + const home = await makeRoot("home"); + const shared = join(home, ".ai/skills/hack-cli/SKILL.md"); + const older = renderCodexSkill().replace( + /generated by hack CLI v\S+?;/, + "generated by hack CLI v0.0.1;" + ); + expect(older).not.toBe(renderCodexSkill()); + await writeFile(shared, older); + + const results = await run({ action: "sync", scope: "global", root: home }); + expect(byTarget(results, "shared-hack-cli-skill").status).toBe("refreshed"); + expect(byTarget(results, "codex-hack-cli-skill").status).toBe("missing"); + expect(await Bun.file(shared).text()).toBe(renderCodexSkill()); +}); + +test("Claude settings without Hack hooks are skipped; partial hooks are completed", async () => { + const project = await makeRoot("project"); + const settingsPath = join(project, ".claude/settings.local.json"); + const foreign = `${JSON.stringify({ permissions: { allow: ["Bash(true)"] } }, null, 2)}\n`; + await writeFile(settingsPath, foreign); + + const skipped = await run({ + action: "sync", + scope: "project", + root: project, + }); + expect(byTarget(skipped, "claude-hooks").status).toBe("missing"); + expect(await Bun.file(settingsPath).text()).toBe(foreign); + + const partial = { + permissions: { allow: ["Bash(true)"] }, + hooks: { + SessionStart: [ + { + matcher: "", + hooks: [{ type: "command", command: "hack agent prime" }], + }, + ], + }, + }; + await writeFile(settingsPath, `${JSON.stringify(partial, null, 2)}\n`); + const refreshed = await run({ + action: "sync", + scope: "project", + root: project, + }); + expect(byTarget(refreshed, "claude-hooks").status).toBe("refreshed"); + const written: unknown = JSON.parse(await Bun.file(settingsPath).text()); + expect(written).toMatchObject({ + permissions: { allow: ["Bash(true)"] }, + hooks: { SessionStart: partial.hooks.SessionStart }, + }); + expect( + (await checkClaudeHooks({ scope: "project", projectRoot: project })).status + ).not.toBe("missing"); + + await writeFile(settingsPath, "{ not json"); + const broken = await run({ action: "sync", scope: "project", root: project }); + expect(byTarget(broken, "claude-hooks")).toMatchObject({ + status: "error", + message: "Claude settings are not a JSON object; left unchanged.", + }); + expect(await Bun.file(settingsPath).text()).toBe("{ not json"); + expect(setupSyncExitCode({ action: "sync", results: broken })).toBe(1); +}); + +test("agent docs and MCP configs without a Hack section are skipped unless installing", async () => { + const project = await makeRoot("project"); + const agentsPath = join(project, "AGENTS.md"); + const mcpPath = join(project, ".mcp.json"); + const agents = "# Team rules\n\nUse pnpm.\n"; + const mcp = `${JSON.stringify({ mcpServers: { other: { command: "true" } } }, null, 2)}\n`; + await writeFile(agentsPath, agents); + await writeFile(mcpPath, mcp); + + const skipped = await run({ + action: "sync", + scope: "project", + root: project, + }); + expect(byTarget(skipped, "agents-md").status).toBe("missing"); + expect(byTarget(skipped, "claude-mcp").status).toBe("missing"); + expect(await Bun.file(agentsPath).text()).toBe(agents); + expect(await Bun.file(mcpPath).text()).toBe(mcp); + + const installed = await run({ + action: "sync", + scope: "project", + root: project, + installMissing: true, + }); + expect(byTarget(installed, "agents-md").status).toBe("installed"); + expect(byTarget(installed, "claude-mcp").status).toBe("installed"); + const agentsAfter = await Bun.file(agentsPath).text(); + expect(agentsAfter.startsWith(agents.trimEnd())).toBe(true); + expect(agentsAfter).toContain(renderAgentDocsSnippet().trimEnd()); + const mcpAfter: unknown = JSON.parse(await Bun.file(mcpPath).text()); + expect(mcpAfter).toMatchObject({ + mcpServers: { other: { command: "true" }, hack: { command: "hack" } }, + }); + + const stale = agentsAfter.replace( + "", + "\nDRIFTED LINE" + ); + await writeFile(agentsPath, stale); + const refreshed = await run({ + action: "sync", + scope: "project", + root: project, + }); + expect(byTarget(refreshed, "agents-md").status).toBe("refreshed"); + expect(await Bun.file(agentsPath).text()).toBe(agentsAfter); + + await writeFile( + agentsPath, + `${agents}\n\nhalf a section\n` + ); + const malformed = await run({ + action: "sync", + scope: "project", + root: project, + }); + expect(byTarget(malformed, "agents-md")).toMatchObject({ + status: "preserved", + reason: "customized-content", + }); +}); + +test("remove deletes only Hack-owned targets and preserves customized ones", async () => { + const home = await makeRoot("home"); + await run({ + action: "sync", + scope: "global", + root: home, + installMissing: true, + }); + const rules = join(home, ".cursor/rules/hack.mdc"); + const customized = `${renderCursorRules()}\nPersonal note.\n`; + await writeFile(rules, customized); + + const removed = await run({ action: "remove", scope: "global", root: home }); + expect(byTarget(removed, "cursor-rules").status).toBe("preserved"); + expect(byTarget(removed, "shared-hack-cli-skill").status).toBe("removed"); + expect(byTarget(removed, "claude-hooks").status).toBe("removed"); + expect(byTarget(removed, "codex-mcp").status).toBe("removed"); + expect(await Bun.file(rules).text()).toBe(customized); + expect(await Bun.file(join(home, ".ai/skills/hack-cli")).exists()).toBe( + false + ); + expect(setupSyncExitCode({ action: "remove", results: removed })).toBe(0); +}); + +test("retired Hack artifacts are reported when present and removed on sync", async () => { + const home = await makeRoot("home"); + const retired = join(home, ".codex/skills/hack-tickets/SKILL.md"); + await writeFile(retired, "---\nname: hack-tickets\n---\nRetired.\n"); + + const checked = await run({ action: "check", scope: "global", root: home }); + expect(byTarget(checked, "retired-artifact")).toMatchObject({ + status: "removed", + path: retired, + }); + expect(await Bun.file(retired).exists()).toBe(true); + expect(setupSyncExitCode({ action: "check", results: checked })).toBe(1); + + const synced = await run({ action: "sync", scope: "global", root: home }); + expect(byTarget(synced, "retired-artifact").status).toBe("removed"); + expect(await Bun.file(retired).exists()).toBe(false); +}); + +test("an unresolved scope root is a closed error and touches nothing", async () => { + const results = await run({ action: "sync", scope: "global", root: " " }); + expect(results).toEqual([ + { + scope: "global", + target: "scope-root", + path: "", + status: "error", + message: "HOME is not set; cannot resolve global integration paths.", + }, + ]); + expect(setupSyncExitCode({ action: "sync", results })).toBe(1); +}); + +test("JSON data lists every target with its status and reason", async () => { + const project = await makeRoot("project"); + await writeFile(join(project, ".cursor/rules/hack.mdc"), "team rules\n"); + const results = await run({ + action: "sync", + scope: "project", + root: project, + }); + const data = buildSetupSyncJsonData({ + action: "sync", + installMissing: false, + results, + }); + expect(data.action).toBe("sync"); + expect(data.dryRun).toBe(false); + expect(data.counts).toEqual({ + preserved: 1, + missing: PROJECT_TARGETS.length - 1, + }); + expect(data.targets[0]).toEqual({ + scope: "project", + target: "cursor-rules", + path: join(project, ".cursor/rules/hack.mdc"), + status: "preserved", + reason: "missing-ownership-marker", + message: expect.stringContaining("no Hack ownership marker"), + }); + const serialized = JSON.stringify(data); + expect(serialized).not.toContain("team rules"); +}); diff --git a/tests/setup.test.ts b/tests/setup.test.ts index 86615693..66edc6a7 100644 --- a/tests/setup.test.ts +++ b/tests/setup.test.ts @@ -139,17 +139,24 @@ test("setup sync collapses healthy artifacts into one scope row", () => { const result = buildSetupSyncScopeResult({ action: "check", scope: "Project", - groups: [ + results: [ { - label: "Cursor", - results: [{ status: "noop", path: "/repo/.cursor/rules/hack.mdc" }], + scope: "project", + target: "cursor-rules", + status: "unchanged", + path: "/repo/.cursor/rules/hack.mdc", }, { - label: "MCP config", - results: [ - { status: "present", path: "/repo/.cursor/mcp.json" }, - { status: "present", path: "/repo/.codex/config.toml" }, - ], + scope: "project", + target: "cursor-mcp", + status: "unchanged", + path: "/repo/.cursor/mcp.json", + }, + { + scope: "project", + target: "codex-mcp", + status: "unchanged", + path: "/repo/.codex/config.toml", }, ], }); @@ -169,14 +176,18 @@ test("setup sync keeps failing artifact paths visible", () => { const result = buildSetupSyncScopeResult({ action: "check", scope: "Global", - groups: [ + results: [ { - label: "Codex", - results: [{ status: "stale", path: "/.codex/skill.md" }], + scope: "global", + target: "codex-hack-cli-skill", + status: "refreshed", + path: "/.codex/skills/hack-cli/SKILL.md", }, { - label: "Claude", - results: [{ status: "noop", path: "/.claude/settings.json" }], + scope: "global", + target: "claude-hooks", + status: "unchanged", + path: "/.claude/settings.json", }, ], }); @@ -186,7 +197,8 @@ test("setup sync keeps failing artifact paths visible", () => { label: "Global", status: "warn", meta: "1/2 current", - detail: "Codex: stale at /.codex/skill.md", + detail: + "Codex hack-cli skill: stale at /.codex/skills/hack-cli/SKILL.md", }); }); @@ -194,15 +206,12 @@ test("setup sync check reports retired agent artifacts as stale", () => { const result = buildSetupSyncScopeResult({ action: "check", scope: "Project", - groups: [ + results: [ { - label: "Retired agent artifacts", - results: [ - { - status: "deprecated", - path: "/repo/.codex/skills/hack-tickets/SKILL.md", - }, - ], + scope: "project", + target: "retired-artifact", + status: "removed", + path: "/repo/.codex/skills/hack-tickets/SKILL.md", }, ], }); @@ -212,6 +221,42 @@ test("setup sync check reports retired agent artifacts as stale", () => { expect(result.item.meta).toBe("0/1 current"); }); +test("setup sync check counts missing targets without failing", () => { + const missing = { + scope: "project", + target: "agents-md", + status: "missing", + path: "/repo/AGENTS.md", + } as const; + const result = buildSetupSyncScopeResult({ + action: "check", + scope: "Project", + results: [ + missing, + { + scope: "project", + target: "cursor-rules", + status: "unchanged", + path: "/repo/.cursor/rules/hack.mdc", + }, + ], + }); + expect(result.exitCode).toBe(0); + expect(result.item.status).toBe("ok"); + expect(result.item.meta).toBe("1 current, 1 not installed"); + + // With --install-missing, check plans an install and fails like a stale file. + const planned = buildSetupSyncScopeResult({ + action: "check", + scope: "Project", + results: [{ ...missing, status: "installed" }], + }); + expect(planned.exitCode).toBe(1); + expect(planned.item.detail).toBe( + "AGENTS.md: not installed (would install) at /repo/AGENTS.md" + ); +}); + test("buildInitAssistantReport captures repo signals", async () => { const repoRoot = await setupTempRepo(); await Bun.write( @@ -388,22 +433,22 @@ test("Codex project and user duplicates keep separate ownership", async () => { test("setup sync reports preserved files without failing the action", () => { const result = buildSetupSyncScopeResult({ - action: "install", + action: "sync", scope: "Global", - groups: [ + results: [ { - label: "Codex", - results: [ - { - status: "preserved", - path: "/.codex/skills/hack-cli/SKILL.md", - message: "Hack left this path unchanged because it was edited.", - }, - ], + scope: "global", + target: "codex-hack-cli-skill", + status: "preserved", + reason: "customized-content", + path: "/.codex/skills/hack-cli/SKILL.md", + message: "Hack left this path unchanged because it was edited.", }, { - label: "Cursor", - results: [{ status: "updated", path: "/.cursor/rules/hack.mdc" }], + scope: "global", + target: "cursor-rules", + status: "refreshed", + path: "/.cursor/rules/hack.mdc", }, ], }); @@ -412,28 +457,72 @@ test("setup sync reports preserved files without failing the action", () => { expect(result.item).toEqual({ label: "Global", status: "warn", - meta: "1 updated, 1 preserved", + meta: "1 refreshed, 1 preserved", detail: - "Codex: preserved at /.codex/skills/hack-cli/SKILL.md. Hack left this path unchanged because it was edited.", + "Codex hack-cli skill: preserved at /.codex/skills/hack-cli/SKILL.md. Hack left this path unchanged because it was edited.", }); const removed = buildSetupSyncScopeResult({ action: "remove", scope: "Project", - groups: [ + results: [ { - label: "Cursor", - results: [ - { status: "preserved", path: "/repo/.cursor/rules/hack.mdc" }, - ], + scope: "project", + target: "cursor-rules", + status: "preserved", + path: "/repo/.cursor/rules/hack.mdc", }, ], }); expect(removed.exitCode).toBe(0); expect(removed.item.meta).toBe("0 removed, 1 preserved"); expect(removed.item.detail).toBe( - "Cursor: preserved at /repo/.cursor/rules/hack.mdc." + "Cursor rules: preserved at /repo/.cursor/rules/hack.mdc." ); + + // A preserved file means sync cannot make it current, so check fails. + const checked = buildSetupSyncScopeResult({ + action: "check", + scope: "Project", + results: [ + { + scope: "project", + target: "cursor-rules", + status: "preserved", + path: "/repo/.cursor/rules/hack.mdc", + }, + ], + }); + expect(checked.exitCode).toBe(1); +}); + +test("setup sync summarizes skipped missing targets", () => { + const result = buildSetupSyncScopeResult({ + action: "sync", + scope: "Global", + results: [ + { + scope: "global", + target: "cursor-rules", + status: "unchanged", + path: "/.cursor/rules/hack.mdc", + }, + { + scope: "global", + target: "codex-mcp", + status: "missing", + path: "/.codex/config.toml", + }, + ], + }); + expect(result.exitCode).toBe(0); + expect(result.item).toEqual({ + label: "Global", + status: "ok", + meta: "already current, 1 missing (skipped)", + detail: + "Missing integrations were skipped; add --install-missing to create them.", + }); }); async function setupTempRepo(): Promise { From 5f8cb79c06a1b28935e9b59c60de1b121dbc4a2f Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 09:07:05 -0400 Subject: [PATCH 48/52] fix(logger): pass messages to gum after an end-of-options marker The gum-backed logger passed the log message as a bare argument, so a message starting with "-" or "--" (for example "--value-stdin requires ...") was parsed by gum's kong CLI as an unknown flag. gum printed its usage and "gum: error: unknown flag" to stderr before the logger fell back to clack. Every gum call site in src/ui/gum.ts now places free text (log message and structured fields, style/join text, confirm prompt, choose/filter options, file path, pager content, format template, spin command, version-check constraint) after "--", and binds valued flags such as --header, --placeholder, --prompt, --value, --title and --affirmative in single-token --flag=value form so a dash-leading value cannot be read as an option. gum invocations now go through a small GumRunner boundary so tests can assert argv without gum installed or any process being spawned. Co-Authored-By: Claude Opus 5.5 --- src/ui/gum.ts | 406 +++++++++++++++++-------------- tests/fixtures/gum | 3 + tests/gum-end-of-options.test.ts | 301 +++++++++++++++++++++++ tests/gum.test.ts | 4 +- 4 files changed, 529 insertions(+), 185 deletions(-) create mode 100644 tests/gum-end-of-options.test.ts diff --git a/src/ui/gum.ts b/src/ui/gum.ts index ccfa7876..914f011c 100644 --- a/src/ui/gum.ts +++ b/src/ui/gum.ts @@ -142,6 +142,76 @@ export async function ensureBundledGumInstalled(): Promise>; + readonly stdin: "inherit" | "ignore" | { readonly text: string }; + readonly stdout: "inherit" | "pipe" | "ignore"; +} + +export interface GumRunResult { + readonly exitCode: number; + /** Captured stdout when the request used `stdout: "pipe"`, otherwise "". */ + readonly stdout: string; +} + +/** Process boundary for every gum invocation in this module. */ +export interface GumRunner { + /** Synchronous run, used by the logger so log lines keep their order. */ + runSync(request: GumRunRequest): number; + run(request: GumRunRequest): Promise; +} + +const bunGumRunner: GumRunner = { + runSync({ cmd, env, stdin, stdout }) { + const res = Bun.spawnSync({ + cmd: [...cmd], + env: { ...env }, + stdin: + typeof stdin === "string" + ? stdin + : new TextEncoder().encode(stdin.text), + stdout, + stderr: "inherit", + }); + return res.exitCode; + }, + async run({ cmd, env, stdin, stdout }) { + const proc = Bun.spawn([...cmd], { + env: { ...env }, + stdin: typeof stdin === "string" ? stdin : streamFromText(stdin.text), + stdout, + stderr: "inherit", + }); + const piped = proc.stdout; + const [text, exitCode] = await Promise.all([ + piped instanceof ReadableStream + ? new Response(piped).text() + : Promise.resolve(""), + proc.exited, + ]); + return { exitCode, stdout: text }; + }, +}; + +let gumRunner: GumRunner = bunGumRunner; + +/** + * Test seam: route gum invocations through `runner` instead of spawning. + * Pass `null` to restore the real Bun process runner. + */ +export function setGumRunnerForTests({ + runner, +}: { + readonly runner: GumRunner | null; +}): void { + gumRunner = runner ?? bunGumRunner; +} + export function tryGumLog({ level, message, fields }: GumLogInput): boolean { const gum = getGumPath(); if (!gum) { @@ -162,22 +232,19 @@ export function tryGumLog({ level, message, fields }: GumLogInput): boolean { const cmd = [ gum, "log", - "--level", - level, + ...valueFlag({ flag: "level", value: level }), ...(kv.length > 0 ? ["--structured"] : []), - message, - ...kv, + ...positionals([message, ...kv]), ]; - const res = Bun.spawnSync({ + const exitCode = gumRunner.runSync({ cmd, env: buildGumRenderEnv(), stdin: "ignore", stdout: "inherit", - stderr: "inherit", }); - return res.exitCode === 0; + return exitCode === 0; } function getBundledGumInstallPath(): string { @@ -268,23 +335,21 @@ export async function gumConfirm({ const cmd = [ gum, "confirm", - ...(prompt ? [prompt] : []), ...(defaultChoice ? ["--default"] : []), ...(showOutput ? ["--show-output"] : []), - ...(affirmative ? ["--affirmative", affirmative] : []), - ...(negative ? ["--negative", negative] : []), + ...valueFlag({ flag: "affirmative", value: affirmative }), + ...valueFlag({ flag: "negative", value: negative }), ...booleanFlag({ value: showHelp, flag: "show-help" }), - ...(timeout ? ["--timeout", timeout] : []), + ...valueFlag({ flag: "timeout", value: timeout }), + ...positionals(prompt ? [prompt] : []), ]; - const proc = Bun.spawn(cmd, { + const { exitCode } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "inherit", - stderr: "inherit", }); - - const exitCode = await proc.exited; if (exitCode === 0) { return { ok: true, value: true }; } @@ -330,29 +395,24 @@ export async function gumInput({ const cmd = [ gum, "input", - ...(placeholder ? ["--placeholder", placeholder] : []), - ...(prompt ? ["--prompt", prompt] : []), - ...(header ? ["--header", header] : []), - ...(value ? ["--value", value] : []), - ...(width === undefined ? [] : ["--width", String(width)]), - ...(charLimit === undefined ? [] : ["--char-limit", String(charLimit)]), + ...valueFlag({ flag: "placeholder", value: placeholder }), + ...valueFlag({ flag: "prompt", value: prompt }), + ...valueFlag({ flag: "header", value: header }), + ...valueFlag({ flag: "value", value }), + ...valueFlag({ flag: "width", value: width }), + ...valueFlag({ flag: "char-limit", value: charLimit }), ...(password ? ["--password"] : []), ...booleanFlag({ value: showHelp, flag: "show-help" }), - ...(timeout ? ["--timeout", timeout] : []), + ...valueFlag({ flag: "timeout", value: timeout }), ...booleanFlag({ value: stripAnsi, flag: "strip-ansi" }), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout.trimEnd() }; } @@ -401,32 +461,27 @@ export async function gumWrite({ const cmd = [ gum, "write", - ...(width === undefined ? [] : ["--width", String(width)]), - ...(height === undefined ? [] : ["--height", String(height)]), - ...(header ? ["--header", header] : []), - ...(placeholder ? ["--placeholder", placeholder] : []), - ...(prompt ? ["--prompt", prompt] : []), + ...valueFlag({ flag: "width", value: width }), + ...valueFlag({ flag: "height", value: height }), + ...valueFlag({ flag: "header", value: header }), + ...valueFlag({ flag: "placeholder", value: placeholder }), + ...valueFlag({ flag: "prompt", value: prompt }), ...(showCursorLine ? ["--show-cursor-line"] : []), ...(showLineNumbers ? ["--show-line-numbers"] : []), - ...(value ? ["--value", value] : []), - ...(charLimit === undefined ? [] : ["--char-limit", String(charLimit)]), - ...(maxLines === undefined ? [] : ["--max-lines", String(maxLines)]), + ...valueFlag({ flag: "value", value }), + ...valueFlag({ flag: "char-limit", value: charLimit }), + ...valueFlag({ flag: "max-lines", value: maxLines }), ...booleanFlag({ value: showHelp, flag: "show-help" }), - ...(timeout ? ["--timeout", timeout] : []), + ...valueFlag({ flag: "timeout", value: timeout }), ...booleanFlag({ value: stripAnsi, flag: "strip-ansi" }), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout.trimEnd() }; } @@ -481,40 +536,29 @@ export async function gumChooseMany( const cmd = [ gum, "choose", - ...input.options, ...(input.ordered ? ["--ordered"] : []), - ...(input.height === undefined ? [] : ["--height", String(input.height)]), - ...(input.cursor ? ["--cursor", input.cursor] : []), + ...valueFlag({ flag: "height", value: input.height }), + ...valueFlag({ flag: "cursor", value: input.cursor }), ...booleanFlag({ value: input.showHelp, flag: "show-help" }), - ...(input.timeout ? ["--timeout", input.timeout] : []), - ...(input.header ? ["--header", input.header] : []), + ...valueFlag({ flag: "timeout", value: input.timeout }), + ...valueFlag({ flag: "header", value: input.header }), ...buildSelectedArg(input.selected), - ...(input.inputDelimiter - ? ["--input-delimiter", input.inputDelimiter] - : []), - ...(input.outputDelimiter - ? ["--output-delimiter", input.outputDelimiter] - : []), - ...(input.labelDelimiter - ? ["--label-delimiter", input.labelDelimiter] - : []), + ...valueFlag({ flag: "input-delimiter", value: input.inputDelimiter }), + ...valueFlag({ flag: "output-delimiter", value: input.outputDelimiter }), + ...valueFlag({ flag: "label-delimiter", value: input.labelDelimiter }), ...booleanFlag({ value: input.stripAnsi, flag: "strip-ansi" }), ...(input.noLimit ? ["--no-limit"] : []), - ...(input.limit === undefined ? [] : ["--limit", String(input.limit)]), + ...valueFlag({ flag: "limit", value: input.limit }), ...(input.selectIfOne ? ["--select-if-one"] : []), + ...positionals(input.options), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { const out = stdout.trimEnd(); const parts = delimiter === "\n" ? out.split("\n") : out.split(delimiter); @@ -576,43 +620,34 @@ export async function gumFilterMany( const cmd = [ gum, "filter", - ...input.options, ...buildSelectedArg(input.selected), ...booleanFlag({ value: input.showHelp, flag: "show-help" }), - ...(input.header ? ["--header", input.header] : []), - ...(input.placeholder ? ["--placeholder", input.placeholder] : []), - ...(input.prompt ? ["--prompt", input.prompt] : []), - ...(input.width === undefined ? [] : ["--width", String(input.width)]), - ...(input.height === undefined ? [] : ["--height", String(input.height)]), - ...(input.value ? ["--value", input.value] : []), + ...valueFlag({ flag: "header", value: input.header }), + ...valueFlag({ flag: "placeholder", value: input.placeholder }), + ...valueFlag({ flag: "prompt", value: input.prompt }), + ...valueFlag({ flag: "width", value: input.width }), + ...valueFlag({ flag: "height", value: input.height }), + ...valueFlag({ flag: "value", value: input.value }), ...(input.reverse ? ["--reverse"] : []), ...booleanFlag({ value: input.fuzzy, flag: "fuzzy" }), ...booleanFlag({ value: input.fuzzySort, flag: "fuzzy-sort" }), - ...(input.timeout ? ["--timeout", input.timeout] : []), - ...(input.inputDelimiter - ? ["--input-delimiter", input.inputDelimiter] - : []), - ...(input.outputDelimiter - ? ["--output-delimiter", input.outputDelimiter] - : []), + ...valueFlag({ flag: "timeout", value: input.timeout }), + ...valueFlag({ flag: "input-delimiter", value: input.inputDelimiter }), + ...valueFlag({ flag: "output-delimiter", value: input.outputDelimiter }), ...booleanFlag({ value: input.stripAnsi, flag: "strip-ansi" }), ...(input.noLimit ? ["--no-limit"] : []), - ...(input.limit === undefined ? [] : ["--limit", String(input.limit)]), + ...valueFlag({ flag: "limit", value: input.limit }), ...(input.selectIfOne ? ["--select-if-one"] : []), ...booleanFlag({ value: input.strict, flag: "strict" }), + ...positionals(input.options), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { const out = stdout.trimEnd(); const parts = delimiter === "\n" ? out.split("\n") : out.split(delimiter); @@ -659,30 +694,25 @@ export async function gumFile({ const cmd = [ gum, "file", - ...(path ? [path] : []), - ...(cursor ? ["--cursor", cursor] : []), + ...valueFlag({ flag: "cursor", value: cursor }), ...(all ? ["--all"] : []), ...booleanFlag({ value: permissions, flag: "permissions" }), ...booleanFlag({ value: size, flag: "size" }), ...(file ? ["--file"] : []), ...(directory ? ["--directory"] : []), ...booleanFlag({ value: showHelp, flag: "show-help" }), - ...(timeout ? ["--timeout", timeout] : []), - ...(header ? ["--header", header] : []), - ...(height === undefined ? [] : ["--height", String(height)]), + ...valueFlag({ flag: "timeout", value: timeout }), + ...valueFlag({ flag: "header", value: header }), + ...valueFlag({ flag: "height", value: height }), + ...positionals(path ? [path] : []), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout.trimEnd() }; } @@ -713,20 +743,18 @@ export async function gumPager({ const cmd = [ gum, "pager", - ...(content ? [content] : []), ...(showLineNumbers ? ["--show-line-numbers"] : []), ...booleanFlag({ value: softWrap, flag: "soft-wrap" }), - ...(timeout ? ["--timeout", timeout] : []), + ...valueFlag({ flag: "timeout", value: timeout }), + ...positionals(content ? [content] : []), ]; - const proc = Bun.spawn(cmd, { + const { exitCode } = await gumRunner.run({ + cmd, env: process.env, stdin: "inherit", stdout: "inherit", - stderr: "inherit", }); - - const exitCode = await proc.exited; if (exitCode === 0) { return { ok: true, value: undefined }; } @@ -771,21 +799,20 @@ export async function gumSpin({ ...(showError ? ["--show-error"] : []), ...(showStdout ? ["--show-stdout"] : []), ...(showStderr ? ["--show-stderr"] : []), - ...(spinner ? ["--spinner", spinner] : []), - ...(title ? ["--title", title] : []), - ...(align ? ["--align", align] : []), - ...(timeout ? ["--timeout", timeout] : []), - ...cmd, + ...valueFlag({ flag: "spinner", value: spinner }), + ...valueFlag({ flag: "title", value: title }), + ...valueFlag({ flag: "align", value: align }), + ...valueFlag({ flag: "timeout", value: timeout }), + // gum consumes this marker; the wrapped command keeps its own flags. + ...positionals(cmd), ]; - const proc = Bun.spawn(gumCmd, { + const { exitCode } = await gumRunner.run({ + cmd: gumCmd, env: process.env, stdin: "inherit", stdout: "inherit", - stderr: "inherit", }); - - const exitCode = await proc.exited; if (exitCode === 0) { return { ok: true, value: 0 }; } @@ -844,37 +871,32 @@ export async function gumStyle({ const cmd = [ gum, "style", - ...text, ...(trim ? ["--trim"] : []), ...booleanFlag({ value: stripAnsi, flag: "strip-ansi" }), - ...(foreground ? ["--foreground", foreground] : []), - ...(background ? ["--background", background] : []), - ...(border ? ["--border", border] : []), - ...(borderBackground ? ["--border-background", borderBackground] : []), - ...(borderForeground ? ["--border-foreground", borderForeground] : []), - ...(align ? ["--align", align] : []), - ...(height === undefined ? [] : ["--height", String(height)]), - ...(width === undefined ? [] : ["--width", String(width)]), - ...(margin ? ["--margin", margin] : []), - ...(padding ? ["--padding", padding] : []), + ...valueFlag({ flag: "foreground", value: foreground }), + ...valueFlag({ flag: "background", value: background }), + ...valueFlag({ flag: "border", value: border }), + ...valueFlag({ flag: "border-background", value: borderBackground }), + ...valueFlag({ flag: "border-foreground", value: borderForeground }), + ...valueFlag({ flag: "align", value: align }), + ...valueFlag({ flag: "height", value: height }), + ...valueFlag({ flag: "width", value: width }), + ...valueFlag({ flag: "margin", value: margin }), + ...valueFlag({ flag: "padding", value: padding }), ...(bold ? ["--bold"] : []), ...(faint ? ["--faint"] : []), ...(italic ? ["--italic"] : []), ...(strikethrough ? ["--strikethrough"] : []), ...(underline ? ["--underline"] : []), + ...positionals(text), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: buildGumRenderEnv(), stdin: "ignore", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout }; } @@ -902,23 +924,18 @@ export async function gumJoin({ const cmd = [ gum, "join", - ...text, - ...(align ? ["--align", align] : []), + ...valueFlag({ flag: "align", value: align }), ...(horizontal ? ["--horizontal"] : []), ...(vertical ? ["--vertical"] : []), + ...positionals(text), ]; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: buildGumRenderEnv(), stdin: "ignore", stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout }; } @@ -950,25 +967,19 @@ export async function gumFormat({ const cmd = [ gum, "format", - ...(template ?? []), - ...(theme ? ["--theme", theme] : []), - ...(language ? ["--language", language] : []), + ...valueFlag({ flag: "theme", value: theme }), + ...valueFlag({ flag: "language", value: language }), ...booleanFlag({ value: stripAnsi, flag: "strip-ansi" }), - ...(type ? ["--type", type] : []), + ...valueFlag({ flag: "type", value: type }), + ...positionals(template ?? []), ]; - const stdin = input !== undefined ? streamFromText(input) : "inherit"; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: buildGumRenderEnv(), - stdin, + stdin: input === undefined ? "inherit" : { text: input }, stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout }; } @@ -1019,37 +1030,27 @@ export async function gumTable({ const cmd = [ gum, "table", - ...(separator ? ["--separator", separator] : []), - ...(columns ? ["--columns", columns.join(",")] : []), - ...(widths ? ["--widths", widths.join(",")] : []), - ...(height === undefined ? [] : ["--height", String(height)]), + ...valueFlag({ flag: "separator", value: separator }), + ...valueFlag({ flag: "columns", value: columns?.join(",") }), + ...valueFlag({ flag: "widths", value: widths?.join(",") }), + ...valueFlag({ flag: "height", value: height }), ...(print ? ["--print"] : []), - ...(file ? ["--file", file] : []), - ...(border ? ["--border", border] : []), + ...valueFlag({ flag: "file", value: file }), + ...valueFlag({ flag: "border", value: border }), ...booleanFlag({ value: showHelp, flag: "show-help" }), ...booleanFlag({ value: hideCount, flag: "hide-count" }), ...(lazyQuotes ? ["--lazy-quotes"] : []), - ...(fieldsPerRecord === undefined - ? [] - : ["--fields-per-record", String(fieldsPerRecord)]), - ...(returnColumn === undefined - ? [] - : ["--return-column", String(returnColumn)]), - ...(timeout ? ["--timeout", timeout] : []), + ...valueFlag({ flag: "fields-per-record", value: fieldsPerRecord }), + ...valueFlag({ flag: "return-column", value: returnColumn }), + ...valueFlag({ flag: "timeout", value: timeout }), ]; - const stdin = input !== undefined ? streamFromText(input) : "inherit"; - const proc = Bun.spawn(cmd, { + const { exitCode, stdout } = await gumRunner.run({ + cmd, env: buildGumRenderEnv(), - stdin, + stdin: input === undefined ? "inherit" : { text: input }, stdout: "pipe", - stderr: "inherit", }); - - const [stdout, exitCode] = await Promise.all([ - new Response(proc.stdout).text(), - proc.exited, - ]); if (exitCode === 0) { return { ok: true, value: stdout.trimEnd() }; } @@ -1071,14 +1072,12 @@ export async function gumVersionCheck({ return { ok: false, reason: "unavailable" }; } - const proc = Bun.spawn([gum, "version-check", constraint], { + const { exitCode } = await gumRunner.run({ + cmd: [gum, "version-check", ...positionals([constraint])], env: process.env, stdin: "ignore", stdout: "ignore", - stderr: "inherit", }); - - const exitCode = await proc.exited; if (exitCode === 0) { return { ok: true, value: true }; } @@ -1138,7 +1137,46 @@ function buildSelectedArg( return []; } if (selected === "*") { - return ["--selected", "*"]; + return ["--selected=*"]; + } + return valueFlag({ flag: "selected", value: selected.join(",") }); +} + +/** + * gum (kong) ends option parsing at this token; every later argument is a + * positional, even when it begins with "-" or "--". + */ +export const GUM_END_OF_OPTIONS = "--"; + +/** + * Builds a valued flag in single-token `--flag=value` form. + * + * Invariant: free text must never be parsed as a gum option. In the + * two-token form kong rejects a value that begins with "-" (it reads it as a + * flag); the joined form always binds the value to the flag. Empty strings and + * `undefined` omit the flag, matching the previous truthiness checks. + */ +function valueFlag({ + flag, + value, +}: { + readonly flag: string; + readonly value: string | number | undefined; +}): readonly string[] { + if (value === undefined || value === "") { + return []; + } + return [`--${flag}=${String(value)}`]; +} + +/** + * Places positional text after the end-of-options marker. Callers must spread + * this last, after every flag, so text such as "--value-stdin requires ..." + * is printed instead of being parsed as an unknown flag. + */ +function positionals(args: readonly string[]): readonly string[] { + if (args.length === 0) { + return []; } - return ["--selected", selected.join(",")]; + return [GUM_END_OF_OPTIONS, ...args]; } diff --git a/tests/fixtures/gum b/tests/fixtures/gum index fadf0cf4..6c790c9a 100755 --- a/tests/fixtures/gum +++ b/tests/fixtures/gum @@ -58,6 +58,9 @@ case "$sub" in exit "${GUM_FIXTURE_CONFIRM_EXIT:-0}" ;; version-check) + if [[ "${1:-}" == "--" ]]; then + shift + fi if [[ "${1:-}" == "fail" ]]; then exit 1 fi diff --git a/tests/gum-end-of-options.test.ts b/tests/gum-end-of-options.test.ts new file mode 100644 index 00000000..24d5cc2a --- /dev/null +++ b/tests/gum-end-of-options.test.ts @@ -0,0 +1,301 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; + +import { + GUM_END_OF_OPTIONS, + type GumRunRequest, + gumChooseMany, + gumConfirm, + gumFile, + gumFilterMany, + gumFormat, + gumInput, + gumJoin, + gumPager, + gumSpin, + gumStyle, + gumTable, + gumVersionCheck, + gumWrite, + resetGumPathCacheForTests, + setGumRunnerForTests, + tryGumLog, +} from "../src/ui/gum.ts"; +import { logger, setLoggerBackendOverride } from "../src/ui/logger.ts"; + +// Never spawned: every invocation goes through the recording runner below. +const FAKE_GUM = "/nonexistent/hack-test/gum"; +const DASH_TEXT = "--value-stdin requires --env"; +const SHORT_DASH_TEXT = "-n is not a number"; + +let calls: GumRunRequest[] = []; +let previousGumPath: string | undefined; + +beforeEach(() => { + calls = []; + previousGumPath = process.env.HACK_GUM_PATH; + process.env.HACK_GUM_PATH = FAKE_GUM; + resetGumPathCacheForTests(); + setGumRunnerForTests({ + runner: { + runSync(request) { + calls.push(request); + return 0; + }, + run(request) { + calls.push(request); + return Promise.resolve({ exitCode: 0, stdout: "picked\n" }); + }, + }, + }); +}); + +afterEach(() => { + setGumRunnerForTests({ runner: null }); + setLoggerBackendOverride({ backend: null }); + if (previousGumPath === undefined) { + Reflect.deleteProperty(process.env, "HACK_GUM_PATH"); + } else { + process.env.HACK_GUM_PATH = previousGumPath; + } + resetGumPathCacheForTests(); +}); + +function onlyCall(): readonly string[] { + expect(calls).toHaveLength(1); + const call = calls[0]; + if (!call) { + throw new Error("expected one gum invocation"); + } + return call.cmd; +} + +/** + * Asserts every listed text is a positional after the end-of-options marker + * and that nothing before the marker is free text (only `--flag[=value]`). + */ +function expectTextAfterMarker({ + cmd, + subcommand, + text, +}: { + readonly cmd: readonly string[]; + readonly subcommand: string; + readonly text: readonly string[]; +}): void { + expect(cmd.slice(0, 2)).toEqual([FAKE_GUM, subcommand]); + const marker = cmd.indexOf(GUM_END_OF_OPTIONS); + expect(marker).toBeGreaterThan(1); + expect(cmd.slice(marker + 1)).toEqual([...text]); + for (const arg of cmd.slice(2, marker)) { + expect(arg.startsWith("--")).toBe(true); + } +} + +test("tryGumLog puts a dash-leading message and fields after the marker", () => { + expect( + tryGumLog({ + level: "warn", + message: DASH_TEXT, + fields: { flag: "--force", b: SHORT_DASH_TEXT }, + }) + ).toBe(true); + + const cmd = onlyCall(); + expect(cmd).toEqual([ + FAKE_GUM, + "log", + "--level=warn", + "--structured", + GUM_END_OF_OPTIONS, + DASH_TEXT, + "b", + SHORT_DASH_TEXT, + "flag", + "--force", + ]); + expect(calls[0]?.stdin).toBe("ignore"); +}); + +test("tryGumLog without fields still ends options before the message", () => { + tryGumLog({ level: "info", message: DASH_TEXT }); + expect(onlyCall()).toEqual([ + FAKE_GUM, + "log", + "--level=info", + GUM_END_OF_OPTIONS, + DASH_TEXT, + ]); +}); + +test("logger selected as gum passes a dash-leading message after the marker", () => { + setLoggerBackendOverride({ backend: "gum" }); + logger.error({ message: DASH_TEXT }); + logger.success({ message: SHORT_DASH_TEXT }); + + expect(calls).toHaveLength(2); + expect(calls[0]?.cmd).toEqual([ + FAKE_GUM, + "log", + "--level=error", + GUM_END_OF_OPTIONS, + DASH_TEXT, + ]); + expect(calls[1]?.cmd).toEqual([ + FAKE_GUM, + "log", + "--level=info", + "--structured", + GUM_END_OF_OPTIONS, + SHORT_DASH_TEXT, + "status", + "success", + ]); +}); + +test("gumStyle and gumJoin pass text blocks after the marker", async () => { + await gumStyle({ text: [DASH_TEXT, "-"], bold: true, padding: "0 1" }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "style", + text: [DASH_TEXT, "-"], + }); + expect(calls[0]?.cmd).toContain("--padding=0 1"); + + calls = []; + await gumJoin({ text: [SHORT_DASH_TEXT, DASH_TEXT], horizontal: true }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "join", + text: [SHORT_DASH_TEXT, DASH_TEXT], + }); +}); + +test("gumConfirm passes the prompt after the marker and binds button labels", async () => { + const res = await gumConfirm({ + prompt: DASH_TEXT, + affirmative: "--yes", + negative: "-no", + timeout: "5s", + }); + expect(res).toEqual({ ok: true, value: true }); + + const cmd = onlyCall(); + expectTextAfterMarker({ cmd, subcommand: "confirm", text: [DASH_TEXT] }); + expect(cmd).toContain("--affirmative=--yes"); + expect(cmd).toContain("--negative=-no"); +}); + +test("gumConfirm without a prompt adds no marker", async () => { + await gumConfirm({ default: true }); + expect(onlyCall()).toEqual([FAKE_GUM, "confirm", "--default"]); +}); + +test("gumChooseMany and gumFilterMany pass options after the marker", async () => { + const chosen = await gumChooseMany({ + options: [DASH_TEXT, "-a"], + header: "--pick one", + selected: [DASH_TEXT], + }); + expect(chosen).toEqual({ ok: true, value: ["picked"] }); + let cmd = onlyCall(); + expectTextAfterMarker({ cmd, subcommand: "choose", text: [DASH_TEXT, "-a"] }); + expect(cmd).toContain("--header=--pick one"); + expect(cmd).toContain(`--selected=${DASH_TEXT}`); + + calls = []; + await gumFilterMany({ + options: ["-x", DASH_TEXT], + placeholder: "-search", + prompt: "--> ", + value: "-", + }); + cmd = onlyCall(); + expectTextAfterMarker({ cmd, subcommand: "filter", text: ["-x", DASH_TEXT] }); + expect(cmd).toContain("--placeholder=-search"); + expect(cmd).toContain("--prompt=--> "); + expect(cmd).toContain("--value=-"); +}); + +test("text prompts bind dash-leading values to their flags", async () => { + await gumInput({ header: DASH_TEXT, placeholder: "-p", value: "--v" }); + let cmd = onlyCall(); + expect(cmd.slice(0, 2)).toEqual([FAKE_GUM, "input"]); + expect(cmd).toEqual([ + FAKE_GUM, + "input", + "--placeholder=-p", + `--header=${DASH_TEXT}`, + "--value=--v", + ]); + + calls = []; + await gumWrite({ header: DASH_TEXT, prompt: "-> ", width: 40 }); + cmd = onlyCall(); + expect(cmd).toEqual([ + FAKE_GUM, + "write", + "--width=40", + `--header=${DASH_TEXT}`, + "--prompt=-> ", + ]); +}); + +test("gumFile, gumPager and gumFormat pass content after the marker", async () => { + await gumFile({ path: "-dir", header: "--h" }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "file", + text: ["-dir"], + }); + + calls = []; + await gumPager({ content: DASH_TEXT, softWrap: true }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "pager", + text: [DASH_TEXT], + }); + + calls = []; + await gumFormat({ template: [DASH_TEXT], type: "template" }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "format", + text: [DASH_TEXT], + }); + + calls = []; + await gumFormat({ input: DASH_TEXT, type: "markdown" }); + expect(onlyCall()).toEqual([FAKE_GUM, "format", "--type=markdown"]); + expect(calls[0]?.stdin).toEqual({ text: DASH_TEXT }); +}); + +test("gumSpin ends gum options before the wrapped command", async () => { + await gumSpin({ cmd: ["bun", "--version"], title: "-loading" }); + const cmd = onlyCall(); + expectTextAfterMarker({ + cmd, + subcommand: "spin", + text: ["bun", "--version"], + }); + expect(cmd).toContain("--title=-loading"); +}); + +test("gumTable feeds rows on stdin and binds flag values", async () => { + await gumTable({ + input: `${DASH_TEXT}\n`, + columns: ["-a", "b"], + print: true, + }); + expect(onlyCall()).toEqual([FAKE_GUM, "table", "--columns=-a,b", "--print"]); + expect(calls[0]?.stdin).toEqual({ text: `${DASH_TEXT}\n` }); +}); + +test("gumVersionCheck passes the constraint after the marker", async () => { + await gumVersionCheck({ constraint: "-1" }); + expectTextAfterMarker({ + cmd: onlyCall(), + subcommand: "version-check", + text: ["-1"], + }); +}); diff --git a/tests/gum.test.ts b/tests/gum.test.ts index 162e501e..02b9c6d7 100644 --- a/tests/gum.test.ts +++ b/tests/gum.test.ts @@ -104,5 +104,7 @@ test("tryGumLog returns true and writes structured args (fixture)", async () => expect(ok).toBe(true); const text = await Bun.file(logFile).text(); - expect(text).toContain("gum log --level info --structured hello a 1 ok true"); + expect(text).toContain( + "gum log --level=info --structured -- hello a 1 ok true" + ); }); From 378f75f4987e0eb67e51cf3aedf53d598f62356b Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 09:10:23 -0400 Subject: [PATCH 49/52] fix(env): refuse invalid env key arguments without echoing them `hack env add KEY=value` printed "Invalid env key: KEY=", and other key/scope refusals echoed the raw argument, which is often a mistyped assignment or a value pasted into the key position. - parseProjectEnvTarget refuses any argument containing "=" before the dotted-scope split, and every key/scope refusal is now fixed text that names the allowed pattern but never the argument. - setProjectEnvValue, assertValidProjectEnvScopeName, legacy env set/unset key resolution and env get use the same value-free messages; a malformed key in an env config file is reported by scope only. - env add validates the key argument before reading --value-stdin or migrating project files; legacy env unset validates the key before resolving a secret store. Tests run the real CLI with a canary inside the argument and assert it is absent from stdout, stderr and spawned argv/env, with files unchanged. Co-Authored-By: Claude Opus 5.5 --- src/commands/env.ts | 33 +- src/lib/project-env-config.ts | 32 +- tests/env-key-argument-redaction-cli.test.ts | 361 +++++++++++++++++++ tests/project-env-config.test.ts | 45 +++ 4 files changed, 455 insertions(+), 16 deletions(-) create mode 100644 tests/env-key-argument-redaction-cli.test.ts diff --git a/src/commands/env.ts b/src/commands/env.ts index a988205b..5f60870d 100644 --- a/src/commands/env.ts +++ b/src/commands/env.ts @@ -61,6 +61,7 @@ import { import { assertValidProjectEnvScopeName, discoverComposeServiceNames, + ENV_KEY_INVALID_MESSAGE, materializeProjectEnv, migrateLegacyProjectEnv, parseProjectEnvTarget, @@ -1018,7 +1019,7 @@ const handleEnvGet: CommandHandlerFor = async ({ }) => { const key = args.positionals.key; if (!ENV_KEY_PATTERN.test(key)) { - throw new CliUsageError("Invalid env key"); + throw new CliUsageError(ENV_KEY_INVALID_MESSAGE); } const scope = assertValidProjectEnvScopeName({ scopeName: args.options.service, @@ -1733,17 +1734,26 @@ const handleEnvAdd: CommandHandlerFor = async ({ }): Promise => { const keyArg = args.positionals.key?.trim(); const valueArg = args.positionals.value; + const valueStdin = args.options.valueStdin === true; // --value-stdin: refuse bad combinations and read the bounded value before // any project file is migrated or written, so a refusal stores nothing. - let stdinValue: string | null = null; - if (args.options.valueStdin === true) { + if (valueStdin) { assertEnvValueStdinUsage({ hasKey: Boolean(keyArg), hasPositionalValue: valueArg !== undefined || keyArg?.includes("=") === true, }); - stdinValue = await readEnvValueFromStdin(); } + // Validate a key argument with fixed, value-free refusals before stdin is + // read or any project file is migrated or written. + const interactive = !(valueStdin || keyArg || valueArg); + const argTarget = interactive + ? null + : parseProjectEnvTarget({ + keyOrPath: keyArg ?? "", + scopeOverride: args.options.service, + }); + const stdinValue = valueStdin ? await readEnvValueFromStdin() : null; const project = await resolveProjectForEnv({ ctx, @@ -1760,7 +1770,6 @@ const handleEnvAdd: CommandHandlerFor = async ({ reason: "mutation", }); - const interactive = stdinValue === null && !(keyArg || valueArg); const prompted = interactive ? await promptForEnvAddInput({ project, @@ -1770,10 +1779,10 @@ const handleEnvAdd: CommandHandlerFor = async ({ const parsedTarget = prompted ? { scope: prompted.scope, key: prompted.key } - : parseProjectEnvTarget({ - keyOrPath: keyArg ?? "", - scopeOverride: args.options.service, - }); + : argTarget; + if (!parsedTarget) { + throw new CliUsageError("Env key is required."); + } const value = prompted?.value ?? stdinValue ?? @@ -2188,10 +2197,12 @@ const handleEnvUnset: CommandHandlerFor = async ({ projectDir: project.projectDir, }); if (!modernExists) { - const store = await resolveConfiguredSecretStore({ project, projectName }); + // Validate the key before resolving a secret store, so a refused key + // never reaches a backend. const key = await resolveEnvKey({ key: (args.positionals.key ?? "").trim(), }); + const store = await resolveConfiguredSecretStore({ project, projectName }); const okLegacy = await confirmSafe({ message: `Unset "${key}" from ${resolveEnvFilePath({ projectDir: project.projectDir, envName })} and ${formatSecretStoreDescriptor({ descriptor: store.descriptor })}${envName ? ` (env ${envName})` : ""}?`, @@ -2433,7 +2444,7 @@ async function resolveEnvKey(opts: { readonly key: string }): Promise { const fromPos = opts.key.trim(); if (fromPos.length > 0) { if (!ENV_KEY_PATTERN.test(fromPos)) { - throw new CliUsageError(`Invalid env key: ${fromPos}`); + throw new CliUsageError(ENV_KEY_INVALID_MESSAGE); } return fromPos; } diff --git a/src/lib/project-env-config.ts b/src/lib/project-env-config.ts index 1d756962..f12519b1 100644 --- a/src/lib/project-env-config.ts +++ b/src/lib/project-env-config.ts @@ -80,6 +80,21 @@ const NATIVE_ENV_WORKLOAD_PATTERN = /^[a-z0-9][a-z0-9._-]{0,62}$/; const NATIVE_ENV_OVERLAY_PATTERN = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; const PROJECT_ENV_KEY_PATTERN = /^[A-Z_][A-Z0-9_]*$/; const PROJECT_ENV_SCOPE_PATTERN = /^[a-z0-9][a-z0-9._-]*$/; + +/** + * Fixed refusals for env key and scope arguments. + * + * Invariant: these messages never interpolate the rejected argument. A key + * argument is often a mistyped `KEY=value` or a value pasted into the key + * position, so echoing any part of it could disclose a secret in terminal + * output, logs, or CI transcripts. + */ +export const ENV_KEY_INVALID_MESSAGE = + "Invalid env key: keys must match ^[A-Z_][A-Z0-9_]*$ (for example API_TOKEN). The argument is not shown because it may contain a value."; +export const ENV_KEY_ASSIGNMENT_MESSAGE = + 'Invalid env key: the key argument cannot contain "=". Pass only the key (matching ^[A-Z_][A-Z0-9_]*$) and supply secret values with --value-stdin or the hidden prompt. The argument is not shown because it may contain a value.'; +export const ENV_SCOPE_INVALID_MESSAGE = + 'Invalid env scope: scopes must be "global" or match ^[a-z0-9][a-z0-9._-]*$ (usually a compose service name). The argument is not shown because it may contain a value.'; const PROJECT_ENV_SECRET_PREFIX = "v1" as const; const PROJECT_ENV_ALGORITHM = "aes-256-gcm"; const PROJECT_ENV_IV_BYTES = 12; @@ -219,7 +234,7 @@ export function assertValidProjectEnvScopeName(opts: { scopeName: opts.scopeName, }); if (!isValidProjectEnvScopeName({ scopeName })) { - throw new Error(`Invalid env scope: ${scopeName}`); + throw new Error(ENV_SCOPE_INVALID_MESSAGE); } return scopeName; } @@ -690,7 +705,9 @@ function parseProjectEnvConfig(opts: { if (!PROJECT_ENV_KEY_PATTERN.test(key)) { return { ok: false, - error: `Invalid env key "${key}" in scope "${scope}".`, + // The scope is validated above; the key is not echoed because a + // malformed entry may hold a pasted value. + error: `Invalid env key in scope "${scope}"; keys must match ^[A-Z_][A-Z0-9_]*$.`, }; } const normalizedValue = parseProjectEnvStoredValue({ valueRaw }); @@ -2213,7 +2230,7 @@ export async function setProjectEnvValue(opts: { await assertLegacyProjectDirectory({ projectDir: opts.projectDir }); await assertLegacyProjectInputFamily({ projectRoot: opts.projectRoot }); if (!PROJECT_ENV_KEY_PATTERN.test(opts.key)) { - throw new Error(`Invalid env key: ${opts.key}`); + throw new Error(ENV_KEY_INVALID_MESSAGE); } const scope = assertValidProjectEnvScopeName({ scopeName: opts.scope, @@ -4074,6 +4091,11 @@ export function parseProjectEnvTarget(input: { if (trimmed.length === 0) { throw new Error("Env key is required."); } + // Refuse `KEY=value` before any dotted-scope split, so no fragment of the + // value can reach a scope or key error. Every refusal below is fixed text. + if (trimmed.includes("=")) { + throw new Error(ENV_KEY_ASSIGNMENT_MESSAGE); + } const dotIndex = trimmed.lastIndexOf("."); if (dotIndex > 0) { @@ -4084,13 +4106,13 @@ export function parseProjectEnvTarget(input: { const key = trimmed.slice(dotIndex + 1).trim(); assertValidProjectEnvScopeName({ scopeName: scope }); if (!PROJECT_ENV_KEY_PATTERN.test(key)) { - throw new Error(`Invalid env key: ${key}`); + throw new Error(ENV_KEY_INVALID_MESSAGE); } return { scope, key }; } if (!PROJECT_ENV_KEY_PATTERN.test(trimmed)) { - throw new Error(`Invalid env key: ${trimmed}`); + throw new Error(ENV_KEY_INVALID_MESSAGE); } return { scope: assertValidProjectEnvScopeName({ diff --git a/tests/env-key-argument-redaction-cli.test.ts b/tests/env-key-argument-redaction-cli.test.ts new file mode 100644 index 00000000..3c4ce87c --- /dev/null +++ b/tests/env-key-argument-redaction-cli.test.ts @@ -0,0 +1,361 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +/** + * Key-argument refusals in `hack env` must never echo the rejected argument. + * + * A key argument is often a mistyped `KEY=value` or a value pasted into the + * key position. Each case runs the real CLI in a fresh subprocess with a + * distinctive canary inside the argument and proves the canary is absent from + * stdout, stderr, and the argv/env of every child process the CLI spawns. + * + * Isolation: temp HOME and global config, a temp project, the project-key + * YAML store or an injected `encrypted_file` backend with an env-provided key. + * No keychain, 1Password, Docker, or agent client config is reached. + */ + +const REPO_ROOT = resolve(import.meta.dir, ".."); +const ENTRYPOINT = resolve(REPO_ROOT, "index.ts"); +const RECORDER = resolve(import.meta.dir, "fixtures/spawn-argv-recorder.ts"); +/** Lowercase canary: valid as a scope fragment, invalid as an env key. */ +const CANARY = "echo-canary-c1-9d4f7a"; +/** Uppercase-bearing canary: invalid as a scope name. */ +const SCOPE_CANARY = "Echo-Scope-Canary-C1-3b6e"; +const SECRETS_FILE_KEY = "env-key-redaction-test-key"; + +const INVALID_KEY = "Invalid env key: keys must match ^[A-Z_][A-Z0-9_]*$"; +const ASSIGNMENT_KEY = 'the key argument cannot contain "="'; +const INVALID_SCOPE = 'Invalid env scope: scopes must be "global"'; +const GET_REFUSAL = "Unable to read env value."; + +type RunResult = { + readonly exitCode: number; + readonly stdout: string; + readonly stderr: string; + readonly spawnLog: string; +}; + +let root: string | null = null; + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "hack-env-key-redaction-")); + await mkdir(join(root, "home"), { recursive: true }); +}); + +afterEach(async () => { + if (root) { + await rm(root, { recursive: true, force: true }); + root = null; + } +}); + +function requireRoot(): string { + if (!root) { + throw new Error("Missing temp root"); + } + return root; +} + +async function runHack(input: { + readonly args: readonly string[]; + readonly cwd?: string; + readonly stdin?: string; +}): Promise { + const base = requireRoot(); + const spawnLog = join(base, `spawn-${crypto.randomUUID()}.jsonl`); + const proc = Bun.spawn( + [process.execPath, "--preload", RECORDER, ENTRYPOINT, ...input.args], + { + cwd: input.cwd ?? REPO_ROOT, + env: { + PATH: process.env.PATH ?? "/usr/bin:/bin", + HOME: join(base, "home"), + TMPDIR: join(base, "home"), + HACK_GLOBAL_CONFIG_PATH: join(base, "home", "hack-config.json"), + HACK_LOGGER: "console", + HACK_SECRETS_FILE_KEY: SECRETS_FILE_KEY, + HACK_TEST_SPAWN_ARGV_LOG: spawnLog, + }, + stdin: input.stdin === undefined ? "ignore" : "pipe", + stdout: "pipe", + stderr: "pipe", + } + ); + if (input.stdin !== undefined && proc.stdin) { + proc.stdin.write(input.stdin); + await proc.stdin.end(); + } + const [stdout, stderr, exitCode] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + const spawnLogText = await readFile(spawnLog, "utf8").catch(() => ""); + return { exitCode, stdout, stderr, spawnLog: spawnLogText }; +} + +function expectRefusedWithoutEcho(input: { + readonly result: RunResult; + readonly message: string; + readonly canary: string; + readonly rawArgument: string; +}): void { + const { result } = input; + expect(result.exitCode).toBe(1); + expect(result.stderr).toContain(input.message); + for (const stream of [result.stdout, result.stderr, result.spawnLog]) { + expect(stream).not.toContain(input.canary); + expect(stream).not.toContain(input.rawArgument); + } +} + +async function createModernProject(): Promise { + const projectRoot = join(requireRoot(), "modern"); + const projectDir = join(projectRoot, ".hack"); + await mkdir(projectDir, { recursive: true }); + await writeFile( + join(projectDir, "docker-compose.yml"), + "services:\n api:\n image: alpine:3.20\n" + ); + await writeFile( + join(projectDir, "hack.config.json"), + `${JSON.stringify({ name: "key-redaction-modern", dev_host: "key-redaction-modern.hack" }, null, 2)}\n` + ); + await writeFile( + join(projectDir, "hack.env.default.yaml"), + 'version: 1\nenvironment: default\nsecretsprovider: project_key\nvalues:\n global:\n GLOBAL_FLAG: "1"\n' + ); + return projectRoot; +} + +async function createLegacyProject(): Promise<{ + readonly projectRoot: string; + readonly storePath: string; +}> { + const base = requireRoot(); + const projectRoot = join(base, "legacy"); + const projectDir = join(projectRoot, ".hack"); + const storePath = join(base, "secrets.enc.json"); + await mkdir(projectDir, { recursive: true }); + await writeFile(join(projectDir, "docker-compose.yml"), "services: {}\n"); + await writeFile( + join(projectDir, "hack.config.json"), + `${JSON.stringify( + { + name: "key-redaction-legacy", + controlPlane: { + secrets: { + backend: "encrypted_file", + encryptedFile: { path: storePath }, + }, + }, + }, + null, + 2 + )}\n` + ); + await writeFile( + join(projectDir, "hack.env.json"), + `${JSON.stringify( + { + version: 1, + vars: [{ key: "API_KEY", required: false, source: "keychain" }], + }, + null, + 2 + )}\n` + ); + return { projectRoot, storePath }; +} + +async function readTree(paths: readonly string[]): Promise { + const contents = await Promise.all( + paths.map((path) => readFile(path, "utf8").catch(() => "")) + ); + return contents.join("\n--\n"); +} + +function modernFiles(projectRoot: string): readonly string[] { + return [ + "hack.env.default.yaml", + "hack.env.local.yaml", + ".env", + "hack.config.json", + ].map((file) => join(projectRoot, ".hack", file)); +} + +test( + "modern env add/set/unset/explain refuse KEY=value without echoing any of it", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + const assignment = `API_TOKEN=${CANARY}`; + // A dotted value would previously split into a scope that carried the + // value prefix into "Invalid env scope: API_TOKEN=...". + const dottedAssignment = `API_TOKEN=${CANARY}.tail`; + const cases: readonly (readonly string[])[] = [ + ["env", "add", assignment], + ["env", "add", assignment, "--secret"], + ["env", "add", dottedAssignment], + ["env", "add", assignment, "--service", "api"], + ["env", "set", assignment], + ["env", "unset", assignment], + ["env", "unset", dottedAssignment], + ["env", "explain", assignment], + ]; + for (const args of cases) { + const raw = args[2] ?? ""; + const result = await runHack({ args: [...args, "--path", projectRoot] }); + expectRefusedWithoutEcho({ + result, + message: ASSIGNMENT_KEY, + canary: CANARY, + rawArgument: raw, + }); + } + expect(await readTree(modernFiles(projectRoot))).toBe(before); + }, + { timeout: 90_000 } +); + +test( + "modern env commands refuse a value pasted as the key without echoing it", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + const cases: readonly { + readonly args: readonly string[]; + readonly message: string; + readonly canary: string; + }[] = [ + { args: ["env", "add", CANARY], message: INVALID_KEY, canary: CANARY }, + { + args: ["env", "add", CANARY, "positional-value"], + message: INVALID_KEY, + canary: CANARY, + }, + { + args: ["env", "add", `api.${CANARY}`], + message: INVALID_KEY, + canary: CANARY, + }, + { + args: ["env", "add", `${SCOPE_CANARY}.API_TOKEN`, "value"], + message: INVALID_SCOPE, + canary: SCOPE_CANARY, + }, + { + args: ["env", "add", "API_TOKEN", "value", "--service", SCOPE_CANARY], + message: INVALID_SCOPE, + canary: SCOPE_CANARY, + }, + { args: ["env", "set", CANARY], message: INVALID_KEY, canary: CANARY }, + { args: ["env", "unset", CANARY], message: INVALID_KEY, canary: CANARY }, + // `env get` replaces every failure with one generic message. + { args: ["env", "get", CANARY], message: GET_REFUSAL, canary: CANARY }, + { + args: ["env", "get", "API_TOKEN", "--service", SCOPE_CANARY], + message: GET_REFUSAL, + canary: SCOPE_CANARY, + }, + { + args: ["env", "explain", CANARY], + message: INVALID_KEY, + canary: CANARY, + }, + ]; + for (const entry of cases) { + const result = await runHack({ + args: [...entry.args, "--path", projectRoot], + }); + expectRefusedWithoutEcho({ + result, + message: entry.message, + canary: entry.canary, + rawArgument: entry.args[2] ?? "", + }); + } + expect(await readTree(modernFiles(projectRoot))).toBe(before); + }, + { timeout: 120_000 } +); + +test( + "a refused key argument is rejected before stdin is read or files change", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + // Empty stdin would be refused as "empty value" if it were read first, so + // seeing the key refusal proves the key is validated before stdin. + const result = await runHack({ + args: ["env", "add", CANARY, "--secret", "--value-stdin"], + stdin: "", + cwd: projectRoot, + }); + expectRefusedWithoutEcho({ + result, + message: INVALID_KEY, + canary: CANARY, + rawArgument: CANARY, + }); + expect(result.stderr).not.toContain( + "--value-stdin received an empty value" + ); + expect(await readTree(modernFiles(projectRoot))).toBe(before); + }, + { timeout: 30_000 } +); + +test( + "legacy env set and unset refuse an invalid key without echoing it or touching the store", + async () => { + const { projectRoot, storePath } = await createLegacyProject(); + const files = [ + storePath, + join(projectRoot, ".hack", ".env"), + join(projectRoot, ".hack", "hack.env.json"), + ]; + const before = await readTree(files); + const cases: readonly (readonly string[])[] = [ + ["env", "set", CANARY], + ["env", "set", CANARY, "positional-value"], + ["env", "set", `${CANARY}=positional-value`], + ["env", "set", CANARY, "--secret"], + ["env", "unset", CANARY], + ]; + for (const args of cases) { + const result = await runHack({ args, cwd: projectRoot }); + expectRefusedWithoutEcho({ + result, + message: INVALID_KEY, + canary: CANARY, + rawArgument: args[2] ?? "", + }); + } + expect(await readTree(files)).toBe(before); + }, + { timeout: 90_000 } +); + +test( + "a malformed key in an env config file is reported without echoing the key", + async () => { + const projectRoot = await createModernProject(); + await writeFile( + join(projectRoot, ".hack", "hack.env.default.yaml"), + `version: 1\nenvironment: default\nsecretsprovider: project_key\nvalues:\n global:\n "API_TOKEN=${CANARY}": "1"\n` + ); + const result = await runHack({ + args: ["env", "list", "--path", projectRoot], + }); + expectRefusedWithoutEcho({ + result, + message: 'Invalid env key in scope "global"', + canary: CANARY, + rawArgument: `API_TOKEN=${CANARY}`, + }); + }, + { timeout: 30_000 } +); diff --git a/tests/project-env-config.test.ts b/tests/project-env-config.test.ts index b0a555c9..44e27f3b 100644 --- a/tests/project-env-config.test.ts +++ b/tests/project-env-config.test.ts @@ -405,6 +405,51 @@ test("project env scopes accept compose-compatible underscores and dots", () => }); }); +test("env key and scope refusals never echo the rejected argument", async () => { + const canary = "echo-canary-lib-5e1c"; + const refusal = (fn: () => unknown): string => { + try { + fn(); + } catch (error: unknown) { + return error instanceof Error ? error.message : String(error); + } + throw new Error("expected a refusal"); + }; + const messages = [ + refusal(() => parseProjectEnvTarget({ keyOrPath: `API_TOKEN=${canary}` })), + refusal(() => + parseProjectEnvTarget({ keyOrPath: `API_TOKEN=${canary}.tail` }) + ), + refusal(() => parseProjectEnvTarget({ keyOrPath: canary })), + refusal(() => parseProjectEnvTarget({ keyOrPath: `api.${canary}` })), + refusal(() => + parseProjectEnvTarget({ keyOrPath: `Bad${canary}.API_TOKEN` }) + ), + refusal(() => + assertValidProjectEnvScopeName({ scopeName: `Bad${canary}` }) + ), + ]; + for (const message of messages) { + expect(message).toMatch(/^Invalid env (key|scope): /); + expect(message).not.toContain(canary); + expect(message).not.toContain("API_TOKEN="); + } + + const repo = await createRepo(); + const write = setProjectEnvValue({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + scope: "global", + key: `API_TOKEN=${canary}`, + value: canary, + secret: true, + }); + await expect(write).rejects.toThrow("Invalid env key: keys must match"); + const error = await write.catch((caught: unknown) => caught); + expect(error instanceof Error ? error.message : "").not.toContain(canary); +}); + test("resolveProjectEnvConfig falls back to HACK_ENV_SECRET_KEY when the key file is missing", async () => { const repo = await createRepo(); From 4acac8132bb1d83c057b267162c21a15806a0b16 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 09:31:11 -0400 Subject: [PATCH 50/52] fix(env): refuse env scope arguments without echoing them `hack env materialize --service API_TOKEN=` printed "Unknown env scope: API_TOKEN=". The same raw --service/--scope value reached scope selection from env exec, env shell, host exec and host shell, and env apply forwarded it to a `hack restart` child process. - New src/cli/env-arguments.ts parses every user-supplied scope with assertValidProjectEnvScopeName and refuses with fixed text before project migration, file access or any child process. env materialize, exec, shell, list, apply, get and host exec/shell use it. - selectProjectEnvValues and resolveProjectEnvValue report an unknown scope with a fixed message that lists only configured scope names from compose and env files, never the request. Scope lookups use Object.hasOwn so names such as "constructor" are unknown rather than inherited. - env get key/scope parsing moved into parseEnvGetArguments so its refusals are tested directly; the command's generic error wrapper would otherwise hide a re-echo. Tests run the real CLI with canaries in the scope for each command, assert the canary is absent from stdout, stderr and spawned argv/env, that no non-git child starts, and that project files are unchanged. Co-Authored-By: Claude Opus 5.5 --- src/cli/env-arguments.ts | 101 ++++++++++ src/commands/env.ts | 74 +++---- src/lib/project-env-config.ts | 49 ++++- tests/env-command-modern.test.ts | 4 +- tests/env-key-argument-redaction-cli.test.ts | 193 ++++++++++++++++++- tests/project-env-config.test.ts | 56 +++++- 6 files changed, 417 insertions(+), 60 deletions(-) create mode 100644 src/cli/env-arguments.ts diff --git a/src/cli/env-arguments.ts b/src/cli/env-arguments.ts new file mode 100644 index 00000000..1423e2fd --- /dev/null +++ b/src/cli/env-arguments.ts @@ -0,0 +1,101 @@ +import { + assertValidProjectEnvScopeName, + ENV_KEY_ASSIGNMENT_MESSAGE, + ENV_KEY_INVALID_MESSAGE, + ENV_SCOPE_INVALID_MESSAGE, +} from "../lib/project-env-config.ts"; +import { CliUsageError } from "./command.ts"; + +const ENV_KEY_PATTERN = /^[A-Z_][A-Z0-9_]*$/; +const SCOPE_CONFLICT_MESSAGE = "Use either --scope or --service, not both."; +const APPLY_SERVICE_REQUIRED_MESSAGE = + "--service is required."; + +/** + * Parsers for the env-scope and env-key arguments of `hack env` and + * `hack host` commands. + * + * Invariant: every refusal is fixed text. A scope or key argument is often a + * mistyped `KEY=value` or a value pasted into the wrong position, so no error + * raised here may contain any part of the argument. Callers must parse + * user-supplied scopes here before resolving env files, reading stdin, or + * spawning a child process, so a refused argument never reaches another + * component that might echo it. + */ + +/** Normalize a `--service`/`--scope` value; empty means `global`. */ +export function parseEnvScopeOption(input: { + readonly scopeName: string | undefined; +}): string { + try { + return assertValidProjectEnvScopeName({ scopeName: input.scopeName }); + } catch { + throw new CliUsageError(ENV_SCOPE_INVALID_MESSAGE); + } +} + +/** + * Resolve the `--scope`/`--service` pair of host execution commands. + * Returns `null` when neither names a scope (the global scope). + */ +export function parseExecutionScopeOptions(input: { + readonly scopeName: string | undefined; + readonly serviceName: string | undefined; +}): string | null { + const scopeName = input.scopeName?.trim() ?? ""; + const serviceName = input.serviceName?.trim() ?? ""; + if (scopeName.length > 0 && serviceName.length > 0) { + throw new CliUsageError(SCOPE_CONFLICT_MESSAGE); + } + const requested = scopeName || serviceName; + if (requested.length === 0) { + return null; + } + return parseEnvScopeOption({ scopeName: requested }); +} + +/** `hack env materialize --service`: omitted or empty is `null` (global). */ +export function parseMaterializeScopeOption(input: { + readonly serviceName: string | undefined; +}): string | null { + if ((input.serviceName?.trim() ?? "").length === 0) { + return null; + } + return parseEnvScopeOption({ scopeName: input.serviceName }); +} + +/** `hack env apply --service`: a single non-global service scope. */ +export function parseApplyServiceOption(input: { + readonly serviceName: string | undefined; +}): string { + if ((input.serviceName?.trim() ?? "").length === 0) { + throw new CliUsageError(APPLY_SERVICE_REQUIRED_MESSAGE); + } + const scope = parseEnvScopeOption({ scopeName: input.serviceName }); + if (scope === "global") { + throw new CliUsageError(APPLY_SERVICE_REQUIRED_MESSAGE); + } + return scope; +} + +/** + * `hack env get [--service ]`. Validated here, before any project + * or backend access. The command also replaces every failure with one generic + * message, but these refusals stay value-free on their own so that removing + * that wrapper cannot reintroduce an echo. + */ +export function parseEnvGetArguments(input: { + readonly key: string; + readonly serviceName: string | undefined; +}): { readonly key: string; readonly scope: string } { + if (input.key.includes("=")) { + throw new CliUsageError(ENV_KEY_ASSIGNMENT_MESSAGE); + } + if (!ENV_KEY_PATTERN.test(input.key)) { + throw new CliUsageError(ENV_KEY_INVALID_MESSAGE); + } + return { + key: input.key, + scope: parseEnvScopeOption({ scopeName: input.serviceName }), + }; +} diff --git a/src/commands/env.ts b/src/commands/env.ts index 5f60870d..9091914b 100644 --- a/src/commands/env.ts +++ b/src/commands/env.ts @@ -9,6 +9,13 @@ import { defineOption, withHandler, } from "../cli/command.ts"; +import { + parseApplyServiceOption, + parseEnvGetArguments, + parseEnvScopeOption, + parseExecutionScopeOptions, + parseMaterializeScopeOption, +} from "../cli/env-arguments.ts"; import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPath, optProject } from "../cli/options.ts"; import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; @@ -59,7 +66,6 @@ import { sanitizeProjectSlug, } from "../lib/project.ts"; import { - assertValidProjectEnvScopeName, discoverComposeServiceNames, ENV_KEY_INVALID_MESSAGE, materializeProjectEnv, @@ -1017,12 +1023,9 @@ const handleEnvGet: CommandHandlerFor = async ({ ctx, args, }) => { - const key = args.positionals.key; - if (!ENV_KEY_PATTERN.test(key)) { - throw new CliUsageError(ENV_KEY_INVALID_MESSAGE); - } - const scope = assertValidProjectEnvScopeName({ - scopeName: args.options.service, + const { key, scope } = parseEnvGetArguments({ + key: args.positionals.key, + serviceName: args.options.service, }); const project = await resolveProjectForEnv({ ctx, @@ -1084,17 +1087,9 @@ const handleEnvList: CommandHandlerFor = async ({ envName, }); if (modern) { - const selectedScope = (() => { - try { - return assertValidProjectEnvScopeName({ - scopeName: args.options.service, - }); - } catch (error: unknown) { - throw new CliUsageError( - error instanceof Error ? error.message : "Invalid env scope." - ); - } - })(); + const selectedScope = parseEnvScopeOption({ + scopeName: args.options.service, + }); const envValues = selectProjectEnvValues({ resolved: modern, scopeName: selectedScope, @@ -1367,10 +1362,9 @@ const handleEnvApply: CommandHandlerFor = async ({ ctx, args, }): Promise => { - const service = args.options.service?.trim() ?? ""; - if (service.length === 0 || service === "global") { - throw new CliUsageError("--service is required."); - } + const service = parseApplyServiceOption({ + serviceName: args.options.service, + }); const project = await resolveProjectForEnv({ ctx, pathOpt: args.options.path, @@ -1934,6 +1928,10 @@ const handleEnvMaterialize: CommandHandlerFor = async ({ ctx, args, }): Promise => { + // Validate before migration or file access so a refused scope changes nothing. + const serviceName = parseMaterializeScopeOption({ + serviceName: args.options.service, + }); const project = await resolveProjectForEnv({ ctx, pathOpt: args.options.path, @@ -1955,7 +1953,7 @@ const handleEnvMaterialize: CommandHandlerFor = async ({ projectRoot: project.projectRoot, projectDir: project.projectDir, envName, - serviceName: args.options.service?.trim() || null, + serviceName, serviceNames, }); logger.success({ @@ -1975,18 +1973,6 @@ function resolveShellCommandCommand(input: { return ["/bin/sh", "-lc", input.command]; } -function resolveExecutionScopeName(input: { - readonly scopeName?: string; - readonly serviceName?: string; -}): string | null { - const scopeName = input.scopeName?.trim(); - const serviceName = input.serviceName?.trim(); - if (scopeName && serviceName) { - throw new CliUsageError("Use either --scope or --service, not both."); - } - return scopeName || serviceName || null; -} - async function runHostCommandWithInjectedEnv(input: { readonly ctx: CliContext; readonly pathOpt: string | undefined; @@ -2000,6 +1986,10 @@ async function runHostCommandWithInjectedEnv(input: { readonly timeout?: number; readonly lifetime?: string; }): Promise { + const executionScope = parseExecutionScopeOptions({ + scopeName: input.scopeName, + serviceName: input.serviceName, + }); if ( input.timeout !== undefined && (!Number.isFinite(input.timeout) || @@ -2051,10 +2041,7 @@ async function runHostCommandWithInjectedEnv(input: { project, projectName, envName, - serviceName: resolveExecutionScopeName({ - scopeName: input.scopeName, - serviceName: input.serviceName, - }), + serviceName: executionScope, target, }); const declaredLifetime = @@ -2085,6 +2072,10 @@ async function openHostShellWithInjectedEnv(input: { readonly serviceName?: string; readonly targetOpt: string | undefined; }): Promise { + const executionScope = parseExecutionScopeOptions({ + scopeName: input.scopeName, + serviceName: input.serviceName, + }); const project = await resolveProjectForEnv({ ctx: input.ctx, pathOpt: input.pathOpt, @@ -2101,10 +2092,7 @@ async function openHostShellWithInjectedEnv(input: { project, projectName, envName, - serviceName: resolveExecutionScopeName({ - scopeName: input.scopeName, - serviceName: input.serviceName, - }), + serviceName: executionScope, target, }); diff --git a/src/lib/project-env-config.ts b/src/lib/project-env-config.ts index f12519b1..c17bffc2 100644 --- a/src/lib/project-env-config.ts +++ b/src/lib/project-env-config.ts @@ -95,6 +95,9 @@ export const ENV_KEY_ASSIGNMENT_MESSAGE = 'Invalid env key: the key argument cannot contain "=". Pass only the key (matching ^[A-Z_][A-Z0-9_]*$) and supply secret values with --value-stdin or the hidden prompt. The argument is not shown because it may contain a value.'; export const ENV_SCOPE_INVALID_MESSAGE = 'Invalid env scope: scopes must be "global" or match ^[a-z0-9][a-z0-9._-]*$ (usually a compose service name). The argument is not shown because it may contain a value.'; +export const ENV_SCOPE_UNKNOWN_MESSAGE = + "Unknown env scope: the requested scope is not global, a compose service, or a scope in the project env files. The argument is not shown because it may contain a value."; +const UNKNOWN_SCOPE_LIST_LIMIT = 20; const PROJECT_ENV_SECRET_PREFIX = "v1" as const; const PROJECT_ENV_ALGORITHM = "aes-256-gcm"; const PROJECT_ENV_IV_BYTES = 12; @@ -185,14 +188,42 @@ export function selectProjectEnvValues(opts: { return { ...opts.resolved.globalEnv }; } - const scoped = opts.resolved.serviceEnv[scopeName]; + const scoped = Object.hasOwn(opts.resolved.serviceEnv, scopeName) + ? opts.resolved.serviceEnv[scopeName] + : undefined; if (!scoped) { - throw new Error(`Unknown env scope: ${scopeName}`); + throw new Error( + unknownProjectEnvScopeMessage({ + configuredScopes: Object.keys(opts.resolved.serviceEnv), + }) + ); } return { ...scoped }; } +/** + * Fixed refusal for a scope that is not configured. It may list scope names + * taken from the project's compose and env files, but never the requested + * argument: a mistyped `--service KEY=value` must not be echoed. + */ +export function unknownProjectEnvScopeMessage(opts: { + readonly configuredScopes: readonly string[]; +}): string { + const configured = [ + "global", + ...[...new Set(opts.configuredScopes)] + .filter((scope) => scope !== "global") + .sort(), + ]; + const shown = configured.slice(0, UNKNOWN_SCOPE_LIST_LIMIT); + const more = + configured.length > shown.length + ? ` (and ${configured.length - shown.length} more)` + : ""; + return `${ENV_SCOPE_UNKNOWN_MESSAGE} Configured scopes: ${shown.join(", ")}${more}.`; +} + export function selectProjectEnvValuesForExecutionTarget(opts: { readonly resolved: ProjectEnvResolvedConfig; readonly scopeName?: string | null; @@ -208,7 +239,10 @@ export function selectProjectEnvValuesForExecutionTarget(opts: { const scopeName = normalizeProjectEnvScopeName({ scopeName: opts.scopeName, }); - return { ...(opts.resolved.hostTargetEnv[scopeName] ?? selected) }; + const hostTarget = Object.hasOwn(opts.resolved.hostTargetEnv, scopeName) + ? opts.resolved.hostTargetEnv[scopeName] + : undefined; + return { ...(hostTarget ?? selected) }; } export function isValidProjectEnvScopeName(opts: { @@ -947,7 +981,14 @@ export async function resolveProjectEnvValue(opts: { !opts.serviceNames.includes(scope) && !Object.hasOwn(layers.merged.values, scope) ) { - throw new Error("Unknown env scope"); + throw new Error( + unknownProjectEnvScopeMessage({ + configuredScopes: [ + ...opts.serviceNames, + ...Object.keys(layers.merged.values), + ], + }) + ); } const entries = resolveEffectiveStoredEntries({ layers: layers.envLayers, diff --git a/tests/env-command-modern.test.ts b/tests/env-command-modern.test.ts index 61956952..f7d3e505 100644 --- a/tests/env-command-modern.test.ts +++ b/tests/env-command-modern.test.ts @@ -88,7 +88,9 @@ test("modern env list rejects unknown service scopes", async () => { ]); expect(result.exitCode).toBe(1); - expect(result.stderr).toContain("Unknown env scope: typo_service"); + expect(result.stderr).toContain("Unknown env scope:"); + expect(result.stderr).toContain("Configured scopes: global, api"); + expect(result.stderr).not.toContain("typo_service"); }); test("env explain reports precedence and delivery without revealing values", async () => { diff --git a/tests/env-key-argument-redaction-cli.test.ts b/tests/env-key-argument-redaction-cli.test.ts index 3c4ce87c..0358001c 100644 --- a/tests/env-key-argument-redaction-cli.test.ts +++ b/tests/env-key-argument-redaction-cli.test.ts @@ -2,12 +2,23 @@ import { afterEach, beforeEach, expect, test } from "bun:test"; import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; +import { CliUsageError } from "../src/cli/command.ts"; +import { + parseEnvGetArguments, + parseExecutionScopeOptions, +} from "../src/cli/env-arguments.ts"; +import { + ENV_KEY_ASSIGNMENT_MESSAGE, + ENV_KEY_INVALID_MESSAGE, + ENV_SCOPE_INVALID_MESSAGE, +} from "../src/lib/project-env-config.ts"; /** - * Key-argument refusals in `hack env` must never echo the rejected argument. + * Key- and scope-argument refusals in `hack env` and `hack host` must never + * echo the rejected argument. * - * A key argument is often a mistyped `KEY=value` or a value pasted into the - * key position. Each case runs the real CLI in a fresh subprocess with a + * A key or `--service`/`--scope` argument is often a mistyped `KEY=value` or + * a value pasted into the wrong position. Each case runs the real CLI in a fresh subprocess with a * distinctive canary inside the argument and proves the canary is absent from * stdout, stderr, and the argv/env of every child process the CLI spawns. * @@ -359,3 +370,179 @@ test( }, { timeout: 30_000 } ); + +const UNKNOWN_SCOPE = "Unknown env scope: the requested scope is not global"; + +/** Program names (argv[0]) of every recorded child process. */ +function spawnedPrograms(spawnLog: string): readonly string[] { + return spawnLog + .split("\n") + .filter((line) => line.trim().length > 0) + .map((line) => { + const record: unknown = JSON.parse(line); + const argv = + typeof record === "object" && record !== null && "argv" in record + ? record.argv + : null; + return Array.isArray(argv) ? String(argv[0]) : ""; + }); +} + +/** + * Every command that accepts a scope through --service or --scope. Options + * precede `--`, after which everything is the host command's own argv. + */ +function scopeOptionCases(input: { + readonly scope: string; + readonly projectRoot: string; +}): readonly (readonly string[])[] { + const path = ["--path", input.projectRoot]; + return [ + ["env", "materialize", "--service", input.scope, ...path], + ["env", "exec", "--service", input.scope, ...path, "--", "true"], + ["env", "shell", "--service", input.scope, ...path], + ["env", "list", "--service", input.scope, ...path], + ["env", "apply", "--service", input.scope, ...path], + ["host", "exec", "--scope", input.scope, ...path, "--", "true"], + ["host", "shell", "--scope", input.scope, ...path], + ]; +} + +test( + "scope options refuse a malformed scope without echoing it or touching files", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + for (const scope of [`API_TOKEN=${CANARY}`, SCOPE_CANARY]) { + const canary = scope === SCOPE_CANARY ? SCOPE_CANARY : CANARY; + for (const args of scopeOptionCases({ scope, projectRoot })) { + const result = await runHack({ args }); + expectRefusedWithoutEcho({ + result, + message: INVALID_SCOPE, + canary, + rawArgument: scope, + }); + // Refused before any shell, command, or restart child starts. + expect( + spawnedPrograms(result.spawnLog).filter((name) => name !== "git") + ).toEqual([]); + } + } + expect(await readTree(modernFiles(projectRoot))).toBe(before); + }, + { timeout: 180_000 } +); + +test( + "a well-formed but unknown scope is refused without echoing it", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + // CANARY matches the scope pattern, so it reaches scope resolution. The + // refusal may list configured scopes from project files, never the request. + const cases = scopeOptionCases({ scope: CANARY, projectRoot }).filter( + (args) => args[1] !== "apply" + ); + for (const args of cases) { + const result = await runHack({ args }); + expectRefusedWithoutEcho({ + result, + message: UNKNOWN_SCOPE, + canary: CANARY, + rawArgument: CANARY, + }); + expect(result.stderr).toContain("Configured scopes: global, api."); + expect( + spawnedPrograms(result.spawnLog).filter((name) => name !== "git") + ).toEqual([]); + } + const get = await runHack({ + args: ["env", "get", "GLOBAL_FLAG", "--service", CANARY], + cwd: projectRoot, + }); + expectRefusedWithoutEcho({ + result: get, + message: GET_REFUSAL, + canary: CANARY, + rawArgument: CANARY, + }); + expect(await readTree(modernFiles(projectRoot))).toBe(before); + }, + { timeout: 120_000 } +); + +test("env get argument refusals are value-free before the generic wrapper", () => { + // The CLI replaces every `env get` failure with one generic message, which + // would hide a regression in the handler's own refusals. Assert the parse + // layer the handler uses so a re-echo of the key or scope is still caught. + const cases: readonly { + readonly key: string; + readonly serviceName?: string; + readonly message: string; + readonly canary: string; + }[] = [ + { key: CANARY, message: ENV_KEY_INVALID_MESSAGE, canary: CANARY }, + { + key: `API_TOKEN=${CANARY}`, + message: ENV_KEY_ASSIGNMENT_MESSAGE, + canary: CANARY, + }, + { + key: "API_TOKEN", + serviceName: SCOPE_CANARY, + message: ENV_SCOPE_INVALID_MESSAGE, + canary: SCOPE_CANARY, + }, + { + key: "API_TOKEN", + serviceName: `api=${CANARY}`, + message: ENV_SCOPE_INVALID_MESSAGE, + canary: CANARY, + }, + ]; + for (const entry of cases) { + let thrown: unknown = null; + try { + parseEnvGetArguments({ + key: entry.key, + serviceName: entry.serviceName, + }); + } catch (error: unknown) { + thrown = error; + } + expect(thrown).toBeInstanceOf(CliUsageError); + const message = (thrown as Error).message; + expect(message).toBe(entry.message); + expect(message).not.toContain(entry.canary); + } + expect( + parseEnvGetArguments({ key: "API_TOKEN", serviceName: " api " }) + ).toEqual({ key: "API_TOKEN", scope: "api" }); + expect( + parseEnvGetArguments({ key: "API_TOKEN", serviceName: undefined }) + ).toEqual({ key: "API_TOKEN", scope: "global" }); +}); + +test("execution scope parsing refuses conflicts and malformed scopes value-free", () => { + expect( + parseExecutionScopeOptions({ scopeName: undefined, serviceName: " " }) + ).toBeNull(); + expect( + parseExecutionScopeOptions({ scopeName: "api", serviceName: undefined }) + ).toBe("api"); + for (const input of [ + { scopeName: `API_TOKEN=${CANARY}`, serviceName: undefined }, + { scopeName: undefined, serviceName: `API_TOKEN=${CANARY}` }, + { scopeName: "api", serviceName: CANARY }, + ]) { + let message = ""; + try { + parseExecutionScopeOptions(input); + } catch (error: unknown) { + message = error instanceof Error ? error.message : String(error); + } + expect(message.length).toBeGreaterThan(0); + expect(message).not.toContain(CANARY); + } +}); diff --git a/tests/project-env-config.test.ts b/tests/project-env-config.test.ts index 44e27f3b..2d4c10fa 100644 --- a/tests/project-env-config.test.ts +++ b/tests/project-env-config.test.ts @@ -25,6 +25,7 @@ import { upsertDotEnvValue } from "../src/lib/hack-env.ts"; import { readProjectDefaultEnvConfig } from "../src/lib/project.ts"; import { assertValidProjectEnvScopeName, + ENV_SCOPE_UNKNOWN_MESSAGE, ensureHackDirGitignore, ensureProjectEnvSecretKey, inspectLegacyComposeEnvFileReferences, @@ -38,8 +39,10 @@ import { resolveProjectEnvConfig, resolveProjectEnvLocalConfigPath, resolveProjectEnvSharedKeyPath, + selectProjectEnvValues, selectProjectEnvValuesForExecutionTarget, setProjectEnvValue, + unknownProjectEnvScopeMessage, unsetProjectEnvValue, } from "../src/lib/project-env-config.ts"; import { resolveSecretStore } from "../src/lib/secret-store.ts"; @@ -378,15 +381,50 @@ test("materializeProjectEnv rejects unknown service scopes", async () => { secret: false, }); - await expect( - materializeProjectEnv({ - projectRoot: repo.projectRoot, - projectDir: repo.projectDir, - envName: null, - serviceName: "typo_service", - serviceNames: ["api", "web"], - }) - ).rejects.toThrow("Unknown env scope: typo_service"); + const error = await materializeProjectEnv({ + projectRoot: repo.projectRoot, + projectDir: repo.projectDir, + envName: null, + serviceName: "typo_service", + serviceNames: ["api", "web"], + }).then( + () => null, + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Error); + const message = (error as Error).message; + // Configured scope names come from compose/env files and may be listed; + // the requested argument never is. + expect(message).toBe( + `${ENV_SCOPE_UNKNOWN_MESSAGE} Configured scopes: global, api, web.` + ); + expect(message).not.toContain("typo_service"); +}); + +test("unknown-scope refusals list configured scopes but never the request", () => { + const canary = "api_token=unknown-scope-canary-7c2e"; + const resolved = { + globalEnv: {}, + serviceEnv: { web: {}, api: {} }, + hostTargetEnv: {}, + } as unknown as Parameters[0]["resolved"]; + for (const scopeName of [canary, "constructor", "__proto__", "toString"]) { + let message = ""; + try { + selectProjectEnvValues({ resolved, scopeName }); + } catch (error: unknown) { + message = error instanceof Error ? error.message : String(error); + } + expect(message).toBe( + `${ENV_SCOPE_UNKNOWN_MESSAGE} Configured scopes: global, api, web.` + ); + expect(message).not.toContain(scopeName); + } + + const many = unknownProjectEnvScopeMessage({ + configuredScopes: Array.from({ length: 30 }, (_, i) => `svc${i + 100}`), + }); + expect(many).toContain("(and 11 more)"); }); test("project env scopes accept compose-compatible underscores and dots", () => { From b1bc6db2af72c6febc18bac67663cc88abc3cccf Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 10:02:07 -0400 Subject: [PATCH 51/52] fix(env): refuse value-bearing --env, --project and --target without echoing them `--env X=` was slugged into the overlay name `x`, printed as env_selection, and `hack env add ... --env X=` created .hack/hack.env.x.yaml. `--project x=` printed Unknown project "x", and `--target x=` was echoed verbatim. - parseEnvConfigSelection no longer slugs a requested overlay name. Apart from surrounding whitespace, a value must already match ^[a-z0-9]+(-[a-z0-9]+)*$; base/default/none (any case) and an empty value still select base layers. The CLI and the daemon env routes share it. - resolveRequestedEnvName refuses with one fixed message that names the grammar, never the argument. runCli resolves the shared --env option before any handler runs, so every command that accepts --env refuses before project discovery, migration, file access, stdin or a child process. - env apply forwards the validated selection to `hack restart`, never the raw argument (an empty --env now forwards `base`, matching the grammar). - env/host --project is parsed by parseProjectSelectorOption: an argument containing "=" or characters outside the selector pattern (letters, digits, -, _, space, /) is refused with fixed text before the registry is read. Legacy alias spellings still select their slug, but Unknown project only echoes a name typed in canonical form. - --target refusals list the allowed values only. env explain, exec, shell and host exec/shell validate --env and --target before resolving the project. - docs/env.md states the overlay name rule instead of describing normalization. Tests enumerate every CLI command that accepts --env (and every env/host command with --project) from CLI_SPEC and run the real CLI with canaries in --env, --project and --target. They assert the canary (including its slugged, lowercased form) is absent from stdout, stderr and spawned argv/env, that no non-git child starts, that no file whose name contains the canary exists, and that project files are unchanged. The --env refusal is checked from a directory without .hack/ to prove it precedes project discovery. Co-Authored-By: Claude Opus 5.5 --- docs/env.md | 13 +- src/cli/env-arguments.ts | 27 ++ src/cli/env-selection.ts | 39 +- src/cli/run.ts | 24 ++ src/commands/env.ts | 74 ++-- src/lib/project-name.ts | 11 + src/lib/project.ts | 20 +- tests/env-key-argument-redaction-cli.test.ts | 352 ++++++++++++++++++- tests/env-selection-parity.test.ts | 22 +- 9 files changed, 517 insertions(+), 65 deletions(-) diff --git a/docs/env.md b/docs/env.md index 75822d81..930761c5 100644 --- a/docs/env.md +++ b/docs/env.md @@ -139,8 +139,11 @@ Projects can set a default overlay in `.hack/hack.config.json`: Use `--env=base` to bypass that default and read only `.hack/hack.env.default.yaml` (plus local overrides). `default` and `none` are aliases for `base`, in any case, on every command that accepts `--env`; an overlay file named after one of those spellings is never -selected by name. Other names are normalized (for example ` QA ` selects `qa`), and a -non-empty value with no usable name characters is refused with `Invalid --env value.` +selected by name. Any other value must already be a canonical overlay name: lowercase +letters and digits joined by single hyphens (`^[a-z0-9]+(-[a-z0-9]+)*$`, for example +`qa` or `qa-2`); surrounding whitespace is ignored. Hack never rewrites a requested name, +so `QA`, `q_a` or `X=1` is refused with a fixed `Invalid --env value.` message that does +not repeat the argument, before any project file is read or any child process starts. See the [command and delivery matrix](#command-and-delivery-matrix) for what each command does with the selection. @@ -216,9 +219,9 @@ For example, `hack host exec --scope api -- bun db:migrate` still runs on the ho Every command that accepts `--env` resolves it with the same rule: omitted selects the configured default overlay (`env.defaultOverlay`); `base`, `default` or `none` -selects base layers only; any other value selects that normalized overlay name. The -table lists every such command; `tests/env-command-matrix.test.ts` fails when a -command gains or loses `--env` without a matching row, or when a row's selectors +selects base layers only; any other value must be a canonical overlay name and selects +that overlay. The table lists every such command; `tests/env-command-matrix.test.ts` +fails when a command gains or loses `--env` without a matching row, or when a row's selectors drift from the CLI. Commands not listed here refuse `--env` as an option that is not valid for them (arguments after a standalone `--` belong to the child command). diff --git a/src/cli/env-arguments.ts b/src/cli/env-arguments.ts index 1423e2fd..a761b54e 100644 --- a/src/cli/env-arguments.ts +++ b/src/cli/env-arguments.ts @@ -4,12 +4,19 @@ import { ENV_KEY_INVALID_MESSAGE, ENV_SCOPE_INVALID_MESSAGE, } from "../lib/project-env-config.ts"; +import { + normalizeProjectName, + PROJECT_NAME_PATTERN, + PROJECT_SELECTOR_PATTERN, +} from "../lib/project-name.ts"; import { CliUsageError } from "./command.ts"; const ENV_KEY_PATTERN = /^[A-Z_][A-Z0-9_]*$/; const SCOPE_CONFLICT_MESSAGE = "Use either --scope or --service, not both."; const APPLY_SERVICE_REQUIRED_MESSAGE = "--service is required."; +export const INVALID_PROJECT_OPTION_MESSAGE = + "Invalid --project value. Project names use ASCII letters, digits, hyphens, underscores, spaces or slashes; run 'hack projects' to see registered projects."; /** * Parsers for the env-scope and env-key arguments of `hack env` and @@ -99,3 +106,23 @@ export function parseEnvGetArguments(input: { scope: parseEnvScopeOption({ scopeName: input.serviceName }), }; } + +/** + * Parse `--project` for `hack env` and `hack host` commands. An argument + * containing `=` or any character outside `PROJECT_SELECTOR_PATTERN` may be a + * pasted value, so it is refused with fixed text before the registry is read + * and is never slugged into a name. `canonical` reports whether the argument + * was typed in canonical form, the only form an error may echo. + */ +export function parseProjectSelectorOption(input: { + readonly projectOpt: string; +}): { readonly name: string; readonly canonical: boolean } { + const trimmed = input.projectOpt.trim(); + const name = PROJECT_SELECTOR_PATTERN.test(trimmed) + ? normalizeProjectName(trimmed) + : null; + if (!name) { + throw new CliUsageError(INVALID_PROJECT_OPTION_MESSAGE); + } + return { name, canonical: PROJECT_NAME_PATTERN.test(trimmed) }; +} diff --git a/src/cli/env-selection.ts b/src/cli/env-selection.ts index e45b9d08..c88abf40 100644 --- a/src/cli/env-selection.ts +++ b/src/cli/env-selection.ts @@ -1,10 +1,13 @@ -import { - ENV_BASE_SELECTION_ALIASES, - parseEnvConfigSelection, -} from "../lib/project.ts"; +import { parseEnvConfigSelection } from "../lib/project.ts"; import { CliUsageError } from "./command.ts"; -const INVALID_ENV_SELECTION_MESSAGE = "Invalid --env value."; +/** + * Fixed refusal for an unusable `--env` value. It names the accepted grammar + * and never the argument, which may be a mistyped `KEY=value` or a pasted + * value. + */ +export const INVALID_ENV_SELECTION_MESSAGE = + "Invalid --env value. Overlay names must match ^[a-z0-9]+(-[a-z0-9]+)*$; use base, default or none for base layers only."; /** * Resolve a CLI `--env` value through the one shared selection grammar. @@ -12,30 +15,28 @@ const INVALID_ENV_SELECTION_MESSAGE = "Invalid --env value."; * - `undefined` (option omitted): callers apply the configured default overlay. * - `null`: base layers only. `base`, `default` and `none` (any case) and an * empty value bypass the configured default overlay. - * - `string`: the canonical overlay name (`normalizeEnvConfigName`). + * - `string`: the overlay name, which must already be canonical + * (`ENV_CONFIG_NAME_PATTERN`); surrounding whitespace is ignored. * - * A non-empty value with no usable overlay characters is refused with a fixed, - * value-free usage error instead of silently selecting the base layers. - * Every CLI command that accepts `--env` must resolve it here so that + * Anything else, including any value containing `=`, is refused with a fixed, + * value-free usage error. A requested name is never slugged: slugging + * `X=secret` to `xsecret` would print part of the value as the selection and + * write it into an overlay file name. + * + * `runCli` resolves `--env` once before any handler runs, so a refusal + * precedes project discovery, migration, file access and child processes. + * Every CLI command that accepts `--env` must also resolve it here so that * `default`/`base` mean the same layers everywhere. */ export function resolveRequestedEnvName(input: { readonly envOption: string | undefined; }): string | null | undefined { - const selection = parseEnvConfigSelection(input.envOption); if (input.envOption === undefined) { - return selection; + return undefined; } + const selection = parseEnvConfigSelection(input.envOption); if (selection === undefined) { throw new CliUsageError(INVALID_ENV_SELECTION_MESSAGE); } - const trimmed = input.envOption.trim(); - if ( - selection === null && - trimmed.length > 0 && - !ENV_BASE_SELECTION_ALIASES.has(trimmed.toLowerCase()) - ) { - throw new CliUsageError(INVALID_ENV_SELECTION_MESSAGE); - } return selection; } diff --git a/src/cli/run.ts b/src/cli/run.ts index 101c63e8..e7fdd714 100644 --- a/src/cli/run.ts +++ b/src/cli/run.ts @@ -18,7 +18,9 @@ import { parsePositionalsForCommand, resolveCommand, } from "./command.ts"; +import { resolveRequestedEnvName } from "./env-selection.ts"; import { printHelpForPath } from "./help.ts"; +import { optEnv } from "./options.ts"; import { CLI_SPEC } from "./spec.ts"; function isTruthyEnv(value: string | undefined): boolean { @@ -80,6 +82,10 @@ export async function runCli(argv: readonly string[]): Promise { resolved.command.options, parsed.values ); + validateSharedEnvSelection({ + command: resolved.command, + parsedValues: parsed.values, + }); const pos = parsePositionalsForCommand( resolved.command.positionals, resolved.remainingPositionals @@ -260,6 +266,24 @@ function validateResolvedCommandOptions(opts: { } } +/** + * Refuse an unusable shared `--env` value before the handler runs, so no + * command discovers a project, migrates or reads files, or starts a child + * process with it. The refusal is fixed text that never includes the value. + */ +function validateSharedEnvSelection(opts: { + readonly command: AnyCommandSpec; + readonly parsedValues: Record; +}): void { + if (!opts.command.options.includes(optEnv)) { + return; + } + const value = opts.parsedValues[optEnv.name]; + if (typeof value === "string") { + resolveRequestedEnvName({ envOption: value }); + } +} + function allowsUnknownOptions(opts: { readonly command: AnyCommandSpec | null; }): boolean { diff --git a/src/commands/env.ts b/src/commands/env.ts index 9091914b..68a8b46b 100644 --- a/src/commands/env.ts +++ b/src/commands/env.ts @@ -15,6 +15,7 @@ import { parseEnvScopeOption, parseExecutionScopeOptions, parseMaterializeScopeOption, + parseProjectSelectorOption, } from "../cli/env-arguments.ts"; import { resolveRequestedEnvName } from "../cli/env-selection.ts"; import { optEnv, optJson, optPath, optProject } from "../cli/options.ts"; @@ -80,7 +81,6 @@ import { setProjectEnvValue, unsetProjectEnvValue, } from "../lib/project-env-config.ts"; -import { normalizeProjectName } from "../lib/project-name.ts"; import { resolveRegisteredProjectByName } from "../lib/projects-registry.ts"; import { formatSecretStoreDescriptor, @@ -627,6 +627,9 @@ function normalizeGlobalEncryptedFileConfigPath(input: { return resolvedPath; } +const UNKNOWN_PROJECT_HINT = + "Run 'hack init' in that repo (or run 'hack projects' to see registered projects)."; + async function resolveProjectForEnv(opts: { readonly ctx: CliContext; readonly pathOpt: string | undefined; @@ -637,14 +640,19 @@ async function resolveProjectForEnv(opts: { } if (opts.projectOpt !== undefined) { - const name = normalizeProjectName(opts.projectOpt); - if (!name) { - throw new CliUsageError("Invalid --project value."); - } - const project = await resolveRegisteredProjectByName({ name }); + const selector = parseProjectSelectorOption({ + projectOpt: opts.projectOpt, + }); + const project = await resolveRegisteredProjectByName({ + name: selector.name, + }); if (!project) { + // Echo only a name the user typed in canonical form; never a slug + // derived from their input. throw new CliUsageError( - `Unknown project "${name}". Run 'hack init' in that repo (or run 'hack projects' to see registered projects).` + selector.canonical + ? `Unknown project "${selector.name}". ${UNKNOWN_PROJECT_HINT}` + : `Unknown project. ${UNKNOWN_PROJECT_HINT}` ); } return project; @@ -775,8 +783,9 @@ function resolveHostEnvTarget(input: { ) { return target as (typeof HOST_ENV_TARGET_VALUES)[number]; } + // Fixed text: the rejected argument may be a pasted value. throw new CliUsageError( - `Invalid --target "${target}". Expected one of: ${HOST_ENV_TARGET_VALUES.join(", ")}` + `Invalid --target value. Expected one of: ${HOST_ENV_TARGET_VALUES.join(", ")}.` ); } @@ -1293,18 +1302,19 @@ const handleEnvExplain: CommandHandlerFor = async ({ ctx, args, }): Promise => { - const project = await resolveProjectForEnv({ - ctx, - pathOpt: args.options.path, - projectOpt: args.options.project, - }); - const projectName = await resolveProjectName(project); + // Refuse malformed arguments with fixed text before any project file is read. const envName = resolveRequestedEnvName({ envOption: args.options.env }); const parsed = parseProjectEnvTarget({ keyOrPath: args.positionals.key, scopeOverride: args.options.service, }); const target = resolveHostEnvTarget({ targetOption: args.options.target }); + const project = await resolveProjectForEnv({ + ctx, + pathOpt: args.options.path, + projectOpt: args.options.project, + }); + const projectName = await resolveProjectName(project); const modern = await loadModernProjectEnv({ project, envName }); if (modern) { renderEnvExplain( @@ -1365,6 +1375,7 @@ const handleEnvApply: CommandHandlerFor = async ({ const service = parseApplyServiceOption({ serviceName: args.options.service, }); + const envName = resolveRequestedEnvName({ envOption: args.options.env }); const project = await resolveProjectForEnv({ ctx, pathOpt: args.options.path, @@ -1379,8 +1390,9 @@ const handleEnvApply: CommandHandlerFor = async ({ "--path", project.projectRoot, ]; - if (args.options.env) { - command.push("--env", args.options.env); + // Forward only the validated selection, never the raw argument. + if (envName !== undefined) { + command.push("--env", envName ?? "base"); } if (args.options.json) { command.push("--json"); @@ -1747,6 +1759,9 @@ const handleEnvAdd: CommandHandlerFor = async ({ keyOrPath: keyArg ?? "", scopeOverride: args.options.service, }); + const envName = resolveRequestedEnvName({ + envOption: args.options.env, + }); const stdinValue = valueStdin ? await readEnvValueFromStdin() : null; const project = await resolveProjectForEnv({ @@ -1755,9 +1770,6 @@ const handleEnvAdd: CommandHandlerFor = async ({ projectOpt: args.options.project, }); const projectName = await resolveProjectName(project); - const envName = resolveRequestedEnvName({ - envOption: args.options.env, - }); await maybeMigrateLegacyProjectEnv({ project, projectName, @@ -2016,18 +2028,18 @@ async function runHostCommandWithInjectedEnv(input: { "--timeout requires non-TTY stdin; pipe input or redirect from /dev/null for a finite job." ); } - const project = await resolveProjectForEnv({ - ctx: input.ctx, - pathOpt: input.pathOpt, - projectOpt: input.projectOpt, - }); - const projectName = await resolveProjectName(project); const envName = resolveRequestedEnvName({ envOption: input.envOpt, }); const target = resolveHostEnvTarget({ targetOption: input.targetOpt, }); + const project = await resolveProjectForEnv({ + ctx: input.ctx, + pathOpt: input.pathOpt, + projectOpt: input.projectOpt, + }); + const projectName = await resolveProjectName(project); const shellCommand = input.shellCommandOpt?.trim(); const positionalCommand = input.command; if (shellCommand && positionalCommand.length > 0) { @@ -2076,18 +2088,18 @@ async function openHostShellWithInjectedEnv(input: { scopeName: input.scopeName, serviceName: input.serviceName, }); - const project = await resolveProjectForEnv({ - ctx: input.ctx, - pathOpt: input.pathOpt, - projectOpt: input.projectOpt, - }); - const projectName = await resolveProjectName(project); const envName = resolveRequestedEnvName({ envOption: input.envOpt, }); const target = resolveHostEnvTarget({ targetOption: input.targetOpt, }); + const project = await resolveProjectForEnv({ + ctx: input.ctx, + pathOpt: input.pathOpt, + projectOpt: input.projectOpt, + }); + const projectName = await resolveProjectName(project); const envState = await resolveEnvInjection({ project, projectName, diff --git a/src/lib/project-name.ts b/src/lib/project-name.ts index 819a990b..ce7a89ed 100644 --- a/src/lib/project-name.ts +++ b/src/lib/project-name.ts @@ -16,6 +16,17 @@ export function normalizeProjectName(input: string): string | null { return name.length > 0 ? name : null; } +/** A canonical project name: the fixed point of `normalizeProjectName`. */ +export const PROJECT_NAME_PATTERN = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; + +/** + * A project selector a user may type: a canonical name or a legacy alias + * spelling (ASCII letters, digits, `-`, `_`, space and `/`). Anything else, + * including any argument containing `=`, may be a pasted value and must be + * refused with text that does not include it. + */ +export const PROJECT_SELECTOR_PATTERN = /^[A-Za-z0-9_ /-]+$/; + export class AmbiguousProjectNameError extends Error { constructor(name: string) { super( diff --git a/src/lib/project.ts b/src/lib/project.ts index 7a54cea5..c11630d3 100644 --- a/src/lib/project.ts +++ b/src/lib/project.ts @@ -133,6 +133,24 @@ export const ENV_BASE_SELECTION_ALIASES: ReadonlySet = new Set([ "none", ]); +/** + * Canonical overlay name: the fixed point of `normalizeEnvConfigName` + * (lowercase letters and digits joined by single hyphens). + */ +export const ENV_CONFIG_NAME_PATTERN = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; + +/** + * Parse a requested overlay selection (CLI `--env`, daemon `env`). + * + * - `undefined` input: no selection. + * - `null`: base layers only (empty, or a base alias in any case). + * - `string`: the overlay name, exactly as given apart from surrounding + * whitespace. + * - `undefined` for a non-empty input: refused. A requested name is never + * slugged, so `X=secret` or `QA` cannot silently become `xsecret` or `qa` + * and then be printed or written into an overlay file name. Callers must + * refuse with fixed text that does not include the input. + */ export function parseEnvConfigSelection( input: string | undefined ): string | null | undefined { @@ -149,7 +167,7 @@ export function parseEnvConfigSelection( return null; } - return normalizeEnvConfigName(trimmed); + return ENV_CONFIG_NAME_PATTERN.test(trimmed) ? trimmed : undefined; } export function sanitizeBranchSlug(input: string): string { diff --git a/tests/env-key-argument-redaction-cli.test.ts b/tests/env-key-argument-redaction-cli.test.ts index 0358001c..8bc3e16c 100644 --- a/tests/env-key-argument-redaction-cli.test.ts +++ b/tests/env-key-argument-redaction-cli.test.ts @@ -1,12 +1,25 @@ import { afterEach, beforeEach, expect, test } from "bun:test"; -import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { + mkdir, + mkdtemp, + readdir, + readFile, + rm, + writeFile, +} from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; -import { CliUsageError } from "../src/cli/command.ts"; +import { type AnyCommandSpec, CliUsageError } from "../src/cli/command.ts"; import { + INVALID_PROJECT_OPTION_MESSAGE, parseEnvGetArguments, parseExecutionScopeOptions, + parseProjectSelectorOption, } from "../src/cli/env-arguments.ts"; +import { INVALID_ENV_SELECTION_MESSAGE } from "../src/cli/env-selection.ts"; +import { optEnv, optProject } from "../src/cli/options.ts"; +import { CLI_SPEC } from "../src/cli/spec.ts"; +import { parseEnvConfigSelection } from "../src/lib/project.ts"; import { ENV_KEY_ASSIGNMENT_MESSAGE, ENV_KEY_INVALID_MESSAGE, @@ -119,6 +132,8 @@ function expectRefusedWithoutEcho(input: { for (const stream of [result.stdout, result.stderr, result.spawnLog]) { expect(stream).not.toContain(input.canary); expect(stream).not.toContain(input.rawArgument); + // A slugged echo (case-folded, punctuation stripped) is still an echo. + expect(stream.toLowerCase()).not.toContain(input.canary.toLowerCase()); } } @@ -546,3 +561,336 @@ test("execution scope parsing refuses conflicts and malformed scopes value-free" expect(message).not.toContain(CANARY); } }); + +/** Every file or directory name under `dir`, recursively. */ +async function listNames(dir: string): Promise { + const entries = await readdir(dir, { recursive: true }); + return entries.map((entry) => String(entry)); +} + +/** No file anywhere under the temp root may carry the canary in its name. */ +async function expectNoCanaryFileNames(canary: string): Promise { + const names = await listNames(requireRoot()); + expect( + names.filter((name) => name.toLowerCase().includes(canary.toLowerCase())) + ).toEqual([]); +} + +function expectNoNonGitChild(result: RunResult): void { + expect( + spawnedPrograms(result.spawnLog).filter((name) => name !== "git") + ).toEqual([]); +} + +/** Command paths (e.g. ["env", "get"]) whose spec includes `option`. */ +function commandPathsWithOption(option: unknown): readonly string[][] { + const out: string[][] = []; + const walk = ( + commands: readonly AnyCommandSpec[], + prefix: readonly string[] + ): void => { + for (const command of commands) { + const path = [...prefix, command.name]; + if (command.options.some((entry) => entry === option)) { + out.push(path); + } + walk(command.subcommands, path); + } + }; + walk(CLI_SPEC.commands, []); + return out; +} + +/** Arguments that let each env/host command reach its handler. */ +function handlerArguments(path: readonly string[]): readonly string[] { + switch (path.join(" ")) { + case "env get": + case "env explain": + case "env unset": + return ["API_TOKEN"]; + case "env add": + case "env set": + return ["API_TOKEN", "value"]; + case "env apply": + return ["--service", "api"]; + case "env exec": + case "host exec": + return ["--", "true"]; + default: + return []; + } +} + +/** `env get` replaces every failure with one fixed message. */ +function expectedRefusal(path: readonly string[], message: string): string { + return path.join(" ") === "env get" ? GET_REFUSAL : message; +} + +/** Value-bearing --env spellings: assignments and non-canonical names. */ +const ENV_CANARY_VALUES: readonly { + readonly value: string; + readonly canary: string; +}[] = [ + // Previously slugged to "xecho-canary-...", printed, and used as a file name. + { value: `X=${CANARY}`, canary: CANARY }, + // Previously case-folded to "echo-scope-canary-...". + { value: SCOPE_CANARY, canary: SCOPE_CANARY }, + { value: `${CANARY}.yaml`, canary: CANARY }, +]; + +test( + "every command accepting --env refuses a value-bearing --env before any project work", + async () => { + const paths = commandPathsWithOption(optEnv); + // Guard against the enumeration silently finding nothing. + expect(paths.map((path) => path.join(" "))).toEqual( + expect.arrayContaining([ + "up", + "down", + "restart", + "run", + "exec", + "env add", + "env apply", + "env get", + "host exec", + "config plan", + "session start", + ]) + ); + // No .hack/ here: any refusal other than the --env one would mean the + // command discovered (or failed to discover) a project before --env. + const empty = join(requireRoot(), "empty"); + await mkdir(empty, { recursive: true }); + for (const path of paths) { + for (const entry of ENV_CANARY_VALUES) { + const result = await runHack({ + args: [...path, "--env", entry.value], + cwd: empty, + }); + expectRefusedWithoutEcho({ + result, + message: expectedRefusal(path, INVALID_ENV_SELECTION_MESSAGE), + canary: entry.canary, + rawArgument: entry.value, + }); + expectNoNonGitChild(result); + } + } + await expectNoCanaryFileNames(CANARY); + await expectNoCanaryFileNames(SCOPE_CANARY); + }, + { timeout: 300_000 } +); + +test( + "env and host commands refuse a value-bearing --env in a real project without creating overlay files", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + const beforeNames = await listNames(join(projectRoot, ".hack")); + const paths = commandPathsWithOption(optEnv).filter( + (path) => path[0] === "env" || path[0] === "host" + ); + for (const path of paths) { + for (const entry of ENV_CANARY_VALUES) { + const result = await runHack({ + args: [ + ...path, + "--env", + entry.value, + "--path", + projectRoot, + ...handlerArguments(path), + ], + }); + expectRefusedWithoutEcho({ + result, + message: expectedRefusal(path, INVALID_ENV_SELECTION_MESSAGE), + canary: entry.canary, + rawArgument: entry.value, + }); + expectNoNonGitChild(result); + } + } + // `env add --value-stdin` refuses before reading stdin: an empty stdin + // would otherwise be refused as an empty value. + const stdinResult = await runHack({ + args: [ + "env", + "add", + "API_TOKEN", + "--secret", + "--value-stdin", + "--env", + `X=${CANARY}`, + ], + cwd: projectRoot, + stdin: "", + }); + expectRefusedWithoutEcho({ + result: stdinResult, + message: INVALID_ENV_SELECTION_MESSAGE, + canary: CANARY, + rawArgument: `X=${CANARY}`, + }); + expect(stdinResult.stderr).not.toContain("empty value"); + expect(await readTree(modernFiles(projectRoot))).toBe(before); + expect(await listNames(join(projectRoot, ".hack"))).toEqual(beforeNames); + await expectNoCanaryFileNames(CANARY); + await expectNoCanaryFileNames(SCOPE_CANARY); + + // The documented base aliases keep working. + for (const alias of ["base", "DEFAULT", "none", ""]) { + const result = await runHack({ + args: ["env", "list", "--env", alias, "--json", "--path", projectRoot], + }); + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain("GLOBAL_FLAG"); + } + }, + { timeout: 300_000 } +); + +test( + "env and host commands refuse a value-bearing --project without echoing it", + async () => { + const paths = commandPathsWithOption(optProject).filter( + // `host ps` only filters its listing by name and never echoes it. + (path) => + (path[0] === "env" || path[0] === "host") && + path.join(" ") !== "host ps" + ); + expect(paths.length).toBeGreaterThanOrEqual(12); + const cases: readonly { + readonly value: string; + readonly canary: string; + readonly message: string; + }[] = [ + // Previously printed as Unknown project "xecho-canary-...". + { + value: `x=${CANARY}`, + canary: CANARY, + message: INVALID_PROJECT_OPTION_MESSAGE, + }, + { + value: `${CANARY}.x`, + canary: CANARY, + message: INVALID_PROJECT_OPTION_MESSAGE, + }, + // A legacy alias spelling still selects by its slug, but an unknown + // non-canonical selector is not echoed back in slugged form. + { + value: SCOPE_CANARY, + canary: SCOPE_CANARY, + message: "Unknown project. Run 'hack init'", + }, + ]; + const cwd = join(requireRoot(), "empty"); + await mkdir(cwd, { recursive: true }); + for (const path of paths) { + for (const entry of cases) { + const result = await runHack({ + args: [...path, "--project", entry.value, ...handlerArguments(path)], + cwd, + }); + expectRefusedWithoutEcho({ + result, + message: expectedRefusal(path, entry.message), + canary: entry.canary, + rawArgument: entry.value, + }); + expectNoNonGitChild(result); + } + } + await expectNoCanaryFileNames(CANARY); + await expectNoCanaryFileNames(SCOPE_CANARY); + }, + { timeout: 300_000 } +); + +test( + "env and host commands refuse a value-bearing --target without echoing it", + async () => { + const projectRoot = await createModernProject(); + const before = await readTree(modernFiles(projectRoot)); + const paths = [ + ["env", "explain"], + ["env", "exec"], + ["env", "shell"], + ["host", "exec"], + ["host", "shell"], + ]; + for (const path of paths) { + for (const entry of [ + { value: `x=${CANARY}`, canary: CANARY }, + { value: SCOPE_CANARY, canary: SCOPE_CANARY }, + ]) { + const result = await runHack({ + args: [ + ...path, + "--target", + entry.value, + "--path", + projectRoot, + ...handlerArguments(path), + ], + }); + expectRefusedWithoutEcho({ + result, + message: "Invalid --target value. Expected one of: host, compose.", + canary: entry.canary, + rawArgument: entry.value, + }); + expectNoNonGitChild(result); + } + } + expect(await readTree(modernFiles(projectRoot))).toBe(before); + await expectNoCanaryFileNames(CANARY); + await expectNoCanaryFileNames(SCOPE_CANARY); + }, + { timeout: 120_000 } +); + +test("overlay and project selectors are never slugged from user input", () => { + // Shared by the CLI and the daemon env routes (which map undefined to a + // fixed `invalid_env` error). + expect(parseEnvConfigSelection(undefined)).toBeUndefined(); + expect(parseEnvConfigSelection(" ")).toBeNull(); + expect(parseEnvConfigSelection("NONE")).toBeNull(); + expect(parseEnvConfigSelection(" qa ")).toBe("qa"); + expect(parseEnvConfigSelection("qa-2")).toBe("qa-2"); + for (const value of [ + `X=${CANARY}`, + SCOPE_CANARY, + "QA", + "q_a", + "q a", + "q--a", + "-qa", + `${CANARY}.yaml`, + "../qa", + ]) { + expect(parseEnvConfigSelection(value)).toBeUndefined(); + } + + expect(parseProjectSelectorOption({ projectOpt: " my-app " })).toEqual({ + name: "my-app", + canonical: true, + }); + // Legacy alias spellings still select their canonical slug. + expect(parseProjectSelectorOption({ projectOpt: "My_App" })).toEqual({ + name: "my-app", + canonical: false, + }); + for (const value of [`x=${CANARY}`, `${CANARY}.x`, "a:b", "", "---"]) { + let message = ""; + try { + parseProjectSelectorOption({ projectOpt: value }); + } catch (error: unknown) { + expect(error).toBeInstanceOf(CliUsageError); + message = error instanceof Error ? error.message : String(error); + } + expect(message).toBe(INVALID_PROJECT_OPTION_MESSAGE); + } +}); diff --git a/tests/env-selection-parity.test.ts b/tests/env-selection-parity.test.ts index 52f174ba..eeb6a7c7 100644 --- a/tests/env-selection-parity.test.ts +++ b/tests/env-selection-parity.test.ts @@ -10,7 +10,10 @@ import { import { tmpdir } from "node:os"; import { join } from "node:path"; import { CliUsageError } from "../src/cli/command.ts"; -import { resolveRequestedEnvName } from "../src/cli/env-selection.ts"; +import { + INVALID_ENV_SELECTION_MESSAGE, + resolveRequestedEnvName, +} from "../src/cli/env-selection.ts"; import { optEnv } from "../src/cli/options.ts"; import { runCli } from "../src/cli/run.ts"; import { __testOnlySessionCommand } from "../src/commands/session.ts"; @@ -216,7 +219,7 @@ test("an unusable --env name is refused by every surface without reading values" for (const result of [plan, list]) { expect(JSON.parse(result.stdout)).toEqual({ ok: false, - error: { code: "E_USAGE", message: "Invalid --env value." }, + error: { code: "E_USAGE", message: INVALID_ENV_SELECTION_MESSAGE }, }); } expect(get.stdout).toBe(""); @@ -262,16 +265,21 @@ test("authored native dispatch refuses an unusable --env before selecting a proj overlay: INVALID_NAME, }).catch((value: unknown) => value); expect(error).toBeInstanceOf(CliUsageError); - expect(error).toHaveProperty("message", "Invalid --env value."); + expect(error).toHaveProperty("message", INVALID_ENV_SELECTION_MESSAGE); }); test("the shared resolver maps base spellings to null and keeps workspace names env-base", () => { const { buildScopedWorkspaceBaseName } = __testOnlySessionCommand; expect(resolveRequestedEnvName({ envOption: undefined })).toBeUndefined(); - expect(resolveRequestedEnvName({ envOption: " QA " })).toBe("qa"); - expect(() => resolveRequestedEnvName({ envOption: INVALID_NAME })).toThrow( - "Invalid --env value." - ); + expect(resolveRequestedEnvName({ envOption: " qa " })).toBe("qa"); + expect(resolveRequestedEnvName({ envOption: "qa-2" })).toBe("qa-2"); + // Names are never slugged: a non-canonical spelling is refused, not + // rewritten into a different overlay name. + for (const spelling of [INVALID_NAME, "QA", "q_a", "q a", "qa.", "X=qa"]) { + expect(() => resolveRequestedEnvName({ envOption: spelling })).toThrow( + INVALID_ENV_SELECTION_MESSAGE + ); + } for (const spelling of BASE_SPELLINGS) { const envName = resolveRequestedEnvName({ envOption: spelling }); expect(envName).toBeNull(); From 63e1d39c040f11529646cec25b3795620bc1f5a6 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Mon, 12 Oct 2026 12:05:03 -0400 Subject: [PATCH 52/52] test(native): use a canonical overlay name in the env selection case #308 makes --env require a canonical overlay name instead of slugging input, so ' QA ' is now refused before any project work. Keep the native dispatch case on the documented path with ' qa ' (still exercises trimming); the refusal itself is covered by the env argument redaction tests. Co-Authored-By: Claude Opus 5.5 --- tests/native-authored-command-dispatch.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/native-authored-command-dispatch.test.ts b/tests/native-authored-command-dispatch.test.ts index cb7806dc..9637a6bd 100644 --- a/tests/native-authored-command-dispatch.test.ts +++ b/tests/native-authored-command-dispatch.test.ts @@ -308,7 +308,7 @@ macTest.each( macTest.each([ { option: "base", overlay: "base", metadata: null }, { option: undefined, overlay: "inherit", metadata: "qa" }, - { option: " QA ", overlay: { named: "qa" }, metadata: "qa" }, + { option: " qa ", overlay: { named: "qa" }, metadata: "qa" }, ])( "source CLI env $option binds the documented selection into the native source before values", async ({ option, overlay, metadata }) => {