feat(env): accept secret values on stdin with --value-stdin

[roodboi]

Summary

HACK-1158 (part of HACK-1127). hack env add and hack env set now accept --value-stdin. You can pipe a value from a secret manager or a file, and it never appears in argv, shell history or process listings.

<secret-source> | hack env add SERVICE_TOKEN --service api --secret --value-stdin

Final behavior

• Reads all of stdin up to 64 KiB. The reader stops and cancels the stream as soon as input goes over the limit, so an endless producer cannot exhaust memory.

• Strips exactly one trailing newline (\n, or \r\n counted as one newline). Any further newlines stay in the value.

• Refuses each of these with a fixed message that never includes the input. The refusal happens before any project file is migrated or written:

  • an interactive TTY on stdin
  • a value argument passed with the flag, including KEY=value
  • a missing key
  • empty input
  • input over 64 KiB
  • invalid UTF-8
  • a NUL byte

• Works on the modern YAML path and on the legacy env set path (keychain or encrypted_file backends).

• The non-TTY E_INTERACTIVE_REQUIRED hints in src/commands/env.ts no longer tell you to pass the value as an argument. They now point to --value-stdin or the hidden prompt.

• Docs and guidance:

  • docs/env.md gains an "Entering secret values" section.
  • docs/reference/cli.md is regenerated.
  • Canonical guidance (instruction-source.ts, env composition) and the onboarding prompt now say to pipe secrets with --value-stdin and never pass them as arguments.
  • The plugin bundle and the checked-in examples (.codex, .cursor, examples/basic) are updated.
  • The content revision is bumped, and the new render fingerprints for hack-cli-skill and cursor-rules are appended to the existing lists.

• Invalid key and scope arguments are refused with fixed text that names the allowed pattern but never echoes the argument. Before this, hack env add KEY=value printed "Invalid env key: KEY=", and hack env materialize --service KEY=value printed "Unknown env scope: KEY=".

  • Keys: env add/set/unset/explain/get, the legacy env set/unset path, setProjectEnvValue, and malformed keys in env config files. env add validates the key before reading stdin or migrating files.
  • Scopes: every --service/--scope value is validated in src/cli/env-arguments.ts before project migration, file access or any child process. This covers env materialize/exec/shell/list/apply/get and host exec/shell. env apply no longer forwards a malformed --service to its hack restart child.
  • Unknown scopes: selectProjectEnvValues and resolveProjectEnvValue now use one fixed "Unknown env scope" message. It lists configured scope names from compose and env files, never the request. Scope lookups use Object.hasOwn, so constructor or __proto__ count as unknown.
  • env get key/scope parsing is now parseEnvGetArguments, tested directly. The command's generic error wrapper would otherwise hide a re-echo.
  • Proven by tests/env-key-argument-redaction-cli.test.ts. It runs the real CLI with a canary in the key or scope for each command. The canary must be absent from stdout, stderr and spawned argv/env, no non-git child may start, and project files must be unchanged. Run against the previous head, the new scope cases fail with the canary in stderr. Source mutations that re-echo the argument, skip scope validation, or reorder validation each fail the suite. These include a library-only unknown-scope echo, skipping exec/apply scope validation, and an env get key echo that the CLI wrapper hides.

• --env, --project and --target values are no longer slugged or echoed. Before this, --env X=<value> became the overlay name x<value>, was printed as env_selection, and hack env add ... --env X=<value> created .hack/hack.env.x<value>.yaml. --project x=<value> printed Unknown project "x<value>", and --target x=<value> was echoed as given.

  • --env: parseEnvConfigSelection (shared by the CLI and the daemon env routes) no longer rewrites a requested name. Apart from surrounding whitespace, the value must already match ^[a-z0-9]+(-[a-z0-9]+)*$. base, default and none (any case) and an empty value still select base layers. Anything else, including any value with = or uppercase letters, gets one fixed Invalid --env value. message that states the rule and never the argument. runCli checks the shared --env option before any handler runs, so every command that accepts --env refuses before project discovery, migration, file access, stdin or a child process. env apply forwards the checked selection to hack restart, not the raw argument; an empty --env now forwards base, which matches the grammar.
  • --project on env/host commands: parseProjectSelectorOption refuses an argument that contains = or any character other than letters, digits, -, _, space and /. The refusal is fixed text and happens before the registry is read. Legacy alias spellings such as My_App still select my-app. Unknown project echoes a name only when it was typed in canonical form.
  • --target: the refusal lists only the allowed values (host, compose). env explain/exec/shell and host exec/shell now check --env and --target before they resolve the project.
  • docs/env.md now states the overlay name rule. Before, it said that names are normalized.
  • Proven by four new CLI tests in tests/env-key-argument-redaction-cli.test.ts. The list of commands comes from CLI_SPEC: every command that accepts --env, and every env/host command that accepts --project. Each command runs as the real CLI with canaries in --env, --project and --target. The tests check that the canary, and its lowercased slug form, is absent from stdout, stderr and spawned argv/env. They also check that no non-git child starts, that no file under the temp root has the canary in its name, and that project files are unchanged. The --env refusal is also checked from a directory without .hack/, which proves it happens before project discovery. Base aliases still work. A unit test covers parseEnvConfigSelection and parseProjectSelectorOption.
  • Mutations, each reverted after the run: putting slugging back in parseEnvConfigSelection, removing the runCli --env check, dropping the --project pattern, echoing a slugged unknown project, and echoing --target. Each one failed the new tests.

Validation

• tests/env-value-stdin.test.ts (in-process, stdin source injected), covering:

  • trailing newline, CRLF and multi-newline handling
  • multi-byte characters split across chunks
  • input exactly at the limit, and oversize input (an endless stream is canceled after a bounded number of pulls)
  • empty and newline-only input
  • a TTY stdin is refused before the stream is opened
  • invalid UTF-8, NUL bytes, and read failures (the error drops its cause)
  • usage checks, and that every refusal message is fixed and value-free

• tests/env-value-stdin-cli.test.ts runs the CLI in fresh subprocesses. Each run gets:

  • a temp HOME and temp global config
  • a minimal environment
  • a --preload recorder (tests/fixtures/spawn-argv-recorder.ts) that logs argv and env for every Bun.spawn, Bun.spawnSync and child_process call

  Stores are either the project-key YAML store or an injected encrypted_file backend with an env-provided key. No keychain, no 1Password, no Docker, and no agent client config. The tests check that:

  • the value is absent from stdout, stderr, the spawn argv/env log and the project files, and that it round-trips exactly
  • ps -ww -o args= for the waiting CLI process does not contain the value
  • all four positional+stdin forms are refused without echoing the value and without changing any file
  • oversize, empty and missing-key input is refused and nothing is stored
  • the non-TTY hint names --value-stdin
  • the legacy env set --secret --value-stdin path writes through encrypted_file

• Required checks:

  • tests/integration-ownership.test.ts, tests/agent-plugin-generation.test.ts, tests/agent-instruction-source.test.ts, tests/onboarding-prompt.test.ts, tests/cli-reference-docs.test.ts, tests/runtime-guidance.test.ts
  • the existing env suites (env-command-modern, env-backend-command, env-get, env, env-command-matrix, interactivity, cli-help, project-env-worktree-key, env-selection-parity)
  • bun run --cwd packages/cli typecheck
  • bun run --cwd packages/cli check (0 errors)
  • For the --env/--project/--target fix (head b1bc6db2), these all passed locally: the env suites (compose-managed-env, daemon-env, env-backend-command, env-command-matrix, env-command-modern, env-exec-command, env-get, env-key-argument-redaction-cli, env-selection-parity, env-value-stdin, env-value-stdin-cli, env, hack-env, project-env-*, project-lifecycle-env, session-env-command, native-env-*), plus integration-ownership, agent-plugin-generation, project-execution-routing, cli-*, session-command and native-config-compiler; root bun run typecheck; and bun run check (warnings only, none in changed files). docs/reference/cli.md regenerates unchanged.

Negative controls

• Unit: the leak detector is shown to catch a value placed in a CliUsageError message, and also an 8-character fragment of the value.
• CLI: I temporarily changed handleEnvAdd to throw a refusal that contains the positional value. The positional+stdin test then failed on expect(stderr).not.toContain(SENTINEL). I reverted the change and did not commit it.
• The spawn recorder is shown to be live: the log of the write path contains the tracked-file probe that the env write makes.
• The output scan is shown to be live: hack env get of the stored key returns the value on stdout.

Remaining gaps

• The macOS keychain backend is not exercised (by design: tests must not touch the real keychain). It stores values through Bun.secrets, which takes no argv. The stdin path hands it the same string as the prompt path does.
• No real-terminal (pty) end-to-end test of the TTY refusal. The TTY branch is covered by an injected isTTY source.
• Outside env/host commands, --project still slugs its argument and echoes the slug in Unknown project "<slug>". This affects src/commands/project.ts, branch.ts, config.ts, tui.ts, remote.ts, gateway.ts and project-admin.ts. The --target refusal for up/down/restart (src/lib/project-execution.ts) also still echoes its argument. Both are out of scope for this change and need the same fix.
• An --env spelling that used to be normalized (for example QA or my_overlay) is now refused. Use the canonical name instead.
• The env apply change, which forwards the checked selection, has no end-to-end test, because running the hack restart child would reach Docker.
• Hosted CI has not run yet on this head.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[blacksmith-sh]

Found 10 test failures on Blacksmith runners:

Failures

Test	View Logs
domain-migration (0.0s) — Host ingress qualification requires HACK_E2E_DOCKER=1/ domain-migration (0.0s) — Host ingress qualification requires HACK_E2E_DOCKER=1	View Logs
domain-migration (0.0s) — Set HACK_E2E_DOMAIN_ROUTING=1 to use the existing shared Cadd y ingress/ domain-migration (0.0s) — Set HACK_E2E_DOMAIN_ROUTING=1 to use the existing shared Cadd y ingress	View Logs
incomplete-owned-cleanup (0.0s) — Owned cleanup refused/ incomplete-owned-cleanup (0.0s) — Owned cleanup refused	View Logs
native-fixture (0.0s) — missing fixture prerequisite/ native-fixture (0.0s) — missing fixture prerequisite	View Logs
ordinary-fixture-cleanup (0.0s) — Earlier assertion failure after complete cleanup/ ordinary-fixture-cleanup (0.0s) — Earlier assertion failure after complete cleanup	View Logs
owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted./ owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted.	View Logs
owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted./ owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted.	View Logs
owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted./ owned-evidence-control (0.0s) — Synthetic fixture refusal; values omitted.	View Logs
retained-no-success (0.0s) — missing prerequisite after retained cleanup/ retained-no-success (0.0s) — missing prerequisite after retained cleanup	View Logs
retained-no-success (0.0s) — Scenario cleanup is incomplete; owned fixtures are retaine d for recovery/ retained-no-success (0.0s) — Scenario cleanup is incomplete; owned fixtures are retaine d for recovery	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}