[coderdan]

Summary

This pull request lets a language SDK query and read an encrypted JSON document (EncryptedJson, EQL's json_tree type) through a data plan, the declaration a non-Rust SDK sends Stack Encrypt (the Rust engine) as data. It is slice C of the plan in #1353: the JSON operators, selector and token preparation, and reads opened against the request that prepared them, in the engine and in the WebAssembly guest (the engine compiled for Go). The Go runtime learns to carry the new answers; generated Go methods follow in the next pull request.

Every operand is the typed Rust chain's, bit for bit. A read's rows open only against the trusted request (operator, path, bound, field and keyset), which the engine re-derives from, so the database never says which path a row is for.

Changes

Operators (packages/stack-encrypt/src/dynamic)

• OpSpec gains the document operators. Each takes an object of named inputs, with JSON values as JSON text:

  operator	input	operand, bound as
  contains, contained_by	{"json"}	needle, eql_v4.query_json_tree
  eq_at	{"path", "json"}	needle holding the scalar there
  lt_at … gte_at	{"path", "json"} at an ordered path	leaf operand, eql_v4.query_json_tree_leaf
  between_at	{"path", "low", "high"}	two leaf operands
  selector	{"path"}	text
  subtree	{"path", "wildcard_bound"?}	text[]
  children, child_count	{"path"}, no wildcard	text (the node's token)

• field_operators lists them from the field's resolved options (the orderings only when the field orders a path), and only those whose SQL type the target names, so it lists exactly what query_term derives. between_at's arity is one: its input is one object.

• The query term's wire form gains encoding (bytes, eql, text, text_array), and Operand gains Text and TextList, so a selector or a token list is a named value, not anonymous bytes. An operator entry gains input (its input's shape), so a generator can tell inputs apart without a copy of the rules.

• Refusals before anything is derived: a position in a path where the operand is a selector (json_path_position; a positional read is a subtree read), a wildcard where the operator names one node (json_path_wildcard), malformed inputs (json_input), and the engine's existing ones (container at a path, unordered path, non-ordered leaf).

• json_open(cipher, plan, field, request, rows, resolver) / check_json_open: open the rows of a selector or subtree read, one group per database row, in one key request. The engine derives the selector or the subtree key again from the request and the field's options under the caller's keyset, runs the index's own row checks, and opens through that keyset alone.

EQL (packages/eql)

• Target::leaf_query_sql_domain, so the engine knows what an ordering at a path binds as.
• tests/json_tree_data_plan.rs: every operand equals the typed chain's; the operator list; the refusals; reads of -> and of a wildcard subtree across two documents, absence as no value, rows refused against another path's request, and a truncated wildcard refused.

Guest (languages/golang/encrypt/guest)

• New export se_json_open; JSON read refusals (JsonWrongEntry, JsonWildcardTruncated) are STATUS_ENCODING with their own diagnostic codes, not internal errors.

Go runtime (languages/golang)

• QueryTerm.Encoding and QueryTerm.Text; record.Operator.Input; record.JSONIndex; Cipher.JSONOpen.
• json_tree_test.go: every operator through the guest with its encoding and domain, a refused positional selector, and a selector read opened against its request (and refused against another's).

Docs: CHANGELOGs (stack-encrypt, eql-bindings, Go) and a changeset for @cipherstash/eql.

Assumptions

The five owner decisions in #1347 are open; this stack takes the plan's recommended answer to each (the Go-facing ones land in the next pull request):

1. Go field types: json.RawMessage / *json.RawMessage.
2. JSON crosses the guest as text: yes; every JSON input here is JSON text.
3. Containment needle: an explicit JSON value ({"json": …}), not the field's struct type.
4. Ordered-path accessors: generated from the validated options; the engine lists orderings only for a field that orders a path.
5. First release scope: the full inventory.

Also assumed:

• Operator names contains, contained_by, eq_at, lt_at, lte_at, gt_at, gte_at, between_at, selector, subtree, children, child_count; input objects follow the plan's examples ({"json"}, {"path", "json"}, {"path", "wildcard_bound"}).
• Read descriptor, not a guest handle (plan section 4): the host keeps the request and hands it back with the rows; the guest re-derives. The read is opened through the keyset that prepared it, never the client-wide scope, so a document from another keyset is refused before any key request.
• Children: the guest prepares the node token for json_tree_children / json_tree_child_count. The plaintext child-opening helper the plan lists (parent-bound child validation) is not in this pull request; children are exposed as encrypted entries for SQL. See "Not done" below.
• Text-valued operands bind as text / text[], named in the term's domain.

Not done here

• A request-bound opener for json_tree_children rows (child scalars without a subtree read). A subtree read of the node returns the same values today.
• A typed struct adapter for a needle (plan section 2): the needle is JSON text.

Verification

macOS arm64, Rust 1.94.1, Go 1.26.10, on this branch. After the review fixes (abdc767, merged with the base in 978f0cd): root clippy and cargo nextest run --workspace --all-features (1,493 passed, including the new in-crate tests/json_operators.rs and the heap-spy test for an operator's input), rustdoc with warnings as errors, packages/eql fmt, clippy and nextest (627 passed), types:check, the four guest builds and wasm:guest:test, go:test (including TestParseQueryTermRefusesAMixedShape), go:lint, go:test:postgres against PostgreSQL 17 with the tree's EQL v4 bundle, crap:stack-encrypt (none over 30), and a local cargo mutants --in-diff run (96 mutants; the six survivors are fixed or, for the visitor's expecting, excluded as equivalent). The original verification:

• cargo fmt --all --check (root, packages/eql, guest): clean. cargo clippy --locked --no-deps --workspace --all-targets --all-features -- -D warnings: clean. EQL cargo clippy --workspace --all-targets --all-features --exclude eql_tests -- -D warnings: clean.
• Root mise x --env test -- cargo nextest run --workspace --all-features --locked: 1,470 passed.
• packages/eql cargo nextest run --workspace --exclude eql_tests: 624 passed, including the 7 tests of tests/json_tree_data_plan.rs. The SQLx crate needs generated fixtures and a database and was not built.
• Guests rebuilt (wasm:guest:build, :deterministic, :eql, :eql:deterministic); mise run go:test passed, including the new TestJSONOperatorsDeriveTaggedTerms and TestJSONSelectorReadsOpenAgainstTheirRequest.
• mise run wasm:guest:test (clippy in every feature set, docs, nextest with and without eql): passed, with the catalog test now pinning EncryptedJson's operators.
• The pinned wire bytes test (dynamic::query::wire_bytes) is re-pinned for the new keys and gains the JSON index's entries and a text and text_array term; the two trybuild snapshots that now print stack_encrypt::op::Contains in full (because OpSpec::Contains exists) are re-recorded.
• Not run: live ZeroKMS tests (no credentials here). PostgreSQL: the next pull request runs the Go data path against PostgreSQL; nothing here changes SQL.

Related

• Refs Tracking: encrypted JSON documents (json_tree) in the Go SDK #1347 (tracking), json_tree operators (@>, <@, equality and ordering at a path, ->) have no data-plan form #1350 (operators), Go SDK: decrypt json_tree extracted entries and subtree walks #1352 (reads; the child-opening helper remains), Stack Encrypt refuses the JSON index in a data plan, so the Go SDK cannot seal a json_tree document #1349 (the query half of data-plan/typed parity).
• Plan: docs(plans): define Go JSON dynamic contract and delivery #1353.

Stack

Merge order is bottom to top; each pull request is based on the one below it.

1. chore(deps): take vitaminc's Go bindings from cipherstash/vitaminc#526 by git reference #1369: vitaminc's Go bindings from the unreleased feat(vcffi): read and write the storage profile in Go vitaminc#526 by git reference (temporary)
2. feat(go): seal an opaque struct as a vitaminc storage blob #1370: Go: an opaque struct sealed as a vitaminc storage blob
3. feat(stack-encrypt,eql): store a JSON document through a data plan as EncryptedJson #1372: EncryptedJson stored through a data plan (engine, EQL catalog, guest, Go runtime)
4. feat(stack-encrypt): answer a document field's JSON operators and open its reads from a data plan #1373: JSON operators and request-bound reads through a data plan (this pull request)
5. feat(go): generate encrypted JSON document fields #1374: Go: generated encrypted JSON document fields

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

[changeset-bot]

🦋 Changeset detected

Latest commit: 8a2488e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@cipherstash/eql	Minor
stash	Patch
@cipherstash/stack-prisma	Patch
@cipherstash/basic-example	Patch
@cipherstash/e2e	Patch
@cipherstash/prisma-example	Patch
@cipherstash/stack-drizzle	Patch
@cipherstash/stack-supabase	Patch
@cipherstash/stack	Patch
@cipherstash/wizard	Patch
@cipherstash/bench	Patch
@cipherstash/test-kit	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

caught	missed	unviable	timeout
60	0	32	0

Every mutant in the changed lines was caught by a test.

[coderdan]

Automated code review of this pull request at 8485fe9fa (the review skill did not post inline comments, so the findings are listed here):

1. packages/stack-encrypt/src/dynamic/query.rs:619: answers() lists a document field's operators from its options alone, without checking that the target names the domain each binds as, so field_operators can list an operator query_term then refuses.
2. packages/stack-encrypt/src/dynamic/json_query.rs:516: json_open's scope has no effect: Pending::all(keyset, …) scopes the batch to the deriving keyset whatever scope is passed.
3. packages/stack-encrypt/src/dynamic/query.rs:268: takes_pair() is true for between_at, so the catalog says pair while the input is one object {path, low, high}.
4. packages/stack-encrypt/src/dynamic/json_query.rs:336: derive() clones the parsed plaintext into an owned index operation that never wipes it.
5. packages/stack-encrypt/src/dynamic/json_query.rs:418: open()'s docs say a non-read operator is operator_not_answered; the code says json_input.
6. languages/golang/encrypt/query.go:210: parseQueryTerm accepts term shapes the engine never writes (encoding vs. text/list/domain mismatches); its doc lists five keys.
7. languages/golang/encrypt/guest/src/ops.rs:376: parse_field took parse_query's doc comment, and parse_query repeats parse_field's parsing.
8. packages/stack-encrypt/src/dynamic/json_query.rs:133: json_reason matches error codes as strings with a catch-all, not the enum.
9. packages/stack-encrypt/src/dynamic/json_query.rs:85: binds() and answered_by end in catch-all arms, so a new operator is silently mis-bound or unanswered.
10. languages/golang/encrypt/query.go:114: public docs not updated for the document operators (Cipher.QueryTerm, parseOperatorName, the guest's query_term, Rust query_term, QueryTerm::domain).
11. .cargo/mutants.toml:134: excluding Parsed's Drop exempts a reachable, behaviour-changing mutant, against AGENTS.md's rule.
12. packages/stack-encrypt/src/dynamic/query.rs:620: every answer() for a document field clones the options twice to look up one operator.

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

[coderdan]

Review response to the findings listed above (fixed in abdc767 unless noted; merged with the base in accb055 and 978f0cd):

#	Finding	Outcome
1	query.rs:619: operators listed without their domain	Fixed: a document field lists an operator only when the target names the SQL type its operand binds as (json_query::domain, now shared with query_term), so field_operators lists exactly what query_term derives.
2	json_query.rs:516: json_open's scope has no effect	Fixed: the parameter is gone (json_open(cipher, plan, field, request, rows, resolver)); a read opens through the keyset that derived it and that keyset alone, as documented. Guest and tests updated.
3	query.rs:268: between_at reported as a pair	Fixed: takes_pair is between alone; between_at's arity is one and its input path_pair. Pinned wire bytes re-pinned for the seven between_at entries.
4	json_query.rs:336: cloned plaintext handed to the index	Not changed: the JSON index takes its input by value and its derivation does not wipe it yet. That is the same gap reported on #1372 for the seal path (json::derive frees key and leaf plaintext unwiped); moving instead of cloning would not change what the index frees.
5	json_query.rs:418: wrong reason in open()'s docs	Fixed: the docs list json_input for every request refusal, as the code does.
6	query.go:210: parseQueryTerm too lenient	Fixed: encoding, term shape and domain must agree as the engine writes them (bytes: bytes, no domain; eql: bytes with a domain; text: one string; text_array: a list of strings). New TestParseQueryTermRefusesAMixedShape (seven accepted shapes, seven refused). Doc updated.
7	ops.rs:376: misplaced doc, duplicated parsing	Fixed: each function has its own doc, and parse_query calls parse_field.
8	json_query.rs:133: reason by string code	Fixed: json_reason matches JsonTextError's variants.
9	json_query.rs:85: catch-all arms	Fixed: binds is exhaustive over OpSpec and returns None for a scalar operator (is_json is defined by it); answered_by lists the document operators in its own arm.
10	query.go:114: stale docs	Fixed: Cipher.QueryTerm, parseOperatorName, the guest's query_term (with encoding), Rust query_term (inputs and refusals) and QueryTerm::domain.
11	mutants.toml:134: Parsed's Drop excluded	Fixed: the exclusion is removed. heap_hygiene's new a_json_operators_input_leaves_no_plaintext_on_the_heap checks a document operator's input and fails when Parsed's Drop is a no-op (checked by making it one). The same was done for JsonPlaintext on #1372.
12	query.rs:620: options cloned per lookup	Not changed: field_operators returns an owned IndexSpec per operator, so a copy per listed operator is part of its contract; the cost is a small path set per call.

CI failures on the previous push of this branch (rustdoc's private links, the CRAP gate, which needed in-crate coverage of the JSON operators, and the mutants gate's survivors) were fixed in e1b21ce and b2b5e1d.

Verified on this branch after the fixes: root cargo clippy and cargo nextest run --workspace --all-features (1,493 passed), rustdoc with warnings as errors, packages/eql fmt, clippy and nextest (627 passed), types:check, the four guest builds and wasm:guest:test, go:test, go:lint, go:test:postgres against PostgreSQL 17 with the tree's EQL v4 bundle, and the CRAP gate (crap:stack-encrypt: none over 30).

https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu