fix(stack-encrypt): check that a json_tree read returns the entry asked for

.changeset/eql-json-tree-extracted-entry-check.md

@@ -0,0 +1,5 @@
+---
+"@cipherstash/eql": patch
+---
+
+EQL v4 `json_tree` (unreleased): the client now checks that what `->` and the subtree walks return is what it asked for. Every entry of a stored document is sealed under its own path selector and position, so any entry of the row opened, and a database could answer `doc -> s($.salary)` with the row's `$.bonus`, or with the sealed root (the whole document), or splice other entries into a walk. In the `eql-bindings` crate an entry `->` returned now opens through the selector that asked for it, `JsonTreeExtracted::from_selector(&selector, entry)`, which refuses an entry carrying any other selector; `JsonTreeEntry` no longer opens on its own. A walk's rows, opened through `JsonTreeSubtree::from_key(&key, rows)` as before, are each checked against the key's path: the node a gated walk reaches must carry the path's selector and position, every other row the selector of its parent's path and its own label, and the rows must follow each other in the document with no gap. A refusal is `stack_encrypt::Error::JsonWrongEntry`. Not checked, by design: which row of the column an entry comes from (ADR 0010). The SQL is unchanged.

docs/plans/2026-10-17-json-tree-index.md

@@ -496,7 +496,7 @@ Two of the three items that gate a release are closed here; the third, the final
 - **Planner fragility (gate 2).** `eql_v4.json_tree_query` is plpgsql with `COST 1000` so that a bitmap scan over the GIN index beats a sequential scan even when the documents are TOASTed and the heap looks like a few pages. That was observed in the benches, at 10,000 and 100,000 rows, on one PostgreSQL version, and nothing held it. `postgres_the_planner_picks_the_indexes_without_help` in the encryption suite now does: on a flat table (1,000 four-node documents, every row inline) and a nested table (300 documents of about 220 nodes, every row TOASTed, both regimes pinned by `pg_column_size`), with every planner setting at its default and after `ANALYZE`, containment plans as a bitmap scan over the GIN index and a range at an ordered path as a scan of the functional btree on `ord_term(doc -> selector)`. A control sets the function's cost to 1 for one transaction and shows the planner taking the sequential scan on the nested table, so the assertion is known to be able to fail. The suite runs in `test-eql.yml` in a new `rust-encryption-postgres` job, a matrix over the supported PostgreSQL versions (17 on a pull request, 14 to 17 on a push, like the SQLx jobs); it was a step of the single-version `rust-crates` job. Measured locally on 14, 15, 16 and 17 before the commit.
 - **Leakage (gate 3).** `packages/eql/docs/reference/json-tree-leakage.md` is the one page: what a stored row shows at rest, field by field; what each query type hands the database; what is authenticated; the side-by-side with v3 from section 6 above, brought up to date (no `g`, position-keyed `p`, ordering opt-in) and corrected where section 6 was wrong (the entry order; see the note under its table); and the two options that change the picture. It says what a token is and that it is column-wide and does not expire, which is the sentence a customer most needs before sending one.
 - **Open for the owner: the stored entry order (2026-10-23, review of #1336).** `sv` and `q` hold the nodes in the walk's order, depth-first, an object's members in the sorted order of their keys (serde_json's `Map` without `preserve_order`; a build that unifies that feature on would store insertion order instead, which leaks the application's field order). Across rows this reveals each row's shape, its parent relation, every array's length and the alphabetical order of sibling key names, which partly undoes what the keyed rank (decision 11) hides from a token holder. The leakage page says so. Closing it is a format change, not a documentation one: a stored order that does not follow the tree (a document-keyed permutation, say, with `j` the permuted position), a rebuild that does not read the tree from the order, and the same for the pairs of `q`, whose nesting would still group array elements. Not taken here.
-- **Open for the owner: an extracted entry opens under the selector the server returns with it.** `JsonNode` and `JsonSubtree` open each entry under the `s` and `j` that arrive beside it, and under the data key the header that arrives with it names. The entry's AAD (`entry_aad` in `packages/stack-encrypt/src/json/derive.rs`) binds the keyset, the column's context, the data key's tag, the selector and the ordinal, and nothing that identifies a row. So a decrypted value is always one sealed in that column, but not necessarily the one at the path asked for, nor one from the row the result puts it in: `->` on `$.salary` can be answered with the row's `$.bonus`, with the sealed root (which opens to the whole document), or with another row's `$.salary` returned together with that row's header. The last is by design: two values sealed under the same context are interchangeable between rows (ADR 0010 in `packages/stack-encrypt/docs/adr`), and row provenance is the database's word. The leakage page states this under "What is authenticated". Checking the returned `s` against the selector the client derived would close the path substitution for `->`, though not the row one; a walk's rows would need each child's selector checked against its parent's path.
+- **Taken (2026-10-24, #1355, PR #1358): an extracted entry is checked to be the one asked for.** Until then `JsonNode` and `JsonSubtree` opened each entry under the `s` and `j` that arrived beside it, and under the data key the header that arrived with it named. The entry's AAD (`entry_aad` in `packages/stack-encrypt/src/json/derive.rs`) binds the keyset, the column's context, the data key's tag, the selector and the ordinal, and nothing that identifies a row. So a decrypted value was always one sealed in that column, but not necessarily the one at the path asked for, nor one from the row the result puts it in: `->` on `$.salary` could be answered with the row's `$.bonus`, with the sealed root (which opens to the whole document), or with another row's `$.salary` returned together with that row's header. Now a row opens only through what the client derived for the read. `JsonSelector::open(node)` refuses an entry whose `s` is not the selector (another path's entry, and the sealed root, whose selector is the root's, in place of a node) before any key is retrieved. `JsonSubtreeKey::open(rows)` refuses a row that does not fit the key's path: a node a gated walk reaches must carry the path's selector at depth `0`, and the position the path names when its last step is one; every other row must carry the selector of its parent's path extended by its own decrypted label (a key, or the wildcard for an element), which the key derives through the keyset's index-key PRF it now holds; and the rows of each subtree must be consecutive in the document, which is what a subtree is in a depth-first `sv`, so a row out of place, a dropped row and an orphan are refused. A reported depth `d` that disagrees is refused too; the rows are placed by their positions and child counts, not by `d`. The refusal is a new `Error::JsonWrongEntry`, not `Aead` (each entry is genuine) and not `OpenedType` (the entry is not of an unexpected shape; it answers another question). `eql-bindings` follows: `JsonTreeExtracted::from_selector(&selector, entry)` is how an entry `->` returned opens, and `JsonTreeEntry` no longer opens on its own. Still not covered: another row's `$.salary` with that row's header opens, by design (two values sealed under the same context are interchangeable between rows, ADR 0010 in `packages/stack-encrypt/docs/adr`, and row provenance is the database's word); and an array position before the last step of a path reads as the wildcard to every selector, so a walk's rows for `$.a[1].b` can be another element's `b` in the same row. Until the ungated walk returns the node it starts from (#1356), a walk with no gates is also not checked for which node at the path's selector it descends from, which matters only for a token chained through array positions (`$.l[3]`), whose children share every other element's selectors; nor for children left out after the last row it returns, or members left out anywhere, since the node's child count is in the node's own entry and a member's place is a keyed rank. An array's elements are checked to start at `0` (review of PR #1358: leaving out element 0 opened `[10, 20, 30]` as `[20, 30]`), and rows that stop short of a node's subtree are `JsonWrongEntry` like a gap.
 
 ## Decisions to take before Phase 1
 

packages/eql/crates/eql-bindings/CHANGELOG.md

@@ -20,8 +20,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   opens at the path the walk's key was derived from: the rows are built
   with `JsonTreeSubtree::from_key(&key, rows)`, and there is no
   constructor without the key.
+  An entry `->` returned opens through the selector that asked for it,
+  `JsonTreeExtracted::from_selector(&selector, entry)`, which refuses an
+  entry carrying another selector (`stack_encrypt::Error::JsonWrongEntry`),
+  and a walk's rows through the key that keyed it, which checks each row
+  against the path the same way; `JsonTreeEntry` does not open on its own.
   The document is an `EncryptFrom<Value>` and `DecryptInto<Value>` target,
-  the entry and the subtree open through `DecryptInto`, and the needle and
+  the extracted entry and the subtree open through `DecryptInto`, and the needle and
   the ordering operand convert from Stack Encrypt's `JsonQuery` and
   `JsonLeafQuery`. `JsonTree::new(options)` is the `json_tree` layout under
   index options other than the defaults (a `Layout<Value>`, so a program

packages/eql/crates/eql-bindings/src/v4/json_tree.rs

@@ -23,8 +23,8 @@ use schemars::{schema_for, JsonSchema, Schema};
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use stack_encrypt::json::{
-    JsonDocument, JsonLeafQuery, JsonNode, JsonOptions, JsonPath, JsonQuery, JsonSubtree,
-    JsonSubtreeKey,
+    JsonDocument, JsonLeafQuery, JsonNode, JsonOptions, JsonPath, JsonQuery, JsonSelector,
+    JsonSubtree, JsonSubtreeKey,
 };
 use stack_encrypt::target::{AeadContext, ExpectedContext, IndexSpec, Json, Layout};
 use stack_encrypt::{
@@ -94,7 +94,13 @@ pub struct JsonTreeDocument {
 /// `c` the node's own ciphertext; `p` the parent tag below the chain's
 /// heads, which the walk probes by ordinal; `op` the leaf ordering term.
 /// `h`, `j`, `d`, `v` and `i` are what the extractors graft onto an entry
-/// they return, so it opens on its own.
+/// they return, so that it can be opened.
+///
+/// An entry a query returned is the database's word and does not open on
+/// its own: open the entry `->` returned through the selector that asked
+/// for it ([`JsonTreeExtracted::from_selector`]), and a walk's rows through
+/// the key that keyed it ([`JsonTreeSubtree::from_key`]). Each checks that
+/// the rows are the ones it asked for.
 #[derive(Clone, Debug, PartialEq, Serialize, Deserialize, TS, JsonSchema)]
 #[serde(deny_unknown_fields)]
 pub struct JsonTreeEntry {
@@ -165,6 +171,42 @@ pub struct JsonTreeLeafQuery {
     pub op: JsonTreeOrderingTerm,
 }
 
+/// The entry `col -> selector` returned, with the selector that asked for
+/// it: opens to the scalar the entry holds, or to the whole document for a
+/// sealed root asked for at `$`. The only constructor,
+/// [`from_selector`](Self::from_selector), takes the selector, and opening
+/// refuses an entry that does not carry it
+/// ([`Error::JsonWrongEntry`](stack_encrypt::Error::JsonWrongEntry)), as
+/// [`JsonSelector::open`] does: a database that answers one path with
+/// another path's entry, or with the sealed root, returns an entry that
+/// would otherwise open.
+#[derive(Clone, Debug, PartialEq)]
+pub struct JsonTreeExtracted {
+    entry: JsonTreeEntry,
+    selector: JsonSelector,
+}
+
+impl JsonTreeExtracted {
+    /// The entry `col -> selector` returned for `selector`, as the server
+    /// sent it.
+    pub fn from_selector(selector: &JsonSelector, entry: JsonTreeEntry) -> Self {
+        Self {
+            entry,
+            selector: *selector,
+        }
+    }
+
+    /// The entry.
+    pub fn entry(&self) -> &JsonTreeEntry {
+        &self.entry
+    }
+
+    /// The selector the read asked for.
+    pub fn selector(&self) -> &JsonSelector {
+        &self.selector
+    }
+}
+
 /// The rows a subtree walk returned (`eql_v4.json_tree_subtree` or
 /// `json_tree_subtree_at`), collected with the key that keyed the walk:
 /// opens to the value under the walked path, or to the list of complete
@@ -396,14 +438,18 @@ where
 }
 
 /// An extracted entry opens to the scalar it holds, or to the whole
-/// document for a sealed root, under the column it was extracted from. A
+/// document for a sealed root asked for at `$`, under the column it was
+/// extracted from, once it is shown to carry the selector asked for. A
 /// container node holds no value of its own; its subtree opens through
 /// [`JsonTreeSubtree`].
-impl DecryptInto<Value> for JsonTreeEntry {
+impl DecryptInto<Value> for JsonTreeExtracted {
     type Context = NonEmpty<Identifier>;
     fn decryption<K: 'static>(self, context: Self::Context) -> Decryption<Value, K> {
-        match self.node() {
-            Ok(node) => node.decryption(AeadContext::from(context)),
+        match self.entry.node() {
+            Ok(node) => self
+                .selector
+                .open(node)
+                .decryption(AeadContext::from(context)),
             Err(error) => Decryption::failed(error),
         }
     }

packages/eql/docs/reference/json-tree-leakage.md

@@ -39,7 +39,12 @@ A query hands the database a term derived from the plaintext it asks about, and
 
 Only the ciphertext `c`. Opening an entry verifies that it was sealed under this keyset, for this column, under the data key its row's header names (a fresh one for every row), with the selector `s` and the position `j` it is opened with; a ciphertext moved into another row or column, or opened under another selector or position, fails. Nothing in it names a row, so an entry that travels with its own row's header still opens; see below. When the client opens a whole document it takes each entry's position from its place in `sv`, so a moved, swapped or re-labelled entry fails to open. The selectors, the projection, the parent tags, the ordering terms, the header's sealed-root flag, and which entries a row holds are not authenticated, by the same rule every EQL stored value follows (ADR 0010 in `packages/stack-encrypt/docs/adr`).
 
-The consequence to design around: tampering with those fields changes **which rows a query returns, where a walk stops, and which entries of a row an extraction or a walk hands back**. An entry the database returns from `->` or a walk carries its own `s` and `j`, and the client opens it under those, and the header it carries names the data key, so a decrypted value is always one that was sealed in this column, but the client does not check today that it is the entry at the path it asked for, or that it comes from the row the result puts it in: a database that wants to can answer `$.salary` with the same row's `$.bonus`, or with another row's `$.salary`. A query's result is not integrity-protected; each decrypted value is genuine, but where it came from is the database's word.
+The consequence to design around: tampering with those fields changes **which rows a query returns, where a walk stops, and which entries of a row an extraction or a walk hands back**. An entry the database returns from `->` or a walk carries its own `s` and `j`, and the header it carries names the data key, so a decrypted value is always one that was sealed in this column. The client also checks that it is the entry it asked for, from what it derived for the query, and refuses anything else (`JsonWrongEntry`):
+
+- **`->`.** The entry must carry the selector the client derived for the path. Another path's entry, such as the row's `$.bonus` returned for `$.salary`, is refused, and so is the sealed root returned for a field (it opens to the whole document); the root is accepted only when `$` itself was asked for.
+- **A walk.** A node a gated walk reaches must carry the path's selector, and the position the path ends in, if it ends in one. Every other row must carry the selector of its parent's path extended by its own key (or the wildcard, for an array element), which the client derives from the decrypted key, and a subtree's rows must follow each other in the document with no gap, through to the last row the node's child count calls for. A row of another path, a reordered row, a row whose parent is not among the rows, and a row left out of a node's subtree are refused.
+
+What the checks do not cover: **which row an entry comes from**. Nothing in an entry names its row, so another row's `$.salary`, returned with that row's header, opens; that is the rule every EQL stored value follows (ADR 0010). And **a position before the last step of a path**: every element of an array shares its selector, so a walk for `$.orders[1].items` can be answered with `$.orders[0].items` of the same row. And, for a walk with no gates (`json_tree_subtree`, whose path is a top-level field, or the root when the root is not sealed, followed only by array positions), **the node the walk starts from**, which it does not return: its children are checked against each other, so a gap between two of them and an array that does not start at element `0` are refused, but children left out after the last one returned, and members left out anywhere, are not (a member's place is a keyed rank the client does not hold), and for a path that ends in a position (`$.orders[3]`) the children can be another element's, since every element's children share their selectors. Both close when the walk returns the node it starts from, whose child count bounds its children (#1356). A query's result is not integrity-protected as a whole: each decrypted value is genuine and sealed at the path asked for, read as a selector reads it, but which row it came from, and which of the same-selector nodes above, is the database's word.
 
 ## Compared with EQL v3 (`public.eql_v3_json_search`)
 
@@ -62,6 +67,6 @@ The consequence to design around: tampering with those fields changes **which ro
 - **`sealed_root`** (default on). The root entry's ciphertext is the whole document, so a full read is one decryption and the broadest token a query can send is one top-level field's. The costs are that the root entry's length reveals the document's size, the row is larger, and the root's direct members are marked by their missing `p`, which puts each top-level field's node count in plain view. Off, there is no document-sized ciphertext, but every subtree read whose path begins with a key sends the root's token, the whole column's shape.
 - **`ordered`** (default none). Each path listed carries an ordering term and takes on the `op` row above; a path not listed cannot be range-queried or sorted, and a query that tries is refused by the client before any key loads. Order only the paths your queries sort or range over.
 
-Two disclosures above are properties of the stored format that no option changes today: the depth-first entry order, with members in sorted key order, and the client opening an extracted entry under the selector the database returns with it. Both are open decisions recorded in the design document (`docs/plans/2026-10-17-json-tree-index.md` at the repository root, Phase 4 notes).
+One disclosure above is a property of the stored format that no option changes today: the depth-first entry order, with members in sorted key order. It is an open decision recorded in the design document (`docs/plans/2026-10-17-json-tree-index.md` at the repository root, Phase 4 notes) and in #1357.
 
 To report a concern with any of this, see `SECURITY.md` at the repository root.