[coderdan]

Summary

EQL v4's json_tree column lets PostgreSQL answer JSON containment (@>, "does this document contain that one?", and <@, the reverse) over encrypted documents. The promise is that it answers exactly as PostgreSQL's own jsonb operators would answer over the plaintexts. Until now that promise was checked against live PostgreSQL only for a fixed, hand-picked set of documents. The only randomized check compared against a Rust model of jsonb containment, never against PostgreSQL itself.

This PR adds a seeded property test that generates a few hundred documents and several hundred queries, and checks every answer against PostgreSQL's real jsonb answer, on both the index path and the sequential-scan path. It is test-only.

Changes

• packages/eql/tests/encryption/tests/json_tree.rs: new test postgres_containment_over_generated_documents_is_jsonb_containment, plus its generator.
  • Stored documents (300): nested objects and arrays up to 4 levels, keys reused across levels and siblings, arrays of objects and arrays of arrays, duplicate array elements, and numbers that are equal but stored differently (1 / 1.0 / 1e0, 100 / 1e2, 0 / -0.0, 0.5 / 5e-1). It also has strings that look like numbers, booleans or null, booleans, null, empty {} and [], root scalars and root arrays. A census asserts that each of these appears, so a later change to the generator cannot quietly drop one.
  • Needles (500): 100 random values. 200 sub-documents taken from stored documents: a pruned copy of the whole document, which is contained by construction (the test checks this against PostgreSQL), or a bare scalar element of a root array, or a pruned node from inside the document. 200 near misses: a pruned copy with one edit (a changed leaf, an added key, an added element, or a changed kind).
  • Oracle: each document is stored twice in the same row: encrypted in a public.eql_v4_json_tree column, and as plaintext jsonb. For every needle, the set of row ids that encrypted @> selects must equal the set that jsonb @> selects over the plaintext. PostgreSQL computes the jsonb answer in one statement. <@ is checked the same way. The only allowed difference is the documented one, a stored root scalar inside a needle root array. The test asserts that this is the whole difference and that the case actually occurs.
  • Both execution paths: all needles run once with enable_seqscan = off and once with every index path off. Each @> plan is checked with EXPLAIN: it must be a bitmap scan on the GIN index in the first run and a plain Seq Scan in the second. <@ has no index path, because GIN jsonb_path_ops only serves @>, so it runs as a scan both times.
  • -> path extraction (small add-on): at 20 generated key paths ($.a, $.a.b, …), the encrypted column has an entry exactly where jsonb #> finds a node. A leaf opens (decrypts) to the value jsonb holds there, compared with PostgreSQL's jsonb =.
  • Seed: fixed at 1336 and logged on every run. EQL_JSON_TREE_SEED overrides it. A failure prints the seed, the needle, and every stored document the two answers disagree on.
• packages/eql/mise.toml: test:encryption:postgres now passes --show-output, so the seed and summary line appear in the CI log even when the test passes. No other suite in that loop prints.

Encryption uses the test crate's fake key source, so no CipherStash credentials are needed. The new test is #[ignore] like the other PostgreSQL tests, so test:encryption:postgres (--ignored) runs it, and in CI that is the rust-encryption-postgres job in test-eql.yml.

Verification

Locally, against the EQL tasks' own Postgres (mise run postgres:up postgres-N, port 7432), with the v4 bundle built from this tree:

• mise run test:encryption:postgres on PostgreSQL 17: every suite passes. The new test logged: 300 documents, 500 needles, 354 needles matching some row (17,028 matches), 162 root scalars excluded from <@, 117 leaves opened and 196 containers found by ->, about 13.4 s.
• The json_tree suite on PostgreSQL 14.24, 15.19 and 16.15: all 6 tests pass. The new test takes 15–17 s on each.
• Seeds tried, all passing with no mismatch: 1336 (default), 1, 2, 3, 5, 6, 7, 42, 100, 777, 2024, 9001, 31337, 65535, 123456789, 987654321, 18446744073709551615.
• I also broke the test on purpose by removing the <@ exclusion. It failed with 68 disagreements, and each one named the seed, the needle and the stored documents (for example, row 97: 10.0 matched by the needle [10,10,[{},{}]]). That confirms the failure report is usable. The change was reverted.
• cargo fmt --check and cargo clippy --locked -p eql-bindings -p eql-encryption-tests --all-features --all-targets -- -D warnings are clean.
• vitest over scripts/__tests__/eql-* and crates-ci (the tests that parse packages/eql/mise.toml and the workflows) pass.

CI: the Rust encryption with PostgreSQL 17 job (job 120517712608) ran postgres_containment_over_generated_documents_is_jsonb_containment ... ok, and the log shows the seed line and the same counts as the local run (seed 1336, 500 needles, 17,028 matches, 162 <@ exclusions), in 10.1 s. On a pull request, test-eql.yml runs PostgreSQL 17 only. The 14–16 matrix runs on a push to main and on the nightly schedule, so for this PR, versions 14–16 were checked locally only (above).

No changeset: test-only.

Related

Refs #1336 (stacked on it: base feat/json-tree-gates).

https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3

[changeset-bot]

⚠️ No Changeset found

Latest commit: b8d7042

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}