Skip to content

EQL SQLx tests find $.hello by guessing from ciphertext lengths: have the fixture generator record its selector #1278

Closed as not planned

Description

@tobyhede

Background

EQL (Encrypt Query Language) is the SQL library we install into a customer's Postgres database to store and query encrypted JSON documents. Each field in an encrypted document is found by a SteVec selector, a keyed hash of the field's JSON path. Because the hash is keyed by the CipherStash workspace, the same path has a different selector in every workspace. That is why the SQLx test suite in packages/eql/tests/sqlx never pins a selector. It reads selectors out of the loaded test fixture (#1200).

Problem

The v3_ste_vec fixture holds ten documents, and the tests need the selector of the $.hello field. Nothing in the fixture records which selector belongs to which path. So is_hello_op (packages/eql/tests/sqlx/src/fixtures/v3_ste_vec.rs) guesses from the shape of the encrypted data. It picks the one selector that:

  • appears on every row,
  • has order terms (op) of exactly two lengths, exactly one character apart, and
  • has only one row at the longer length.

That only works because of three facts about the fixture:

  • the CLLW-OPE order-term encoding is 16 hex characters per character plus 20;
  • ROW_COUNT = 10 makes world-1..world-9 against world-10 a 9-to-1 split;
  • no other string leaf has that same profile.

Change the order-term layout, the row count or the fixture documents, and every test that uses hello_op_selector fails. The failure is loud and lists every candidate, so nothing passes silently. But the rule has to be re-derived and patched each time. coderdan raised this in review on #1204 and called it fine as a follow-up.

Proposal

  1. When the fixture is generated, compute the $.hello selector with the client that already does this: ste_vec_query_selector in src/fixtures/cipherstash.rs. An e2e test already uses it for $.hello, so its output matches the stored selector. Write the result into v3_ste_vec.sql as a small table, fixtures.v3_ste_vec_selectors (path text primary key, selector text).
  2. Let FixtureSpec::run pass extra SQL through to the script. The driver's write_script already has an extra_header slot, but run() does not expose it yet.
  3. Make hello_op_selector read the $.hello row from that table. Keep one cheap shape check (the selector carries op on every row) so a bad table still fails loudly. Then delete is_hello_op, HELLO_OP_RULE and their unit tests.
  4. Update the AGENTS.md and tests/sqlx/README.md paragraphs that describe rule-based selection.

v3_doc_integer's $.field does not need this, because its rule is just "the only op-carrying selector".

Trade-off

The lookup would trust the same client that wrote the fixture, where today it is derived independently from the data. That matters little here, because the selector only finds the entry, and the SQL tests still check every result against plaintext. Regenerating the fixture needs CipherStash credentials, so CI's credentialed prep job is the first place the change can be checked.


Generated by Claude Code

Activity

  1. added
    enhancementNew feature or request
    rustPull requests that update Rust code
    on Oct 19, 2026
  2. coderdan commented on Oct 25, 2026

    @coderdan
    Contributor

    Won't fix. This is EQL v3 behaviour, and v3 will not be changed further. The fix is in EQL v4's json_tree family (#1288, #1325, #1329, #1363), which does not reuse the v3 SteVec design.

    https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    SDKenhancementNew feature or requestrustPull requests that update Rust code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions