[Upd4ting]

Why

orderBy("title.fr"), lookup(other, localKey, "title.fr") and an index { fields: ["title.fr"] } read a top-level key literally named title.fr instead of the fr member of title. Localized and _internal sorts and lookups therefore answered wrong on postgres: every row tied in a sort, and a lookup found nothing. The mysql benchmark (AntelopeJS/mysql bench/RESULTS.md) marks these postgres rows ✗. ANT-2089.

Summary

A dotted name now reads a nested path, with the rule AntelopeJS/mysql uses. A field declared under the exact name wins. Otherwise the name is a path below its first segment, which may be a declared JSON field (title.fr) or an undeclared key (_internal.data.title.fr).

• NamedField(source, name) in row-source.ts is the one reading.
• IndexColumns (orderBy, getAll, between, group) and the other key of lookup read through it.
• The index DDL builds its key with the same NamedField over the table's own columns. The index expression is therefore the expression the read compiles to, and the planner uses it.
• Two cases PostgreSQL cannot index are handled at registration, because a failed build stops the schema's whole index phase:
  • A path below a declared value with no members (name.fr over name: "string") reads through to_jsonb, which is not IMMUTABLE, and always reads NULL. Registration refuses it with a clear message, as mysql does.
  • A date member below a declared field reads through a timestamptz cast, which is not IMMUTABLE. Its index keys on the stored JSON member, with a JSON null read as absent. It builds and enforces unique, and serves no read.
• Index signatures of non-dotted names (declared fields, _id, undeclared keys, GIN, crossInstance twins) are byte-identical to main, so no deployed index rebuilds except those over dotted names.
• docs/storage.md documents the rule; docs/errors.md lists the new refusal and the reworded reserved-column message.
• @antelopejs/interface-database rises to >=0.3.3 <0.4.0, the release of feat(staged-query): read dotted index fields, orderBy names and lookup keys as nested paths interface-database#87 that carries the shared dotted_paths conformance suite.

flowchart LR
  N["name: 'title.fr'"] --> D{"declared under<br/>the exact name?"}
  D -- yes --> C["that column"]
  D -- no --> P["source.field('title')<br/>then member 'fr'"]
  C --> R["read: orderBy / getAll / between /<br/>group / lookup other key"]
  P --> R
  C --> I["index key DDL"]
  P --> I
  I --> X{"IMMUTABLE?"}
  X -- "path below a scalar" --> F["refused at registration"]
  X -- "date member" --> J["key on stored JSON member"]
  X -- yes --> S["same expression as the read:<br/>index serves it"]

Loading

Related issue

ANT-2089. The shared contract and conformance suite come from ANT-2090 (AntelopeJS/interface-database#87).

Verification

src/test/nested-paths.test.ts, run with pnpm exec ajs module test . -f dist/test/nested-paths.test.js.

Before (origin/main source, same test file):

  dotted names read nested paths
    1) orderBy sorts by a path below a declared field
    2) orderBy sorts by a path below an undeclared key
    3) orderBy by an index over a path sorts by that path
    4) getAll by an index over a path and between by a path match that path
    5) lookup finds the books row by a path below a declared field
    6) lookup finds the books row by a path below an undeclared key
    7) lookup finds the booksIndexed row by a path below a declared field
    8) lookup finds the booksIndexed row by a path below an undeclared key
    9) a unique index over a path refuses a second row with the same nested value
    10) an index over a date below a declared field builds, refuses a duplicate, and the sort reads dates
    ✔ an index over a date below a declared field holds a stored null as absent
    ✔ the index over a path serves a sort and a match by that path

  2 passing
  10 failing

  1) orderBy sorts by a path below a declared field:
      AssertionError: expected [ 'book-0', 'book-1', 'book-2', …(2) ] to deeply equal [ 'book-1', 'book-4', 'book-3', …(2) ]
  4) getAll by an index over a path and between by a path match that path:
      AssertionError: expected [] to deeply equal [ 'book-3' ]
  5) lookup finds the books row by a path below a declared field:
      AssertionError: expected [ undefined, undefined ] to deeply equal [ 'book-3', 'book-1' ]
  9) a unique index over a path refuses a second row with the same nested value:
     AssertionError: the insert of book-9 was not refused

After:

  dotted names read nested paths
    ✔ orderBy sorts by a path below a declared field
    ✔ orderBy sorts by a path below an undeclared key
    ✔ orderBy by an index over a path sorts by that path
    ✔ getAll by an index over a path and between by a path match that path
    ✔ lookup finds the books row by a path below a declared field
    ✔ lookup finds the books row by a path below an undeclared key
    ✔ lookup finds the booksIndexed row by a path below a declared field
    ✔ lookup finds the booksIndexed row by a path below an undeclared key
    ✔ a unique index over a path refuses a second row with the same nested value
    ✔ an index over a date below a declared field builds, refuses a duplicate, and the sort reads dates
    ✔ an index over a date below a declared field holds a stored null as absent
    ✔ the index over a path serves a sort and a match by that path

  12 passing

The two tests that pass on main guard regressions of the fix rather than the original bug. The plan test fails when the reads move to the nested path and the DDL does not: the index then no longer matches the read. The null test fails when the date key drops its NULLIF: two rows without a date then conflict under unique. Both were checked by patching the built dist.

Shared conformance suite dotted_paths from @antelopejs/interface-database 0.3.3 (-f node_modules/@antelopejs/interface-database/dist/tests/dotted_paths.test.js): origin/main 1 passing, 6 failing; this branch 7 passing, 0 failing.

src/test/unit/ddl.spec.ts adds the refusal of a path below a scalar or a date member, and a declared dotted field (_id.x) keying on its own column.

pnpm run checks: Checks: 11 passed, 0 failed.

Upgrade notes for the release

• An index declared over a dotted name is rebuilt on the next registration with its nested key. A unique one fails to rebuild if the nested values hold duplicates; the previous index keeps serving and the failure is logged.
• orderBy, getAll, between, group and a lookup other key no longer read a top-level key literally named a.b from _extra; they read the b member of a.
• A schema declaring an index on a path below a scalar field (name.fr over name: "string") is now refused at registration. Its index never served a read.

Checklist

• The pull request title follows Conventional Commits.
• I added or updated tests when behavior changed.
• I updated documentation when users or contributors are affected.
• I documented breaking changes and migration steps.
• I did not include credentials or unrelated changes.