Skip to content

fix(compilers/openapi): keep a keyword at every schema outcome - #1217

Merged
OmarAlJarrah merged 12 commits into
mainfrom
fix/openapi-census-every-outcome
Nov 2, 2026
Merged

OmarAlJarrah merged 12 commits into
mainfrom
fix/openapi-census-every-outcome

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Summary

A schema keyword reached the IR only on some of the outcomes a schema lowers through. Three outcomes returned before the keyword census ran, the gate that gives a shared position a node of its own never asked about what the census keeps, and a null was read a different way depending on the keyword and the node it sat on. On Cloudflare's schema this loses minimum and maximum beside the oneOf of spectrum-config_origin_port.

Every outcome now answers the census:

  • A Union keeps what it has no field for, a value bound included.

  • A type array and a distributed union ask their variants, since each variant is built from the schema itself. What a variant consumes is neither kept nor reported, and anything else is kept on the union. A format beside a distributed properties is kept; the properties are not.

  • An expanded $dynamicRef lowers to a reference, so it takes a $ref site's census with a collapsed union (referenceOutcomeKeywords), and a carrier reaches the same function.

  • A parameter keeps what a property and a header keep. Its filler carried its own copy of that list, which had lost the content vocabulary and an unexpanded $dynamicRef, so both vanished without a report. All three carriers now go through schema.PreserveCarriedKeywords, so they cannot drift again.

The hoist gate reads what the census would keep (annotation.WritesUnknownKeyword, undefinedKeywords) rather than a second list, so an unknown keyword, and a nullable the dialect does not define, lands on an alias at a position with no node or carrier.

A null for a Schema Object keyword reads as the keyword left out. That is decided once, in annotation.RawPropertyNode, which every keeper goes through, rather than per keeper, with const, default and example excepted because their value is an instance. schema.writesType and effectiveTypes do the same for type, which the parser decodes as a type spelled "" (docs/parser-library.md#null-type-is-empty, with its canary). unmergedBranchKeys lists keys rather than asking for one, so it asks annotation.WritesKeywordLeftOut. The two readers of the raw tree that decide whether a $dynamicRef expands read it the same way: a null $id starts no resource, and a null $dynamicAnchor is no anchor named null. The table grades every such null as a field left out, numeric bounds included, whose shape it otherwise leaves to the lowering.

Why #847 and #764 are in here

An unknown keyword beside a $ref now hoists an alias, and an alias is exactly what a mapping misses, so a document that compiled with the keyword silently dropped would have started failing pass/discriminator-missing-variant. Two shapes reach it, and fixing only the first would make this PR worse than main on the second.

A mapping naming a component a branch aliases (#847). A mapping target now routes to the variant whose branch names it (referencedVariants). That reads the branches, not the registry, so it turns on no lowering order. Two options were open in the issue: route in the lowering, or teach validation to read through aliases. The lowering is the right side because an emitter routes by Mapping, which has to name a variant.

A type two branches name maps to neither, in either branch order. If one of them is the type itself the mapping stays on that variant, and if both alias it the pass reports it, which is the stated rule the issue asked for, though it refuses a valid document (below).

A mapping naming a position whose $ref has something beside it (#764). readsThrough used to stop there, keeping an alias that is no variant or subtype. It no longer asks what is beside the $ref, only whether the position is itself a variant, so the mapping means what the $ref names. That also retires refSiteHomesNothing.

Behaviour to check on review

Test plan

  • For every keyword of the schema table at 3.0.3, 3.1.0 and 3.2.0, at 17 positions and 24 outcomes, 58,786 pairs compiled with the keyword and without it, comparing IR and diagnostics past the source hash and line numbers, with irverify on each. Identical output means it vanished silently: 6,015 pairs on main and 3,078 here, with 2,937 fixed and no pair worse, and no violation on either. The 3,078 are 2,888 for the four exemptions docs/compilers/openapi.md#vocabulary-coverage names, and 190 where the keyword says nothing new: a type a string enum already implies, an allOf: [{type: object}] on an object, and a 3.0 nullable on a component nothing references. This check is a throwaway script; the conformance cases and operation.TestParams_KeepWhatAnyCarrierKeeps hold the shapes.
  • e2e.TestCompile_ANullKeywordLowersAsTheKeywordLeftOut reads every keyword off the table at 3.0.3, 3.1.0 and 3.2.0, in four positions and several contexts, and requires the IR to equal the same document with the key removed. It found what a list of the issue's examples would not: a 3.0 exclusiveMinimum: null, a 3.0 $ref: null still reading its siblings as beside a $ref, dependentRequired and $vocabulary, which have no model field, and an allOf branch counting a null as residue. e2e.TestCompile_ANullIdentifierLowersAsTheKeywordLeftOut holds the two readers that sweep cannot reach, as it writes no $dynamicRef. A wider throwaway run of 54,740 pairs, adding a union, a type array and a $ref as what each keyword sits beside, differs only under not and in an allOf branch, which keep the schema verbatim, and at openapi: a keyword failing to decode beneath itself turns the outer schema false #1299's shape. A third run of 136,568 pairs adds outcomes (type arrays mixing scalars with objects, a discriminated union, a distributed union with allOf or required, a tuple, a 3.0 nullable), positions (component parameters, headers, bodies and responses, path-level parameters, callbacks, encoding headers, propertyNames, if, contentSchema) and a second value per keyword, each empty or falsy: none is worse than main, 6,414 are fixed, and what both builds leave alike is a value that states nothing, such as title: '' or deprecated: false, or openapi: unevaluatedProperties false on a position lowering to no model is dropped #1283's shape.
  • New conformance cases with goldens: keywords-at-every-outcome, unknown-keywords-at-shared-nodes, null-keywords, discriminator-aliased-branch.
  • 44 planted defects across the guards this adds, each confirmed compiling and each reddened by a named test. Survivors along the way showed four redundant guards in the mapping code, since removed, and gaps in the sweep and in the tests of a routed defaultMapping and of a null $ref outside a Schema Object, since filled.
  • sweep.py with --verify against main over testdata and morphic-references (2,175 specs): 1,652 stay 0, 519 stay 1, 4 go 1→0 (the two specs above and two of the new cases, which main refuses), and none get worse. 16 specs differ. I read the 12 real-world ones and each is one of: a bound beside a union kept (api_constrained, anyOf.json), $recursiveRef at a branch kept, a null error turned warning, or an invalid type, such as an unquoted type: [string, null], no longer lowering a variant for the empty type.
  • Injecting a keyword beside every $ref, beside every type, and a bound and a format beside every oneOf, anyOf and type array, into 1,396 documents drawn from those two corpora, through harness.Check on both builds: nothing panics, nothing breaks irverify, determinism or order-invariance, and the one spec going ok to error is the openapi: a mapping to a type two union branches alias is refused #1298 shape above. Compared with the same spec before injection, with each alias the keyword minted read through to its target and its own entry set aside, every type, property, parameter and mapping is unchanged.
  • Cloudflare's openapi.json @319dc756af: one new info, at spectrum-config_origin_port, and that type's unmodeled is the only change to the type registry.
  • BenchmarkCompile_Petstore: 52 more allocations of about 12,051 (0.4%), and on Cloudflare's schema 0.3% more instructions retired, medians of three alternating runs whose ranges overlap. A first version returned a slice from a helper, which pushed GetType's stack-allocated slice to the heap at every read; a profile diff found it, and the predicate that replaced it does not.
  • CI's gate on the final head: go test ./... -race with morphic-covergate at 100% of statements, vet, build, lint and the planted-lint check.
  • A trial merge with feat(compilers/openapi): read each document's schema identifiers #1227, which deletes internal/anchors and declaresResourceIDAbove, passes go test ./compilers/openapi/internal/schema ./compilers/openapi/internal/bind and the e2e conformance, lowering-spec and null tests, as bind already reads a null identifier as left out. It conflicts in five files, the dynamic-ref rule among them, and needs two fixes no conflict marks, which that PR lists. TestBuild_ANullAnchorNamesNothing and its citation in null-keywords go with the package. fix(compilers/openapi)!: lower allOf by what its conjuncts say #1269 conflicts in five files and adds a caller of collapsedUnhomedKeywords, which this removes, feat(ir)!: type the secrecy and stability a document states #1229 conflicts in three, and fix(compilers/openapi): treat a restated union branch as bare #1278 in docs/compilers/openapi.md.

Closes #848
Closes #852
Closes #1053
Closes #1069
Closes #847
Closes #764

A keyword reached the IR only on some of the outcomes a schema lowers
through. Three outcomes returned before the census ran: a Union, a
multi-typed schema and an expanded $dynamicRef. The gate that hoists a
node for a shared position never asked about a keyword the census keeps,
so an unknown keyword there went nowhere. A null keyword was read three
ways depending on the keyword and the node it sat on.

Every outcome now runs the census. A collapsed union and an expanded
$dynamicRef take a $ref site's, and a type array and a distributed union
ask their variants, each built from the schema itself, so what they
consume is neither kept nor reported. The gate reads what the census
would keep rather than a second list.

RawPropertyNode, the one reader every keeper goes through, reads a null
as absent, unless the keyword's value is an instance. declaredTypes does
the same for `type`, which the parser decodes as a type spelled "". The
table grades each such null as a field left out, a bound included.

An unknown keyword beside a union branch's $ref now hoists an alias, and
an alias is what a mapping naming that component missed, so a document
that compiled with the keyword dropped would have failed validation. A
mapping target now routes to the one variant aliasing it.
A null keyword reads as the keyword left out, decided in
annotation.RawPropertyNode, but two readers of the raw tree decide
whether a $dynamicRef expands and go through neither it nor the model.

declaresResourceIDAbove took a null $id for a resource boundary, so a
null at or above either end stopped the expansion. anchors.record
indexed a null $dynamicAnchor under its text, so `$dynamicRef: '#null'`
expanded to a schema naming no anchor. Each now reads the key as absent,
as bind's own reader of those keywords already does.
Two guards this branch adds had no test that fails without them: routing
a defaultMapping to the variant aliasing its target, and reading a null
$ref as no reference only in a Schema Object. The second is out of a
document's reach while the parser refuses a null $ref elsewhere (#1284),
so it is pinned where the check runs.
The rules now say a mapping to a type two variants alias is refused
(#1298), and that a null beneath the same keyword turns the outer schema
false (#1299), neither of which the branch fixes. The lines it added past
the column limit are rewrapped, and a comment that named DecidedKeywords
after the census stopped passing them is corrected.
A limitation the branch leaves open belongs where the next reader of
that code reaches it, so referencedVariants names #1298 and the boolean
reading of a schema names #1299.
The type-arrays rule said each variant holds what its own type admits,
but a value bound lands on any variant with a Constraints field, one
whose type it cannot constrain included, which #1310 tracks; it now says
so where the rule and the census state it. The unknown-keys rule said a
keyword the dialect does not define is left out, when the census keeps
it as it keeps an unknown one, and hoistDeclarationHome named only a
collapsed union among the outcomes lowering to a reference.
The specs these tests compile spelled newlines and quotes by hand, which
rule 12.11 of the styleguide asks to write as raw string literals.
@OmarAlJarrah
OmarAlJarrah merged commit 94561e9 into main Nov 2, 2026
2 checks passed
@OmarAlJarrah
OmarAlJarrah deleted the fix/openapi-census-every-outcome branch November 2, 2026 00:37
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
#1219 landed 0.24.0 first, so this change keeps the row and the history
entry main holds and appends its own under 0.25.0, with the names hash
the pin main added requires. Its history entry now names 0.24.0 as the
generation a pinned consumer holds. The conformance and golden cases
#1217 and #1219 added each declare the unresolved auth namespace.
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
main took IRVersion 0.24.0 for #1219, so this change moves to 0.25.0,
with its own generations row and history entry.

#1217 made an expanded $dynamicRef run the keyword census. That census
asks keywordHome at no node, which now homes a collection bound the
reader fills, so the keywords-at-every-outcome carrier holds its maxItems
in Constraints. The case now writes a false uniqueItems beside it too,
which stays verbatim, so the census at that carrier is still pinned. The
rule's Status drops the items #1217 closed and the clause saying a bound
beside an expanded $dynamicRef is dropped.
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
Resolves the conflicts with #1217 and #1219, and what their rules ask
of the readers this branch adds:

- keepsAtOutcome takes referenceOutcomeKeywords for every outcome that
  is not a refinement, so an expanded $dynamicRef keeps its census, and
  unmergedBranchKeys keeps #1217's null check beside the predicate this
  branch passes it.
- A null keyword is the keyword left out (#1217). writtenKinds and
  saysKind read `type: null` as no type written (writesType), where a
  null type had admitted nothing and kept `{allOf: [{type: null}]}` a
  Model; voicesAmong and restates pass over a null keyword, where
  `description: null` on one branch had kept another's description
  verbatim as a disagreement.
- declaresScoped passes the context declaresPositionScoped now takes.
- TestAllOf_ANullTypeExcludesNoObject composes, so it still reaches the
  branch residue it pins: its old spec now refines.
- A node at an inline allOf branch takes the name of the schema holding
  it, which moves a hint the name pin (#1219) holds, so IRVersion moves
  to 0.25.0 with a row keeping 0.24.0's shape.
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
#1219 took 0.24.0 and #1226 0.25.0, so this change moves to 0.26.0,
keeps both rows and history entries, and appends its own.

#1219 turned the field list of docs/ir/unmodeled.md#promotion into a
table held to the code both ways. It gains the fields this change
fills: a Deprecation itself at each carrier, Operation.Stability and
Parameter.Secret, with TypeCommon named among the nodes keeping a
deprecation's or a Secret's extension, as a $ref's target lends both
to a use. TestExtensionTarget_EveryTargetIsInThePromotionTable now
names every field a target fills rather than one, since TargetRedaction
fills three. OperationGroup joins Variant and EnumMember as a node rule
4 names, and TestPromotionTable_EveryNodeCanRecord holds it there.

#1219's rule 3 records a promotion on the node keeping the extension.
A field a $ref's target lends a use is the one case this change records
twice: the target keeps the entry and its marker, and the use lists the
marker too, since a use is read without resolving its target and its
type reaches the entry. Rule 3, extension-promotion and the 0.26.0
history entry say so.

#1217 ended fillPropertyAnnotations in PreserveCarriedKeywords; the
promotion this change moved to PromoteUse stays out of it. #1226
rewrote lowerSecurityScheme's return, which now calls the Promoter.
The two new conformance goldens gain the unresolved auth namespace
every OpenAPI document declares since #1226.
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
The merge of #1217 taught voicesAmong and restates to pass over a key
written as null, which reads as the keyword left out: a branch writing
`description: null` had disagreed with one writing `description: d`,
keeping d verbatim, and items restating a $ref's but for a null
keyword had kept their branch. Neither reader had a test of its own,
and TestCompile_ANullKeywordLowersAsTheKeywordLeftOut writes one branch
alone, where no disagreement can arise. These pin both, in both orders
for the first.
OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026
A refinement's inline principal that lowers as a type array dropped
what none of its variants holds, as a type array written alone did, and
the allof-refinement rule and keepsAtOutcome named that as a limit
(#852). #1217 closed it: the census asks the variants and keeps such a
keyword on the union, and a principal reaches the same census. So
`{allOf: [{type: [string, integer], minLength: 1}]}` keeps minLength on
its union, as `{type: [string, integer], minLength: 1}` does. The limit
leaves the rule's status and the comment, and
TestRefinement_ATypeArrayPrincipalKeepsWhatNoVariantHolds holds both
spellings to it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment