Skip to content

fix(compilers/openapi): read an unquoted date or << as a string - #1371

Open
OmarAlJarrah wants to merge 8 commits into
mainfrom
fix/openapi-plain-date-strings
Open

OmarAlJarrah wants to merge 8 commits into
mainfrom
fix/openapi-plain-date-strings

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Summary

YAML 1.2, which OpenAPI recommends (3.1.2 §Format), has no timestamp type: its core schema reads an unquoted date such as 2021-01-01 as a string, and OpenAPI's limit on tags ("Tags MUST be limited to those allowed by YAML's JSON schema ruleset") leaves !!str the one tag that can hold it. The YAML parser tags it !!timestamp, and the parser library's model decodes no string from a timestamp (docs/parser-library.md#undecoded-scalar). So every string field written that way lowered as if it were left out, with exit 0 and nothing reported:

Written as a plain date Before After
info.title, info.version name and version absent the date
a tag's name, an operation's tags a tag definition with no name, and the group g/tags with the empty name g/tags/2021-01-01
operationId, summary the name falls back to the hint get_b, no summary the date
required: [2021-01-01] the property is optional required
a schema's title, a response's description absent the date
$ref on a path item, parameter, request body, response, header, link, callback or 3.2 content entry the object is gone and nothing says a reference failed followed like any other $ref, and reported as openapi/unresolved-ref when it reaches nothing
a path item's summary dropped when alone; replaced by the referent's beside a $ref, with no collision reported kept, and the collision is reported
two parameters named by different dates a false openapi/duplicate-parameter error, and IDs that spell no name two parameters, each ID spelling its name
an http scheme's scheme a false openapi/incomplete-security-scheme error, and the scheme lowers as unresolved a custom scheme with that token
a discriminator's propertyName a discriminator naming no property: IR that --verify refuses as ir/empty-prop-ref the property it names

A << written as a value failed the same way. The parser tags a plain << !!merge wherever it stands, and the model decodes no string from a merge, so title: << lowered with no name and $ref: << dropped the object it stood for with exit 0, where $ref: "<<" is followed and reported as openapi/unresolved-ref. Only a mapping key merges, and YAML 1.2 reads << as a string.

The load phase now tags each such scalar !!str before the model is built: any date written with no tag, and any plain << but a mapping key (load.mistagged). This happens in the walk that already clears anchors, so it adds no walk; releaseAnchors is renamed readyForModel since it now does both. The walk judges each scalar from its parent, which knows a key from a value, so a << key still merges. The walk runs on the source after an overlay applies, which covers a date the overlay grafts, and on every external document the reader prepares. One route reached the model without that walk: an external document named by a URL that net/url respells is parsed by the resolver itself. That document was rebuilt only when its parse held an anchor. Now it is also rebuilt when the parse holds such a scalar (anyUnready).

A default, enum member, example or extension already kept a date or a << as its text (value.Classify), so the retag changes nothing there.

A scalar the source tags outright keeps its tag. Retagging an explicit !!timestamp soon would keep as text a default that values refuses today, which is #245's question to answer. And !!binary aGk= means the bytes hi, not the text aGk=. A field under an explicit tag outside the ruleset (!local, !!binary, !!timestamp) still reads as left out with nothing reported, the same as on main. That is filed as #1368, and the rule Status lines and comments that cited this issue for local tags now cite #1368. nodeview.Strings, auth.unresolvableSchemeDiags and auth.docsBeside read a requirement's scopes and a scheme's $ref from the node. They stay, because they still serve those spellings.

Exit status moves in both directions. A document whose $ref is written as a plain date or a plain << now exits 1 where it exited 0, because the reference is followed and reported when it reaches nothing. A document that drew a false duplicate-parameter or incomplete-security-scheme error from a plain date now exits 0. The IR's shape and IRVersion are unchanged.

Skills. Two skill rules list the spellings a reader of raw YAML is probed with, and neither named a scalar the parser tags as no string: a plain date, a plain << value, a !!binary, a local tag. That is where the model and the rule table part, and two defects in #1226 got through grids built from those lists. morphic-openapi-compiler's yaml-alias-anchor-merge-spellings now names them in its fixture set and its Probe, and morphic-verifying-changes' generator-width now names them among the spellings a generator writes. The Probe is the comparison this PR's tests make: at each string position, diff the IR and diagnostics against the ordinary spelling.

Landing beside other PRs. #1227 landed on main and conflicted with this change in one Status paragraph of docs/compilers/openapi.md#references, which both rewrite. The merge of main keeps #1227's text with its sentence citing #1280 replaced by this change's, citing #1368. No open PR names releaseAnchors, anyAnchored, hasAnchor or plainDate, which this change renames, none adds a spec writing a plain date or << value, and none conflicts with this head that does not already conflict with main.

Closes #1280
Closes #1282

Test plan

  • e2e.TestCompile_APlainDateCompilesAsItsQuotedTwin compiles each site in the issue and its comments, an http scheme token, the other spellings yaml.v3 resolves as a timestamp (one-digit fields, a date and time with a T or a space), a value, an extension and a key. It requires the IR, diagnostics included, to equal that of a twin that quotes every date. Twins also cover a date in an external document, read from a file and over HTTP, and a date an overlay grafts. load.TestResolveExternal_RecoversAPlainDateBehindARespelledURL covers the respelled URL. e2e.TestCompile_ADateTaggedOutrightKeepsItsTag checks that an explicit !!timestamp soon default is still refused with the warning it drew before.
  • e2e.TestCompile_APlainMergeValueCompilesAsItsQuotedTwin runs the same rows with each date written <<, all but the three that write a date as a key, where << merges, against a twin quoting each <<. load.TestUnready holds the rebuild gate to a << value and item, and not to a << key or a << quoted or tagged outright. The undecoded-scalar canary pins that the model reads a plain << as it reads a plain date.
  • The yaml-timestamp-scalars conformance case now writes a plain-date info.version and operation summary, so the harness sweep and the order oracle reach plain dates at string fields. The library's validator calls both type mismatches. They are recorded in validation_disagreements.txt under a new reason, plain-date, which cites §Format and YAML 1.2's core schema.
  • Defects planted one at a time in a throwaway worktree, each run against the new tests, at 55716bf:
    • plainDate always false, rerun on the pushed tree, reddens 13 of the 15 twin rows, both external-document subtests, the overlay twin, the yaml-timestamp-scalars case, TestUnready and the respelled-URL test. The value and key rows lowered alike on main already; they stay as guards that the retag changes nothing there.
    • Dropping the TaggedStyle check reddens TestCompile_ADateTaggedOutrightKeepsItsTag and TestUnready.
    • Skipping the retag on external documents reddens both external-document subtests and the respelled-URL test.
    • Restoring the anchors-only rebuild gate reddens TestUnready and the respelled-URL test.
  • And on the final tree:
    • The << arm always false reddens 11 of the 12 << twin rows and TestUnready's value and item rows; the value row lowered alike already.
    • Retagging a << key too reddens the yaml-merge-keys conformance case, TestUnready's merge-key row, and tests reading merged schemas, servers and media types.
    • Judging the key position off by one reddens the << twin rows, TestUnready and the yaml-merge-keys case.
    • Retagging a << only inside a mapping reddens the << twin row for an operation's tags and TestUnready's item row.
    • After main was merged in, dropping the date arm, the << arm or the whole predicate still reddens the tests above.
  • Corpus, final heads (main @3188471d against this branch): every YAML spec in testdata and the reference corpus, 1,758, compiled on both. Exit codes match on every spec (1,418 exit 0, 340 exit 1 on each side). Two specs differ, each only where a date becomes text: the yaml-timestamp-scalars case gains its version and summary, and one reference spec gains version: "2021-03-25", which it writes unquoted at info.version. No spec in either writes a plain << value, and JSON writes no plain scalar of either kind. At 55716bf, .claude/skills/morphic-verifying-changes/scripts/sweep.py over 2,181 specs, JSON included, against main @6996ce45 matched exit codes on every spec (1,661 exit 0, 520 exit 1 on each side) with the same two differences, and the case's dates as defaults, enum members, examples and extensions lowered the same on both sides.
  • Golden spec, cloudflare/api-schemas @c312f753, compiled with --verify at 55716bf: exit 1 on both builds, with the same IR byte for byte past sources, and the same 1,862 diagnostic lines. It is JSON, so it writes no plain date or <<.
  • A twin sweep over the same 2,181 specs, at 55716bf, to find any site the twin test does not name. For each spec, a pair of documents re-encoded alike, one with every string value but structural ones (openapi, type, in, style, format, $ref and the schema identifiers) replaced by an unquoted 2021-01-01, the other with the same date quoted: 2,177 pairs and 367,785 values. On this branch the two halves of every pair agree in exit code, IR and diagnostics. On main, 2,075 pairs differ, and 353 plain halves break an IR invariant under --verify, every one of them ir/empty-prop-ref from a discriminator's propertyName, the site that sweep added to the table and the twin test. The 4 specs left over write no replaceable value or do not parse (a stream whose first document is empty, a deliberately malformed spec).
  • morphic-harness at 55716bf, over testdata (227 files) and the reference corpus (2,079): line for line the same outcomes on main and this branch. Over the twins on this branch, each plain half gets the verdict its quoted half does, the paths it names aside, on all 2,173 pairs it reads; 678 plain halves pass every oracle, determinism, idempotence and order-invariance among them, and the rest stop at an error diagnostic the replacement causes in both halves.
  • Cost. The retag is one predicate per node, each judged once from its parent, in a walk the load phase already runs. Instructions retired, main @3188471d and this head run alternately on one thread with the collector off, 7 runs a side, on a 722 KB YAML spec from the reference corpus: +0.05% at the median against a 0.3% range on each side. At 55716bf, the golden spec as JSON (27 MB) and as block YAML (21.7 MB), too large to hold with the collector off, read −1.2% and −0.8% with each side's range at 1–9%, which is noise either way. A document read under a respelled URL that writes a plain date or << value anywhere, a value included, now costs a second resolution pass, as one writing an anchor already did.
  • At 55716bf, go test ./... and make fmt vet lint passed locally. On the final tree, go build ./..., go vet ./compilers/openapi/..., internal/archtest, the plain-date, <<, lowering-spec, conformance and external-resolution tests, and skills_lint.py pass; the two skill rules sit within the 150-word cap. The full gate, coverage included, is left to CI.

OpenAPI holds YAML to its JSON schema ruleset, where an unquoted date such
as 2021-01-01 is a string. The YAML parser tags it a timestamp, and the
parser library's model decodes no string from a timestamp, so every string
field written that way lowered as left out: the document's title and
version, a tag's name, an operation's ID and summary, a required property
name, a parameter's name, an http scheme's token. A $ref so written read as
no reference, so the object it stood for vanished with nothing reported.
Nothing reported any of it, and two parameters named by different dates
drew a false duplicate-parameter error.

The load phase now tags such a scalar a string in the walk that already
releases anchors, before the model is built, in the source and in each
external document it prepares. A document the resolver parsed itself under
a respelled URL is rebuilt when it holds a plain date, as when it holds an
anchor. A scalar the source tags outright keeps its tag, so what an
explicit !!timestamp or a local tag reads as stays undecided here.
The retag of plain dates reads the parser's tag, not the text, so every
spelling yaml.v3 resolves as a timestamp is a string: a date with
one-digit fields, a date and time with a T or a space, and a date under
YAML's non-specific tag. Pin each, at the predicate and end to end, so a
reading narrowed to one spelling fails.
A plain date, a !!binary and a local tag are where the parser library's
model and the rule table part: the table reads any scalar as a string,
and the model decodes none of these. The fixture set the raw-YAML
reader rule names and the spellings a generator is told to write left
all three out, so two defects of a security requirement sat at a plain
date through grids of about 200 specs. Name the three in both rules.
A twin sweep over the corpus, each spec compiled with its string values
swapped for a plain date and for the same date quoted, found one site the
twin test did not name: on main a plain-date discriminator propertyName
reads as empty, so the discriminator names no property and the IR fails
ir/empty-prop-ref under --verify. The load phase's retag already reads it
as written; hold it there.
The YAML parser tags a plain `<<` as a merge wherever it stands, and the
parser library's model decodes no string from a merge, as it decodes
none from a timestamp. So a `<<` written as a value read as left out
exactly as a plain date did: `title: <<` lowered with no name, and
`$ref: <<` dropped the parameter it stood for with exit 0 and nothing
reported, where `$ref: "<<"` is followed and reported unresolved.

Only a mapping key merges. YAML 1.2, which OpenAPI recommends, has no
merge or timestamp type, and its core schema reads both scalars as
strings. The load phase's walk now judges each scalar from its parent,
so it knows a key from a value, and tags a string any date written with
no tag and any plain `<<` but a key (mistagged). A `<<` key still
merges, and a scalar tagged outright keeps its tag.

The documents stated the reading as the JSON schema ruleset's. That
ruleset calls a plain scalar matching none of its patterns an error
(YAML 1.2.2 §10.2.2); it is the core schema (§10.3.2) that reads one as
a string, and OpenAPI's limit on tags leaves !!str the one tag that
holds it. The yaml-decoding and undecoded-scalar entries now say so.

The twin test's rows run again with each date written `<<`, but those
writing a date as a key, where `<<` merges. The undecoded-scalar canary
pins that the model reads a plain `<<` as it reads a plain date.
The validation oracle's plain-date reason said OpenAPI holds YAML to
its JSON schema ruleset and so reads a plain date as a string. That
ruleset reads such a scalar as an error; YAML 1.2's core schema reads
it as a string, and OpenAPI's limit on tags leaves !!str the one tag
that holds it. The reason now says that.

The references Status named only a local tag among the $ref spellings
the model decodes no string from. A $ref written `!!binary aGk=` or
`!!timestamp 2021-01-01` drops its object with nothing reported too, so
the line names every tag outside the JSON schema ruleset, as the
yaml-decoding Status does. A security Status line left unwrapped is
rewrapped.
The parser tags a plain `<<` a merge wherever it stands, and the model
decodes no string from it, so a `<<` written as a value reads as left
out just as a plain date does. The two skill rules naming the scalars
the parser tags as no string listed a plain date, a !!binary and a
local tag, and a fix for dates built from them left the `<<` value
standing. Name it beside the plain date in both.
@OmarAlJarrah OmarAlJarrah changed the title fix(compilers/openapi): read an unquoted date as a string fix(compilers/openapi): read an unquoted date or << as a string Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant