feat(ir)!: settle doc references, promotion records and names by OmarAlJarrah · Pull Request #1219 · dexpace/morphic

OmarAlJarrah · 2026-11-01T04:31:05Z

Summary

Three places where the IR's contract said less than a consumer needs, closed together because each is a statement about what a document means rather than how it is lowered.

References in prose (closes #721). Docs.Description was documented to hold {t:TypeID} tokens, which a text scan cannot tell from author prose of the same shape, and which pass/validate left unchecked for that reason. No compiler wrote one. Docs gains Refs: byte spans of Description that name a type, so the prose stays as the source wrote it.

The escape rule the issue offered fails on real input. CommonMark reads no backslash escape in a code span, and of the 253 descriptions compiled from the golden spec that hold a brace, 121 hold it in one. docs/ir/annotations.md gives the reason beside each design it rejects.
A span's offsets carry their unit in their names, StartByte and EndByte, because the JSON is what a consumer outside Go reads, and start invites an index into that language's own strings, which is wrong on non-ASCII text with nothing to say so.
The check the issue wanted needs no parser: a DocRef's Type is a TypeID, so both checkers report a dangling one as ir/dangling-type-ref. irverify adds ir/doc-ref-span and ir/doc-ref-order.
A reference is a mention, not a use, so it is no edge of the type graph. pass/validate_graphql.go needed a change for this: its reflective walk read every TypeID under an operation, so a type named in an operation's docs would have become a GraphQL root.
It is still a reference to whatever resolves one: CollapseAliases lowers a span naming a decorator to the type the decorator stands for, and a span whose type renders as nothing a link can reach stays text.
A span indexes the description it was written against, so each fold of Docs that takes a field from the first source setting it, the use-site merge and CollapseAliases' fold of a decorator chain, counts a description and its Refs as one field.

Where a promotion is recorded (closes #674). The target list named nodes the fourth rule forbade, and nothing said whose provenance records a promotion that writes into another node. The issue offered a Provenance on each struct or the nearest ancestor. Neither is what this does: the record lands on the node keeping the extension, in Provenance.Inferred, or in a bare Inferred where that node has none, as TypeRef already does. ErrorCase and OAuthFlow gain one.

The ancestor fails on a concrete input: a key on one response would mark its operation, where nothing beside the marker explains it, so a declared deprecated: true reads as the guess and two error cases cannot be told apart.
A full Provenance is correct but locates what the entry's own Provenance already locates, at the cost of a pointer on each of the golden spec's 5,753 error cases and a rewrite of every golden holding one.
docs/ir/unmodeled.md now has a table of each target, the node keeping it and its marker, TestPromotionTable_EveryNodeCanRecord reads that table against what ir declares, and TestExtensionTarget_EveryTargetIsInThePromotionTable requires a row for every target the compiler declares, so the list can drift from neither.

Name stability (closes #680). Emitters render identifiers from Canonical and Hint, so a change to how either is derived renames generated symbols with no key of the document moved. docs/ir/ir-version.md now states that this is versioned like a change of shape, and the issue's golden goes further than review visibility: ir/testdata/names.golden.txt hashes into a new names column of the generations table, so a name changed at an unmoved version fails.

The grammar decides at a rune from that rune and its two neighbours, so the golden lists every name of up to three runes over one rune of each class the Unicode tables tell apart, which reaches every decision it makes, and each ASCII rune between two letters, which pins the runes that separate. It records unicode.Version too: a Go release moving those tables changes derived names, and now fails instead of passing silently.
The grammar's conformance table stays out of the hash. A row added there changes no name, so it must not cost a breaking bump.
The issue asked for a golden of the grammar alone. Hint derivation is the other half of the same defect, so the file also lists the hints of testdata/golden/openapi/hint_positions.yaml, a spec holding one position for each way the naming composes a hint from one.

Limitations, stated where they live. No compiler sets Docs.Refs, ErrorCase.Inferred or OAuthFlow.Inferred yet, so each is a line of testdata/unwitnessed.golden.txt; OpenAPI has no syntax for the first (docs/compilers/openapi.md#docs-and-deprecation). The hint pin reaches what its spec writes, so a position outside it, such as a document a $ref reads whole, remains a bump no test requires (docs/ir/ir-version.md), and adding a position moves the hash, so it lands with whichever change bumps next.

Open issues this touches. It settles the marker question #877 lists, and the ordering #656 waits on; both stay open for the promotions themselves. #662 will need a bump under this policy, since it renames hints. #469's Protobuf compiler is one that would fill Docs.Refs, recorded with EDoc as the matrix row docs-type-references. #713's naming epoch is an emitter's and stays separate. #1221's keys read under a scope stay open: the rule its cause cites is reworded here, and still keeps a key off a node that cannot record it. #677 stays open too: a DocRef resolves in the document holding it, so these documents promise nothing across revisions, and they say a link follows what its target renders as rather than that it survives a rename.

Test plan

main was merged in at 608a5a46, so this head holds #1191 and #1192. CI's gate passed on 6dd4dfed, and the one commit after it changes two documents; the gate on this head is CI's.

Breaking

IRVersion moves to 0.24.0, with Docs.Refs, ErrorCase.Inferred and OAuthFlow.Inferred new in the shape. A consumer pinned to 0.23.0 refuses the document as another generation's, and one that scanned descriptions for {t: tokens must read Refs. #1226 and #1302 also claim 0.24.0 and #1229 a version from an older base, so each that merges after another takes the next version, and its row takes the names hash TestIRVersion_PinsTheNames prints.

Docs.Description was documented to hold {t:TypeID} tokens, which a text scan could not tell from author prose of the same shape, and which pass/validate left unchecked for that reason. No compiler wrote one. Docs gains Refs, spans of Description that name a type, so the prose stays as the source wrote it. A DocRef's Type is a TypeID like any other, so the existing dangling-reference checks reach it with no parser, and irverify holds each span to whole runes inside the text and the list to ascending order. A reference is a mention, not a use: no edge of the type graph, and no root of GraphQL reachability. IRVersion moves to 0.23.0. A consumer pinned to 0.22.0 refuses the document, and one that scanned descriptions for tokens must read Refs.

The promotion rules listed targets the fourth rule forbade: ErrorCase and EnumMember have no Provenance, and some promotions write into another node than the one carrying the extension, with nothing saying whose provenance records them. The record now lands where the extension is kept, so a marker always sits beside the entry that explains it, whichever node holds the field. ErrorCase and OAuthFlow gain a bare Inferred, as TypeRef has, so a key on a response or a flow can be read. A table states each target's node and marker, and a test reads it against what ir declares, so a target no node can record fails rather than reading as settled. The ancestor and a Provenance on every struct were weighed and rejected; docs/ir/unmodeled.md gives the reason beside each. No promotion the OpenAPI compiler makes moves.

Emitters render identifiers from Naming.Canonical and Naming.Hint, so a change to how either is derived renames generated symbols with no key of the document moved. Nothing held that to the version, and the compatibility policy spoke only of the shape. ir/testdata/names.golden.txt now describes those names: the Unicode tables the grammar reads, its answer for every window it decides over one rune of each class, and the hints of a spec written to reach each position the OpenAPI compiler composes one from. Its hash is a new column of the generations table, so a name changed at an unmoved version fails, and a Go release moving Unicode fails with it. docs/ir/ir-version.md states the policy. A hint at a position that spec does not write remains the one bump no test requires, and it says so.

irverify finds an inferred-marker list by its type, so ErrorCase.Inferred and OAuthFlow.Inferred were held from the moment they existed, but no case showed it. markerCarriers now reaches them, and TypeEdges says why a DocRef holding a TypeID is no edge, beside the positions it lists that are.

The sentence that names DocRef took the doc comment past the cap. DocRef's own comment gives the reason, so this one need only name the exclusion.

The table of what a promotion can fill was held to the structs in one direction only: a row naming a field or a node ir does not declare fails, and a target with no row passes. A list of targets kept apart from the code is how the list before it drifted, and the next target added to the vocabulary would have repeated that with every test green. Each ExtensionTarget now has to name a field in the table's first column, read from the cells so that prose around the table does not count as a row.

The IRVersion constant still said only a change of shape bumps it, and promotion.go still said a promoted field's own node is marked, where the rule is now the node keeping the extension. The bump procedure said no other golden moves, which is false where a hint moved: the bump rewrites none, and the change behind it moves every corpus golden holding that hint. The rule for two PRs claiming a version named only the shape test, though the row now also takes the hash the names test prints.

Added directly under the hand lists bullet, the goldens bullet sat against that bullet's last line, which #1238 rewrites, so the two conflicted on nothing but position. One bullet down, they merge. A shorter values bullet keeps the body within its cap once #1219 lands too, which together with this branch's additions it would pass.

…-pin 0.23.0 landed on main with the file body, enum and union changes, so this branch takes 0.24.0: main's row and history entry stay as they are, and its own follow them.

TestValidate_DanglingDocRef was inserted under the doc comment of TestValidate_DanglingRefInMessagePayload, so that comment described the wrong test and the test it belongs to had none. TestPromotionRows_ReadsTheTableOnly compared a struct of unexported fields with assert.Equal; a value like that goes through cmp.Diff.

checkDocRefs and appendTypeIDs read v.Type() for every value a walk reaches, where the checks beside them ask v.Kind() first and pay for the type only on a struct.

"Survives a rename" read as a claim that an ID outlives a rename of what it was derived from, which a pointer-derived ID does not. A span resolves against the document that holds it, so what holds is that a link says what its target renders as in that SDK.

describeNames listed every spelling in canonicalCases, so a row added to the grammar's conformance table moved the hash with no name changed: the pin then reported that generated symbols had been renamed, and asked for a breaking bump to add a test case. The table stays out of the description now. It held those spellings to pin the runes that separate, which one class hides from the three-rune names, so the description lists each ASCII rune between two letters instead: that is derived, covers every ASCII rune rather than the few the table happened to spell, and catches a plant the table passes. The finding also says what an added hint position does, since that moves the hash with no name changed too.

The spec behind the names pin was said to write every position one document can, and did not. Rebuilding the plants from the naming code left nine that moved a hint with the pin still green: a callback operation's name, the server OpenAPI supplies, a composition branch of an allOf, and each map whose key names a schema, since no key in the spec was spelled like a keyword the walk treats as a role. The spec now writes those, a webhook named by its route, a header on an encoding and the positions under a component path item and callback. docs/ir/ir-version.md says what was checked rather than what cannot be: one position for each decision the naming takes.

CollapseAliases listed each reference to an aliased type it rewrites, and DocRef.Type is a new one: a span naming a decorator would have pointed at a type that emits no declaration. The list names it now. The same reading left one case open in docs/ir/annotations.md: a span whose type renders as nothing a link can reach, such as a primitive, stays text, which is what a reader that resolves no references sees anyway.

isSpanOf's comment said a byte that does not decode counts as a boundary, but utf8.RuneStart rejects a continuation byte whether or not it belongs to a rune, so a stray one reads as a split rune. The comment says that now and two rows hold both sides of it.

Each cmp.Diff this branch adds now prints which side is which, and the GraphQL reachability test holds its two cases as rows rather than closures in a map, each planted on its own: removing the exclusion in appendTypeIDs reddens the operation's row, and making a DocRef an edge of ir.TypeEdges reddens the reachable type's.

A second set of plants over the naming code left seven that moved a hint with the pin green. Beneath a component schema a hint read from the pointer alone reads the same as one read from its owner, so a role only written there cannot tell the two apart: dropping a kind from the walk up to the owner changed nothing the spec wrote, though it would rename response_item, the name of every array response without a component. The spec now writes the roles under a response, a request body, a parameter and a header, a structure a reference reads inside raw YAML, a key under an extension spelled like a role, and the two shapes variantHint tells apart. "Every position one document can", and the "valid document" this branch first narrowed it to, were both false: the spec keeps two component keys the validator refuses. What holds is one position for each way the naming composes a hint from a position, and docs/ir/ir-version.md says that. How an owner's name is read, null and non-scalar spellings included, is TestAt_ReadsAnOwnerAsTheParserDoes's.

TestIRVersion_PinsTheNames reads OpenAPI's hint spec alone, and a hint is composed per compiler, so one a later compiler moves would rename what emitters render with nothing to report it. The recipe says so at the step that adds a compiler's corpus, and says nothing fails there. The spec's header is rewrapped, which moves the golden's source hash.

Start and End count bytes of UTF-8, and only their comments said so. The JSON is what a consumer outside Go reads, and {"start":4} invites an index into that language's own strings, which is wrong on non-ASCII text with nothing to say so. StartByte and EndByte state the unit where it is read, as MaxSourceBytes does. No compiler writes either field and the 0.24.0 row is unpublished, so this is the last point the rename costs nothing. The same section's account of a span that stays text is rewritten as two sentences.

isSpanOf returned one expression of five terms across two lines. Each half now has a name, and the rune edges are read only once the span is known to lie inside the text, which is what lets them index it.

An additionalOperations key reaches an operation's hint verbatim, where a fixed field's name is upper-cased, so upper-casing it would rename m_search_structures to msearch_structures with the pin green, and a query operation's hint was not written at all.

A span indexes the description it was written against, so a fold that takes each field of Docs from the first source setting it can pair one source's Description with another's Refs, and link the wrong words. docs/ir/annotations.md said whatever replaces a description replaces its spans, but only in its own section. CollapseAliases' fold of a decorator chain and the use-site merge in "What a reference carries down" each fold field by field and said nothing of it; both now count a description and its Refs as one field. No compiler sets Refs yet, and the fold happens in a refiner, past anything irverify reads.

#1219 landed 0.24.0 first, so this change keeps the row and the history entry main holds and appends its own under 0.25.0, with the names hash the pin main added requires. Its history entry now names 0.24.0 as the generation a pinned consumer holds. The conformance and golden cases #1217 and #1219 added each declare the unresolved auth namespace.

main took IRVersion 0.24.0 for #1219, so this change moves to 0.25.0, with its own generations row and history entry. #1217 made an expanded $dynamicRef run the keyword census. That census asks keywordHome at no node, which now homes a collection bound the reader fills, so the keywords-at-every-outcome carrier holds its maxItems in Constraints. The case now writes a false uniqueItems beside it too, which stays verbatim, so the census at that carrier is still pinned. The rule's Status drops the items #1217 closed and the clause saying a bound beside an expanded $dynamicRef is dropped.

Resolves the conflicts with #1217 and #1219, and what their rules ask of the readers this branch adds: - keepsAtOutcome takes referenceOutcomeKeywords for every outcome that is not a refinement, so an expanded $dynamicRef keeps its census, and unmergedBranchKeys keeps #1217's null check beside the predicate this branch passes it. - A null keyword is the keyword left out (#1217). writtenKinds and saysKind read `type: null` as no type written (writesType), where a null type had admitted nothing and kept `{allOf: [{type: null}]}` a Model; voicesAmong and restates pass over a null keyword, where `description: null` on one branch had kept another's description verbatim as a disagreement. - declaresScoped passes the context declaresPositionScoped now takes. - TestAllOf_ANullTypeExcludesNoObject composes, so it still reaches the branch residue it pins: its old spec now refines. - A node at an inline allOf branch takes the name of the schema holding it, which moves a hint the name pin (#1219) holds, so IRVersion moves to 0.25.0 with a row keeping 0.24.0's shape.

#1219 took 0.24.0 and #1226 0.25.0, so this change moves to 0.26.0, keeps both rows and history entries, and appends its own. #1219 turned the field list of docs/ir/unmodeled.md#promotion into a table held to the code both ways. It gains the fields this change fills: a Deprecation itself at each carrier, Operation.Stability and Parameter.Secret, with TypeCommon named among the nodes keeping a deprecation's or a Secret's extension, as a $ref's target lends both to a use. TestExtensionTarget_EveryTargetIsInThePromotionTable now names every field a target fills rather than one, since TargetRedaction fills three. OperationGroup joins Variant and EnumMember as a node rule 4 names, and TestPromotionTable_EveryNodeCanRecord holds it there. #1219's rule 3 records a promotion on the node keeping the extension. A field a $ref's target lends a use is the one case this change records twice: the target keeps the entry and its marker, and the use lists the marker too, since a use is read without resolving its target and its type reaches the entry. Rule 3, extension-promotion and the 0.26.0 history entry say so. #1217 ended fillPropertyAnnotations in PreserveCarriedKeywords; the promotion this change moved to PromoteUse stays out of it. #1226 rewrote lowerSecurityScheme's return, which now calls the Promoter. The two new conformance goldens gain the unresolved auth namespace every OpenAPI document declares since #1226.

A message #1219 wrote on main named PromoteDeprecation, which this change replaced with Promoter.Deprecation, and the merge kept it.

OmarAlJarrah added 5 commits November 1, 2026 05:51

docs(ir): keep the TypeEdges comment within the word cap

d5d99e1

The sentence that names DocRef took the doc comment past the cap. DocRef's own comment gives the reason, so this one need only name the exclusion.

This was referenced Nov 1, 2026

skills: a new TypeID position that is no edge has no rule for the reflective readers #1223

Closed

fix!: keep a security requirement naming no scheme #1226

Merged

feat(ir)!: type the secrecy and stability a document states #1229

Merged

OmarAlJarrah added 2 commits November 1, 2026 17:27

This was referenced Nov 1, 2026

chore(skills): harden sweep.py and close gaps in the rules #1210

Merged

fix(compilers/openapi)!: lower allOf by what its conjuncts say #1269

Merged

OmarAlJarrah added 10 commits November 2, 2026 00:39

Merge remote-tracking branch 'origin/main' into feat/ir-doc-refs-name…

ff95256

…-pin 0.23.0 landed on main with the file body, enum and union changes, so this branch takes 0.24.0: main's row and history entry stay as they are, and its own follow them.

perf: check a value's kind before its type in doc ref walks

ca75f7b

checkDocRefs and appendTypeIDs read v.Type() for every value a walk reaches, where the checks beside them ask v.Kind() first and pay for the type only on a struct.

OmarAlJarrah added 3 commits November 2, 2026 02:31

docs(ir): rewrap the paragraph on a span's unit

b0e9998

OmarAlJarrah added 3 commits November 2, 2026 03:04

refactor(ir/irverify): name the operands of a span's check

044014c

isSpanOf returned one expression of five terms across two lines. Each half now has a name, and the rune edges are read only once the span is known to lie inside the text, which is what lets them index it.

OmarAlJarrah merged commit 39afce9 into main Nov 2, 2026
2 checks passed

OmarAlJarrah deleted the feat/ir-doc-refs-name-pin branch November 2, 2026 00:38

fuad-daoud mentioned this pull request Nov 2, 2026

ci: run coverage in set mode, and -race only where code is concurrent #1271

Open

OmarAlJarrah mentioned this pull request Nov 2, 2026

skills: edit-pr-body-from-live-copy sends a body without checking the fetch #1326

Open

OmarAlJarrah added a commit that referenced this pull request Nov 2, 2026

test(compilers/openapi): name Promoter.Deprecation in a carrier check

cd0e837

A message #1219 wrote on main named PromoteDeprecation, which this change replaced with Promoter.Deprecation, and the merge kept it.

Navigation Menu

Repository navigation

feat(ir)!: settle doc references, promotion records and names - #1219

feat(ir)!: settle doc references, promotion records and names#1219
OmarAlJarrah merged 23 commits into
mainfrom
feat/ir-doc-refs-name-pin

OmarAlJarrah commented Nov 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

OmarAlJarrah commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Breaking

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

OmarAlJarrah commented Nov 1, 2026 •

edited

Loading