Skip to content

docs(emitters): verify a generation before, during and after it - #1339

Open
OmarAlJarrah wants to merge 5 commits into
docs/emitters-docs-artifactsfrom
docs/emitters-verification
Open

OmarAlJarrah wants to merge 5 commits into
docs/emitters-docs-artifactsfrom
docs/emitters-verification

Conversation

@OmarAlJarrah

Copy link
Copy Markdown
Member

Summary

How a generation is checked, at the three points where it can be wrong (docs/emitters/verification.md). These issues are one contract, since each check reads what another records.

Before generation: unhonored rows (#707).

  • What a document uses is a new derived fact, irfacts.Rows (docs/ir/derived-facts.md#capability-rows): for each capability-matrix row, by its stable key, the locations exercising it. A test holds one detector per row to the matrix's keys, as the conformance corpus is held.
  • What a target honors is Capabilities.Rows. The engine reports the difference before Generate as emitters/<target>/unhonored-row.
  • The same sets generate each target's support matrix.

During: step versions (#702).

After: the output builds (#704).

  • This is an engine stage over the staged output, run before the writer moves files into place, so a failing generation never reaches the user's tree unless the caller lets it.
  • Each target declares toolchains (probe command, version pattern, minimum, install guidance) and checks (build, lint, test) in Capabilities.Verify.
  • A missing or old toolchain skips its checks and says so: a check that could not run never reads as a pass.
  • A failure maps to the entity whose declaration holds the failing line, through a new Artifact.Spans, rather than to a path alone, which is all Origins gives.
  • The usage samples from docs(emitters): design the docs emitter and its shared examples #1337 compile in a snippet project, and CI runs the stage over every golden.

After: the manifest is checked (#584).

  • Coverage is exact, every emitted operation and type with exactly one entry, because the manifest is built from Origin rather than inferred, so a tolerance would only hide our bugs.
  • Resolution checks that every manifest symbol exists in the built output, through the same extractor surface verification reads an SDK with.

Change records and severities (#715) (docs/emitters/surface-verification.md).

  • Records carry direction (from Facts.Usage), response optionality, and bindings in the behavioral channel.
  • The seeded severity table gives one verdict per channel. The issue's proposal mixed two questions. A response enum gaining a member does not break a caller's Go code; it does break a deployed SDK reading a closed enum, and a Kotlin when that must be exhaustive.
  • The structural channel answers whether callers' code builds, and the behavioral channel whether a deployed SDK still works against the new API.
  • The bump rule: structural breaking is major, additive minor, any behavioral record at least minor, any other artifact change a patch (a moved step included), and no change no bump. It never comes from a spec checksum.

apidiff's role (#795). It is decided now rather than deferred, on a spike against golang.org/x/exp/apidiff at its October 2026 revision (BSD-3-Clause, actively developed). apidiff is the test oracle for Go's structural severity: a differential test over the surface corpus holds verify.Severity's Go verdicts to apidiff's.

  • It cannot be the differ, since its records carry no IR ID.
  • It cannot be the severity, since it answers only whether Go code builds. On the spike, an added enum constant came back compatible, correct for callers' code, while the behavioral channel must still report it.
  • As a test-only dependency it needs no compatibility promise from x/exp.

Spike verdicts (apidiff.Changes on two type-checked packages): a field string → *string was incompatible, an added constant compatible, an added struct field compatible, an added method parameter incompatible.

T-41 to T-44 pin the four checks.

Closes #707
Closes #702
Closes #704
Closes #584
Closes #715
Closes #795

Test plan

  • apidiff spike run as described; licence and commit history read through the GitHub API.
  • The Verify, Toolchain, Check and Span declarations, extracted verbatim, pass go vet.
  • CI: the doc tests over the new document, its map entry and anchors.

Stacked on #1337 (#1329 ← #1331 ← #1332 ← #1334 ← #1335 ← #1336 ← #1337). Merge in order. After each base lands, "Update branch" is a merge of main that conflicts nowhere.

Nothing required generated output to build, a target found the
constructs it could not lower one at a time inside Generate, a
regeneration could not say what changed or why, nothing checked the
manifest, and surface verification's records lacked the facts its
verdicts turn on.

The engine now checks a generation at three points. Before it, the
capability-matrix rows a document exercises, a new derived fact, are
compared with the rows a target declares, and each one unhonored is
reported. During it, every refine step and template carries a version
the manifest records. After it, a verify stage builds, lints and tests
the staged output and compiles the usage samples, mapping each toolchain
message to the entity whose declaration holds it, and the manifest is
checked for exact coverage and for symbols that resolve in the build.

Change records now carry direction, response optionality and bindings,
the seeded severity table gives one verdict per channel, structural for
callers' code and behavioral for deployed SDKs, and the version bump
follows from them. apidiff is the test oracle for Go's structural
verdicts, neither the differ nor the severity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant