Repository navigation
Keep the result of a native build whose inputs did not change - #2946
Merged
Merged
Conversation
A native build that found its frontend entry, its objects and its executable in the caches still walked the whole pipeline to say so: it read and parsed the frontend entry (which carries every source text), wrote the LLVM text back out, recovered the objects, re-checked the target and the feature set and, when the executable cache is unavailable — a helper from SCRIPTC_LLVM_HELPER, an FFI manifest, a profile, a linker flag — re-emitted and re-linked the program. On fx-ts a no-change build took 72 ms, on tsc-ts 959 ms, and 427 ms on fx-ts with a helper set. Record the build instead: the frontend cache key, the identities of the toolchain and the linker, the content digests of the profile and the FFI libraries, the frontend's own input probes, and the identity of the executable, the kept LLVM text and whatever the build left beside it. A later build whose key matches and whose every probe still matches returns the executable already on disk, and costs a few dozen stat calls. A miss is the pipeline it ran before, including the sources edited while a build ran and the outputs another tool replaced. No-change builds: fx-ts 71.9 ms to 16.6 ms, fx-ts with a helper set 426.8 ms to 17.6 ms, tsc-ts with a profile and an FFI manifest 959 ms to 58.6 ms. Cold builds and one-line edits are unchanged, with the same binaries and the same outputs.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
A record of a native build could outlive inputs it does not name. The record of a Darwin dev build held the identity of the `.dSYM` bundle's directory, which a debug tool rewriting the DWARF file inside it leaves as it was. The key held the runtime pack's package.json only, so a pack whose artifacts or manifest were edited in place, or a link input replaced under it, still reported the old executable for unchanged sources. The key held the profile's digest only, so a block-count file beside the profile could change without one. And a build that writes block counts with SCRIPTC_RECORD_PROFILE_BLOCKS gets that file from the helper, which a kept output never runs. Record every file of a beside directory, and the runtime pack artifacts the build linked, and verify them on every hit. Key the pack's manifest and the profile's block-count file. Record nothing at all when a build writes block counts, which is what such a build already did to the caches, or when it is sanitized: a sanitizer runtime is compiled from sources outside the pack, and the key does not name them.
An FFI manifest can name a library by its linker name (`-l<name>`) and, on Darwin, a framework by bundle name. The linker resolves both through search paths, and the unchanged-build key names the manifest's bytes, not the files those names resolve to: replacing the resolved library in place left a build with unchanged sources writing the executable that was linked against the old one. The complete-artifact cache already refuses to serve a link that supplies either form for this reason; the unchanged-build record now keeps nothing for such a build, so it links again and the link sees the file. A manifest that names its libraries by path is unaffected: the key digests each of those files.
A toolchain manifest's linker arguments reach the link unchanged: a `-l` name, a `-L` directory or a driver name such as `cc` makes the linker resolve archives through search paths of its own. The unchanged-build key hashed the arguments but not the files they resolve, so replacing an archive in place left a build with unchanged sources writing the executable that was linked against the earlier one. No record is taken when the toolchain supplies linker arguments. The complete-artifact cache already refuses a link that supplies any for the same reason (`executable-cache.ts` requires `toolchain.linkerArgs.length === 0`), and a build without them keeps its record.
`SCRIPTC_LINKER` can name a wrapper with inputs of its own, and `SCRIPTC_LLVM_HELPER` can name a code-generation wrapper that produces the objects the link consumes. The unchanged-build key names such a wrapper's file, its arguments and the environment's spelling, not the archives, libraries or plugin files the wrapper reads or adds, so a file replaced in place would leave a kept executable from the earlier build. The complete-artifact cache and the linker job cache already refuse a build that sets either variable (`openNativeExecutableCache` and `LinkerJobs.open` return null); the unchanged-build record now keeps nothing for them as well. The mutable toolchain variables name files and directories that change behind a stable spelling — search paths, SDKs, config files, include paths — while the key hashes only the spelling. A build with any of them set keeps nothing either, through the same `toolchainEnvironmentCachePolicy().completeArtifacts` the executable cache consults. A variable the fingerprint hashes by value (`CC`, `CFLAGS`, `LDFLAGS`, `AR`, `RANLIB`, `LC_ALL`, ...) keeps its record: changing the value changes the key. The toolchain manifest, the runtime pack and the dSYM tool are recorded by identity: the key names the resolved linker, its arguments, the pack's manifest and consumed artifacts and, for a dev build, the dSYM tool's file.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SCRIPTC_LLVM_HELPERset 426.8 ms → 17.6 ms (4.1% [3.7, 4.5]); tsc-ts with a profile and an FFI manifest 959 ms → 58.6 ms (6.1% [5.0, 7.2]). Cold builds and one-line edits are unchanged, including the binaries they produce.