Skip to content

sea: generate a compile cache for the VFS modules - #66768

Open
mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:sea-vfs-code-cache
Open

mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:sea-vfs-code-cache

Conversation

@mcollina

@mcollina mcollina commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

"useCodeCache": true is currently rejected together with "useVfs": true. The reason is a plumbing gap rather than a V8 limitation: the SEA code cache is only consumed by the embedder compile paths (compileFunctionForCJSLoader(..., is_sea_main=true) and the kEmbedder branch in module_wrap.cc), while a main script in the VFS runs through the regular CJS/ESM loaders. A main-only cache would also be of little use for a VFS bundle, where the dependency graph is what costs at startup.

This PR implements the build-time half of the fix:

  • When both flags are set, --build-sea / --experimental-sea-config spins up a Node.js instance (CommonEnvironmentSetup + LoadEnvironment), mounts the assets (or the vfsArchive) exactly like the runtime does, and compiles every .js/.cjs/.mjs (and .ts/.cts/.mts with --strip-types) module in the mount without executing it, via wrapSafe / compileSourceTextModule, with a portable CompileCacheHandler whose keys are relative to the mount point.
  • The resulting entries are serialized in the compile cache file layout and embedded as reserved node:sea:compile-cache/<key> assets. The SEA provider hides node:sea: keys so they don't show up as files in the mount. Modules that fail to compile are skipped.
  • CompileCacheHandler gains a key_root for portable caches and SerializeEntries(), which shares its serialization with Persist(). The internal enableCompileCache binding takes the key root as a 4th argument.
  • The build instance runs with a cloned environment where NODE_COMPILE_CACHE* is removed, so the user's compile cache settings cannot hijack the build.

Reading the embedded entries at startup (a read-only, blob-backed CompileCacheHandler enabled from initSeaVfs, including in workers under vfsArchive) is left to a follow-up PR, which will also decide whether the main-only code cache blob still needs to be generated under VFS.

Tests: test/sea/test-single-executable-application-vfs-code-cache.js checks the number of embedded entries for the plain, ESM and ZIP fixtures (5/4/3), that NODE_COMPILE_CACHE / NODE_DISABLE_COMPILE_CACHE in the build environment do not affect the result, and that the built executable still runs. The rejection case was removed from test-build-sea-vfs-incompatible-options.js. The existing VFS SEA tests and test/parallel/test-compile-cache* pass.

—-

AI generated, reviewed by me

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/single-executable

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. single-executable Issues and PRs related to single-executable applications. vfs Issues and PRs related to the virtual filesystem subsystem. labels Oct 15, 2026
@mcollina
mcollina force-pushed the sea-vfs-code-cache branch 2 times, most recently from a35d240 to 5ee0908 Compare October 15, 2026 15:58
"useCodeCache" was rejected together with "useVfs" because the SEA code
cache is only consumed by the embedder compile paths, while a main
script in the VFS runs through the regular loaders and a main-only cache
would not cover the modules it depends on anyway.

Compile every JavaScript (and, with --strip-types, TypeScript) module in
the VFS when the preparation blob is generated, without executing them,
through a portable CompileCacheHandler keyed by the path relative to the
mount point. The entries are serialized in the cache file layout and
embedded as reserved node:sea:compile-cache/<key> assets, which the SEA
provider hides from the mount. Modules that fail to compile are skipped.

CompileCacheHandler gains a key root for portable caches and in-memory
serialization of its entries, shared with Persist(). The build runs in a
Node.js instance whose environment ignores NODE_COMPILE_CACHE.

Reading the embedded entries at startup is left to a follow-up.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.02902% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.64%. Comparing base (98dbf60) to head (5d06ff8).
⚠️ Report is 58 commits behind head on main.

Files with missing lines Patch % Lines
src/node_sea.cc 81.52% 8 Missing and 9 partials ⚠️
lib/internal/vfs/sea_compile_cache.js 93.58% 14 Missing and 1 partial ⚠️
src/compile_cache.cc 94.44% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66768      +/-   ##
==========================================
- Coverage   90.65%   90.64%   -0.02%     
==========================================
  Files         797      798       +1     
  Lines      278268   278618     +350     
  Branches    53517    53587      +70     
==========================================
+ Hits       252261   252550     +289     
- Misses      16386    16438      +52     
- Partials     9621     9630       +9     
Files with missing lines Coverage Δ
lib/internal/vfs/providers/sea.js 100.00% <100.00%> (ø)
lib/internal/vfs/sea.js 100.00% <100.00%> (ø)
src/compile_cache.h 100.00% <ø> (ø)
src/env.cc 84.23% <100.00%> (-0.07%) ⬇️
src/env.h 97.40% <ø> (ø)
src/node_modules.cc 79.67% <100.00%> (+<0.01%) ⬆️
src/compile_cache.cc 81.18% <94.44%> (+1.12%) ⬆️
lib/internal/vfs/sea_compile_cache.js 93.58% <93.58%> (ø)
src/node_sea.cc 87.81% <81.52%> (+0.08%) ⬆️

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 28, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 28, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollina mcollina added the resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. label Oct 31, 2026
@github-actions github-actions Bot added resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention. and removed resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. labels Oct 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Failed to resume CI

   ✖  test/sea/test-single-executable-application-vfs-code-cache.js
   ℹ  https://ci.nodejs.org/job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/44172/consoleText
✖  Refusing to resume CI: failures reference files changed by this PR
Full Auto Start CI output
�[36m⠋�[39m Validating Jenkins credentials
�[36m⠋�[39m Validating Jenkins credentials
✔  Jenkins credentials valid
�[36m⠙�[39m Looking for CI runs for pull request 66768
�[36m⠙�[39m Looking for CI runs for pull request 66768
�[36m⠙�[39m Getting PR from nodejs/node/pull/66768
�[36m⠙�[39m Getting reviews from nodejs/node/pull/66768
�[36m⠙�[39m Getting comments from nodejs/node/pull/66768
✔  Found PR CI job 78696
�[36m⠹�[39m Querying data for job/node-test-pull-request/78696/
�[36m⠹�[39m Querying data for job/node-test-pull-request/78696/
�[36m⠹�[39m Querying API for job/node-test-pull-request/78696/
✔  Build data downloaded
�[36m⠹�[39m Checking whether PR CI job 78696 can be resumed
�[36m⠹�[39m Checking whether PR CI job 78696 can be resumed
✔  Jenkins offers a Resume build action
�[36m⠹�[39m Checking failures against changed PR files
�[36m⠹�[39m Checking failures against changed PR files
�[36m⠸�[39m Querying data for job/node-test-pull-request/78696/
�[36m⠸�[39m Querying API for job/node-test-pull-request/78696/
✔  Build data downloaded
�[36m⠸�[39m Querying failures of job/node-test-commit/93624/
�[36m⠸�[39m Querying failures of job/node-test-commit/93624/
�[36m⠸�[39m Querying API for job/node-test-commit-windows-fanned/81480/
�[36m⠼�[39m Querying API for job/node-test-binary-windows-js-suites/44172/
�[36m⠼�[39m Querying API for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/44172/
�[36m⠼�[39m Querying API for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win11-COMPILED_BY-vs2022_clang/44172/
�[36m⠼�[39m Querying API for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win2019-COMPILED_BY-vs2022_clang/44172/
�[36m⠼�[39m Querying API for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win2022-COMPILED_BY-vs2022_clang/44172/
�[36m⠴�[39m Querying console text for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/44172/
�[36m⠴�[39m Querying console text for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win11-COMPILED_BY-vs2022_clang/44172/
�[36m⠴�[39m Querying console text for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win2019-COMPILED_BY-vs2022_clang/44172/
�[36m⠴�[39m Querying console text for job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win2022-COMPILED_BY-vs2022_clang/44172/
✔  Data downloaded
   ✖  test/sea/test-single-executable-application-vfs-code-cache.js
   ℹ  https://ci.nodejs.org/job/node-test-binary-windows-js-suites/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/44172/consoleText
not ok 1431 sea/test-single-executable-application-vfs-code-cache
  ---
  duration_ms: 6725.23600
  severity: fail
  exitcode: 1
  stack: |-
    1..0 # Skipped: Cannot find signtool: Error: - process terminated with status 1, expected 0
        at signSEA (c:\workspace\node-test-binary-windows-js-suites\node\test\common\sea.js:220:7)
        at buildSEA (c:\workspace\node-test-binary-windows-js-suites\node\test\common\sea.js:132:3)
        at main (c:\workspace\node-test-binary-windows-js-suites\node\test\sea\test-single-executable-application-vfs-code-cache.js:104:22)
        at process.processTicksAndRejections (node:internal/process/task_queues:104:5) {
      command: 'where signtool'
    }
    Mismatched noop function calls. Expected exactly 1, actual 0.
        at Proxy.mustCall (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:470:10)
        at Object.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\sea\test-single-executable-application-vfs-code-cache.js:128:20)
        at Module._compile (node:internal/modules/cjs/loader:1968:14)
        at Object..js (node:internal/modules/cjs/loader:2108:10)
        at Module.load (node:internal/modules/cjs/loader:1690:32)
        at Module._load (node:internal/modules/cjs/loader:1480:12)
        at wrapModuleLoad (node:internal/modules/cjs/loader:261:19)
        at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:173:5)
        at node:internal/main/run_main_module:33:47
    (node:1932) ExperimentalWarning: The zlib ZIP archive API is an experimental feature and might change at any time
    (Use `node --trace-warnings ...` to show where the warning was created)
    [process 2576]: --- stderr ---
    INFO: Could not find files for the given pattern(s).
    
    [process 2576]: --- stdout ---
    
    [process 2576]: status = 1, signal = null
  ...
✖  Refusing to resume CI: failures reference files changed by this PR

View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention. single-executable Issues and PRs related to single-executable applications. vfs Issues and PRs related to the virtual filesystem subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants