Skip to content

perf(plugins): read each source once across the prepasses and compilers - #1033

Merged
owjs3901 merged 7 commits into
mainfrom
perf/webpack-prepass-reuse
Nov 19, 2026
Merged

owjs3901 merged 7 commits into
mainfrom
perf/webpack-prepass-reuse

Conversation

@owjs3901

@owjs3901 owjs3901 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Before the bundler starts, the import graph, the class-slot reservation and the prewarm each read the project's sources from disk, and the prewarm re-reads the files the reservation just read. A Next production build then runs all of them again for each of its three compilers (server, edge-server, client), because the engine is reset between compilers. So every source was read about 6–9 times per next build.

They now read through one stamp-checked cache per build:

  • plugin-utils: new readSource(path) / withSourceCache(run, cache). Inside withSourceCache, a file is served from the cache while its mtime/ctime/size stamp is unchanged. Checking the stamp costs about a fifth of reading the file (300 files: stat 2–3 ms vs read 10–16 ms on Windows). Outside it, readSource reads the disk as before. The import graph, the reservation discovery, Next's prewarm collection and the module resolver's cache misses read through it.
  • next-plugin: one cache for a production build's webpack compilers, handed to each DevupUIWebpackPlugin (sourceCache, adapter-owned like productionPrewarmFiles); one for the Turbopack config-phase graph and prewarm.
  • webpack-plugin: initialize and each compilation's preparation (reservation + prewarm) run inside the build's cache, or a fresh one per call.
  • Watch and dev builds get no shared cache (sourceCache is only passed when !dev), so each compilation still starts from a fresh cache and nothing can go stale between edits.

Engine work (scratch reservation and prewarm extraction) still runs per compiler: the engine is reset between compilers and the server/client resolve conditions differ. Sharing it would change the engine-state contract, so it is out of scope here.

Measurements (local, Windows, 4 alternating A/B runs, native engine)

CPU sampled in Devup frames (--cpu-prof, all processes of the build):

Fixture before (main) after
next-devup-ui (webpack) 423 / 399 / 389 / 273 ms (median 394) 262 / 188 / 153 / 157 ms (median 172)
300-component app (webpack, same fixture with 300 generated Flex/Box/Text components) 3962 / 2216 / 1478 / 1172 ms (median 1847) 1239 / 1093 / 937 / 675 ms (median 1015)
Turbopack next.setup (DEVUP_UI_PROFILE, single-turbo) 164 / 93 / 84 / 94 ms 87 / 60 / 55 / 60 ms

Per compiler on the 300-component app (instrumented, main): the reservation discovery took 36–70 ms, of which reads were most, and the prewarm read 300 files again (23–31 ms) every compiler. The local machine is shared with other builds, so absolute numbers are noisy; the direction held in every pair. On Linux CI reads are cheaper, so expect a smaller gain there. The CI benchmark job of this PR shows it.

Output

Byte-identical CSS and page JS (md5) on next-devup-ui, next-devup-ui-single, next-devup-ui-single-turbo, next-devup-ui-turbo, next-devup-ui-collapse and the 300-component app.

Notes

Test plan

  • New source-cache.test.ts: no cache outside a run, a cache hit under any spelling of the path, a changed file read again, a deleted file reported and evicted, nested runs keep the outer cache, the active cache is restored when the run throws
  • next-plugin: one Map shared by the server and client compilers of a production build, none for dev
  • webpack-plugin: a compiler prewarms from what an earlier compiler of the build read, without reading again
  • bun test packages/plugin-utils packages/next-plugin packages/webpack-plugin: 5213 pass (with rsbuild-plugin), 0 fail; 100% for every changed source file
  • eslint

요약

번들러가 시작되기 전에 import graph, class-slot 예약(reservation), prewarm이 각각 프로젝트 소스를 디스크에서 읽고, prewarm은 예약이 방금 읽은 파일을 다시 읽습니다. Next 프로덕션 빌드는 컴파일러 사이에 엔진을 리셋하므로 이 과정을 컴파일러 3개(server, edge-server, client)마다 다시 돕니다. 그래서 next build 한 번에 소스 하나를 약 6~9번 읽었습니다.

이제 빌드당 하나인, stamp로 검증하는 캐시를 거쳐 읽습니다.

  • plugin-utils: readSource(path) / withSourceCache(run, cache)를 추가했습니다. withSourceCache 안에서는 mtime/ctime/크기 stamp가 그대로인 동안 캐시에서 내줍니다. stamp 확인 비용은 읽기의 약 1/5입니다(Windows에서 300개 파일: stat 23 ms, read 1016 ms). 밖에서는 지금처럼 디스크를 읽습니다. import graph, 예약 탐색, Next prewarm 수집, 모듈 resolver의 캐시 미스가 이걸 거쳐 읽습니다.
  • next-plugin: 프로덕션 빌드의 webpack 컴파일러들이 캐시 하나를 함께 쓰고, 각 DevupUIWebpackPlugin에 넘깁니다(sourceCache, productionPrewarmFiles처럼 어댑터 전용). Turbopack config 단계의 graph와 prewarm도 캐시 하나를 씁니다.
  • webpack-plugin: initialize와 컴파일마다의 준비(예약 + prewarm)가 빌드의 캐시 안에서, 없으면 호출마다 새 캐시 안에서 돕니다.
  • watch·dev 빌드에는 공유 캐시가 없습니다(!dev일 때만 넘김). 그래서 컴파일마다 새 캐시로 시작하고, 편집 사이에 오래된 내용이 남을 수 없습니다.

엔진 작업(예약의 scratch 추출, prewarm 추출)은 여전히 컴파일러마다 돕니다. 컴파일러 사이에 엔진이 리셋되고 server/client의 resolve 조건이 다르기 때문입니다. 이걸 공유하려면 엔진 상태 계약이 바뀌므로 이 PR 범위 밖입니다.

측정 (로컬, Windows, A/B 교대 4회, native 엔진)

Devup 프레임에서 샘플링된 CPU(--cpu-prof, 빌드의 모든 프로세스):

픽스처 이전 (main) 이후
next-devup-ui (webpack) 423 / 399 / 389 / 273 ms (중앙값 394) 262 / 188 / 153 / 157 ms (중앙값 172)
컴포넌트 300개 앱 (webpack, 같은 픽스처에 Flex/Box/Text 컴포넌트 300개 생성) 3962 / 2216 / 1478 / 1172 ms (중앙값 1847) 1239 / 1093 / 937 / 675 ms (중앙값 1015)
Turbopack next.setup (DEVUP_UI_PROFILE, single-turbo) 164 / 93 / 84 / 94 ms 87 / 60 / 55 / 60 ms

컴포넌트 300개 앱에서 컴파일러마다(main 계측): 예약 탐색이 3670 ms였고 대부분이 읽기였습니다. prewarm은 컴파일러마다 파일 300개를 다시 읽었습니다(2331 ms). 로컬 머신을 다른 빌드와 함께 쓰고 있어 절대값은 흔들리지만, 모든 짝에서 방향은 같았습니다. Linux CI에서는 읽기가 더 싸서 효과가 더 작을 것입니다. 이 PR의 CI benchmark job에 나옵니다.

출력

next-devup-ui, next-devup-ui-single, next-devup-ui-single-turbo, next-devup-ui-turbo, next-devup-ui-collapse, 컴포넌트 300개 앱 모두 CSS와 페이지 JS가 바이트 단위로 같습니다(md5).

참고

테스트

  • 새 source-cache.test.ts: 실행 밖에서는 캐시 없음, 경로 표기가 달라도 캐시 적중, 바뀐 파일은 다시 읽음, 삭제된 파일은 오류를 내고 캐시에서 빠짐, 중첩 실행은 바깥 캐시 유지, 실행이 throw해도 활성 캐시 복원
  • next-plugin: 프로덕션 빌드의 server·client 컴파일러가 Map 하나를 공유하고 dev는 없음
  • webpack-plugin: 컴파일러가 같은 빌드의 앞 컴파일러가 읽은 내용으로 prewarm하고 다시 읽지 않음
  • bun test packages/plugin-utils packages/next-plugin packages/webpack-plugin: 5213 pass (with rsbuild-plugin), 0 fail; 바뀐 소스 파일 모두 100%
  • eslint

🤖 Generated with Claude Code

The import graph, the class-slot reservation and the prewarm each read the
project's sources from disk, and a Next production build runs all of them
again for its server, edge and client compilers. They now read through a
stamp-checked cache that one build shares: plugin-utils' `readSource` serves
a file again from the cache while its modification stamp is unchanged, inside
`withSourceCache`. Next hands one cache to the webpack plugins of all its
production compilers and uses one for the Turbopack setup; a plain webpack
build shares one per preparation. Watch and dev builds still start from a
fresh cache for each compilation. The module resolver keeps its own cache and
reads misses through the shared one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
packages/next-plugin/src/plugin.ts 100.00% <100.00%> (ø)
packages/next-plugin/src/prewarm.ts 100.00% <100.00%> (ø)
packages/plugin-utils/src/import-graph.ts 100.00% <100.00%> (ø)
packages/plugin-utils/src/index.ts 100.00% <100.00%> (ø)
packages/plugin-utils/src/reservation-discovery.ts 100.00% <100.00%> (ø)
packages/plugin-utils/src/source-cache.ts 100.00% <100.00%> (ø)
packages/webpack-plugin/src/plugin.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

owjs3901 and others added 6 commits November 19, 2026 14:06
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ions

The Turbopack loader options no longer carry singleCss (#1036), so the
one-route test (#1035) checks what the coordinator extracts into.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@owjs3901
owjs3901 merged commit 3594eb3 into main Nov 19, 2026
71 of 73 checks passed
@owjs3901
owjs3901 deleted the perf/webpack-prepass-reuse branch November 19, 2026 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant