Repository navigation
Conversation
|
Review requested:
|
|
Welcome to Node.js, and thank you for your first contribution! Before review, please take a moment to read:
Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal. Caution AgentScan found account activity patterns that may be consistent with automation. This is a heuristic, not proof that this pull request was opened by an agent or violates policy. AI-assisted contributions are permitted, but automated tooling must not open pull requests without advance approval, and contributors must personally understand, test, verify, and take responsibility for every submitted change. See the AgentScan analysis, AI use policy, and automation policy for additional context. |
feb58df to
233f483
Compare
The memory corruption that led to disabling CRC32_SIMD_SSE42_PCLMUL came from crc_fold_copy() in deps/zlib/crc_folding.c. For the final (len % 16) bytes of a copy it did a 16-byte unaligned store, writing up to 15 bytes past the destination. With gzip at level 0, deflate_stored() copies input straight into strm->next_out through read_buf() and copy_with_crc(), so the overrun landed past the end of the caller's output buffer and corrupted the heap. Chromium replaced that store with zmemcpy(), and the fix arrived here with the zlib update in 70a0510 (v22.0.0). With the old code, the added test aborts v19.0.0 and v20.6.0 builds every time. With the current code it passes, and the CRC32 SIMD path is clean under ASan and valgrind. The restored zlib_crc32_simd target predates clang-cl support on Windows, so also pass -msse4.2 and -mpclmul through msvs_settings, as zlib_adler32_simd does. The bundled zlib only detects CPU features in deflateInit2() and when crc32() or adler32() is called with a NULL buffer. zlib.crc32() always passes a buffer, so it used the portable code until a stream had triggered the detection. Trigger it when the binding is initialized so zlib.crc32() uses the SIMD path from the start. Refs: nodejs#45268 Refs: nodejs#49511 Refs: nodejs#52123 Assisted-by: Claude Code Signed-off-by: Parker Loflin <parker@cronstable.dev>
233f483 to
e180d3c
Compare
Since #49511, x86 builds compile the bundled zlib without
CRC32_SIMD_SSE42_PCLMUL. That worked around the heap corruption in #45268, but left gzip, gunzip, andzlib.crc32()on the portable CRC path.The corruption came from
crc_fold_copy()indeps/zlib/crc_folding.c, which handled the trailinglen % 16bytes of a copy with a 16-byte unaligned store—writing up to 15 bytes past the destination. With gzip at level 0 (as in the #45268 repro usingcreateGzip({ level: 0 })),deflate_stored()copies input directly intostrm->next_outviaread_buf()→copy_with_crc(), landing the overrun past the end of the caller's output buffer. Chromium replaced that store withzmemcpy(), and Node picked up the fix in #52123 (v22.0.0, backported to v20.13.0 and v18.20.3). The CRC values themselves were always correct; the out-of-bounds store was the only bug.This PR:
zlib_crc32_simdindeps/zlib/zlib.gyp. Because the commented-out block predatesclang-clsupport on Windows (build: add option to enable clang-cl on Windows #52870), this also passes-msse4.2 -mpclmultoclang-clviamsvs_settings, matchingzlib_adler32_simd.deflateInit2()or whencrc32()/adler32()is called with aNULLbuffer; without this, standalonezlib.crc32()calls use the portable path until a stream is created.test-zlib-gzip-stored-output-boundary.js, which runs level-0 gzip into output buffers ending at every offset modulo 16.Verification (AMD Ryzen 7 9800X3D, Ubuntu 24.04, GCC 13.3, Valgrind 3.22):
x86_cpu_enable_simd=1): Built bundled zlib with the exact defines and per-file SIMD flags fromzlib.gyp. Round-tripped 29,340 gzip streams (all levels, whole and piecewise input, into exact-sizemalloc()buffers at every mod-16 end offset) and verified gzip trailers plus 335,766crc32()calls against a reference implementation (lengths 0–4200, alignments 0–63, incremental updates). Clean under both ASan and Valgrind (0 errors).heap-buffer-overflowunder ASan and invalid writes under Valgrind atcrc_folding.c:419(viaread_buf()).test-zlib-gzip-stored-output-boundary.jsaborts the official v19.0.0 and v20.6.0 linux-x64 binaries on every run with glibc heap-corruption errors (free(): invalid pointeron this machine; Memory corruption and crash when streaming to zlib #45268 reported an_int_mallocassertion). It passes on v20.8.0 (SIMD disabled) and on this branch.zlib_crc32_simdtarget builds and links, and all 100test-zlib*tests pass undertools/test.py.make lint-json the new test andtools/cpplint.py src/node_zlib.ccpass.Benchmark, same toolchain (AMD Ryzen 7 9800X3D, Ubuntu 24.04 on WSL2, GCC 13.3). "Before" is the parent commit (8d514c7), built from the same tree with the same configure flags, so the only difference is this PR's
zlib.gypandnode_zlib.ccchanges.node benchmark/compare.js --old <before> --new <after> --runs 30 --filter crc32 --exclude crc32c --analyze zlib:len=32is unchanged because the SIMD path only handles inputs of 64 bytes or more (Z_CRC32_SSE42_MINIMUM_LENGTHindeps/zlib/crc32_simd.h).The same comparison with
--runs 20 --filter pipe --set algorithm=gzipshows no significant change (+1.06% string, −0.62% buffer). That benchmark pipes 1 KB writes throughcreateGzip()at the default level intocreateGunzip(), so compression dominates.Whole-buffer throughput on the first 14 MB of a tar of
lib/,doc/api/,deps/zlib/anddeps/v8/src/. Each figure is the median of 3 rounds that alternate the two binaries, and each round takes the best of 5–30 calls:zlib.crc32zlib.crc32, before any zlib stream existsgzipSync(level 0)gunzipSync(level-0 input)gunzipSync(level-6 input)gzipSync(level 6)A build with only the
zlib.gypchange (and the parent'snode_zlib.cc) runszlib.crc32()at 8.5 GB/s until the process creates its first zlib stream, then at 20.4 GB/s. With this PR it runs at 20.4 GB/s from the start (median of 3 runs on the same input).Benchmark (Xeon, 14 MB tar, best-of-N):
Only
CRC32_SIMD_SSE42_PCLMULis enabled;CRC32_SIMD_AVX512_PCLMULremains off. Note that "before" uses the official v22.22.0 binary, so the baseline also differs in bundled zlib version (1.3.1 vs. 1.3.2.1-motley onmain) and toolchain:zlib.crc32gzipSync(level 0)gunzipSyncgzipSync(level 6)AI assistance: I used Claude Code for parts of this PR. I personally reviewed the diff, traced the root cause in the zlib sources, reproduced the crash on v20.6.0, and ran the ASan/Valgrind harness and zlib test suite.
Refs: #45268
Refs: #49511
Refs: #52123