Skip to content

keccak_gkr: AVX-512 Skip kernel with each chi row's products paired by a shared factor - #742

Open
nconsigny wants to merge 2 commits into
leanEthereum:mainfrom
nconsigny:pr/keccak-skip-paired
Open

nconsigny wants to merge 2 commits into
leanEthereum:mainfrom
nconsigny:pr/keccak-skip-paired

Conversation

@nconsigny

Copy link
Copy Markdown
Contributor

What and why

Skip, the first rounds of each Keccak GKR layer, sums per chi row sum_x w_x e_{x+1} e_{x+2} at the 65 mixed points of a group, which costs the AVX-512 kernel 40 VPCLMULQDQ per eight positions. The sum factors as e_2 (w_0 e_1 + w_1 e_3) + e_4 (w_2 e_3 + w_3 e_0) + w_4 e_0 e_1, and the bracketed sums are linear in the packs, so SkipBlock forms them once at the group's 16 binary points and extends them by XOR like the embeddings. A mixed point then takes 24 VPCLMULQDQ: 1,944 instead of 2,600 per row chunk and group (-25%). Only the AVX-512 mod x86 kernel changes; the portable, AVX2 and NEON paths are untouched. The PR applies on main alone; it does not need the input-opening BitFold PR.

Measured

Ryzen 9 9955HX3D (Zen 5, 16C/32T), 16 threads (taskset -c 0-15), --repeat 3 --cooldown 1, main (e5d75f8) vs branch in turns (ABBA). Δ is the median of the paired differences (branch - main, ms), [min, max]. S500/S480 = aggregate --sphincs 500/480 (leanSPHINCS), X1400 = aggregate --xmss 1400, H18 = hash --log-n 18, rate 1/4 unless noted.

workload pairs Skip Δ Prove Δ
S500 6 -12.5 [-29.3, +7.7] -18.8 [-69.0, +0.0] (-1.5%), 5/6 faster
S480 6 -18.2 [-34.3, -8.1] -37.7 [-67.3, -20.0] (-3.2%), 6/6 faster
S500, rate 1/2 3 -27.8 -32.3
S480, rate 1/2 3 -23.0 -8.7
H18 6 -23.3 [-32.7, -13.8] -27.7 [-55.7, -15.0] (-7.1%)
X1400 3 -11.1 +7.0 [-16.3, +14.7] as run, -4.0 by each run's median pass (within noise)

Caveats:

  • The measured branch also carried the input-opening BitFold change (separate PR), so the Prove column includes it; alone it measured Openings -4 to -8 ms and Prove -4.5 (S500), -2.3 (S480), -1.8 (H18), within noise. It does not touch Skip.
  • Our kill line was Skip at least 12 ms faster at S500 and H18. S500 clears it by 0.5 ms, so that margin is thin; Skip spans come from one traced pass and vary by about ±10 ms with no code change. More S500 pairs are pending.
  • One machine, one target-cpu=native (AVX-512 + VPCLMULQDQ + GFNI) build; the box carried about one core of foreign niced load. One thread, indicative: SkipBlock::run 95-100 to 73-74 us per block.

Proof bytes and soundness

Prover-only. Every slot sums the same polynomial, so the Skip sums, the transcript and the proof are unchanged; get returns the sum as three parts that carry to the same words. A recursion --n 1 --sphincs-per-leaf 16 --leaf-log-inv-rate 2 leaf proof (199,521 B) has main's sha256, and every benchmark proof verified. The verifier is untouched. The new code is in mod x86 (compiled only with AVX-512F/BW, VPCLMULQDQ and GFNI); its register helpers are safe fns over unsafe blocks with // SAFETY: comments, and every index is bounds-checked.

Tests

  • paired_skip_equals_skip (replaces the single-block check): SkipBlock against the portable block kernel, slot by slot, on random, sparse and all-ones blocks with random, sparse, zero and all-ones weights.
  • paired_skip_sums_equal_the_original: a layer's Skip sums against those summed block by block from the portable kernel, padded and full batches (1,100 and 4,096 instances), three steps.
  • Upstream CI builds for haswell and arm64, where this kernel is not compiled, so it is exercised only by AVX-512 runs (checked locally on Zen 5; the haswell build passes clippy and tests). The aarch64 build was not checked locally.

Memory

Per-worker SkipBlock scratch shrinks by about 10 KB (952 registers instead of 1,105). Peak RSS is unchanged in every paired run.

…s at 30 points

The Skip pass's hot loop is carry-less products: on the AVX-512 arm, 208,000
VPCLMULQDQ a block, 40 a slot. On Zen 5 one takes two cycles at any width,
where a VGF2P8MULB or an XOR of a register takes half a cycle or less, so the
pass runs at the multiplier's rate: 476K cycles a block counted, where
208,000 multiplies at two cycles are 416K.

A block's sums are binary polynomials of at most 238 coefficients. Such a
polynomial is its residues modulo the 30 irreducible polynomials of degree 8,
whose product has degree 240, and a residue is the polynomial's value at a
root in the field of the GFNI byte product: one byte, nothing redundant, and
the values of a product are the byte products. The new kernel (skip_gfni)
holds every polynomial as those 30 bytes per slice, four registers of eight
points by eight slices, and has no carry-less product:

- affine byte maps read the residues off the witness's bytes, and once a
  layer off the weights';
- a row is e[2] (w[0] e[1] + w[1] e[3]) + e[4] (w[2] e[3] + w[3] e[0])
  + e[0] (w[4] e[1]), three stars built on the group's packs and extended
  like the values, so a slot is three byte products;
- one interpolation a block returns each slot's polynomial: over the
  conjugates of a point it is a trace, so affine byte maps again.

A group is summed over the whole block, then the other, which keeps a
group's sums in the first-level cache.

SkipBlock::new takes the layer's weights, whose residues the kernel holds,
and run the block; get returns what it did, the slot's polynomial as double
words from words 0, 1 and 2. The kernel is compiled with AVX-512F and GFNI
and replaces the AVX-512 five-term one; the portable, AVX2 and NEON arms
compute what they did. The sums are the same polynomials on every input, so
the proof is unchanged: the kernel test against the portable row over random
blocks and over a witness's first and last layers, with random and all-ones
weights; a test of the residues and of the interpolation; 15 proof and claim
digests equal to main's on native, znver4, haswell, x86-64 and aarch64
builds, and 23 whole batch proofs on the first three.

One thread, Ryzen 9 9955HX3D (Zen 5), six alternated rounds. SkipBlock::run
on one block: 95.1 -> 28.9 us (medians), 476K -> 127K cycles (best runs),
1.30M -> 0.74M uops. hash --log-n 16, medians: the Skip spans summed over
the layers 323.8 -> 109.1 ms, the layers and openings 647.2 -> 420.4 ms, the
proof 775.5 -> 550.5 ms.
The kernel's operations are quadword by quadword or byte by byte, so only
its six register operations depend on the width. With AVX2 and GFNI but no
AVX-512F (Alder Lake and the Intel clients after it) they are written on
256-bit registers: four points by eight slices a register, eight registers
a value, the same kernel otherwise.

Such a build ran the AVX2 row, 422,400 VPCLMULQDQ a block at 256 bits. The
proof is unchanged: the kernel test and the residue test on an alderlake
build, 15 proof and claim digests and 23 whole batch proofs equal to main's.

One thread, an alderlake build run on a Ryzen 9 9955HX3D (Zen 5; no Intel
machine was measured), six alternated rounds, medians. SkipBlock::run on one
block: 185.0 -> 75.3 us, 939K -> 334K cycles, 2.89M -> 1.63M uops.
hash --log-n 16: the Skip spans summed over the layers 636.7 -> 264.9 ms,
the layers and openings 1239.5 -> 873.2 ms, the proof 1470.7 -> 1098.0 ms.
@nconsigny
nconsigny force-pushed the pr/keccak-skip-paired branch from a159501 to 569cd18 Compare November 11, 2026 20:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant