Repository navigation
jitv2: inline the FPU integer conversions - #332
Merged
Merged
Conversation
CVT.W/L.S/D, ROUND/TRUNC/CEIL/FLOOR.W/L.S/D and the reverse CVT.S/D.W/L each emitted a call_indirect into the interpreter's own conversion helper (core.fpu_cvt_to_int_fn / fpu_cvt_int_to_float_fn), which cost 17-19 ns per converted value against 3.1 ns for the already-inline cvt.d.s. The common case is now compiled: read the source, round it to an integral value, saturate-cast, clear Cause if it was set, and write the destination. Rounding for the two dynamic instructions is an add/subtract of a format-sized power of two in the source's sign direction under FCSR.RM == 0 -- the only case in which host MXCSR is known to hold nearest-even, since the JIT syncs MXCSR from FCSR on every FCSR write. ROUND/TRUNC/CEIL/FLOOR keep the existing mode-independent bit manipulation. Everything not proven inline branches to the unchanged callout before any guest state is written: NaN, infinities, out-of-range values, non-RN-even rounding modes and every Invalid/Inexact trap, so the exception model remains the interpreter's own function rather than a reimplementation. Measured on an R4400 with --cache shadow, per compiled instruction (a region of 128 conversions called through the JitFn, best of 3, release build): cvt.w.s 17.5 -> 6.45 ns, cvt.w.d 18.6 -> 6.97, cvt.l.d 18.5 -> 6.73, cvt.l.s 18.6 -> 6.06, trunc.w.s 17.1 -> 9.0, cvt.s.w 8.6 -> 5.2, cvt.d.w 7.2 -> 4.7, cvt.s.l 8.2 -> 5.1, cvt.d.l 7.7 -> 5.0. The untouched cvt.s.d control is unchanged (8.9 -> 8.7). Guest benchmark kernels built for this measurement show 1.76x on cvt.w.s and cvt.l.d at the suite's per-op granularity. The equivalence tests pin the whole core snapshot including fpu_fcsr, with no flag mask, across the range edges (+2^31, the F32 slack the interpreter admits), all four rounding modes, both FR modes and the enable-bit/trap paths; they caught a real bug in the first version of the rounding (2^23+1 came back as 2^23).
belikh
force-pushed
the
perf/jitv2-cvt
branch
from
October 25, 2026 18:40
c224834 to
db62bb1
Compare
belikh
added a commit
to irix7/iris
that referenced
this pull request
Oct 25, 2026
Adopt the upstream work that landed since the fork's last sync: - jitv2: skip the FP exception model for ordinary operands (techomancer#328) - jitv2: inline the FPU integer conversions (techomancer#332) - serial: wall-clock timestamps for --serial-log (techomancer#325) - physical: a sparse, host-mapped decode table over 40 bits - pit: implement the 8254 read-back command - O2: the AIC-7880 specification, shared [scsi.N] disks, fresh serial log - docs: Odyssey references and the AIC-7880 emulation notes Conflict resolutions: - serial log: take upstream's one shared sink and the off|wall timestamp modes; delete the fork's guest/both synthetic-clock sink and its partial-line flusher. The CI and TCP paths still share the sink, and the fork's per-consumer console cursors are unchanged. - config: use dev::serial_log::SerialLogTimestamps; remove the local enum, the microsecond formatter and the guest/both parsing tests. - CHANGELOG/FEATURES/HELP: keep the fork's October sections and the instance/fleet entries, adopt upstream's JIT v2 and O2 text, and drop the duplicated O2 sync and machine-start entries. Tests: cargo test --workspace --features lightning,jitv2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CVT.W/L.S/D,ROUND/TRUNC/CEIL/FLOOR.W/L.S/Dand the reverseCVT.S/D.W/Lwere the only FPU instructions still compiled to acall_indirectper operation (core.fpu_cvt_to_int_fn/fpu_cvt_int_to_float_fn, added to get the exact MIPS Inexact/roundingflags by value instead of from host MXCSR). That callout costs 17-19 ns per
converted value, against 3.1 ns for the already-inline
cvt.d.s.This compiles the common case inline and keeps the callout for everything
that cannot be proven:
destination's range with the interpreter's own bounds,
fcvt_to_sint_sat,clear Cause if set, write the destination.
CVT.W/L.S/D) underFCSR.RM == 0isx ± 2^23|2^52 ∓ ...(add/subtract a format-sizedpower of two in the source's sign direction, selected only while
|x| < 2^mantissa_bits, above which every value is already integral).This is RN-even for every finite magnitude, ties included, and relies on
host add/sub rounding to nearest-even — which is exactly what MXCSR holds
whenever
FCSR.RMis 0, sinceemit_fpu_set_modesyncs MXCSR from FCSRon every FCSR write.
ROUND/TRUNC/CEIL/FLOORkeep the existing mode-independentemit_round_to_int_modebit manipulation; their mode is a constant, soit is cheaper than a gate plus the magic pair.
so
(result as f32/f64) != xcan never fire. That includes the +2^31edge an F32 source is admitted at (
i32::MAX as f32rounds up), whereboth sides answer
i32::MAX.does, so a trapping conversion still leaves the converted value in the
FPR), then
(result as i64) != srcfor Inexact, then Cause/Flag/trap. Onlyunder
FCSR.RM == 0, becausefcvt_from_sintiscvtsi2ss/cvtsi2sdandrounds per MXCSR while the interpreter's
ascast is always RN-even.±infinity, out-of-range values, every non-RN-even rounding mode for the
dynamic instructions, and every Invalid/Inexact trap. Those run the very
cvt_to_int_and_commit/cvt_int_to_float_and_committhe interpretercalls, so the exception model is the interpreter's own function rather
than a reimplementation.
No change to FP arithmetic, the memory guard, the interpreter or the cache
model.
Before / after
Host: Intel Core i5-8500T (6 cores), Linux x86_64, load average 1.2-2.6.
Build:
cargo build --release -p iris --features lightning,jitv2, rustc1.101.0-nightly (2026-10-08), R4400,
--cache shadow. Same source tree,only
src/cpu/jitv2/codegen.rsdiffering; the probe compiles a region of128 conversions (destination registers rotated) and calls the real
JitFn20 000 times, best of 3, three runs; medians below, spread ±3%.
cvt.w.scvt.w.dcvt.l.dcvt.l.strunc.w.scvt.s.wcvt.d.wcvt.s.lcvt.d.lcvt.s.d(control, untouched)cvt.d.s(control, untouched)The two controls are untouched emitters and measure flat.
Guest-level cross-check with the
bench/suite, extended locally with sevenconversion kernels (one conversion per L1-resident element,
cvt.w.sandcvt.l.dforced with inline asm so the instruction under test is exact,cvt.s.das the control; kernels not committed because the suite's prebuiltimage would have to be regenerated with the MIPS toolchain). Three
interleaved runs each, medians:
fpu/cvt_w_sfpu/cvt_l_dfpu/cvt_s_wfpu/cvt_d_wfpu/cvt_s_lfpu/cvt_d_lfpu/cvt_s_d(control)The last three rows are the whole-page compiler's layout sensitivity, not the
instruction: in that image
cvt_d_lnever enters the JIT at all (128 ns inboth builds), and
cvt_s_l/cvt_s_dshare a code page with the kernelswhose emitted code grew. The same kernel run from a different image layout
swings 15.9 ↔ 129 ns in one binary; the per-instruction probe above, which
has no dispatch or page-layout in it, shows these instructions unchanged or
improved. Everything else in the suite is flat (
int/alu2.85→2.85,mem/copy1.46→1.42,fpu/scalar_s10.94→11.28,fpu/scalar_d9.29→9.55,sys/tlb_hit8.87→8.97) and the suite stays at 40/40 accuracy.Tests
cargo test --workspace --features lightning,jitv2: 1755 passed, 0 failed(27 ignored).
New in
src/cpu/jitv2/equiv_test.rs, all comparing the whole coresnapshot —
fpu_fcsrincluded, deliberately with no flag mask — against theinterpreter:
cvt_to_int_range_edges_match_interpreter— 38 values × W/L × S/D × bothFR modes: zero, ±0.25, ±0.5, ±0.9999999, half-integers, 2^23±, 2^24±,
±(2^31−1/2^31/2^31+1), ±2^63, subnormals, NaN, ±inf.
round_trunc_ceil_floor_edges_match_interpreter— the 16 fixed-modeconverts on half-way values, NaN and out-of-range.
cvt_to_int_all_rounding_modes_match_interpreter— FCSR.RM 0-3 throughCVT.W/L.S/D.cvt_to_int_invalid_enable_bits_match_interpreter— NaN/inf/overflow withEV clear (sticky Flag.V, destination written) and set (trap, destination
and Flag untouched), plus a dirty FCSR.
cvt_to_int_dirty_fcsr_matches_interpreter— a stale Cause and stickyFlags across the inline path.
cvt_from_int_edges_match_interpreter— 0, ±1, 2^24±, ±2^31, 2^53±,i64::MAX/MIN, both widths, both FR modes, all four rounding modes
(exercising both the inline RN-even and the callout paths).
cvt_from_int_inexact_enable_bits_match_interpreter— Inexact with EIclear (sticky Flag.I) and set (trap with the destination already
written), and dirty Flag preservation.
The edge tests caught a real bug in the first version of this patch: the
magic-number rounding is only valid below 2^mantissa_bits, and 2^23+1 came
back as 2^23 until the
|x| < 2^23select was added.cvt_perf_probe(ignored by default) reproduces the per-instruction table:cargo test --release -p iris --features lightning,jitv2 --lib cvt_perf_probe -- --ignored --nocapture.