Skip to content

jitv2: inline the FPU integer conversions - #332

Merged
techomancer merged 1 commit into
techomancer:mainfrom
irix7:perf/jitv2-cvt
Oct 25, 2026
Merged

techomancer merged 1 commit into
techomancer:mainfrom
irix7:perf/jitv2-cvt

Conversation

@belikh

@belikh belikh commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

CVT.W/L.S/D, ROUND/TRUNC/CEIL/FLOOR.W/L.S/D and the reverse
CVT.S/D.W/L were the only FPU instructions still compiled to a
call_indirect per operation (core.fpu_cvt_to_int_fn /
fpu_cvt_int_to_float_fn, added to get the exact MIPS Inexact/rounding
flags by value instead of from host MXCSR). That callout costs 17-19 ns per
converted value, against 3.1 ns for the already-inline cvt.d.s.

This compiles the common case inline and keeps the callout for everything
that cannot be proven:

  • float → int: read the source, round to an integral value, test the
    destination's range with the interpreter's own bounds, fcvt_to_sint_sat,
    clear Cause if set, write the destination.
    • Rounding for the two dynamic-rounding instructions (CVT.W/L.S/D) under
      FCSR.RM == 0 is x ± 2^23|2^52 ∓ ... (add/subtract a format-sized
      power of two in the source's sign direction, selected only while
      |x| < 2^mantissa_bits, above which every value is already integral).
      This is RN-even for every finite magnitude, ties included, and relies on
      host add/sub rounding to nearest-even — which is exactly what MXCSR holds
      whenever FCSR.RM is 0, since emit_fpu_set_mode syncs MXCSR from FCSR
      on every FCSR write.
    • ROUND/TRUNC/CEIL/FLOOR keep the existing mode-independent
      emit_round_to_int_mode bit manipulation; their mode is a constant, so
      it is cheaper than a gate plus the magic pair.
    • Inexact is provably unreachable in range: the rounded value is integral,
      so (result as f32/f64) != x can never fire. That includes the +2^31
      edge an F32 source is admitted at (i32::MAX as f32 rounds up), where
      both sides answer i32::MAX.
  • int → float: convert, write the destination first (the interpreter
    does, so a trapping conversion still leaves the converted value in the
    FPR), then (result as i64) != src for Inexact, then Cause/Flag/trap. Only
    under FCSR.RM == 0, because fcvt_from_sint is cvtsi2ss/cvtsi2sd and
    rounds per MXCSR while the interpreter's as cast is always RN-even.
  • Callout, unchanged, before any guest state is written: NaN,
    ±infinity, out-of-range values, every non-RN-even rounding mode for the
    dynamic instructions, and every Invalid/Inexact trap. Those run the very
    cvt_to_int_and_commit/cvt_int_to_float_and_commit the interpreter
    calls, so the exception model is the interpreter's own function rather
    than a reimplementation.

No change to FP arithmetic, the memory guard, the interpreter or the cache
model.

Before / after

Host: Intel Core i5-8500T (6 cores), Linux x86_64, load average 1.2-2.6.
Build: cargo build --release -p iris --features lightning,jitv2, rustc
1.101.0-nightly (2026-10-08), R4400, --cache shadow. Same source tree,
only src/cpu/jitv2/codegen.rs differing; the probe compiles a region of
128 conversions (destination registers rotated) and calls the real JitFn
20 000 times, best of 3, three runs; medians below, spread ±3%.

instruction upstream this PR speedup
cvt.w.s 17.50 ns 6.45 ns 2.71×
cvt.w.d 18.58 ns 6.97 ns 2.67×
cvt.l.d 18.45 ns 6.73 ns 2.74×
cvt.l.s 18.56 ns 6.06 ns 3.06×
trunc.w.s 17.11 ns 9.00 ns 1.90×
cvt.s.w 8.63 ns 5.17 ns 1.67×
cvt.d.w 7.15 ns 4.71 ns 1.52×
cvt.s.l 8.20 ns 5.08 ns 1.61×
cvt.d.l 7.69 ns 4.96 ns 1.55×
cvt.s.d (control, untouched) 8.88 ns 8.73 ns 1.02×
cvt.d.s (control, untouched) 3.11 ns 3.15 ns 0.99×

The two controls are untouched emitters and measure flat.

Guest-level cross-check with the bench/ suite, extended locally with seven
conversion kernels (one conversion per L1-resident element, cvt.w.s and
cvt.l.d forced with inline asm so the instruction under test is exact,
cvt.s.d as the control; kernels not committed because the suite's prebuilt
image would have to be regenerated with the MIPS toolchain). Three
interleaved runs each, medians:

kernel upstream this PR speedup
fpu/cvt_w_s 22.76 ns/op 12.91 ns/op 1.76×
fpu/cvt_l_d 24.54 ns/op 13.94 ns/op 1.76×
fpu/cvt_s_w 19.54 ns/op 17.63 ns/op 1.11×
fpu/cvt_d_w 17.63 ns/op 16.15 ns/op 1.09×
fpu/cvt_s_l 23.56 ns/op 25.46 ns/op 0.93×
fpu/cvt_d_l 126.0 ns/op 132.8 ns/op 0.95×
fpu/cvt_s_d (control) 15.37 ns/op 16.35 ns/op 0.94×

The last three rows are the whole-page compiler's layout sensitivity, not the
instruction: in that image cvt_d_l never enters the JIT at all (128 ns in
both builds), and cvt_s_l/cvt_s_d share a code page with the kernels
whose emitted code grew. The same kernel run from a different image layout
swings 15.9 ↔ 129 ns in one binary; the per-instruction probe above, which
has no dispatch or page-layout in it, shows these instructions unchanged or
improved. Everything else in the suite is flat (int/alu 2.85→2.85,
mem/copy 1.46→1.42, fpu/scalar_s 10.94→11.28, fpu/scalar_d 9.29→9.55,
sys/tlb_hit 8.87→8.97) and the suite stays at 40/40 accuracy.

Tests

cargo test --workspace --features lightning,jitv2: 1755 passed, 0 failed
(27 ignored).

New in src/cpu/jitv2/equiv_test.rs, all comparing the whole core
snapshot — fpu_fcsr included, deliberately with no flag mask — against the
interpreter:

  • cvt_to_int_range_edges_match_interpreter — 38 values × W/L × S/D × both
    FR modes: zero, ±0.25, ±0.5, ±0.9999999, half-integers, 2^23±, 2^24±,
    ±(2^31−1/2^31/2^31+1), ±2^63, subnormals, NaN, ±inf.
  • round_trunc_ceil_floor_edges_match_interpreter — the 16 fixed-mode
    converts on half-way values, NaN and out-of-range.
  • cvt_to_int_all_rounding_modes_match_interpreter — FCSR.RM 0-3 through
    CVT.W/L.S/D.
  • cvt_to_int_invalid_enable_bits_match_interpreter — NaN/inf/overflow with
    EV clear (sticky Flag.V, destination written) and set (trap, destination
    and Flag untouched), plus a dirty FCSR.
  • cvt_to_int_dirty_fcsr_matches_interpreter — a stale Cause and sticky
    Flags across the inline path.
  • cvt_from_int_edges_match_interpreter — 0, ±1, 2^24±, ±2^31, 2^53±,
    i64::MAX/MIN, both widths, both FR modes, all four rounding modes
    (exercising both the inline RN-even and the callout paths).
  • cvt_from_int_inexact_enable_bits_match_interpreter — Inexact with EI
    clear (sticky Flag.I) and set (trap with the destination already
    written
    ), and dirty Flag preservation.

The edge tests caught a real bug in the first version of this patch: the
magic-number rounding is only valid below 2^mantissa_bits, and 2^23+1 came
back as 2^23 until the |x| < 2^23 select was added.

cvt_perf_probe (ignored by default) reproduces the per-instruction table:
cargo test --release -p iris --features lightning,jitv2 --lib cvt_perf_probe -- --ignored --nocapture.

CVT.W/L.S/D, ROUND/TRUNC/CEIL/FLOOR.W/L.S/D and the reverse CVT.S/D.W/L
each emitted a call_indirect into the interpreter's own conversion helper
(core.fpu_cvt_to_int_fn / fpu_cvt_int_to_float_fn), which cost 17-19 ns per
converted value against 3.1 ns for the already-inline cvt.d.s.

The common case is now compiled: read the source, round it to an integral
value, saturate-cast, clear Cause if it was set, and write the destination.
Rounding for the two dynamic instructions is an add/subtract of a
format-sized power of two in the source's sign direction under FCSR.RM == 0
-- the only case in which host MXCSR is known to hold nearest-even, since the
JIT syncs MXCSR from FCSR on every FCSR write. ROUND/TRUNC/CEIL/FLOOR keep
the existing mode-independent bit manipulation. Everything not proven inline
branches to the unchanged callout before any guest state is written: NaN,
infinities, out-of-range values, non-RN-even rounding modes and every
Invalid/Inexact trap, so the exception model remains the interpreter's own
function rather than a reimplementation.

Measured on an R4400 with --cache shadow, per compiled instruction (a region
of 128 conversions called through the JitFn, best of 3, release build):
cvt.w.s 17.5 -> 6.45 ns, cvt.w.d 18.6 -> 6.97, cvt.l.d 18.5 -> 6.73,
cvt.l.s 18.6 -> 6.06, trunc.w.s 17.1 -> 9.0, cvt.s.w 8.6 -> 5.2,
cvt.d.w 7.2 -> 4.7, cvt.s.l 8.2 -> 5.1, cvt.d.l 7.7 -> 5.0. The untouched
cvt.s.d control is unchanged (8.9 -> 8.7). Guest benchmark kernels built for
this measurement show 1.76x on cvt.w.s and cvt.l.d at the suite's per-op
granularity.

The equivalence tests pin the whole core snapshot including fpu_fcsr, with no
flag mask, across the range edges (+2^31, the F32 slack the interpreter
admits), all four rounding modes, both FR modes and the enable-bit/trap
paths; they caught a real bug in the first version of the rounding (2^23+1
came back as 2^23).
@techomancer
techomancer merged commit fba85e7 into techomancer:main Oct 25, 2026
13 checks passed
belikh added a commit to irix7/iris that referenced this pull request Oct 25, 2026
Adopt the upstream work that landed since the fork's last sync:

- jitv2: skip the FP exception model for ordinary operands (techomancer#328)
- jitv2: inline the FPU integer conversions (techomancer#332)
- serial: wall-clock timestamps for --serial-log (techomancer#325)
- physical: a sparse, host-mapped decode table over 40 bits
- pit: implement the 8254 read-back command
- O2: the AIC-7880 specification, shared [scsi.N] disks, fresh serial log
- docs: Odyssey references and the AIC-7880 emulation notes

Conflict resolutions:

- serial log: take upstream's one shared sink and the off|wall timestamp
  modes; delete the fork's guest/both synthetic-clock sink and its
  partial-line flusher. The CI and TCP paths still share the sink, and the
  fork's per-consumer console cursors are unchanged.
- config: use dev::serial_log::SerialLogTimestamps; remove the local enum,
  the microsecond formatter and the guest/both parsing tests.
- CHANGELOG/FEATURES/HELP: keep the fork's October sections and the
  instance/fleet entries, adopt upstream's JIT v2 and O2 text, and drop the
  duplicated O2 sync and machine-start entries.

Tests: cargo test --workspace --features lightning,jitv2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants