Use talc as the guest heap allocator by ludfjig · Pull Request #1928 · hyperlight-dev/hyperlight

ludfjig · 2026-10-12T20:01:13Z

Switch the guest heap allocator from buddy_system_allocator to talc. Snapshots split into fewer memory ranges, and guest init touches fewer heap pages. KVM sandbox_from_snapshot gets 20 to 26% faster for small to large heaps, restore_alternate gets 11 to 28% faster, and MSHV sandbox creation gets 27 to 36% faster.

buddy_system_allocator init writes a free-list pointer at the start of every power-of-two heap block. Each written page splits the snapshot's base layer into another live range. Each live range costs a memory slot on KVM and a map call on MSHV, and buddy needs 5 to 16 of them depending on heap size. talc writes only the start and end of the heap, and every size needs 4.

The host populates heap pages on first guest touch. MSHV populates a whole 2 MiB chunk per touch, and buddy's init touches 2 to 7 more chunks than talc.

talc runs behind a spin mutex, as talc recommends. spin is already a dependency.

Benchmarks

Compared to the PR below.

MSHV: Linux 6.6 mshv.
KVM: Linux 6.17.

Benchmark	MSHV	KVM
`sandboxes/sandbox_from_snapshot/large`	6.14 ms → 6.06 ms (~)	1.59 ms → 1.17 ms (−26%)
`snapshots/restore_alternate/medium`	129 µs → 113 µs (−12%)	462 µs → 334 µs (−28%)
`sandboxes/create_initialized/medium`	5.6 ms → 3.59 ms (−36%)	2.9 ms → 2.82 ms (~)
`snapshot_files/cold_start_via_evolve/small`	3.25 ms → 2.02 ms (−38%)	2.9 ms → 2.76 ms (−5%)

No benchmark regresses. On main, talc makes no measurable difference. Flat snapshots use one memory slot, and main populates all snapshot memory up front.

Full results

All benchmarks on both hosts (click to expand)

Before → after (change). Negative is faster. Bold is a regression. ~ is no consistent change.

snapshots

Benchmark	Size	MSHV	KVM
`call_and_create`	default	376 µs → 372 µs (~)	413 µs → 405 µs (~)
`call_and_create`	small	433 µs → 422 µs (~)	474 µs → 456 µs (~)
`call_and_create`	medium	781 µs → 760 µs (~)	808 µs → 793 µs (~)
`call_and_create`	large	1.98 ms → 1.94 ms (~)	1.93 ms → 1.92 ms (~)
`create`	default	242 µs → 244 µs (~)	282 µs → 275 µs (~)
`create`	small	304 µs → 289 µs (−5%)	325 µs → 329 µs (~)
`create`	medium	646 µs → 625 µs (~)	667 µs → 661 µs (~)
`create`	large	1.85 ms → 1.86 ms (~)	1.79 ms → 1.76 ms (~)
`restore`	default	66 µs → 66.7 µs (~)	20.7 µs → 20.6 µs (~)
`restore`	small	66.6 µs → 67 µs (~)	21.6 µs → 20.7 µs (−4%)
`restore`	medium	71.6 µs → 70 µs (~)	25.9 µs → 24.4 µs (−6%)
`restore`	large	96.1 µs → 91.7 µs (−5%)	47.7 µs → 45.9 µs (~)
`restore_alternate`	default	124 µs → 110 µs (−11%)	455 µs → 333 µs (−27%)
`restore_alternate`	small	124 µs → 109 µs (−11%)	449 µs → 336 µs (−25%)
`restore_alternate`	medium	129 µs → 113 µs (−12%)	462 µs → 334 µs (−28%)
`restore_alternate`	large	156 µs → 135 µs (−14%)	484 µs → 368 µs (−24%)

guest_calls

Benchmark	Size	MSHV	KVM
`call`	default	43.6 µs → 44.5 µs (~)	18 µs → 17.5 µs (~)
`call`	small	43.7 µs → 44.9 µs (~)	17.9 µs → 18.1 µs (~)
`call`	medium	43.8 µs → 44.7 µs (~)	17.9 µs → 18.3 µs (~)
`call`	large	43.8 µs → 44.7 µs (~)	17.8 µs → 17.9 µs (~)
`call_with_host_function`	default	73.8 µs → 73.2 µs (~)	34.4 µs → 35.2 µs (~)
`call_with_host_function`	small	73 µs → 73.7 µs (~)	34 µs → 31.3 µs (~)
`call_with_host_function`	medium	74.2 µs → 72.6 µs (~)	34.2 µs → 34.4 µs (~)
`call_with_host_function`	large	72 µs → 72.8 µs (~)	33.8 µs → 34.3 µs (~)
`call_with_restore`	default	134 µs → 138 µs (~)	62.7 µs → 64.7 µs (~)
`call_with_restore`	small	135 µs → 139 µs (~)	63.8 µs → 65.3 µs (~)
`call_with_restore`	medium	139 µs → 142 µs (~)	69.5 µs → 69.6 µs (~)
`call_with_restore`	large	161 µs → 166 µs (~)	89.5 µs → 95.9 µs (~)
`different_thread`	-	66.6 µs → 67.6 µs (~)	41.5 µs → 40.9 µs (~)
`interrupt_latency`	-	45.9 µs → 46.8 µs (~)	24.7 µs → 24.2 µs (~)

guest_functions_with_large_parameters

Benchmark	Size	MSHV	KVM
`guest_call_with_large_parameters`	-	101 ms → 102 ms (~)	867 ms → 862 ms (~)

sample_workloads

Benchmark	Size	MSHV	KVM
`24K_in_8K_out_c`	-	54.5 µs → 54.5 µs (~)	27.2 µs → 27.7 µs (~)
`24K_in_8K_out_rust`	-	54.2 µs → 54.7 µs (~)	27.3 µs → 26.7 µs (~)

sandboxes

Benchmark	Size	MSHV	KVM
`create_initialized`	default	1.47 ms → 1.51 ms (~)	2.55 ms → 2.55 ms (~)
`create_initialized`	small	3 ms → 2.12 ms (−30%)	2.68 ms → 2.64 ms (~)
`create_initialized`	medium	5.6 ms → 3.59 ms (−36%)	2.9 ms → 2.82 ms (~)
`create_initialized`	large	11.1 ms → 8.11 ms (−27%)	3.41 ms → 3.27 ms (−4%)
`create_initialized_and_drop`	default	22.4 ms → 20.7 ms (~)	14 ms → 13.2 ms (~)
`create_initialized_and_drop`	small	25.5 ms → 24.1 ms (−5%)	12.6 ms → 13.4 ms (~)
`create_initialized_and_drop`	medium	27.4 ms → 28.8 ms (~)	13.4 ms → 11.7 ms (~)
`create_initialized_and_drop`	large	31.2 ms → 28.2 ms (−9%)	13.7 ms → 12.5 ms (~)
`sandbox_from_snapshot`	default	643 µs → 637 µs (~)	945 µs → 886 µs (~)
`sandbox_from_snapshot`	small	911 µs → 848 µs (−7%)	1.18 ms → 947 µs (−20%)
`sandbox_from_snapshot`	medium	2.12 ms → 2.06 ms (~)	1.31 ms → 1.02 ms (−22%)
`sandbox_from_snapshot`	large	6.14 ms → 6.06 ms (~)	1.59 ms → 1.17 ms (−26%)

snapshot_files

Benchmark	Size	MSHV	KVM
`cold_start_via_evolve`	default	1.56 ms → 1.53 ms (~)	2.69 ms → 2.73 ms (~)
`cold_start_via_evolve`	small	3.25 ms → 2.02 ms (−38%)	2.9 ms → 2.76 ms (−5%)
`cold_start_via_evolve`	medium	6.11 ms → 3.8 ms (−38%)	3.08 ms → 2.93 ms (−5%)
`cold_start_via_evolve`	large	11.3 ms → 8.3 ms (−26%)	3.78 ms → 3.31 ms (−12%)
`cold_start_via_snapshot`	default	1.95 ms → 1.96 ms (~)	2.56 ms → 2.56 ms (~)
`cold_start_via_snapshot`	small	8.64 ms → 8.45 ms (~)	8.6 ms → 8.49 ms (~)
`cold_start_via_snapshot`	medium	53.5 ms → 53.2 ms (~)	50.3 ms → 50.1 ms (~)
`cold_start_via_snapshot`	large	204 ms → 203 ms (~)	192 ms → 193 ms (~)
`cold_start_via_snapshot_unverified`	default	1.37 ms → 1.4 ms (~)	2.13 ms → 2.07 ms (~)
`cold_start_via_snapshot_unverified`	small	1.58 ms → 1.54 ms (~)	2.38 ms → 2.11 ms (−11%)
`cold_start_via_snapshot_unverified`	medium	2.79 ms → 2.74 ms (~)	2.14 ms → 2.21 ms (~)
`cold_start_via_snapshot_unverified`	large	7.39 ms → 7.19 ms (~)	2.75 ms → 2.3 ms (−16%)
`load_snapshot`	default	707 µs → 704 µs (~)	697 µs → 698 µs (~)
`load_snapshot`	small	6.7 ms → 6.64 ms (~)	6.37 ms → 6.35 ms (~)
`load_snapshot`	medium	50.6 ms → 50.4 ms (~)	48 ms → 47.9 ms (~)
`load_snapshot`	large	197 ms → 197 ms (~)	190 ms → 190 ms (~)
`load_snapshot_unverified`	default	134 µs → 134 µs (~)	139 µs → 139 µs (~)
`load_snapshot_unverified`	small	142 µs → 134 µs (−5%)	141 µs → 141 µs (~)
`load_snapshot_unverified`	medium	144 µs → 135 µs (−6%)	143 µs → 141 µs (~)
`load_snapshot_unverified`	large	139 µs → 138 µs (~)	146 µs → 145 µs (~)
`save_snapshot`	default	933 µs → 937 µs (~)	38.6 ms → 43.1 ms (~)
`save_snapshot`	small	9.1 ms → 9.15 ms (~)	74.8 ms → 73.7 ms (~)
`save_snapshot`	medium	67.2 ms → 67.1 ms (~)	590 ms → 526 ms (~)
`save_snapshot`	large	268 ms → 265 ms (~)	2.53 s → 2.51 s (~)

buddy_system_allocator writes a free-list pointer at the start of every power-of-two heap block during guest init. Each written page leaves a hole in the snapshot's base layer and splits its live range. On KVM each live range is a memory slot, so a snapshot needs 5 to 16 slots depending on heap size. talc writes only the start and end of the heap, and every size needs 4 slots. The host populates snapshot memory on first guest touch. Fewer written pages also speed up sandbox creation. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

hyperlight-gh-bot · 2026-10-12T20:54:07Z

Benchmark Results

Measured commit: 6919f7f16312
Baseline commit: 96fab649f47c

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`778.08 ns` (➖ 1.02x faster)
`vec_bytes`	`581.93 ns` (➖ 1.02x faster)
		`373.60 µs` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`522.70 ns` (➖ 1.01x faster)
`65536`	`142.41 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.09 ms` (➖ 1.12x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`8.11 ns` (➖ 1.05x slower)	`7.74 ns` (➖ 1.03x faster)	`7.76 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`141.97 µs` (➖ 1.00x slower)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.27 µs` (➖ 1.03x slower)	`7.00 µs` (➖ 1.11x faster)
`65536`	`2.05 µs` (➖ 1.07x slower)	`1.92 µs` (➖ 1.08x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`6.27 µs` (➖ 1.03x faster)	`6.41 µs` (➖ 1.01x faster)
`8192`	`1.11 µs` (➖ 1.05x slower)	`1.04 µs` (➖ 1.00x faster)
`262144`		`28.30 µs` (➖ 1.03x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`812.54 ns` (➖ 1.05x faster)
`vec_bytes`	`614.16 ns` (➖ 1.01x faster)
		`762.67 µs` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`627.70 ns` (➖ 1.01x faster)
`65536`	`162.66 ns` (➖ 1.03x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.04 ms` (➖ 1.18x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`8.33 ns` (➖ 1.04x faster)	`8.33 ns` (➖ 1.06x faster)	`8.33 ns` (➖ 1.04x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`82.10 µs` (➖ 1.00x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.11 µs` (➖ 1.02x faster)	`9.18 µs` (➖ 1.01x slower)
`65536`	`2.56 µs` (➖ 1.01x faster)	`2.57 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.52 µs` (➖ 1.03x faster)	`8.43 µs` (➖ 1.05x faster)
`8192`	`869.55 ns` (➖ 1.09x faster)	`873.67 ns` (➖ 1.06x faster)
`262144`		`35.06 µs` (➖ 1.04x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`996.72 ns` (➖ 1.03x slower)
`vec_bytes`	`695.55 ns` (➖ 1.06x faster)
		`353.32 µs` (➖ 1.32x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`724.94 ns` (➖ 1.00x faster)
`65536`	`192.65 ns` (➖ 1.00x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`61.40 ms` (➖ 1.04x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`9.74 ns` (➖ 1.00x faster)	`9.71 ns` (➖ 1.01x faster)	`9.90 ns` (➖ 1.02x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`123.47 µs` (➖ 1.08x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.99 µs` (➖ 1.08x faster)	`9.85 µs` (➖ 1.01x faster)
`65536`	`2.42 µs` (➖ 1.02x faster)	`2.24 µs` (➖ 1.02x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.06 µs` (➖ 1.19x faster)	`8.04 µs` (➖ 1.23x faster)
`8192`	`1.24 µs` (➖ 1.02x faster)	`1.25 µs` (➖ 1.03x slower)
`262144`		`36.70 µs` (➖ 1.03x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`916.61 ns` (➖ 1.02x faster)
`vec_bytes`	`687.00 ns` (➖ 1.05x slower)
		`658.87 µs` (➖ 1.04x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`618.88 ns` (➖ 1.01x faster)
`65536`	`171.87 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`74.56 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`9.72 ns` (➖ 1.03x slower)	`8.77 ns` (➖ 1.00x faster)	`8.78 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`63.17 µs` (➖ 1.10x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`8.00 µs` (➖ 1.00x faster)	`7.98 µs` (➖ 1.01x slower)
`65536`	`2.28 µs` (➖ 1.02x faster)	`2.28 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`7.62 µs` (➖ 1.00x slower)	`7.48 µs` (➖ 1.04x faster)
`8192`	`920.19 ns` (➖ 1.02x faster)	`915.46 ns` (➖ 1.01x slower)
`262144`		`38.26 µs` (➖ 1.06x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.17 µs` (➖ 1.03x faster)
`vec_bytes`	`812.12 ns` (➖ 1.01x slower)
		`1.86 ms` (➖ 1.02x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`792.83 ns` (➖ 1.01x slower)
`65536`	`233.24 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`10.16 ms` (➖ 1.15x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.19 ns` (➖ 1.02x slower)	`10.07 ns` (➖ 1.00x faster)	`10.16 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`805.99 µs` (➖ 1.03x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.09 µs` (➖ 1.08x faster)	`9.23 µs` (➖ 1.02x slower)
`65536`	`2.43 µs` (➖ 1.04x faster)	`2.41 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`9.35 µs` (➖ 1.02x slower)	`8.90 µs` (➖ 1.02x slower)
`8192`	`1.36 µs` (➖ 1.02x faster)	`1.35 µs` (➖ 1.03x slower)
`262144`		`40.04 µs` (➖ 1.05x slower)

hyperv-ws2025 / intel (Windows) (✅ 1.78x)

Top improvements

sandboxes/create_initialized_and_drop/medium — ✅ 1.78x faster

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.23 µs` (➖ 1.03x faster)
`vec_bytes`	`784.97 ns` (➖ 1.02x faster)
		`2.98 ms` (➖ 1.04x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`746.43 ns` (➖ 1.01x faster)
`65536`	`211.80 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.76 ms` (✅ 1.78x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.58 ns` (➖ 1.03x slower)	`10.02 ns` (➖ 1.00x slower)	`10.62 ns` (➖ 1.01x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`858.45 µs` (➖ 1.19x slower)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.80 µs` (➖ 1.01x slower)	`7.81 µs` (➖ 1.01x slower)
`65536`	`2.41 µs` (➖ 1.05x slower)	`2.34 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.52 µs` (➖ 1.04x faster)	`8.49 µs` (➖ 1.01x faster)
`8192`	`1.16 µs` (➖ 1.12x faster)	`1.12 µs` (➖ 1.03x slower)
`262144`		`41.88 µs` (➖ 1.01x faster)

_{Reported by cargo ci bench-report --candidate run:38301513915 --config-file bench_report.toml.}

ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Oct 12, 2026

ludfjig requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, simongdavies, squillace and syntactically as code owners October 12, 2026 20:01

ludfjig added this pull request to stack #1929 October 12, 2026 20:01

ludfjig mentioned this pull request Oct 12, 2026

Layered/incremental snapshots #1902

Closed

Conversation

ludfjig commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Full results

snapshots

guest_calls

guest_functions_with_large_parameters

sample_workloads

sandboxes

snapshot_files

Uh oh!

hyperlight-gh-bot Bot commented Oct 12, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Top improvements

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ludfjig commented Oct 12, 2026 •

edited

Loading