[CMGS]

Guest side of the advanced network mode (#471): silkd owns a tun0 and carries its raw IPv4 packets to the host over one vsock stream. The proxy path (default network mode) is unchanged. The sandboxd side follows in a separate PR.

What changes

• tun0: 100.64.0.2/30 in every guest, default route via 100.64.0.1, IFF_VNET_HDR. The proxy variables stay set, so clients that honour them keep using 127.0.0.1:3128 over loopback. Everything else takes tun0.
• Switch: tun_up / tun_down verbs on the 2048 control channel (Rust and Go wire types, fixtures 61 → 63).
  • silkd also asks on vsock 2051 at boot and after every stream it sees end. The host hello answers on or off. A refused dial at boot means off.
  • A repeated tun_up ends the current stream and dials again, so the host can force a reconnect after a wake, unarchive or move.
  • Nothing is stored in the guest, so the switch is hot and goldens are unchanged.
• Wire protocol v1 (docs/advanced-network.md):
  • 12-byte hellos carrying version, offloads, MTU and flags. Flag bits 1–15 are reserved; one is meant to open several streams per guest later.
  • Frames are u32 len | virtio_net_hdr | packet, so GSO/USO packets up to 64 KiB cross vsock whole.
  • Each side enables only the offloads both kernels accept.
  • The host-side contract holds for either host design: one stream per claim, host-side enforcement, how to update checksums after a rewrite, and only sending GSO types the guest offered.
• Packet paths: io_uring is the default. tun reads stay queued, and one submission carries a batched stream write or many tun writes. epoll is the fallback when io_uring setup fails, or when SILKD_TUN_IO=epoll is set.
• CI: the root-only step also runs tun_e2e. It covers both paths in two network namespaces, behind a relay that rewrites addresses the way the host will. The throughput case is skipped in CI.

Tests

Linux container (privileged, linux/arm64; cargo test --test tun_e2e -- --ignored):

• Correctness, for io_uring and epoll each:
  • TCP 256 MiB each way.
  • UDP echo with the rewritten source seen by the server; a forged guest source leaves as the claim address.
  • USO and GSO frames in both directions, largest frame 63,774 B.
  • Partial and full checksums through the rewrite.
  • A host off answer removes tun0.
• The e2e suite passed 30 runs in a row after the fixes below.
• Container throughput (12 vCPU), with arms interleaved:

Arm	TCP up / down (Gbit/s)	UDP 64 B up (kpps / guest ns per pkt)	UDP 64 B down (kpps / guest ns per pkt)
io_uring + offloads	29.8–31.9 / 23.4–27.1	1277–1288 / 457–459	1576–1675 / 541–543
epoll + offloads	29.4–35.4 / 36.1–37.1	460–1338 / 313–1147	471–530 / 1364–1600
io_uring, no offloads	10.3–10.8 / 10.5–11.0	790–978 / 596–828	854–1092 / 664–1006
epoll, no offloads	3.6–4.1 / 3.5–4.7	150–298 / 1370–2126	227–420 / 1539–2057

Real guest (bare-metal x86_64 testbed, NUMA-fenced idle node; cocoon v0.7.8, CH dev acded4b0, guest kernel 6.18; rt:24.04 with this silkd; a test-only host stub ran in a scratch netns on the VM's vsock.uds_2051):

• tun_up brings up tun0: 100.64.0.2/30, default route, vnet_hdr on. The guest offered offloads 0x6f (USO included), and io_uring is enabled in the guest kernel.
• Arms run in the order uring, epoll, epoll, uring. Guest CPU is summed over all silkd threads, including io_uring workers.

Arm	TCP up	TCP down	UDP 64 B up	UDP 64 B down	UDP RTT p50 / p99
io_uring	7.29–8.34 Gbit/s, 0.62–0.65 s CPU/GiB	3.04–3.05 Gbit/s, 2.43–2.46 s/GiB	~570 kpps, 1082–1085 ns/pkt	~455 kpps, 1003–1404 ns/pkt	107–120 / 152–161 µs
epoll	7.05–7.47 Gbit/s, 0.64–0.65 s/GiB	2.93–3.03 Gbit/s, 1.99–2.07 s/GiB	~570 kpps, 1430–1435 ns/pkt	~430 kpps, 2299–2344 ns/pkt	87–91 / 135–137 µs
3128 proxy path, same vsock	8.10–8.15 Gbit/s, 0.64–0.67 s/GiB	3.28–3.53 Gbit/s, 1.97–2.35 s/GiB	not possible	not possible	—

• Bulk TCP on the tun path matches the 3128 relay over the same vsock in both directions. Downloads arrive as GSO frames (4.3 GB in 67k frames), so the ~3.3 Gbit/s download ceiling is the vsock host→guest direction, shared by both paths. io_uring cuts guest CPU per small packet by ~25 % up and ~40–55 % down against epoll.
• Hibernate/restore (no sandboxd; the stub keeps its UDS):

Case	New stream after restore	Result
Download in flight, no tun_up	≤200 ms	4 GiB download resumed and completed
Idle, no tun_up, IPv6 on	≤200 ms
Idle, no tun_up, IPv6 off on tun0	≤200 ms
tun_up after restore	≤100 ms	tun_up answered in 39–46 ms

• Android (android-base cocoonbox image, redroid Android 16 in a Linux guest, --nics 0, this silkd copied in):
  • With tun_up, redroid's ICMP (RTT 0.35 ms) and UDP reach the host through docker NAT and tun0.
  • The image redirects all redroid TCP into redsocks → the 3128 proxy (iptables -t nat PREROUTING -i docker0 -p tcp -j REDIRECT --to-ports 3129), so TCP takes the proxy path as designed. With that rule removed, redroid TCP runs over tun0 (~3 GB in 8 s down, 5 MB up). An image that should send TCP over tun0 in the advanced mode has to skip that redirect.
• Windows guests run no silkd and get no advanced network mode.

Review

A full-file review of the first commit found no memory-safety hole in the io_uring path. Applied fixes are in 0d7f0821:

• A repeated tun_up redials.
• tun_up replaces a boot link that has no route yet.
• A boot link stops on a failed hello.
• The offload set is trimmed to combinations TUNSETOFFLOAD accepts.
• The completion queue is sized for the most tun writes one stream read can queue.
• A write that finds the peer gone ends the stream like an EOF; this was a 4-in-25 teardown flake in the e2e.
• tun_up and tun_down run off the tokio workers.

Kept without code, both contrived: a failed re-arm after an SQ submit error, and a self-inflicted IPv6 GSO read error.

Not covered

• Live move on hardware: needs a second node, and comes with the sandboxd PR. The guest path is the same as restore: the stream ends, then silkd dials again or gets tun_up.
• The sandboxd side: the packet port, VXLAN to the exit, ARP, address allocation, EvalFlow, enforcement and audit come in the next PR, per sandboxd: advanced network mode, an L3 tun path over VXLAN for none-lane claims (design) #471.

Gates:

• cargo fmt --check, cargo clippy --all-targets -D warnings and cargo test on mac and linux/arm64.
• clippy for aarch64/x86_64 musl.
• go test for protocol/wire.