feat(networking): ipv6-based routing

.github/scripts/provision-kvm-host.sh

@@ -18,9 +18,6 @@
 #     the node)
 #   - DEBIAN_FRONTEND=noninteractive (install.md is written for an interactive
 #     operator session; CI has no tty to prompt on)
-#   - [network] is always written here (install.md presents it as an optional
-#     section) since the runner's default route interface is detected fresh
-#     each run and integration coverage wants egress networking exercised
 set -euo pipefail
 
 export DEBIAN_FRONTEND=noninteractive
@@ -34,20 +31,12 @@ sudo udevadm trigger --name-match=kvm
 
 sudo apt-get update
 sudo apt-get install -y \
-  coreutils e2fsprogs iproute2 libc-bin lvm2 nftables skopeo thin-provisioning-tools util-linux \
+  coreutils e2fsprogs libc-bin lvm2 skopeo thin-provisioning-tools util-linux \
   "linux-modules-extra-$(uname -r)"
 
 # -a is load-bearing: without it modprobe reads the 2nd+ names as module
 # PARAMETERS of the first, and the load fails.
-#
-# nf_tables + nf_nat back host-init's `hyper` nftables table and its
-# masquerade/SNAT/DNAT rules. A fresh GitHub-hosted runner does not have them
-# loaded, and the netfilter family does not always autoload on first `nft` use,
-# so `nft add table ip hyper` fails (empty stderr, non-zero exit) and the node
-# refuses to start. Load the base explicitly, as we do for the dm modules; the
-# nat expression sub-modules (nft_chain_nat, nft_masq) autoload once the base is
-# present and a rule is issued.
-sudo modprobe -av dm_snapshot dm_thin_pool loop nf_tables nf_nat
+sudo modprobe -av dm_snapshot dm_thin_pool loop
 targets="$(sudo dmsetup targets)"
 echo "dmsetup targets: ${targets}"
 grep -q thin-pool <<<"${targets}" || { echo "ERROR: thin-pool dm target missing" >&2; exit 1; }
@@ -59,33 +48,6 @@ command -v thin_dump >/dev/null || { echo "ERROR: thin_dump missing (thin-provis
 sudo install -o root -g root -m 0755 "$(readlink -f "$(command -v thin_dump)")" /usr/local/sbin/thin_dump
 [ ! -L /usr/local/sbin/thin_dump ] || { echo "ERROR: /usr/local/sbin/thin_dump is still a symlink" >&2; exit 1; }
 
-# host-init (run by the node at start) asserts ip_forward=1 rather than
-# setting it, so provisioning must turn it on and persist it across reboots.
-sudo sysctl -w net.ipv4.ip_forward=1
-echo 'net.ipv4.ip_forward=1' \
-  | sudo tee /etc/sysctl.d/99-hyper-ip-forward.conf >/dev/null
-
-# GitHub-hosted runners ship Docker, which sets the iptables `filter` FORWARD
-# policy to DROP and only admits its own bridge traffic. Hyper's guest egress is
-# *forwarded* (per-VM netns veth -> uplink), so Docker's drop silently eats
-# every packet the guest sends — the guest configures its NIC fine but nothing
-# ever returns. Admit the clone pool (172.31.0.0/16, the helper's default) both
-# directions via DOCKER-USER, which Docker evaluates before its own drops; the
-# `hyper` nftables forward chain still enforces guest isolation. Fall back to an
-# accepting FORWARD policy on a host without Docker's chain.
-clone_pool="172.31.0.0/16"
-if sudo iptables -L DOCKER-USER >/dev/null 2>&1; then
-  sudo iptables -I DOCKER-USER -s "${clone_pool}" -j ACCEPT
-  sudo iptables -I DOCKER-USER -d "${clone_pool}" -j ACCEPT
-else
-  sudo iptables -P FORWARD ACCEPT
-fi
-
-# The default-route interface is the uplink guests NAT egress through.
-uplink="$(ip route show default | awk '{print $5; exit}')"
-[ -n "${uplink}" ] || { echo "ERROR: could not detect default-route uplink interface" >&2; exit 1; }
-echo "detected uplink: ${uplink}"
-
 sudo mkdir -p /etc/hyper
 sudo tee /etc/hyper/config.toml >/dev/null <<EOF
 work_dir = "/srv/hyper"
@@ -104,11 +66,6 @@ cgroup = "hyper"
 # load is still decaying, and nothing else runs here worth protecting.
 [budget]
 cpu_max_load = 4.0
-
-# Enables per-VM egress NAT. The nft table itself is owned by host-init
-# (created idempotently at node start), not by this script.
-[network]
-uplink = "${uplink}"
 EOF
 sudo chown root:root /etc/hyper/config.toml
 sudo chmod 0644 /etc/hyper/config.toml

CHANGELOG.md

@@ -14,6 +14,15 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   value is now rejected with `INVALID_ARGUMENT`; `ListVms` is now paginated
   (`page_size` / `page_token` / `next_page_token`).
 
+### Removed
+- **Breaking:** VM networking (per-VM netns + veth + TAP + nftables NAT egress,
+  and the guest `/etc/resolv.conf` from `hyper.resolver=`) has been removed
+  pending a ground-up rewrite. VMs boot with no NIC, a node no longer requires
+  or reads a `[network]` config table (an existing one is ignored), and the
+  `Hyper.SuidHelper.Network`, `Hyper.Node.FireVMM.Net` and `Hyper.Cfg.Network`
+  modules and the setuid helper's `network` subcommands remain as interfaces
+  but return/raise unimplemented.
+
 ## [0.1.0]
 
 First public release: a distributed orchestrator for Firecracker microVMs on the

README.md

@@ -43,7 +43,7 @@ Then load an OCI image and boot it:
 {:ok, %{stdout: "hello\n"}} = Hyper.exec(vm, ["/bin/echo", "hello"])
 ```
 
-That's a real Firecracker microVM with a copy-on-write rootfs -- the
+That's a real Firecracker microVM with a copy-on-write rootfs – the
 [quickstart guide](https://hexdocs.pm/hypervm/quickstart.html) has the
 details. To embed Hyper in your own application instead, add it to your Mix
 project:
@@ -63,29 +63,32 @@ integrating Hyper.
 
 ## Features
 
-- **Fully distributed** -- nodes that are added to the cluster automatically
+- **Fully distributed** – nodes that are added to the cluster automatically
   become VM runners.
-- **Affinity-based scheduling** -- Hyper automatically schedules new VMs on
+- **Affinity-based scheduling** – Hyper automatically schedules new VMs on
   nodes with the most shared resources. Forked VMs prefer being scheduled on
   nodes where the original VM ran, with fallbacks across the cluster.
-- **Disk layering** -- Forking virtual machines creates thin COW layers rather
+- **Disk layering** – Forking virtual machines creates thin COW layers rather
   than full disk snapshots. This gives Hyper a significant performance edge
   over sandbox providers which do not implement this.
-- **Telemetry** -- Hyper is mostly fully instrumented with
+- **Telemetry** – Hyper is mostly fully instrumented with
   [Otel](https://opentelemetry.io/) so you get full traces on if/why things are
   not performing as expected.
-- **Minimal stack** -- Hyper makes very few assumptions on your cloud, and only
+- **Minimal stack** – Hyper makes very few assumptions on your cloud, and only
   requires a Postgres database as a minimal external dependency.
-- **🔮 BEAM-native** -- Hyper is written on the
+- **🔮 BEAM-native** – Hyper is written on the
   [BEAM](https://en.wikipedia.org/wiki/BEAM_(Erlang_virtual_machine)). This
   means that fault-tolerance is built into the virtual machine, and allows you
   to interactively debug any issues you run into.
-- **gRPC Interface** -- although the author exclusively uses Hyper through the
+- **gRPC Interface** – although the author exclusively uses Hyper through the
   BEAM-native interfaces, we recognize this may not be ideal for all languages
   and existing stacks. For that reason, Hyper has a GRPC interface, so you can
   call it from any language you already use.
   ⚠️ The gRPC API is unauthenticated and off by default — bind it to a trusted network or front it with an authenticating proxy. See [the gRPC guide](docs/grpc.md).
 
+> **VM networking** is being rewritten from the ground up and is currently
+> unavailable: VMs boot with no NIC and have no network egress.
+
 ## Docs
 
 Full docs on getting started, as well as useful diagrams are available on
@@ -104,7 +107,7 @@ not enjoy existing products:
 
 - [Daytona](http://daytona.io/) (the best overall, in my experience) was a
   container-first SAAS. Their recent additions of VMs have been unreliable and
-  buggy for me -- between snapshots not getting committed and their VM feature
+  buggy for me – between snapshots not getting committed and their VM feature
   not being available open-source, I was ultimately dissatisfied and decided to
   write my own. **Daytona has much better support than Hyper ever will. If you
   do not want to maintain your code and want a Slack channel with really awesome
@@ -125,7 +128,7 @@ The reasons to use Hyper are:
 - You need good distributed performance. Hyper is designed to scale to
   extremely high numbers of host nodes.
 - You do not want limits. All other providers have limits, and although those
-  are in place to avoid abuse, I believe that limits should not exist -- just
+  are in place to avoid abuse, I believe that limits should not exist – just
   charge per compute hour and abuse should be curbed financially. Why can you
   not get a 128-core VM? With Hyper, you can.
 - You need good disk forking. Hyper has great support for forking block storage

docs/cookbook/architecture.md

@@ -209,7 +209,7 @@ $$
 
 The $\beta_m^{\text{max}}(N)$ value should never be exceeded, just as the
 $\alpha_m(N)$ is never to be exceeded. On the other hand, the $\beta_m^i(N)$
-value, acts as a cap -- if a machine is above that threshold, then no further
+value, acts as a cap – if a machine is above that threshold, then no further
 VMs can be scheduled. This acts as an instantaneous measurement.
 
 ## Scheduling
@@ -260,19 +260,19 @@ NVMe drives on each node, there is a desirable property in colocating VMs on
 machines that already have the pre-requisite layers loaded, as that enables
 avoiding the relatively long and painful download of each layer.
 
-> A further optimization, as it turns out, exists -- colocation can be improved
+> A further optimization, as it turns out, exists – colocation can be improved
 > not only by spawning VMs on nodes that have layers mounted, but by also
 > colocating VMs on nodes which have _recently_ had the required layers
 > mounted.
 >
 > At current, a layer is considered mounted if the layers are all mounted as
 > block devices. The net result is that even though a block device is umounted
-> (and by proxy -- `Hyper` believes the layer is not locally available), the
+> (and by proxy – `Hyper` believes the layer is not locally available), the
 > shared layer medium may cache the data itself, which could be useful to
 > speed up bootup.
 >
 > It is worth noting that in this document, the author hand-waves what
-> "caching a layer" means -- many network-attached media cache at the block
+> "caching a layer" means – many network-attached media cache at the block
 > layer, so in certain cases it is more valuable to ask "how much of an image
 > is cached".
 >

docs/cookbook/config.md

@@ -110,29 +110,10 @@ uid_gid_range = [900000, 999999]
 
 ## VM Networking Configuration
 
-**Required.** Every VM gets NAT'd egress out through a physical host interface,
-and a node refuses to start without a `[network]` table (see the
-[install guide](install.md#vm-networking-required) for the host prerequisites:
-`iproute2`/`nftables` and `net.ipv4.ip_forward=1`). `config.toml`-only — the
-setuid helper reads the same `uplink` and `clone_pool` to build each VM's netns,
-veth pair, and NAT rules, so it cannot live in `config.exs`.
-
-### `config.toml`
-
-```toml
-[network]
-# **required**. The physical uplink interface guest egress is NAT'd out through.
-# On a single-uplink host, the default-route NIC is the usual choice:
-#   ip route show default | awk '{print $5; exit}'
-uplink = "eth0"
-
-# optional -- defaults shown. The IPv4 CIDR the per-VM /30s are carved from.
-clone_pool = "172.31.0.0/16"
-
-# optional -- defaults shown. DNS resolver handed to guests via the kernel
-# cmdline (the guest agent writes it to /etc/resolv.conf).
-resolver = "1.1.1.1"
-```
+VM networking is being rewritten from the ground up and is currently
+unavailable: VMs boot with no NIC, and a node no longer requires or reads a
+`[network]` table. An existing `[network]` table in `config.toml` is accepted
+but ignored (see the [install guide](install.md#vm-networking)).
 
 ## gRPC Configuration