I kept trying to make the network story in MVM simpler.
Every design that started with a normal guest NIC became complicated in the wrong place.
A virtual NIC is familiar. Linux knows what to do with it. Existing software gets sockets, DNS, routes, and all the usual tools. The host can put a firewall around it and call the result isolated.
That is a perfectly reasonable design for a lot of virtual machines.
It was not the default I wanted for untrusted and AI-generated workloads.
Once a guest has a routable network device, the runtime is no longer deciding whether the workload has network authority. It is deciding how much of an authority already granted should be blocked.
That distinction bothered me enough to remove the NIC.
The ordinary path
A conventional microVM network looks roughly like this:
flowchart LR
W[Workload] --> S[Guest socket]
S --> K[Guest TCP/IP stack]
K --> N[Virtual NIC]
N --> T[Host TAP or bridge]
T --> F[Firewall and NAT]
F --> I[External network]
This shape is well understood. It is also a lot of machinery:
- virtual device configuration
- address assignment
- TAP or bridge lifecycle
- routes and NAT
- DNS
- firewall rules
- backend-specific cleanup
- state that must disappear when the VM does
None of that is automatically unsafe. The issue is what becomes ambient.
The workload has a network stack attached to a network device. Policy comes after that fact.
MVM takes the opposite starting point:
flowchart LR
W[Workload] --> A[Narrow guest adapter]
A --> V[Authenticated vsock channel]
V --> G[Host-owned gateway]
G --> P[Policy, limits, secrets, audit]
P --> I[External network]
The guest can ask for a connection. The host decides whether a connection exists.
That is the line I wanted.
The host owns the last mile
This is the load-bearing part of the design: the host originates the outbound connection.
The guest does not receive a reusable API token and then make an arbitrary request. It sends a request across a machine-scoped channel. The host can bind that request to the VM’s identity, the signed execution plan, the destination, the protocol, and the current policy epoch.
Only then does the host open the socket.
That gives the runtime a place to do things that are awkward or impossible once authority has already crossed into the guest:
- deny egress by default
- bind a secret to one destination
- add credentials at the final hop
- cap bytes and active flows
- reject known-bad ranges
- record a decision without logging payloads
- stop a stale session from reconnecting after restore
The workload can still be useful. It just does not get a general-purpose network identity for free.
No NIC does not mean no sockets
This is where the design gets less obvious.
MVM has two network shapes.
The narrow path exposes specific host services over vsock. That is the strongest fit for workloads that only need a few known operations.
Some software expects a normal socket API. Rewriting every library is not realistic, so MVM also has an opt-in L3-over-vsock path.
In that mode the guest gets a point-to-point TUN interface named mvm0. It still does not get a virtio NIC, a TAP-backed Ethernet device, a MAC address, a bridge, or an L2 broadcast domain.
The guest sees IP. The host still owns transport.
flowchart TB
APP[Application] --> SOCK[Normal guest socket]
SOCK --> TCP[Guest TCP/IP stack]
TCP --> TUN[mvm0 point-to-point TUN]
TUN --> FRAME[Bounded IP frame]
FRAME --> VSOCK[vsock control and data channels]
VSOCK --> NETD[Machine-scoped host gateway]
NETD --> POLICY[Admission and flow policy]
POLICY --> OUT[Host networking]
That compromise matters.
Applications keep the interface they already understand. The runtime keeps the boundary it needs to enforce.
The complexity did not disappear
Removing the NIC made one thing simpler: there is no second path around the host gateway.
It made several other things my problem.
The first length field is hostile
A guest can send any bytes it wants. The first parser on the host cannot trust a frame length and allocate accordingly.
The L3 protocol uses a fixed header, explicit endianness, hard size limits, and rejection before allocation. IPv6 extension headers have a bounded walk. Fragment handling is explicit. Unknown message types do not fall through to a permissive default.
This is not protocol polish. The parser sits on the security boundary.
Backpressure is a security property
The happy path is easy: one sender, one receiver, both moving at the same speed.
Real systems do not behave that way.
A guest can write faster than the host can connect. An upstream server can return data faster than the guest can read it. DNS requests can pile up. UDP peers can multiply. A workload can be buggy without being malicious and still consume the host.
Every queue needs a ceiling. Every flow table needs a ceiling. Credit needs to be tracked in both directions.
A useful permission is never “may send.” It is closer to:
Leave any dimension unbounded and the permission is wider than it looks.
A socket path is not an identity
One of the easiest mistakes in local systems software is to authorize the transport coordinate.
A CID is not a VM identity. Neither is an IP address, a port, or a Unix socket path. All of them can be reused.
MVM binds the session to host-owned boot state:
VmInstanceIdentity {
node_id,
vm_id,
boot_id,
plan_digest,
}The important field is not any one value. It is the combination.
A network lease belongs to this boot, under this plan, on this node. A restored VM gets a new session. A socket left on disk does not resurrect yesterday’s authority.
DNS has to live under the same policy
A domain allow-list is mostly theater if the guest can:
- use another resolver
- connect directly to an IP
- follow a CNAME outside policy
- accept a rebinding answer into a private range
- keep using an address after the policy binding expired
The controlled DNS path normalizes names, checks the same policy as egress, pins approved answers for a bounded lifetime, and blocks private or reserved destinations unless the plan says otherwise.
The audit log records the decision. It does not need the full DNS payload.
Teardown is not cleanup trivia
Creating a TUN interface is easy.
Removing every route, namespace, nftables chain, listener, lease, and process after every possible failure is the real work.
A machine may stop cleanly. It may crash. Startup may fail halfway through. The operator may cancel it while the gateway is still coming up.
If a forwarding rule survives the VM, authority survives without an owner.
That is a security bug, not untidy housekeeping.
Backends must be allowed to say no
Cross-platform abstractions are good at making things look equivalent.
Hypervisors are not equivalent.
If a signed plan requires L3-over-vsock plus the guarantee that no routable guest NIC exists, the selected backend has to provide both. A backend that cannot do that should fail admission with a named shortfall.
It should not quietly substitute a nearby feature.
This is a pattern I keep coming back to in MVM: refusal preserves meaning.
A degraded success is often worse than a clear failure because the interface stays the same while the guarantee changes underneath it.
TLS is an actual boundary
There is one place where the network design has to be blunt.
The host can authorize an encrypted connection by destination, port, workload identity, and policy. It cannot inspect arbitrary end-to-end TLS payloads unless it deliberately terminates TLS.
That limits what the raw-IP path can promise.
Secret substitution and content redaction work when MVM owns a cleartext boundary. They do not magically work inside ciphertext. A plan that requires those features cannot select a transport that makes them impossible.
The right behavior is to refuse the combination before boot.
I would rather have a smaller feature with a real guarantee than a large feature whose security story stops at the first encrypted byte.
What no NIC buys
“No NIC” is not a performance trick or a claim that virtual networking is broken.
It is a choice about where authority lives.
The guest does not get a route to the world merely because it booted. Network access becomes a request against a signed contract. The host can answer:
- who is asking
- which boot is asking
- which plan admitted the workload
- where it wants to connect
- how much it may send
- whether the backend can enforce the request
- what evidence should remain afterward
The price is real. MVM has to own framing, flow control, DNS, identity, policy, and teardown.
I am comfortable paying that price because the complexity is now on the trusted side of a narrow boundary.
The safest NIC, for this system, is no NIC.