Networking Isolated Agent VMs at Scale

With Firecracker micro-VM isolation, each AI agent has its own kernel. Networking those isolated VMs at scale requires deliberate choices about session routing, host forwarding, egress, and tenant separation. The architectural goal is to preserve each agent’s identity and isolation across its full lifecycle without making the network a bottleneck.

Why networking becomes the hard problem once you commit to per-agent VM isolation

With one VM per agent, the host must handle a separate network attachment for each agent. Firecracker exposes only a basic network device, block device, vsock device, balloon device, serial console, and minimal keyboard controller. Its minimalism leaves network orchestration outside the VM. Operators must manage TAP interfaces, IP tables, and host routing rather than assume that the mature CNI plugin machinery familiar from container deployments is available.

Maritime follows this isolation principle by putting each agent in its own Firecracker micro-VM, behind a hardware boundary that container escape vulnerabilities cannot cross. That choice changes the networking surface its operators manage. A fleet of isolated kernels does not inherit the operational model of a Kubernetes pod fleet simply because both run on Linux hosts. The failure modes, performance characteristics, and security requirements differ. Container networking has years of tooling around its assumptions, while Firecracker leaves more of these responsibilities to the infrastructure layer.

Operating beyond a handful of VMs requires fleet-wide coordination of network attachment, routing policy, and cleanup, beyond what can be set up on each host alone.

What the network topology of a multi-tenant agent fleet actually looks like

A production multi-tenant agent fleet has at least five independently scaling layers, namely Gateway, Orchestrator, Scheduler, Memory, and Sandbox. Each has its own traffic patterns and isolation requirements. The network spans all five, even though the guest’s network device exposes only one part of that topology. Endpoint tracking and VM lifecycle management must therefore remain coordinated as the layers grow independently.

The defining boundary is one VM per user session. Reusing that VM for a different user or task reconstructs the shared-worker leak in another form. Each VM also needs an addressable endpoint. AWS Lambda MicroVMs implements this by returning a dedicated HTTPS endpoint at launch, rather than placing interchangeable sessions behind one shared endpoint.

At thousands of VMs, the fleet contains thousands of endpoints, TAP interfaces, routing entries, and egress paths. All require lifecycle management as VMs launch and suspend. This is the infrastructure burden that platforms such as Maritime are built to absorb, because separate kernels imply separate network attachments. Cognition’s engineering work on Devin illustrates the orchestration cost. Its team spent over three quarters of dedicated engineering effort building infrastructure to manage thousands of concurrent VMs, including routing, endpoint tracking, and lifecycle.

How do you route traffic to thousands of ephemeral, per-session endpoints without a shared load balancer?

Traditional load balancing assumes a homogeneous pool of interchangeable backends. A web-service request can go to any suitable backend, but a user’s agent state lives in exactly one VM.

Routing therefore requires a live mapping from user or session identifiers to VM endpoints. That mapping must support low-latency reads and consistent, atomic updates as VMs launch, suspend, and terminate. Lambda MicroVMs makes the division of responsibility explicit. It supports vertical scaling within a running session up to 4× the configured baseline, but creating more VMs, tracking their endpoints, routing users, and cleaning up remain application-layer responsibilities.

Slow lookups delay every interaction, while inconsistent mappings risk cross-session routing and contamination. Infrastructure for persistent agents, including Maritime, has this dependency because it keeps the same agent state across wake cycles and user sessions. It would disappear for stateless serverless functions, but those would not preserve the same agent model. Profound’s AI Marketer launches several hundred thousand MicroVMs per month. At that volume, the registry is a production database with strict latency and consistency requirements.

How TAP interface proliferation and host routing tables degrade at fleet scale

The logical registry is only one routing burden. Each Firecracker VM also adds a TAP interface and routing entries to the host kernel. The TAP interface is a virtual Ethernet device through which the VMM connects the guest’s network stack to the host. At low VM counts, that overhead can remain unobtrusive. At hundreds of VMs per host, interfaces, iptables rules, and routing entries accumulate together.

Rule evaluation adds a further scaling cost. Linux iptables evaluation is O(n) in the number of rules, so a growing rule set can increase packet-processing latency. A 36-core server can provision roughly 150 new microVMs per second, but even a host that provisions that fast can have routing-table problems when the load stays high.

The design direction is to move beyond per-VM rule accumulation. eBPF-based forwarding provides a constant-time evaluation approach rather than a linear rule walk, while IP-masquerading bridges or macvtap offer alternatives to per-VM NAT rules. Operators must choose and set up these options when they use Firecracker. Cognition says VM-based isolation takes a lot of work, and host networking accounts for much of that work.

Egress control for agents that can call anything on the public internet

A separate kernel does not authorize outbound traffic. An agent can remain isolated from other guests while sending credentials to an external endpoint or calling an unauthorized API. Arbitrary or AI-generated code makes that distinction consequential. Without an explicit egress policy, access to the public internet leaves an attack surface outside the compute boundary.

Lambda MicroVMs provides public outbound internet access by default. VPC egress through Lambda Network Connectors requires deliberate configuration. Isolation consequently leaves identity, permissions, tool authorization, logging, and state cleanup as separate responsibilities. An isolated agent is not necessarily an authorized agent.

The host should enforce deny-by-default egress. All VM outbound traffic passes through a policy-enforcing proxy or firewall that permits only the external endpoints the agent is authorized to reach. That policy belongs in provisioning. Enforcement inside the guest is insufficient because the agent cannot be trusted to police its own outbound access. Before a snapshot is saved, API keys injected through environment variables must be scrubbed, or credentials must be injected only at resume time, so tokens in saved state cannot be used for unintended outbound calls.

Multi-tenancy and keeping network paths between tenant VMs truly separate

Outbound controls do not settle the lateral network boundary. A tenant’s VM must also be blocked from reaching another tenant’s VM, even by accident. Otherwise, separate kernels can coexist with a network path to another user’s data, credentials, or session state. Network separation must match the compute boundary.

Static segmentation is poorly matched to continuous provisioning. Firewall rules are slow to update, and redesigning VLANs around every workload is operationally impractical when hundreds of VMs are created and destroyed per hour. The practical approach assigns each VM a unique IP in a non-overlapping private range during provisioning. Host routing rules prevent VM-to-VM traffic at the same enforcement layer as internet egress controls, while network namespaces keep the host’s internal routing table out of the guest’s view.

Tool access is a related but distinct boundary. MCP servers expose capabilities through lightweight JSON-RPC 2.0 interfaces over stdio or Streamable HTTP, allowing per-tenant tool sets without requiring shared network paths between tenants. One tenant can have code execution tools while another has only read-only search. That does not relax the one-VM-per-session rule. Reusing a VM across tenants risks inherited open connections and cached DNS results as well as credentials.

How snapshot-restore and idle suspension change the network lifecycle problem

Snapshot-restore makes persistent per-agent VMs economically viable by suspending idle execution. The installed dependencies, working directory, running processes, and warm page cache all stay frozen. Nothing runs while the agent is paused. Idle cost therefore trends toward zero, with snapshot storage and a small pool of pre-built network slots remaining as standing costs.

The network does not freeze with the guest. A remote endpoint may time out and close a network connection while the snapshot retains a kernel that believes the connection is open. The first network call after resume can consequently fail silently. The host must also recreate and reattach the TAP interface, restore routing, and sometimes reassign the IP address. The guest must tolerate transient network unavailability during that work.

Lambda MicroVMs preserves its HTTPS endpoint address across suspend and resume. The platform is built to keep that address, and a raw Firecracker implementation must do the same. SpecBox, a 2026 study evaluated on high-concurrency multi-turn agent traces, uses speculative sandbox preallocation and reduces P99 end-to-end latency by up to 2.9× against an on-demand baseline. It also uses less memory at its peak than deployments that stay permanently reserved. For networking, preallocated slots can remove resume-time TAP attachment from the critical path.

Credential lifetime must follow authorization. When a VM resumes, it must get the credentials for the intended session instead of inheriting tokens left in memory when it was saved.

Latency at the network layer and where the real bottlenecks appear in practice

Network-facing latency often originates outside packet forwarding itself. Host-side I/O, checkpoint overhead, and registry lookup time can dominate before inter-VM communication or external API calls become the limiting factors. Optimizing VM boot time alone misses those dependencies. A running kernel is not yet a responsive agent.

Cursor’s engineering identified disk I/O as a hotspot after addressing RAM when running hundreds of agents on shared machines. That finding has a network consequence. Agents waiting on disk can keep idle connections open, tying up file descriptors and connection-table entries. DeltaBox’s 2026 work addresses checkpoint overhead through incremental delta checkpointing hidden under inference time, reporting approximately 10.83 ms checkpoint time and 1.86 ms rollback time. Its approach keeps connections fresh rather than leaving them stale through long interruptions.

Request routing adds another unavoidable part of the latency budget. An incoming request that needs to find the correct VM endpoint must wait for a round trip to the central registry. Wake latency then compounds with host network reattachment and connection recovery. The practical measurement is time to the first successful network response. A sub-second wake requirement must include that full path, including the time after the kernel begins running.

A practical framework for making networking decisions for an agent fleet

The first choice is the isolation tier. Sensitive data, multi-tenant SaaS, and AI-generated code execution call for full VM-level separation, while container-level separation may be enough for other workloads. A hybrid architecture can route routine tasks to shared machines and high-sensitivity tasks to isolated VMs. That first choice sets how much network separation the infrastructure needs to provide.

The second decision is the endpoint model. Stable dedicated endpoints, as in the Lambda MicroVMs model, avoid a class of client-side session-tracking bugs, but their addresses must stay the same across suspension. Ephemeral addresses instead make reassignment part of routing and resume. This choice comes before routing is built because it sets what the registry must keep.

The third decision is the egress policy. Each agent tier needs an external-endpoint allowlist enforced at the host, with secrets kept out of snapshots and injected at resume. Tenant separation follows as the fourth decision. eBPF-based per-VM policy or a network namespace per tenant must scale with provisioning and apply automatically.

The fifth choice is how to design the registry and routing. Infrastructure commitments include production-grade latency for reads and writes atomic with provisioning to avoid a running VM that is not yet routable. Maritime is one option that embeds them into a platform, with a Firecracker micro-VM on bare metal, persistent disk, and isolated network namespace for each agent. Its snapshot-and-restore scheduler maintains a stable endpoint model and wakes agents in under a second, while abstracting registry management, TAP management, and resume-time reattachment.

The governing principle is one VM per user and one persistent identity per agent. It gives isolation, endpoint continuity, egress scope, tenant separation, and routing a common unit of ownership from the start.