Bare Metal vs Virtualized Cloud for Agent Compute

Firecracker micro-VM isolation for AI agents gives each agent its own guest kernel through KVM hardware virtualization. For bursty, stateful, multi-tenant agents, lightweight micro-VMs on bare metal combine per-agent isolation with dense hardware use. Virtualization earns its overhead when it adds enough isolation, persistence, and savings on idle cost, whether on bare metal or in the cloud.

Why the bare metal vs. cloud framing misleads developers building agent infrastructure

The familiar comparison starts with model training, where direct hardware access serves a workload that runs continuously and saturates GPUs. In that setting, an intermediary layer is mainly a possible source of lost throughput. Applying the same test to agent infrastructure leaves out what the execution environment must provide. A machine can have excellent throughput yet lack a safe boundary between users or lose their execution state through sleep.

Bare metal is the physical host, while a micro-VM is the environment given to each agent. Platforms such as Maritime, which runs each agent in its own Firecracker micro-VM on bare metal, combine both. Developers must assess which layer supplies the required properties at an acceptable cost.

What bare metal actually delivers and where its advantages are real

Bare metal gives a single tenant direct access to GPUs, CPUs, NVMe storage, and network interfaces through native drivers. No hypervisor scheduler stands between the workload and those resources. NVLink interconnects, PCIe lanes, and network interfaces are available without virtualization overhead. Because the physical machine belongs to one tenant, another tenant cannot compete for its memory bus, storage controller, or network interface.

These advantages matter when GPU performance depends on keeping data moving at full bandwidth. Virtualized environments can introduce latency and throughput variability, and they may not expose the complete NVLink topology. Sharing PCIe bandwidth across co-located VMs can also constrain the path feeding the GPUs. On long training runs, those costs change how much work the hardware can finish.

The financial case follows the same pattern. Paying a fixed cost for hardware can be cheaper than per-hour billing when training or inference keeps the machine busy for days or weeks. Bare metal also serves strict data-sovereignty requirements that rule out an abstraction layer between the workload and auditable hardware. Its strongest performance and cost case rests on sustained, saturated use.

Why agent workloads fail the assumptions bare metal optimization depends on

A coding agent follows a different activity pattern. It spends much of its wall time waiting for model API responses and network fetches, interspersed with bursts of tool execution, compilation, or testing. Hardware reserved for that agent remains a cost even when the agent has nothing to execute.

The unit of tenancy also changes. A product offering one agent per user may need hundreds or thousands of isolated environments. Single-tenant hardware separates that product from other hardware customers, but it does not separate the product’s users from each other. Per-agent isolation must come from another layer.

That layer must also keep files, credentials, installed packages, and session memory through restarts and sleep cycles. Bare metal supplies the hardware on which those properties, including execution continuity, can be built. Meanwhile, user activity can make the fleet expand sharply and then fall nearly quiet overnight. Fixed provisioned capacity is mismatched with a changing population of stateful agents.

Why containers do not solve the isolation problem for agents that execute untrusted code

Containers offer a lightweight way to divide a host into execution environments, but standard containers share its kernel. Sharing a kernel has risks when agents install arbitrary packages, run shell commands, use browsers, or run user-submitted code. A vulnerability in the shared kernel can expose the host and other tenants.

Hardening still has value. Seccomp profiles, dropped capabilities, and read-only root filesystems restrict what container processes can do. They provide defense in depth alongside the isolation supplied by separate guest kernels. For a fleet executing untrusted code, the distinction determines how far a single compromise can reach.

Containers alone also do not supply the complete persistent execution environment these agents need. Attaching durable storage does not preserve a running browser or a paused execution stack. A container also leaves an agent without control over a guest kernel of its own. The required primitive combines a kernel-level boundary with low overhead, fast boot, and persistent state.

How do you give each AI agent its own isolated kernel with Firecracker micro-VMs?

Firecracker runs each micro-VM with its own guest kernel through KVM hardware virtualization. An exploit against that guest kernel therefore meets a VM boundary, while a container’s host kernel is shared by design. This is the central difference from standard containers for untrusted agent execution. The security boundary sits around the agent’s operating environment.

Firecracker’s lightweight design lets a whole fleet have that separation. Its boot time is approximately 125ms, with approximately 5MB of memory overhead per VM. Firecracker’s design document states that a 36-core server can create 180 new micro-VMs per second, or five per CPU core per second. Those figures describe virtualization overhead and creation capacity, while an agent’s own work has additional memory and CPU requirements.

The host still matters because Firecracker requires KVM access. Bare metal exposes the hardware virtualization layer directly, while nested-virtualization cloud environments can constrain access to it. With persistent disks and lifecycle management, each guest can support package installation, browser sessions, and arbitrary code inside its own boundary. Fixed hardware cost is then spread across tenants through density.

This architecture does not give Firecracker guests direct GPU passthrough. Its minimal device model excludes PCI and VFIO machinery. GPU workloads requiring direct device access need a different VMM, such as Cloud Hypervisor.

How snapshot-restore scheduling turns idle agent compute into near-zero storage cost

Kernel isolation limits how far a compromise can reach, but idle guests would still cost money if they all kept running. Snapshot-restore scheduling changes that lifecycle. A micro-VM reaches a ready state, its memory and block-device state are preserved on local NVMe, and its process is suspended. The execution context survives without the agent continuing to run.

Maritime builds its snapshot-and-restore scheduler around the observation that most agents are idle most of the time. In this model, an inactive agent occupies only storage. Its persistent disk, kernel state, and credential store stay with its isolated environment. When activity returns, restoration brings back that environment.

For a product with thousands of users, the economic consequence is substantial. The physical host still has a cost, but snapshot scheduling frees its compute capacity to serve active work.

What per-user agent fleet economics look like across the 2026 infrastructure landscape

In the 2026 agent sandbox market, prices depend on whether a persistent agent has to keep running. An hourly rate describes active capacity, but it does not reveal the cost of preserving an inactive user’s environment. Even at a lower rate, a fleet can cost a lot if every agent keeps its allocated compute through long idle periods. Snapshot-based suspension changes the amount of compute purchased.

Dormant snapshots cost as much as storage-tier space instead of compute that runs all the time, which changes idle costs by an order-of-magnitude. That distinction matters most when inactive users greatly outnumber active users. A fleet comparison must therefore account for the running population as well as the total number of agents. Pricing persistence as continuous execution obscures the workload’s defining characteristic.

Self-hosting has a different cost structure because hardware amortization and colocation create a fixed monthly carrying cost. That fixed cost pays off when the hardware stays busy enough to spread it over enough work. Bursty fleets often fail that condition, making managed infrastructure more economical for most startups. Maritime’s flat-rate per-agent model uses sleep and restoration to keep persistent environments without charging for every dormant agent as an always-running VM. The economic advantage comes from the lifecycle.

What persistent state actually requires at the infrastructure layer and why it cannot be bolted on

Persistent agent state includes more than the records an application writes to a database. It includes installed packages, local files, credentials, authenticated browser state, and work paused partway through execution. These properties live across the filesystem and running processes.

An external database can retain key-value state, but it does not by itself retain an installed Python environment or a running browser session. Application checkpointing, too, does not save the full execution environment on its own. At a human-in-the-loop gate, an agent may need to resume with its call stack and pending continuations intact. That requirement reaches below the application’s data model.

The micro-VM model addresses the requirement through a persistent disk and a snapshot of memory. The disk retains filesystem changes, while the snapshot preserves execution context through sleep. Credentials stay in the agent’s own isolated environment, where other tenants do not share them. Isolation and persistence thus apply to the same unit, which is the agent’s machine. This is why infrastructure choice precedes agent design, since it determines what continuity the application can rely on and what it must reconstruct.

How developer tooling and deployment workflows build on this infrastructure foundation

A stronger execution boundary need not create a more complicated deployment workflow. The platform can manage the micro-VM, storage, and sleep-wake lifecycle while exposing a small deployment interface. The developer works with an agent endpoint. That separation is useful only if the platform handles the lifecycle properties the application needs.

Amazon Bedrock AgentCore’s MCP server illustrates this direction. It handles containerization, scaling, IAM authentication, and session isolation through standard MCP tool calls, with each user session in a dedicated micro-VM. Its tools span seven operational primitives, namely Runtime, Memory, Identity, Gateway, Policy, Browser, and Code Interpreter. MCP clients including Kiro, Cursor, Claude Code, and Amazon Q CLI can use that interface without putting ECS cluster and load balancer configuration on the developer’s deployment path.

Related work includes OpenAI’s Agents SDK, Google’s Agent2Agent (A2A) protocol, and Anthropic’s Model Context Protocol (MCP). These efforts make support for standard interfaces important without making the underlying runtimes interchangeable. With Maritime, a single CLI push produces a live API endpoint while the platform provisions the micro-VM, persistent disk, snapshot scheduler, and network endpoint. The practical goal is a small deployment surface backed by explicit isolation and persistence guarantees.

Making the infrastructure decision: which workloads belong on bare metal, which belong on micro-VM fleets

Direct bare metal is the right choice for continuous training or high-utilization inference that saturates GPUs for days or weeks. It also fits workloads that keep the hardware busy enough for its fixed cost to be lower than per-hour billing. Strict data-sovereignty requirements can independently justify that choice.

Micro-VM fleets on bare metal fit products that give each user a stateful agent executing untrusted code. Separate guest kernels provide the required tenant boundary, while persistent disks and snapshots retain the environment through idle periods. Snapshot scheduling is especially important when most agents spend most of their time inactive. Products that respond quickly to users need boot and wake latency low enough for that interaction, along with cheap storage for dormant agents.

Maritime is an option for this second profile, combining Firecracker isolation, persistent disks, and snapshot-based sleep with a managed deployment workflow. It does not replace the case for direct GPU access or sustained single-tenant execution. The decision follows from the resource being optimized, whether that is saturated hardware or a large population of intermittent, isolated environments. Virtualization earns its place when the isolation, continuity, and density it adds are worth more than the overhead it introduces.