Threat Modeling Untrusted AI Agents to Pick the Right Runtime Boundary

Firecracker micro-VM isolation gives each AI agent its own kernel through hardware-assisted virtualization via KVM, rather than sharing the host kernel through containers. Choosing that boundary is a threat-modeling decision. The right design matches user misuse, model misbehavior, and external injection to separate enforcement mechanisms, pairing infrastructure containment with independent control over what the agent is allowed to do.

Why the runtime boundary question is really a threat-modeling question

“Which sandbox should be used?” starts with a mechanism. “What can go wrong, and through which channel?” starts with the conditions that mechanism must withstand. The distinction changes the design process before any configuration begins. Agents generate and execute instructions dynamically from model output, so their execution surface is not fully known at deployment. A boundary chosen because it is the usual choice can thus block harmless work while leaving a dangerous path open.

Prompt instructions cannot supply the missing architectural separation. The Parallax paper argues that prompt injection and safety instructions work through the same attention mechanism. Both rely on the reasoning system that is under attack. Consequently, an enforceable safety guarantee must hold even when the model stops following instructions.

The three attacker classes that organize the threat model

User misuse comes from the agent’s own user. That user may submit inputs intended to trigger dangerous tool calls, extract data the agent legitimately holds, or escalate privileges granted for ordinary work. Policy and identity are the relevant controls because the system must distinguish permitted use from abuse of existing capabilities. The user must be entitled to cause the action.

Model misbehavior can occur even when no user means harm. A model can generalize badly on novel inputs or drift during an extended conversation. Parallax describes gradual “salami slicing,” in which cumulative context shifts the model’s effective constraint boundary. This class calls for architectural separation between reasoning and execution, so that a faulty proposal cannot turn straight into an action.

External injection comes in through files, web pages, tool outputs, or retrieved documents. Parallax reports that indirect injection accounts for a majority of observed incidents and achieves meaningfully higher success rates than direct injection. The same paper reports that a successful attack in a single multi-agent incident reaches 48% of co-running agents. Trust can thus spread beyond the point where the attack came in. Data-provenance tracking and taint propagation address that spread in ways that neither a prompt nor a container provides alone.

What a threat model must answer before any boundary is drawn

The analysis begins with six concrete questions. First, what assets does the agent hold or reach, including source code, secrets, user data, production APIs, browser sessions, and private MCP services? Second, through which channels can attacker-influenced data enter, including package scripts, downloaded files, tool outputs, generated commands, MCP responses, and retrieved web content?

Third, what execution or privilege can those inputs trigger? A poisoned tool response that induces code execution can reach the host OS directly when no infrastructure boundary exists. Supply-chain channels also belong in this examination, including agent configuration files, MCP tokens, IDE task definitions, CI/CD OIDC configuration, and development-environment persistence hooks. The entry point need not resemble a prompt. Fourth, which attacker class dominates each route, and which other classes remain plausible?

Fifth, which rules must hold no matter how the model behaves? Those rules must be machine-readable, versioned, testable, and auditable rather than expressed only as natural-language intent. Sixth, what evidence will demonstrate that the boundary holds, including isolation configuration and boundary-check records? The output is a threat profile that precedes sandbox selection. It connects assets and attack channels to controls capable of enforcing the required limits.

What runtime isolation options fit each threat profile?

Process and container isolation fit trusted, first-party code exposed through a narrow API, where user misuse is the dominant concern. Namespaces and resource controls constrain execution, but containers share the host kernel. A kernel-level escape can cross that boundary. Mounting /var/run/docker.sock also gives a compromised container direct control of the Docker daemon. Even without such mistakes, container isolation neither separates reasoning from execution nor tracks the provenance of instructions.

gVisor provides a different enforcement primitive through user-space syscall mediation. The gVisor documentation describes a Linux-like interface implemented in user space, reducing direct exposure to the host kernel. This introduces a compatibility surface to maintain. It fits workloads where exploitation through agent-generated code is a central concern but full VM overhead is operationally prohibitive. Its value follows from that combination of risk and operating constraints.

Firecracker provides lightweight microVMs through a virtual machine monitor. Hardware-assisted virtualization via KVM gives each agent its own kernel, so compromising a guest kernel does not itself compromise another guest kernel or the host kernel. That is a different boundary from container namespaces. One VMM process per microVM also avoids a complex multi-tenant VMM daemon. Kata Containers is a project distinct from Firecracker that runs sandboxed containers through VM-backed runtimes.

MicroVMs contain execution resulting from model misbehavior or external injection, but they do not validate an instruction’s legitimacy. WebAssembly offers another fit for a different profile. It is highly constrained and fast, though tool compatibility may be limited when agent workloads require full Linux system calls. A narrow, well-defined toolset whose primary threat is resource exhaustion can suit Wasm without requiring a full Linux environment.

The counter-case is a normal backend service handling a narrow, server-side-authorized request, such as checking billing status or reading a calendar entry. That profile does not require microVM overhead. Adding a VM introduces latency and operational complexity without meaningfully reducing the dominant risk. The engineering test is whether the boundary reduces the identified risk enough while leaving the agent operable.

Where infrastructure containment ends and semantic enforcement must begin

Kernel separation leaves the meaning of an action unresolved. An agent can use a legitimate database tool to retrieve an entire customer table and return it through an authorized HTTP egress port. Every syscall may be valid, every tool approved, and every destination allowed. To the sandbox, that sequence stays within the boundary. Infrastructure containment and runtime policy enforcement therefore have distinct jobs, and neither replaces the other.

Parallax places an architectural boundary between cognition and execution. Its Cognitive-Executive Separation principle prevents the reasoning system from directly triggering actions, while Adversarial Validation with Graduated Determinism interposes an independent, multi-tiered validator. Information Flow Control propagates data sensitivity labels through workflows to detect context-dependent threats. Reversible Execution captures pre-destructive state for rollback.

The validator must still work even if an attacker takes over the whole reasoning system. OpenParallax, the open-source reference implementation in Go, uses Assume-Compromise Evaluation to test that condition by bypassing the reasoning system entirely. Parallax reports that it blocks 98.9% of attacks with zero false positives under its default configuration. That result applies to the reported evaluation.

In operation, this policy layer has four responsibilities. It defines the rules, blocks dangerous actions before they happen, observes behavior through logging and tracing, and supports safe recovery through rollback or containment. A VM can constrain where execution occurs while the policy layer permits or blocks the proposed action in its current context.

How the snapshot-restore pattern and persistent state change the threat model for long-lived agents

With persistent agents, the attack surface spans sessions. Parallax describes memory poisoning that implants false instructions in retained memory, turning an agent into a latent threat. Those instructions may remain after an execution session ends. The threat model must therefore include the state from which later execution resumes, rather than treating each request as an independent security event.

Execution checkpointing saves workflow state to a durable store so that work can pick up where it stopped without doing finished steps again. That continuity is useful, but the durable state requires the same integrity controls as live execution. Snapshot-restore also enables copy-on-write forks of a running VM’s memory and disk. Multiple fix attempts can then begin from the same starting state. Each branch is still a separate execution context with its own threat surface, and their shared starting state matters when checking the integrity of restored state.

Per-user fleets add a separate segregation requirement. Identity, files, caches, and state must remain distinct for each tenant, with isolation configuration and boundary-check records retained as evidence. Lifecycle choices also change exposure. Eager teardown after each tool call reduces idle attack surface but raises restore overhead, while retaining a running sandbox preserves that surface during inactivity. The boundary decision consequently includes which runtime starts execution, what survives it, and for how long.

How production deployments encode their threat model choices — three concrete cases

AWS announced AWS Lambda MicroVMs on June 22, 2026, as a resource providing dedicated, Firecracker-based environments for user-supplied or AI-generated code. Each invocation receives VM-level isolation with preserved in-session state, a dedicated HTTPS endpoint, and direct lifecycle control. The default timeout is 5 minutes, and persistent sandboxes auto-snapshot on idle suspend. AWS built the resource on the same Firecracker virtualization powering over 15 trillion Lambda requests each month. The encoded threat assumption is that supplied or generated code cannot be trusted at the kernel level.

Docker Sandboxes reached general availability in September 2026, bringing Firecracker microVM isolation to AI coding agents with 180ms cold start times. Its boundary is built for agents that execute dynamic code and need more than the container namespaces used for trusted processes. Kernel separation is the relevant design choice here. Firecracker’s mechanism also minimizes exposed device emulation to what the workload requires, shrinking the VMM attack surface structurally.

The Codex CLI three-mode design illustrates a different division of responsibility. It focuses on agent capability and treats platform-level isolation as a precondition supplied outside the tool. That is a product-boundary application of Cognitive-Executive Separation. Unlike the first two cases, the defining choice concerns where responsibility for VM isolation resides.

The contested frontier: where today's boundaries still don't hold

An adversarial host changes the assumptions behind microVM isolation. Trezala Labs argued in June 2026 that Firecracker remains a software boundary because a compromised host places the hypervisor within the attacker’s reach. Its position is that silicon-enforced Trusted Execution Environments are the boundary that survives that threat. The claim remains contested in production practice. As of October 2026, no production agent platform had adopted TEEs as standard.

That debate concerns a narrower profile than most agent builders face today. MicroVM isolation remains a strong and economically viable boundary for most threat profiles, but its suitability depends on which components the model assumes are trusted. Host-adversarial deployments need to be assessed differently from those where hostile code runs inside a platform that is otherwise trusted. A stronger guest boundary cannot, by itself, settle that distinction.

The semantic gap is the more immediate unresolved challenge. Information flow control can propagate sensitivity labels through agent workflows, but it still requires an independent validation layer to interpret those labels in context. Increasing infrastructure isolation does not supply that layer. Nor does a policy engine remove the need to contain the code that eventually runs.

Good threat modeling produces a calibrated boundary with limits. Working through the three attacker classes and six practical questions identifies the dominant risks, the enforcement primitives that address them, and the behaviors left outside their reach. Knowing what the boundary cannot do is itself a useful security result, because it defines the policy layer that must stand beyond it.