Agent Continuity After Crashes and Unexpected Restarts

Agent continuity after a crash or unexpected restart is an infrastructure problem, not a software pattern problem. An agent that lives in a persistent, isolated environment with durable state and fast wake times can pick up where it left off. An agent in a stateless or shared environment cannot recover meaningfully, no matter how sophisticated its checkpointing logic becomes. The difference is determined by what the environment itself holds when the process dies.

Why agents crash mid-task and what they lose when they do

The leading cause of production agent failure is not hallucination or bad prompts. It is the loss of state when a long-running execution is interrupted. The failure modes are distinct enough to name separately: planned restarts from deployments or configuration changes, process crashes from out-of-memory errors or unhandled exceptions, context overflow when conversation history exceeds the model's context window, and hangs or timeouts where the agent is alive but stuck on an external call. Each destroys a different layer of what the agent was holding.

Consider an agent twelve steps into a twenty-step task. It crashes with no way to resume. No amount of application-layer retrying recovers the tool outputs, filesystem state, and external effects already produced. The agent does not lose its model weights or its prompt template. It loses the material residue of the work it was doing, and that residue is what made the work recoverable.

The scale at which this matters is now concrete. A 2026 enterprise survey found that most respondents had reversed an agent action, and a significant share had lost revenue because an agent failed. Production deployments at the scale of Tata Steel, with over 300 specialized agents, and Cognizant, with over 200 agents serving hundreds of thousands of employees, make restarting from scratch increasingly costly. This is not a theoretical concern about hypothetical agents. It is an operational reality for organizations that have already committed to agent-based workflows.

The severity of a crash is determined by what the environment was holding. Process state, open connections, filesystem artifacts, and external effects already produced are precisely the things that vanish when a stateless execution environment terminates. That distinction, between what survives and what does not, is where the infrastructure argument begins.

Why crash recovery is an infrastructure problem, not a software pattern problem

You can't recover meaningfully from a crash in a stateless or shared environment, no matter how sophisticated your application-layer checkpointing logic is. Maritime is one example of a platform that rejects it from the ground up, deploying each agent inside its own persistent micro-VM with a durable filesystem and running processes that wake from sleep in under a second. Recovery there is not a reconstruction problem. It is a resume problem.

Application-layer patterns have real value, but they have a hard boundary. A 2026 security study of agent checkpoint-and-rollback systems found that C/R designs span all three boundaries, but all leave a gap between restored state and the dependencies required for secure continuation. That gap produces real exploitable failures, not just inconvenience. The claim is that a LangGraph checkpointer on PostgreSQL provides durable state. Recovery capability is therefore a property of the infrastructure layer, and the rest of this examination concerns what that infrastructure must provide.

What state an agent actually needs to survive a crash

Agent state is not a single thing. It is four distinct tiers, and losing any one of them produces a different and serious failure even if the others survive. In-context working memory is what the model can see right now: the current prompt, conversation history, and tool results. Stored in Redis, DynamoDB, or PostgreSQL, it survives crashes if backed by durable storage, but only if the agent wrote to it before crashing.

Episodic memory is the append-only log of every observation, action, and outcome. This is the foundation of the "log is the agent" pattern. Semantic vector memory keeps embeddings, so past context comes back by meaning, not by exact key. It survives if the vector store is external and durable, but it cannot reconstruct tool outputs or filesystem artifacts.

Tool state is a fifth category, distinct from conversational memory. Database-backed checkpointing captures tiers one through three if correctly implemented, but tool state requires the environment itself to be persistent. No application library can substitute for a durable disk.

How application-layer checkpointing works and where it runs out

Database-backed checkpointing is the strongest application-layer recovery pattern you can get, and it earns its keep. LangGraph's checkpointer interface writes the full agent state, including message history, current execution node, tool outputs, and metadata, to SQLite, PostgreSQL, Redis, or other community backends such as MongoDB and DynamoDB at each state transition. On resume, the agent loads the latest checkpoint and continues from that node.

LangGraph offers three durability modes that map to a tradeoff between safety and throughput. Exit mode writes only when graph execution ends, so you get the best performance but no crash recovery mid-run. If an agent crashes after calling a tool but before checkpointing the result, it will retry the tool call on recovery, and API writes, database inserts, and file uploads need careful handling to avoid double effects.

The security dimension sharpens the boundary problem considerably. The 2026 systematic study of checkpoint-and-rollback in agent systems identifies five recurring failure modes: incomplete internal state coverage, inconsistent checkpoint state, external state mismatch, unbound nondeterministic replay, and unrecorded external effects. The study demonstrates these through end-to-end attacks on Hermes, Cline, and LangGraph, producing malware-verification bypass, unauthorized mail forwarding, and double payment.

A checkpoint restores what the framework recorded. A coding agent removes malware, scans the cleaned repository, and records a passing result. Most systems only cover the first.

What a persistent, isolated VM gives an agent that no application pattern can

A persistent, isolated VM covers all three recovery boundaries at once because the whole execution environment, not just the message log, is what persists. Running processes survive too, if memory is snapshotted. A dev server or database the agent started is already up in every restored fork, with no warm-up and no cache rebuild.

This means tool state is automatically protected. A VM snapshot captures what happened inside the environment. Effects applied to external services, such as API writes, emails sent, and payments processed, still require idempotency or a compensating transaction at the application layer.

Infrastructure that persists the full execution environment can restore these layers in a way that checkpointing the conversation history alone cannot. The agent can install packages, open browser sessions, and store credentials, and all of that state persists across sleep and redeploy. The environment is as durable as the agent itself.

Snapshot-restore as the recovery primitive that makes persistence affordable

With snapshot-restore scheduling, a persistent environment does not have to be a running environment. When the signal arrives, the harness restores the agent from the snapshot in milliseconds. This is what makes persistence affordable: agents can persist across sleep cycles, paying for storage rather than idle compute, and wake in under a second.

Firecracker's snapshot-restore API is the mechanism underlying this. Guest memory restores copy-on-write, so the Nth restore does not copy gigabytes of RAM. Pages are shared until something writes to them. A bex.co cost analysis from August 2026 compared three approaches for a sandbox fleet. CRIU checkpoint and restore reduces it further still, to a fraction of the always-on cost.

The idle billing problem is structural. Maritime's snapshot-and-restore scheduler is built on this insight: most agents are idle most of the time, so users pay for storage during sleep and compute only during activity. You get the flat rate of roughly one dollar per agent per month because of this scheduling model, not because isolation is weak. SpecBox, a 2026 Beihang University runtime middleware, uses speculative sandbox preallocation to cut P99 end-to-end latency significantly relative to the on-demand baseline while also cutting peak memory consumption relative to permanently reserved deployments. The warm/cold tradeoff is a scheduling problem, not a fixed constraint.

The rollback security problem that fast infrastructure alone doesn't solve

Restoring a VM to a prior snapshot recovers internal state faithfully but cannot un-apply effects the agent already sent into the world. Confusing "checkpoint restored" with "execution safely resumed" is a serious and underexplored failure mode. The 2026 systematic security study names this "execution continuity": the requirement that security-relevant states, decisions, assumptions, and effects carried across recovery remain jointly consistent with the valid execution history that justifies them.

The five failure modes the study identifies span all recovery designs. Incomplete internal state coverage, inconsistent checkpoint state, external state mismatch, unbound nondeterministic replay, and unrecorded external effects recur across heterogeneous C/R designs because they stem from a common architectural gap, not from implementation bugs. It includes unauthorized mail forwarding, where an authorization decision is carried forward in agent state but the mailbox configuration it authorized against has changed. It includes malware-verification bypass, where a scan result is retained in agent state while the workspace it applied to is rolled back.

A snapshot covering framework state, workspace state, and OS/VM state together, as a VM snapshot does, eliminates the workspace and OS mismatch failure modes. It does not eliminate the external-effect problem. That requires idempotency, compensating transactions, or explicit effect recording at the application layer. API writes, emails, payments, and file uploads to external services need their own idempotency keys or effect logs that survive independently of the VM snapshot. This is application-layer work, but it is well-defined and tractable once the infrastructure layer is sound.

What identity continuity requires beyond state recovery

State recovery gets an agent back to its last checkpoint. Identity continuity demands more. The agent before and after compaction behaves as two different entities. Users describe this as "we just discussed this; why are you asking again?"

The soul.py architecture paper from March 2026 draws on neurological case studies of human memory disorders to argue that human identity survives damage because it is distributed across multiple systems: episodic memory, procedural memory, emotional continuity, and embodied knowledge. When one system fails, others compensate. The soul.py implementation achieves this through persistent identity files and memory logs, with a hybrid retrieval system that routes queries to appropriate memory access patterns.

The multi-anchor insight maps directly to infrastructure design. A VM with a persistent disk provides the embodied anchor: the filesystem, installed packages, and running processes that constitute the agent's material continuity. Agents on persistent VM infrastructure get a strong embodied anchor by default. The remaining work is ensuring that episodic and semantic memory are stored durably outside the VM, so they survive even a VM-level failure, and that the retrieval system can reconstruct context from whichever anchors survived.

What recovery-capable agent infrastructure looks like in practice

A recovery-capable agent stack has three layers, and each one does work the others cannot. The infrastructure layer runs each agent in its own persistent, isolated VM with a durable disk and a real kernel. This covers framework state, workspace state, and OS/VM state at once, and makes tool state durable by default with no application code. Maritime's Firecracker micro-VM architecture on bare metal is built for exactly this, with sub-second wake times from snapshot and a flat per-agent pricing model that makes the per-user agent pattern economically viable at scale.

The application layer keeps the message log append-only and stored externally to the VM so it survives a VM-level failure. Context overflow is handled by a retrieval system that reconstructs from distributed memory anchors rather than relying on a single summarization step. The scheduling layer separates persistence from continuous compute cost, with snapshot-restore on idle so agents sleep to storage and wake in milliseconds. Each layer cannot substitute for the others. The infrastructure layer should be invisible except when something fails, at which point it should be the reason the agent recovered rather than the reason it did not.

When infrastructure makes isolated, persistent agents affordable at roughly one dollar per agent per month, the right primitive for an agent-native product is one VM per user, each with its own space, memory, and continuity across sessions. That is a different model from a shared stateless pool with application-layer state reconstruction. The agent lives in the cloud the way the architecture demands: as an entity that persists, not a function that gets called.