DM Explains · No. 004 · Agents
Persistent agent states: why AI agents forget everything unless you make them remember
An AI agent does not remember anything by nature. If a crash or a days-long approval wait is not supposed to wipe out its progress, something has to write that progress down after every single step.
· 13 min read
Read me first
An AI agent genuinely does forget everything the moment its process stops, and no amount of model intelligence fixes that; only a separate, deliberately engineered checkpoint written to external storage does.
A CRM-writing CrewAI Flow researches a lead → 1 conversation history and cursor captured → 2 @persist snapshots Pydantic state to SQLite → 3 kickoff(inputs={'id':...}) keys the run → 4 pauses for manager approval, no tokens spent waiting → 5 rerun with same id rehydrates state, skips completed steps → 6 a different worker resumes the run from Postgres → CRM record updated
An agent has spent forty minutes and a stack of paid LLM calls working through a multi-step task — researching, drafting, calling tools — when the server it runs on restarts. Nothing was malicious, nothing was even unusual; servers restart. What happens next depends entirely on a decision made long before the crash, not on how capable the underlying model is.
Without a deliberate mechanism to save progress, the agent does not resume from where it stopped; it starts again from step one, silently re-running and re-paying for everything already done. Worse, any workflow that was waiting days for a human's approval simply evaporates — there is no "where was I" to return to, because nothing wrote that down.
Everyday picture. Dadi is halfway through a long story to Arjun when the power cuts out. When the lights return she does not start over — she keeps a bookmark in her notebook, not just in her head, so she picks up from the exact sentence she had reached.
Where the picture breaks. Dadi's memory is hers alone; a real durable-execution system deliberately writes state to an external database precisely because trusting any single "head" to remember is unsafe — the whole point is that the bookmark survives even if Dadi herself does not come back.
Why it exists now
Agent frameworks have only recently converged on treating this as a first-class feature rather than an afterthought. CrewAI documents a @persist() decorator for its Flows that snapshots a Pydantic state object to SQLite after each step, and its documentation walks through the exact failure it is designed to prevent: a Flow that crashes at step four and, without persistence, restarts at step one, re-running every LLM call already paid for (dreaming.press). Microsoft has published a Durable Extension for its Agent Framework that layers Durable Task infrastructure — including a Durable Task Scheduler and Azure Functions hosting — onto agents specifically for checkpointing, crash recovery, and pausing for human input (Microsoft). And open-source reference projects such as ForgeFlow combine LangGraph's graph-based workflow model with PostgreSQL checkpointing and human-in-the-loop interrupts as a deliberately "opinionated" blueprint for deploying agents safely (GitHub).
What changed is not the model underneath — it is that agents are now expected to run for hours, days or weeks, touch real systems with real side effects, and wait on real people, which a single in-memory process was never built to survive.
How it works
Capture the state
Nothing survives that was never written down
What it is. After each step — a tool call, an LLM response, a decision node — the agent's full working state is serialised, not just carried forward in the running process.
Why it exists. Agent state, as described in durable-execution literature, includes conversation history, the execution cursor showing which step is current, pending external effects, and external references; missing any of these means resuming later loses context or repeats reasoning already done.
Where it breaks. If capture is partial — history saved but a pending effect not — resuming can duplicate a side effect like an email or a CRM write.
How that's handled. The snapshot is treated as a single atomic unit representing the whole step, not a partial log entry.
In technical terms
In technical terms: CrewAI's @persist decorator snapshots the Flow's Pydantic state after each step, defaulting to a SQLiteFlowPersistence store, while frameworks such as LangGraph capture equivalent state as a graph node payload.
Write it durably
If it only lives in memory, it dies with the process
What it is. The captured snapshot lands in storage outside the running process, before the next step is allowed to begin.
Why it exists. Durable execution requires that every completed step be written externally before the next one starts, rather than held in memory or buffered — otherwise a crash between two steps loses the one not yet written (agentwiki.org).
Where it breaks. LangGraph's in-memory state saver loses all workflow state on restart; it looks like persistence right up to the first crash, at which point developers are pointed towards a persistent checkpointer such as PostgresSaver backed by a real database instead (Substack).
How that's handled. Production deployments swap the default in-memory saver for a database-backed one deliberately, rather than assuming the default is already safe.
In technical terms
In technical terms: CrewAI writes to SQLite by default, ForgeFlow checkpoints every node of its LangGraph StateGraph to PostgreSQL, and Microsoft's Durable Extension persists through Durable Task Scheduler infrastructure — any of these outlives the process that wrote to it.
Key it by a stable id
A run only comes back if you know its name
What it is. The saved state is only ever retrieved again through a stable identifier chosen when the run starts.
Why it exists. This id is the only link between a crashed run and its saved checkpoint; CrewAI Flow persistence is keyed on the Flow id, passed via kickoff(inputs={'id': ...}), and LangGraph equivalents use a thread_id.
Where it breaks. Pass a fresh random id each run and the checkpoint is orphaned forever, even though it was written correctly — there is no error message, just a brand-new run starting from zero next to an unreachable old one.
How that's handled. The id is generated once, stored alongside whatever triggered the run, and reused deliberately on every retry.
In technical terms
In technical terms: resuming a CrewAI Flow means calling kickoff again with the same id so the persistence layer can find and rehydrate the matching snapshot (dreaming.press).
Pause for a human
Waiting for a person should not cost tokens
What it is. Execution can stop at a named point and hold its full state for hours or days without spending compute.
Why it exists. Many workflows need sign-off before a high-impact action — an email send, a CRM write — and suspend/resume primitives let a workflow pause for hours or days awaiting approval without losing state, with full context persisting across the pause.
Where it breaks. If the pause point sits after the side effect rather than before it, approval becomes theatre — the email is already sent by the time anyone reviews it.
How that's handled. ForgeFlow pauses via LangGraph's interrupt_before, resuming only on an approve/reject webhook or a Slack button, and Microsoft's Durable Extension allows agents to pause for human input or external events without consuming compute or model tokens while waiting.
In technical terms
In technical terms: interrupt_before halts execution ahead of a specified node; Durable Task's suspend/resume primitives achieve the same effect on Microsoft's infrastructure.
Resume, skip done work
A rerun should cost nothing it already paid for
What it is. The same id rehydrates the saved state, and the engine continues past steps already marked complete rather than re-executing them.
Why it exists. Re-running a CrewAI Flow with the same id rehydrates self.state and skips completed steps, which is the direct fix for the motivating failure case — a Flow crashing at step four and restarting at step one, re-running every already-paid-for LLM call.
Where it breaks. If a step had an untracked side effect, "skip on resume" can also mean "never actually verify it happened".
How that's handled. Side effects worth protecting are checkpointed as explicitly completed only after they are confirmed, not merely attempted.
In technical terms
In technical terms: resumption compares the rehydrated execution cursor against the workflow's step list and short-circuits anything the cursor already passed.
Hand off across workers
Progress belongs to storage, not to one machine
What it is. Because state lives in shared storage rather than one machine's memory, any worker in a fleet can pick up and continue any run.
Why it exists. Durable agents built with Microsoft's extension can scale work across distributed, stateless workers based on demand without changing core agent logic, and Microsoft's extension supports both managed Azure Functions hosting and bring-your-own-compute self-hosted execution for exactly this reason.
Where it breaks. This only works if every worker reads from the same store with the same schema; a worker on an older schema version can silently misread state.
How that's handled. Schema changes to the persisted state are versioned and rolled out to all workers together, not staggered.
In technical terms
In technical terms: ForgeFlow's PostgreSQL checkpoints and Microsoft's Durable Task Scheduler both decouple a run's logical progress from any single process, letting stateless workers claim whichever run needs attention.
Compared with what came before
| Stateless / in-memory agent | Persistent agent state | |
|---|---|---|
| Survives a crash | No — restarts from step one | Yes — resumes from last checkpoint |
| Cost of a restart | Full re-run of every LLM call | Only steps after the last checkpoint |
| Human approval | Must happen inline, holding the process open | Can wait days without consuming compute or tokens |
| Scaling | Tied to the process/machine that started the run | Any worker reading the same store can resume it |
| Failure mode | Loud — it visibly restarts | Can be silent — a wrong id orphans a checkpoint with no error |
The judgement is not that the old, stateless way is wrong — a short single-turn request genuinely has nothing worth checkpointing. It is that once a run is expensive, long-running, or waiting on a person, treating it as stateless is a choice to redo work and lose approvals whenever something ordinary, like a restart, happens.
Where it breaks
Illustrative: run id lead-4471 kicks off → 1 researches the lead, state saved → 2 drafts the CRM update, checkpoint written → 3 worker dies mid-write → 4 approval sits unresolved for two days, zero tokens spent → 5 manager clicks approve in Slack → 6 a fresh worker rehydrates state from lead-4471, skips steps 1-2 → CRM record written, exactly once
Persistence helps, but only when every layer of it is actually built, not assumed. A run that crashes at step four and restarts cleanly at step four looks identical from the outside to one that silently restarted at step one — the difference only shows up in the bill and in whether an approval that was waiting is still there.
Not a security or correctness boundary
Storing state durably says nothing about who is allowed to read or resume it, and skipping completed steps on resume says nothing about whether those steps' side effects were actually verified. Both have to be enforced separately from the persistence mechanism itself.
The most common way this fails in practice is quiet, not loud: a fresh id on retry orphans a perfectly good checkpoint, a default in-memory saver is mistaken for a real one until the first restart, or a step's side effect is skipped on resume without ever confirming it happened the first time. None of these produce an error message — they produce a system that looks durable until the day it is tested.
Numbers to carry
| Detail | Value |
|---|---|
| CrewAI Flow persistence default store | SQLite, via SQLiteFlowPersistence |
| CrewAI Flow resume key | Flow id, passed via kickoff(inputs={'id': ...}) |
| ForgeFlow checkpoint store | PostgreSQL, checkpointing every LangGraph StateGraph node |
| ForgeFlow human approval mechanism | interrupt_before, resumed via approve/reject webhook or Slack button |
| Microsoft Durable Extension hosting models | Azure Functions (managed) or bring-your-own-compute (self-hosted) |
| Microsoft Durable Extension underlying infrastructure | Durable Task Scheduler |
| LangGraph unsafe default | In-memory state saver, loses all state on restart |
| LangGraph safe alternative | PostgresSaver, backed by a real database |
Myths worth killing
| Commonly repeated | What is actually true |
|---|---|
| "The agent remembers what it did because it's intelligent." | It remembers nothing by nature; a separate checkpointing system writes its state to storage after each step, and without that system it starts over. |
| "In-memory state counts as persistence." | LangGraph's in-memory state saver loses all workflow state on restart; only a database-backed checkpointer such as PostgresSaver survives one. |
| "Pausing for approval keeps burning tokens while it waits." | Suspend/resume primitives hold full context across a pause of hours or days without consuming compute or model tokens. |
| "Long-term memory and step checkpointing are the same feature." | CrewAI's crew long-term memory (backed by LanceDB) stores durable knowledge across separate runs; @persist checkpoints one run's execution spine — conflating them means persisting the wrong thing. |
| "If the run resumes correctly, it must be safe." | Resuming and skipping completed steps can also mean skipping verification that a side effect, like a CRM write, actually happened. |
Worked example
- Papa's team runs a CRM-writing agent Flow: research a lead, draft a CRM update, pause for a manager's approval, then write to the CRM.
- The Flow starts with a fixed id tied to the lead's record, not a random one generated per attempt.
- After the research step and again after the draft step,
@persistsnapshots the Flow's Pydantic state to SQLite. - Illustrative: forty minutes and several LLM calls into the run, the worker process crashes during the drafting step.
- Because the id is stable, restarting the Flow with the same id rehydrates the last saved state — research and drafting up to the last checkpoint are not re-run.
- Before the CRM write, the Flow pauses via an interrupt-style approval gate and waits for the manager, holding state without spending further tokens.
- Illustrative: the manager is travelling and approves three days later; the Flow resumes from exactly that gate and performs the CRM write once.
- Because the state lived in shared storage rather than the crashed worker's memory, a different worker in the fleet could have picked up the resume — and did.
Should you use it?
Build this when a run is expensive or long enough that redoing it would hurt, when a step genuinely must wait on a person rather than a timeout, or when a fleet of workers needs to share and resume the same runs. Skip it for short, single-turn, stateless interactions where a restart costs nothing to repeat, and do not reach for it as a substitute for a proper memory or retrieval layer — that is a different feature solving a different problem. The cheapest first experiment is not a new agent: take one existing multi-step flow, add a persistence decorator or checkpointer to it, kill the process mid-run on purpose, and see whether it actually resumes rather than silently restarting.
Say it in the room
"Persist state after every step and key it by a stable id, or the agent isn't durable — it just hasn't crashed yet."
Lines to carry
- "Nothing survives that was never written down."
- "If it only lives in memory, it dies with the process."
- "A run only comes back if you know its name."
- "Waiting for a person should not cost tokens."
- "Progress belongs to storage, not to one machine."
- "Skipping a step on resume is not the same as verifying it happened."
FAQ
Doesn't the model itself remember the conversation?
No — a language model predicts text from whatever is placed in its context window for that single call; it does not retain anything between calls unless a separate system writes the conversation down and feeds it back in next time. That separate system is exactly what persistent agent state is.
Is this the same as giving an agent long-term memory?
No, and conflating them causes real bugs. CrewAI's crew long-term memory, backed by LanceDB, stores durable knowledge across many separate runs. Step checkpointing via @persist is about one run's execution spine surviving a crash within that single run.
What happens if I forget to pass the same id on retry?
The checkpoint is written correctly but becomes unreachable — a fresh random id starts an entirely new run from scratch, sitting silently next to the old, orphaned one, with no error to flag the mistake.
Does pausing for approval cost anything while it waits?
Not in compute or model tokens, if the pause is built on genuine suspend/resume primitives rather than a process kept alive in a loop. Full context persists across the wait, whether that wait is minutes or days.
Can I just rely on the default settings in these frameworks?
Not safely. LangGraph's default in-memory state saver loses everything on restart; it has to be swapped for a database-backed checkpointer such as PostgresSaver deliberately, and CrewAI's Flow persistence only helps if a stable id is actually passed at kickoff every time.
Glossary
- Durable execution — an approach where a workflow's progress is written to external storage after each step, so it survives crashes and restarts without repeating work.
- Checkpointing — saving the current state of a running process to persistent storage at defined points, so it can be reloaded later.
- Human-in-the-loop — a workflow design where execution pauses to require a person's approval or input before continuing.
- LangGraph — a framework for building agent workflows as a graph of steps, supporting persistence and interruption of that graph.
- StateGraph — a graph-based representation of a workflow's steps and transitions used by LangGraph, where each node can be checkpointed.
- Interrupt_before — a LangGraph mechanism that pauses execution before a specified node, typically to await human approval.
- Flow id / thread_id — a stable identifier passed into a workflow run, used to look up and resume its saved state later.
- Persistence store — the database or storage system, such as SQLite or PostgreSQL, where an agent's state snapshots are written.
- Durable Task Scheduler — infrastructure used by Microsoft's Durable Extension to manage checkpointed orchestration and workflow execution.
- Crew long-term memory — a separate CrewAI feature storing durable knowledge across multiple runs, backed by LanceDB, distinct from Flow step-checkpointing.
Sources
- JoelJohnsonThomas/ForgeFlow — GitHub
- Durable Extension for Microsoft Agent Framework — Microsoft
- Make Your CrewAI Flow Survive a Crash: A Hands-On Guide to @persist — dreaming.press
- Durable Execution for Agents — AI Agent Knowledge Base
- Durable Agents: Surviving Crashes & Restarts — Diagrid
- Designing Agent State Machines for Long-Running Business Workflows — Substack