DM Explains · No. 003 · Orchestration
Stateful orchestration: why AI agents now save their place before continuing
Agents that remember where they stopped can pause for approval, survive a crash and pick up exactly where they left off. Here is the machinery that makes that possible, and when it is not worth building.
· 13 min read
Read me first
Stateful orchestration genuinely does let AI agents survive crashes and pause for human sign-off, but only when every step writes its progress to durable storage; the graph itself remembers nothing, the database does.
Illustrative: loan agent pulls documents (1) into typed state (2) → checkpointed to Redis after every node (3, f9) → router decides risk is borderline (4) → pauses for a human approver via interrupt (5, f2) → short-term memory holds this run, long-term memory holds prior loans (6, f3) → resumes behind a stateless gateway and disburses funds (7, f8)
A multi-step agent is three-quarters through a loan approval workflow — documents pulled, risk checked, waiting on a human sign-off — when the server restarts. Without explicit state, that restart means starting the whole workflow from scratch: re-running tool calls, re-asking the model, possibly re-charging an API, and losing the human's pending review entirely. The difference between an agent that shrugs off a crash and one that loses an afternoon of work is not intelligence. It is whether anything was written down.
Everyday picture. Dadi is cooking a multi-day festival meal. Each evening before bed she writes down exactly which dish is half-done and what is prepped, and puts the note in the kitchen drawer, so if the power goes out overnight she can pick up from that exact point rather than starting the whole meal over.
Where the picture breaks. Dadi's notebook is one object in one drawer; real checkpointers replicate the same state across several databases at once for reliability, which a single notebook cannot show.
Why it exists now
Agent frameworks have run multi-step tool calls for years, but until recently the "state" of a workflow lived inside one process's memory: kill the process, lose the state. LangGraph reached general availability at version 1.0 on 22 October 2025, converting years of production patterns into a stable, backward-compatible application programming interface with standardised checkpointer interfaces across SQLite, PostgreSQL and Redis, along with new interrupt and Command primitives for human-in-the-loop control (LangGraph 2.0: Stateful Multi-Agent Orchestration Framework — Jose Nobile, 2026). CrewAI, a competing framework, expanded its Flows feature over the same period to give event-driven, stateful control alongside its autonomous "Crews" (GitHub - crewAIInc/crewAI — GitHub, 2026).
What changed is not the idea of remembering progress — workflow engines have done that for decades — but that agent frameworks now interleave deterministic business logic with model-driven, non-deterministic steps inside the very same persisted structure. LangGraph is described as trusted by companies including Klarna, Uber and J.P. Morgan for building long-running, stateful agents in production (langchain-ai/langgraph — GitHub, 2026). That is the stake: once an agent's job is a loan approval or a multi-day research task rather than a single chat reply, "restart from zero" stops being an acceptable failure mode.
How it works
Graph of nodes
Behaviour becomes a map, not a hidden loop.
What it is. A developer defines a StateGraph where nodes are functions — usually a call to a language model or a tool — and edges, some conditional, decide what runs next.
Why it exists. Drawing the workflow as an explicit graph, rather than a hidden loop inside one function, lets deterministic code and model-driven steps sit inside the same structure and be inspected separately.
Where it breaks. For a single question-and-answer exchange, drawing a graph is overhead with no payoff.
How that's handled. Reach for a graph only once a task has more than one step that might need to branch, retry or pause.
In technical terms
In technical terms: LangGraph lets developers mix deterministic, hand-coded steps with LLM-driven agentic steps within the same graph (LangGraph Overview — LangChain).
Typed shared state
Every step reads the same notebook and writes back to it.
What it is. State is a defined shape, not a loose blob passed hand to hand — commonly a Python TypedDict.
Why it exists. Each node receives the current state and returns a partial update that the framework merges in, which forces disciplined data design and makes it obvious what each step actually changed.
Where it breaks. Rigid typing helps debugging but adds design work upfront that a quick prototype rarely needs.
How that's handled. Start with a loose shape for a prototype, then tighten the type once the workflow is heading to production.
In technical terms
In technical terms: LangGraph checkpoints and merges partial state updates returned by each node, combined with PostgreSQL or SQLite checkpoint backends, so workflows can pause, resume and recover from failures (Multi-Agent Orchestration: LangChain vs CrewAI vs AutoGen — Inductivee).
Checkpoint each step
Nothing is lost that was already written down.
What it is. This is the actual mechanism behind "survives a crash": a graph compiled with a checkpointer saves state after every node transition, keyed by a thread ID, to a backend such as SQLite, PostgreSQL or Redis.
Why it exists. A graph compiled without a checkpointer runs stateless — every invoke call starts fresh; one compiled with a checkpointer saves state after every node transition instead (LangGraph in Production — Kalvium Labs). LangGraph provides this as durable execution, letting agents persist through failures and resume from exactly where they left off (langchain-ai/langgraph — GitHub, 2026).
Where it breaks. Checkpointing adds a database hop on every step, which costs latency that latency-sensitive requests cannot afford.
How that's handled. Checkpoint at node boundaries, not inside a node, and reserve the pattern for workflows measured in seconds or longer, not milliseconds.
In technical terms
In technical terms: a graph compiled with a checkpointer serialises and stores the full state object after each node, keyed by thread ID, so a later resume call reloads exactly that state rather than starting the invoke chain over.
Router
One decision, feeding straight back into the map.
What it is. A function looks at the state and names the next node.
Why it exists. Conditional routing inspects intermediate results — a risk score, a tool failure — and dynamically returns the next node's name instead of following a fixed sequence.
Where it breaks. Too many branches and the graph becomes as hard to reason about as the free-form agent it replaced.
How that's handled. Keep routing functions small and testable, and log every routing decision alongside the state that produced it.
In technical terms
In technical terms: a conditional edge is a function bound to a node that returns the string name of the next node based on fields in the current state, rather than a fixed destination.
Human interrupt
Progress waits for a person instead of vanishing for one.
What it is. The agent pauses mid-task and hands its exact state to a person for review.
Why it exists. LangGraph supports human-in-the-loop workflows by allowing inspection and modification of agent state at any point during execution (langchain-ai/langgraph — GitHub, 2026). Interrupt and Command primitives were made first-class in LangGraph 1.0, generally available since 22 October 2025, replacing the older breakpoint approach (LangGraph 2.0 — Jose Nobile, 2026).
Where it breaks. If no step genuinely needs sign-off, adding interrupts just adds waiting.
How that's handled. Reserve interrupts for the few steps where a wrong decision is expensive or irreversible — a disbursement, a legal filing — not every step a model touches.
In technical terms
In technical terms: calling interrupt within a node halts graph execution and returns the current state to the caller; a subsequent Command call resumes the same thread from that exact point, optionally with edited state.
Memory layers
One list for this week, one book for every year.
What it is. Short-term memory reasons within a run; long-term memory spans sessions.
Why it exists. LangGraph offers both short-term working memory for ongoing reasoning inside one run and long-term persistent memory across otherwise unrelated sessions (langchain-ai/langgraph — GitHub, 2026).
Where it breaks. A one-off task with no future session has nothing for long-term memory to serve.
How that's handled. Turn on long-term memory only for agents a person or business genuinely returns to, such as an ongoing customer account, not one-shot tasks.
In technical terms
In technical terms: short-term memory lives inside the checkpointed thread state; long-term memory is a separate store keyed by a user or entity identifier rather than a thread identifier, so it survives across many unrelated threads.
Deploy behind gateway
The server forgets; the database remembers.
What it is. Compiled graphs run behind a stateless application programming interface gateway, with the actual state held in Redis or similar.
Why it exists. A production architecture places a StateGraph behind a stateless gateway, backed by Redis for checkpointing, often running on Kubernetes, and traced with a tool like LangSmith to visualise execution paths, state transitions and runtime metrics (LangGraph Production Architecture — Markaicode). This gives per-node retries and crash resumability at the cost of an extra latency hop from the stateful dependency.
Where it breaks. This is genuine infrastructure — running it well needs the same operational care as any other stateful production service, not a toggle in a settings menu.
How that's handled. Treat the checkpoint store like any production database: back it up, monitor it, and size it for the workflow's actual write volume.
In technical terms
In technical terms: compute nodes scale horizontally and hold no state between requests; the checkpointer backend (Redis, PostgreSQL) is the single source of truth for a thread's progress, so any gateway instance can resume any thread.
Compared with what came before
| Stateless agent call | Stateful orchestration | |
|---|---|---|
| Failure recovery | Restart from the beginning | Resume from the last checkpointed node |
| Human review | Outside the workflow, or not at all | Interrupt pauses the run, exposes state, then resumes it |
| Debugging | Re-run and hope to reproduce | Inspect the exact state at any past step |
| Setup cost | Minimal — one prompt, one call | Graph design, typed state, a checkpoint backend |
| Best fit | Single-turn question and answer | Long-running, multi-step, auditable workflows |
The trade is explicit: stateful orchestration buys resumability and auditability at the price of designing a graph and running a database. A practitioner account of six LangGraph projects concludes the framework is only worth the added complexity under certain conditions (LangGraph in Production — Kalvium Labs), which is the honest verdict for the whole pattern: it fits long-running, reviewable work, not every agent.
Where it breaks
Stateful orchestration solves the "lost progress" problem, but only for the parts of a workflow that were actually checkpointed; it does not make the underlying model calls more accurate, and it does not by itself secure anything.
Not a security boundary
Checkpointed state decides what the workflow can resume, not who is allowed to trigger, inspect or edit that state. Access control has to be enforced around the gateway and the checkpoint store, not assumed from the graph.
Latency is the other honest cost: every checkpoint write is a database hop, so a workflow that needs sub-second responses will feel the stateful machinery as drag, not safety. And the graph itself can become the very complexity it was meant to tame — too many conditional edges, and tracing why a workflow took a particular path is as hard as debugging the free-form agent loop it replaced.
Numbers to carry
| Item | Value |
|---|---|
| LangGraph 1.0 general availability | 22 October 2025 |
| LangGraph release cadence | major releases every 6 to 12 months, per its semantic versioning policy |
| CrewAI's claimed developer base | over 100,000 certified developers, as of September 2026 |
| Companies cited using LangGraph in production | Klarna, Uber, J.P. Morgan |
Myths worth killing
| Commonly repeated | What is actually true |
|---|---|
| "State makes an agent smarter." | State makes an agent's progress durable and inspectable; it does not change what a model call outputs. |
| "A checkpoint stores what the model understood." | A checkpoint stores the data structure the graph was passed, nothing about comprehension. |
| "Human-in-the-loop means a human runs the whole workflow." | It means the graph pauses at chosen points and hands over just the current state; the rest still runs unattended. |
| "Any agent framework needs a graph and a database." | Single-turn or short chains gain little; the pattern earns its cost on multi-step, resumable, auditable work. |
| "CrewAI's Crews and Flows are the same thing." | Crews model autonomous role-based collaboration; Flows are the explicit, checkpointed, event-driven control layer. |
Worked example
- Papa's team (system) submits a loan application; the graph's first node pulls the applicant's documents and writes the extracted data into the typed state.
- The checkpointer saves that state, keyed by a thread ID for this specific application, to the PostgreSQL backend before the next node starts.
- A risk-check node runs a model call over the documents and writes a risk score into the state; the checkpointer saves again.
- The router reads the risk score and, since it falls above a threshold, sends the workflow to a human-review node instead of straight to disbursement.
- The interrupt halts execution and exposes the full state — documents, risk score, reasoning so far — to a human approver, who reviews it the next morning.
- The server restarts overnight (illustrative: due to a routine deployment); because the state was checkpointed after step 3, nothing before the interrupt needs to be re-run when the process comes back.
- The approver's Command call resumes the exact thread, optionally edits the state to add an approval note, and the workflow continues to disbursement.
- A long-term memory store records that this applicant's loan was approved with conditions, available to any future workflow touching the same customer, separate from this thread's own short-term state.
Should you use it?
Reach for stateful orchestration when a workflow has more than a couple of steps, must survive a restart without losing the work already done, or needs a genuine human sign-off partway through — the loan approval case, a multi-day research agent, anything regulated enough that auditability matters. Skip it for a single prompt-and-response tool, a quick prototype, or anything where sub-second latency matters more than resumability; the graph, the typed state and the checkpoint database are real engineering cost, not a free upgrade.
The cheapest first experiment is to take one existing multi-step agent, wrap its steps as nodes in a graph, compile it with a checkpointer against a simple SQLite file, and deliberately kill the process mid-run to watch it resume rather than restart.
Say it in the room
"Use it when a workflow must survive a crash or pause for a human mid-task; skip it when one model call and one response is the whole job."
Lines to carry
- "State is a note left for your future self."
- "A crash should cost a restart, not a rewrite."
- "Pausing for a person is a feature, not a failure."
- "The server forgets; the database remembers."
- "Structure is what lets a model be trusted with a step."
FAQ
Does stateful orchestration make an agent more accurate?
No. It changes whether progress survives a crash or a pause, not what any individual model call outputs. Accuracy still depends on the model, the prompt and the data it is given at each node.
Is LangGraph the only way to do this?
No — CrewAI offers a comparable pattern through its Flows feature, which adds event-driven state, branching and checkpointing alongside its autonomous Crews. Other orchestration frameworks exist too; LangGraph and CrewAI are simply two named examples with public production adoption.
Does checkpointing slow everything down?
It adds at least one database write per step, which is a real latency cost. For workflows measured in seconds or longer, that cost is usually negligible next to a model call; for anything needing sub-second responses, it is a genuine tax.
Who can see the state during a human interrupt?
Whoever the surrounding application lets query the checkpoint store — the graph itself enforces no permissions. Access control has to be built around the gateway and the storage backend, not assumed from the interrupt mechanism.
Do I need Redis or Postgres specifically?
No specific backend is mandatory; LangGraph's checkpointer interface supports SQLite, PostgreSQL and Redis, among others, and the right choice depends on expected write volume and how the workflow is deployed.
Glossary
- Stateful orchestration — coordinating multi-step AI workflows where progress and data are explicitly tracked and saved so the workflow can pause, resume or recover.
- Checkpointing — saving a workflow's current state to storage after each step so execution can resume from that point later, even after a crash.
- StateGraph — LangGraph's representation of a workflow as nodes and edges, with an explicit typed state passed between them.
- Node — a single function or step in the graph, typically a model call or a tool call.
- Edge — a connection between nodes that determines what runs next; fixed or conditional.
- Conditional routing — choosing the next node dynamically based on the current state.
- Human-in-the-loop — a design where a person can inspect or modify an agent's state mid-workflow before it continues.
- Durable execution — a system continuing correctly after failure by resuming from saved progress rather than restarting.
- Thread ID — a unique identifier used to key a particular workflow run's saved state.
- Interrupt/Command primitives — LangGraph's mechanisms for pausing a running graph and controlling its resumption or state edits.
- Crews (CrewAI) — teams of role-based AI agents collaborating autonomously on a task.
- Flows (CrewAI) — CrewAI's event-driven, stateful workflow mechanism offering branching and checkpointing alongside Crews.
- TypedDict — a Python typing construct declaring a dictionary's field shape, used to define graph state structure.
- LangSmith — a LangChain tool for tracing and visualising agent execution, including state transitions and runtime metrics.
Sources
- langchain-ai/langgraph — GitHub, 2026
- LangGraph Overview — LangChain
- Multi-Agent Orchestration: LangChain vs CrewAI vs AutoGen — Inductivee
- GitHub - crewAIInc/crewAI — GitHub, 2026
- LangGraph in Production: StateGraph, set_entry_point, and the Patterns We Use for Real Agents — Kalvium Labs
- LangGraph Production Architecture: Scaling AI Agents — Markaicode
- LangGraph 2.0: Stateful Multi-Agent Orchestration Framework — Jose Nobile, 2026