When the loop should be a graph

Choice is a cost as well as a feature

A free-running loop gives the model a decision at every step. That is exactly right when nobody knows the path in advance, and exactly wrong when everybody does. A refund process has been the same five steps for eleven years. Asking a model to rediscover them forty times a day is paying for creativity in a place where creativity is a defect.

The alternative is a state machine: named states, defined transitions, and a model call only where a genuine judgement is required.

verify_order ──> check_policy ──> decide ──┬──> issue_refund ──> notify
                                            ├──> escalate ─────> notify
                                            └──> reject ───────> notify

verify_order is a database lookup, no model. check_policy reads the policy text against the order and is a model call. decide is a model call returning one of three enum values. issue_refund is code with a limit. notify is a template. One agent turned into three model calls and four functions.

The containment argument

The strongest reason is not tidiness, it is tool exposure.

In a free loop, every tool is available on every turn. The refund tool sits in the tool list while the agent is reading an email, and the only thing standing between an injected instruction and a payment is an instruction in the prompt asking it not to.

In a state machine, each state carries its own tool list. The notify state has no refund tool, not a forbidden one, an absent one. The check_policy state can read and cannot write. A prompt injection that arrives while reading the customer’s message lands in a state where the dangerous capability does not exist, and the class of attack does not apply there. This is the difference between a rule and a boundary, and it is available to you for the price of splitting one prompt into three.

The other benefits, briefly

• Enumerable paths. Seven states and eleven transitions can be listed, and each tested. A twenty-turn free loop has no path list at all.
• Resumability. A run that dies in check_policy restarts at check_policy, because the state is a value you stored, not a position in a transcript.
• Cost. Each state gets a short prompt and only the context it needs. Three short calls beat one twenty-turn transcript, and the arithmetic from the context-window lesson makes that difference large.
• Small prompts. Each state’s instruction is a paragraph about one decision, which is far easier to write, test and ablate than a page trying to govern everything at once.

Keep loops where they belong

Inside a state, when the work is exploratory (searching until something is found, retrying a flaky lookup, reading files until a question is answered) use a loop with its own budget. The graph is the outer structure; loops live inside individual nodes. Most good agent systems are shaped like this rather than being purely one or the other.

What to build it with

A dictionary of handler functions and while state != "done" is a complete implementation and stays debuggable at 3am:

state, ctx = "verify_order", {"order_id": order_id}
while state != "done":
    state, ctx = HANDLERS[state](ctx)
    log(run_id, state, ctx)

LangGraph gives you this with persistence and streaming attached. Temporal gives durable execution, meaning a workflow survives the machine dying mid-step, which matters when a state takes ten minutes and calls a payment API. Both are real tools with real costs: a framework you cannot read is a debugging problem you have deferred, not avoided. Start with the dictionary; move when you can name what you need.

The limitation

A state machine encodes a process you already understand. If nobody can draw the process, you cannot draw the graph, and forcing one prematurely produces a rigid system that handles the twelve cases somebody imagined and breaks on the thirteenth.

So the sequence that works is: prototype with a free loop, run it on real inputs, then read fifty traces and extract the graph the traces already show you. The states will be obvious by then, and so will the two transitions you would never have predicted. Building the loop first is not wasted work; it is how the map gets drawn.