We are in the part of the cycle where the word "agent" has outrun the thing. Almost every company building on top of language models now describes its product as agentic. The demos are genuinely impressive: a model that books the trip, files the report, navigates the website, writes and runs the code. And yet I'll make a prediction that I think will age well — most of these systems will fail, not because the models aren't good enough, but because they're built on a misunderstanding of what makes something an agent in the first place.
The misunderstanding is this: people believe that autonomy is what you get when you chain enough model calls together and give the chain some tools. It isn't. Chaining language models produces something that looks autonomous in a demo and is brittle everywhere else. Real autonomy is a property of a system — one with feedback, grounding, a model of its world, and hard constraints on what it can do — and almost none of that lives inside the language model. This essay is about the gap between those two things, and why it's the gap that will decide who survives.
A chain is not an agent
Start with the dominant mental model, because it's the source of the trouble. The intuitive picture of an agent is a loop: the model thinks, calls a tool, sees the result, thinks again. Wrap that in a prompt that says "you are an autonomous agent," give it access to a few APIs, and you have something that will, for a while, do remarkable things.
The problem is structural, and it's the same problem I've written about elsewhere: the reliability of a chain is the product of the reliability of its steps. If each step works 95% of the time — already optimistic for an open-ended task — then ten steps in sequence succeed barely more than half the time, and twenty steps is a coin flip you keep losing. This isn't a flaw you prompt your way out of. It's arithmetic.
What makes this seductive — and dangerous — is that the failures are invisible at demo length. A three-step task in a controlled setting works beautifully. A twelve-step task against real systems, run a thousand times, fails in ways the builder never saw coming. The demo and the deployment are different regimes, and the chain abstraction only survives in the first one. Most "agent" companies are, right now, mistaking the demo regime for the product.
The four things the model doesn't have
If chaining isn't enough, what's missing? Four things, none of which the language model provides, and all of which have to be built around it.
Feedback. A real agent observes the consequences of its actions and adjusts. Most "agents" act and move on, with no mechanism to notice that an action didn't produce the intended effect, and no way to recover when it didn't. Without a feedback loop, you don't have an agent — you have an open-loop script that happens to be written by a model at runtime. The intelligence people attribute to the model is often really the intelligence of a loop that isn't there.
Grounding. The model generates fluent, plausible text. Plausible is not the same as true, and in an agent that acts, a plausible-but-wrong belief becomes a wrong action in the world. A real agent checks its beliefs against ground truth — a database, a tool result, a constraint — before committing. Ungrounded agents are the ones that confidently do the wrong thing, because nothing in the loop ever told them they were wrong.
A model of the world. To act well, a system needs some representation of the state of the world it's acting in: what's true now, what its actions will change, what's still pending. Language models are extraordinary at language and notoriously shaky at maintaining a consistent model of a changing situation across many steps. An agent that loses track of what it has already done, or what's true right now, will eventually act on a world that no longer exists.
Action constraints. An autonomous system that can act in the real world needs hard limits on what it's allowed to do — not suggestions in a prompt, but constraints in the architecture. The difference matters enormously. A prompt that says "don't do X" is a request the model can be talked, confused, or hallucinate its way past. A constraint that makes X unreachable is a guarantee. Systems that put their safety in prompts are systems whose safety is optional.
Autonomy is a property of the loop, not the model
Notice what those four missing pieces have in common: they're all about the system around the model, and they're all about closing a loop with the world. Feedback closes the loop. Grounding tethers the loop to reality. A world model is what the loop carries forward. Constraints bound what the loop is allowed to do.
This is why I keep insisting that the closed loop is the unit of analysis, not the model call. A model call is a function: input, output, done. An agent is a loop: it senses the state of the world, represents it, decides, acts, and then the world changes and it senses again. The hard engineering — and the actual autonomy — lives in how those steps close on each other, in how the system behaves when a step is wrong, in whether it can recover. The model is a single capable step inside that loop. A better model gives you a better step. It does not give you the loop.
The companies that internalize this build something fundamentally different from the ones that don't. The chain-thinkers spend their effort on prompts and on waiting for the next model release. The loop-thinkers spend their effort on verification, state, recovery, and constraints — and find that their systems get more reliable even when the model underneath stays the same. Over time those are not small differences in quality. They're the difference between a system that works and one that doesn't.
Why "better models will fix it" is a trap
The most common objection to all of this is that it's temporary — that the next generation of models will be reliable enough that you won't need the scaffolding. I think this is the single most expensive mistake a team in this space can make, for two reasons.
First, better models raise the reliability of a step, but the workflows worth automating are getting longer and more consequential at least as fast as the models are getting better. A 99%-reliable step is wonderful until you chain fifty of them, and the tasks people actually want agents for are not three steps long. The compounding math doesn't go away; it just moves to a longer chain. You will always be able to construct a task long enough to break an unscaffolded agent, and those are exactly the tasks with the most value.
Second — and this is the deeper point — feedback, grounding, world-state, and constraints are not deficiencies of current models that better models will patch. They are different kinds of things. A language model, however good, is still a function from context to text. Closing a loop with the world, checking a belief against an external source of truth, maintaining durable state across hours, enforcing a hard limit on actions — none of these are things a better next-token predictor becomes. They're properties of a system you build. Waiting for the model to provide them is waiting for a category error to resolve itself.
What this looks like when the stakes are real
It helps to ground this in a domain where the difference between a chain and an agent isn't academic. Take a healthcare workflow — the kind of multi-step, consequential task where an agent might gather information across systems, coordinate with people, make decisions, and act. This is exactly the kind of work the chain abstraction promises and the chain abstraction can't deliver.
Walk through where the naive version breaks. The agent retrieves a piece of information and treats it as true without checking it against an authoritative source — a grounding failure, and now a wrong belief is propagating toward an action with consequences. It takes a step that has a real-world side effect, then the process hiccups and retries, and the side effect happens twice — a recovery failure that no prompt prevents. It runs for long enough that it loses track of what it already did — a world-model failure. And at no point is there a hard constraint that makes the genuinely dangerous actions simply unreachable; there's only a prompt politely asking the model not to, which is not a safety property at all.
Every one of these failures is invisible in the demo, because the demo is short, clean, and supervised. Every one of them is fatal in deployment, because deployment is long, messy, and unattended. And not one of them is fixed by a better model — they're fixed by the verification, the idempotency, the durable state, and the hard constraints that the chain abstraction simply doesn't have. The high-stakes domain is where the gap between looking autonomous and being autonomous stops being a philosophical distinction and starts being the difference between a system you can deploy and one you can't.
What the survivors will look like
If most agentic systems will fail, the interesting question is what the survivors have in common. From where I sit, building agents in a domain where being wrong has real consequences, the pattern is fairly clear.
The survivors will treat the model as a component, not the product. Their moat won't be a clever prompt — prompts are copied in an afternoon — but the orchestration around the model: the verification layer that catches plausible-wrong outputs, the durable state that survives a crash, the recovery logic that resumes safely, the constraints that make dangerous actions unreachable, the human-in-the-loop handoffs that fire at exactly the right moment. This is unglamorous systems engineering, and that's precisely why it's defensible. It's hard, it's specific to the domain, and it doesn't fall out of the next model release.
The survivors will also pick domains where reliability is the whole game, because that's where the scaffolding is worth building and where the gap between a real agent and a chain-with-a-prompt is most visible. In low-stakes settings, a flaky agent is a curiosity you tolerate. In high-stakes settings, a flaky agent is unusable, and the only systems that work are the ones built correctly. Counterintuitively, the harder, more unforgiving domains are the better places to build, because they force you to build the real thing and they reward you for it.
And the survivors will be honest about the loop. They'll know what their system senses, how it grounds its beliefs, how it carries state, how it behaves when it's wrong, where the human sits, and how feedback actually changes future behavior. Those are loop-shaped questions, and a team that can answer them crisply is a team that has built an agent. A team that answers every one of them with "the model handles it" has built a demo.
The hype will burn off, as it always does. What it leaves behind will not be the companies with the best prompts or the most impressive launch videos. It'll be the ones that understood, early, that they were never really in the business of chaining model calls — they were in the business of building systems that can act in the world and be trusted to. That's a much harder business. It's also the only one that was ever real.