The Demo That Dies on Day Two
It is still busy. It still produces output. Somewhere around hour three it quietly stopped doing the work it was given, and nothing in the stack noticed — because nothing in the stack was watching for that. Everyone was watching for errors. There weren’t any.
I’ll come back to that, because it’s the failure mode that defines the whole category. But it starts, as these things do, with a demo that goes well.
So picture one. A KYC periodic-review agent — I’ll use it as the worked example throughout, because it has every property that makes long-horizon work hard, and you can substitute your own without losing anything. Feed it a corporate client and it pulls the entity record, chases down the ultimate beneficial owners, checks the screening hits, notices the expired registration document, drafts the outreach, and hands an analyst a tidy package. Fifteen minutes for work that used to take half a day. The room applauds. Somebody says “transformational,” which is usually the moment to start looking for the exit.
Now point it at a real remediation backlog. Not one client — thousands, across a program that runs for months.
It comes apart in ways the demo could never have shown. A case stalls for nine days waiting on a document from the client, and by the time the document arrives the agent has no idea why it asked. Two runs work the same entity from different queues and file contradictory conclusions. A run crashes on a rate limit forty steps into a sixty-step case and starts again from step one, re-sending outreach the client already received. And then the one I opened with: on long cases the agent slowly loses its hold on what it was trying to establish. It never announces this. It never errors. It drifts off the job and keeps working, fluently, on something adjacent to it.
The scenario is illustrative — assembled so that every failure sits in one place, which no single program would be unlucky enough to manage. The failures are not.
None of them is a model problem. Every one is an architecture problem, and it is the same architecture problem each time: an agent whose entire existence is one conversation, asked to do work that lasts a season.
That gap has a name now. Long-horizon agents.
What “Long Horizon” Actually Means
The term gets used loosely, usually as a synonym for “ambitious.” Let me make it precise, because the precision is what makes it buildable.
A long-horizon agent is one whose task outlives the thing it’s thinking with.
Three ways a task can outlive it, and most real work does all three:
- It outlives the context window. The work generates more evidence, more tool output, more intermediate reasoning than fits. The transcript can no longer be the memory.
- It outlives the session. The work pauses — for a human approval, a batch window, a client response, a market close — and resumes hours or weeks later, in a different process, possibly on a different model version.
- It outlives the human’s attention. Nobody is watching turn by turn. The supervision has to be structural, not conversational.
Notice what’s not in that definition: autonomy level, number of tools, whether it’s “multi-agent.” Those are implementation choices. The horizon is a property of the task, and it’s the property that dictates everything else.
Where the Regular Agent Model Quietly Stops Working
Most agent frameworks are built around a loop: get a request, reason, call tools, observe, repeat until done, return an answer. It’s a good loop. I’ve shipped a lot of it. Its assumptions are just very specific, and every one of them breaks at horizon.
| Request-scoped agent | Long-horizon agent | |
|---|---|---|
| State lives in | the transcript | an external, queryable record |
| Memory strategy | keep everything | curate a working set |
| Failure mode | it crashes, you retry | it drifts, and keeps going |
| Recovery | start over — it’s cheap | resume — starting over is unaffordable and often unsafe |
| Side effects | few, small, forgivable | real money, real letters, real records |
| Correctness check | the human reading the answer | a verifier that isn’t the agent |
| Supervision | every turn | by exception, against a contract |
| The dangerous outcome | a wrong answer | a plausible answer nobody can reconstruct |
That last row is the whole discipline in one line. A short agent that goes wrong gives you a bad answer and you notice. A long agent that goes wrong gives you ninety percent of a correct case file, assembled over three weeks, with the wrong beneficial owner in it, and a reasoning trail that got compacted away in week one. It fails the way a bad employee fails, not the way a bad script fails — and you can’t debug it by reading a stack trace.
The three specific killers, in my experience:
Context rot. As the context grows, model performance degrades — measurably, across every frontier model tested, and as a function of length itself rather than task difficulty. It is not that the instruction falls out of the window, and it is not simply that recent tokens win: attention is biased toward both ends of the context, so the opening instruction is one of the better-protected positions. What degrades is everything buried in the middle, and, separately, the grip the goal keeps on the next action once it is competing with a hundred thousand tokens of accumulated evidence. The agent doesn’t lose the goal. It stops being governed by it. That is why a long run wanders.
Goal decay through summarization. Compaction is necessary; it’s also lossy in a directional way. Summaries preserve narrative and drop constraints. Do it four times and the hard requirement from step two — the source of truth for legal entity name is the registry, not the CRM — has evaporated, and the agent is confidently using the CRM.
Unrepeatable side effects. In a chat agent, retry is free. In an agent that emails clients, posts journal entries, opens tickets, or files reports, “just run it again” is how you send the same regulator two contradictory submissions.
Seven Principles I’d Now Refuse to Build Without
These aren’t theory. Each one is scar tissue.
1. The task is the state. The transcript is a cache. Every long-horizon agent needs an external task record — goal, constraints, decisions made and why, evidence gathered with provenance, open questions, current step, budget consumed. It is written to durable storage, it is the authority, and the agent’s context is rebuilt from it rather than accumulated. When I got this wrong, the system was a chat log with delusions of durability. When I got it right, I could kill any run mid-flight and lose nothing.
2. Context is a working set, not a memory. Deliberately decide what gets loaded for each step. The goal and hard constraints are pinned — always re-injected verbatim, never summarized. Everything else is retrieved on demand from the task record. Compaction should be structured — update fields in the record — not prose-summarize-the-conversation. The distinction between “pinned” and “summarizable” is one of the highest-leverage design decisions in the whole system, and almost nobody makes it explicitly.
3. Every step is resumable and idempotent. Borrow this wholesale from durable workflow engines; it’s a solved problem in a neighboring field. Each step has an ID, its result is checkpointed, and re-executing a completed step is a no-op that returns the stored result. The test: pull the plug at any moment and restart. If the system can’t continue from where it stood, you don’t have a long-horizon agent — you have a long-running one, which is a completely different and much sadder thing.
4. Classify actions by reversibility, not by risk score. Three buckets, and they drive everything: reversible (read a record, draft a document) — let it run; compensable (open a ticket, place a hold) — let it run, but the compensating action must exist and be tested; irreversible (send the client letter, submit the filing, move the money) — gated, always, and gated at the action, not at the plan. Plans get approved and then mutate. Actions don’t lie.
5. The verifier cannot be the actor. An agent asked to check its own work will tell you it did the work. Progress has to be certified by something with independent standing: a deterministic rule set, a separate model with only the goal and the artifact (not the reasoning that produced it), a schema, a reconciliation against the source system. In banking this is easy to justify — it is maker-checker, which we have run for generations. Everywhere else you’ll have to argue for it. Argue for it.
6. Budget the horizon explicitly. Wall-clock time, token spend, tool calls, retries, and blast radius all get ceilings, declared up front, decremented, and enforced. An agent without a budget doesn’t fail — it idles expensively, which takes far longer to discover. Every long-horizon system I’ve built has eventually needed a “you have used 60% of your budget and completed 20% of your plan, stop and escalate” rule. Build it on day one.
7. Assume the world moved while you weren’t looking. This is the principle people miss, and it’s the one that connects to everything else I write about. A long-horizon agent’s beliefs about the enterprise are snapshots. Over a three-week task, the schema changes, the policy is updated, the client’s status flips, the rate is repriced. An agent that can’t be told what changed since it last looked doesn’t fail loudly — it confabulates with institutional confidence. Long-horizon agency is a change-propagation problem wearing a different hat, and the organizations that already invested in a living, versioned map of their estate are about to find out they accidentally built the foundation for this.
A Reference Architecture
Eight components. Deliberately boring — the value is in the arrangement, not the novelty.
1. Goal contract. A structured, machine-readable statement of the objective, success criteria, hard constraints, authority limits, budget, and escalation triggers. Written once, versioned, never summarized away. If a human can’t read this and say “yes, that’s the job,” you’re not ready to start.
2. Task ledger. The durable state: plan, step status, decisions with rationale, evidence with provenance and timestamps, open questions, budget consumed. Append-mostly, queryable, bitemporal if you can manage it. This is the artifact that survives everything.
3. Durable orchestrator. Checkpointing, retries with backoff, resumption after crash, timers for long waits, and the ability to park a task for two weeks without holding a process open. Use a real workflow engine. This is not where you innovate.
4. Context assembler. The policy layer that builds the prompt for each step from the ledger: pinned constraints, current step, retrieved relevant evidence, and nothing else. Treat it as a component with its own tests, because it is the single biggest lever on both quality and cost.
5. Effect-typed tool layer. Every tool declares its reversibility class, its compensating action, its idempotency key, and its authority requirement. The runtime enforces the gates. The agent doesn’t get to decide what’s dangerous.
6. Verifier. Independent checks at step and milestone level: deterministic validators where the rules are crisp, an LLM critic with a narrow mandate where they aren’t, reconciliation against systems of record where possible. Verification failures are first-class events that can roll a step back, not warnings in a log.
7. Supervisor and escalation channel. Humans enter on exception, not on schedule: verifier failure, budget threshold, irreversible action, novel situation outside the contract. The escalation must carry enough context that a person can decide in two minutes. An escalation that requires reading a transcript is an escalation that gets rubber-stamped.
8. Trace and audit. Every decision, every input, every tool call, every human intervention — reconstructable months later. In a regulated industry this is what makes the whole thing deployable rather than a science project. Build it first, not after the audit request arrives.
Where This Lands, Industry by Industry
Banking, because it’s where the horizon is longest and the tolerance is lowest.
KYC and periodic review remediation. The canonical case: multi-week, multi-party, evidence-heavy, interruption-dominated, and audited. The agent’s job isn’t to decide — it’s to assemble a defensible file, keep it current as documents trickle in, and escalate exactly when a human judgment is required. Value is in the assembly, not the verdict.
Credit and underwriting file preparation. Spreads, covenant extraction, comparables, site-visit notes, exception memos — assembled over weeks against a policy contract, with every figure traceable to a source document. The horizon here is defined by how long it takes counterparties to send things.
Regulatory change to control mapping. A rule changes. Something has to trace it to affected policies, processes, controls, models, and reports, then draft the change requests and track them to closure across quarters. This is a long-horizon agent whose entire product is impact propagation.
Core and data platform modernization. Understanding a decades-old system, proposing a migration slice, generating and testing it, and doing that for years. The horizon is the program.
Dispute and fraud case handling. Regulated clocks, third-party evidence, provisional credits with compensating actions. A near-perfect fit for effect-typed tooling.
Insurance: complex claims from first notice to settlement — reserve updates, adjuster reports, medical records arriving over months. Healthcare: prior authorization and denial appeals, where the task is inherently a multi-week correspondence with an external party. Pharma: regulatory submission assembly, measured in quarters. Manufacturing and supply chain: supplier qualification, or a multi-week disruption response with real-world side effects that cost money to reverse. Telecom: network change programs across thousands of sites. Legal: discovery review and matter management, where the case file is the task ledger. Public sector: benefits and permit adjudication, where the audit trail is the deliverable.
The pattern behind all of them: the work is long because it waits on the world, not because the model is slow. Long-horizon architecture is fundamentally the engineering of waiting well.
What To Do in the Next Ninety Days
Don’t start with the ambitious one.
Weeks 1–4. Pick one workflow that already takes humans days or weeks and is currently done badly by a queue. Write the goal contract by hand, in a document, with the business owner. If you can’t get agreement on success criteria and authority limits on paper, the agent will not resolve that disagreement for you — it will just automate it.
Weeks 5–8. Build the ledger and the durable orchestrator before you build anything clever. Run the workflow with a deliberately dumb agent — few tools, everything gated, humans doing most steps. You’re testing the skeleton: can it pause for nine days, resume, and know exactly where it stood?
Weeks 9–12. Add the verifier, then start moving actions from “gated” to “reversible” one at a time, based on measured verifier pass rates. Autonomy is earned per action, with evidence. Anyone who wants to grant it per agent is describing a future incident.
Then expand along the horizon, not the headcount. The instinct to spin up a fleet of specialized agents is strong and mostly wrong at this stage; one agent that can reliably survive three weeks is worth ten that impress for thirty minutes.
The Real Test
We spent the last two years learning to make agents that are impressive in a conversation. The work the enterprise actually runs on — the remediation program, the migration, the claim, the case, the filing — was never a conversation. It’s a thing that starts on a Monday, waits on eleven other people, gets interrupted twice, changes shape in the middle, and has to be defensible a year later in front of somebody who wasn’t there.
Can it be interrupted on Tuesday and still be right on Thursday?
That is the question I’d now ask about any agent before it goes anywhere near production — not whether it can do the task. If the answer depends on nobody restarting the process, nobody changing the underlying data, and nobody asking how it reached its conclusion, you don’t have an agent. You have a very expensive demo. Most of us in this field have given one. The point of writing all this down is that the next one doesn’t have to be.
This is the operational half of an argument I’ve been making for a while: agents need a stable semantic backbone to reason over and a nervous system that tells them what moved. See The Knowledge Spine for the anatomy, and Your Data Strategy Is Already Dead for the nerves.