AI Agents Have Left the Chat

Jan Collaert I, New Inventions of Modern Times [Nova Reperta], The Invention of the Clockwork, plate 5, ca. 1600. The Metropolitan Museum of Art, public domain.

If you run a finance team or a legal desk, the number that matters in this release is how often an AI system finishes work that nobody is watching it do. Anthropic shipped Claude Fable 5.1 on 1 September 2026, and its headline figure is 52.6 percent on Terminal-Bench-Science 0.1, an agentic benchmark of 70 tasks that ask a model to work in a terminal, the text window programmers type commands into: read a data set, run the computations, submit a result. Fable 5, three months earlier, scored 24.7 percent on the same tasks, and Opus 5 scored 29.0. On Terminal-Bench 4.0 the jump was 42.0 to 55.8 percent.

The line my CFO clients should read twice is about price. Headline pricing stayed at 10 dollars per million input tokens (a token is a fragment of a word, the unit a model reads and writes) and 50 per million output. Cache reads dropped 75 percent, from one dollar to 25 cents per million. An agent re-reads the same context over and over as it works through a task, so cached input dominates its bill. Anthropic cut the price of the usage pattern agents produce, which tells you what it expects customers to run.

How an agent differs from the chat tool your staff already use

For most firms, AI over the last two years has meant an employee typing a question into a browser tab and reading what comes back. The employee decides what to ask, judges the answer and carries it into the real work. Everything the model does is bounded by one person's attention.

An agent is handed an assignment. It plans the steps, picks its tools and works until it has something to hand back: a research report, a reconciliation, a first-pass code change, an operational review. The person writes the brief and reads the result. The middle, where most of the labour used to sit, now happens without them, and that changes both the cost of the work and who answers for it.

What a 52.6 percent score tells a board

A frontier agent now completes about half of a bounded technical workload without help, up from a quarter three months ago. It still failed 47.4 percent of the tasks. Anthropic reports an error margin of roughly 3.5 to 4.5 percentage points on Terminal-Bench-Science 0.1, so two nearby scores on that benchmark often mean the same thing.

Half is a large step up from a quarter, and it is nowhere near enough to remove review. Boards tend to hear whichever half of that sentence their vendor is selling. My rule for reading these releases: use the rate of improvement to decide when to pilot, and use the failure rate to decide how much output a person still checks.

Vendors can withdraw an agent without notice

While Anthropic was tuning a model for long-running autonomy, OpenAI was withdrawing the equivalent capability from its consumer product. Operator, the original browser agent, was folded into ChatGPT agent in July 2025 and shut down on 31 August 2025. Agent mode was removed from ChatGPT in early August 2026, with no advance deprecation notice and no migration guide, and the Atlas browser shut down on 9 August 2026. Users were pointed to ChatGPT Work and a separate cloud browser feature. Neither drives a logged-in browser session and hands control back to a person at the login and payment steps, which is what agent mode did.

I read that as a large company consolidating a product line, and I do not think OpenAI is leaving the category. It still shows a risk that agentic deployment carries and chat deployment mostly did not. Build a workflow on an agent that drives a browser, lose the agent, and you have a broken process and a team that no longer remembers the manual steps. Replacing a chat tool took an afternoon.

So the second question in any agent deployment, right after what it does, is what happens to the work if the capability disappears in 90 days. Before 9 August most boards treated that as a formality.

What working deployments have in common

The deployments I have been closest to this year are unglamorous and share a shape. Somebody took a task already defined tightly enough to write down, moved it into the overnight window, and put a person on the front of it in the morning.

The clearest case is a contract review queue. Agents draft first-pass responses to standard commercial contracts overnight, each redline carrying its reasoning, and the associates arrive to a queue instead of a pile. Turnaround on standard agreements went from days to inside a working day. The firm could do this because it already had a written house position on the twenty clauses that come up repeatedly, so the agent had a standard to meet and the associate had a standard to check against. Firms without that document got a queue of confident, well-formatted drafts that nobody could review quickly.

Reconciliation has moved the same way, from teams running queries and phoning branches to a nightly run and a morning exception list. So has overnight preparation of analytical drafts, and first-pass triage of flagged transactions before a compliance officer starts the day. In each case the first hour now starts with a decision instead of data collection.

I do not have clean before-and-after productivity numbers I would defend in front of a CFO for any of these, and nobody else does this early either. What I can say is which deployments held up over six months and which got switched off. The difference was almost never model quality. It was whether a written standard and a named owner existed.

Who owns an agent's output

A junior associate who misreads a clause makes that mistake once, and a partner catches it. An agent that misreads the same clause applies the identical reasoning to all forty contracts in the queue. The error has no variation to make it visible, and the output is well formatted whether the thinking was sound or not.

Ownership therefore comes down to four specific duties, all held by one named person:

• setting the written standard the agent's output is reviewed against;
• deciding what share of output stays sampled once the agent is trusted;
• watching that share, and the error rate inside it, month by month;
• switching the agent off without waiting for a committee.

In a mid-sized firm that is usually the COO or CTO wearing another hat. In a larger one it is an AI Officer with the role in writing. If you cannot name the person, the deployment is not ready, whatever the benchmark said. Boards that have not answered this carry an exposure that appears nowhere in their current reporting, because nothing has gone wrong yet.

Where the gap between firms will come from

Within a year, running agents on defined recurring processes will be ordinary in firms that compete on cost, and firms that have not started will be about a year behind on unit economics. Every serious vendor will have a competent agent, and the models have converged every year since 2023. Picking the right model will matter less than the preparation: knowing which processes are defined tightly enough to hand over, writing the standard that makes output checkable, and naming the person who answers for it. None of that needs a licence, and most of it can be done before you choose a vendor.

The weak point in my own argument is the evidence base. Six months of deployments in a handful of firms is not a study, and the benchmark tasks are technical ones that may not predict performance on a Caribbean insurer's claims file.

What a board should do this month

1. The CEO picks one hard recurring task and runs it twice. Give a frontier agent a real brief with a real deadline, run it in parallel with the person who normally does the work, and have both results read by the same reviewer. Budget one week.
2. The board names one owner before the second run. Minute the name and the four duties above. If no one will accept them, stop the pilot there.
3. The owner writes the review standard. For contracts that is the house position on your recurring clauses; for reconciliations it is the tolerance that triggers an exception. Keep it to two pages so reviewers use it.
4. The owner sets a sampling rate and keeps it. Start by checking every output, and only reduce the share once a month of results shows an error rate the owner will sign for in writing.
5. The COO writes the withdrawal plan. List which workflows break if the vendor removes the capability, how long the manual fallback takes, and who still knows how to run it. Test that fallback once a quarter.