Research newsletter
Who Can Stop an AI Decision Before It Costs You?
In a simulation of four workflows, expected annual cash loss more than doubled under junior-led AI. A package with stronger review and tighter execution controls brought expected loss below the human baseline.
An unchecked claim can become a spending decision
Imagine that an analyst uses AI to size a new market. The estimate goes into a strategy memo. A manager approves the budget, and an agent starts preparing orders. By the time someone checks whether the source counted paying customers or everyone who expressed interest, the company may have committed money.
Put the source check before the first spending commitment. Give the reviewer authority to hold the order while they compare the estimate with the original market data.
What we put in the simulation
The model covers research briefs, strategy proposals, investment approvals and agent actions. At 50% adoption, half of existing decision opportunities go to AI. An autonomy setting of 0.75 on a 0–1 scale controls how much faster AI can act and how widely an error can spread.
Each oversight case includes 250,000 simulated years. The loss rates, review assumptions and dollar figures describe a constructed portfolio. Use the model to test choices with data from your own operation.
What changed when experts could intervene
Only the full expert package brought expected loss below the human baseline. It assumes fewer harmful errors before review, more relevant checks before action and stronger controls after an error gets through.
Expected loss in the expert case rose to US$447,000 when the model halved both review capacity and the relevance of the checks and doubled the frequency of shared failures. In Table 4 of its six-function preprint Decision speed, autonomy and organisational loss: a simulation of AI integration, Maestro AI Labs reported that expert oversight came in 3.2% above the human baseline under different assumptions.
An independent meta-analysis by Michelle Vaccaro and colleagues found that, on average, human–AI teams scored below the stronger of people or AI working alone. The researchers reported “performance losses in tasks that involved making decisions”. A reviewer needs to know which assumption could change the decision.
| Oversight | Expected annual loss | Annual P99 |
|---|---|---|
| Human baseline | US$431,000 | US$1.42m |
| Junior-led AI | US$874,000 | US$2.76m |
| Partial expert review | US$647,000 | US$2.11m |
| Expert orchestration | US$317,000 | US$1.16m |
Price the hours before you delegate more
Upgrading junior-led AI to the expert package avoided about US$557,000 in expected annual loss. It required 1,461 more review hours and cost about US$191,000 more a year, leaving US$366,000 in net annual risk-reduction value in this scenario.
Try it on one workflow
Pick a decision that releases money, changes a live system or makes a promise to a customer. Write down the evidence the reviewer must see, the exposure limit and the person who can stop the action. Introduce a plausible error in a test case and measure whether the reviewer catches it before the commitment. Track what happens when the reviewer is unavailable.
Model assumptions and sources
For each workflow, estimate how many decisions the team attempts, how many harmful errors get past review and what each escaped error costs. The simulation combines routine incidents with the possibility of one shared failure, such as several teams relying on the same data. It reports cash losses from disruption, direct financial damage and downstream effects such as lost contribution from customer departures.
The four-workflow model is a new scenario analysis. The two linked Maestro AI Labs preprints share one simulator. Pages 5–6 of The economic value of expert orchestration under AI integration explain how the authors price review and controls in their six-function model. Read the peer-reviewed human–AI meta-analysis.