The thing standing between you and an agent that actually acts on your behalf, buys, books, executes, without a human hovering over every commit, is not a smarter model. It is the absence of a market that will price the agent's risk and put a solvent third party behind it. An autonomous agent bears no consequence for being wrong, so someone exposed to the downside will not let it act unsupervised. Commerce has faced this exact problem for four centuries and always solved it the same way: not by making the untrustworthy party trustworthy, but by pricing its unreliability and having a well-capitalized stranger stand behind the loss. The agent economy is waiting on that market to form, and its arrival, not the next benchmark, is the leading indicator of real autonomy.
I want to be precise about the claim, because "AI needs insurance" is the kind of sentence that sounds either obvious or glib. The claim is narrower and load-bearing: the binding constraint on delegating consequential actions to agents is a missing price. We have no honest, continuously updated number that says how much to trust a given agent doing a given task. Until that number exists and someone will sell you protection at that price, autonomy stays capped at whatever a single nervous human is personally willing to eat.
The problem is structural, and capability does not touch it
I have argued the underlying constraint at length: your AI agent has no skin in the game, and that, not raw capability, is the real ceiling on autonomy. A human professional you delegate to is bonded, licensed, insurable, suable. The consequence of being wrong lands on them, which is the specific engineering that makes it rational to trust a stranger with something expensive. An agent has no bond to forfeit, no license to lose, no assets to attach. You cannot align an incentive that does not exist, and you cannot deter a thing that has nothing to protect.
So the exposure relocates onto the nearest human with standing, and that human, sensibly, refuses to grant autonomy proportional to a downside they cannot inspect or cap. This is why "human in the loop" persists even as models get better. It is not a capability crutch. It is the only risk-management instrument currently available, and it is a terrible one: it does not scale, it burns the scarcest resource in the building, and it puts the reviewer on the hook for decisions they did not fully make.
Here is the move the whole essay turns on. "Put a human on the hook" is one way to handle delegation to an unaccountable party. It is not the way commerce actually settled on for the general case. Commerce settled on pricing the risk and transferring it to someone who chose to hold it.
Every mechanism for trusting a party with no skin in the game already exists
Long before AI, the economy was full of necessary delegations to parties whose good faith you could not assume and whose downside did not naturally land on them. The instruments invented to make those delegations safe are precisely the instruments an autonomous agent needs, because an agent recreates the exact risk they were built for.
- Surety and performance bonds. A contractor can take your deposit and vanish, or botch the roof and walk. So construction runs on bonds: a surety company guarantees the contractor's performance and pays out if he fails, having underwritten him first. You trust the contractor because a solvent third party has staked its own capital on his reliability.
- Professional indemnity / errors-and-omissions insurance. A lawyer, an accountant, an architect will occasionally be wrong in a way that costs you a great deal. E&O insurance means there is a pool that makes you whole, and a premium that reflects how error-prone that professional's practice is.
- Escrow and letters of credit. When a buyer and seller who do not trust each other must transact, a bank holds the value and releases it only on verified performance. The bank has no stake in the underlying deal; it sells the standing-in-the-middle as a service.
- Fidelity bonds cover employees who handle money, insuring the employer against the employee's dishonesty, an agent-with-your-credentials problem in its purest form.
None of these makes the delegated party accountable in the moral sense. The contractor still has no real skin in the game beyond his bond. What they do is convert an unbounded, uninspectable trust problem into a bounded, priced, transferable one, and interpose a balance sheet that can actually pay. Marine insurance was born this way in Edward Lloyd's London coffeehouse in the 1680s: shipowners faced catastrophic, uncontrollable loss, and a market formed where underwriters literally wrote their names under a risk and the price they would carry it for. An autonomous agent booking travel, moving money, or committing you to a purchase is a cargo ship leaving harbor. The infrastructure for that is not new. It is unbuilt for this asset class.
What the underwriter prices, and why the premium is the signal we lack
Forecast, labeled as such: agent-liability underwriting will price three things, and building your agent so those three things are favorable will be the difference between "deployable with autonomy" and "stuck behind a human."
One, the agent's track record. Frequency and severity of loss-producing actions over a real deployment history, tied to a persistent identity. This is ordinary actuarial input, and it is exactly what a cash-on-delivery seller's delivery history provides in the trust-scarce markets I build for.
Two, the reversibility and blast radius of the action. A reversible action (a draft, a cancellable hold, a staged change behind a confirmation) has near-zero severity no matter how often it goes wrong. An irreversible one (a wire, a legally binding order, a deletion) carries full severity. Underwriters price severity times frequency, so the architecture of the action, not just the model's accuracy, sets the premium.
Three, the verification wrapping around the agent. Sandboxing, permission scoping, human-confirmation gates on high-severity steps, tamper-evident logs. These are loss-control measures, the agent equivalent of a sprinkler system, and insurers discount for them the way they discount a warehouse with fire suppression.
Run the arithmetic, because it makes the mechanism concrete. Say a purchasing agent executes 100,000 actions a month. Suppose 0.1% produce a loss (1 in 1,000) at an average of $200 each. Expected monthly loss is 100,000 × 0.001 × $200 = $20,000. Add a load for capital, administration, and model uncertainty, call it 40%, and the premium is $28,000 a month, or $0.28 per action. That number is something we have never had: a market-made, honest estimate of the agent's unreliability, denominated in dollars per action. Now wrap the agent in verification that cuts the loss rate tenfold to 0.01%. Expected loss falls to $2,000, and the premium falls roughly tenfold with it, to about three cents an action. The price signal rewards exactly the engineering that makes autonomy safe, and it does so continuously, which a static benchmark never will.
That last point matters more than it looks. A benchmark score is a claim made once, under a fixed distribution. An agent is non-stationary: it degrades when the model is updated, when the world drifts, when an adversary finds a new injection. A premium repriced against live loss experience is the only trust estimate that updates when the agent quietly gets worse. It is trust with a feedback loop.
This is the trust primitive that scales where a human cannot
I have called the next bottleneck the coming agent trust crisis: intelligence is commoditizing, trust is not, and the winners will compete on verifiable trust primitives rather than raw IQ. A liability market is the economic layer of that stack. Cryptographic identity, provenance, scoped permissions, and audited reputation answer is this agent who it claims to be and can I inspect what it did. Underwriting answers the question those primitives set up but cannot themselves settle: given all that, what is this agent's reliability actually worth, and who pays when it is wrong?
The technical primitives are the underwriter's inputs. You cannot insure a party you cannot name, whose actions you cannot reconstruct, whose history you cannot bind to a non-transferable identity. Which is why the insurable entity is never the weights. It is the operator running the agent under a durable identity, or an agent identity backed by a bond its operator posts. The underwriting object is the operator-plus-agent-plus-controls bundle. Provenance and identity are what make that bundle underwritable at all.
The Kommerce lens is not a metaphor here; it is the working template. I build commerce infrastructure for markets where cash on delivery dominates, where neither party assumes the other's good faith because the enforcing institutions are thin. Those economies do not run on trust. They run on risk-pricing and staking: value transfers at verification, reputation is staked and slow to build, and the cost of the residual risk is priced into the transaction and often carried by a third party. The high-trust world imagines these mechanisms as exotic. In low-trust commerce they are the load-bearing structure, and the agent economy is about to become a low-trust economy at machine speed. The people who learned to price risk where good faith was never assumable have seen this movie.
The honest counterargument, and where it bites
The strongest objection is that this market may not form, or may cover far less than the hype implies, and the objection has real teeth. Insurance needs stable actuarial data, and an agent is a moving target: the loss distribution you underwrote last quarter can be invalidated by a single model update. Non-stationarity is genuinely hostile to underwriting, and it will keep the earliest policies narrow, short-tenor, and heavily loaded for uncertainty.
Adverse selection bites too. Akerlof's market-for-lemons logic (1970) applies directly: if operators know their agents' true reliability and insurers do not, the operators most eager to buy coverage are the ones running the worst agents, which pushes premiums up and drives good operators to self-insure, potentially unraveling the pool. The fix is the same as in every insurance line, legible track records and priced controls that let the underwriter tell a good risk from a bad one, which is another reason auditable action logs are not optional. Moral hazard is the mirror image: an operator who is fully insured may stop caring how the agent behaves, so policies will carry deductibles, coverage caps, and control requirements, exactly as they do everywhere else.
The fairest concession is that insurance does not remove the accountable human. It relocates and reshapes the exposure. A named party is still underwritten against. That is not a refutation of the thesis; it is the mechanism of it. Converting an unbounded personal liability into a bounded, priced, transferable one is the unlock, because a founder who will never personally stand behind an unlimited wrongful wire will happily pay a known premium for a capped, backed exposure. The human is not removed. The human is made willing.
What to watch, and what to build
The forecast reduces to one falsifiable prediction: the leading indicator of genuine autonomous deployment is not a model release, it is the appearance of agent-liability products, bonded-agent offerings, per-action coverage, operator E&O written specifically against agent actions. When an underwriter is willing to quote a price, someone has done the work of deciding that agents in that niche are a real, boundable risk. Watch the premiums before you watch the demos. A falling per-action premium in a category is the market telling you autonomy there has actually arrived.
If you deploy agents, build now for the two things the underwriter will price. Make actions reversible, staged, and capped, so the blast radius of any single action is small and known. Emit a legible, tamper-evident log, so your agent's history can be bound to its identity and repriced against reality. You are not adding compliance overhead. You are pre-negotiating your premium, which is to say the price of your own autonomy.
The agent that gets to act on its own is not the smartest one. It is the one a stranger will insure.