Someone on the trading desk at a Fortune 100 financial services firm asked an AI chatbot to summarise a client portfolio. One question, one tool, one person. A week later the security team found 47 autonomous agents running across six business units. None of them had been approved. None had been audited. None had even been named.
That anecdote, from a Security Boulevard investigation published this month, is not an outlier. It is the pattern. Eighty percent of Fortune 500 companies now have active AI agents in production, according to Microsoft's security research. And the majority of those organisations have no centralised record of what those agents are, what they can access, or who authorised them.
The word for this is sprawl. It is the same dynamic that gave us shadow IT a decade ago — tools adopted faster than governance could track them — except the stakes are categorically different. A shadow SaaS subscription leaks data if someone misconfigures a sharing setting. A shadow AI agent acts on data autonomously, makes decisions without review, and compounds its own errors across every interaction. The failure mode is not a leak. It is an unaudited decision chain.
Deloitte's 2026 State of AI in the Enterprise survey — 3,235 IT and business leaders across 24 countries — found that only 21 percent of organisations have a mature governance model for agentic AI. Nearly three-quarters plan to deploy agentic AI within two years. The gap between those two numbers is where risk is accumulating.
The instinct is to reach for tooling. Monitoring dashboards. Compliance checklists. Agent registries. And those tools are arriving — Gartner projects the AI governance platform market will approach half a billion dollars in 2026 alone. But tooling solves the visibility problem. It does not solve the design problem.
A registry tells you that 47 agents exist. It does not tell you whether any of them should. A monitoring dashboard shows you what an agent did. It does not show you whether the rules governing that agent connect to the workflow it operates in. An observability layer catches the failure after it happens. A governance design prevents the category of failure from being possible.
This is the distinction that the current market is not making. Building effective human-in-the-loop checkpoints remains one of the hardest governance challenges organisations face as they scale agentic AI. Most are settling for what researchers call "human-on-the-loop" — a passive model where a person watches but does not intervene. The result is the appearance of oversight without the mechanism of control.
Yale's Chief Executive Leadership Institute published a governance framework this month that names four requirements: transparency (can stakeholders reconstruct how the agent reached its decision?), accountability (who bears responsibility when it goes wrong?), bias (does the system perpetuate or amplify systematic disadvantage?), and data privacy (how is information protected when agents access and combine data across systems without per-transaction review?).
Those are the right questions. They are also design questions, not technology questions. You cannot solve them by adding a layer of monitoring on top of agents that were deployed without design. You solve them by designing how the agent operates before it reaches the workflow — deciding what it can do, what it cannot, who reviews its output, and what happens when the rules do not fit the situation.
The organisations that avoid the sprawl pattern share one characteristic: they designed the control layer before the agents multiplied. Not a policy document. Not a risk register. An operating layer — rules embedded in the workflow that determine what the agent does, who decides when it stops, and how the decisions connect to the work.
That is governance design. It is the layer the market has not built. And until it exists, every new agent is another entry in an inventory nobody is keeping.
One workflow at a time.
Sources: Security Boulevard, "The Shadow AI Governance Crisis: Why 80% of Fortune 500 Companies Have Already Lost Control of Their AI Infrastructure," Deepak Gupta, 4 May 2026. Microsoft Security Blog, "80% of Fortune 500 Use Active AI Agents: Observability, Governance, and Security Shape the New Frontier," 10 February 2026. Deloitte, "State of AI in the Enterprise," 2026, 3,235 IT and business leaders across 24 countries. Deloitte Insights, "AI Agents Scaling Faster Than Guardrails," 2026. Fortune / Yale Chief Executive Leadership Institute, "Anthropic's Most Powerful AI Model Just Exposed a Crisis in Corporate Governance," Jeffrey Sonnenfeld et al., 2 May 2026. Gartner, "AI Governance Market Forecast," February 2026.