The Illusion of Oversight
A finance team spends three weekends building an automated workflow. It pulls data from three systems, generates forecasts, and emails results to department heads. No approval was sought. No security review occurred. The tool works, so nobody questions it.
This is the deception at the heart of enterprise AI. The governance apparatus — the review queues, the vendor assessments, the integration approvals — operates on what it can see. The work that never enters those queues remains invisible, accumulating structural risk while appearing to be someone else’s problem.
The misconfigured tool that passed through review is a governed risk. The production workflow built in isolation is not. At most large organizations, the second category is growing faster than the first, and the gap between what governance claims to cover and what it actually covers widens with every quarter.
Why the Official Path Fails
A predictable bottleneck plays out inside large enterprises. A team needs AI support. The formal channel is slow, backlogged, or unclear about who owns the decision. So the team builds something independently.
That instinct is correct. The structure around it is not.
Effort gets duplicated across departments with no visibility into each other’s work. Siloed tooling emerges without shared standards. Security and data handling practices diverge. People get pulled away from their core responsibilities to maintain AI tools that three other teams independently built last quarter.
Meanwhile, the governance function stays occupied with the queue it can see: integrations passing through approvals, vendors appearing in procurement requests. The work happening outside that queue — the experiments, the workarounds, the shadow infrastructure — never enters the review process because it was never designed to.
The costs compound quietly. By the time the organization recognizes the problem, undoing it costs far more than governing it would have.
The Scale Problem Nobody Planned For
The numbers make the visibility gap harder to dismiss. By 2028, the average global Fortune 500 enterprise is projected to use more than 150,000 agents, Gartner predicts. [1] That figure describes a world where centralized review of every AI deployment becomes impractical.
The governance models that enterprise organizations rely on were not designed for this moment. They assumed a manageable flow of requests through a defined pipeline. They assumed that what needed governing would announce itself by seeking approval.
Neither assumption holds.
The concentration on which AI risks to govern — bias, hallucination, data leakage, model drift — obscured a more fundamental question. What operating model should underpin the governing itself? The answer cannot be more review layers. Adding gates to a process that people already route around simply pushes more work underground.
Where the Real Risk Lives
A distinction matters here. The AI that deceives through wrong answers, fabricated citations, or confident nonsense gets attention because it is visible. The AI that deceives through invisible adoption gets less scrutiny because nothing appears broken.
Consider what happens when a team deploys an agent to handle customer inquiries without telling anyone. The agent works. Customers get responses. Metrics look fine. But the agent also makes commitments the company cannot track, accesses data it should not touch, and creates liability that surfaces only when something goes wrong.
The deception is structural, not technical. The system appears governed because the visible parts are governed. The invisible parts — which may represent the majority of actual AI usage — operate without oversight, standards, or even awareness that they exist.
This is not a failure of individual employees cutting corners. It is a failure of operating models that treat governance as a gatekeeping function rather than an enabling one. When the sanctioned path is slower than building alone, rational actors build alone.
What an Operating Model That Works Looks Like
Experimentation is essential for finding valuable AI use cases. Suppressing it through tight controls is the wrong approach — and it often pushes experimentation further underground. The goal is not to stop the work but to make the sanctioned path faster than the unsanctioned one.
Three structural changes move organizations toward that goal.
A central team focused on policy and standards creates a single interface for risks surfaced by security, legal, and compliance. This team sets technical guardrails, owns vendor selection, and builds reusable patterns that every function can draw from. Its purpose is to make responsible AI the default, so that individual review becomes the exception rather than the rule.
Embedded AI transformation owners inside each function provide the second piece. These people work day-to-day within a function but report to the centralized governance team. They understand local context and apply controls consistently. They serve as the single intake point for AI requests within their function, so teams building in isolation have someone who can triage, prioritize, and connect the work to what already exists elsewhere.
The reason this structure works where formal channels fail is speed. When the sanctioned path is faster than building alone, teams stop routing around it.
Measurement provides the third piece. A good signal that governance is enabling sanctioned AI experimentation is how much work across the organization compounds in value. Look at reuse: a pattern built for finance getting picked up by operations, a workflow proven in one team becoming the default in three others. A dropping volume of IT requests combined with compounding, visible AI usage tells you experimentation is both maturing and staying visible.
The Legal Dimension Nobody Escaped
The governance question has moved beyond IT and security. Legal teams now confront questions about liability for AI outputs, documentation of AI decisions, privileged data fed into AI tools, and preparation for future regulation. These questions reach into risk management, compliance, and corporate governance.
Gartner has warned that insufficient governance controls could force companies to roll back AI initiatives. That warning describes a future where the invisible risk becomes visible through failure — a costly way to discover what a functioning operating model would have caught earlier.
The organizations treating AI governance as a gatekeeping function have missed the point. AI needs governance, but the role of governance needs to change. Significant risks live in the work that never enters the queue. The only way to govern that work is to be present where it happens, before it compounds into something much harder to fix.
AI moves fast. Governance standards need to travel with it, rather than chasing behind.
The Figma Case: Governance as Competitive Question
The tension between AI partnership and AI competition surfaced publicly in 2026 when activist shareholder Findell Capital urged Figma to review its governance structure and its partnership with Anthropic. The push followed Anthropic’s launch of a competing product, raising questions about conflicts of interest and Figma’s AI product roadmap.
Figma CEO Dylan Field responded by outlining the company’s AI investment approach and disputing claims that AI makes traditional product design obsolete.
The episode illustrates a version of the visibility problem at the corporate level. A partnership that appeared to strengthen Figma’s position — deep integration with a leading AI platform — also gave a potential rival insight into product direction and distribution advantages. The governance review Findell requested was less about short-term earnings than about whether Figma was arming a competitor while believing it was building a moat.
The challenge from Findell Capital arrived when investors were already focused on the business case for Figma’s AI spending and the durability of its core design platform.
The outcome may influence Figma’s competitive positioning in AI design tools, its board priorities, and how future capital is allocated. If AI tools such as Anthropic’s competing product can own more of the creation process, Figma’s position as the central design system could be tested. If Figma remains the core collaboration layer that orchestrates multiple AI engines, the partnership and any governance refresh could support execution on its existing AI roadmap.
The call to tighten governance and clarify AI partnerships aligns with the narrative that Figma is trying to broaden use cases for AI-powered workflows across product managers, engineers, and marketers — a strategy that relies on strong platform control. The emergence of Anthropic as a direct competitor challenges the assumption that deep integrations with large AI platforms automatically make a company more embedded in customer workflows.
The Expertise Question
The governance conversation often assumes that selecting the right AI tools is the primary challenge. In FulcrumGT CEO Ahmed Shaaban’s view, that assumption misses something more fundamental.
As Shaaban puts it, AI is a force multiplier — it amplifies what a team already knows how to do. If the organization has not invested in human capability first, it is not accelerating anything. It is just making mistakes faster.
This observation reframes the governance problem. The issue is not only which tools get approved or which workflows get reviewed. It is whether the people directing those tools have the expertise and vision to use them well. A governance model that focuses on process without building capability produces compliance theater — the appearance of control without the substance.
The deception here is subtle. A company with extensive AI policies, review committees, and approval workflows may believe it has governance. But if the people building AI tools lack the judgment to recognize when they are creating risk, the policies govern nothing. They create documentation of decisions that should never have been made.
What Compounding Value Looks Like
Reuse is the clearest signal that governance is enabling sanctioned AI experimentation rather than obstructing it.
If AI usage grows while reuse stays flat, the organization is building silos faster than it is building capability. If reuse grows while the volume of IT requests drops, experimentation is maturing and staying visible.
Governance as pure gatekeeping produces the opposite pattern. Usage migrates underground. Reuse becomes impossible because no one knows what anyone else has built. The visible queue shrinks while actual AI deployment grows, and the gap between what governance claims to cover and what it actually covers becomes the organization’s largest unmanaged risk.
The Cost of Waiting
As AI compute costs consume larger portions of IT budgets, the cost of a poor operating model does not become visible until it is expensive to fix. Shadow tooling that started as a workaround becomes load-bearing infrastructure. Teams that built in isolation develop technical debt through approaches that are hard to standardize later.
There is a lot more to undo than there would have been to govern.
The organizations that recognized this early are building operating models that accommodate experimentation rather than suppress it. They are embedding governance where the work happens, not where the queue forms. They are measuring reuse and compounding value rather than counting approvals.
The ones that did not recognize it are still reviewing the queue, still wondering why the risk they can see does not match the risk they experience, still treating governance as a gate rather than a capability.
The gap between what AI appears to do and what it actually does is not only a property of the technology. It is a property of the organizations deploying it. The governance that appears to work and the governance that actually works are different things — and the difference lives in the work that never enters the queue.