Beyond the Policy: 3 Key Components of AI Governance in Healthcare

If asked about how they govern artificial intelligence, most hospital CIOs provide the answer in the form of a document, like a policy or a charter, or any other set of principles that a governance committee approved. While that work matters, a written policy is not the actions of governance.

The tension between passive written policy and active governance sits at the center of healthcare’s AI challenge. As healthcare delivery organizations (HDOs) rapidly adopt AI, their governance capabilities have failed to keep pace. According to an August 2026 study of health systems by the Center for Connected Medicine at UPMC and KLAS Research, 93 percent of organizations had deployed third-party AI, while 63 percent described their AI governance as “developing” or “ad hoc.”

In short, slightly more than two thirds of the HDOs that implement AI know how to govern it. Increasingly, that gap is less about whether written governance policies exist and more about whether organizations have the right people with the skills necessary to carry out the policies.

In an HDO that uses Epic Systems, a routine update may add a generative AI feature to help doctors and clinicians write case notes. The vendor, contract, and procurement event remain the same, so nothing triggers another review. While the tool in production has changed, the governance process may never register it.

When governance lives primarily on paper, the policy may still be sound, even as the environment it’s meant to govern changes around it. The answer is not simply to write a better policy, but to build the operational capabilities that transform passive policy into active practice.

AI governance consists of three capabilities:

• • Seeing what’s running.

• • Enforcing rules in real time.

• • Keeping monitoring once a system is live.

None of those disciplines are new. Each extends a capability that mature security programs already know well. The asset they now have to govern is new. It changes quickly, behaves unpredictably, and can enter the enterprise in ways existing controls were never designed to catch.

 

Component 1: Visibility — Know What AI Is Running

Security rests on a principle that predates AI by decades. Security teams cannot protect what they don’t know they have. Building a complete and accurate asset inventory is foundational precisely because every other downstream control requires an accurate picture of the assets being defended.

Artificial intelligence corrodes that picture in a particular way. Conventional assets arrive through channels built to be tracked, like a purchase order, an onboarded vendor, or a provisioned server. AI arrives through channels that are not meant to be tracked, including:

• • A feature toggled on inside an application already in use: no purchase order, no new vendor, just a new capability.

• • A browser tab and a personal account: operating beyond the managed environment entirely.

• • A capability a trusted vendor adds without ceremony: implementation as an update, similar to the above-mentioned documentation vendor.

In a recent Wolters Kluwer survey of more than 500 healthcare professionals, 40 percent reported encountering an unauthorized AI tool inside their organization, and nearly one in five admitted to using one, a sign of “shadow AI” operating outside formal inventories and review processes.

While building an asset inventory is not a new discipline, AI enters the organization in ways traditional inventory processes fail to capture. A hardware inventory is a procurement-driven process that uses periodic reviews. Applying this approach to AI will miss tools added through browser use, embedded vendor features, and other channels that may never trigger a formal review.

Those blind spots carry forward into every governance decision that follows. A system no one has catalogued cannot be assessed, restricted, or monitored. Visibility may not be the most ambitious part of AI governance, but it is the foundation the rest depends on.

 

 

Component 2: Enforcement — Apply the Rules in Real Time

A rule that cannot be applied at the moment of risk is a policy, not a control. Security programs already understand this distinction. Access controls do more than define who should access a system. They also prevent unauthorized access. Data-loss controls do not just describe how sensitive information should be handled. They also intervene when that information is at risk.

While traditional governance mitigates risks related to how people interact with data, AI governance focuses on risk that occurs from interacting with the tool, like the prompt a user enters, the response a patient chat AI generates, or the next action an autonomous agent takes. This is where sensitive data can be exposed, fabricated information can enter a patient record, or an AI-enabled workflow can take an action that was never authorized.

Effective enforcement means applying governance rules at the interaction point to allow appropriate use and block or restrict activity that violates written policy. Without that capability, an organization can define how AI should be used but lacks the ability to ensure those rules are followed in practice.

Visibility can tell an organization that a risky interaction is happening. Enforcement determines whether it can do something about it. Without the ability to act, awareness alone isn’t control.

 

 

Component 3: Continuous Monitoring — Make Validation Ongoing

Most healthcare AI oversight concentrates on pre-deployment activities, like validation, bias testing, and a review board’s approval. That rigor is necessary, but it can create a false sense of finality, as if a model that is safe and effective at go-live will remain that way.

AI systems aren’t static. Models can drift as patient populations, workflows, and underlying data change. Performance can degrade over time, sometimes without an obvious signal. For example, when researchers at Michigan Medicine externally validated a sepsis-prediction model embedded in a widely used electronic health record, it missed roughly two-thirds of sepsis cases in real-world use, with performance well below what its original validation had suggested.

The potential for errors that impact human health and safety is why validation cannot end at deployment. Despite these risks, many health systems still lack the infrastructure to continuously assess how AI performs once it is live. The same CCM/KLAS study found that while 92 percent of health systems test AI tools before deployment, only 44 percent have a dedicated environment to validate accuracy, safety, and drift after go-live. Oversight is strongest at the front door and much thinner once a system enters production.

Security programs already operate with the right mindset. Monitoring is continuous because risk is continuous. AI governance requires the same posture, focused on whether a system has been compromised and whether it still behaves and performs as intended.

That distinction matters. A model can become less accurate, drift from its approved use, or begin producing unsafe outputs without any traditional security event taking place. Nothing has necessarily been breached. The AI has simply changed in ways the organization needs to detect.

Without continuous monitoring, “we validated it” only notes that the system met the standard at one point in time. Effective governance requires knowing whether it still does.

 

 

Why the three are inseparable

The reason these amount to a framework rather than a list is that each fails in isolation:

• • Visibility without enforcement lets an organization watch a risk arrive and play out.

• • Enforcement without visibility blocks actions without considering the impact on legitimate clinical work and still missing the exposures no one knew to look for.

• • Monitoring without enforcement detects a model going wrong with no means to stop it.

Effective governance depends on all three working together. Visibility identifies the AI being used so that the HDO can enforce a policy in real time while continuously monitoring to ensure the AI model continues to perform as intended.

Similarly, a governance committee, on its own, isn’t enough. A committee can define policy, establish accountability, and make decisions about acceptable use. However, it cannot replace the operational capabilities needed to put those decisions into practice.

 

 

A short test

The gap between policy and capability can be measured with three straightforward questions:

• • Visibility: Could you produce today a complete list of every AI system and AI-enabled feature touching patient data, including capabilities added by vendors already in place?

• • Enforcement: If an unsafe AI interaction were happening right now, could you stop it  or only record that it occurred?

• • Monitoring: Would you know if a tool that passed review last quarter began behaving differently this quarter?

For many organizations, the answers are still some version of “partially,” “no,” and “no.” That is not a reflection of poor intent or lack of effort. It is a practical way to identify where governance is strongest, where the gaps remain, and which capabilities need to be built next.

 

 

What AI governance in healthcare comes down to

Strong security remains the foundation of any AI governance program. Visibility, enforcement, and continuous monitoring are built into the disciplines and tools health systems already use. Extending them to AI and the new risks it introduces means organizations gain greater value from those processes and solutions.

Putting them into practice will require a combination of technology, process, and people. No single tool is the answer, and no governance framework should depend on one alone.

The goal, then, is not to build more governance around AI, but to make governance part of how AI is actually deployed and operated across the enterprise. That means moving from periodic review to an ongoing system of visibility, control, and validation that can keep pace as tools, models, and use cases change.

Health systems will continue to adopt AI faster and in more places. The organizations best prepared for that future will be the ones that make governance just as dynamic as the technology itself.