Across the Fortified ecosystem, leaders keep telling me the same thing. AI is already in their environments, taking work off overloaded teams, speeding up routine tasks, and helping people get answers faster.
They also tell me departments are adopting AI before security or governance teams know it is there. It arrives quietly, with vendors building it into tools organizations already own, and clinical workflows being updated for efficiencies.
This makes it challenging for any organization to know where AI is being used, and one of the biggest risks is assuming you do.
Healthcare Impact
Agents can connect to systems, pull data, and act on their own. When teams do not understand what an agent can reach, the risk can spread quickly into clinical workflows, identity, operations, and incident response.
Governance follows visibility. Know where AI is showing up before you try to govern it by answering what data the AI can see, where that data goes, what systems it connects to, and what it can do.
Leadership Recommendations
Inventory AI capabilities
Create an inventory of AI capabilities, not just products, across vendor applications, productivity suites, clinical technologies, agents, APIs, and automated workflows. For each one, document what it does, what data it can access, which systems it connects to, what permissions it holds, what actions it can take, which vendor supports it, and how it is monitored.
Then tie that inventory to procurement and change management. When a vendor adds an AI feature or changes how an existing feature works, review the capability again.
Make ownership explicit
Every AI use case should have a named business owner who is accountable for the use case itself and not just the underlying technology. Document who can approve that use case, which governance group reviews it, how performance and risk are monitored, and where issues are escalated.
Extend existing governance
Build AI into the controls your healthcare organization already run: security assessments, privacy reviews, compliance, procurement, third-party risk management, and incident response. Set clear data-access boundaries and minimum permissions for automated workflows and require vendors to disclose significant changes to AI functionality.
Give employees approved options so they are less likely to turn to unmanaged tools.
Preserve accountable human oversight
Define where human review is required and who is responsible for it. Ensure that the review has clearly defined review requirements that can be tested and audited. Focus most on AI outputs and actions that affect clinical care, security, operations, finances, or patient communication.
The review should cover potential impact and document when an automated process must stop, escalate, or hand control back to a person.
Essential Healthcare Leadership Cybersecurity Questions:
- Where is AI operating across our clinical, administrative, security, and vendor environments?
- What data can each AI capability access, and what actions can it take?
- Who owns the use case, the risk, the monitoring, and the outcome?
- Where must accountable human review remain mandatory?
Executive Takeaways:
- Inventory AI capabilities, not just products
- Governance must keep pace with AI adoption
- Approved AI is not the same as understood AI
- Maintain accountable human oversight
THREAT NEWS AND INSIGHTS
- Beyond the Checklist: Why Third-Party Risk Management (TPRM) is Shifting from Compliance to Resilience
- Healthcare Security Tool Sprawl: Why More Means Less Protection
- Healthcare Security Cost Reduction Efforts Have Become Strategic Initiatives
Peer Pulse: Shadow AI, Agentic AI, and Healthcare Governance
In this month’s Peer Pulse, Russell Teague speaks with Preston Duren (VP, Threat Services) and T.J. Ramsey (Senior Director, Threat Operations) about why shadow AI remains difficult to govern, how agentic AI changes the risk conversation, and where healthcare leaders should focus next.
Russell Teague: We’ve been talking about shadow AI for nearly two years. Why is it still such a challenge?
Preston Duren: Because I don’t think it’s been solved. Most organizations can identify some AI usage, but visibility into agents, vendor integrations, APIs, and embedded capabilities is much harder. The challenge isn’t finding ChatGPT anymore. It’s understanding where AI is operating and what it’s doing across your entire organization.
T.J. Ramsey: Most organizations understand the risks. The question is what has prevented them from moving from discussion to action. Building policies, governance processes, and enforcement mechanisms has proven harder than most people expected.
Russell Teague: What’s the biggest misconception healthcare leaders have about AI today?
Preston Duren: That the risk is limited to the chatbot itself. The bigger concern is AI agents and agentic workflows that create automated decision based on agent reasoning. Once AI starts interacting with systems, identities, permissions, and business processes, the conversation changes. We’re no longer talking only about generated content. We’re talking about actions being taken with what oversight?
T.J. Ramsey: Regardless of the tool, human validation still matters. We’ve seen AI generate convincing analysis while getting key facts wrong. A human still must validate the outcome and remain accountable for the decision and ensure its defensible.
Russell Teague: How should healthcare organizations respond when they discover shadow AI usage?
Preston Duren: My first reaction usually isn’t to block it. The better question is, “what are you trying to accomplish?” If there’s a legitimate business need, we should look for approved alternatives and better controls. Simply saying no usually sends users somewhere else.
T.J. Ramsey: That’s where governance becomes practical. Organizations will have more success providing approved tools, clear expectations, and reasonable guardrails than relying entirely on restrictions.
Russell Teague: What’s the one thing healthcare leaders should focus on over the next year?
Preston Duren: Governance maturity. We’ve moved past asking whether AI will be part of healthcare. The organizations that succeed will establish ownership, accountability, and repeatable governance processes before AI becomes embedded in every workflow.
T.J. Ramsey: Visibility. If you don’t know where AI exists, what data it touches, or what permissions it has, governance becomes nearly impossible.
CLOSING PERSPECTIVE
The risk starts when organizations assume they already know where AI is operating.
The healthcare organizations getting the most from AI will know where it is being used, who owns it, and who is accountable for it before an incident, audit, or compliance issue exposes the gaps.
