AI Privacy Risks in Healthcare: Where Patient Data Actually Leaks

You adopted an AI scribe to save clinicians two hours a day. You approved a chatbot for patient triage. Maybe you didn't approve anything at all, and your staff found their own tools. Either way, AI privacy risks in healthcare are no longer theoretical. They are structural. This piece walks through the specific points where patient data escapes clinical workflows when AI enters the picture, who holds that data once it does, and what you can actually do about it beyond writing another policy memo nobody reads.

Key Takeaways

• The ambient AI scribe on your clinician's phone generates far more protected health information than what ends up in the chart. Some vendors retain that surplus data (audio, interim transcripts, metadata) to retrain models. Your BAA may not address this.
• Shadow AI usage in healthcare is endemic: 57% of healthcare professionals have used unauthorized AI tools for work. Blocking tools doesn't fix the problem; deploying compliant alternatives does.
• Third-party vendor incidents now account for 58% of all healthcare data breaches, up from 44% in 2023. A single compromised AI vendor can expose millions of records across dozens of hospital systems simultaneously.
• Consumer AI health products generally fall outside HIPAA. Patients sharing symptoms with a chatbot from a non-covered entity have essentially no federal privacy protection.
• Governance is lagging adoption across the industry. A 2026 benchmarking study of 54 healthcare organizations found that AI implementation is growing faster than the oversight mechanisms meant to contain it.

Where Does Patient Data Go When You Deploy an Ambient AI Scribe?

It goes to more places than you think. An ambient scribe captures live audio of a clinical encounter, generates an interim transcript, produces a machine-drafted note, and collects metadata about the clinician and patient. The final chart note is one artifact among several. The others are the ones that matter for privacy.

Some vendors delete raw audio within hours. Others retain recordings and transcripts indefinitely, sometimes to retrain their models, sometimes to let clinicians replay visits. Medcurity's 2026 compliance guide lays this out clearly: the scribe itself is not the leak. The vendor's retention policy is.

This is the due-diligence question almost nobody asks during procurement. Administrators evaluate whether a scribe vendor is "HIPAA compliant" in the abstract, check a box, and move on. The actionable question is different: what happens to the audio and interim transcript after the note is finalized? Is it deleted? Retained? Used for model training? And is the answer to that question actually specified in your Business Associate Agreement, or did you sign a template?

Legal exposure is already materializing. The American Bar Association's Health Law Section has documented cases where patients were never informed that recordings were occurring, where transcripts were transmitted to and stored by vendors without adequate controls, and where AI-generated records inaccurately stated that patient consent had been obtained when it had not. That last one is worth reading twice. An AI tool fabricating a consent record is a compliance event and a liability event simultaneously.

What Should You Demand in a BAA for an AI Scribe?

Specific retention timelines for each data type: raw audio, interim transcript, final note, and metadata. Explicit language about whether any of that data is used for model training or improvement. A deletion schedule that is auditable, not just promised. And a clause that addresses what happens to data if the vendor is acquired or goes bankrupt. These are not exotic asks. They are the minimum for a tool that records doctor-patient conversations.

How Bad Is the Shadow AI Problem in Healthcare?

It is pervasive and largely invisible to IT. A 2026 report from Paubox found that 95% of healthcare organizations suspect their staff is already using generative AI for work-related email or content. A quarter of those organizations have not formally approved any staff use of AI in email at all. The gap between what staff is doing and what leadership has sanctioned is enormous.

A February 2026 survey by Healthcare Brew put a finer point on it: 57% of healthcare professionals have encountered or used unauthorized AI tools at work. The same Paubox report found that only 42% of healthcare organizations have signed a BAA covering an AI tool used in email, and 21% of teams believe a BAA is not required for an AI assistant at all.

That last number is the one that should keep you up. One in five teams thinks HIPAA simply doesn't apply to their AI email tool. It does.

Why Do Clinicians Use Unauthorized AI Tools?

Because the sanctioned tools are too slow, too clunky, or don't exist. Shadow AI is a symptom. The disease is that most health systems haven't deployed compliant-by-default AI tooling embedded in existing clinical workflows. When a physician can paste a discharge summary into a consumer chatbot and get a plain-language patient handout in twelve seconds, and the approved alternative requires submitting a ticket to medical records, the physician is going to use the chatbot. Every time.

Policy memos and blocklists do not fix this. You can block one consumer AI product and three more will surface next week. The structural fix is giving clinicians tools that are as fast as the unauthorized ones, with privacy controls built in rather than bolted on. This is a product problem, not a policy problem.

What Is the Real Blast Radius of a Third-Party AI Vendor Breach?

It is far larger than most risk assessments assume. Third-party vendor incidents now account for 58% of all healthcare data breaches, up from 44% in 2023. According to the Verizon 2026 Data Breach Investigations Report, third-party breaches in healthcare rose 60% year over year. Seven of the ten largest breaches reported to OCR in the first half of 2026 involved a business associate or vendor system.

The Xsolis incident is the case study. A single phishing attack gave attackers a two-day window inside the AI vendor's system, exposing Social Security numbers, health insurance details, and medical treatment records for nearly 1.4 million individuals across seven major hospital systems. One vendor. One phishing email. Seven health systems. 1.4 million patients.

Separately, NYC Health + Hospitals disclosed a breach via a third-party vendor exposing medical records, government IDs, geolocation data, and biometric data for at least 1.8 million people.

Why Is Fourth-Party Concentration Risk the Underreported Story?

Because most vendor risk assessments evaluate a single vendor in isolation. They ask: is this vendor secure? They rarely ask: how many other hospitals share this same multi-tenant AI backend? When a widely deployed AI platform serves dozens or hundreds of healthcare organizations, breaching that one platform is equivalent to breaching all of them at once. The blast radius is not proportional to your organization's size. It is proportional to the vendor's market share.

This is a different kind of risk than traditional vendor management was designed to handle. It means that the more popular and successful an AI platform becomes, the more attractive it is as a target, and the more catastrophic a single compromise. Administrators evaluating AI vendors need to reframe their risk assessment around blast radius, not just individual vendor audit scores.

How Much Does a Healthcare Data Breach Actually Cost?

A 2025 IBM report found that the average security breach in the healthcare industry totaled over $7.4 million. The same analysis found that 97% of organizations with AI-related security incidents lacked proper AI access controls. Nearly all of them. The cost is real and the controls are absent.

That $7.4 million figure includes direct costs like forensics, notification, and regulatory fines, but also indirect costs: lost patient trust, diverted clinical resources, legal fees, and the operational disruption of running an incident response while also trying to provide patient care. For a mid-size health system, a single breach can consume the entire IT security budget for multiple years.

Does HIPAA Actually Cover Consumer AI Health Tools?

Mostly, no. AI companies and health-app developers are generally not covered entities or business associates under HIPAA. The Center for Democracy & Technology has documented this gap clearly: while some states have health privacy laws, protections vary widely and don't cover most Americans.

This matters as patients increasingly use consumer chatbots for health questions. A patient who describes their symptoms, medications, and medical history to a consumer AI product has no federal privacy protection for that data. The AI company can, in many cases, use that information for model training, sell aggregated insights, or share data with third parties, depending on their terms of service. Most patients do not realize this.

Two major AI labs announced new consumer AI products focused on health in early 2026, both of which raise privacy questions that remain unanswered for users considering sharing personal health information. The regulatory gap between clinical AI (subject to HIPAA) and consumer AI (largely unregulated at the federal level) is widening as adoption grows on both sides.

What New Regulations Are Coming in 2026?

State-level regulation is arriving faster than federal action. The Texas AI Policy Act and similar state laws now require explicit patient consent for AI data use and clear disclosures about how an AI model makes decisions. In Canada, British Columbia's privacy commissioner released guidelines in January 2026 for healthcare organizations using AI scribes, and Ontario's privacy commissioner released parallel guidance.

These regulations share a common thread: they shift the burden of transparency onto the deploying organization, not the vendor. If your health system uses an AI scribe, you are responsible for informing patients, obtaining consent, and ensuring the vendor's data practices comply. "The vendor told us it was compliant" is not a defense that regulators are accepting.

Where Does Governance Stand Relative to Adoption?

Behind. Significantly behind. A 2026 benchmarking study of 54 healthcare organizations found that AI implementation is growing faster than the oversight mechanisms needed to ensure safe use. This creates compounding privacy risk, especially where AI models train on sensitive patient data without clear governance frameworks.

Nearly two-thirds of hospitals using Epic Systems have adopted ambient AI charting tools. A major competing EHR vendor began offering its ambient AI scribe free to all customers in February 2026, removing cost barriers for hundreds of thousands of providers. Adoption is now outpacing governance in most systems. Free tools accelerate adoption faster than compliance teams can evaluate them.

The pattern is consistent: a clinical champion discovers an AI tool that saves time, deploys it in their department, and by the time compliance, legal, and IT are aware, it is embedded in clinical workflows and producing records of care. Removing it at that point is operationally painful, so the tool gets retroactively approved with minimal scrutiny. This is how privacy debt accumulates.

What Should You Actually Do About This?

Five concrete steps, none of which require buying anything.

Audit your AI vendor contracts for data retention and training-use clauses. Not whether the vendor is "HIPAA compliant" in the abstract, but what specifically happens to audio, transcripts, metadata, and intermediate outputs. If the answer isn't in the BAA, it isn't enforceable.

Map your fourth-party exposure. For every AI vendor you use, ask how many other healthcare organizations share the same backend infrastructure. If your ambient scribe vendor serves fifty hospital systems on the same multi-tenant platform, your breach risk is coupled to theirs. Price that into your risk model.

Deploy compliant alternatives before you write the shadow AI policy. Clinicians use unauthorized tools because authorized alternatives don't exist or are too slow. Identify the three most common shadow AI use cases in your organization (discharge summaries, prior auth letters, and patient communication are typical) and deploy sanctioned tools that match the speed of the unauthorized ones. Then write the policy.

Require explicit patient notification for ambient AI recording. Even if your state doesn't mandate it yet, the regulatory direction is clear. Building the notification workflow now is cheaper than retrofitting it under a consent decree later. Document the notification in the patient record, and don't let the AI tool itself generate the consent documentation (because it will hallucinate consent that was never given).

Treat AI vendor access controls as a board-level risk item. When 97% of organizations with AI-related security incidents lack proper AI access controls, the problem is not technical. It is organizational. Access controls for AI systems need the same governance rigor as access controls for the EHR itself.

How Does This Apply to AI Tools Your Staff Uses for Non-Clinical Work?

The same principles apply, with one additional complication: the line between clinical and non-clinical data is blurrier than most people assume. A clinician pasting a patient's medication list into a consumer chatbot to draft a referral letter has just sent PHI to a non-covered entity. An administrator emailing a spreadsheet of patient appointment data to a colleague using an AI-powered email tool may have transmitted PHI through an AI system with no BAA.

We build Selina as a privacy-focused AI assistant that encrypts content at rest and routes requests through a stack of frontier models via API, with a short retention window for non-content operational metadata. That architecture reflects a specific position: the AI tool your staff reaches for daily should have privacy controls built into its foundation, not layered on after the fact. But we are an AI assistant, not a clinical documentation system. The ambient scribe problem, the EHR integration problem, and the vendor consolidation problem are distinct from the general-purpose AI assistant problem, and they require distinct solutions.

What Is the Cost of Doing Nothing?

Your staff is already using AI. The question is whether they are using tools you chose, with contracts you negotiated, under governance you control, or tools they found on their own, with terms of service nobody read, sending patient data to servers you cannot audit. The breach statistics suggest most health systems are closer to the second scenario than the first. The cost of doing nothing is not zero. It is $7.4 million, plus the patients whose data you were entrusted to protect.

If you want an AI assistant that encrypts content by default and doesn't train on your data, start a free 7-day trial, no card required.

Michael C.

Founder & Principal Engineer, Selina Labs

Michael builds Selina, a privacy-first AI that remembers you across conversations. He ships security-sensitive AI in production — real attacks, real fixes, measured in minutes and dollars — and writes about privacy, security, and LLMs from that seat. Top Rated Plus and expert-verified on Upwork.

• www.upwork.com
• x.com
• medium.com
• selinalabs.io