Data scientists are concerned that HIPAA security laws are quickly becoming outdated with the progress of AI in healthcare. A New York University research team found that LLMs can re-identify patients from clinical notes.
And yet the use of AI has become widespread among care providers. They use it to transcribe patient calls and predict what’s ailing them.
What they do not check is whether the patient details they enter have been deidentified under HIPAA Safe Harbor rules before they input it in the AI tools.
Then there are also AI-powered third-party tools that have access to Protected Health Information (PHI). The possibility of AI revealing confidential patient information to the public or sharing it with advertising providers poses far greater risk.
That is why it is important to learn about the vulnerabilities you expose your system to when using AI. The usage of AI in healthcare is not a problem in itself. It is the ignorance of the users that violates HIPAA compliance most of the time.
So, let’s take a look at some ground reports to assess what threats arise from rampant use of AI.
How AI threatens HIPAA compliance – Some Case Studies
AI technology may seem alluring because of how easier it makes our day-to-day tasks. But once you feed health data into it, the privacy risk becomes real.
Here are some cautionary tales that show AI and HIPAA compliance do not always go hand-in-hand.
Advocate Aurora Health’s $12.225 Million Lawsuit
In 2022, a non-profit hospital network called Advocate Aurora Health suffered a Pixel tracking-related data breach. While Meta Pixels are nothing but simple code snippets, they are powered by AI for tracking on platforms like Google and Meta.
The purpose of the tracking tool was to gain insights into improving the website and the app. Instead, it compromised the private data of roughly 3 million patients.
Advocate Aurora Health settled the subsequent lawsuit for $12.225 million.
Serviceaide, Inc.’s Data Breach of 483,126 Patients
As a BA, Serviceaide, Inc. provided Agentic AI-powered database and IT management services to Catholic Health. This included access to patients’ PHIs.
In 2024, Serviceaide exposed the private data of 483,126 patients. This happened due to improper security configurations in the AI-powered Elasticsearch database.
MMG Fusion’s Massive Data Breach
A company selling AI-driven, all-in-one oral solutions software, MMG Fusion suffered a data breach in 2020. The unauthorized actor stole the PHI of 15 million patients and posted them on the dark web.
MMG Fusion reached a settlement of $10,000 with OCR earlier this year for its HIPAA violations.
Healthcare Workers Uploading Patient Information to AI
One of the most concerning revelations came from the research conducted by the cybersecurity company Netskope.
71% of healthcare workers are using generative AI tools such as ChatGPT and Gemini on personal accounts. And the egregious part is that they are sharing sensitive data like PHIs.
However, using public versions of AI in healthcare does not make them automatically fall under HIPAA compliance. And then there is also the process of de–identification of PHI, which is an important step of HIPAA for healthcare workers before it is inputted into AI.
All these examples prove that it is either negligence or ignorance that violates HIPAA compliance.
In the case of Advocate Aurora Health, it is their fault for not having a stricter BAA and a regular inspection to see whether the security controls in the BAA are being met.
Serviceaide shows the risks you bring onboard when you do not properly vet your vendors or the technologies they use. AI-connected vendors with weak controls are simply a recipe for disaster.
But according to Netskope’s research, the biggest risk to HIPAA for healthcare comes from caregivers themselves. As more data is pushed into chatbots and other AI services, healthcare providers are not only opening themselves up to lawsuits but also putting their patients’ trust at risk.
Why Do These Gaps Keep Happening? - Threats AI presents to HIPAA compliance
While human negligence remains a big part of the conversation, organizational policies and governance can also leave security gaps.
Let’s discuss them in depth.
- Data Sharing Without Controls: HIPAA violations occurs every time patient data enters an unapproved AI tool without a signed BAA. And most of the time this happens silently, behind the back of the administration.
- Weak Risk Analysis: Management often gives priority to partnership with vendors rather than assessing the latter’s HIPAA compliance. By the time the risk analysis happens, it is already too late, and PHIs are already moving through the system.
- Misconfigured AI/Cloud Environments: Misconfigured AI systems, although an overlooked aspect, can cause unauthorized data breach. The Serviceaide incident is proof of that.
- Use of Unauthorized AI Tools: Hospital staff use public AI tools to work more efficiently in a hectic environment. It automates time-consuming tasks like scheduling and avoids crashing out under the organization’s burden. But the ease also puts the privacy of patients at risk.
What Mitigating Measures Should Organizations Look to Deploy?
HIPAA has always been technology neutral. As a result, it doesn’t prohibit the use of AI. However, it does mandate that any AI tool accessing PHI must adhere to its security and privacy rules.
Since compliance starts before AI deployment and continues after launch, let’s look at how it will look from start to finish.
Before Deployment
- Before handling PHIs, all AI tools must go through a formal risk analysis. From how the data flows through the system to how they are stored, you must document everything.
- This is also the step where you vet AI vendors thoroughly. Simply signing a BAA is not enough. You need to verify that they are following HIPAA compliance.
- Hospital staff must ensure that all data has been properly de-identified to avoid re-identification of the patient. This is a crucial step of HIPAA for healthcare providers.
- You need to be transparent with patients about the way their personal information will be used before deploying an AI model or tool.
- The most important mitigation is staff training. If patient data is needed to improve AI models, make sure it meets HIPAA’s Safe Harbor rules first.
During Deployment
Organizations must remain vigilant during this process and employ the following three strategies to marry AI and HIPAA compliance.
- Clear AI Policy: You need to specify which AI tools are approved for use and which are off limits. Staff must know that tools not on the approved list are not allowed and teaching them the reason will prevent future miscommunication.
- Active Monitoring: If clarifying the policy doesn’t help, use network monitoring and endpoint controls to detect unauthorized PHI sharing. HIPAA violations can happen without any alarms, and it will be too late before a compliance officer takes notice.
- Data Training: You must ensure that the staff understands what patient data they cannot share with AI under any circumstances.
After Deployment
- Compliance is never a one-time thing. It is a continuous process that requires regular audits.
- For cloud-hosted AI tools, always ensure the configurations are working as intended. Misconfigured systems can be disastrous.
- Always remember to reassess vendors at least annually. Just because there is a BAA in place doesn’t mean vendors cannot stray from it.
- Restricting staff to PHI according to the role they serve in the organization brings accountability. And adding additional security measures such as multi-factor authentication (MFA) ensure only authorized people have access to sensitive data.
Why You Need to Tackle AI-Related Risks Now
Organizations are already running late. The Department of Health and Human Services (HHS) published their most significant update to the HIPAA Security Rule in nearly 20 years.
It acknowledges the requirement of stronger safeguards for AI systems, proposing measures such as mandatory encryption, and regular risk analysis.
Still, it is a proposed rule, and the final implementation is yet to get a timeline.
What this means is that there are more AI-related security lapses happening each day due to oversight.
Take, for example, the MMG Fusion data breach that happened in 2020, and the company reached a settlement in 2026.
Within that time frame, AI has already reached the capability to identify patients from nothing but the doctor’s note.
This is why organizations should take measures now instead of waiting for AI-specific regulations.
How Can Kleap Help You?
There may already be a security gap that you are unaware of, and you need some guidance on how exposed your PHI is or whether your AI-enabled workflow is truly meeting HIPAA compliance.
This is where Kleap steps in.
KLEAP does not simply give you a platform score, or a vendor attestation, or a checklist to tick. That is not what KLEAP does.
KLEAP’s concierge-style model gives healthcare organizations year-round compliance and delivers a tangible risk assessment with proper documentation.
The entire process evaluates your actual environment. It starts with your AI tools, before moving on to your data flows, your vendor relationships, and your workforce behavior. A generic framework is not what constitutes KLEAP’s risk assessment.
For organizations planning to integrate AI, Kleap can help by:
- Reviewing where PHI enters AI-enabled tools and chatbots.
- Identifying policy gaps, vendor risk, and missing controls around data sharing.
- Validating the configuration of cloud-based AI apps.
- Testing whether healthcare web apps are vulnerable to data breaches.
Healthcare teams should not wait for a perfect AI compliance memo before acting.
If you do not know where you’re facing data exposure, that is exactly where the conversation should start.
