AI Training Data for Healthcare: HIPAA, Compliance, and What to Check

Why healthcare AI training data requires a different procurement approach

Healthcare AI programs routinely require training data that includes Protected Health Information (PHI) -- patient records, diagnostic images, clinical notes, insurance claims, medical audio. Under HIPAA, any vendor who handles PHI on behalf of a covered entity or business associate is itself a business associate and must sign a Business Associate Agreement (BAA) before receiving any data.

The consequences of mishandling are significant. A HIPAA breach involving PHI can cost a covered entity $100 to $50,000 per violation (HHS OCR tiered penalty structure), up to $1.9 million per violation category per year, plus reputational damage and state AG enforcement. The 2024 Change Healthcare breach -- which affected 100+ million records -- showed how a single vendor relationship can create systemic healthcare sector risk.

Beyond HIPAA, healthcare AI programs may also need to address the EU Medical Device Regulation (MDR) and In Vitro Diagnostic Regulation (IVDR) if the model will be deployed in Europe, the FDA's AI/ML Software as Medical Device (SaMD) guidance if deployed in the US, and HITRUST if the organization uses that framework as an overlay to HIPAA.

What a HIPAA BAA covers and what it does not

A BAA (Business Associate Agreement) is a contract between a covered entity (or business associate) and a downstream vendor who will handle PHI. Under HIPAA, the BAA must specify: the permitted uses and disclosures of PHI by the business associate; the business associate's obligation to safeguard PHI; breach notification obligations; the right to audit compliance; and the requirement to pass BAA obligations through to any sub-business-associates (subprocessors).

What a BAA does not do: it does not create HIPAA compliance on its own. A signed BAA with a vendor who does not have technical and administrative safeguards in place is a contract, not a control. The practical question is whether the vendor's security posture -- endpoint security, access controls, encryption in transit and at rest, workforce training, audit logging -- is actually HIPAA-compatible, not just whether they will sign the paper.

The BAA also does not cover de-identified data. If PHI has been properly de-identified under HIPAA's Safe Harbor or Expert Determination method, the data is no longer PHI and HIPAA requirements do not apply to it. However, re-identification risk -- the risk that de-identified data can be re-identified by combining it with other datasets -- is a practical concern even when HIPAA technically does not apply.

PHI de-identification for AI training data

HIPAA's Safe Harbor method requires removal of 18 specified identifiers (name, address, dates except year, phone, SSN, medical record numbers, etc.) and requires that the covered entity have no actual knowledge that the remaining information could identify an individual. The Expert Determination method allows a statistician to certify that re-identification risk is very small.

For AI training data, the challenge is that models can sometimes learn to re-identify data from patterns that are not in the 18-identifier list. A radiology model trained on chest X-rays that retain bone density and anatomical patterns may enable re-identification through features that Safe Harbor does not address. This is an area where FDA SaMD guidance and emerging AI-specific data governance frameworks are stricter than HIPAA alone.

When sourcing medical imaging annotation, clinical NLP, or diagnostic data for AI, the safest approach is to use de-identified data where possible and apply BAA-covered handling where PHI is necessary. The annotation vendor should be able to describe their de-identification validation process and whether they use a human review step to catch residual identifiers before annotation begins.

Questions to ask every healthcare AI data vendor

Will you sign a HIPAA BAA? This is the threshold question. A vendor who will not sign a BAA for PHI-involving work is not viable for regulated healthcare programs. The BAA should cover the specific service being procured, not a generic template.

What are your HIPAA technical safeguards? Specifically: encryption at rest and in transit, access controls with least-privilege principles, audit logging of who accessed what data and when, workstation controls for any remote annotation work, and a breach response plan with defined notification timelines. HIPAA requires 'reasonable and appropriate' safeguards -- ask what that means operationally for their service.

Do you conduct annual HIPAA workforce training? Every member of the workforce who handles PHI must receive annual training on HIPAA's requirements. For annotation teams, this training needs to be specific to the handling of clinical data -- not a generic data handling module. Ask how training is documented and whether you can review the training materials.

What is your subprocessor list? Annotation vendors often use subprocessors for data storage, platform tooling, or overflow annotation capacity. Each subprocessor who handles PHI is a sub-business-associate and must be covered by a BAA from the primary vendor. Request the subprocessor list and ask whether BAAs are in place for each.

Medical imaging annotation: specific requirements

Medical imaging annotation -- radiology, pathology, dermatology, ophthalmology, cardiology -- requires clinical domain expertise that general annotation vendors cannot provide. A bounding box around a 'suspicious lesion' drawn by an annotator without medical training will have low clinical validity. The practical requirement for diagnostic AI training data is that initial annotation be performed by, or validated by, qualified clinicians: radiologists, pathologists, or trained medical imaging technicians depending on the task.

DICOM format handling is a baseline technical requirement for radiology data. DICOM files contain both image data and embedded metadata that may include PHI in header fields. An annotation vendor working with DICOM files must be able to scrub PHI from DICOM headers before annotation begins -- this is often done with a DICOM anonymization pipeline, and the specific tags stripped should be documented.

For the FDA SaMD pathway, training data documentation is a component of the device file. Records of who annotated each case, what qualifications they held, what QA process was applied, and what inter-annotator agreement was achieved are expected as part of the algorithm's evidence package. Vendors who cannot produce structured annotation provenance records are not viable for FDA-track programs.

Clinical NLP and EHR data annotation

Clinical notes, discharge summaries, and EHR records are among the most PHI-dense data types in healthcare AI. NLP models for clinical decision support, coding automation, and clinical trial matching all require annotation of these records. The annotators need clinical context: understanding of medical terminology, ICD coding conventions, drug naming standards, and clinical document structure.

Annotation of clinical NLP data should be performed in a de-identified dataset wherever possible. Where PHI is necessary -- for example, for a model that must learn to identify PII for downstream redaction -- annotation should occur in a HIPAA-secured environment with full audit logging, limited workforce access, and BAA coverage.

A common procurement error is treating clinical NLP annotation as if it were general text annotation and awarding to a low-cost general annotation vendor. The resulting labels are often clinically invalid because the annotators lack the medical domain knowledge to distinguish relevant from irrelevant clinical findings. Structuring quality evaluation around clinical expert review -- at least on a sample basis -- is worth the additional cost.

Frequently asked questions

Does an AI training data vendor need to sign a HIPAA BAA?

Yes, if the vendor will handle Protected Health Information (PHI) on behalf of a covered entity or business associate, they are a business associate under HIPAA and must sign a BAA before receiving any PHI.

Can I use de-identified medical data for AI training without HIPAA compliance?

Properly de-identified data under HIPAA's Safe Harbor or Expert Determination method is no longer PHI, so HIPAA technically does not apply. However, re-identification risk from combined datasets remains a practical concern, and for FDA SaMD programs, data governance requirements extend beyond HIPAA de-identification.

What security certifications should a healthcare AI data vendor have?

At minimum: SOC 2 Type II (preferably including privacy Trust Services Criteria), ISO 27001:2022, HIPAA BAA capability, and ideally HITRUST CSF certification for organizations using that framework. For EU healthcare AI, also require ISO 13485 alignment and MDR-compatible data governance.

Can Corpshore AI handle healthcare AI training data?

Yes. Corpshore AI holds a HIPAA BAA (available for qualifying healthcare engagements), SOC 2 Type II, and ISO 27001:2022. Healthcare program scoping is discussed during the pilot process at corpshore.ai/start-a-pilot.

What is required for medical imaging annotation?

Medical imaging annotation requires clinical domain expertise (radiologists, pathologists, or trained medical imaging technicians for diagnostic validation), DICOM format handling with PHI header scrubbing, BAA coverage, and audit logging. For FDA SaMD programs, structured annotation provenance records are required as part of the device file.