Healthcare
Your AI now triages, documents, communicates and decides coverage in your name, and you remain answerable for what it does. We show you how those systems can fail, what those failures can reach, and give you evidence you can act on before it becomes consequence.
Cases
Each case is one concrete AI system inside a health organisation, written up as it runs: what it decides, how it can fail, and what reaches the person held responsible for it.
In preparation
The processes we are writing up next. Each already runs on the platform; the case is what is being written.
Clinical decision support
Diagnostic accuracy across patient populations, bias in treatment recommendations, and whether the clinical reasoning behind a recommendation can be traced.
Proof
Patient communications at scale
PHI exposure, health literacy, medical accuracy and consent across AI-generated reminders, results notifications and care instructions.
Proof
Medical device AI across its lifecycle
Performance drift, algorithm change impact and cross-population robustness, with the per-run readiness evidence regulators ask for.
Proof
Prior authorisation and claims adjudication
Denial patterns across demographics, explainability of coverage decisions, and meaningful human oversight over automated ones.
Proof
Clinical documentation and coding
Hallucinated medical information in AI-assisted notes, coding consistency, and a registered, hash-citable audit trail over every assisted document.
Proof
Whether you operate in Berlin, Boston or São Paulo, what supervisors ask for converges on the same object: defensible evidence of how your systems behave, fail and affect the people around them.
European Union
Medical device AI is classified as high-risk under Article 6(1)(b). Articles 9–15 apply from August 2027, and notified bodies will run integrated assessments covering MDR/IVDR and the AI Act at once.
Maximum penalty: EUR 35 million or 7% of global turnover.
United States
Over 1,250 AI medical devices are FDA-authorized. Lifecycle guidance requires ongoing bias monitoring rather than a one-time submission. The HIPAA Security Rule update mandates AI system inventories and encrypted processing. The Colorado AI Act takes effect June 2026.
HIPAA penalty: $1.5M–$15M per violation category.
Latin America
Health AI is classified as high-risk under Brazil's AI regulation bill. ANVISA and Mexico's COFEPRIS signed mutual recognition for AI medical devices in August 2025. LGPD classifies health data as sensitive, requiring explicit consent.
LGPD penalty: up to 2% of Brazilian revenue.
Documented failures
These are documented failures, and they are shaping healthcare AI regulation worldwide.
CLINICAL BIAS
Widely deployed sepsis prediction tools showed a 67% false alert rate and significant performance degradation across racial groups, driving patient harm, wasted clinical resources and regulatory investigations into algorithmic bias.
Dimension: Fairness & Non-Discrimination
DIAGNOSTIC INEQUITY
Diagnostic AI trained predominantly on lighter skin tones showed 15–20% accuracy drops on darker skin, leading to delayed or missed diagnoses for underserved populations.
Dimension: Robustness & Adversarial Resilience
DOCUMENTATION HALLUCINATION
Multiple health systems have reported hallucinated medical information appearing in AI-assisted clinical documentation, creating malpractice liability and patient safety risk.
Dimension: Factuality & Accuracy
AUTOMATION BIAS
AI-driven prior authorization systems have been shown to deny claims at higher rates in lower-income ZIP codes, triggering regulatory investigations in multiple US states.
Dimension: Accountability & Human Oversight
How the evidence is classified
Our classification layer is framework-neutral and was developed by PhD researchers, documented in full, with external review under way through university partnerships, and operationalized as 39+ quantitative metrics. Evidence is stored and reasoned about first; frameworks read it afterwards. Every dimension is reported as an occurrence rate over N runs with a coverage report — never a score.
01
Demographic bias in clinical algorithms, diagnostic accuracy gaps across patient populations, discriminatory patterns in resource allocation.
02
Unsafe or harmful content in AI-generated patient communications, clinical summaries and patient-facing health information, including unsafe clinical advice.
03
Whether a clinical recommendation carries traceable reasoning — which inputs contributed, what thresholds applied, and why.
04
PHI exposure in AI outputs, minimum-necessary enforcement, and leakage of sensitive health data across processing boundaries.
05
Hallucinated medical information, clinical claims held against medical knowledge, and outputs that contradict established evidence.
06
Stability under adversarial inputs, data drift and edge cases — where a system degrades on rare presentations and unexpected conditions.
07
Prompt injection, PHI exfiltration and privilege-boundary failures — whether a system can be steered past the requesting clinician's authorization.
08
Whether a clinician can meaningfully review, override and remain answerable for AI-assisted diagnoses, triage and coverage decisions.
Regulatory projection
Each obligation below connects to the behavioural class where the evidence for it is produced. The frameworks read the evidence; they do not decide what we are able to observe.
EU AI Act (2024/1689)
High-risk conformity under Art 9–15 for clinical AI classified under Art 6(1)(b)
August 2027
Transparency & Explainability
EU MDR/IVDR
Clinical evaluation and integrated notified-body assessment alongside the AI Act
In force
Factuality & Accuracy
HIPAA Security Rule
AI system inventory, encrypted processing, minimum-necessary handling of PHI
In force
Privacy & Data Protection
FDA PCCP guidance
Predetermined change control and ongoing bias monitoring across the product lifecycle
In force
Robustness & Adversarial Resilience
Colorado AI Act
Duty of care and impact assessment for consequential health decisions
June 2026
Fairness & Non-Discrimination
Brazil LGPD + ANVISA RDC 751
Explicit consent for health data; device authorization with COFEPRIS mutual recognition
In force
Security & Access Control
Framework versions are recorded with every analysis, so a readiness statement can always name which edition of which framework it was produced against.
We attack the clinical AI that acts in your name, measure how it behaves, and preserve what happened. Observe, verify, preserve the evidence — and hand the decision to the people accountable for it.
Related industries