# Five gaps in healthcare AI evaluation that nobody fills well

> Calibrated graders, causal diagnosis, buyer-run testing, monitoring and language-specific evaluation: the gaps we found in healthcare AI evaluation today.

By Precision research team, Healthcare AI evaluation research. Published 2026-11-09. Topic: Landscape and gaps. Source: https://precision.example/blog/healthcare-ai-evaluation-gaps-nobody-fills

**TL;DR:** Across vendors, benchmarks and health-system programmes, five gaps recur: graders that report their own error rates, causal diagnosis of which component failed, independent testing on a buyer's own encounters, continuous monitoring after deployment, and evaluation in languages other than English. Existing tools cover tracing, generic scores and benchmarks well.

## Gap 1: Do graders report their own error rates?

Most scores arrive as a single number. In published tests, LLM judges discriminated note completeness at AUC 0.49 to 0.66. A score from a grader with a known miss rate, a confidence interval and an abstain option is more useful than one without. We found no product that reports grader error bars calibrated against expert labels end to end.

## Gap 2: Can anyone say which component failed?

Logs show what happened, not why. Published attribution methods find the faulty step in failed agent runs only 14% to 52% of the time, and we found no healthcare study that separates speech-recognition errors from model or prompt errors in scribes. Swapping one component on a recorded encounter and measuring the change is an experiment, not a guess.

## Gap 3: Who tests your product on your own encounters?

Benchmarks rank models. Vendors grade themselves. Health systems run slow local pilots. Independent, buyer-side testing with the buyer's clinicians as the reference is thinly served.

## Gap 4: Who keeps watching after go-live?

Epic's open-source seismometer covers predictive models. For generative AI, monitoring is largely manual. In the UPMC and KLAS survey, 63% called their strategy developing or ad hoc.

## Gap 5: What about languages other than English?

Clinical benchmarks and voice testing are English-first. Indian languages and code-mixed speech are the clearest example, but the pattern holds across the UK, the Nordics and Germany, where dialects and data-sovereignty rules shape what can be tested.

| Gap | What exists | What is missing |
| --- | --- | --- |
| Grader error rates | Generic LLM judges | Calibration against clinician labels, with error bars |
| Causal diagnosis | Tracing, clustering | Counterfactual replay in healthcare |
| Buyer-side testing | Benchmarks, vendor claims | Independent testing on your encounters |
| Monitoring | Seismometer for predictive models | Generative AI, scribes, voice agents |
| Languages | English benchmarks | Clinician-labelled sets per language |

## What did we not verify?

We did not test every vendor's claims, and "no product found" means not found in our searches. See [who evaluates healthcare AI in 2026](/blog/who-evaluates-healthcare-ai-2026).

## Frequently asked questions

### What is missing in healthcare AI evaluation?

Five things stand out: graders that report their own error rates, causal diagnosis of which part of a system failed, independent testing on a buyer's own data, continuous monitoring after deployment, and evaluation outside English. Tracing and generic scores are widely available.

### Why does continuous monitoring matter?

Vendors update models on their own schedule. In a UPMC and KLAS survey, fewer than half of hospitals had a dedicated AI testing environment and 63% called their approach developing or ad hoc, so a regression can pass unnoticed.

### Is language a real gap?

Yes. On 162 real Indian doctor-patient recordings, the best system's word error rate rose from 14% in English to 35% in Kannada, and standard metrics mis-score mixed-language speech. Few clinical benchmarks cover Indian languages.
