What others are doing, and what they miss.

Sourced analysis of healthcare AI evaluation: who does what, what the evidence supports, and where the gaps are.

Featured · Landscape

Who evaluates healthcare AI in 2026, and what do they miss?

Qualified Health, Vals AI, Alterra Labs, voice-testing vendors, CHAI and Epic: who evaluates healthcare AI today, what each does, and the gaps that remain.

Precision research team · 15 Nov 2026

All posts

  1. Evaluation Why do AI graders miss omissions in clinical notes?Two 2026 preprints find LLM judges near chance at spotting what a clinical note leaves out, yet good at catching invented content. What the numbers say and do not.
  2. Evaluation Why can't you compare AI scribe error rates between studies?Published scribe error rates run from 1.47% of sentences to 31% of notes. Unit, sampling and review instructions change the answer: what that means for buyers.
  3. Landscape Abridge, GAO and Ontario's auditor on evaluating AI scribesAbridge's published method excludes omissions. GAO found few independent accuracy studies. Ontario's auditor found 11 of 20 scribe vendors had no third-party audit.
  4. Landscape Voice AI testing for healthcare: Coval, Hamming, Cekura, BluejayFour funded voice-agent testing vendors, what each does, what they charge where public, and why healthcare call failures still lack independent benchmarks.
  5. Evaluation Why chatbot benchmark scores miss real-user medical outcomesA 2026 Oxford randomised trial of 1,298 people found chatbot users no better than controls at naming conditions or the right action. What it means for evaluation.
  6. Landscape Five gaps in healthcare AI evaluation that nobody fills wellCalibrated graders, causal diagnosis, buyer-run testing, monitoring and language-specific evaluation: the gaps we found in healthcare AI evaluation today.
  7. Evidence What do health-system AI governance committees need from vendors?Local validation, bias testing, PHI safeguards, monitoring plans and audit trails: what buyers say they require, and how little testing capacity they have.