Evaluation science
How healthcare AI is actually measured: judges, error rates, severity grading and why the numbers rarely compare.
- Evaluation Why do AI graders miss omissions in clinical notes?Two 2026 preprints find LLM judges near chance at spotting what a clinical note leaves out, yet good at catching invented content. What the numbers say and do not.
- Evaluation Why can't you compare AI scribe error rates between studies?Published scribe error rates run from 1.47% of sentences to 31% of notes. Unit, sampling and review instructions change the answer: what that means for buyers.
- Evaluation Why chatbot benchmark scores miss real-user medical outcomesA 2026 Oxford randomised trial of 1,298 people found chatbot users no better than controls at naming conditions or the right action. What it means for evaluation.