Hallucination rate
Share of sentences or notes with unsupported content; sentence-level clinician annotation against transcript.
What hallucination rate means
Share of sentences or notes with unsupported content; sentence-level clinician annotation against transcript.
Why it matters. Invented content in a note enters the medical record and can mislead later care.
A note records a normal chest exam that never took place.
How to measure it
Collect visits
Use recorded or simulated visits with the source transcript.
Compare note to source
Check every fact in the visit against the note, and every claim in the note against the visit.
Grade severity
Clinicians rank each error by the harm it could do.
Stratify
Split results by specialty, accent and visit type.
Rates from different studies use different definitions and aren't comparable.
A per-note error rate with severity bands, specialty splits and a named review instruction.
Report whether the rate is per sentence or per note, how notes were sampled, which review instruction clinicians were given, and the severity of each error. Rates from different studies are not comparable.
Reviewed by the Precision research team.
How strong is the evidence
E1 · Peer-reviewed, regulator, audit or court source. Backed by a peer-reviewed paper, a regulator, an audit or a court record. Usually read through a summary, not the full text.
What the sources say
CREOLA 1.47% of sentences; JMIR 11.5% of notes; Palm 31% of ambient vs 20% of human notes (Frontiers) (units differ).
- Clinician judgement. Needs a clinician to judge or label, because the answer depends on clinical meaning.
- Automatable. Can be scored by software, including an AI grader that is spot-checked against clinicians.
- Frontiersfrontiersin.org
Found something out of date? Write to hello@precision.example.
Agents that measure this
Quick answers
What is hallucination rate in healthcare AI?
Share of sentences or notes with unsupported content; sentence-level clinician annotation against transcript. It sits in the ambient scribes and clinical documentation group.
Why does hallucination rate matter?
Invented content in a note enters the medical record and can mislead later care. A note records a normal chest exam that never took place.
How is hallucination rate measured?
Collect visits, then Compare note to source, then Grade severity, then Stratify. Report whether the rate is per sentence or per note, how notes were sampled, which review instruction clinicians were given, and the severity of each error. Rates from different studies are not comparable.
What should I watch out for with hallucination rate?
Rates from different studies use different definitions and aren't comparable. A good report gives: a per-note error rate with severity bands, specialty splits and a named review instruction.
Can hallucination rate be automated?
Only in part. It needs clinician judgement and automatable. Software can screen, but a clinician must judge the cases that matter.