AI scribes quality.
Notes that miss allergies, doses or decisions. An AI scribe listens to the visit and writes the note.
A core focus: we deliver this as a service today.
Where ai scribes breaks.
An AI scribe listens to the visit and writes the note. That note enters the medical record, drives billing and guides the next clinician. A quiet mistake travels a long way.
It leaves things out
An allergy mentioned in passing, a follow-up plan, a family history detail. Omissions are the most common error in published scribe reviews.
It invents things
A physical exam that never happened, or a symptom the patient denied. Invented content reads fluently, so reviewers miss it.
It flips meaning
'No chest pain' becomes 'chest pain'. A dose changes, or left becomes right. Small edits, serious consequences.
Three steps, in the open.
Independent, local-data vendor bake-offs; omission/hallucination audits; ASR-vs-LLM attribution; monitoring after model updates.
- 01
Replay real visits
We run recorded or realistic simulated visits through the scribe.
- 02
Compare note to source
An omission auditor lists every fact in the visit and checks the note. A second agent checks every claim in the note against the visit.
- 03
Clinicians rank the errors
Your clinicians grade severity, so a missed allergy isn't counted the same as a missed pleasantry.
Audit, bake-off, drift monitoring.
Omission and hallucination auditors.
Who builds it, who buys it.
Abridge ($5.3B, >90 disclosed customers, UPMC 12,000+ clinicians), Ambience (~$1.25B), Nabla, Suki, Microsoft Dragon Copilot (>100,000 clinicians), Epic native AI Charting (released Feb 2026). Buyers: health systems
The stakes
Scribe spend is large and growing, and buyers lack a shared metric. Rates reported by different studies aren't comparable, so each product needs testing on your own visits.
Rules and reading
Measure ai scribes on your own data
What we measure for ai scribes.
Hallucination rate
Share of sentences or notes with unsupported content; sentence-level clinician annotation against transcript
Omission rate
Relevant transcript content missing from note
Accidental inclusion
Off-topic or other-person content entered
Error severity (major/minor)
Could it change diagnosis or management
PDQI-9 / modified PDQI-9
Nine to eleven criteria on 5-point Likert (accurate, thorough, organised, succinct, internally consistent)
Targeted error classes: attribution, negation, laterality, medication and dose
Test sets built to provoke each class
Agents we use here.
Omission auditor
Finds what the note left out.
Hallucination auditor
Finds claims the source never supports.
Dose and negation checker
Catches wrong doses, flipped negations and left-versus-right slips.
Replay diagnostician
Re-runs a failure with one part swapped to find the cause.
Common questions.
How often do AI scribes make mistakes?
It depends on the study and the definition. One hospital review found about 18% of 356 notes had an omission. Different studies count differently, which is why you should test on your own visits.
What should I measure first?
Start with omissions and invented content, graded by severity. Then check high-harm classes such as negation, laterality and medication doses.
Do you test the vendor's product directly?
Yes. We only need the same inputs and outputs a clinician sees: the audio in and the note out.
Know your healthcare AI is safe to trust
before it reaches a patient.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example