Scribes drop what matters
In one hospital pilot, nearly 1 in 5 AI-written notes left something out.
We test your AI scribe, voice agent or clinical chat with your own clinicians. You see what's wrong, why, and which fix works.
Your encounters · Your clinicians · Fixed scope
Sources: AMA · JMIR Med Inform 2026 · MedCity News
Working with teams building healthcare AI
Six patterns from studies, audits and regulators. Each one has a source and a fix.
In one hospital pilot, nearly 1 in 5 AI-written notes left something out.
People using chatbots picked the right action less than 44% of the time, no better than people without one.
Source · Older models
Microphone, transcript, prompt or model? Logs show what happened, not why.
Source · Not healthcare data
A '95% accurate' claim for AI coding tools traced back to one vendor's own report.
Source · Secondary summary
Fewer than half of hospitals have a test environment for the AI they buy.
Source · Small sample
Change the reviewer's instruction and the share of errors that hold up swings from 9% to 79%.
Source · Preprint
Pictures show illustrative data.
Clinicians decide what right looks like. Precision does the measuring.
Clinicians define what right meansTheir reactions become the rubric
Clinicians mark real outputs pass or fail and say why. Precision turns that into written criteria.
Read the evidenceScores with error barsEvery grader is checked first
We check each automated grader with your clinicians, then show every score with its range.
Read the evidenceRe-run any visit or callSame encounter, one change
Record a visit once, then re-run it as often as you need. Simulated patients fill the gaps.
Read the evidenceFind the part that brokeSwap one thing, measure the change
Swap the speech model, prompt, retriever or model on the same visit. Causes come ranked by effect size.
Read the evidenceFixes that hold, with a paper trailGates, approvals, ledger
A fix is accepted only if it holds on new cases. Exact rules become hard checks. Every step is logged.
Read the evidence
Clinician reactions · Illustrative
Note omits the penicillin allergy discussedFail
Plan matches what was agreedPass
"No chest pain" recorded as "chest pain"Fail
Failure types induced
60 reactions
Grader vs clinicians · Illustrative
The grader's own miss rate is reported next to every score.
Rate with 95% interval
Repeated runs
Recorded encounter #0412 · Illustrative
Fitted caller and patient simulators fill the gaps real data cannot.
Same encounter, re-run
No live patientrun 1OriginalOmits allergyrun 2New speech modelOmits allergyrun 3New promptCaptured
Why omissions happen · Illustrative
Causes come from experiments on the same input, not from reading logs.
Ranked causes
Effect on omission rate
Hard gates · Illustrative
Doses, escalation and policy rules are checked on the action, not left to the model.
Evidence ledger
Every change recordedfix 14Prompt v7, held-out +9 ptsApprovedfix 15Retriever updateAwaiting reviewfix 16Model swapRejected
Your AI, tested on your encounters and graded by your clinicians.
Run AI vendors side by side on your own cases, not their demos.
Find out within days when a model update makes things worse.
The proof that security and clinical-safety reviews ask for.
The whole loop as software, for teams that outgrow audits.
Mask identifiers in clinical text on your own machine.
Each kind of healthcare AI breaks differently. Pick yours.
Notes that miss allergies, doses or decisions.
Calls that fail and nobody knows why.
Codes that cost you denials or audits.
Fluent answers built on stale or wrong guidance.
Triage advice that fails on real, messy messages.
Agents that act inside the patient chart.
Each one explained in plain words: what it means, how to measure it, and what the evidence says.
Relevant transcript content missing from note
Share of sentences or notes with unsupported content; sentence-level clinician annotation against transcript
Major/minor (could the error change diagnosis or management if uncorrected), then none/minor/major/critical and reached-patient status
PHI present in outputs, logs, prompts, third-party calls
Metric shift after model, prompt, ASR or EHR changes; black-box drift tests
Error rates by race, language, gender, age, dialect, literacy
They measure, find causes and block. A person approves every change.
Finds what the note left out.
Finds claims the source never supports.
Re-runs a failure with one part swapped to find the cause.
Plays real patients, with messy wording.
Forces a handoff to a human when red flags appear.
Assembles results into the documents reviewers ask for.
We track the rules. Each card opens a sourced page that says what we couldn't verify.
Single-recommendation tools get more room. AI gets no new rule.
A government benchmark for health AI, run with IIT Kanpur.
HTI-5 removes federal source-attribute duties. Buyers must ask for them.
Consent, security and breach rules reach health data.
Standalone high-risk AI under the AI Act.
AI built into regulated devices joins the AI Act.
We test healthcare AI such as scribes, voice agents and clinical chat with your own clinicians, show what is wrong and why, and prove each fix holds. Today that is a fixed-scope Quality Audit. A self-serve platform is coming.
A benchmark ranks models on fixed questions. Precision tests your product on your encounters, with your clinicians as the reference, and shows how often our own graders are wrong.
Generic AI judges catch invented content but miss omissions, scoring near chance in published tests. We calibrate graders against clinician labels and report their error rates.
An audit uses real encounters, so we agree the data route first: de-identified files, or tests run inside your environment where only metrics leave. We work under your agreements.
No. We check another party's product and never make or change clinical decisions or contact patients. A named person approves every change. Your counsel decides how rules apply to your use.
AI scribes, healthcare voice agents, and coding and prior-authorisation tools, where errors are measurable and buyers lack a trusted number. Imaging is partner-only.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example