Healthcare AI quality

Know if your healthcare AI
is actually right.

We test your AI scribe, voice agent or clinical chat with your own clinicians. You see what's wrong, why, and which fix works.

Your encounters · Your clinicians · Fixed scope

Quality Audit report AI scribe · primary care · 124 encounters Illustrative sample Fix omissions first
Severity-weighted
score
Reviewed by
3 clinicians
  • Omissions 79%
  • Invented content 92%
  • Negation and laterality 96%
  • Medication and dose 98%
  • Speaker attribution 94%
  • PHI in outputs 100%
Share of encounters with no error of that type · every encounter scoredPass Fix Risk
81%of US physicians used AI in early 2026, up from 38% in 2023
18%of reviewed AI scribe notes left something out
<50%of health systems have a test environment for third-party AI
138ways to measure healthcare AI, defined in our library

Sources: AMA · JMIR Med Inform 2026 · MedCity News

Working with teams building healthcare AI

  • Aldermoor Health
  • Quillon Care
  • Vesper Clinics
  • Tamarind Medical
  • Halcyon Ridge
  • Ostrava Labs
  • Fennel & Pine Health
  • Corvane Diagnostics
The problem

Where healthcare AI quietly goes wrong.

Six patterns from studies, audits and regulators. Each one has a source and a fix.

18%of 356 reviewed notes had omissions

Scribes drop what matters

In one hospital pilot, nearly 1 in 5 AI-written notes left something out.

Source

<44%right action, 1,298 people

Exam scores don't survive real patients

People using chatbots picked the right action less than 44% of the time, no better than people without one.

Source · Older models

14–52%best published accuracy at finding the faulty step

No one knows which part broke

Microphone, transcript, prompt or model? Logs show what happened, not why.

Source · Not healthcare data

>95%claimed, no independent audit

Vendors grade themselves

A '95% accurate' claim for AI coding tools traced back to one vendor's own report.

Source · Secondary summary

63%call their approach ad hoc or still developing

Models change while you sleep

Fewer than half of hospitals have a test environment for the AI they buy.

Source · Small sample

9% → 79%from the instruction alone

Error rates can't be compared

Change the reviewer's instruction and the share of errors that hold up swings from 9% to 79%.

Source · Preprint

Pictures show illustrative data.

See all 8 problems

How it works

From “something's off” to proof it's fixed.

Clinicians decide what right looks like. Precision does the measuring.

  1. Clinicians define what right meansTheir reactions become the rubric

    Clinicians mark real outputs pass or fail and say why. Precision turns that into written criteria.

    Read the evidence
  2. Scores with error barsEvery grader is checked first

    We check each automated grader with your clinicians, then show every score with its range.

    Read the evidence
  3. Re-run any visit or callSame encounter, one change

    Record a visit once, then re-run it as often as you need. Simulated patients fill the gaps.

    Read the evidence
  4. Find the part that brokeSwap one thing, measure the change

    Swap the speech model, prompt, retriever or model on the same visit. Causes come ranked by effect size.

    Read the evidence
  5. Fixes that hold, with a paper trailGates, approvals, ledger

    A fix is accepted only if it holds on new cases. Exact rules become hard checks. Every step is logged.

    Read the evidence
  • Failure types in clinical words
  • Rubrics clinicians can edit
  • A blind set to check against
  • How often each grader is wrong
  • Ranges on every comparison
  • Repeat runs, not one lucky run
  • Reproducible failures
  • Simulators scored for realism
  • No live patient contact
  • Ranked causes
  • Speech vs model vs prompt
  • Evidence an engineer can act on
  • Fixes proven on held-out cases
  • Hard gates for doses and escalation
  • A ledger reviewers can read

The five layers in detail

Solutions

Pick where you want to begin.

ServiceAvailable now

Quality Audit

Your AI, tested on your encounters and graded by your clinicians.

  • Ranked failures
  • Likely causes
  • Tests you keep
Fixed scope · your own dataSee how it works
ServiceAvailable now

Vendor bake-off

Run AI vendors side by side on your own cases, not their demos.

  • Same cases for all
  • Scores with ranges
  • A decision memo
Head to head · on local dataSee how it works
ServiceAvailable now

Drift monitoring

Find out within days when a model update makes things worse.

  • Scheduled test runs
  • Alerts that mean something
  • A readable change log
Monthly · repeated trialsSee how it works
ServiceEarly access

Evidence pack

The proof that security and clinical-safety reviews ask for.

  • Validation summary
  • Bias results
  • Monitoring plan
For governance and procurementSee how it works
ProductComing soon

Precision Platform

The whole loop as software, for teams that outgrow audits.

  • Clinician review queue
  • Scores with error bars
  • Evidence ledger
Self-serve · clinician review queueSee how it works
ProductFree tool

Precision De-ID

Mask identifiers in clinical text on your own machine.

  • Runs locally
  • Apache-2.0 models
  • You review the output
Free · runs on your machineSee how it works

All 8 solutions

Product map

Whatever you're building, here's what we do.

Each kind of healthcare AI breaks differently. Pick yours.

Core focus

AI scribes

Notes that miss allergies, doses or decisions.

ServiceProduct
Core focus

Voice agents

Calls that fail and nobody knows why.

ServiceProduct
Core focus

Coding and billing

Codes that cost you denials or audits.

ServiceProduct
Next

Clinical Q&A

Fluent answers built on stale or wrong guidance.

ServiceProduct
Next

Patient chat

Triage advice that fails on real, messy messages.

ServiceProduct
Next

EHR agents

Agents that act inside the patient chart.

ServiceProduct

All 12 product types

Questions

Straight answers.

More? hello@precision.example

What does Precision do?

We test healthcare AI such as scribes, voice agents and clinical chat with your own clinicians, show what is wrong and why, and prove each fix holds. Today that is a fixed-scope Quality Audit. A self-serve platform is coming.

How is this different from a benchmark like HealthBench?

A benchmark ranks models on fixed questions. Precision tests your product on your encounters, with your clinicians as the reference, and shows how often our own graders are wrong.

Why not let an AI judge grade the notes?

Generic AI judges catch invented content but miss omissions, scoring near chance in published tests. We calibrate graders against clinician labels and report their error rates.

Do you need patient data?

An audit uses real encounters, so we agree the data route first: de-identified files, or tests run inside your environment where only metrics leave. We work under your agreements.

Is Precision a medical device?

No. We check another party's product and never make or change clinical decisions or contact patients. A named person approves every change. Your counsel decides how rules apply to your use.

Where do you start?

AI scribes, healthcare voice agents, and coding and prior-authorisation tools, where errors are measurable and buyers lack a trusted number. Imaging is partner-only.

Know your healthcare AI is safe to trust
before it reaches a patient.

Book a 30-minute call. We'll look at your product and tell you what we would measure first.