Imaging quality.
Models that slip when the scanner or site changes. Imaging AI reads scans.
Partner pilots only: this needs radiologist-labelled local data, so we work with a clinical partner.
Where imaging breaks.
Imaging AI reads scans. Performance can fall when a hospital uses a different scanner, protocol or patient mix from the one the model learned on.
Site shift
A model that works at one hospital slips at another.
Hidden subgroup gaps
Average accuracy hides weaker results for some patients.
Too many false alarms
Alert burden tires radiologists and they start to ignore flags.
Three steps, in the open.
Local validation and drift monitoring; but regulated, reader-study-centred, and the LLM-judge loop barely applies.
- 01
Validate locally
Run the model on your scans with radiologist-adjudicated labels.
- 02
Break results down
Report by scanner, site and patient group, with calibration.
- 03
Track change
Re-test after software updates or protocol changes.
Method work with a radiology partner.
Local validation (with partner).
Who builds it, who buys it.
~1,600 FDA-listed AI devices, ~76% radiology; Aidoc ($150M Series E, Apr 2026)
The stakes
This is a separate regulatory world built around reader studies. We work on methods with a radiology partner rather than selling it as a service today.
Rules and reading
Measure imaging on your own data
What we measure for imaging.
Sensitivity, specificity, AUROC
Against reference standard
PPV and NPV at deployment prevalence
Local prevalence adjustment
Calibration
Predicted vs observed
Subgroup performance tables and model cards
FDA draft lifecycle guidance expects them; finalisation is on the FY2027 agenda
Out-of-distribution scanner and site failure
Performance on new devices and protocols
Reader-study uplift and turnaround
Radiologist accuracy and time with and without AI
Common questions.
Do you evaluate imaging AI now?
Only in partner pilots with radiologists. It needs radiologist-labelled local data.
Why not offer it widely?
The evidence standards differ from text and voice tools, and we'd rather be honest about what we don't cover.
What would a pilot look like?
A local validation on a few hundred scans with subgroup tables and calibration plots.
Know your healthcare AI is safe to trust
before it reaches a patient.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example