Predictive models quality.
Risk scores that drift without warning. Predictive models score risk: sepsis, readmission, deterioration.
Not offered yet: we are being honest about what we do not do today.
Where predictive models breaks.
Predictive models score risk: sepsis, readmission, deterioration. They're often built once and left running while the patient population and the hospital change.
Drift
Performance falls as patients, coding or workflows change.
Alert fatigue
A low positive predictive value means most alerts are false.
Miscalibrated risk
A 20% score doesn't mean 20% of those patients have the event.
Three steps, in the open.
Local validation and drift (Epic's free seismometer already covers tabular models).
- 01
Validate locally
Check discrimination and calibration on your own patients.
- 02
Measure alert burden
Positive predictive value and alerts per day.
- 03
Watch over time
Track subgroup results and drift.
Local validation and drift checks.
Drift monitor.
Who builds it, who buys it.
Epic Sepsis Model and risk models, Bayesian Health
The stakes
Free tools already serve much of this need, so it isn't our wedge.
Rules and reading
Measure predictive models on your own data
What we measure for predictive models.
Discrimination (AUROC)
Local validation
Calibration
Predicted vs observed risk
PPV and alert burden
7.6% PPV in a county-ED validation
Lead time
Hours of warning before event
Subgroup performance and drift
Per group, over time
Net benefit
Decision-curve analysis
Common questions.
Do you evaluate predictive models?
Not as a focus. We include drift and local validation in other services when they apply.
What is calibration?
How well predicted risk matches what actually happens.
Why does local validation matter?
Joint Commission and CHAI guidance expects it before go-live, because performance elsewhere may not hold in your hospital.
Know your healthcare AI is safe to trust
before it reaches a patient.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example