Subgroup performance and drift

Per group, over time.

Who judgesAutomatable
EvidenceGrade E3
Product typePredictive modelsID PM5

What subgroup performance and drift means

Per group, over time.

Why it matters. Populations change and so does performance.

Example, illustrative

A model works for adults but not for a subgroup, and slips over time.

How to measure it

01

Use local data

Validate on your own recent patients.

02

Check discrimination and calibration

Both, not just AUROC.

03

Measure alert burden

Positive predictive value and alerts per day.

04

Monitor over time

Track subgroups and drift after go-live.

Watch out for

A strong AUROC can coexist with poor calibration and heavy alert fatigue.

What a good report shows

Local discrimination, calibration, PPV at your prevalence, and drift tracked over time.

Report discrimination and calibration on local data, the alert rate, positive predictive value, and drift over time.

Reviewed by the Precision research team.

How strong is the evidence

E3 · Secondary or single-source report. Reported by trade press or a single source. Directional only until checked against the primary document.

What the sources say

HTI-1 source attributes (31 for predictive tools); most were removed by the HTI-5 final rule, published 28 October 2026 and effective 28 December 2026 (CASRAI, EBG).

  • Automatable. Can be scored by software, including an AI grader that is spot-checked against clinicians.
  1. CASRAIcasrai.org

Found something out of date? Write to hello@precision.example.

Quick answers

What is subgroup performance and drift in healthcare AI?

Per group, over time. It sits in the predictive models group.

Why does subgroup performance and drift matter?

Populations change and so does performance. A model works for adults but not for a subgroup, and slips over time.

How is subgroup performance and drift measured?

Use local data, then Check discrimination and calibration, then Measure alert burden, then Monitor over time. Report discrimination and calibration on local data, the alert rate, positive predictive value, and drift over time.

What should I watch out for with subgroup performance and drift?

A strong AUROC can coexist with poor calibration and heavy alert fatigue. A good report gives: local discrimination, calibration, PPV at your prevalence, and drift tracked over time.

Can subgroup performance and drift be automated?

Largely yes, including AI graders that are spot-checked against clinicians. Report the grader's error rate beside every score.

Measure subgroup performance and drift on your own data