Harm severity grade

Major/minor (could the error change diagnosis or management if uncorrected), then none/minor/major/critical and reached-patient status.

Who judgesClinician judgement
EvidenceGrade E1
Product typeCross-cuttingID X3

What harm severity grade means

Major/minor (could the error change diagnosis or management if uncorrected), then none/minor/major/critical and reached-patient status.

Why it matters. Counting errors treats a typo and a missed allergy the same. Severity tells you what to fix first.

Example, illustrative

Fourteen errors in a sample: two could change management, twelve could not. The two get fixed first.

How to measure it

01

Define the unit

Decide whether you count per sentence, per note, per patient or per call.

02

Sample on purpose

Pick cases that cover specialties, languages and rare events, not only easy ones.

03

Judge with clinicians

Clinicians label a sample. Graders are then checked against those labels.

04

Report the margin

Show an interval, and the grader's own error rate beside the score.

Watch out for

A single average hides rare, serious errors and weaker subgroups.

What a good report shows

A number with a clear unit, a stated sample, a margin and the grader's agreement with clinicians.

Report the unit (per sentence, per note, per patient), the sampling method, the severity scale, and how often the automated grader agreed with clinicians compared with how often clinicians agreed with each other.

Reviewed by the Precision research team.

How strong is the evidence

E1 · Peer-reviewed, regulator, audit or court source. Backed by a peer-reviewed paper, a regulator, an audit or a court record. Usually read through a summary, not the full text.

What the sources say

CREOLA major/minor (npj Digit Med); E1. NCC MERP and AHRQ Harm Scale mapping is E4; no published mapping to LLM errors found.

  • Clinician judgement. Needs a clinician to judge or label, because the answer depends on clinical meaning.
  1. npj Digit Medpmc.ncbi.nlm.nih.gov

Found something out of date? Write to hello@precision.example.

Quick answers

What is harm severity grade in healthcare AI?

Major/minor (could the error change diagnosis or management if uncorrected), then none/minor/major/critical and reached-patient status. It sits in the every product type group.

Why does harm severity grade matter?

Counting errors treats a typo and a missed allergy the same. Severity tells you what to fix first. Fourteen errors in a sample: two could change management, twelve could not. The two get fixed first.

How is harm severity grade measured?

Define the unit, then Sample on purpose, then Judge with clinicians, then Report the margin. Report the unit (per sentence, per note, per patient), the sampling method, the severity scale, and how often the automated grader agreed with clinicians compared with how often clinicians agreed with each other.

What should I watch out for with harm severity grade?

A single average hides rare, serious errors and weaker subgroups. A good report gives: a number with a clear unit, a stated sample, a margin and the grader's agreement with clinicians.

Can harm severity grade be automated?

Only in part. It needs clinician judgement. Software can screen, but a clinician must judge the cases that matter.

Measure harm severity grade on your own data