Gold-set curator and labelling triage
Picks the few cases clinicians most need to label.
Checked against clinicians
Measures labelling time and judge-gain per label.
Where a person decides
Clinicians label; no auto-labels enter gold.
Works on
All.
Delivered inside a Quality Audit today. Reviewed .
The problem it solves
Clinician time is the scarcest resource. Labelling random cases wastes it, because most are easy and tell you nothing new.
How it works
Score each case
The agent estimates which cases the grader is unsure about, which carry high harm and which cover new ground.
Build the queue
Clinicians see the most informative cases first.
Keep the set balanced
It tracks coverage by specialty, language and error type.
An example, illustrative
500 random cases to label.
120 chosen cases that cover the same ground, including every high-severity type.
A pool of unlabelled cases and a clinician reviewer.
It chooses what to label, never the label itself.
Quick answers
What does the gold-set curator do?
Choose which cases clinicians label (uncertain, high-severity, new-specialty); minimise clinician minutes.
How is the gold-set curator checked against clinicians?
Measures labelling time and judge-gain per label.
Does picking cases bias the results?
Chosen cases improve the grader. A separate random holdout is kept for reporting, so headline numbers stay unbiased.
What is a gold set?
A set of cases labelled by clinicians that graders are tested against.
What can the gold-set curator not do?
Clinicians label; no auto-labels enter gold. No agent makes a clinical decision.