Gold-set curator and labelling triage

Picks the few cases clinicians most need to label.

In audits Layer 1 · Specify
Takes inProduction samples, judge uncertainty
The agentGold-set curator
Hands backLabelling queues, stratified gold set

Checked against clinicians

Measures labelling time and judge-gain per label.

Where a person decides

Clinicians label; no auto-labels enter gold.

Works on

All.

Delivered inside a Quality Audit today. Reviewed .

The problem it solves

Clinician time is the scarcest resource. Labelling random cases wastes it, because most are easy and tell you nothing new.

How it works

01

Score each case

The agent estimates which cases the grader is unsure about, which carry high harm and which cover new ground.

02

Build the queue

Clinicians see the most informative cases first.

03

Keep the set balanced

It tracks coverage by specialty, language and error type.

An example, illustrative

Before

500 random cases to label.

With the agent

120 chosen cases that cover the same ground, including every high-severity type.

What it needs from you

A pool of unlabelled cases and a clinician reviewer.

Its limits

It chooses what to label, never the label itself.

Quick answers

What does the gold-set curator do?

Choose which cases clinicians label (uncertain, high-severity, new-specialty); minimise clinician minutes.

How is the gold-set curator checked against clinicians?

Measures labelling time and judge-gain per label.

Does picking cases bias the results?

Chosen cases improve the grader. A separate random holdout is kept for reporting, so headline numbers stay unbiased.

What is a gold set?

A set of cases labelled by clinicians that graders are tested against.

What can the gold-set curator not do?

Clinicians label; no auto-labels enter gold. No agent makes a clinical decision.

Use the gold-set curator on your product