Clinician reaction elicitor
Turns clinician reactions into written test criteria.
Checked against clinicians
Blind-labelled holdout; second clinician pass on induced criteria.
Where a person decides
Clinicians own the taxonomy; agent proposes.
Works on
All.
Delivered inside a Quality Audit today. Reviewed .
The problem it solves
Teams struggle to write down what 'good' means. Clinicians know a bad note when they see one, but can't always say why. Rubrics invented at a desk miss what actually matters.
How it works
Collect reactions
Clinicians mark real outputs pass or fail and add a short comment.
Find the patterns
The agent groups comments into failure types, such as omission or wrong negation.
Draft criteria
It proposes written criteria, and clinicians edit or reject each one.
An example, illustrative
'This note feels off.'
Criterion: the note must list every allergy mentioned in the visit.
A few dozen real outputs and about an hour of clinician time.
It proposes. Clinicians own the taxonomy and can overrule any criterion.
Quick answers
What does the reaction elicitor do?
Turn clinicians' pass/fail and comments on real outputs into a failure taxonomy and candidate criteria.
How is the reaction elicitor checked against clinicians?
Blind-labelled holdout; second clinician pass on induced criteria.
Why not write the rubric ourselves?
You can, and the agent starts from your draft. Reactions to real outputs usually surface criteria you wouldn't have thought of.
How many reactions are enough?
Around 60 reactions is a useful start. More are added where clinicians disagree.
What can the reaction elicitor not do?
Clinicians own the taxonomy; agent proposes. No agent makes a clinical decision.