Repair agent (evaluator-bounded)
Proposes fixes, accepted only if they hold on new cases.
Checked against clinicians
Held-out clinician labels; checks for judge overfit.
Where a person decides
Human approval; canary and rollback; PCCP-style change record.
Works on
Scribes, Q&A, chat, voice.
On the roadmap. Tell us if you need it sooner. Reviewed .
The problem it solves
Fixing a prompt for one failure can break three others. Fixes need proof, not hope.
How it works
Propose a fix
The agent suggests a prompt, retrieval or routing change for a diagnosed cause.
Test on held-out cases
The fix is judged on cases it hasn't seen.
Accept only if it holds
It's accepted only if it improves the target and doesn't harm other scores.
An example, illustrative
Allergy omitted in 23% of notes.
Prompt v7: 14%, held-out. No regressions. Awaiting approval.
A diagnosis and held-out cases from earlier layers.
Independent evidence for automated repair is thin. A person approves every change.
What it helps measure
Quick answers
What does the repair agent do?
Propose prompt, retrieval or routing fixes; accept only if held-out clinician-labelled results hold within judge error bounds.
How is the repair agent checked against clinicians?
Held-out clinician labels; checks for judge overfit.
Does it change production by itself?
Never. A named human approves each change.
What are held-out cases?
Cases kept aside so the fix can't be tuned to them.
What can the repair agent not do?
Human approval; canary and rollback; PCCP-style change record. No agent makes a clinical decision.