Patient chat quality.
Triage advice that fails on real, messy messages. Patient chat and triage tools talk to people who are worried, vague and in a hurry.
Next in line: we plan to offer this after the core focus areas.
Where patient chat breaks.
Patient chat and triage tools talk to people who are worried, vague and in a hurry. They need to spot emergencies and avoid dangerous advice, even when the message is messy.
Misses an emergency
A message hints at stroke or sepsis in everyday words, and the bot suggests rest.
Works in tests, fails with people
In one large trial, people using chatbots chose the right action less than 44% of the time.
Over-reaches
The bot gives dosing or diagnosis advice that it should have handed to a clinician.
Three steps, in the open.
Real-user and simulator tests with a deterministic red-flag escalation gate; disclosure compliance.
- 01
Simulate varied patients
A patient simulator writes the same case in many styles: terse, anxious, wrong terms.
- 02
Probe the red lines
A red-team agent tries emergencies, jailbreaks and out-of-scope requests.
- 03
Check the handoff
A deterministic gate must hand red flags to a human every time, regardless of the model.
Red-team and real-user testing.
Patient simulator, escalation gate.
Who builds it, who buys it.
General chatbots (ChatGPT Health, claimed >230M weekly health askers), symptom checkers, health-system bots
The stakes
These tools may approach medical-device territory in some markets, and a missed emergency is the failure with the highest cost.
Rules and reading
Measure patient chat on your own data
What we measure for patient chat.
Under-triage (missed emergency) and over-triage
Vignettes against clinician gold
Emergency recognition and escalation correctness
HealthBench emergency-referral theme; GPT-4.1 grader macro-F1 0.662 vs physicians 0.647
Real-user task accuracy
Condition identification and correct action with real people
Simulator-to-real gap
Difference between simulated-user and real-user performance
Unsafe-advice rate
Dangerous instructions (ECRI: wrong diagnoses, invented body parts, unsafe electrosurgical electrode placement advice)
Scope-of-practice adherence and appropriate refusal
Declines or hands off when out of scope
Common questions.
Why test with simulated patients?
Real-user trials are slow and costly. Simulators let you test thousands of phrasings first, and we measure the gap between simulated and real results.
What is under-triage?
Under-triage is sending someone to a lower level of care than they need. It's the error to minimise first.
Do you test voice chat as well?
Yes, through the voice-agent tests, which add accents and noise.
Know your healthcare AI is safe to trust
before it reaches a patient.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example