EHR agents quality.
Agents that act inside the patient chart. EHR agents don't just answer.
Next in line: we plan to offer this after the core focus areas.
Where ehr agents breaks.
EHR agents don't just answer. They look things up and act inside the chart: ordering, scheduling, summarising. Mistakes change records.
Wrong data pulled
The agent retrieves the wrong lab or the wrong date and reasons correctly from bad input.
Arithmetic and units
Sums, conversions and aggregations that look right and aren't.
Unapproved actions
A write or order goes through without anyone confirming it.
Three steps, in the open.
FHIR-sandbox replay, action-safety gates, write-back approval policies.
- 01
Run tasks in a FHIR sandbox
Tasks run against a test record, and we check the final record state.
- 02
Separate retrieval from reasoning
We measure which failures come from fetching the wrong data and which from thinking.
- 03
Review every action
We log actions taken and check which required human approval.
FHIR-sandbox replay and action review.
Replay diagnosis, policy compiler.
Who builds it, who buys it.
Epic Agent Factory, Art/Emmie/Penny; health-system-built agents
The stakes
This area is rising fast. How much we can test depends on how open the EHR vendor is.
Rules and reading
Measure ehr agents on your own data
What we measure for ehr agents.
Task completion
Programmatic check of FHIR end state; MedAgentBench 300 tasks; <70% (2025) to >90% (2026) per secondary summary
Retrieval versus reasoning failure share
FHIR-AgentBench: 2,931 real-world questions
Action safety
No unauthorised writes, orders or deletions
Order and medication accuracy
Correct resource, dose, route
Unit and aggregation correctness
Example: 7015 ml vs 7515 ml fluid-output sum (anecdote, vendor blog)
Human-confirmation coverage
Share of writes gated by approval
Common questions.
What is FHIR?
A standard format for sharing health records. Many EHRs expose it, so agents can be tested against it.
How do you stop an agent writing to a real chart?
Testing happens in a sandbox. In production, writes are gated by approval.
Is this offered now?
It's next on our list, behind scribes, voice and coding.
Know your healthcare AI is safe to trust
before it reaches a patient.
Book a 30-minute call. We'll look at your product and tell you what we would measure first.
hello@precision.example