Workflow pass rate (all repeats pass)

Scenario success across repeated calls.

Who judgesAutomatable
EvidenceGrade E2
Product typeVoice agentsID V1

What workflow pass rate (all repeats pass) means

Scenario success across repeated calls.

Why it matters. A call that works once and fails the next time is not reliable.

Example, illustrative

A scenario passes only if all five repeats succeed.

How to measure it

01

Script scenarios

Cover booking, refills, intake and escalation.

02

Add real conditions

Layer accents, noise and interruptions.

03

Repeat every scenario

A scenario passes only if all repeats pass.

04

Check what was written back

Compare the chart entry with what was said.

Watch out for

A single clean run proves little because outputs vary between calls.

What a good report shows

Pass rates over repeated calls, split by accent and noise, with the number of calls shown.

Report the pass rate across repeated calls (a scenario passes only if every repeat passes), the accents and noise conditions tested, and the number of calls.

Reviewed by the Precision research team.

How strong is the evidence

E2 · Preprint, benchmark page or vendor-run result. Backed by numbers from a preprint, a benchmark page or a vendor-run test. Treat as strong but not settled. At least one figure is a vendor claim.

What the sources say

Cekura Medicare test: ElevenLabs 91.3%, Pipecat 82.6%, LiveKit 73.9%, Synthflow 69.6%, Vapi 65.2%; single scenario, vendor-run (Cekura).

  • Automatable. Can be scored by software, including an AI grader that is spot-checked against clinicians.
  1. Cekurabenchmarks.cekura.ai

Found something out of date? Write to hello@precision.example.

Quick answers

What is workflow pass rate (all repeats pass) in healthcare AI?

Scenario success across repeated calls. It sits in the voice agents group.

Why does workflow pass rate (all repeats pass) matter?

A call that works once and fails the next time is not reliable. A scenario passes only if all five repeats succeed.

How is workflow pass rate (all repeats pass) measured?

Script scenarios, then Add real conditions, then Repeat every scenario, then Check what was written back. Report the pass rate across repeated calls (a scenario passes only if every repeat passes), the accents and noise conditions tested, and the number of calls.

What should I watch out for with workflow pass rate (all repeats pass)?

A single clean run proves little because outputs vary between calls. A good report gives: pass rates over repeated calls, split by accent and noise, with the number of calls shown.

Can workflow pass rate (all repeats pass) be automated?

Largely yes, including AI graders that are spot-checked against clinicians. Report the grader's error rate beside every score.

Measure workflow pass rate (all repeats pass) on your own data