# Who evaluates healthcare AI in 2026, and what do they miss?

> Qualified Health, Vals AI, Alterra Labs, voice-testing vendors, CHAI and Epic: who evaluates healthcare AI today, what each does, and the gaps that remain.

By Precision research team, Healthcare AI evaluation research. Published 2026-11-15. Topic: Landscape and gaps. Source: https://precision.example/blog/who-evaluates-healthcare-ai-2026

**TL;DR:** Healthcare AI evaluation is split across governance platforms (Qualified Health), benchmark publishers (Vals AI, HealthBench), simulation environments (Alterra Labs), horizontal voice-testing tools and free validation software for predictive models. We found no dedicated, funded evaluator of deployed AI scribes, and no one measuring a product on a health system's own encounters with calibrated error bars.

## Who builds evaluation for healthcare AI?

Six kinds of player cover parts of the problem. None covers all of it.

| Who | What they do | Evidence of scale | Limit |
| --- | --- | --- | --- |
| Qualified Health | Governance and deployment platform for health systems | $125M Series B, March 2026 (NEA); about $155M total; valuation press-estimated | Positioned as a platform, not an independent measurement lab; pricing not public |
| Vals AI | Benchmarks, including MedScribe and MedCode (February 2026) | $40M Series A at $400M, 13 August 2026 (a16z) | Benchmarks rank models; they do not test your product |
| Alterra Labs | Simulated medical environments scored by clinicians | $12.5M seed, 10 November 2026 (Oak HC/FT) | Customers are mostly frontier AI labs |
| Voice-testing vendors (Coval, Hamming, Cekura, Bluejay) | Simulation and monitoring for voice agents | Coval $28M Series A, June 2026 | Horizontal tools; healthcare is a use case, and failure rates are vendor-run |
| Epic seismometer | Open-source validation of predictive models | 272 GitHub stars | Tabular models; generative AI not covered at launch |
| Health-system in-house programmes | Local validation by informatics teams | Stanford's FURM is documented | Slow, expensive and not shared |

## Which attempts at third-party assurance failed?

CHAI planned a network of assurance labs. Fierce Healthcare reports the plan was scrapped. The Joint Commission's 2026 certification assesses an organisation's governance, not a product. Third-party validation of the tools themselves remains open.

## What do horizontal evaluation tools miss in healthcare?

General LLM evaluation platforms gate HIPAA agreements, private hosting and data residency behind enterprise tiers, according to third-party comparison write-ups we read. They also measure generic quality, not omissions, negation or dose errors in clinical notes.

## Where are the gaps?

Five stand out: calibrated graders that report their own error rates; causal diagnosis of which component failed; independent testing on a buyer's own encounters; continuous monitoring of scribes, voice agents and other agents after deployment; and language-specific evaluation outside English.

## What did we not verify?

Qualified Health's valuation is a press estimate. Vals AI's customers come from a secondary source. We did not search every vendor, and pricing for most evaluation services is not public.

## Frequently asked questions

### Is there a company that independently evaluates AI scribes?

We did not find a dedicated, funded healthcare scribe-evaluation startup. Vals AI publishes a MedScribe benchmark, and health systems run their own pilots, but independent evaluation of deployed scribes on a buyer's own encounters is largely unserved.

### What does Qualified Health do?

Qualified Health raised a $125M Series B led by NEA in March 2026, about $155M in total. Press calls it AI evaluation, but it positions itself as a platform to build, deploy and govern generative AI in health systems. Pricing is not public.

### Did CHAI's assurance labs work?

No. The plan for a network of assurance labs was scrapped, according to Fierce Healthcare, and replaced by assurance resource providers. CHAI's model-card registry continues, and the Joint Commission's certification covers organisations, not products.
