Automation bias and deskilling

Performance of unaided clinicians after AI exposure.

Who judgesReal-world outcomes
EvidenceGrade E1/E3
Product typeCross-cuttingID X20

What automation bias and deskilling means

Performance of unaided clinicians after AI exposure.

Why it matters. Reliance on AI can erode skills, and users may accept wrong output without checking.

Example, illustrative

Clinicians perform worse unaided after months of relying on an alert.

How to measure it

01

Define the unit

Decide whether you count per sentence, per note, per patient or per call.

02

Sample on purpose

Pick cases that cover specialties, languages and rare events, not only easy ones.

03

Judge with clinicians

Clinicians label a sample. Graders are then checked against those labels.

04

Report the margin

Show an interval, and the grader's own error rate beside the score.

Watch out for

A single average hides rare, serious errors and weaker subgroups.

What a good report shows

A number with a clear unit, a stated sample, a margin and the grader's agreement with clinicians.

Report the unit (per sentence, per note, per patient), the sampling method, the severity scale, and how often the automated grader agreed with clinicians compared with how often clinicians agreed with each other.

Reviewed by the Precision research team.

How strong is the evidence

E1 · Peer-reviewed, regulator, audit or court source. Backed by a peer-reviewed paper, a regulator, an audit or a court record. Usually read through a summary, not the full text.

What the sources say

Adenoma detection 28.4% to 22.4% without AI after exposure (observational, four Polish centres) (Warwick); 88% of physicians concerned (AMA).

  • Real-world outcomes. Needs outcome data from live use, such as time saved, burnout or patient results.
  1. Warwickwrap.warwick.ac.uk

Found something out of date? Write to hello@precision.example.

Quick answers

What is automation bias and deskilling in healthcare AI?

Performance of unaided clinicians after AI exposure. It sits in the every product type group.

Why does automation bias and deskilling matter?

Reliance on AI can erode skills, and users may accept wrong output without checking. Clinicians perform worse unaided after months of relying on an alert.

How is automation bias and deskilling measured?

Define the unit, then Sample on purpose, then Judge with clinicians, then Report the margin. Report the unit (per sentence, per note, per patient), the sampling method, the severity scale, and how often the automated grader agreed with clinicians compared with how often clinicians agreed with each other.

What should I watch out for with automation bias and deskilling?

A single average hides rare, serious errors and weaker subgroups. A good report gives: a number with a clear unit, a stated sample, a margin and the grader's agreement with clinicians.

Can automation bias and deskilling be automated?

Only in part. It needs real-world outcomes. It depends on data from real use, not a test set.

Measure automation bias and deskilling on your own data