← Back to insights AI in Medicine

How Accurate Are AI Medical Scribes? What the Studies Actually Measured

AI medical scribe accuracy cannot be reduced to one percentage. Studies measure different things—from transcription hallucinations and omissions to semantic agreement, note quality and clinician review. This blog explains why the numbers differ and what healthcare teams should actually evaluate before deployment.

Kamna Singh Thakur • Sep 29, 2026 • 14 min read

How Accurate Are AI Medical Scribes? What the Studies Actually Measured

There is no defensible single accuracy rate for AI medical scribes.

Published studies report very different numbers because they are measuring different things.

One study found fabricated speech-to-text content in roughly 1% of audio segments. Another found hallucinations in 31% of complete AI-generated clinical notes. A new September 2026 study reported 89.3% semantic agreement across a large routine-care dataset—but semantic agreement is not the same thing as clinical accuracy.

Those figures should not be placed on one scale.

The useful question is not simply:

“How accurate are AI medical scribes?”

It is:

Accurate compared with what, measured how, in which setting, for which clinical task, and judged by whom?

Seven studies, seven different measurements

Study	What was measured	Sample	Main finding
Koenecke et al., ACM FAccT 2024	Fabricated content in Whisper speech recognition	13,140 short speech segments	Hallucinated content in roughly 1% of segments
Taylor et al., JMIR Medical Informatics, April 2026	Physician-rated errors in real clinical notes	356 reviewed notes from 7,545 generated	18% omissions, 11.5% hallucinations, 9.3% accidental inclusions
Anderson et al., Mayo Clinic Proceedings: Digital Health, December 2025	Note-level errors across five platforms	14 simulated encounters	Mean clinical-element error rate 26.3%
Palm et al., Frontiers in AI, 2025	Hallucinations and documentation quality	97 paired encounters	31% hallucinations in AI notes vs 20% in physician notes
Blease et al., BMJ Health & Care Informatics, 2026	Clinician-reported errors during routine UK use	1,003 GPs; 141 current users	32% of users reported errors often/always; 14% reported significant-to-critical implications
Groote Schuur Hospital — preprint	AI notes compared with handwritten documentation	49 encounters	Moderate-to-severe errors reported at least fivefold more often in handwritten notes; preliminary evidence
Álvaro de la Parra et al., Scientific Reports, September 2026	Semantic agreement, readability and structured documentation quality	>2.3 million outpatient consultations across 45 specialties; quality sample of 300 reports	Weighted semantic agreement 89.3%; structured quality scores improved from 5.79 to 8.02 with scribe support

Seven studies.

Seven measurement approaches.

They do not produce seven competing estimates of the same thing.

Why the numbers diverge

The definition of “error” does most of the work

Koenecke and colleagues evaluated speech recognition.

Their question was essentially:

Did the transcription system create words that were not spoken?

That is important.

But it is not an end-to-end clinical-note accuracy measure.

An ambient documentation workflow contains several stages:

conversation → speech recognition → interpretation → note generation → clinician review

An error can enter at any one of them.

Taylor and colleagues evaluated complete clinical notes rather than raw transcription. Their framework separated:

accidental omissions;
hallucinations;
accidental inclusions;
bias.

In 356 reviewed notes, omissions were the most common problem at 18%, followed by hallucinations at 11.5% and accidental inclusions at 9.3%.

The Mayo platform comparison used another method again: case-specific clinical elements were scored as correct, omitted, partially correct or incorrectly included.

Across five platforms, the mean error rate was 26.3%.

Different definitions produce different numbers.

That is expected.

The new September 2026 evidence makes the measurement problem even clearer

A large peer-reviewed Spanish study published in Scientific Reports on 23 September 2026 adds a different kind of evidence.

Researchers analysed more than 2.3 million outpatient consultations across 45 specialties between October 2024 and December 2025.

Instead of producing a conventional hallucination or omission rate, the study evaluated:

semantic agreement;
readability;
structured documentation quality;
selected safety-related documentation fields.

The AI-supported notes showed a weighted mean semantic agreement of 89.3%.

Readability scores improved, and structured quality scores increased from 5.79 to 8.02 in the evaluated sample. The manual audit also reported better documentation of past medical history and allergy information.

But:

89.3% semantic agreement does not mean “the scribe is 89.3% clinically accurate.”

Semantic similarity measures how closely the content aligns in meaning.

It does not, by itself, answer whether:

a medication instruction was omitted;
negation changed;
laterality was wrong;
the wrong speaker was attributed;
an unsupported diagnosis appeared;
a clinically important qualification disappeared.

The study therefore strengthens the central argument of this article:

quality, similarity, completeness and clinical accuracy overlap—but they are not interchangeable measurements.

Omissions matter as much as hallucinations

“Hallucination” receives most of the attention because it is easy to understand:

the system wrote something that was not supported.

Omission is quieter.

The system fails to include something that was present.

In the UC Davis study:

Error type	Notes affected
Accidental omission	18%
Hallucination	11.5%
Accidental inclusion	9.3%
Bias	1.1%

A missing allergy, medication instruction, negative finding or follow-up condition can matter even when every sentence that remains is factually correct.

That is why asking only:

“Does the scribe hallucinate?”

produces an incomplete evaluation.

The product matters as much as the technology

The Mayo study is particularly useful because five ambient-scribe platforms processed the same fourteen simulated encounters.

Mean error across the platforms was 26.3%.

But the stronger finding was variability.

Of the clinical elements captured correctly, only 35.8% were consistently correct across all five systems.

Researchers also found an average of three errors per case with the potential for moderate-to-severe harm.

So the useful procurement question is not:

“Are ambient scribes accurate?”

It is:

“How does this specific system perform on consultations like ours?”

That is a much harder question.

It is also the one that matters.

Who judges the note changes what gets found

The studies also use different reviewers.

In the UC Davis project, treating physicians reviewed generated notes from their own clinical encounters.

They knew:

what was actually said;
what they intended;
which missing details mattered.

Palm and colleagues used blinded speciality reviewers comparing paired ambient and physician-authored notes through a structured documentation-quality instrument.

The Mayo study used case-specific rubrics and separately considered potential clinical harm.

The Spanish study combined automated metrics with a structured quality rubric and manual audit.

Each method detects different weaknesses.

None captures all of them.

Different evaluation methods producing different measures of AI medical scribe quality.

AI notes are not being compared with perfect notes

This is another important distinction.

Clinical documentation created by humans also contains omissions, inconsistencies and unsupported content.

Palm and colleagues compared ambient notes with physician-authored “gold” notes across 97 encounters and five specialties.

The physician notes scored slightly higher overall:

4.25 vs 4.20 out of 5.

Physician notes performed better on:

accuracy;
succinctness;
internal consistency.

Ambient notes performed better on:

thoroughness;
organisation.

Hallucinations appeared in 31% of ambient notes and 20% of physician-authored notes.

Yet reviewers preferred ambient notes in 47% of comparisons, compared with 39% for physician notes.

That is an important combination.

A note can be:

more polished + more organised + preferred by reviewers

while still containing more hallucinations.

Quality perception is not the same thing as factual reliability.

A preliminary counterpoint from handwritten documentation

A preprint from Groote Schuur Hospital in South Africa compared ambient AI notes with contemporaneous handwritten notes across 49 encounters.

The study reported higher documentation-quality scores for the AI notes, while moderate-to-severe hallucinations, distortions, omissions and clinically significant errors were at least fivefold more frequent in handwritten documentation.

That finding should be treated cautiously.

The study:

remains a preprint;
comes from one site;
has a small sample;
includes a vendor relationship that should remain visible alongside the findings.

It does not establish that AI notes are generally safer than clinician documentation.

It supports a narrower point:

The correct comparator for an AI scribe is the documentation process it is replacing—not an imaginary error-free note.

What randomized evidence says about efficiency

Accuracy is not the only reason health systems consider ambient AI.

Documentation burden is.

A three-arm pragmatic randomized trial enrolled 238 outpatient physicians across 14 specialties and compared DAX Copilot, Nabla and usual care.

Time-in-note changed as follows:

Arm	Change in time-in-note	Result
Control	4m22s → 4m4s	−18 seconds
DAX	4m29s → 4m6s	−23 seconds; not significantly different from control
Nabla	4m30s → 3m49s	−41 seconds; 9.5% improvement relative to control, p=.02

Approximately 15% of physicians assigned to one of the treatment groups never used their assigned scribe.

That matters.

A tool can demonstrate benefit among users and still fail to become part of routine workflow for a meaningful group of clinicians.

Efficiency therefore has at least two questions:

Does it save time when used?

and:

Will clinicians actually use it consistently?

Large-scale evidence shows modest time savings

A separate JAMA study published in April 2026 followed 8,581 ambulatory clinicians across five academic health systems.

Of these, 1,809 adopted an AI scribe.

Scribe adoption was associated with approximately:

13.4 fewer minutes of total EHR time;
16.0 fewer minutes of documentation time;

per eight scheduled patient-hours.

Those are meaningful changes.

They are also modest at the scale of a full clinical day.

And importantly:

this study measured time and productivity—not clinical-note accuracy.

An efficiency study cannot answer an accuracy question simply because the technology is the same.

What UK clinicians report

Two UK studies published in 2026 provide quite different estimates of adoption.

Blease and colleagues surveyed 1,003 UK GPs in August 2025.

Fourteen per cent were current users.

Among the 141 current users:

80% reported less documentation time;
70% reported reduced cognitive load;
55% considered AI-generated documentation better than standard notes;
32% reported errors often or always;
14% reported errors with significant-to-critical implications;
63% routinely sought patient consent.

Reported error risk was highest in:

multi-party consultations — 38%;
complex histories — 35%;
non-English encounters — 31%.

A separate npj Digital Medicine survey of 598 UK GPs found 40% were current AI-scribe users, with a further 23% reporting previous use.

Fourteen per cent versus forty per cent.

That is not necessarily a contradiction.

The samples, recruitment and timing differ.

It is another example of why:

measurement design travels with the number.

Clinician review is necessary — but “reviewed” is not a measurement

Most ambient documentation systems rely on the same safety model:

AI generates → clinician reviews → clinician signs

That makes sense.

But the word review hides enormous variation.

In the UC Davis dataset:

14.9% of notes with available editing data were left entirely unedited;
median AI-generated words changed was 9%;
individual physician editing patterns ranged from 1.9% to 69.3%.

Those numbers do not prove that an unedited note was not reviewed.

But they show why a system should not treat the existence of a review button as proof that review is functioning as a strong safety control.

The more useful questions are:

What does the clinician check?
What information is available during review?
Which sections get corrected most?
How often are notes accepted unchanged?
Does edit behaviour change over time?
Are high-risk errors being detected before filing?

What to measure before deploying

Published evidence should inform local evaluation.

It should not replace it.

1. Omission rate

Measure clinically relevant information that was spoken but failed to reach the final note.

2. Unsupported additions

Identify content that cannot be supported by the encounter or selected source data.

3. Error severity

A formatting problem and an incorrect medication instruction should not receive the same weight.

The UC Davis study found 94.7% of reviewed notes free from significant errors, while 5.3% contained an error rated as posing serious or imminent potential harm if left uncorrected.

Both numbers belong together.

4. Clinician editing behaviour

Track:

percentage of notes edited;
amount changed;
which sections change;
percentage accepted unchanged.

Edit rate is not a direct accuracy measure.

But it is an important workflow signal.

5. Performance by specialty

Do not collapse primary care, paediatrics, mental health and emergency medicine into one number.

Clinical conversations differ.

So do documentation requirements.

6. Performance by language and encounter complexity

Test:

accents;
non-English encounters;
code-switching;
multi-speaker consultations;
interruptions;
noisy environments;
long encounters;
complex histories.

The UK evidence already suggests these conditions deserve particular attention.

7. Your existing documentation

Compare AI-assisted documentation with the process you currently use.

Measure:

omissions;
corrections;
turnaround;
completeness;
clinician time;
review effort.

The relevant question is:

Did the overall documentation process become better?

not:

Did the AI produce a perfect note?

For broader ambient documentation evidence, see Claude in Clinical Documentation: What Ambient AI Evidence Actually Shows on KastHunt.

Seven things healthcare organisations should measure when evaluating an AI medical scribe.

The procurement question worth asking

Instead of asking:

“What is your accuracy rate?”

ask:

“Show us your results for patients and consultations that look like ours—and show us what clinicians had to correct before signing.”

Then ask:

what was measured;
how errors were defined;
which specialties were included;
who reviewed the notes;
what happened in difficult encounters;
whether performance changes by language;
how model updates are revalidated.

That is harder to answer with one marketing percentage.

That is exactly why it is more useful.

Where this leaves the accuracy question

Current evidence does not support a universal AI-scribe accuracy percentage.

It supports something more nuanced.

AI medical scribes can produce useful documentation and can reduce documentation burden in some settings.

They can also:

omit relevant information;
introduce unsupported content;
behave differently across products;
perform differently across consultation types;
generate notes that look highly polished despite containing factual problems.

The newest large-scale evidence adds another useful point: AI-supported documentation can improve readability and structured documentation quality in routine clinical care, but those gains should not be mistaken for proof that every clinical fact is correct.

The defensible conclusion is:

AI medical-scribe performance depends on the product, clinical setting, patient population, language, measurement method, workflow and review process.

That makes local evaluation part of procurement.

Not an optional step after deployment.

Want to know how an ambient scribe performs in your clinical setting?

Published studies tell you what happened in other organisations, specialties and workflows.

They cannot tell you how a scribe will perform with your clinicians, your patients, your languages, your note templates and your review process.

KastHunt’s Discovery Workshop helps teams define what should be tested before committing to a larger healthcare AI build or deployment—including the workflow, evaluation measures, clinical-review points and evidence required to make the decision.

Discuss Your Ambient Scribe Evaluation

Frequently asked questions

How accurate are AI medical scribes?

There is no reliable category-wide percentage. Studies measure different things, including speech-recognition hallucination, omissions, unsupported additions, semantic agreement, structured note quality and clinician-reported error. Their results therefore cannot be combined into one universal accuracy score.

What is the most common error in AI-generated clinical notes?

In the UC Davis real-world pilot, accidental omissions were the most frequently identified error, affecting 18% of reviewed notes, compared with hallucinations in 11.5% and accidental inclusions in 9.3%.

Do AI scribes hallucinate?

Yes. Peer-reviewed studies have documented unsupported content in AI-generated clinical notes, although the reported frequency depends strongly on the definition and evaluation method. Palm and colleagues detected hallucinations in 31% of ambient notes compared with 20% of physician-authored comparison notes.

Do clinicians also make documentation errors?

Yes. Physician-authored notes are not an error-free comparator. Palm and colleagues detected hallucinations in 20% of physician-authored notes, while preliminary Groote Schuur evidence also reported clinically meaningful errors in handwritten documentation.

What does the new 2026 Spanish study show?

A Scientific Reports study published 23 September 2026 analysed more than 2.3 million outpatient consultations across 45 specialties. AI-assisted documentation showed 89.3% weighted semantic agreement and improved readability and structured documentation-quality scores. These are quality and agreement measurements, not a universal clinical-accuracy percentage.

Do AI scribes save clinicians time?

Evidence suggests they can, although the magnitude varies. In a randomized trial of 238 physicians, Nabla produced a statistically significant 9.5% relative reduction in time-in-note compared with control, while DAX did not show a statistically significant difference from control.

A separate study of 8,581 clinicians found AI-scribe adoption associated with 13.4 fewer minutes of total EHR time and 16.0 fewer documentation minutes per eight scheduled patient-hours.

Are AI-scribe errors clinically serious?

Most evaluated errors in the UC Davis study were not rated at the highest severity. However, 5.3% of reviewed notes contained errors rated as posing serious or imminent potential harm if uncorrected.

Do clinicians always correct AI-generated notes?

No consistent editing pattern exists. In the UC Davis editing dataset, 14.9% of notes were left entirely unedited, while physician editing behaviour varied substantially. An unchanged note does not necessarily mean it was not reviewed, but it shows why actual review behaviour should be monitored.

Do different AI-scribe products perform differently?

Yes. A five-platform controlled comparison found substantial variation across systems processing identical encounters. Only 35.8% of correctly captured clinical elements were consistently correct across all five platforms.

Are simulated studies useful?

Yes. They are particularly useful for controlled product comparison because multiple systems can process identical input. However, simulated encounters cannot reproduce every aspect of routine clinical work.

What does UK evidence show?

UK surveys provide different adoption estimates. One study of 1,003 GPs found 14% current use, while another survey of 598 GPs found 40% current use. The difference illustrates why sample, timing and recruitment methodology need to accompany any adoption figure.

Should we trust a vendor’s accuracy claim?

Treat it as one piece of evidence. Ask what was measured, how error was defined, what sample was used, which specialties were represented, who performed the review and whether the evaluation was independent.

What should we test before deploying?

Test omissions, unsupported additions, error severity, clinician-edit behaviour, performance by specialty, performance across your languages and audio conditions, and performance against your existing documentation process.

About the author

Kamna Singh Thakur is CEO and Founder of KastHunt, an AI-native healthcare technology firm serving clinics, healthtech founders and health systems across the UK, US and Australia. She has spent nine years in healthcare technology across business development, account management and digital transformation, and grew up in a clinical household — her mother, Leela Verma, spent 40 years in frontline nursing, medicine and behavioural health, including NHS service.

She writes about clinical AI governance, healthcare product delivery, and what actually survives deployment in a real clinical environment.

Questions or corrections: info@kasthunt.com