Why Clinicians Preferred AI Notes Even When They Hallucinated More Often
One study found reviewers preferred ambient AI-generated clinical notes in 47% of comparisons versus 39% for physician-authored notes, despite higher hallucination rates. The finding shows that clinician preference and note accuracy are different measures and both need to be evaluated separately.
In one peer-reviewed study, reviewers preferred AI-generated ambient clinical notes more often than physician-authored notes — 47% versus 39% — even though hallucinations were detected more often in the AI-generated notes.
Across 97 outpatient encounters, hallucinations were detected in 31% of ambient AI notes versus 20% of physician-authored notes. Physician-authored notes also scored slightly better overall for quality, accuracy, succinctness and internal consistency.
So why were the AI notes still preferred more often?
The study found that ambient notes were rated as more thorough and better organised. Those characteristics can make documentation easier to read and navigate, even when another dimension of quality is weaker.
That does not prove automation bias.
But it highlights an important implementation problem:
A clinical note can feel better before it is actually safer.
What did the study actually find?
The study evaluated 97 outpatient encounters across five specialties: general medicine, paediatrics, obstetrics and gynaecology, orthopaedics, and adult cardiology. Two blinded speciality reviewers compared ambient AI-generated notes with physician-authored reference notes using an adapted PDQI-9 quality framework.
| Measure | AI-generated ambient note | Physician-authored note |
| Overall quality | 4.20 / 5 | 4.25 / 5 |
| Hallucinations detected | 31% | 20% |
| Preferred overall | 47% | 39% |
| Thoroughness | Higher | Lower |
| Organisation | Higher | Lower |
| Accuracy | Lower | Higher |
| Succinctness | Lower | Higher |
| Internal consistency | Lower | Higher |
The important point is not simply that the AI notes contained errors.
It is the apparent contradiction:
The notes with higher hallucination prevalence were also preferred more often.
Why preference and accuracy are different measurements
A clinical note has several jobs.
It must represent the encounter accurately, but it must also be readable, organised, sufficiently complete and useful to another clinician.
Those qualities do not always move together.
In the Palm study, physician-authored notes performed better for accuracy, succinctness and internal consistency, while ambient notes performed better for thoroughness and organisation.
That means a note can be easier to consume without necessarily being more reliable.
This distinction matters because healthcare organisations sometimes evaluate ambient AI using clinician satisfaction alone.
A clinician may genuinely prefer a system because it makes documentation easier.
That does not automatically answer whether the generated note is clinically correct.
Why polished documentation can increase verification risk
Well-structured documentation carries a strong signal of competence.
A note with a clear history, organised assessment, complete plan and familiar clinical structure may appear trustworthy because it resembles the kind of documentation clinicians expect.
The problem is that presentation quality and factual reliability are separate properties.
A highly polished note can still contain an unsupported medication, diagnosis, symptom, examination finding or plan.
This makes one design principle increasingly important:
The easier AI-generated documentation becomes to read, the more deliberate its verification process may need to become.
That point is even more relevant in 2026 because leading ambient-AI companies are moving beyond simple note-generation metrics toward continuous quality assessment.
For example, Abridge now describes a layered evaluation model using pre-deployment testing, continual monitoring and in-line error detection before clinicians receive the draft.
Does this study prove automation bias?
No.
Automation bias generally refers to over-reliance on automated output or failure to detect errors introduced by automated systems.
A longstanding systematic review found that trust, confidence, workload, experience and task conditions can influence reliance on automation.
A 2026 systematic review of generative AI in healthcare also identified automation bias, deskilling and changes in clinical reasoning as recurring concerns across the literature, while noting that much of the evidence remains observational, experimental or conceptual rather than long-term real-world evidence.
But the Palm study itself did not experimentally test automation bias.
The reviewers were blinded to note sources.
Therefore, the 47% versus 39% result should not be presented as proof that clinicians trusted AI because it was AI.
What it does show is that:
clinicians can prefer characteristics of AI-generated documentation even when another measured quality dimension performs worse.
Why speciality matters
The overall preference result also hides substantial speciality variation.
In the study, reviewers in general medicine, orthopaedics and cardiology tended to favour ambient notes more often, while reviewers in obstetrics/gynaecology and paediatrics more often preferred physician-authored notes.
That leads to an important deployment lesson:
There may be no universal definition of the best AI-generated clinical note.
Different specialties value different levels of detail, structure and brevity.
An AI scribe therefore should not be evaluated only with a pooled, organisation-wide score.
It should also be tested by:
- speciality;
- encounter type;
- documentation template;
- clinician workflow;
- error type;
- clinical significance.
Preference is useful — but it is not a safety metric
Clinician preference still matters.
If clinicians dislike a system, adoption will suffer.
Preference can tell healthcare organisations about:
- usability;
- workflow fit;
- readability;
- cognitive burden;
- perceived usefulness.
But it should not substitute for:
- factual accuracy;
- omission rate;
- hallucination rate;
- clinically significant error rate;
- review behaviour;
- patient-safety outcomes.
Abridge, for example, now publicly emphasises structured evaluation across clinical, operational and workflow outcomes rather than relying on a single quality score.
Heidi similarly reports documentation quality together with clinician experience and human review rather than presenting output quality as a standalone metric.
Do not ask only whether clinicians like the AI-generated note. Ask whether the review process is actually reducing residual documentation risk.
The more important metric may be what happens during review
14.9% of analysed AI-generated notes were left entirely unedited.
The important distinction is:
unchanged does not mean unreviewed.
An unchanged note may mean:
- the AI generated an accurate draft;
- the clinician reviewed the note and found no changes necessary;
- the clinician skimmed rather than deeply reviewed it;
- trust increased with familiarity.
Edit rate alone cannot distinguish those scenarios.
Therefore:
A low edit rate should be treated as a monitoring signal, not automatically as evidence of high quality.
Why “human in the loop” is not enough
Healthcare AI discussions often stop at:
“The clinician remains in the loop.”
But this describes architecture, not behaviour.
A clinician can technically sit between AI generation and final sign-off while performing very different levels of review:
- line-by-line verification;
- targeted review of clinically important sections;
- comparison with transcript or source evidence;
- rapid skim;
- approval with minimal review.
Those controls are not equivalent.
The more important question is:
What does the clinician actually do in the loop?
That framing is becoming increasingly relevant across the ambient-AI market. Abridge now highlights source-linked verification and evaluation workflows, while Heidi’s governance content emphasises review, accountability and safe deployment rather than simply saying a human is involved.
How should healthcare organisations evaluate an AI scribe?
Healthcare organisations should evaluate AI scribes across accuracy, error severity, clinician preference, review behaviour, speciality performance and post-deployment monitoring rather than relying on one accuracy or satisfaction score.
1. Measure clinician preference
Ask whether clinicians prefer the generated note, but also ask why.
2. Measure specific error types
Track:
- omissions;
- unsupported additions;
- contradictions;
- medication errors;
- diagnosis errors;
- incorrect plans;
- incorrect referrals.
3. Weight errors by clinical significance
Do not treat a minor wording issue and a fabricated medication instruction as equivalent.
4. Measure review behaviour
Track:
- whether the note was changed;
- which sections were edited;
- review time;
- critical corrections;
- unchanged notes;
- escalation events.
5. Measure performance by speciality and workflow
A pooled average can hide meaningful speciality-level differences.
6. Continue monitoring after deployment
Pilot performance should not be treated as permanent evidence.
Usage patterns, clinician behaviour, model versions and workflow conditions can all change over time.
What can product design do to reduce AI-scribe verification risk?
A safer AI-scribe workflow can consider:
- showing the draft alongside source transcript or evidence;
- highlighting transformed or inferred information;
- flagging uncertain clinical content;
- requiring explicit confirmation of high-risk fields;
- preserving note-version history;
- recording material edits;
- monitoring clinically significant correction patterns;
- sampling unchanged notes for quality review;
- creating speciality-specific templates and review rules.
Abridge’s current public quality approach is moving in this direction: it describes source-linked verification, continual monitoring and specialised in-line systems that identify and correct potential errors before the clinician sees the note.
How can KastHunt help healthcare organisations evaluate AI scribes safely?
KastHunt approaches AI scribe implementation as a workflow and measurement problem, not only a note-generation problem.
Before scaling an AI documentation workflow, KastHunt can help healthcare organisations define:
1. What should be measured
Accuracy, omissions, unsupported additions, clinician preference, review effort and clinically significant corrections should be kept as separate measures.
2. Where clinician review occurs
The workflow should define which generated fields require active verification and who is responsible for final approval.
3. How errors are observed
Edit history, note versions, review timestamps and quality sampling can make review behaviour visible rather than assumed.
4. How speciality variation is handled
Clinical documentation requirements can differ across specialties, encounter types and organisational workflows.
5. What success means before deployment
KastHunt’s current healthcare-AI delivery approach starts by mapping the problem, stakeholders, technology, compliance requirements and measurable success criteria before build or deployment.
This allows the organisation to test a more useful question:
Does this AI scribe reduce documentation burden without increasing residual clinical-documentation risk?
What should healthcare providers ask AI-scribe vendors?
Healthcare organisations should ask:
- How do you measure hallucinations, omissions and clinically significant documentation errors?
- Do you measure preference separately from accuracy?
- How do clinicians verify generated notes?
- Can reviewers see the source information behind generated statements?
- How do you monitor whether review quality changes as clinicians become familiar with the system?
- How does performance vary by speciality, encounter type and workflow?
- What happens when your model, prompt, template or generation pipeline changes?
This is much closer to where leading competitors are now positioning quality evaluation in 2026. Abridge, for example, explicitly discusses continual monitoring and pre-/post-deployment evaluation rather than treating quality testing as a one-time exercise.
What does the 47% versus 39% finding actually mean?
The finding does not mean clinicians prefer inaccurate notes.
It does not prove automation bias.
It does not prove ambient AI documentation is unsafe.
And it does not mean physician-authored notes are error-free.
The study detected hallucinations in both note groups, with higher prevalence in ambient notes.
The more important conclusion is:
clinical-documentation quality is multidimensional.
A note may be easier to read, more organised or more complete while still being weaker in another quality dimension.
That is why usability and correctness should be measured separately.
Where this leaves healthcare organisations
AI-generated notes can be easier to read and more structured.
They can also contain omissions, unsupported additions and other errors.
Healthcare organisations therefore should not choose between clinical usability and clinical accuracy.
They need to measure both.
Then they need to measure what happens during review.
The stronger deployment question is:
Does clinician review remain effective as the AI becomes familiar, fast and increasingly polished?
Evaluating an AI scribe?
Preference, accuracy and review behaviour should be measured separately.
KastHunt helps healthcare organisations map clinical workflows, define measurable AI-scribe evaluation criteria and design review controls before wider deployment.
Frequently asked questions
Why can clinicians prefer AI-generated notes even when those notes contain more hallucinations?
Preference and accuracy measure different things. In the Palm study, ambient notes were rated as more thorough and better organised, while physician-authored notes performed better for accuracy, succinctness and internal consistency. Reviewers preferred ambient notes in 47% of comparisons versus 39% for physician-authored notes.
Did the Palm study prove automation bias in AI medical scribes?
No. The reviewers were blinded and the study did not experimentally test automation bias. The findings show that preference and measured documentation quality can move in different directions, but they do not establish automation bias as the cause.
What is automation bias in healthcare AI?
Automation bias is the tendency to rely too heavily on automated output or fail to recognise errors introduced by automation. Research suggests that trust, workload, experience and task conditions can affect reliance on automated systems.
How often did the AI-generated notes hallucinate in the study?
Hallucinations were detected in 31% of ambient AI notes compared with 20% of physician-authored notes across 97 encounters. These figures come from one study and should not be treated as a universal hallucination rate for all AI medical scribes.
Does a low AI-scribe edit rate mean the system is accurate?
Not necessarily. A low edit rate can mean the AI produced a strong draft, but it can also reflect differences in clinician review behaviour. Edit rate should therefore be interpreted alongside independently measured errors and review quality.
Should clinicians review every AI-generated clinical note?
AI-generated clinical documentation should be treated as a draft that requires appropriate clinician review before it becomes the final clinical record. The review process should reflect the type of information generated, the workflow and applicable professional requirements.
How should hospitals evaluate AI medical scribes?
Hospitals should evaluate AI scribes across accuracy, omissions, unsupported additions, clinically significant errors, clinician preference, review behaviour, speciality performance and post-deployment monitoring. A single accuracy or satisfaction score is not sufficient.
What is the difference between AI-scribe accuracy and clinician preference?
Accuracy measures whether the documentation correctly represents the encounter, while preference reflects how useful, readable or convenient clinicians find the note. A clinician can prefer a note without that note necessarily being more accurate.
Can an accurate-looking AI medical scribe note still contain clinical errors?
Yes. An AI-generated clinical note can be fluent, detailed and well organised while still containing an unsupported, omitted or incorrect clinical detail. Presentation quality therefore should not be treated as evidence of clinical accuracy.
How can healthcare organisations safely review AI-generated clinical notes?
Healthcare organisations can reduce review risk by defining explicit clinician-review steps, measuring clinically significant corrections, monitoring error patterns, testing performance by speciality and continuing quality evaluation after deployment.
How does KastHunt approach AI-scribe evaluation?
KastHunt approaches AI-scribe evaluation as a combination of clinical workflow design, measurable quality criteria, clinician review controls and post-deployment monitoring. Its healthcare-AI delivery model begins by mapping the problem, stakeholders, technology requirements, compliance context and success metrics before scaling a solution.
Related reading
- How Accurate Are AI Medical Scribes? What the Studies Actually Measured
- Who Is Accountable When an AI Scribe Gets a Note Wrong?
- Is an AI Scribe a Medical Device? MHRA, FDA and TGA Compared
Sources
- Frontiers in Artificial Intelligence — Assessing the quality of AI-generated clinical notes: validated evaluation of a large language model ambient scribe
- PubMed — Automation bias: a systematic review of frequency, effect mediators, and mitigators
- PubMed — Automation Bias in AI-Decision Support: Results from an Empirical Study
- PubMed — Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications — A Systematic Review
- Abridge — The Science of Ambient AI Evaluation in Clinical Settings at Large Health Systems
