How accurate are AI medical scribes?

AI medical scribes are accurate enough to be useful and inaccurate enough to require review of every note. Independent evaluations put roughly 2–3 errors in an average draft note, with at least one error in the majority of notes; most errors are mild, but a small minority could cause serious harm if signed unchecked. The commonest error is omission — something discussed that never reached the note — which is also the hardest error for a clinician to spot.

That sentence is deliberately unglamorous, because the honest answer to "how accurate are AI medical scribes?" is not a single percentage. Vendors quote accuracy figures of 95–99%; independent researchers measuring the same category of product report error counts per note rather than percentages, and a Canadian government audit that tested 20 pre-approved products found problems in all 20. This article sets out what the published evidence actually says, why the numbers disagree so wildly, which failure modes matter, and how to review and test a scribe in your own practice.

How accurate are AI medical scribes?

Current ambient AI scribes typically produce a usable draft note from a routine consultation, with overall error rates reported at roughly 1–3% of content — better than the 7–11% error rates typical of older automated dictation systems. But the errors they do make are qualitatively different from mishearings, and in clinical documentation a small percentage of content can carry a large consequence.

A 2025 commentary in npj Digital Medicine frames the position bluntly: adoption is outpacing validation. The authors note that modern ambient scribes "report lower overall error rates (≈1–3%) but introduce distinct failure modes" — hallucination, critical omission, misattribution and contextual misinterpretation — and that even studies reporting low rates "acknowledge that in healthcare, even a small percentage of errors can have profound implications for patient safety" (Topaz, Peltonen & Zhang, npj Digital Medicine, 2025).

The practical reading for a working clinician is this: treat the draft as a first pass by a fast, fluent, context-blind assistant. It will rarely produce nonsense. It will occasionally produce something confidently wrong, and it will quietly leave things out. For how the underlying pipeline creates those specific failure modes, see how AI medical scribes work; for the category as a whole, start with our pillar on what an AI medical scribe is.

What do independent studies actually report?

Three independent evaluations give the clearest picture available. They disagree on headline framing but agree on substance: errors are common, omissions dominate, and serious errors are rare but real. None of the three was funded or run by a scribe vendor.

MedStar Health: 70% of draft notes contained an error

Researchers at MedStar Health's National Center for Human Factors in Healthcare tested two popular commercial ambient scribes in a simulated setting. Recordings of 11 real outpatient encounters across ten service lines were turned into scripts, each read by two residents per product, producing 44 draft notes. Two reviewers independently classified every error.

The result: 127 errors (mean 2.9, SD 2.7 per draft note) in 31 of 44 (70%) draft notes. Omissions were the most frequent error type for both products — 55 of 66 errors (83%) for one product and 33 of 61 (54%) for the other. Error patterns differed significantly between the two products (Fisher exact test, P = .002), which matters: you cannot generalise one vendor's accuracy to another's (Biro et al., Journal of Medical Internet Research, 2025).

The authors' observation about omissions is the single most useful line in the paper for everyday practice: omission "may be the most difficult for clinicians to identify since the identification process requires memory recall of details from the patient encounter. If clinicians review their documentation after several patient encounters, recalling omitted details may be challenging."

Six scribes in primary care: good to excellent, none error-free

A Canadian team at Women's College Hospital built a structured evaluation framework covering usability, technical performance and accuracy, then ran six AI scribes used in Canadian and US primary care through audio from four standardised patient encounters, benchmarked against verbatim transcripts and physician-written notes.

Their finding: "While all AI scribes produced good to excellent quality medical notes, none were consistently error-free. Common errors included deletion, omission, and SOAP structure errors." Crucially, they also identified what degrades accuracy: "Factors such as extraneous conversations and multiple speakers impacted the accuracy of both the transcript and medical note." Most products generated a SOAP-format note in about a minute for a 15-minute encounter, with time rising as encounters got longer and more complex (Ha et al., JMIR Human Factors, 2025).

A 7,545-note pilot: 94.7% clean, 5.3% potentially serious

The largest real-world safety evaluation published to date comes from a two-month pilot in which 31 physicians across multiple specialties used an ambient scribe to help create 7,545 clinic notes. Physicians formally reviewed 356 of them (4.7%) against four error types and rated the potential for patient harm on a 0–5 scale.

• Accidental omissions — 64 notes (18%), again the most frequent error.
• Hallucinations — 41 notes (11.5%).
• Accidental inclusions — 33 notes (9.3%).
• Bias — 4 notes (1.1%), rare in this sample.
• Severity: 119 of 142 errors (83.8%) were mild to moderate; 19 notes (5.3%) contained errors rated as posing serious or imminent risk if left uncorrected.

The authors' conclusion is the one to take into clinic: "AI-generated clinical notes were generally of high quality, with 94.7% (337/356) free from significant errors. However, because a small number contained errors that carried the risk of serious harm if not corrected, careful clinician review of notes remains imperative" (Taylor et al., JMIR Medical Informatics, 2026).

Put the three studies together and a reasonable working estimate for a mainstream product in 2026 is: expect one or more errors in most notes, expect most of them to be trivial, and expect something that would matter clinically in roughly one note in twenty. That is a tolerable risk profile with disciplined review and an unacceptable one without it.

What did Ontario's audit find when it tested 20 approved scribes?

In May 2026 the Office of the Auditor General of Ontario published a special report on the provincial government's use of AI, including a direct examination of the AI scribe products pre-qualified for purchase by health-care providers. It is the most consequential independent test of this product category so far, because the products tested were ones a public health system had already approved.

Two simulated clinician–patient recordings were given to all bidding vendors, whose generated notes were then assessed by clinical evaluators from OntarioMD and Ontario Health. The report states that "all 20 of the approved vendors showed one or more of these types of inaccuracies in their system-generated notes" (Auditor General of Ontario, Use of Artificial Intelligence in the Ontario Government, May 2026).

• Hallucinations — 9 of 20 (45%). Systems "fabricated information and made suggestions to patients' treatment plans, such as referring the patient for therapy or ordering blood tests, even though these steps were not mentioned in the simulated recordings." Evaluators also flagged fabrications with direct clinical weight — notes stating there were "no masses found", or that the patient had anxiety, when neither was discussed.
• Incorrect information — 12 of 20 (60%). Notes "captured a different drug than what was prescribed by the doctor."
• Missing or incomplete information — 17 of 20 (85%). Notes "missed key details about the patients' mental health issues in at least one of the two tests, even though this was mentioned in the simulated recordings." Six of 20 (30%) missed them across both tests.

The procurement findings are arguably more useful to a buyer than the error counts. "Accuracy of medical notes generated" accounted for just 4% of the maximum procurement score, with no minimum passing threshold — so a vendor could score zero on accuracy and still be approved. Eleven of the 20 approved vendors submitted no SOC report, HITRUST certification or ISO 27001 certification. Vendors were not required to demonstrate their systems live, and generated the evaluation notes offline. Bias controls were worth 2% of the score, and vendors were asked to describe bias controls rather than provide evidence of bias testing.

Why do accuracy figures disagree so much?

Because "accuracy" is measured at least four different ways, and the four produce numbers that are not comparable. A vendor quoting 98% and a researcher reporting 2.9 errors per note can both be telling the truth about the same product.

• Word error rate (WER) — the proportion of transcribed words that differ from a human transcript. This measures the speech-to-text stage only. A note can have a near-perfect WER and still be clinically wrong, and a note with a messy transcript can summarise correctly; the six-scribe evaluation found exactly that in both directions.
• Error counts per note — how the independent studies above report. This captures clinical substance, but depends entirely on who counts and what they count as an error.
• Structured note-quality instruments — most commonly the Physician Documentation Quality Instrument (PDQI-9), scoring attributes such as accuracy, thoroughness and succinctness on five-point scales. One validated evaluation across 97 encounters in five specialties scored specialist-drafted notes at 4.25/5 against 4.20/5 for the AI scribe's notes — a small but statistically significant gap (Palm et al., 2025).
• Severity-weighted review — classifying each error by its potential to cause harm, as in the 7,545-note pilot. This is the most decision-relevant measure and the least often published.

When a vendor quotes an accuracy percentage, the only useful follow-up is: accuracy of what, measured against what reference, on whose recordings, scored by whom? If the answer is a transcription benchmark on clean studio audio, it tells you very little about a noisy consulting room with a relative interpreting for the patient.

What are the failure modes, and which is hardest to catch?

Five failure modes recur across every study. Learning them by name is the single highest-yield thing a clinician can do, because each one is caught by a different reading habit — and the most dangerous one is invisible on the page.

• Omission — discussed but absent. The most frequent error in all three independent studies, and the hardest to detect, because nothing on the page looks wrong. It is caught only by recalling the encounter, not by proofreading the note.
• Hallucination (fabrication) — content with no basis in the encounter: an examination that never happened, a negative finding never elicited, a referral never discussed. Reads perfectly plausibly, which is precisely the problem.
• Wrong output — a real element recorded incorrectly: the wrong drug, the wrong laterality, the wrong test paired with the wrong result. The Ontario audit found a different drug from the one prescribed in 60% of approved systems.
• Speaker misattribution — the patient's words recorded as yours, or vice versa, when speaker separation fails in a room with a relative, a student or a chaperone present.
• Irrelevant or misplaced text — technically true content in the wrong section, or disproportionate emphasis. Not usually dangerous, but it bloats the note and buries the signal.

A sixth, subtler risk: because a comprehensive draft arrives already written and fluent, reviewing it is a recognition task rather than a recall task. Recognition is easier and feels complete — which is exactly why omissions slip through. Any review habit that does not deliberately force recall will systematically miss the commonest error type.

Is AI scribe accuracy the same for every patient?

No. The speech-recognition layer these tools are built on performs measurably worse for some speakers, which means documentation quality is not distributed evenly across a patient list. This is an equity issue, not only a technical one.

The foundational study tested five state-of-the-art commercial speech-recognition systems — from Amazon, Apple, Google, IBM and Microsoft — on matched audio snippets from sociolinguistic interviews. It found "substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers" (Koenecke et al., PNAS, 2020). The authors attribute the gap primarily to training data drawn predominantly from white native speakers of American English.

Modern clinical models are not the 2020 consumer systems, and vendors rarely publish subgroup performance — which is itself the finding. Ontario's auditor noted that bias in scribe systems "can occur, for example, through misinterpretations of conversations between health-care professionals and patients, particularly if either party speaks with a diverse accent", and that vendors were not required to provide evidence of bias testing. If your patients speak with accents the training data under-represents, or through a family member, assume lower accuracy and review harder. Our guide to language barriers in the consultation covers the wider problem these tools sit inside.

How much do clinicians actually edit AI-drafted notes?

Less than the error rates imply — and this gap between the review that is required and the review that happens is the real safety risk, more than the raw accuracy of any product.

The 7,545-note pilot captured editing behaviour directly. Across all AI-created notes, a median of 9.0% of AI-generated words were changed (IQR 2.5%–21.9%), and 14.9% of notes (143 of 960) were left entirely unedited. Individual practice varied enormously: average proportion of words changed ranged from 1.9% to 69.3% between physicians (Taylor et al., 2026). In the same dataset, roughly one note in twenty carried an error with potential for serious harm.

Clinician accounts match. In a qualitative study of 22 physicians piloting an ambient scribe at a California health system, 28 of 33 comments about accuracy (85%) were negative, 36 of 45 comments about note construction (80%) were negative, and the verbosity of the generated history drew consistent complaint. One physician summarised the trade-off: "You have to go read it over and edit it, which just takes time" (Shah et al., JAMA Network Open, 2025). Note that these are counts of comments, not of physicians, and the sample was weighted towards high users.

The implication for anyone building a business case is uncomfortable but important: time saved by not typing is partly spent reading, and a product that produces longer drafts can shift work rather than remove it. Our breakdown of the hidden cost of clinical documentation sets out the baseline that any saving is measured against.

How should you review an AI-generated note?

Review in a fixed order that targets the dangerous error classes first, before the encounter fades from memory. Reading a draft for internal plausibility does not work — a hallucinated note reads well. The sequence below takes under a minute once habitual.

• 1. Medications and doses first. The most common wrong-output error in the Ontario audit was a drug other than the one prescribed. Check drug, dose, route, frequency and any change you made.
• 2. The Plan. Confirm every investigation, referral, safety-net instruction and follow-up interval is one you actually gave. Delete anything you did not say — fabricated referrals and test orders were the signature hallucination in the Ontario tests.
• 3. Negatives and examination findings. Any stated normal finding or explicit negative that you did not elicit is a fabrication with legal weight. "No masses found" in a note where you did not examine is not a wording problem.
• 4. Then, from memory, what is missing? Before re-reading the draft, name the two or three things the patient was most worried about. If any is absent from the note, add it. This is the only step that catches omissions, and it must be done from recall.
• 5. Laterality, numbers and attribution. Left versus right, values and units, and whether each statement is attributed to the right speaker.
• 6. Sign in the same session. Reviewing several notes at the end of a clinic is the condition under which omissions become undetectable, because recall is gone.

Our SOAP note guide sets out what a complete note should contain, and the free printable SOAP note template gives the same structure as a checklist you can keep on the desk while you build the habit.

How can you test a scribe's accuracy in your own clinic?

Run your own small evaluation before committing, using your own case mix. The published studies all used simulated or standardised encounters precisely because that makes comparison possible; you can borrow the method cheaply. The Ontario audit's weakest point — vendors generating notes offline, unobserved, with no live demonstration required — tells you what to insist on.

• Insist on a live demonstration on your own recordings, in your own room, not on vendor-prepared audio.
• Use five to ten representative encounters, deliberately including your hard cases: a multi-problem review, a consultation through a family member or interpreter, a patient with a strong accent, a paediatric consultation with a parent speaking, and a noisy room.
• Score against a reference note you write yourself immediately after each encounter, and count errors by type — omission, hallucination, wrong output, misattribution, misplaced text. Error counts by type are far more informative than a single accuracy score.
• Rate severity, 0–5, for potential harm if uncorrected. One serious error in a ten-note test is a meaningful signal.
• Ask for subgroup data on accents, languages and specialties. "We don't publish that" is itself an answer.
• Check whether the product requires an explicit review sign-off. Ontario's auditor recommended vendors be required to "enforce an attestation from users of the AI Scribe to confirm their review of notes produced" — at the time of the audit, doctors were not required to attest that they had verified the note.
• Re-test after updates. These models change; last quarter's evaluation does not describe this quarter's product.

Shifaa AI's AI medical scribe — currently in private beta ahead of launch — is built on this premise rather than around an accuracy claim: it drafts a structured note from the consultation and the clinician reviews, edits and signs it. It is decision support, not a decision-maker, and it does not diagnose. We would rather you tested it against the method above than took a percentage on trust.

Who is responsible when an AI-generated note is wrong?

You are. Every professional body and regulator that has addressed the question has landed in the same place: the clinician who signs the record is accountable for its accuracy, regardless of what drafted it. No current guidance treats "the AI wrote it" as a defence.

The RACGP's guidance for Australian GPs states that AI scribes "can make errors which impact the meaning and accuracy of clinical information" and that "GPs are ultimately responsible for ensuring that the patient health record is accurate and up to date", flagging over-reliance as a specific risk — clinicians paying less attention to clinical detail because a draft already exists (RACGP newsGP).

The regulatory position is narrower than many clinicians assume, and it cuts both ways. In guidance published on 29 July 2026, the UK's MHRA confirmed that ambient voice technology products "intended solely for transcription, summarising of clinical conversations, drafting letters, or suggesting clinical codes for a clinician to review are not regulated as medical devices under the current framework", adding that "users in healthcare should carefully consider what information you may need and practices you should follow" for safe deployment. Products intended to support diagnosis, treatment or prevention — or that act without clinician review, such as placing orders — are regulated devices (MHRA, Ambient voice technology-enabled products, 2026).

Read that carefully: a documentation-only scribe sits largely outside device regulation precisely because a clinician is expected to review its output. The clinician's review is not a courtesy step; in regulatory terms it is the thing that keeps the product out of scope. A tool that is used without review is being used outside the basis on which it was permitted.

Frequently asked questions

How accurate are AI medical scribes?

Independent studies report roughly 2–3 errors in an average draft note, with at least one error in most notes. In a MedStar Health evaluation of two commercial products, 70% of 44 draft notes contained an error (mean 2.9 per note). In a 7,545-note pilot, 94.7% of reviewed notes were free of significant errors, but 5.3% contained errors rated as posing serious or imminent risk if uncorrected. Omissions are consistently the commonest error.

Do AI scribes make things up?

Yes, though not often. Hallucination — fluent content with no basis in the encounter — was found in 11.5% of notes in one real-world pilot, and 9 of 20 products tested by Ontario's Auditor General fabricated information, including referrals and test orders never discussed, and negative examination findings never elicited. Fabrications read plausibly, so they are caught by checking the note against the encounter, not by proofreading it.

What is the most common AI scribe error?

Omission — something discussed in the consultation that never reaches the note. It was the most frequent error type in all three independent evaluations, and it is the hardest to detect, because nothing on the page looks wrong. Detecting it requires recalling the encounter rather than reading the draft, which is why notes should be reviewed in the same session, not batched to the end of a clinic.

Do I still have to check every AI-generated note?

Yes. Professional guidance is consistent that the clinician who signs the record is responsible for its accuracy. It is also the regulatory basis on which documentation-only scribes avoid classification as medical devices in the UK: they are out of scope because output is provided for clinician review and correction. Evidence suggests review is often light — a median of 9% of AI-generated words edited, and about 15% of notes left entirely unedited.

References

• Biro J, Handley JL, Cobb NK, et al. Accuracy and Safety of AI-Enabled Scribe Technology: Instrument Validation Study. Journal of Medical Internet Research, 2025;27:e64993. pmc.ncbi.nlm.nih.gov/articles/PMC11811668
• Ha E, Choon-Kon-Yune I, Murray L, et al. Evaluating the Usability, Technical Performance, and Accuracy of Artificial Intelligence Scribes for Primary Care: Competitive Analysis. JMIR Human Factors, 2025;12:e71434. pmc.ncbi.nlm.nih.gov/articles/PMC12309782
• Taylor SL, Jost M, MacDonald S, et al. Quality of Clinical Notes Created by Ambient Listening Generative AI: Pragmatic Prospective Pilot Study. JMIR Medical Informatics, 2026. pmc.ncbi.nlm.nih.gov/articles/PMC13089619
• Office of the Auditor General of Ontario. Use of Artificial Intelligence in the Ontario Government — Special Report, May 2026 (section 4.3, Safe Use of AI Scribe Systems). auditor.on.ca
• Topaz M, Peltonen LM, Zhang Z. Beyond human ears: navigating the uncharted risks of AI scribes in clinical practice. npj Digital Medicine, 2025;8:569. nature.com/articles/s41746-025-01895-6
• Shah SJ, Crowell T, Jeong Y, et al. Physician Perspectives on Ambient AI Scribes. JAMA Network Open, 2025;8(3):e251904. jamanetwork.com
• Koenecke A, Nam A, Lake E, et al. Racial disparities in automated speech recognition. PNAS, 2020;117(14):7684–7689. pmc.ncbi.nlm.nih.gov/articles/PMC7149386
• Palm E, Manikantan A, Pepin ME, et al. Assessing the Quality of AI-Generated Clinical Notes: A Validated Evaluation of a Large Language Model Scribe. arXiv:2505.17047, 2025. arxiv.org/abs/2505.17047
• Medicines and Healthcare products Regulatory Agency. Ambient voice technology-enabled products — guidance, 29 July 2026. gov.uk
• Royal Australian College of General Practitioners. Are AI scribes risking patient safety? newsGP (reporting RACGP guidance on AI scribes). racgp.org.au