How to test dictation accuracy on your own Mac

To test dictation accuracy on your Mac, read one fixed passage into each app with the same microphone and language, keep the raw output, and count the wrong, missing and extra words. Divide that count by the passage's word count and you have a word error rate you can compare. Then time how long it takes to turn each result into text you'd actually send, because that's the number you feel every day.

The whole test takes about 15 minutes per app. Here's how to run it without fooling yourself.

#The 15-minute version

1. Pick your passage. Use the sample below, plus three or four sentences from your real work: names of clients, products, colleagues, the jargon you type all day.
2. Fix the setup. Same microphone, same room, same distance from the mic, same language setting in every app. Write it down: Mac model, macOS version, app version, speech model.
3. Read the passage once in each app. Don't stop to correct anything. Copy the output into a text file untouched.
4. Count errors against the original (how to count is below).
5. Do it twice more. Three runs per app shows you whether a result was a fluke.

If one app clearly makes fewer mistakes on your own names and sentences across all three runs, you have your answer. If the differences are a word or two, the apps are tied on accuracy and you should decide on everything else.

#A sample passage to read

Download the passage as a text file, or read it from here:

Before the review meeting, please check the project notes and update the draft. We need a clear summary of what changed, which questions remain open, and who will handle the next step. Keep the original file so that we can compare the two versions later.

The first option is simpler to maintain, but it does not cover every case. The second option takes more setup and gives us more control. Do not describe either option as finished until the checks have passed. If a result is uncertain, write down the uncertainty instead of guessing.

After the meeting, send the revised outline to the team. Include the decision, the reason for it, and the date of the next review. Put any unanswered questions in a separate paragraph. A short, accurate note is more useful than a polished explanation that changes what people agreed to do.

It's ordinary office prose with a few traps: "does not" versus "doesn't", negations that flip the meaning if a word drops, and similar-sounding pairs like "two" and "to". We wrote it for this test; we haven't published results for it.

Ordinary prose only tells you part of the story. Add a second passage in your own words, and a third that you speak without a script, the way you'd explain something to a colleague. Unscripted speech has pauses, restarts and "um", which is where dictation apps differ most.

#Counting errors: word error rate in plain terms

Word error rate (WER) = (substitutions + deletions + insertions) ÷ words in the original. It's the standard measure in speech recognition, and the Hugging Face WER documentation spells out the formula and its main weakness: the score depends heavily on the material it's measured on.

• Substitution: you said "draft", it wrote "graph".
• Deletion: you said "do not describe", it wrote "do describe". One missing word, and the meaning is reversed.
• Insertion: it added a word you never said.

A worked example from a real measurement: in our Parakeet and Whisper benchmark, Parakeet v3 made 5 errors across 1,082 words of LibriSpeech audiobook speech. 5 ÷ 1,082 = 0.46%. Whisper Base made 18 errors on the same audio, 1.66%. Those are clean read recordings on one M4 Max Mac, so treat them as a demonstration of the arithmetic, not as what you'll see at your desk.

Normalize before you count

Decide your rules first and use them for every app:

• Lowercase everything and ignore punctuation, or count punctuation as a separate score.
• Decide whether "10" and "ten" match. Most simple scripts say they don't.
• Decide whether "don't" and "do not" match.

Change any of these and the percentage moves, sometimes a lot. That's why a score you read on a website rarely matches the one you get on your own Mac.

#Keep the comparison fair

Use one recording when you can. Some apps transcribe audio files as well as live speech. Record yourself once (Voice Memos is fine), then give that same file to each app. Now every app hears exactly the same audio, and the microphone and your delivery are no longer variables.

Compare like with like. Many apps clean up text after recognition: removing fillers, fixing punctuation, rewriting with an AI model. A rewrite that drops a hesitant "I think" will look like a deletion. Either test every app in its plainest mode, or label which results include cleanup.

Separate the first run. The first dictation after installing a model can be slower while it loads. Time later runs, and note the first one separately.

Use harmless text for cloud apps. If one candidate sends audio to a server, test with material you don't mind uploading. Where the audio goes is its own question; our local vs cloud dictation guide covers it.

#Score the work, not just the words

A tool with a slightly higher WER can still win if its mistakes are easier to fix, or if it gets your names right. Track these alongside the percentage:

Run each app for a normal working day after the formal test. A day of real messages tells you more than any single passage.

#Testing Sayburst

Sayburst has a few settings that make a clean test easy:

• Raw mode inserts the transcript without rewriting, so you're measuring recognition. Switch back to Clean or Email afterwards to see how much editing the cleanup saves you.
• Same file, different models. Drop your recording on the History window and Sayburst transcribes it locally. Change the speech model in Settings (Parakeet v3, Parakeet Ultra, Whisper Large v3 Turbo and others), drop the file again, and compare.
• Vocabulary. Add the names from your work passage and run it again. With Whisper models the vocabulary nudges recognition; with Parakeet, Sayburst corrects close misses to your spelling without touching ordinary words.

Here's what that looks like on a real sentence. Spoken, with vocabulary set up for the names:

"um, can you loop in Siobhan and, and Tadhg before we send the Kestrel proposal"

Inserted in Clean mode:

Can you loop in Siobhan and Tadhg before we send the Kestrel proposal?

For your scorecard, count the names separately. Before you add vocabulary, rare names like these are exactly the words any recognizer is most likely to spell its own way.

#Reading small differences

Two errors apart on a 150-word passage is a tie. Even our benchmark's gap between Parakeet v3 (5 errors) and Parakeet Ultra (7 errors) on over a thousand words is too small to crown a winner. Look for differences that repeat across runs and across your passages.

Keep your test notes. When an app or model updates, you can rerun the same passage under the same conditions and see whether anything actually changed. And if you're choosing between apps, our roundup of Mac dictation apps covers price, privacy and platforms, the things a WER score won't tell you.

Questions people ask

How do I test dictation accuracy on a Mac?

Read the same passage into each app with the same microphone and language, save the untouched output, and count substituted, missing and extra words against the original. Divide by the number of words in the passage to get the word error rate.

What is word error rate in dictation?

Word error rate is substitutions plus deletions plus insertions, divided by the number of words in the reference text. Lower is better. It ignores punctuation unless you choose to count it, and it doesn't measure how long you spend fixing the draft.

How can I compare two dictation apps fairly?

Keep the microphone, room, language and material the same. If both apps accept audio files, feed them the same recording; if not, read the passage three times in each and look at the spread, not one run.

What is a good dictation accuracy score?

There's no universal number, because the score depends on the material, the microphone and how you normalize the text. Compare apps on your own passage and conditions, and pay as much attention to names, numbers and correction time as to the percentage.