Local audio transcription with speaker labels in Claude Code

TLDR; claude-transcribe is an MIT-licensed Claude Code skill: type /transcribe meeting.m4a and it returns a timestamped transcript with speaker labels, produced on your Mac with no audio upload and no API key. The default backend, NVIDIA Parakeet plus pyannote on Core ML, ran at about 100 times real time on my M5 Pro. A Whisper backend covers other languages and gets proper nouns right more often. The binaries do the repeatable work: words, timings and speaker turns. Claude makes the calls that need context: which backend to run, then who each speaker is and what the call decided.

In the small businesses I work with, most of the useful material is spoken: a client discovery call, a supplier negotiation, a team retro. Turning it into a brief used to mean an hour of listening back, or uploading the recording to a transcription service and accepting whatever its terms say about your client’s words.

I wanted a third option: the recording stays on my Mac, and only the transcript text reaches Claude, where I already do the next step. In my case that’s Claude Code, which can turn a labelled transcript into a brief in the same session. claude-transcribe is the skill that does it.

If you only need a live transcript of something you’re recording now, Apple’s Notes app can record and transcribe in the note itself. claude-transcribe is for the file you already have, with speaker labels.

What it produces

You give Claude a file:

/transcribe meeting.m4a

and get back timestamped lines with a speaker label each, printed and saved next to the input as meeting.transcript.txt. The worked example below shows real output.

The Swift backends read mp3, wav, m4a, flac and anything else Apple’s AVFoundation can decode. Because the skill runs inside Claude Code, the transcript is already in context: “summarise this and list who owes what” is the next prompt, with no copy and paste.

A worked example: Apollo 11 and Apollo 13

NASA’s mission audio is public domain, famous enough that you can check the result by ear, and full of radio static, which makes it a fair test. I ran two clips from NASA’s historical sounds page through both backends.

Apollo 11’s landing, 15 seconds, the default Parakeet backend:

[00:00 - 00:14] Speaker 1: Listen, uh Tranguility Base here, the angle has
landed. Roger Twang Tranquility, we copied you on the ground. You got a bunch
of guys about to turn blue. We're breathing again. Thanks a lot.

The same clip through Whisper:

[00:00 - 00:13] Speaker 1: Houston, Tranquility Base here. The Eagle has
landed. Roger, Tranquility. We copy you on the ground. You got a bunch of guys
about to turn blue. We're breathing again. Thanks a lot.

Whisper got every word. Parakeet got the sentence structure and missed the names: “Houston” became “Listen”, “Eagle” became “angle”. Both put Neil Armstrong, on the Moon, and Charlie Duke, in Houston, under one speaker.

Apollo 13’s “Houston, we’ve had a problem”, 17 seconds, has three voices: Jack Swigert and Jim Lovell in the spacecraft, Jack Lousma in Houston. Whisper again got every word, under one speaker. Parakeet found two speakers, but drew the line in the wrong place, and heard Lovell’s repeat as “we’ve had a bowl”:

[00:00 - 00:02] Speaker 1: Okay, Houston, we've had a problem here.

[00:02 - 00:12] Speaker 2: Mrs. Houston, say again, please. Uh Houston we've had
a bowl. We've had a main B plus undervolt. Roger, main B undervolt.

[00:14 - 00:16] Speaker 1: Okay, stand by 13, we're looking at it.

“Mrs. Houston” is “This is Houston”, and Lousma’s lines sit under both labels.

Half a minute of Apollo-era radio is a harder test than a Zoom call recorded on a laptop, and the lesson carries over: Whisper wins on words, and on radio neither backend got the speakers right. The transcript is a draft that someone who was in the room should skim. Names are where it fails first.

Two backends, and when to switch

On the Open ASR Leaderboard, Parakeet v3’s model card reports a 6.34% average word error rate; Whisper large-v3-turbo’s card lists 7.83%. On my M5 Pro, with models loaded, Parakeet transcribed and labelled a clean 53-second two-speaker clip in 0.5 seconds and found both speakers. Whisper took 8.4 seconds on the same clip and found three. At that rate an hour of audio would take well under a minute, though I haven’t timed a full hour.

Whisper got the proper nouns right on both Apollo clips and covers 99 languages, so it stays as the second pass. You can force it with --whisper, but you rarely need to: mention the language, or ask for a careful pass on names, and Claude switches.

Where the code stops and the model starts

The skill is a small case of a split I use in every agent workflow: a step that must give the same answer twice goes in code, and the model gets the calls that need context. claude-transcribe draws the line at the transcript file.

Code does the signal work

Each Swift binary decodes the audio once, then runs transcription and diarization at the same time. Diarization decides who spoke when; it produces speaker segments with no words. The default Parakeet binary assigns each word to the segment that covers the word’s midpoint, or the nearest segment when none does; the Whisper binary hands the join to SpeakerKit, which groups words and matches them to segments by overlap, so words near a change of speaker can land differently between the two. Both merge consecutive words from one speaker into a line and write the timestamps.

Speech recognition and diarization are machine-learning models, but here they run with fixed weights and no prompt. On the Parakeet path the same file gave a byte-identical transcript on four runs of the Apollo 13 clip, in half a second with models loaded, for no tokens. Whisper is looser: on hard audio its decoder can fall back to sampling, so a rerun can differ. Joining words to speakers is plain arithmetic on timestamps either way; a language model doing it would cost tokens on every line and could drift between runs.

Claude does the judgement over text

The skill’s SKILL.md leaves Claude three choices, each made from the request or the first transcript, never the raw audio:

Once the transcript lands, the rest is Claude’s: who each speaker is, what was decided, who owes what, and the follow-up email.

Here is what that looks like on Apollo 13. Given both transcripts above and asked to take Whisper’s words, Parakeet’s speaker turns and name the speakers, Claude produced:

Swigert:          Okay, Houston, we've had a problem here.
Lousma (Houston): This is Houston, say again, please.
Lovell:           Houston, we've had a problem. We've had a main B bus undervolt.
Lousma (Houston): Roger, main B undervolt.
Lousma (Houston): Okay, stand by, 13, we're looking at it.

That’s correct, and it’s correct because the model knows the Apollo 13 transcript and “This is Houston” marks the capsule communicator. On a client call it would work from the meeting invite and how people address each other, and every name is a guess to check. It also dropped the timestamps: it has no word timings for the lines it split, and inventing them would be the model doing the code’s job.

The test I use for which side a step belongs on: would I accept a different answer on a second run? A word’s timestamp, no. A reading of who “Speaker 2” is, yes, as long as it’s flagged. Moving the first kind out of the model is also where the token savings come from, which I measured on a reporting workflow in Stop letting your AI agent do a script’s job.

Offline after the first run

The binaries run on the Neural Engine and GPU through Core ML. The first transcription downloads about 2 GB of models from Hugging Face and compiles them, which takes a few minutes; every transcription after that works offline. Each /transcribe call starts a fresh process that loads the models again; the cache of compiled models on disk is what keeps the reload fast. My first Apollo 13 run took 18 seconds for 17 seconds of audio while that cache warmed, and the next file took under a second.

What I tried and rejected

I tested the alternatives in September 2026 and recorded why each lost, so the next review starts from evidence:

• Apple’s Speech framework ran at 60 times real time but made five word errors on a test clip where WhisperKit made none, and has no speaker labels.
• parakeet-mlx runs the same Parakeet model, slower than the Core ML port, with no diarization.
• senko diarizes faster than pyannote but can’t label overlapping speech, which meetings need. Speed hasn’t been the constraint yet; I’ll revisit senko if multi-hour files make it one.
• SpeakerKit Pro is Argmax’s paid diarizer; the free pyannote pipeline already did what I needed.

Limits

• The fast path needs an Apple Silicon Mac on macOS 14 or later, Xcode Command Line Tools and 8 GB of memory, 16 GB for multi-hour files. The Python backend runs elsewhere, but without a CUDA GPU it’s slow on anything over an hour.
• Speaker labels are numbers until Claude or you name them. Diarization fails both ways: similar voices split into extra speakers, and distinct voices on a poor line merge, as on the Apollo clips. --num-speakers 2 fixes the count when you know it.
• The audio never leaves the Mac, but the transcript does once Claude works on it: Claude Code sends it to Anthropic like any other prompt, under Anthropic’s terms. If the words themselves must stay local, run the binary directly, as the README shows, and keep Claude out of it.
• Overlapping speech gets assigned to one speaker.
• The models carry their own licences: Parakeet and pyannote community-1 are CC-BY-4.0, which allows commercial use with attribution; Whisper is MIT.

Install

git clone https://github.com/kam/claude-transcribe.git
cd claude-transcribe
./install.sh

The script builds both Swift binaries and links the skill into ~/.claude/skills/. Restart Claude Code, then /transcribe a short recording first so the model download happens on something you don’t need urgently.

Turning recorded calls into briefs and follow-ups for a team is the kind of narrow, repeated job I build in Applied AI engagements.

Related reading

• Skills, hooks, agents, and commands: how to use each, how a skill like this one loads and runs
• Claude Code hook examples: turning mistakes into tested rules, my open-source hook-rule plugin
• Stop letting your AI agent do a script’s job, the same split measured on a reporting workflow
• A local 27B model shipped code through my dev workflow, more on running models on your own machine

Sources

• nvidia/parakeet-tdt-0.6b-v3, model card: Open ASR Leaderboard average, 25 languages, CC-BY-4.0. Model card
• openai/whisper-large-v3-turbo, model card: Open ASR Leaderboard mean WER, MIT licence. Model card
• pyannote/speaker-diarization-community-1, the diarization model, CC-BY-4.0. Model card
• NASA historical sounds, the Apollo 11 and Apollo 13 clips. Public-domain audio
• claude-transcribe on GitHub, the skill’s source, benchmarks and the alternatives record. Repository