A local 27B model shipped code through my dev workflow. The harness did the hard part.

TLDR; On a 64 GB M5 Pro I measured Qwen3.8-27B at 20.7 tokens a second with multi-token prediction switched on, and 29 to 30 on the oMLX author’s 6-bit quant. oMLX ships MTP off, and the community figure for that default is about 10. Quant and decoder experiments after that moved throughput by single-digit percentages. Then I put the model inside my test-first dev workflow and it built a 270-line Ruby CLI end to end and caught its own mistake on the red run. A later flag-sized task took Sonnet 33 seconds and 20 cents; the local 35B did it in 2 minutes 51 for the electricity. Local buys privacy and free tokens. The harness buys accuracy.

I keep a six-task Ruby eval for local models: token bucket, LRU cache with TTL, interval merging, a cron parser, CSV diff, markdown table. Forty-nine minitest assertions the model never sees: it gets the prose spec, writes the code, and the harness runs the tests afterwards. One shot, Qwen’s official sampling, medium reasoning effort. When I first ran it a fortnight ago, Claude Sonnet and Opus scored 49/49 and Qwen3.8-27B matched them in about 22 minutes against Sonnet’s 52 seconds. This week I went looking for the speed, and then for what the model could do inside a real workflow instead of a one-shot prompt.

Everything below ran on an M5 Pro with 64 GB and oMLX 0.6.3. oMLX is an Apple-silicon serving stack; MTP is multi-token prediction, the drafter built into the model’s own weights; oQ6e is the oMLX author’s 6-bit quant and MTPLX a community one. Every configuration ran once (“run 1”); where I reran a failed task, the table shows a “best of 2”. Nothing ran more than twice.

Two changes, three times the speed

oMLX ships with multi-token prediction switched off. The community figures on omlx.ai put this model on an M5 Pro at about 10 tokens a second, and that is close to the memory-bandwidth wall: 28 GB of 8-bit weights over a 307 GB/s bus caps plain decoding near 11. Speculative decoding beats the wall by spending one weight read on several tokens: draft three, verify them in a single pass, keep the ones that survive. Switch mtp_enabled on and the same 8-bit weights run at 20.7 tokens a second, with 86% of drafts accepted. Swap to Jundot/Qwen3.8-27B-oQ6e-mtp, the oMLX author’s 6-bit quant at 22 GiB, and it runs at 29 to 30 with 87 to 90% of drafts accepted.

That matched my previous best, a llama.cpp Q8 build with a Q4 draft model. The oMLX run scored 39/49 on the first pass and 49/49 after rerunning the one failure, 20.3 minutes at 29.5 tokens a second, against llama.cpp’s 49/49 in 19.0 minutes at 22 to 25. The llama.cpp figure is from a fortnight earlier on a different code path, so treat the wall-clock gap as indicative.

Run 1 is a single pass of all six tasks. Where shown, Best of 2 reruns only the tasks that failed and keeps the better result, so it flatters that row. Wall clock is noisier than it looks: at temperature 1.0 the model thinks a different amount each run, 27,000 tokens one time and 36,000 the next for identical prompts. The score column carries the same variance. Treat wall-clock gaps and score differences under about ten assertions as unresolved; tokens a second, the median across the six tasks, is the number to compare.

DFlash2 is a real but small win on Apple silicon. It is a 1.9-billion-parameter drafter from Inco AI that proposes tokens in blocks and stands in for the model’s built-in MTP head. Inco AI’s model card claims 3.11x over plain decoding on HumanEval on an H200, against 2.20x for MTP; their numbers, their hardware. On my Mac, with the draft quantised to 4-bit, per-task throughput ran 30.7 to 33.5 tokens a second against MTP’s 25.6 to 29.9, about 8% at the median, one run each. oMLX has not published a DFlash2 speedup for this model on Apple silicon; the nearest figure in its notes is about 1.4x for DFlash on a 30B vision model, against plain decoding rather than MTP, so it does not bound the gain either way. As of oMLX 0.6.3, DFlash2 also runs one request at a time and keeps its own prompt cache, so I went back to MTP for daily use.

Four-bit weights buy almost nothing once MTP is on. With MTP each weight read yields several tokens, so weight size matters less: the 4-bit MTPLX build at 19 GB against the 6-bit oQ6e at 22 GiB, different quant families, bought 4%. The 4-bit MTPLX build was also the only one to fail the markdown-table task, on both attempts (5/7, then 1/7); the three 6-bit and 8-bit runs passed it 7/7.

The fast mixture-of-experts model is fast and wrong. Qwen3.6-35B-A3B runs 3.5x quicker and thinks twice as many tokens on the same tasks, so wall clock only halves. Its misses are logic errors rather than variance: wrong interval merges, wrong cron dates, a bad CSV.parse keyword. Best of 2 it reached 35/49 against the 27B’s 49/49, and its 32/49 on the first pass is what the no-thinking Qwen3-Coder-Next, 80B with 3B active, scored a fortnight ago.

The 27B’s miss was always the same miss

Across every 27B configuration I ran, never more than twice per stack, the cron task flipped between 1/11 and 11/11 on every decoder; the 4-bit build’s markdown-table failure was the only other miss that recurred. Each failing run contained one line: Time#change, an ActiveSupport method, called on a plain Ruby Time. The model writes Rails by reflex. None of the quant or decoder settings I tried removes that reflex, and a rerun at temperature 1.0 only sometimes dodges it; it lives in the weights.

A one-shot eval punishes that reflex at full price. A workflow that runs tests does not, so I ran one.

The build

My dev workflow is a Claude Code plugin built on the tech-lead pattern. For each piece of work it runs, in order:

1. Triage, then an approved brief with acceptance criteria written as tests.
2. Decomposition into tasks.
3. Per task: a feature branch, failing tests first, implementation until green.
4. A code-review agent.
5. A merge that a git hook refuses until the review field is filled in.

I pointed a headless Claude Code at oMLX with the 27B as the model and gave it an approved brief for a small tool: a Ruby CLI that reads a Claude Code session transcript and prints turns, tool counts, token totals and duration.

It triaged the lane correctly, read the task-file format before decomposing, verified the brief’s expected counts against the fixture with a Ruby one-liner before writing tests, and split the work into two tasks. On the first task it wrote the tests, ran them red, wrote the implementation, ran them again, and hit this:

Array#index_with isn’t available in this Ruby. Let me check the version and fix.

Same reflex as the eval. index_with is ActiveSupport. The difference is that the red test caught it and the model replaced it with a hash literal on the next turn.

The review agents also ran locally, the first on the 35B and the second on the 27B. Each approved with a verdict-first report, ran the suite itself, and re-derived the fixture counts with its own script; the first left two medium findings that the main loop applied before committing.

Two tasks, two reviews, two no-fast-forward merges, a 270-line CLI, 13 tests and 50 assertions green. Run against its own .jsonl, the finished CLI reported the build session as 335 user turns, 486 assistant turns and 149 Bash calls over 13 hours 46 minutes.

What broke was plumbing.

• My global settings pin a cloud model, so the first run asked oMLX for claude-fable-5 and got a 404. The fix is passing --model opus, which oMLX maps to the local id.
• I had mapped Claude Code’s Opus tier to the 27B and its Sonnet tier to the 35B: a careful planner over a fast implementer. Two resident models plus the prompt cache came to 43 GB, and oMLX’s memory guard rejected the reviewer’s 58,000-token prompt, then the main loop at 30,000. On 64 GB it is one local model per box.
• Even with one model, a resumed session sat at 70,000 tokens of context after one task and was rejected again. Claude Code never compacted because nothing told it the window was small. CLAUDE_CODE_AUTO_COMPACT_WINDOW=100000 plus CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=50 fixes it (the variable clamps to a 100K floor).
• My git hook blocks git add -A && git commit as a single command, because it checks for unstaged task files before the add runs. The model read the hook’s message, split the commands and carried on.

That took five headless invocations and three server restarts. The loop survived every one, because the state lives in the branch and the task file, not in the session.

One brief, three models

With the plumbing fixed I ran a third task, a --top N flag for the same CLI, three times from the same commit: locally on the 35B, and in the cloud on Sonnet and Opus 5. The runs were not identical. The local run branched from main; the cloud runs came after that work had been merged, so their brief named the commit and told them to ignore main. A first Sonnet run under the original wording found the work done and stopped (14 seconds, 25 cents); I discarded it and reran. I also wrote the brief and the test spec, reviewed the local model’s diff and fixed its gap. The judgement below is not blind, and the task is about 50 lines of diff.

Costs are Claude Code’s own estimates. All three produced the same core change, near line for line: sort the tools, take the first N. All three wrote tests first, used a conventional commit, stayed on their branch. The local model added a fifth test I had not asked for. I would have merged any of the three.

The gap is in what the brief did not spell out. The prose said non-integer input should exit 2. I gave a numbered test for --top 0 and none for --top abc. Sonnet and Opus inferred the case; the local model did not, and --top abc crashed. It also read the 290 KB fixture six times, 25 tool calls to Sonnet’s 12.

The local model does what the tests say; the cloud models do what you meant. The accuracy of a local model in a workflow is set by how completely the brief is written as tests, and that is under my control in a way the weights are not.

Where local pays

• Privacy and offline. Every token stayed on the laptop. For client work with data I cannot send anywhere, this is now a workable path.
• Cost, at volume. Sonnet did the task for 20 cents. The local model did it for the electricity. That only matters once the task count is large or the orchestrator is also local.
• Speed, no. On this task Sonnet finished in half a minute against nearly three for the local model and Opus; one task, and largely a difference in tool calls rather than decode speed.

The setup I have kept:

• A cloud model as orchestrator; it writes the briefs and reviews the diffs.
• A headless local Claude Code as the implementer, for standard tasks with an approved brief and a runnable test command.
• One local model resident at a time.
• A compaction window under the memory guard.
• Every requirement in a brief written as a numbered test, because the model will meet the tests and stop.