Finding the best local coding model for your machine

A repeatable process to answer, for one specific computer: which local model, runtime, and configuration should I code with? Everything runs against OpenAI-compatible servers that a coding harness can actually use.

What this project measures

• Usable speed at real session depth, not benchmark speed. Decode speed falls as the context fills. A model that benchmarks at 60 tok/s can crawl at 2 tok/s mid-session.
• Context that fits the machine while it stays a desktop. Memory footprints are measured so the Mac remains usable while a model serves.
• Quality per quantization. Published scores cover full-precision models. What you run is a quant. One score per model and thinking mode; runtimes at standard quants share it.
• A pick per use, not a single winner. Models are used in more than one way, and the data is kept so each use can be read from it.

Every config gets a KV cache pick, a decode-vs-used-context sweep and an honest "capped by" verdict: speed floor, memory OOM, or model window. The usability floor here is 8 tok/s. EvalPlus gates every config; Mendel ranks the survivors.

Read the methodology before running anything. The flow is binding.

Setups

M1 Max, 32 GB

Apple Silicon, wired limit 25000 MB. Five models, three runtimes: llama-server, mlx_lm.server, and the PrismML llama.cpp fork. LM Studio was tried and retired. Depth sweeps and EvalPlus scores are complete for every model; two models have finished the agent task. The rule: MLX runtimes barely slow down but hit hard memory ceilings; llama runtimes hold their speed deeper at f16 KV, and their ceiling is the largest -c that loads.

Model / Config	Ctx	Cap	tok/s	EvalPlus	Coding
Qwen3.8-27B Q4_K_Mbartowski,llama-server–mtp/3f16xhigh	72k	mem	11.8→ 8.6	0.957/0.93996% completion	93mendel-blind
Qwen3.6-35B-A3B UD-Q4_K_XLunsloth,llama-server–mtp/3q8_0on	82k	speed	43.7→13.0	0.939/0.92197% completion	83mendel-guided
Qwen3.8-27B IQ3_S-mtpISTA-DASLab,llama-server–f16xhigh	147k	speed	† 14.1→ 8.3	0.945/0.92197% completion	80.5mendel-blind
Qwen3.8-27B AD-IQ3_SAtomicChat,llama-server–mtp/3f16medium	104k	mem	† 15.8→10.3	0.988/0.927100% completion	38% / 37.5mendel-blind
Gemma-4-26B-A4B UD-Q4_K_XLunsloth,llama-server–mtp/2f16on	197k	mem	† 60.3→17.3	0.884/0.86089% completion	47.5mendel-blind
Gemma-4-12B Q4_K_XLunsloth,llama-server–f16off	245k	mem	† 24.6→ 8.9	0.976/0.939100% completion	38% / 37.5mendel-guided
Ternary-Bonsai-27B 2-bitprism-ml,mlx_lm.server–f16on	53k	mem	† 24.5→17.3	0.915/0.88497% completion	38% / 37.5†mendel-blind
Ternary-Bonsai-27B Q2_g64prism-ml,prism-llama–q4_0+biason	33k	speed	† 14.8→ 7.9	0.927/0.89098% completion	38% / 31.5mendel-guided
Qwen3.8-27B 4-bitmlx-community,mlx_lm.server–f16low	25k	mem	† 17.0→15.3	0.976/0.927100% completion	13% / 12.5†mendel-blind
Qwen3.6-35B-A3B 4-bitmlx-community,mlx_lm.server–f16on	37k	mem	† 55.1→37.4	0.939/0.92197% completion	pending
Gemma-4-26B-A4B 4-bitmlx-community,mlx_lm.server–f16on	66k	mem	† 51.0→12.8	0.713/0.70172% completion	pending

† from an earlier serving config or method; re-run pending.

¹ LM Studio's MLX engine — the only runtime that loads this model's gemma4_unified architecture. It is retired on that machine; see why.

² PrismML's llama.cpp fork, an approved exception to the no-forks rule.

Legend

• Ctx, the usable context: the deepest context the config served above the floor, set by Cap. The coding harness gets the same window, rounded down to a multiple of 4096. On MLX the cell shows the harness window, 5 percent under the measured ceiling, because that runtime often triggers macOS memory compression near it.
• Cap, what stops the context from growing: memory holds the weights, the drafter, a vision adapter and the runtime's buffers, and what is left is context. Some models do not fit their trained window; others fit it and then decode too slowly to use. The floor is 8 tok/s. mem means memory ran out first, speed means decode fell under the floor first.
• tok/s, decode speed shallow, near an empty context, then deep, at Ctx. Most tools report the shallow number only, but engineering work and long documents run at depth, where speed falls. A drafter (MTP, speculative decoding) often changes the picture, and for some models it can help at one depth and hurt at another; nobody knows before measuring, so every drafter row is read on real text at more than one draft depth, with its acceptance. See the measurement rules.
• EvalPlus, scored once per model and thinking mode; runtimes serving the same model at a standard quant share the score. Aggressive quants (for example the prism fork's calibrated q4 KV) do not share; they pass the gate separately. Each run gets an output budget from a ten-problem calibration, capped at 30000 tokens. A problem that runs to the cap counts as failed; the completion percentage says how many finished. The cap is what this machine can wait for, not the model's ceiling, so a capable model at a high reasoning level can lose points to it.
• Coding, a simulated pull request: the pi coding agent fixes a real issue in a real repository with known traps, over many turns, not one prompt. A stalled agent gets a fixed number of nudges; a nudge the model caused costs points. Mendel blind gives the terse issue and the model plans the work itself. Mendel guided gives the same task as steps with the traps disclosed, so a smaller model can serve as an executor rather than a planner. The pill names the test. Of the config's valid runs the cell shows the one with the most libraries done, then the higher score; a muted percentage before the score is the share of libraries done when the run did not finish, invalid when every attempt was. Rows sort by the average of the EvalPlus base score and this one; a row with only one of the two sorts after every row with both.

model	report	benchmarks
Qwen3.6-35B-A3B (MoE)	report	data
Gemma-4-26B-A4B (MoE)	report	data
Gemma-4-12B-it	report	data
Ternary Bonsai-27B	report	data
Qwen3.8-27B	report	data

Also on this setup: the comparison page with the full depth and quality tables, the setup overview with the machine configuration, and historical measurements taken under retired memory limits.

More setups

A PC with an NVIDIA GPU comes next: Bonsai on the CUDA builds of the prism fork, and lower quants of the other models. It gets the same shape — setup overview, comparison, reports, benchmarks.

Why this exists

This site is the worked example for one machine, but the process applies to any box: substitute your memory budget and your candidates.

The reason the depth axis matters more than any published benchmark: a real coding session here measured 1.7 tok/s at 135K used tokens, on a config whose near-empty benchmark said 62 tok/s. Context maxima alone are storage, not speed. So every config is swept against used context until it drops under the usability floor or runs out of memory, and the floor — not the window — sets the harness compaction threshold.

Published quality scores have the same problem. They cover full-precision weights, and what fits on a desktop is a quant, so quality is measured on the quant actually served. Narrow differences between runtimes' standard quants do not count: one score per model and thinking mode covers them. Aggressive or calibrated quants get their own gate.