Local LLM speed: prefill, decode and Drafter Fusion.

Local LLM speed depends on prefill, decode and Drafter Fusion; each measured result keeps its conditions.

A round orange-ringed speed sign and a small blank plate stand on a stone bridge deck with an orange lane. — 120+ tok/s decode · Drafter Fusion.Catalog decode · Qwen3.5 2B · 4-bit · light tasks only · short context · Apple M1 Max, 64 GB · measured.

Long prompts can spend minutes in prefill before decode starts. Matching-prefix cache reuse avoids repeating input work; prediction paths can change output speed. Read the measured results and their limits.

Checkpoint and prediction paths

Drafter Fusion is Bridge1's speed layer between the engine, the model and its drafter.

Each catalog entry names its checkpoint, built-in prediction head, prompt lookup or standard generation path. The table below identifies the method and its evidence.

DFlash2 is the second-generation checkpoint family; the main model checks proposed tokens before accepting them.

llama.cpp’s n-gram methods suggest draft tokens by matching patterns in token history, without an additional model. Method source

The dashboard reports the active path. If a checkpoint is missing, the model can load without it and report the gap. Read the measured result with its baseline and conditions.

Results depend on the model and workload. MLX checkpoints are Experimental. A measured decode gain does not guarantee a shorter reply time; inconclusive results remain labelled below.

Drafter Fusion: A prediction method proposes several tokens. Main model checks the proposed tokens. Matching tokens are kept; the rest are discarded. Only tokens the main model accepts become the reply.

Which drafting method each model uses

Model and profile	Method	Measured result
Qwen3.6 35B-A3Bmxfp4 · editor's choice · MLXExperimental	DFlash	Catalog decode measurement
Qwen3.6 27B4-bit · MLXExperimental		Measured comparison
Muse Glimmer 30BGGUF Q6_K_L · GGUF / llama.cppwith prompt lookupMature		Catalog decode measurement
Ornith 1.5 35B-A3BOptiQ 4-bit · default for coding · MLXExperimental	DFlash2	Measured comparison
Qwen3.8 27B4-bit · MLXExperimental	DFlash2	Measured comparison
Qwen3.6 27BGGUF Q8_0 · GGUF / llama.cpp	MTP	Catalog decode measurement
Qwen3.6 35B-A3BGGUF UD-Q3_K_M · GGUF / llama.cppwith prompt lookup		Catalog decode measurement
Qwen3.8 27BGGUF UD-Q4_K_S · GGUF / llama.cppwith prompt lookup		Catalog decode measurement
Ornith 1.5 35B-A3BGGUF Q4_K_M · editor's choice · GGUF / llama.cpp	n-gram lookup	Measured comparison
Ornith 1.5 9BGGUF Q4_K_M · editor's choice · GGUF / llama.cpp	n-gram lookup	Measured comparison
Gemma 4 26B-A4BQAT 4-bit · see note · MLX	standard generation	Catalog decode measurement
Ternary Bonsai 2 27B2-bit ternary · shipped artifact · MLX		Catalog decode measurement
Ternary Bonsai 27B2-bit ternary · shipped artifact · MLX		Catalog decode measurement
Ornith 1.5 9BOptiQ 4-bit · MLX		Catalog decode measurement
Qwen3.5 4B4-bit · light tasks only · MLX		Catalog decode measurement
Qwen3.5 2B4-bit · light tasks only · MLX		Catalog decode measurement
FrogNano-4B-2609Q4_K_M · Experimental · GGUF		Decode not measured
Qwen3.5 2BQ6_K · Experimental · GGUF		Decode not measured

Checkpoint and method credits: Jian Chen, Yesheng Liang and Zhijian Liu (Z Lab); Inco AI; Modal; jzinno; Meta; GGUF files by bartowski. NVIDIA’s Nemotron Post-Training Dataset v2 is credited by the Ornith checkpoint card. Qwen: Alibaba Cloud. Ornith: ornith-ai. Prediction-head GGUF publishers: ggml-org and unsloth.

Sources and credits: Z Lab paper · Inco AI · Z Lab and Modal checkpoint · jzinno checkpoint · Muse Glimmer files · NVIDIA dataset credit.

Measured on one Mac, with baselines and conditions

Model and path	Compared with	Result	Conditions
Qwen3.6 27B · checkpoint	without checkpoint	1.69x average on short prompts (1.57x to 1.79x); 1.36x average at depth (1.17x to 1.67x)	M1 Max 64 GB · MLX · Experimental · one request at a time 6 paired short prompts; 4 depth shapes
Qwen3.8 27B · catalog checkpoint profile	checkpoint default profile	1.37x (16.98 vs 12.38 tok/s)	M1 Max 64 GB · MLX · Experimental · through Bridge1's own path 4 runs per profile
Ornith 1.5 35B-A3B · checkpoint	without checkpoint	1.35x (range 1.26x to 1.44x, 24 paired runs); MIXED: registered statistic inconclusive twice	M1 Max 64 GB · MLX · Experimental · one request at a time 24 paired runs
Qwen3.8 27B · checkpoint	without checkpoint	1.20x, not separable from no change (4 runs per arm)	M1 Max 64 GB · MLX · Experimental 4 runs per arm
Ornith 1.5 9B · prompt lookup	no lookup	Edit-shaped gains are recorded; sample size unavailable, so no ratio is published here.	M1 Max 64 GB · llama.cpp Not recorded
Ornith 1.5 35B-A3B · prompt lookup	no lookup	Edit-shaped gains are recorded; sample size unavailable, so no ratio is published here.	M1 Max 64 GB · llama.cpp Not recorded

These are single-machine results. An edit-shaped workload can benefit where prose does not. Mixed and inconclusive results remain labelled; a gain on one model is not a promise for another.

Prefill reads the prompt. Decode generates the reply.

Prefill processes the input tokens before output starts. Decode generates output tokens; its rate is reported in tokens per second. Long input work and reasoning can dominate the wait.

Matching-prefix cache reuse avoids repeated input work when eligible. See time to first token and cache evidence →

Pinned files and visible runtime state

Bridge1 verifies model and checkpoint downloads against pinned revisions and hashes. The dashboard and Live Log report the active path, so a downloaded file cannot stand in for evidence.

A result names its artifact, engine, machine, memory, context, cache state, thermal state, method and date. Inspect model profiles and memory fit →

Compare Bridge1, Ollama and LM Studio designs →

Local speed questions

What is a drafter in speculative decoding?

A small prediction component proposes tokens that the main model checks before accepting. Bridge1 can use an external checkpoint, an embedded head or prompt lookup; some catalog entries use none. Availability does not guarantee shorter replies.

Public source · 2026-10-17 Catalog entries · 2026-10-17

What is DFlash?

It is a checkpoint-based method that proposes tokens for the target model to verify. Bridge1 uses this family on the two Qwen3.6 MLX entries and the Muse Glimmer GGUF entry; second-generation checkpoints are identified separately in the table.

Public source · 2026-10-17 Catalog entries · 2026-10-17

What is MTP (multi-token prediction)?

Multi-token prediction can use multiple output heads over a shared model trunk to predict future tokens. Bridge1’s three GGUF entries with prediction heads are listed in the method table.

Public source · 2026-10-17 Catalog entries · 2026-10-17

What does Drafter Fusion manage for my local model?

It manages the catalog’s prediction paths and reports the active one in the dashboard. MLX checkpoints are Experimental. If a checkpoint is unavailable, the model can load without it and report the gap. The method table links each entry to its measured evidence.

Inspect Drafter Fusion evidence

Does Drafter Fusion reduce every local LLM response time?

No. A decode gain can coexist with a long prefill, and an edit-shaped workload can benefit where prose does not. Experimental and inconclusive results remain labelled, so one favorable measurement does not become a universal latency promise.

Which prediction method actually ran on my model?

The dashboard and Live Log report the active method. A downloaded checkpoint does not prove it executed or shortened the request. The measured rows keep their baseline and conditions, so an installed file cannot stand in for evidence.

Why does a long coding prompt wait before the first token?

Prefill must process the input before decode generates output; reasoning can add more waiting before visible text. Matching-prefix cache reuse avoids repeated input work when eligible. The cache example records different turns, not a controlled first-token guarantee for every model or prompt.

See cache reuse measurements

Can I expect the published Qwen3.8 27B speed on my Mac?

The recorded result only describes its model artifact, engine, workload and reference Mac. Memory pressure, context depth and thermal state can change decode and prefill. Bridge1 exposes the measured path and offers optional calibration on your own machine instead of projecting the M1 Max result onto unmeasured hardware.