BRIDGE1MODEL QUALITY + LOCAL RUNTIME
Compare local LLM benchmarks by model artifact, workload and cache state, with source conditions beside every result.
Avoid choosing a coding model by decode rate alone. Long-prompt waits and cache misses can dominate a session. Bridge1 records model artifacts, hardware, baselines and sample limits; published coding scores describe a separate question under the publisher’s test setup.
The benchmark, the score, the exact tested artifact, the quantization, the test setup, the source, the date, and whether the publisher or an independent party ran it.
A score for the base weights is not a score for Bridge1's quantized artifact.
The machine, memory, engine, context, cache state, prefill and decode, the thermal state, and the method.
A runtime result is not a ranking of model quality.
Time to first token (TTFT) is the wait until the first output token, including work before generation starts. A KV cache stores attention state computed from earlier tokens; prompt caching reuses eligible matching prefixes to avoid repeated prefill. Cache reuse requires compatible model state and input, not merely a long chat.
These are historical measurements behind the home page. Each row states its model, engine and what was measured, and the number of runs where the record gives one. Cold start has two parts: loading the model, and the time to first token (TTFT) on the first long prompt. For a reasoning model the first visible answer can come after a thinking phase that follows prefill.
Every figure is from one Apple M1 Max with 64 GB, on a build before the 0.7.3 release. We have not yet repeated these on the released build.
| KPI | RESULT | WHAT WAS MEASURED | MODEL AND ENGINE | RUNS |
|---|---|---|---|---|
| Model load | 8.1 to 9.1 s | Cold load of a 21.7 GB model with the weights locked in memory. The ledger characterises it as close to disk-bandwidth-bound. The first request then took 1.5 s to its first token. The record does not state that prompt's length, so it does not show what a long first turn costs, and it does not say how warm the operating system's file cache was. | Ornith 1.5 35B-A3B, Q4_K_M · llama.cpp | Not recorded |
| Cold TTFT, long prompt | Not measured | A valid cold-start measurement on the released 0.7.3 build is pending. An earlier attempt ran on a loaded machine and is not used. A prefill rate is not a time to first token, so none is shown. | Ornith 1.5 35B-A3B · MLX (planned) | — |
| Cache after a restart | 99.7% of prompt tokens read from cache | The same conversation resent unchanged after fully quitting and reopening the app. Qwen3.8 27B, through the gateway: 5,077 of 5,093 prompt tokens from cache. Qwen3-VL-2B, run engine-direct: 3,952 of 3,963, logged at 0.099 s; its cold first turn of 3,898 tokens logged 3.579 s. | Qwen3.8 27B (DFlash2) and Qwen3-VL-2B · MLX | 1 per model |
| Second turn of one long chat | 15.8 s instead of 359.7 s (22.8x) | The second turn of one session, with the whole 28,722-token prefix reused. A historical single-session characterisation, not a time-to-first-token measurement. | Qwen3.8 27B, 4-bit · MLX | 1 run |
| Switching between two long chats | 33.9x and 17.9x (16.8 s against 567.4 s cold, 24.7 s against 441.5 s cold) | Two unrelated conversations of about 21K tokens each, alternated A, B, A, B. 99.9% of the tokens came from cache on each return. Elapsed time including decode, not time to first token. Measured before the current memory plan sized the cache. | Qwen3.8 27B, 4-bit · MLX | 1 per direction |
| Warm TTFT, same chat | 49 to 51 ms to the first token | After a first request that took 1.5 s, the next three requests in the same conversation reused the shared prefix. llama.cpp reuses a prefix within one conversation only. The record does not state the prompt's length. | Ornith 1.5 35B-A3B, Q4_K_M · llama.cpp | 3 requests |
No valid measurement yet: cold start through the gateway on the released 0.7.3 build, and cache reuse after a restart on the default coding model, Ornith 1.5 35B-A3B on MLX.
What the cache is: on MLX, a prompt cache that matches the start of a conversation exactly and is kept on disk, so it survives a restart. It is not a compiled-kernel cache. llama.cpp does not keep a cache across conversations.
These are the published coding scores for the base weights of each model, with their source. Scores from interested parties, competitors and community cards are not shown.
Scores from different setups are not a ranking. Use them to see what evidence exists, not to rank unlike runs.
| MODEL | SCORE DESCRIBES | SWE-BENCH VERIFIED | TERMINAL-BENCH | LIVECODEBENCH |
|---|---|---|---|---|
| Qwen3.6 35B-A3BMLX · mxfp4 | base weights | 73.4 creator | 2.0 · 51.5 creator | v6 · 80.4 creator |
| Qwen3.6 35B-A3B MTPGGUF · UD-Q3_K_M | base weights | 73.4 creator | 2.0 · 51.5 creator | v6 · 80.4 creator |
| Qwen3.6 27BMLX · 4-bit | base weights | 77.2 creator | 2.0 · 59.3 creator | v6 · 83.9 creator |
| Qwen3.6 27B MTPGGUF · Q8_0 | base weights | 77.2 creator | 2.0 · 59.3 creator | v6 · 83.9 creator |
| Qwen3.8 27BMLX · 4-bit | base weights | — | 2.1 · 73.0 creator | v6 · 90.3 creator |
| Qwen3.8 27BGGUF · UD-Q4_K_S | base weights | — | 2.1 · 73.0 creator | v6 · 90.3 creator |
| Qwen3.5 2BMLX · 4-bit · light tasks only | base weights | 5.6 independent | 3.0 independent | — |
| Qwen3.5 4BMLX · 4-bit · light tasks only | base weights | — | 25.84 independent | v6 · 55.8 creator |
| Gemma 4 26B-A4BQAT MLX · 4-bit | base weights | — | 2.1 · 38.95 independent | v6 · 77.1 creator |
| Ornith 1.5 35B-A3BMLX · OptiQ 4-bit | base weights | 79.0 creator | 2.1 · 67.8 creator | — |
| Ornith 1.5 35B-A3BGGUF · Q4_K_M | base weights | 79.0 creator | 2.1 · 67.8 creator | — |
| Ornith 1.5 9BMLX · OptiQ 4-bit | base weights | 70.6 creator | 2.1 · 46.2 creator | — |
| Ornith 1.5 9BGGUF · Q4_K_M | base weights | 70.6 creator | 2.1 · 46.2 creator | — |
| Muse Glimmer 30BGGUF · Q6_K_L | base weights | 76.0 creator | 2.1 · 51.7 creator | — |
| Ternary Bonsai 27BMLX · 2-bit ternary | shipped artifact | — | — | 82.75 creator |
| Ternary Bonsai 2 27BMLX · 2-bit ternary | shipped artifact | 60.8 creator | 2.1 · 52.8 creator | v6 · 90.07 creator |
Not one leaderboard: versions, test setups, test dates, thinking modes and quantizations differ between rows.
Provenance: “creator” means the model's publisher reported the score; “independent” means a third party ran it. A dash means no verified published value is shown.
Artifact boundary: “base weights” means the score describes the model family's original weights, not Bridge1's quantized artifact. Only the Ternary-Bonsai rows describe the shipped artifact.
These records answer specific questions about how an artifact runs locally. Decode speed for each model is on the Models page, where it is shown with its basis.
Compare numbers only when the model, artifact, engine, workload and conditions match.
On 25 and 26 September 2026 we ran 40 small coding tasks in tiny Python repositories through Claude Code on local models. Each task is scored by hidden unit tests. Each model's result is shown below with its 95% range.
This is our own test, not a public benchmark. The reference model passed 39 of 40, so these tasks separate strong models only weakly.
| MODEL | TASKS PASSED | TIME PER PASSED TASK, AGAINST THE REFERENCE | VERDICT AT A 10-POINT MARGIN |
|---|---|---|---|
| Qwen3.8 27B · reference | 39 of 40 | 1x | Reference |
| Ornith 1.5 35B-A3B | 40 of 40 | About 5.1x faster (range 4.2 to 6.3) | Not inferior |
| Qwen3.6 35B-A3B | 38 of 40 | About 4.9x faster (range 4.3 to 5.8) | Inconclusive |
| Qwen3.5 4B · 10 tasks | 9 of 10 | Not compared | Not compared |
Limits: Claude Code only; small repositories; the reference passed 39 of 40. A 10-point margin is the test's own threshold, not a product standard. We have not run a public benchmark such as SWE-bench on Bridge1 itself.
The chip, the unified memory, the macOS version, the Bridge1 release and the exact engine build.
The model, quantization, prompt length, output length, concurrency and scenario.
Cold or warm, cache status, date, method and known limits.
CHOOSE THE INPUT
No. Time to first token includes work before output starts, while decode rate measures output generation. Long prefill, loading and reasoning can dominate the wait. Bridge1’s evidence distinguishes those stages, so a tokens-per-second figure does not conceal the cost of a long coding prompt.
Prefill and decode definitionsCompare the benchmark, tested artifact, quantization and harness before comparing scores. A publisher’s SWE-bench result is not a local Bridge1 runtime measurement. Keeping those sources separate avoids choosing a model for fast decode when the real requirement is reliable multi-step coding.
They are historical measurements on one M1 Max 64 GB, mostly single runs, with their original baselines. A released-build cold time to first token has not been measured. The evidence retains that gap instead of turning an older warm or cache-hit result into a fresh-start promise.
MLX restart reuse was verified on Qwen3.8 27B and the Qwen3-VL-2B helper, one run each; it is not yet verified on default Ornith 1.5 35B-A3B. Matching prompt prefixes can avoid repeated prefill, but a changed prefix or different model may miss. The evidence row states the gateway versus engine-direct conditions.
No. Runtime latency and task correctness answer different questions. Bridge1 reports model-quality evaluations separately from prefill, decode and cache measurements. That lets you weigh task evidence against waiting time without treating a drafter speed result as proof of better code.