Performance · Results 2026-10-25b

Measured,
not promised

Loculus Core against industry-standard llama.cpp on 12 popular open models: the same GGUF file for both engines, on the same machine, one engine at a time. Two kinds of test: the standard llama-bench measurements, pp512 and tg128 as the context fills, and real serving through each engine's OpenAI-compatible API.

Processor: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
GPU: AMD Radeon Graphics (RADV GFX1151) · Mesa 25.2.8
Memory: 92.9 GiB unified, shared by CPU and GPU
System: Ubuntu 24.04.4 LTS · Linux 7.0.0-38-generic
Loculus Core: build b1ed73f · Vulkan (RADV)
llama.cpp: b10702 (e42214804) · Vulkan (RADV) · -ngl 99 -fa on

1.49×

Faster prompt processing

pp512, geometric mean over 48 model and depth pairs; up to 3.4× at long context

1.54×

Sooner first token

Serving a ~13k-token prompt, geometric mean over 12 models; up to 2.3×

1.52×

Faster agent turns

Wait before each answer in an 8-turn coding-agent session, geometric mean over 10 models; up to 2.6×

17%

Less peak memory

Geometric mean over 48 engine runs; lower in 44 of them

Prompt processing · pp512 at 32K depth

Tokens per second processing a 512-token prompt with 32,768 tokens already in context (llama-bench pp512 @ d32768): where agents and long documents spend their time. Higher is better.

Qwen3.8 27B 228 +74%

Gemma 4 12B 326 −4%

Gemma 4 26B-A4B 487 −14%

gpt-oss 20B 1,102 +147%

gpt-oss 120B 520 +99%

Nemotron 3.5 Lightning 30B-A3B 851 +8%

Qwen3.8 Flash-Next 177B-A3B 357 +84%

Qwen3 30B-A3B 364 +99%

Llama 3.2 3B 786 +184%

Mistral 7B v0.3 450 +239%

Phi-4 mini 689 +188%

GLM-4 9B 381 +36%

View as table

Model	llama.cpp	Loculus Core	Gain
Qwen3.8 27B	131	228	+74%
Gemma 4 12B	341	326	−4%
Gemma 4 26B-A4B	567	487	−14%
gpt-oss 20B	447	1,102	+147%
gpt-oss 120B	262	520	+99%
Nemotron 3.5 Lightning 30B-A3B	790	851	+8%
Qwen3.8 Flash-Next 177B-A3B	194	357	+84%
Qwen3 30B-A3B	183	364	+99%
Llama 3.2 3B	277	786	+184%
Mistral 7B v0.3	133	450	+239%
Phi-4 mini	239	689	+188%
GLM-4 9B	281	381	+36%

Time to first token · ~13k-token prompt

Served over each engine's OpenAI-compatible API and streamed: seconds until the first token of the answer arrives. Lower is better.

Qwen3.8 27B 45.2 1.2× faster

Gemma 4 12B 25.7 1.2× faster

Gemma 4 26B-A4B 16 1.1× faster

gpt-oss 20B 8.5 1.6× faster

gpt-oss 120B 23.4 1.4× faster

Nemotron 3.5 Lightning 30B-A3B 6.9 2.3× faster

Qwen3.8 Flash-Next 177B-A3B 28.6 2.2× faster

Qwen3 30B-A3B 12.3 1.6× faster

Llama 3.2 3B 6.3 1.6× faster

Mistral 7B v0.3 17.6 1.7× faster

Phi-4 mini 7.1 1.6× faster

GLM-4 9B 14.8 1.4× faster

View as table

Model	llama.cpp	Loculus Core	Gain
Qwen3.8 27B	56.1	45.2	1.2× faster
Gemma 4 12B	30.6	25.7	1.2× faster
Gemma 4 26B-A4B	17.5	16	1.1× faster
gpt-oss 20B	13.4	8.5	1.6× faster
gpt-oss 120B	33.5	23.4	1.4× faster
Nemotron 3.5 Lightning 30B-A3B	15.6	6.9	2.3× faster
Qwen3.8 Flash-Next 177B-A3B	63.7	28.6	2.2× faster
Qwen3 30B-A3B	19.9	12.3	1.6× faster
Llama 3.2 3B	10.1	6.3	1.6× faster
Mistral 7B v0.3	29.9	17.6	1.7× faster
Phi-4 mini	11.6	7.1	1.6× faster
GLM-4 9B	20.5	14.8	1.4× faster

Agent session · wait per turn

A coding-agent session served over the API: a 6k-token system prompt and tool list, then turns adding ~500 tokens of tool output each. Median seconds to first token over the follow-up turns, where reusing the shared prefix decides the wait. Lower is better.

Qwen3.8 27B 2.14 1.2× faster

Gemma 4 12B 1.26 1.2× faster

Gemma 4 26B-A4B 0.77 1.2× faster

gpt-oss 20B 0.38 2.0× faster

gpt-oss 120B 0.98 1.7× faster

Nemotron 3.5 Lightning 30B-A3B 0.66 2.1× faster

Qwen3.8 Flash-Next 177B-A3B 1.58 2.6× faster

Qwen3 30B-A3B 0.59 1.4× faster

Llama 3.2 3B 0.33 1.2× faster

GLM-4 9B 0.7 1.3× faster

View as table

Model	llama.cpp	Loculus Core	Gain
Qwen3.8 27B	2.5	2.14	1.2× faster
Gemma 4 12B	1.54	1.26	1.2× faster
Gemma 4 26B-A4B	0.92	0.77	1.2× faster
gpt-oss 20B	0.75	0.38	2.0× faster
gpt-oss 120B	1.63	0.98	1.7× faster
Nemotron 3.5 Lightning 30B-A3B	1.38	0.66	2.1× faster
Qwen3.8 Flash-Next 177B-A3B	4.05	1.58	2.6× faster
Qwen3 30B-A3B	0.84	0.59	1.4× faster
Llama 3.2 3B	0.4	0.33	1.2× faster
GLM-4 9B	0.88	0.7	1.3× faster

Model by model

pp512 and tg128 as the context fills

Generation (tg128) is bound by memory bandwidth, so the engines land close: Loculus Core runs at 1.03× llama.cpp's speed on geometric mean, ahead on 4 of 12 models with an empty context and 8 of 12 at 32K.

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	371	301	+23%
4K	322	265	+21%
16K	275	208	+32%
32K	228	131	+74%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	12.9	12.9	+0%
4K	12.6	12.6	+0%
16K	12	12	+0%
32K	11.1	11.3	−2%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	930	646	+44%
4K	693	549	+26%
16K	436	430	+1%
32K	326	341	−4%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	28.2	28.5	−1%
4K	26.1	26.1	+0%
16K	25.6	25.3	+1%
32K	24.7	24.2	+2%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,614	1,136	+42%
4K	1,331	944	+41%
16K	698	753	−7%
32K	487	567	−14%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	55.7	53.7	+4%
4K	49.8	47.4	+5%
16K	47.7	44.5	+7%
32K	43.6	40.6	+7%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,894	1,257	+51%
4K	1,797	1,079	+66%
16K	1,347	724	+86%
32K	1,102	447	+147%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	75.2	78.1	−4%
4K	69.3	74.5	−7%
16K	62.9	66.2	−5%
32K	56.1	58.7	−4%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	758	553	+37%
4K	670	494	+36%
16K	594	355	+67%
32K	520	262	+99%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	51.7	56.5	−8%
4K	49.3	53.6	−8%
16K	44.6	47.9	−7%
32K	39.6	41.9	−5%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,061	1,077	−1%
4K	1,056	997	+6%
16K	906	905	+0%
32K	851	790	+8%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	65.9	61.7	+7%
4K	64.2	59.6	+8%
16K	60.9	58.8	+4%
32K	60.1	56.7	+6%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	611	378	+61%
4K	533	287	+86%
16K	443	233	+90%
32K	357	194	+84%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	34.9	32.8	+6%
4K	30.9	24.2	+28%
16K	29.8	17.9	+66%
32K	27.3	16.9	+62%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,599	1,176	+36%
4K	1,150	766	+50%
16K	590	343	+72%
32K	364	183	+99%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	102	118	−13%
4K	75.1	92.9	−19%
16K	54.4	60.7	−10%
32K	47.5	41.9	+13%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	3,276	2,409	+36%
4K	2,387	1,625	+47%
16K	1,224	641	+91%
32K	786	277	+184%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	99.9	99.1	+1%
4K	81	80.5	+1%
16K	62.8	53	+18%
32K	48.8	36.7	+33%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,284	1,040	+23%
4K	981	743	+32%
16K	657	325	+102%
32K	450	133	+239%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	47	48.1	−2%
4K	41.6	42.9	−3%
16K	35.4	32.8	+8%
32K	29.8	25.1	+19%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	2,823	2,089	+35%
4K	2,090	1,398	+49%
16K	1,080	561	+92%
32K	689	239	+188%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	79.5	81.5	−2%
4K	65.8	67	−2%
16K	51.9	44.4	+17%
32K	40.7	31.9	+28%

Prompt processing · pp512

Tokens per second on a 512-token prompt, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	1,130	926	+22%
4K	877	726	+21%
16K	564	422	+34%
32K	381	281	+36%

Generation · tg128

Tokens per second generating 128 tokens, by context depth

View as table

Context	Loculus Core	llama.cpp	Difference
Empty	34.3	35.5	−3%
4K	31	34.6	−10%
16K	27.3	31.8	−14%
32K	23.4	29.1	−20%

Serving four clients at once

Combined output tokens per second with four 1k-token requests in flight, 128 tokens each. Higher is better. Models marked one-at-a-time don't batch in Loculus Core yet.

Qwen3.8 27B · one at a time 9.3 −41%

Gemma 4 12B 35.2 +15%

Gemma 4 26B-A4B 57.5 +19%

gpt-oss 20B 72.3 +11%

gpt-oss 120B · one at a time 31.7 +6%

Nemotron 3.5 Lightning 30B-A3B · one at a time 45.3 +14%

Qwen3.8 Flash-Next 177B-A3B · one at a time 19.7 +2%

Qwen3 30B-A3B 73.5 +3%

Llama 3.2 3B 131 −1%

Mistral 7B v0.3 56 −4%

Phi-4 mini 115 +2%

GLM-4 9B 52 −4%

View as table

Model	llama.cpp	Loculus Core	Gain
Qwen3.8 27B · one at a time	15.8	9.3	−41%
Gemma 4 12B	30.5	35.2	+15%
Gemma 4 26B-A4B	48.3	57.5	+19%
gpt-oss 20B	65.1	72.3	+11%
gpt-oss 120B · one at a time	29.8	31.7	+6%
Nemotron 3.5 Lightning 30B-A3B · one at a time	39.7	45.3	+14%
Qwen3.8 Flash-Next 177B-A3B · one at a time	19.4	19.7	+2%
Qwen3 30B-A3B	71.1	73.5	+3%
Llama 3.2 3B	132	131	−1%
Mistral 7B v0.3	58.6	56	−4%
Phi-4 mini	113	115	+2%
GLM-4 9B	54.3	52	−4%

Peak memory at 32K depth

Memory each engine's process held at its peak during the pp512 / tg128 run, CPU and GPU together, in GiB. Lower is better.

Qwen3.8 27B 17.5 −12%

Gemma 4 12B 8 −17%

Gemma 4 26B-A4B 17.1 −10%

gpt-oss 20B 11.6 −13%

gpt-oss 120B 59.8 −3%

Nemotron 3.5 Lightning 30B-A3B 23.8 +1%

Qwen3.8 Flash-Next 177B-A3B 49.4 −35%

Qwen3 30B-A3B 11.5 −23%

Llama 3.2 3B 5.5 −43%

Mistral 7B v0.3 8.1 −35%

Phi-4 mini 6.4 −44%

GLM-4 9B 6.8 −22%

View as table

Model	llama.cpp	Loculus Core	Gain
Qwen3.8 27B	20	17.5	−12%
Gemma 4 12B	9.6	8	−17%
Gemma 4 26B-A4B	18.9	17.1	−10%
gpt-oss 20B	13.4	11.6	−13%
gpt-oss 120B	61.9	59.8	−3%
Nemotron 3.5 Lightning 30B-A3B	23.6	23.8	+1%
Qwen3.8 Flash-Next 177B-A3B	76.5	49.4	−35%
Qwen3 30B-A3B	14.9	11.5	−23%
Llama 3.2 3B	9.6	5.5	−43%
Mistral 7B v0.3	12.4	8.1	−35%
Phi-4 mini	11.4	6.4	−44%
GLM-4 9B	8.7	6.8	−22%

Speculative decoding is off for both engines in these runs. Loculus Core turns it on by default, so its real-world generation is faster on repetitive output such as code edits.
Even at temperature 0 the two engines write answers of different lengths, so serving is compared on time to first token and throughput rather than total time.
The agent session didn't complete on both engines for Mistral 7B v0.3 and Phi-4 mini, so they are left out of it; a dash in the tables marks a missing result.

Methodology

How we tested

pp512: Prompt processing: a 512-token prompt, in tokens per second. The same test llama-bench reports as pp512.
tg128: Text generation: 128 tokens, one at a time, in tokens per second. llama-bench's tg128.
Context depth: Tokens already in the KV cache when the test starts (llama-bench -d): 0, 4,096, 16,384, 32,768. Each engine test runs twice and the average is reported.
Peak memory: The process's anonymous memory plus its resident GPU memory (GTT and VRAM) from the kernel's per-process accounting, sampled every 20 ms. The model file's memory-mapped pages are page cache either engine shares, so neither is charged for them.
Time to first token: Each engine's own server (llama-server for llama.cpp) over the OpenAI chat completions API, streamed, temperature 0, the model's chat template. Prompts of ~1k, ~3.8k, ~13k tokens (exact counts depend on the tokenizer), each one unique so no cache helps, and 128 tokens out.
Agent session: A 6k-token system prompt and tool list, then 8 turns, each adding ~500 tokens of tool output and asking for a short answer. Reported: time to first token on the first turn, which processes the whole prompt cold, and the median over the follow-up turns, where reusing the shared prefix decides the wait. Total session time isn't compared: even at temperature 0 the engines write answers of different lengths.
Concurrent clients: 1, 2 and 4 clients at once with 1k-token prompts and 128 tokens out; combined output tokens per second. llama-server runs one slot per client.
Settings: The whole model on the GPU for both engines (llama.cpp -ngl 99 -fa on), a 32,768-token context per request. Speculative decoding is off in both, and Loculus Core's on-disk prompt cache is off, so both start cold. One engine at a time, with nothing else on the GPU.

Raw results

Every number

Every figure on this page, per model and context depth. The download holds the same results with the test system and engine builds.

Download results (JSON)

Engine: pp512, tg128 and peak memory by context depth

Model	Depth	pp512 · tok/s			tg128 · tok/s			Peak memory · GiB
Model	Depth	Loculus	llama.cpp	Δ	Loculus	llama.cpp	Δ	Loculus	llama.cpp
Qwen3.8 27BUD-Q4_K_M · 15.3 GiB	0	371	301	+23%	12.9	12.9	+0%	15.3	15.8
	4,096	322	265	+21%	12.6	12.6	+0%	15.8	16.5
	16,384	275	208	+32%	12.0	12.0	+0%	16.5	18.0
	32,768	228	131	+74%	11.1	11.3	−2%	17.5	20.0
Gemma 4 12BQ4_K_M · 6.6 GiB	0	930	646	+44%	28.2	28.5	−1%	7.0	8.0
	4,096	693	549	+26%	26.1	26.1	+0%	7.5	8.7
	16,384	436	430	+1%	25.6	25.3	+1%	7.7	9.1
	32,768	326	341	−4%	24.7	24.2	+2%	8.0	9.6
Gemma 4 26B-A4BUD-Q4_K_M · 15.8 GiB	0	1,614	1,136	+42%	55.7	53.7	+4%	16.1	17.3
	4,096	1,331	944	+41%	49.8	47.4	+5%	16.5	17.8
	16,384	698	753	−7%	47.7	44.5	+7%	16.8	18.3
	32,768	487	567	−14%	43.6	40.6	+7%	17.1	18.9
gpt-oss 20BMXFP4 · 11.3 GiB	0	1,894	1,257	+51%	75.2	78.1	−4%	10.9	11.8
	4,096	1,797	1,079	+66%	69.3	74.5	−7%	11.0	12.0
	16,384	1,347	724	+86%	62.9	66.2	−5%	11.2	12.6
	32,768	1,102	447	+147%	56.1	58.7	−4%	11.6	13.4
gpt-oss 120BMXFP4 · 59 GiB	0	758	553	+37%	51.7	56.5	−8%	58.7	59.6
	4,096	670	494	+36%	49.3	53.6	−8%	58.8	59.9
	16,384	594	355	+67%	44.6	47.9	−7%	59.2	60.7
	32,768	520	262	+99%	39.6	41.9	−5%	59.8	61.9
Nemotron 3.5 Lightning 30B-A3BUD-Q4_K_M · 23.5 GiB	0	1,061	1,077	−1%	65.9	61.7	+7%	23.2	23.0
	4,096	1,056	997	+6%	64.2	59.6	+8%	23.3	23.1
	16,384	906	905	+0%	60.9	58.8	+4%	23.5	23.4
	32,768	851	790	+8%	60.1	56.7	+6%	23.8	23.6
Qwen3.8 Flash-Next 177B-A3BUD-Q2_K_XL · 73.5 GiB	0	611	378	+61%	34.9	32.8	+6%	48.4	74.2
	4,096	533	287	+86%	30.9	24.2	+28%	48.6	74.6
	16,384	443	233	+90%	29.8	17.9	+66%	49.0	75.4
	32,768	357	194	+84%	27.3	16.9	+62%	49.4	76.5
Qwen3 30B-A3BUD-IQ1_S · 8.4 GiB	0	1,599	1,176	+36%	102	118	−13%	8.6	8.8
	4,096	1,150	766	+50%	75.1	92.9	−19%	9.0	9.6
	16,384	590	343	+72%	54.4	60.7	−10%	10.0	11.9
	32,768	364	183	+99%	47.5	41.9	+13%	11.5	14.9
Llama 3.2 3BQ4_K_M · 1.9 GiB	0	3,276	2,409	+36%	99.9	99.1	+1%	2.0	2.6
	4,096	2,387	1,625	+47%	81.0	80.5	+1%	2.5	3.5
	16,384	1,224	641	+91%	62.8	53.0	+18%	3.7	6.1
	32,768	786	277	+184%	48.8	36.7	+33%	5.5	9.6
Mistral 7B v0.3Q4_K_M · 4.1 GiB	0	1,284	1,040	+23%	47.0	48.1	−2%	4.2	4.3
	4,096	981	743	+32%	41.6	42.9	−3%	4.7	5.3
	16,384	657	325	+102%	35.4	32.8	+8%	6.1	8.3
	32,768	450	133	+239%	29.8	25.1	+19%	8.1	12.4
Phi-4 miniQ4_K_M · 2.3 GiB	0	2,823	2,089	+35%	79.5	81.5	−2%	2.5	3.3
	4,096	2,090	1,398	+49%	65.8	67.0	−2%	3.0	4.3
	16,384	1,080	561	+92%	51.9	44.4	+17%	4.4	7.4
	32,768	689	239	+188%	40.7	31.9	+28%	6.4	11.4
GLM-4 9BQ4_K_M · 5.7 GiB	0	1,130	926	+22%	34.3	35.5	−3%	5.5	6.2
	4,096	877	726	+21%	31.0	34.6	−10%	5.7	6.5
	16,384	564	422	+34%	27.3	31.8	−14%	6.1	7.4
	32,768	381	281	+36%	23.4	29.1	−20%	6.8	8.7

Serving: seconds to first token and throughput over the API

Model	TTFT · ~1k prompt · s		TTFT · ~3.8k prompt · s		TTFT · ~13k prompt · s		Agent first turn · s		Agent follow-up · s		4 clients · tok/s
Model	Loculus	llama.cpp	Loculus	llama.cpp	Loculus	llama.cpp	Loculus	llama.cpp	Loculus	llama.cpp	Loculus	llama.cpp
Qwen3.8 27BUD-Q4_K_M	3.74	3.87	12.7	15.0	45.2	56.1	20.3	24.4	2.14	2.50	9.3one at a time	15.8
Gemma 4 12BQ4_K_M	1.65	2.11	5.92	7.82	25.7	30.6	9.56	12.8	1.26	1.54	35.2	30.5
Gemma 4 26B-A4BUD-Q4_K_M	1.05	1.26	3.65	4.45	16.0	17.5	5.92	7.52	0.77	0.92	57.5	48.3
gpt-oss 20BMXFP4	0.57	0.93	2.25	3.33	8.53	13.4	3.74	7.00	0.38	0.75	72.3	65.1
gpt-oss 120BMXFP4	1.64	2.34	6.74	8.92	23.4	33.5	10.2	14.6	0.98	1.63	31.7one at a time	29.8
Nemotron 3.5 Lightning 30B-A3BUD-Q4_K_M	0.76	2.08	1.82	5.01	6.89	15.6	3.51	8.06	0.66	1.38	45.3one at a time	39.7
Qwen3.8 Flash-Next 177B-A3BUD-Q2_K_XL	3.23	5.80	9.74	17.7	28.6	63.7	12.4	27.9	1.58	4.05	19.7one at a time	19.4
Qwen3 30B-A3BUD-IQ1_S	0.72	0.89	2.84	3.98	12.3	19.9	5.03	7.48	0.59	0.84	73.5	71.1
Llama 3.2 3BQ4_K_M	0.37	0.47	1.39	1.78	6.26	10.1	2.40	4.61	0.33	0.40	131	132
Mistral 7B v0.3Q4_K_M	1.06	1.25	4.15	5.40	17.6	29.9	–	–	–	–	56.0	58.6
Phi-4 miniQ4_K_M	0.37	0.51	1.47	1.98	7.06	11.6	–	–	–	–	115	113
GLM-4 9BQ4_K_M	0.95	1.15	3.73	4.65	14.8	20.5	6.22	8.03	0.70	0.88	52.0	54.3

Beyond the head-to-head

Loculus Core on its own

Development measurements of what the standard tests leave out, Loculus Core alone on the same machine.

748B

parameter model on a 93 GB machine

DeepSeek V4.1 Flash (233 GB file) with experts streamed from the SSD: 10.5–14.6 tok/s decode, 6.3–6.8 GB of process memory.

2.3×

Faster code edits

Lossless speculative decoding, best case

0.67 s

Resume a 7.9k-token prompt

After a restart; 71 s to recompute

Token context

Memory grows only as it fills

Model architectures

Dense, MoE, hybrid SSM, latent attention

Lossless speculative decoding

Decode tokens per second on a code-edit workload; outputs identical

Qwen3 4B 106 2.3×

Qwen3-30B-A3B 133 1.5×

Nemotron 3.5 Lightning 90 1.4×

Qwen3.8 Flash Next 73 2.0×

DeepSeek V4.1 Flash 31.7 2.2×

View as table

Model	Standard decode	With speculative decoding	Gain
Qwen3 4B	47	106	2.3×
Qwen3-30B-A3B	91	133	1.5×
Nemotron 3.5 Lightning	66	90	1.4×
Qwen3.8 Flash Next	36	73	2.0×
DeepSeek V4.1 Flash	14.7	31.7	2.2×

All results from one machine so far; more hardware follows as benchmarking continues. Results set 2026-10-25b, published 2026-10-25.