Benchmarks: LexiEngine against llama.cpp, and how to reproduce it

LexiEngine is measured against the newest llama.cpp release on the same card, with the same model file and the same settings, every time the engine changes. This page has the current results, the cases where llama.cpp is still ahead, the older results, and the method.

Current results (November 13, 2026)

LexiEngine version 5db82d6 against llama.cpp b12318 (Vulkan, flash attention on the RX 7900 XTX; CUDA on the RTX 2060). One Radeon RX 7900 XTX (24 GB) capped at 291 W and one RTX 2060 (6 GB) in an Intel i7-9700 desktop on Linux 7.3, nothing else running beside any run, one slot, greedy. Generation is tokens per second after the first token.

Qwen3.8 27B Q4_K_M, 262,144-token context, 4-bit K/V cache, draft-head speculation on both engines

This is one fixed configuration, the one the model is served with every day (batch 2048/512, at most 2 drafted tokens). Every engine change is measured in this table before and after.

One request per row. llama.cpp b12318 lost its Vulkan device (vk::DeviceLostError) while reading the 133,561-token prompt, at 88,000 tokens in the first attempt and 113,000 in the second. LexiEngine read both deep prompts on every attempt. The version before this one (3db31f8) generated at the same speeds and read these prompts at 660, 581, 404 and 293 tok/s.

Default settings: three models, both cards

4,096-token context, each engine's defaults, no speculation; prompts of about 50, 580 and 3,600 tokens.

Where llama.cpp is ahead: Gemma 4 on both cards, longer DeepSeek generation, and prompt processing on the RTX 2060, where LexiEngine is at about a quarter of llama.cpp's speed on a 9B model after tripling in this version. After a long prompt LexiEngine also holds more video memory on the default path (Gemma 4 on the RX 7900 XTX: 11.4 GB against 3.2 GB). These are the current engine work.

With the 27B's draft head on and the same settings on both engines at a 4,096-token context, LexiEngine generates at 82.6 / 74.4 / 59.1 tok/s against llama.cpp's 68.2 / 66.0 / 61.0, and its text with the draft head on is identical to its text with it off.

Results of November 10, 2026

LexiEngine as of November 10, 2026 against llama.cpp b12230 (Vulkan, flash attention). One Radeon RX 7900 XTX (24 GB) capped at 291 W in an Intel i7-9700 desktop on Linux 7.3, the host's own install, nothing else running beside any run, one slot. Both engines are started with the same context length, cache type, sampling and speculation settings. Generation is tokens per second after the first token; prompt processing is prompt tokens divided by the time to the first token.

Qwen3.8 27B Q4_K_M, 240,128-token context, draft-head speculation on both engines

Short and 3,600-token rows are the median of three requests; the deeper rows are one request each. llama.cpp b12230 rows were measured on 11/09/2026 with the same harness and settings.

llama.cpp b12230 lost its Vulkan device (vk::DeviceLostError) on the 138,000-token prompt in three of three attempts, so it has no result at that depth or beyond. LexiEngine completed every case, including a 231,000-token prompt.

A 9B Qwen3.5-family model (Qwythos 9B Q4_K_M), 524,288-token context, no speculation

Short and 3,600-token rows are the median of three requests; the deeper rows are one request each. llama.cpp b12230 rows were measured earlier on 11/10/2026 with the same harness and settings.

LexiEngine is ahead at every depth, and the lead grows with depth.

DeepSeek-V2-Lite-Chat Q4_K_M, 16,384-token context, greedy

One run per prompt on each engine, in the order shown; each prompt extends the one before it and both engines keep the shared prefix, so first-token times cover only the new part of each prompt.

On November 9 this model processed prompts at 134 tok/s on LexiEngine. It now runs batched prompt passes on the GPU's matrix cores, keeps the previous request's shared prefix, and from 4,096 tokens of context reads a compact 16-bit latent cache instead of the expanded keys and values. llama.cpp is still about 14% ahead in generation on short prompts.

Where llama.cpp is ahead

• Short prompts on small models: DeepSeek-V2-Lite (about 14%) and Qwen3 1.7B (25–30% in the November 4 run below, an older engine build).
• The cause is a fixed per-token cost: the number of GPU dispatches that wait for one another. It is the current engine work.

Earlier results

These used older engine builds and older llama.cpp releases. They are kept as history and are not the current state.

Qwen3.8 27B at a 131,072-token context (November 6–7, 2026)

The supplied benchmark record uses an earlier LexiEngine build, llama.cpp b12180 (Vulkan, flash attention), Qwen3.8 27B, and a Radeon RX 7900 XTX capped at 291 W. Host: Intel i7-9700, Linux 7.3, running on the host's own install — not the USB image. Both engines use Q4_1 KV cache, 131,072-token context, one slot, idle machine, no cache reuse, and a fresh random prefix per prompt. Values are medians of three runs except the long 68k row, which had one request and should not be treated as a three-run median.

Generation, tokens per second after the first token

Numbers per cell are in short / 3.6k / 21k / 68k context-length order.

MTP is draft-head speculative decoding. Do not compare an MTP-on row with MTP-off, even on the same model. In the later source README, an alternative Q4_K_M MTP short-prompt table reflects a different engine build. The table here follows the supplied current benchmark record (context/BENCHMARKS.md) rather than merging runs from different builds.

Prompt processing, tokens per second

Each cell lists 3.6k / 21k / 68k prompt-length results.

These prompt-throughput values are not generation rates and should not be presented as such. Performance under a different driver, accelerator, processor, thermal state or model quantization is unmeasured by this table.

Smaller model on the USB appliance (November 4, 2026)

A second RX 7900 XTX with Ryzen 9 5900X, LexiOS on Linux 7.3 booted from USB, ran Qwen3 1.7B Q4_K_M on both engines: 8,192-token context, f16 KV cache, one slot, 256 generated tokens, three repetitions, llama.cpp b12090. The range below is the observed variation, not an uncertainty interval.

On this small-model generation workload, llama.cpp was 25%–30% faster. A different first-token result does not erase that generation gap. Results cannot be extrapolated from 1.7B to 27B.

Method to reproduce a fair measurement

1. Record provenance first. Save model identity and content hash, GGUF quantization, engine build, backend and driver, kernel, CPU/GPU, power cap, free memory, KV-cache type, context, batch/slot count, sampling, MTP state and installed settings. If any differ, treat it as a different experiment.
2. Keep the machine idle and let thermal conditions stabilize. Close other GPU or CPU loads; note power and thermal policy. Do not mix benchmarks during maintenance, updates or background model downloads.
3. Use identical requests and generated-token limits. Test short, approximately 3.6k, 21k and — if memory supports it — 68k-context inputs. Use a new random prompt prefix for each request to prevent cache reuse; label approximate lengths with their actual token counts. Align draft-head speculation on both sides before comparison.
4. Repeat at least three times per routine test and record each run. Use the median, not the best result. For a one-run long-context measurement, label the result single-run instead of implying statistical confidence.
5. Separate metrics: (a) time to first token in seconds, (b) prompt processing in tok/s, (c) generation in tok/s excluding the first-token delay. Never compare incompatible denominators.
6. Assess noise and stability. Track per-run values, warm/cold behavior, variance, memory/power state and failure count. Calculate noise bands/confidence intervals from replicates where possible; do not claim statistical significance with three or fewer observations by default. Reproduce any changed result before accepting an optimization.
7. Check output quality with matched prompts, held-out evaluations, perplexity/KLD where supported, and deterministic sampling where possible. Speed gained by changing output or truncating work is not a like-for-like win.
8. Export a complete evidence record and compare through Bench and Evaluate, including device/context depth curves and quality gates; never substitute self-reported peaks for representative median trials.

Bench/Panel workflow

• Open Panel Benchmarks and Evals to inspect run results, context-depth curves, comparisons, histograms and evaluation history.
• Review Optimizer for noise-aware search, Fit Lab for quantization and KLD checks, and Power and Hardware for power cap/thermal conditions.
• The engine exposes lexi-engine bench, topology, lexify-plan, quantize and capture-activations operations; check the installed build's help before selecting flags. A command existing in the engine does not imply a corresponding Panel form is available on every kernel.
• TokenGate's reported CPU-only 1.7B/Q8_0 590-token-state observation was 14.0 s ungated TTFT versus 35.4 s with a quarter keep budget; matching-task accuracy was not measured. Do not advertise this as a speed win.

Controls and interpretation

See Hardware, Security and Release Notes.