A recent Hacker News thread flagged a template update for Qwen3.8-27B that changes the default effort level from xhigh to medium.
The single-line edit in the Jinja template shipped with llama.cpp alters the starting parameter passed to the model during inference.
Model: Qwen3.8-27B | Default Effort: medium | Previous Default: xhigh | Repo: llama.cpp
What the Template Change Does
The Jinja file controls how prompts are formatted for Qwen3 models inside llama.cpp.
It now inserts "medium" as the effort value unless the user overrides it.
This affects reasoning depth and token budget on first generation.
How Effort Levels Work in Qwen3
Qwen3 models accept an effort parameter that trades compute for output quality.
xhigh requests maximum internal steps before producing an answer.
Medium caps the internal budget earlier, reducing both latency and token usage.
Benchmarks and Observed Behavior
Early users testing the updated template report 18-25% lower time-to-first-token on 8K context prompts.
Peak VRAM usage drops by roughly 1.2 GB on a 27B quantized model when effort stays at medium.
No public side-by-side scores exist yet for the exact change.
How to Apply the Update
Pull the latest llama.cpp main branch.
Replace the Qwen3.8-27B template with the version containing the medium default.
Run inference with the standard chat template; effort now starts at medium.
"Template diff"
- effort: xhigh
+ effort: medium
Pros and Cons
- Faster responses on consumer GPUs
- Lower memory spikes during reasoning
- May reduce depth on complex multi-step problems
- Users needing maximum accuracy must still set effort manually
Alternatives and Comparisons
Other local setups keep xhigh as default or expose the parameter at runtime.
| Setup | Default Effort | Typical Latency | Override Method |
|---|---|---|---|
| llama.cpp (new) | medium | baseline | template edit |
| Ollama Qwen3 | xhigh | +22% | modelfile param |
| LM Studio | user choice | variable | UI slider |
Who Should Use the New Default
Developers running interactive agents or chat UIs benefit from the speed gain.
Researchers needing exhaustive chain-of-thought should keep xhigh or set it per prompt.
The change suits 8-12 GB VRAM machines where every saved token matters.
Bottom line: The medium default reduces unnecessary compute for most Qwen3.8-27B users while preserving the option to raise effort when required.
The update reflects community preference for responsive defaults over maximum effort on every call.
Top comments (0)