Colibri surfaced on Hacker News with 318 points and 84 comments as a minimal toolkit for running GLM 5.2 on machines with 8–16 GB RAM and no dedicated GPU.
Project: Colibri | Target model: GLM 5.2 | Focus: CPU-only inference | Source: github.com/JustVugg/colibri
What It Is
Colibri applies aggressive layer-wise quantization and memory-mapping to GLM 5.2 weights. It loads only active layers into RAM while keeping the rest on disk, cutting peak memory use by roughly half compared with standard GGUF loads.
The script uses existing llama.cpp kernels with custom mmap flags and a simple Python wrapper. No new model training or fine-tuning is required.
How It Works
Users clone the repo and point it at a GLM 5.2 GGUF file. Colibri then applies 4-bit or 3-bit quantization on the fly for selected layers and streams weights from disk during generation.
The approach avoids full model decompression in memory. Token generation stays sequential, trading speed for the ability to run on hardware that would otherwise OOM.
Benchmarks and Numbers
Early HN reports list the following on an Intel i5-8250U with 16 GB RAM:
- 7–9 tokens per second at 4-bit
- Peak RAM usage of 9.2 GB for the 32B variant
- Cold start time of 45 seconds from SSD
No GPU is used. Numbers come from user-submitted logs in the thread; official benchmarks are not yet published.
How to Try It
Clone the repository and install the listed Python dependencies. Place a GLM 5.2 GGUF file in the models folder, then run:
python colibri.py --model glm-5.2-32b-q4.gguf --max-ram 12
Generation commands follow the standard llama.cpp pattern. The repo README lists exact flags for 8 GB and 12 GB systems.
Pros and Cons
- Works on CPUs with no GPU required
- Reduces RAM footprint enough for 32B-class models on 16 GB machines
Simple one-file script with few external dependencies
Speed drops to single-digit tokens per second
Disk I/O becomes the bottleneck on HDDs
No support for batching or speculative decoding yet
Alternatives and Comparisons
| Tool | Min RAM (32B) | Tokens/s (CPU) | Quant support | License |
|---|---|---|---|---|
| Colibri | 9 GB | 7–9 | 3/4-bit layers | MIT |
| llama.cpp | 18 GB | 12–15 | Full file | MIT |
| Ollama | 20 GB | 10–14 | Full file | Apache 2.0 |
Colibri trades speed for lower memory. llama.cpp remains faster when RAM is available; Ollama adds a higher-level interface but no extra memory savings.
Who Should Use This
Developers testing GLM 5.2 on laptops or older desktops without upgrading hardware will find it useful. Teams needing production throughput should skip it and allocate proper GPU resources instead.
Researchers comparing quantization strategies on consumer hardware can use the layer-mapping approach as a baseline.
Bottom line: Colibri lowers the hardware bar for GLM 5.2 experimentation without new model releases or paid APIs.
The project shows continued demand for practical quantization scripts that fit large models into everyday machines rather than waiting for smaller distilled versions.
Top comments (0)