Frontier models should not be sealed inside datacenters.
Anyone curious enough should be able to open one up —
run it, watch every expert fire, and make it better.
Run it
A 744B frontier-class MoE answering on a 25 GB machine — the proven floor — up to 9+ tok/s on a workstation. No cluster, no cloud bill, no API key.
Study it
The engine is a microscope: live routing telemetry, per-expert heat, and the first published expert atlas of a 700B-class model — measured on machines like yours.
Improve it
One C file, readable in an afternoon. Every optimisation this project found came from someone measuring something — the next one can come from you.
A JIT, but for weights
A compiler JIT never compiles the whole program — it watches what actually runs and compiles the hot paths, just in time. colibrì applies the same bet to a 744B parameter space: a token activates only ~40B parameters, and just ~11 GB of those change from token to token — the routed experts.
The engine keeps the dense weights resident and treats the 19,456 experts as a heterogeneous storage hierarchy: measured routing heat decides which experts earn VRAM, which earn pinned RAM, and which stream from NVMe. The router runs a layer ahead, so prefetch hides the staging latency — and because routing has measurable structure (the atlas below), the hierarchy keeps getting the answer right. Like a JIT, it learns your workload: the more you run, the hotter the right experts get.
Everything is validated token-exact against the reference transformers implementation — speed never buys drift.
Watch it think
A replay of the engine at work, paced to measured decode speeds from real community hardware. The grid is all 19,456 experts (colour = storage tier, brightness = routing heat, routed = white flash); the galaxy below it is the measured atlas — watch different questions light different regions of the mind.
Simulation replays a fixed transcript at each profile's measured decode rate (#82, #389, #387, #273); “—” means that machine's report did not publish the number — nothing here is invented. Atlas flashes sample real measured affinity (atlas v1) for the active topic.
Measured, not assumed
Every point below is a real measured expert from the published canonical atlas v1: 721 canonical specialists confirmed across three independent kernel families (AVX-512 VNNI · NEON · AVX2, engine-pinned, replication-gated) plus 637 gate-sensitive ones. Position is the measured 10-topic affinity vector — a specialist sits at its topic's anchor, a generalist drifts to the centre. Colour = top topic, large points = canonical. Drag to spin.
Same engine, any hardware
Measured decode on real community machines — the hardware only changes where the experts live. Every row links to its public report.
| hardware | decode | where the experts live | |
|---|---|---|---|
| 6× RTX 5090 · full residency + selective NUMA (#82) | 9.0–9.2 tok/s | all in VRAM+RAM · disk 0 | |
| 2× Xeon Gold 6430 · 1 TB DDR5 · CPU-only (#389) | 5.42 tok/s | every expert pinned in RAM | |
| DGX Spark GB10 · 121 GB unified (#161) | 3.33 tok/s | unified memory + CACHE_ROUTE | |
| MacBook Pro M5 Max · 128 GB · Metal (#387) | 2.0 tok/s | Metal tier + warm pin | |
| Ryzen AI Max+ 395 · 128 GB (#200) | 1.83 tok/s | hot set pinned in RAM | |
| Ryzen 9 9950X3D2 · RTX 5090 · Gen5 NVMe (#120) | 1.23 tok/s | 28 GB VRAM tier + 11.5 GB/s disk | |
| single RTX 5070 Ti laptop-class (#273) | 1.07 tok/s | GPU-resident pipeline | |
| 25 GB dev box · cold — the proven floor | 0.05–0.1 tok/s | streamed from NVMe |
…and 15+ more community machines, from an M4 Pro Mac Mini to a 430 GB EPYC server. Full tables, methodology and quality ablations: docs/benchmarks.md
One engine, more minds coming
The tiering algorithm is model-agnostic: any Mixture-of-Experts with routed experts can be staged the same way. GLM-5.2 is live today; support for more open-weight families is on the roadmap.
GLM-5.2
The flagship target: int4 container, token-exact vs reference, full atlas published.
OLMoE
The small research workhorse — quantization A/Bs and quality ablations run here first.
Kimi K2
The next scale step: a trillion-parameter mind on the same tiered engine.
Qwen3 MoE
The most widely-deployed open family — broad hardware coverage meets broad adoption.
MiniMax
Long-context specialists — a different routing profile for the atlas to map.
Thank you to the teams whose open weights make this project possible — Z.ai (GLM), Moonshot AI (Kimi), Alibaba Qwen, MiniMax, Allen AI (OLMoE), and everyone releasing frontier-class models in the open. An engine is nothing without a mind to run.
Come build the microscope
Every optimisation in this project started with someone measuring something on their own machine — a disk swap, a NUMA experiment, an atlas replication on a Mac Mini. You don't need a datacenter to move frontier-model research. That is the whole point.