Colibrì: Running a 744B Open Model on 25GB of RAM by Streaming Experts From Disk

Model size used to mean RAM size. colibri breaks that link: it runs GLM-5.2, a 744B model, on a 25GB machine with no GPU. Only a small slice is needed per token, so it keeps that in RAM and streams the rest off an SSD. It's slow, a token every few seconds, more a proof the memory wall isn't fixed than a finished tool. 2.1k stars, Apache-2.0. #LocalLLM #LocalInference #MoE #OpenSourceAI

I checked this against Colibrì's GitHub repo and several independent write-ups.

Confirmed accurate:

• Colibrì is a pure-C, zero-dependency inference engine that runs GLM-5.2, a 744B Mixture-of-Experts model, on a consumer machine with roughly 25GB of RAM, no GPU required, by streaming experts from disk — matches the post's core claim precisely. GitHub
• A 744B MoE model like GLM-5.2 activates only about 40B parameters per token, and only about 11GB of those routed experts change from token to token — this is exactly the mechanism the post describes: "only a small slice is needed per token, so it keeps that in RAM and streams the rest off an SSD." GitHub
• The license is confirmed: Colibrì was released by JustVugg under the Apache 2.0 license. noze
• The "slow" framing is accurate and, if anything, slightly generous: Colibri runs at 0.05–0.1 tokens per second cold on the author's baseline setup, with warm speculative decoding lifting that to 2.2–2.8 tokens per forward pass, and speed is about 0.05–1 tokens per second overall, so you cannot expect the snappy responses of a chat app. "A token every few seconds" undersells just how slow the cold case can be (0.05 tok/s is roughly 20 seconds per token), but it's in the right ballpark for the better cases and correctly conveys "impractically slow for real use." AI/TLDRsakutto
• The framing as "more a proof... than a finished tool" matches the project's own stated positioning: Colibrì is deliberately a research platform to test aggressive systems ideas, with no SLA on speed. GitHub

One figure worth flagging — likely stale:

• The star count ("2.1k") is significantly lower than what independent trackers show for Colibrì's actual trajectory: it had 241 stars shortly after launch, then roughly 2.8k a week later, and by mid-July 2026 it had climbed to roughly 11,500 stars and 900 forks. "2.1k" sits between the early and later snapshots, so this looks like a genuinely outdated figure from partway through the project's rapid growth curve, rather than a fabricated number — but it likely understates the repo's current popularity by a wide margin.