2.8 Trillion Parameters on 4GB VRAM? Here’s How AirLLM Does It

Grounded / Real Inflated / Uruttu

80% real

20% uruttu

article Original Content

You can now run a 2.8 TRILLION parameter model on a 4GB GPU.
It’s called AirLLM, an open-source tool that uses "Layer-wise Inference." It only loads one layer onto your GPU at a time. so the VRAM you need depends on the layer size, not the model size.
No quantization. No distillation. No pruning.
→ DeepSeek-V3 (671B) on 12GB
→ Llama 3.1 405B on 8GB
→ Kimi K3 (2.8 TRILLION params) on under 4GB
→ works with almost every open model
The biggest model on it needs the LEAST VRAM. K3 is sparse MoE, so it streams only the experts a token actually routes to instead of a whole dense layer.
2.8 trillion parameters running in less VRAM than a 70B.

verified Validated Content

What’s real ✅

AirLLM is real and open-source. Its current repository advertises layer-wise inference that reduces GPU memory requirements by loading model components incrementally.
Kimi K3 is a 2.8T-parameter model. Kimi's own announcement confirms the 2.8T parameter count.
The Kimi K3 + AirLLM 3.72 GB claim is actually documented. AirLLM's v3.1 release reports 3.72 GB peak VRAM, measured end-to-end on an RTX 6000 Ada, while processing the full 1.56 TB checkpoint.
The reason isn't simply "one layer at a time" for K3. K3 is a sparse MoE model, and AirLLM's newer implementation streams individual experts. The release says K3 has 896 experts per layer but routes each token to 16 experts.
AirLLM's repository currently advertises DeepSeek-V3 at ~12 GB and Llama 3.1 405B at 8 GB.

What needs correction ⚠️

“No quantization.”
This is the biggest problem.

The Kimi K3 implementation specifically requires MXFP4 weights / compressed-tensors, according to AirLLM's own current documentation.

So saying:

“No quantization.”

as a blanket statement alongside the K3 3.72 GB claim is misleading.

“The biggest model needs the least VRAM.”
Technically possible in this particular comparison, but the wording is clickbait. VRAM depends heavily on architecture, layer/expert size, weight format, routing, and implementation, not simply total parameter count.

“works with almost every open model.”
AirLLM supports many model families, but “almost every” is a broad marketing claim rather than something that can be independently established. The project itself advertises broad support.

Important caveat: an open GitHub issue specifically questions the reproducibility of AirLLM's headline VRAM claims and notes that some advertised examples lack reproducible benchmark evidence