The GPU Math Behind Deploying Large MoE Models in 2026

I deleted 62.5% of a 120B-parameter model. Production couldn't tell the difference. At reach.jobs we score resumes against job postings with OpenAI's gpt-oss-120b. The full model is 61 GiB of weights: that's RTX 6000 Pro territory, a $10,000 card, before you even talk about context. So I built the world's first REAP-pruned gpt-oss-120b: 80 of 128 experts deleted per layer, calibrated on our real traffic. It now runs on a single $1,000 32 GB GPU with the full 128k context, in stock vLLM, on NVIDIA and Intel alike. On a frozen replay of our production workload it scores inside the full model's own run-to-run consistency range, at 100% valid JSON. 10x cheaper hardware, same answers. And it beats gpt-oss-20b (the model you'd otherwise run at this size) on every metric we measured. One lesson worth stealing: calibrate on prompts PLUS the model's own generations. Prompt-only calibration silently deletes the experts that write your output format, and the model fails by format collapse, not gradual decay. Weights, code (Apache 2.0), and the full write-up: https://lnkd.in/gfYqYcWx https://lnkd.in/gzK8yHkm https://lnkd.in/gWXyCWqF If your workload is narrow, your MoE is mostly dead weight. Measure it. #LLM #OpenSource #GPU #MachineLearning #Inference

Confirmed Accurate

• gpt-oss-120b specifications. OpenAI's official model card confirms gpt-oss-120b has 36 layers, 116.8B total parameters and 5.1B active parameters per token, with a checkpoint size of 60.8 GiB — matching the post's "61 GiB of weights" almost exactly, and the "120B" figure matches OpenAI's own public naming convention (technically 116.8B). arxivarxiv
• 128 experts per layer. Confirmed. gpt-oss-120b uses 36 layers, each containing 128 experts, with 4 active for any token. OpenAI
• "80 of 128 experts deleted per layer" = 62.5%. This math is internally consistent: 80/128 = 62.5%, matching the headline claim exactly.
• REAP as a real, published pruning method. Confirmed. REAP (Router-weighted Expert Activation Pruning) is a real technique from a peer-reviewed paper that considers both router gate-values and expert activation norms to minimize the reconstruction error bound, and was shown to be consistently outperforming merging and other pruning methods on generative benchmarks, especially at 50% compression, validated on models ranging from 20B to 1T parameters. arXiv + 2
• "No retraining required" pruning approach. Consistent with REAP's design — no retraining is required, though optional post-pruning distillation can be used to further recover performance. Emergent Mind
• gpt-oss-120b runs in vLLM, supports 128k context. Both are accurate, documented capabilities of the base model.
• Apache 2.0 licensing of gpt-oss and REAP code. Both gpt-oss and the original REAP research code are released under Apache 2.0, consistent with the post's licensing claim for its own release.

Mostly Accurate

• "RTX 6000 Pro territory, a $10,000 card" for the full model. This was a reasonably accurate price for the RTX PRO 6000 Blackwell (96GB) earlier in its lifecycle, but as of August 2026 — the timeframe of this post — that card's official price has risen to $16,000, an 87% increase over the launch MSRP, with street prices in a similar range. The $10,000 figure is stale and understates current cost by 40-60%, likely reflecting a slightly older reference point or informal rounding. Gagadget
• "Runs on a single $1,000 32GB GPU." The obvious candidate 32GB consumer card is the RTX 5090, but its street pricing is running $4,300 to $5,000 against a $1,999 MSRP as of mid-2026 — even the original MSRP is double the figure cited, and current street price is 4-5x higher. Unless the author means a specific discounted/used/non-consumer 32GB card not identified in the post, this figure appears substantially understated relative to the current market (August 2026). ShopBack

Partially Accurate

• "It scores inside the full model's own run-to-run consistency range, at 100% valid JSON." This is a specific, checkable-sounding claim, but it's self-reported and evaluated only on the author's own "frozen replay of production workload" — a narrow, task-specific, non-public benchmark. This is exactly the kind of author's-own-benchmark claim that warrants the standard caveat: strong performance on a narrow, calibrated internal task doesn't necessarily generalize to broader use cases, and no external or third-party validation is presented.
• "It beats gpt-oss-20b... on every metric we measured." Same caveat applies — "every metric we measured" is self-selected and self-reported, with no disclosure of which metrics, sample size, or statistical significance.

Not Fully Verified

• "World's first REAP-pruned gpt-oss-120b." A "world's first" claim is inherently difficult to verify and is asserted without evidence of a comprehensive search for prior art; treat as an unverified superlative.
• reach.jobs as a company and its production resume-scoring use case. No independent confirmation available of the company's existence, scale, or actual production deployment of this pruned model; this is a first-party claim about the author's own work.
• The specific "calibrate on prompts PLUS the model's own generations" lesson and its claimed failure mode ("format collapse"). This is a plausible, technically coherent insight consistent with how REAP-style calibration works (the method scores experts based on activation during calibration data, so unrepresentative calibration data could indeed miss experts responsible for specific output patterns like structured JSON formatting), but the specific claim that prompt-only calibration "silently deletes the experts that write your output format" is the author's own empirical finding, not something independently documented in the public REAP literature reviewed here.

Opinion / Promotional Language

• "Production couldn't tell the difference" — punchy framing of the author's own internal evaluation.
• "If your workload is narrow, your MoE is mostly dead weight. Measure it." — general advocacy/takeaway framing, reasonable as a heuristic but stated as a broad rule rather than a qualified finding specific to narrow, well-calibrated tasks.
• Overall post structure (dramatic opening claim → technical detail → call to action with hashtags) is typical LinkedIn technical-flex format, though the technical content itself is unusually substantive and verifiable compared to typical promotional posts in this series.

Missing Context

• Domain narrowness caveat. The post's own REAP paper source notes pruning performance varies by task and compression ratio, with larger drops expected at higher pruning percentages on general-purpose tasks; 62.5% expert pruning is a very aggressive compression ratio (above the ~50% level most extensively validated in the original paper), and the post doesn't clarify whether "resume scoring against job postings" is an especially narrow, structured task that tolerates more aggressive pruning than general-purpose use would.
• No mention of quality trade-offs outside the tested workload. A model this aggressively pruned for one domain (resume/JD matching) would very likely perform worse on unrelated tasks — this is implied by the "narrow workload" framing but not explicitly stated as a limitation of the released weights for other users.
• Current GPU market conditions (2026 memory shortage) are not acknowledged, which materially affects the accuracy of both hardware price comparisons cited in the post.
• No sample size, confidence interval, or methodology detail given for the "run-to-run consistency range" comparison, which is central to the post's core claim.