Is On-Prem Trillion-Parameter Inference Worth It? A TCO Look at HP's GB300-Powered ZGX Fury

For only ~$120K you can now get a SOTA model inference server and never send a sensitive prompt to a cloud API again. I had to pinch myself when I first heard the news. It is true. NVIDIA and OEMs like HP are shipping desktop servers with GB300 super-chips and 748GB of coherent memory to an investment firm near you. With a GB300 server like HP's ZGX Fury it is now possible to run 1 trillion parameter open models like GLM 5.2 that are competitive with Claude Sonnet and Opus class models from Anthropic on a variety of benchmarks. So should you buy it? -- I ran the math along with data from our own Elendil Labs' use of Claude and OpenAI tokens. The numbers are eye opening: Over a 4-year TCO factoring in the cost of power and AI ops support ($60K per server) an investment research or quant team stands to save well over $1.5 million dollars vs. paying for the latest Anthropic and OpenAI models through API use. Between client deliverables and our own development, we are consuming tokens at a clip of about 4-5B per month (Opus, Fable, GPT 5.5 and Sol). This is well over $500K / year of token use if paid at the API list prices for Anthropic and OpenAI. I think the math is pretty convincing. DM me if you want the full details and assumptions.

I checked the hardware, model, and pricing claims in this post.

Confirmed accurate:

• HP ZGX Fury is powered by the NVIDIA GB300 Grace Blackwell Ultra Desktop Superchip with 748GB of coherent memory, delivering trillion-parameter inference at the deskside — the 748GB figure matches the post exactly. HP
• The memory breaks down as up to 496GB LPDDR5X for the CPU and up to 252GB HBM3e for the GPU, confirming this is a real, currently-marketed HP product, not a rumor. HP
• HP positions ZGX Fury specifically for organizations that must run AI on-premises due to data sensitivity, compliance, or cost concerns — supports the "never send a sensitive prompt to a cloud API" framing. HP

Partially accurate / needs a caveat:

• GLM 5.2 is real and open-weight, but it's not actually a trillion-parameter model — it's a 753-billion-parameter open-weight model released by Zhipu AI (Z.ai) under an MIT license. The post's phrasing ("run 1 trillion parameter open models like GLM 5.2") blurs the hardware's trillion-parameter capacity with GLM 5.2's actual size, which is about three-quarters of that. MindStudio
• "Competitive with Claude Sonnet and Opus on a variety of benchmarks" is fair but slightly generous depending on which benchmark: GLM-5.2 scores 62.1% on SWE-bench Pro and comes within about four points of Claude Opus 4.8 on Terminal-Bench 2.1 (81.0% vs 85.0%), and on the Artificial Analysis Intelligence Index, Claude Sonnet 5 scores 53 versus GLM-5.2's 51 — genuinely close, but Claude models still edge it out on most head-to-head intelligence measures, even as GLM wins clearly on price and speed. Build Fast with AIArtificial Analysis

Unverifiable / plausible but unconfirmed:

• The ~$120K price tag: HP hasn't published official ZGX Fury pricing. Analysts have inferred pricing will likely track Nvidia's comparable DGX Station, which resellers have offered from around $94,000 up to sub-$200,000 for higher-end configurations — so $120K sits squarely in a plausible range, though it's an estimate, not a confirmed HP price. TechRadar
• The token-volume figures (4-5B tokens/month), the $60K/year AI-ops support cost, and the "$1.5M savings over 4 years" TCO conclusion are internal company figures from "Elendil Labs" that I can't independently verify. The arithmetic is internally consistent — if you assume roughly $500K/year in current API spend against a ~$360K all-in 4-year hardware+ops cost, the stated savings roughly check out — but the inputs themselves rest entirely on the poster's own numbers.