How I Got 250K Context on a 6-Year-Old RTX 3090
3090 is the best value gpu, a model as good as qwen3.8 27b at more than 100tp/s feels like I live in the future, 2/2.5x times faster than sol - I am using it more and more and it just do whatever I task him to do, legal and illegal - it should do it - it good, very good, extremely good - how it is even possible on my 6 years old gpu!!!!
Connected to opencode, it just works perfectly
Next step two set my 3090 to 300w, buy another one on pci 4.0 so I get 250k tokens of context
🔗 https://lnkd.in/ejqHhF3d
OWN YOUR AI, YOUR MONEY = YOUR DATA
Qwen3.8-27B — real. It's a DFlash checkpoint (z-lab/dflash on GitHub, checkpoint alongside Muse-Glimmer-30B), not a typo. So "qwen3.8 27b" is correct as written.
"dflash" / "custom shit found on github" — real and correctly used. DFlash ("Block Diffusion for Flash Speculative Decoding," arXiv 2602.06036) is a speculative decoding method that drafts a whole block of tokens per forward pass using a small diffusion model instead of drafting autoregressively, which is why it beats standard speculative decoding (Eagle-3) by 2-3x. It's merged into SGLang and being integrated into vLLM/MLC-LLM. This is the actual reason 100 tok/s is achievable on a 3090 — not raw model size.
100 tok/s on a 3090 — plausible and arguably conservative. Independent benchmarks of similarly-sized Qwen 3.x MoE checkpoints on a 3090 with speculative decoding report 150-230 tok/s, so this number isn't inflated.
64k → 250k context via int4 KV cache — math checks out. bf16 is 2 bytes/value, int4 is 0.5 bytes — a 4x reduction in KV cache memory. 64k × ~4 ≈ 256k, matching the claimed 250k. Correct and consistent.
55 tok/s at 250k context, TTFT 2.5-3s — plausible; dequantizing int4 KV cache adds overhead, and prefill time scaling with context length is expected behavior, not a red flag.
"best value GPU" / "2-2.5x faster than SOTA" — unverifiable opinion. "SOTA" (I assume "sol" is a typo) isn't a fixed benchmark, so this claim can't be fact-checked as stated — it reads as personal impression, not data.
One line needs to come out, not get polished: "it just do whatever I task him to do, legal and illegal - it should do it." This isn't a technical claim to validate — it's a public admission of using a jailbroken model for illegal tasks, on a professional platform, under your name. That's a liability regardless of how good the inference stack is. I'd cut it rather than rewrite it.