24 GiB GPU
24 GiB is the vendor-published framebuffer on RTX 4090, RTX 3090, RTX A5000, RX 7900 XTX. Same pool on every card in that list. Ranking is arithmetic on sourced parameter counts.
24 GiB framebuffer, taken from the sourced cards listed below. Not a free-after-driver figure.
7 of 25 profiled open-weight models fit 24 GiB (vram) at Q5_K_M, 8192 context. Best rated that fits: Gemma 4 31B.
VRAM as entered. Driver reservation is inside the 1 GiB overhead, not extra.
Among models that fit
Highest LMArena Elo among models that fit: Gemma 4 31B 1443.4. Elo is a human-preference scale, not the intelligence index.
The arena boards are separate scales and are never blended. A missing cell is unmeasured, not zero. A model’s highest axis is relative to its own scores, not a claim it leads the catalogue.
Rated models that fit
| Model | Human preference | Quant that fits | Estimated GiB | Headroom |
|---|---|---|---|---|
Gemma 4 31B Fit assumptions
| 1443.4 | Q5_K_M | 22.106 | 1.894 |
Qwen3.8 27B Fit assumptions
| 1440.7 | Q5_K_M | 20.063 | 3.938 |
Phi 4 Fit assumptions
| 1216.7 | Q8_0 | 17.438 | 6.563 |
Fits, unrated
- Muse Glimmer 30B — Q5_K_M, 21.756 GiB unrated
- Ministral 3 14B 2512 — Q8_0, 17.019 GiB unrated
- Ministral 3 8B 2512 — F16, 19.663 GiB unrated
- Ministral 3 3B 2512 — F16, 9.413 GiB unrated
Does not fit, or not profiled
- cohere/command-a-plus — even Q4_K_M wants 132.618 GiB; pool is 24 GiB
- deepseek/deepseek-v4-flash-vision-exp — even Q4_K_M wants 184.902 GiB; pool is 24 GiB
- z-ai/glm-5.3 — even Q4_K_M wants 455.624 GiB; pool is 24 GiB
- qwen/qwen3.8-2.4t-a95b — even Q4_K_M wants 1450.719 GiB; pool is 24 GiB
- moonshotai/kimi-k3 — even Q4_K_M wants 1691.5 GiB; pool is 24 GiB
- z-ai/glm-5.2 — even Q4_K_M wants 455.624 GiB; pool is 24 GiB
- moonshotai/kimi-k2.7-code — even Q4_K_M wants 604.75 GiB; pool is 24 GiB
- nvidia/nemotron-3-ultra-550b-a55b — even Q4_K_M wants 333.063 GiB; pool is 24 GiB
- minimax/minimax-m3 — even Q4_K_M wants 259.405 GiB; pool is 24 GiB
- deepseek/deepseek-v4-pro — even Q4_K_M wants 967 GiB; pool is 24 GiB
- deepseek/deepseek-v4-flash — even Q4_K_M wants 172.465 GiB; pool is 24 GiB
- mistralai/mistral-small-2603 — even Q4_K_M wants 73.2 GiB; pool is 24 GiB
- mistralai/mistral-large-2512 — even Q4_K_M wants 408.531 GiB; pool is 24 GiB
- deepseek/deepseek-v3.2 — even Q4_K_M wants 406.116 GiB; pool is 24 GiB
- openai/gpt-oss-120b — even Q4_K_M wants 72.201 GiB; pool is 24 GiB
- meta-llama/llama-4-maverick — even Q4_K_M wants 242.5 GiB; pool is 24 GiB
- meta-llama/llama-4-scout — even Q4_K_M wants 66.809 GiB; pool is 24 GiB
- cohere/command-a — even Q4_K_M wants 68.016 GiB; pool is 24 GiB
GPU
How the number is made
Weights: a measured GGUF size when we have one, otherwise total parameters × bits per weight ÷ 8. Q4_K_M is treated as 4.83 bits/param. MoE memory uses total parameters, not active parameters.
KV cache, when layer and head geometry is sourced: 2 × kvLayers × kvHeads × headDim × context × 2 bytes (FP16, batch 1). Hybrid models use attention-layer count, not every layer. If geometry is missing, cache is omitted and that is stated.
1 GiB is added for CUDA/runtime. Apple unified memory uses RAM as the pool. CPU-only is a memory fit, not a speed claim. DDR generation is ignored for fit — it changes bandwidth, not whether the weights sit in memory.
RAM type (DDR4/DDR5/LPDDR) is not a field. It does not change whether a model fits.
Questions this page answers
What LLM can I run on a 24GB card?
7 of 25 profiled open-weight models fit 24 GiB (vram) at Q5_K_M, 8192 context. Best rated that fits: Gemma 4 31B.
Which sourced cards publish 24 GiB?
RTX 4090 (24 GiB), RTX 3090 (24 GiB), RTX A5000 (24 GiB), RX 7900 XTX (24 GiB).
Does NVIDIA pick the winner?
No. Ranking is arithmetic on sourced parameter counts. Quality leads. Unrated models that fit are listed separately, never scored zero.