--- title: "16 GiB GPU: open-weight models that fit · Undominated.ai" canonical: https://undominated.ai/self-host/16gb/ description: "16 GiB is the vendor-published framebuffer on RTX 5080, RTX 5070 Ti, RTX 5060 Ti 16, RTX 4080, RTX 4060 Ti 16, RTX A4000, T4, RX 6900 XT, RX 6800 XT. Catalogue open-weight models that fit that pool, ranked by measured quality. Unrated is not zero. NVIDIA does not pick the winner." --- # 16 GiB GPU: open-weight models that fit · Undominated.ai > 16 GiB is the vendor-published framebuffer on RTX 5080, RTX 5070 Ti, RTX 5060 Ti 16, RTX 4080, RTX 4060 Ti 16, RTX A4000, T4, RX 6900 XT, RX 6800 XT. Catalogue open-weight models that fit that pool, ranked by measured quality. Unrated is not zero. NVIDIA does not pick the winner. [Self-host](/self-host/) # 16 GiB GPU 16 GiB is the vendor-published framebuffer on RTX 5080, RTX 5070 Ti, RTX 5060 Ti 16, RTX 4080, RTX 4060 Ti 16, RTX A4000, T4, RX 6900 XT, RX 6800 XT. Same pool on every card in that list. Ranking is arithmetic on sourced parameter counts. 16 GiB framebuffer, taken from the sourced cards listed below. Not a free-after-driver figure. 4 of 25 profiled open-weight models fit 16 GiB (vram) at Q5_K_M, 8192 context. Best rated that fits: Phi 4. ** ** ** ** Weights 9.625 GiB KV cache 1.563 GiB Runtime 1 GiB Headroom 3.8 GiB / 16 GiB VRAM as entered. Driver reservation is inside the 1 GiB overhead, not extra. ## Among models that fit Highest LMArena Elo among models that fit: Phi 4 1216.7. Elo is a human-preference scale, not the intelligence index. The arena boards are separate scales and are never blended. A missing cell is unmeasured, not zero. A model’s highest axis is relative to its own scores, not a claim it leads the catalogue. ## Rated models that fit | Model | Human preference | Quant that fits | Estimated GiB | Headroom | | --- | --- | --- | --- | --- | | [Phi 4](/models/microsoft__phi-4/) Fit assumptions Hub safetensors listing says 15B params; the card states 14B. We use the card. KV from config.json: 40 layers, 10 KV heads, hidden 5120 → head dim 128. | 1216.7 | Q5_K_M | 12.188 | 3.813 | ## Fits, unrated - [Ministral 3 14B 2512](/models/mistralai__ministral-14b-2512/) — Q5_K_M, 11.806 GiB unrated - [Ministral 3 8B 2512](/models/mistralai__ministral-8b-2512/) — Q8_0, 11.413 GiB unrated - [Ministral 3 3B 2512](/models/mistralai__ministral-3b-2512/) — F16, 9.413 GiB unrated ## Does not fit, or not profiled - cohere/command-a-plus — even Q4_K_M wants 132.618 GiB; pool is 16 GiB - deepseek/deepseek-v4-flash-vision-exp — even Q4_K_M wants 184.902 GiB; pool is 16 GiB - z-ai/glm-5.3 — even Q4_K_M wants 455.624 GiB; pool is 16 GiB - qwen/qwen3.8-27b — even Q4_K_M wants 18 GiB; pool is 16 GiB - qwen/qwen3.8-2.4t-a95b — even Q4_K_M wants 1450.719 GiB; pool is 16 GiB - meta/muse-glimmer-30b — even Q4_K_M wants 19.277 GiB; pool is 16 GiB - moonshotai/kimi-k3 — even Q4_K_M wants 1691.5 GiB; pool is 16 GiB - z-ai/glm-5.2 — even Q4_K_M wants 455.624 GiB; pool is 16 GiB - moonshotai/kimi-k2.7-code — even Q4_K_M wants 604.75 GiB; pool is 16 GiB - nvidia/nemotron-3-ultra-550b-a55b — even Q4_K_M wants 333.063 GiB; pool is 16 GiB - minimax/minimax-m3 — even Q4_K_M wants 259.405 GiB; pool is 16 GiB - deepseek/deepseek-v4-pro — even Q4_K_M wants 967 GiB; pool is 16 GiB - deepseek/deepseek-v4-flash — even Q4_K_M wants 172.465 GiB; pool is 16 GiB - google/gemma-4-31b-it — even Q4_K_M wants 20.6 GiB; pool is 16 GiB - mistralai/mistral-small-2603 — even Q4_K_M wants 73.2 GiB; pool is 16 GiB - mistralai/mistral-large-2512 — even Q4_K_M wants 408.531 GiB; pool is 16 GiB - deepseek/deepseek-v3.2 — even Q4_K_M wants 406.116 GiB; pool is 16 GiB - openai/gpt-oss-120b — even Q4_K_M wants 72.201 GiB; pool is 16 GiB - meta-llama/llama-4-maverick — even Q4_K_M wants 242.5 GiB; pool is 16 GiB - meta-llama/llama-4-scout — even Q4_K_M wants 66.809 GiB; pool is 16 GiB - cohere/command-a — even Q4_K_M wants 68.016 GiB; pool is 16 GiB ## GPU [RTX 5080](/self-host/rtx-5080/)[RTX 5070 Ti](/self-host/rtx-5070-ti/)[RTX 5060 Ti 16](/self-host/rtx-5060-ti-16/)[RTX 4080](/self-host/rtx-4080/)[RTX 4060 Ti 16](/self-host/rtx-4060-ti-16/)[RTX A4000](/self-host/rtx-a4000/)[T4](/self-host/t4/)[RX 6900 XT](/self-host/rx-6900-xt/) ## How the number is made Weights: a measured GGUF size when we have one, otherwise total parameters × bits per weight ÷ 8. Q4_K_M is treated as 4.83 bits/param. MoE memory uses total parameters, not active parameters. KV cache, when layer and head geometry is sourced: 2 × kvLayers × kvHeads × headDim × context × 2 bytes (FP16, batch 1). Hybrid models use attention-layer count, not every layer. If geometry is missing, cache is omitted and that is stated. 1 GiB is added for CUDA/runtime. Apple unified memory uses RAM as the pool. CPU-only is a memory fit, not a speed claim. DDR generation is ignored for fit — it changes bandwidth, not whether the weights sit in memory. RAM type (DDR4/DDR5/LPDDR) is not a field. It does not change whether a model fits. [Self-host](/self-host/) ## Questions this page answers What LLM can I run on a 16GB card? 4 of 25 profiled open-weight models fit 16 GiB (vram) at Q5_K_M, 8192 context. Best rated that fits: Phi 4. Which sourced cards publish 16 GiB? RTX 5080 (16 GiB), RTX 5070 Ti (16 GiB), RTX 5060 Ti 16 (16 GiB), RTX 4080 (16 GiB), RTX 4060 Ti 16 (16 GiB), RTX A4000 (16 GiB), T4 (16 GiB), RX 6900 XT (16 GiB), RX 6800 XT (16 GiB). Does NVIDIA pick the winner? No. Ranking is arithmetic on sourced parameter counts. Quality leads. Unrated models that fit are listed separately, never scored zero. ## Continue your investigation - [Inspect open-weight models](/open-weights/) - [Compare model variants](/families/) - [Review requirements](/compare/) - [Find serving tools](/tools/)