--- title: "Self-host — which open-weight model fits this machine · Undominated.ai" canonical: https://undominated.ai/self-host/ description: "From RAM and GPU memory, rank the catalogue open-weight models that actually fit. Quality leads. Unrated is not zero. A missing parameter count is not a guess." --- # Self-host — which open-weight model fits this machine · Undominated.ai > From RAM and GPU memory, rank the catalogue open-weight models that actually fit. Quality leads. Unrated is not zero. A missing parameter count is not a guess. # Self-host RAM and whether you have a GPU are enough. Everything else is optional. The ranking is arithmetic on sourced parameter counts — NVIDIA does not pick the winner. 20 catalogue rows marked open-weight. Nothing else is a candidate. 20 of those have a sourced parameter count. The rest are listed as unknown, not guessed. 1 GiB runtime allowance, named, not measured on your machine. System RAM GiB GPU None (CPU) NVIDIA AMD Apple Intel GPU memory GiB Required unless you name a card we list. Apple: leave this; RAM is the pool. GPU name (optional) GPU count OS (optional — software only) Skip Linux Windows macOS Context to budget 4K 8K 16K 32K KV cache grows with this. We do not assume the model card maximum. 6 of 20 profiled open-weight models fit 24 GiB (vram) at 8192 context. Best rated that fits: Qwen3.8 27B (Q5_K_M). ** ** ** ** Weights 18.563 GiB KV cache 0.5 GiB Runtime 1 GiB Headroom 3.9 GiB / 24 GiB VRAM as entered. Driver reservation is inside the 1 GiB overhead, not extra. 32 GiB raises the quant of Qwen3.8 27B to Q8_0. Same model, not a higher intelligence score. ## Among models that fit Qwen3.8 27B has the highest Artificial Analysis intelligence (52), coding (68.1) and agentic (50.9) scores among models that fit. Highest LMArena Elo among models that fit: Gemma 4 31B 1441.7. Elo is a human-preference scale, not the intelligence index. Artificial Analysis indices and LMArena Elo are different scales. A missing cell is unmeasured, not zero. A model’s highest axis is relative to its own scores, not a claim it leads the catalogue. ## Rated models that fit | Model | General | Coding | Agentic | Human preference | Quant that fits | Estimated GiB | Headroom | | --- | --- | --- | --- | --- | --- | --- | --- | | Qwen3.8 27B AA peak: Coding vision video reasoning tools | 52 | 68.1 | 50.9 | 1440.8 | Q5_K_M | 20.063 | 3.938 | | Gemma 4 31B AA peak: Coding vision video reasoning tools | 29.7 | 43.4 | 14.4 | 1441.7 | Q5_K_M | 22.106 | 1.894 | | Olmo 3 32B Think reasoning | — | — | — | 1298.5 | Q4_K_M | 22.32 | 1.68 | | Granite 4.1 8B tools | — | 9.5 | — | 1291.6 | F16 | 18.25 | 5.75 | | Phi 4 | — | — | — | 1216.8 | Q8_0 | 17.438 | 6.563 | - Hybrid: only 16 of 64 layers are Gated Attention. KV uses kvLayers=16, not nLayers=64. Measured Q4 is Unsloth UD-Q4_K_M, not a vanilla Q4_K_M. ## Fits, unrated - [Muse Glimmer 30B](/models/meta__muse-glimmer-30b/) — Q5_K_M, 21.756 GiB unrated vision reasoning tools ## What an upgrade actually buys The fit bottleneck is GPU memory. More system RAM will not load extra weights onto this card. | Estimated GiB | Unlocks | Example | List price | | --- | --- | --- | --- | | 32 GiB | Qwen3.8 27B — Q8_0 same model, higher quant | GeForce RTX 5090 (32 GiB) | USD 1999 starting at | The next higher-rated catalogue model is Kimi K3, and even Q4_K_M wants 1691.5 GiB. That is not a desktop card. NVIDIA 'Starting at' list prices as published 2026-08-25, not a store quote and not a used-market index. Street prices in August 2026 are often much higher (Tom's Hardware Newegg median RTX 5090 $4,699 vs $1,999 list). Last-gen 24 GB cards (4090, 3090) have no current NVIDIA starting-at in this file. Datacenter cards have no consumer list here. RAM DIMMs are not priced — they move weekly. Upgrades use the same fit formula as the ranking. List prices are NVIDIA starting-at, dated 2026-08-25. Street prices are often higher. Used 24 GB cards and datacenter GPUs have no list here. RAM DIMMs are not priced. ## Does not fit, or not profiled - qwen/qwen3.8-2.4t-a95b — even Q4_K_M wants 1450.719 GiB; pool is 24 GiB - moonshotai/kimi-k3 — even Q4_K_M wants 1691.5 GiB; pool is 24 GiB - z-ai/glm-5.2 — even Q4_K_M wants 455.624 GiB; pool is 24 GiB - moonshotai/kimi-k2.7-code — even Q4_K_M wants 604.75 GiB; pool is 24 GiB - nvidia/nemotron-3-ultra-550b-a55b — even Q4_K_M wants 333.063 GiB; pool is 24 GiB - minimax/minimax-m3 — even Q4_K_M wants 259.405 GiB; pool is 24 GiB - deepseek/deepseek-v4-pro — even Q4_K_M wants 967 GiB; pool is 24 GiB - deepseek/deepseek-v4-flash — even Q4_K_M wants 172.465 GiB; pool is 24 GiB - mistralai/mistral-small-2603 — even Q4_K_M wants 73.2 GiB; pool is 24 GiB - deepseek/deepseek-v3.2 — even Q4_K_M wants 406.116 GiB; pool is 24 GiB - openai/gpt-oss-120b — even Q4_K_M wants 72.201 GiB; pool is 24 GiB - meta-llama/llama-4-maverick — even Q4_K_M wants 242.5 GiB; pool is 24 GiB - meta-llama/llama-4-scout — even Q4_K_M wants 66.809 GiB; pool is 24 GiB - cohere/command-a — even Q4_K_M wants 68.016 GiB; pool is 24 GiB ## Extra details (optional) Does not change the ranking. Optional AI analysis of your notes against the numbers above, using NVIDIA NIM’s strongest models. Analyze with AI ## How the number is made Weights: a measured GGUF size when we have one, otherwise total parameters × bits per weight ÷ 8. Q4_K_M is treated as 4.83 bits/param. MoE memory uses total parameters, not active parameters. KV cache, when layer and head geometry is sourced: 2 × kvLayers × kvHeads × headDim × context × 2 bytes (FP16, batch 1). Hybrid models use attention-layer count, not every layer. If geometry is missing, cache is omitted and that is stated. 1 GiB is added for CUDA/runtime. Apple unified memory uses RAM as the pool. CPU-only is a memory fit, not a speed claim. DDR generation is ignored for fit — it changes bandwidth, not whether the weights sit in memory. Only catalogue rows with openWeights=true are candidates. 20 such rows exist; 20 have sourced sizes. Unrated models that fit are listed separately. A higher score does not mean a drop-in replacement. RAM type (DDR4/DDR5/LPDDR) is not a field. It does not change whether a model fits.