---
name: undominated-kv-cache-sizing-audit
description: Audits KV cache memory footprint per token and max concurrent sequence limits under FP16, FP8, and INT4 KV caches to prevent serving OOMs.
license: MIT
metadata:
  author: Undominated.ai
  version: "1.0.0"
  tags: ["kv-cache", "vram", "hardware", "memory", "concurrency"]
---

# KV Cache Memory Sizing Audit

Audits KV cache memory footprint per token and max concurrent sequence limits under FP16, FP8, and INT4 KV caches to prevent serving OOMs.

## When to Use
Deploy before sizing serverless concurrency limits or self-hosted GPU node requirements across long contexts.

## Limitations
Hardware memory models assume standard Multi-Head Attention (MHA) or Grouped-Query Attention (GQA). Verify key-value head counts explicitly.

## Verification Script
Run the deterministic stdlib checker:
```bash
python3 scripts/check.py examples/synthetic.json
```
Exits `0` on verified pass, `1` on failed assertion, `2` on malformed input.