Disagreement
Evaluators are not interchangeable. We publish one licensed quality axis and we do not blend a second.
Condition: standard-delivery rows with a measured CI half-width. LMArena data is CC BY 4.0.
Why we refuse to blend evaluators
We never blend evaluators into one quality number. Different labs measure different things. A composite would invent a precision neither source claims, and it would hide the models they would rank differently. The disagreement is the signal. A blend erases it.
A second lab we do not sublicence
A second evaluation lab exists. We do not sublicence those figures, so they do not appear here — not as scores, not as ranks, not as a correlation. Internal use is not a public surface.
Uncertainty we can publish
On the LMArena board, adjacent ranks that sit inside published error bands are ties. The share in the answer is recomputed from this catalogue: standard-delivery rows with a measured CI half-width, CC BY 4.0. Details live on Significance.
Unrated is not zero
206 of 348 standard, publishable models have no LMArena score. Unrated is not zero. Absence is not a favourable verdict. Those models stay unrated until a licensed measurement exists.
Cite this claim
Plain text and BibTeX
As of 2026-11-05, 94% of adjacent LMArena pairs on this catalogue — 132 of 141 — fall inside the models' own published error bands. That is the uncertainty we can publish. We do not blend a second lab's figures into this rank, and we do not invent a crisp first place. As of 2026-11-05. Lens: LMArena Elo (CC BY 4.0). https://undominated.ai/disagreement/
Publication: sha256:5d4c4fe0b25c71badf22f8e487146fa4dec33728cb96eb590d4c545fe54021b7; calculation SHA-256: fe6e66bba404c2382400785273cdd10c512261febefcd31122d0a3785c1a83b7; permitted inputs: https://undominated.ai/data/publications/manifests/5d4c4fe0b25c71badf22f8e487146fa4dec33728cb96eb590d4c545fe54021b7.json@misc{undominated-claim-2026-11-05,
author = {{Undominated.ai}},
title = {Disagreement and score uncertainty — Undominated.ai},
year = {2026},
url = {https://undominated.ai/disagreement/},
note = {as of 2026-11-05; LMArena Elo (CC BY 4.0)},
annote = {Publication: sha256:5d4c4fe0b25c71badf22f8e487146fa4dec33728cb96eb590d4c545fe54021b7; calculation SHA-256: fe6e66bba404c2382400785273cdd10c512261febefcd31122d0a3785c1a83b7; permitted inputs: https://undominated.ai/data/publications/manifests/5d4c4fe0b25c71badf22f8e487146fa4dec33728cb96eb590d4c545fe54021b7.json}
}LMArena scores used under CC BY 4.0 · numbers recomputed from the catalogue on this page. Input identity & calculation SHA.