Disagreement
Evaluators are not interchangeable. We publish one licensed quality axis and we do not blend a second.
Condition: standard-delivery rows with a measured CI half-width. LMArena data is CC BY 4.0.
Why we refuse to blend evaluators
We never blend evaluators into one quality number. Different labs measure different things. A composite would invent a precision neither source claims, and it would hide the models they would rank differently. The disagreement is the signal. A blend erases it.
A second lab we do not sublicence
A second evaluation lab exists. We do not sublicence those figures, so they do not appear here — not as scores, not as ranks, not as a correlation. Internal use is not a public surface.
Uncertainty we can publish
On the LMArena board, adjacent ranks that sit inside published error bands are ties. The share in the answer is recomputed from this catalogue: standard-delivery rows with a measured CI half-width, CC BY 4.0. Details live on Significance.
Unrated is not zero
213 of 358 standard, publishable models have no LMArena score. Unrated is not zero. Absence is not a favourable verdict. Those models stay unrated until a licensed measurement exists.
Cite this claim
Plain text and BibTeX
As of 2026-10-06, 94% of adjacent LMArena pairs on this catalogue — 136 of 144 — fall inside the models' own published error bands. That is the uncertainty we can publish. We do not blend a second lab's figures into this rank, and we do not invent a crisp first place. As of 2026-10-06. Lens: LMArena Elo (CC BY 4.0). https://undominated.ai/disagreement/
Publication: sha256:ecef83f767cd4d2c21766f6a5b4dffa36c06351f23657542d473bc1a030c0932; calculation SHA-256: 5ae4d0bee9b70298dd437b05df87c32acb407040b50bf88240677f295f25955d; permitted inputs: https://undominated.ai/data/publications/manifests/ecef83f767cd4d2c21766f6a5b4dffa36c06351f23657542d473bc1a030c0932.json@misc{undominated-claim-2026-10-06,
author = {{Undominated.ai}},
title = {Disagreement and score uncertainty — Undominated.ai},
year = {2026},
url = {https://undominated.ai/disagreement/},
note = {as of 2026-10-06; LMArena Elo (CC BY 4.0)},
annote = {Publication: sha256:ecef83f767cd4d2c21766f6a5b4dffa36c06351f23657542d473bc1a030c0932; calculation SHA-256: 5ae4d0bee9b70298dd437b05df87c32acb407040b50bf88240677f295f25955d; permitted inputs: https://undominated.ai/data/publications/manifests/ecef83f767cd4d2c21766f6a5b4dffa36c06351f23657542d473bc1a030c0932.json}
}LMArena scores used under CC BY 4.0 · numbers recomputed from the catalogue on this page. Input identity & calculation SHA.