---
title: "Agents · bash reliability: model rankings and source evidence · Undominated.ai"
canonical: https://undominated.ai/rankings/arena-agent-bash-reliability/
description: "Agents · bash reliability ranked by IPS score. Inspect exact model variants, source dates, measurement limits and downloadable evidence."
---

# Agents · bash reliability: model rankings and source evidence · Undominated.ai

> Agents · bash reliability ranked by IPS score. Inspect exact model variants, source dates, measurement limits and downloadable evidence.

[All rankings](/rankings/) / Coding and agents

# Agents · bash reliability

Agent Arena signal scores from the official dataset. Each signal is a separate measurement; it is not a success percentage.

**58 entries** · IPS score · Higher is better

Source publication: 2026-11-04. Retrieved 2026-11-05.

Source: [Arena (lmarena-ai)](https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset) · [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/).

Ranks use confidence bounds. Several models can share a rank; the rows within a shared rank are displayed by score.

Agents · bash reliability · IPS score

| Significance rank | Model / evaluated variant | IPS score | 95% confidence interval | Observations |
| --- | --- | --- | --- | --- |
| 1 | Claude Sonnet 5 (High) anthropic | 0.150823 | 0.145215 – 0.15643 | 231,347 |
| 1 | Claude Opus 5.5 (Max) anthropic | 0.150062 | 0.145787 – 0.154336 | 264,620 |
| 3 | GPT-6.1 Terra (Max) openai | 0.135199 | 0.131378 – 0.13902 | 528,425 |
| 3 | Claude Sonnet 5.5 (Max) anthropic | 0.126451 | 0.119686 – 0.133216 | 140,397 |
| 4 | GPT-5.6 Sol (xHigh) openai | 0.113142 | 0.106509 – 0.119775 | 213,048 |
| 5 | GPT-6 Sol (Max) openai | 0.112677 | 0.107167 – 0.118186 | 227,574 |
| 5 | Claude Fable 5 (High) anthropic | 0.10193 | 0.093162 – 0.110699 | 125,868 |
| 7 | Claude Opus 5 (Max) anthropic | 0.10026 | 0.094497 – 0.106024 | 243,903 |
| 5 | Claude Fable 5.1 (Max) anthropic | 0.100216 | 0.091307 – 0.109124 | 114,914 |
| 7 | Gemini 3.8 Flash (High) google | 0.099694 | 0.09578 – 0.103608 | 846,362 |
| 5 | Grok 4.6 (xHigh) xai | 0.098167 | 0.088 – 0.108334 | 72,255 |
| 7 | GPT-6 Luna (Max) openai | 0.095959 | 0.093178 – 0.09874 | 2,849,515 |
| 7 | Grok 4.7 (xHigh) xai | 0.094426 | 0.088185 – 0.100666 | 256,072 |
| 12 | Muse Wonder Full (Max) meta | 0.084011 | 0.079628 – 0.088394 | 563,468 |
| 11 | GPT-6 Astra (Max) openai | 0.080033 | 0.068525 – 0.091541 | 42,658 |
| 14 | Kimi K3.1 Pro (Max) moonshot | 0.077946 | 0.07234 – 0.083552 | 376,791 |
| 15 | Claude Haiku 5.5 (High) anthropic | 0.07362 | 0.067977 – 0.079263 | 471,690 |
| 15 | GPT-6 Luna (Medium) openai | 0.069343 | 0.065101 – 0.073585 | 715,939 |
| 15 | Hy4 tencent | 0.066155 | 0.05465 – 0.07766 | 117,034 |
| 16 | Gemini 3.7 Flash (High) google | 0.062608 | 0.054019 – 0.071197 | 257,912 |
| 19 | GPT-6.1 Sol (Max) openai | 0.050516 | 0.043575 – 0.057457 | 240,656 |
| 19 | Muse Wonder (Max) meta | 0.049739 | 0.040897 – 0.05858 | 221,013 |
| 19 | Qwen3.7-Max alibaba | 0.035785 | 0.01134 – 0.06023 | 42,865 |
| 23 | DeepSeek-V4.1-Flash-20261017 (Max) deepseek | 0.032041 | 0.02893 – 0.035153 | 2,502,923 |
| 19 | Beam 1.0 501B A23B reflection | 0.03068 | 0.002456 – 0.058904 | 93,649 |
| 24 | MiMo-V2.6-Pro xiaomi | 0.022276 | 0.015638 – 0.028914 | 621,400 |
| 21 | Inkling-Small thinky | 0.021174 | -0.008848 – 0.051196 | 16,293 |
| 24 | GPT-6 Luna (Low) openai | 0.019372 | 0.012095 – 0.026648 | 275,807 |
| 24 | Muse Spark 1.3 (Max) meta | 0.01869 | 0.010873 – 0.026506 | 532,135 |
| 23 | DeepSeek-V4-Pro (High, 20260813) deepseek | 0.017976 | 0.005069 – 0.030884 | 86,502 |
| 21 | Gemini 3.1 Pro Preview google | 0.016872 | -0.015215 – 0.048959 | 35,300 |
| 22 | Inkling thinky | 0.013742 | -0.015964 – 0.043449 | 17,111 |
| 24 | GPT-5.6 Terra (xHigh) openai | 0.008166 | 0.000449 – 0.015884 | 211,380 |
| 23 | Qwen3.7-Plus alibaba | 0.008145 | -0.020271 – 0.036562 | 32,326 |
| 25 | MiMo-V2.5-Pro xiaomi | -0.008052 | -0.028464 – 0.012361 | 55,140 |
| 31 | MiniMax-M3 minimax | -0.020824 | -0.038838 – -0.00281 | 80,936 |
| 32 | Muse Spark 1.1 meta | -0.024236 | -0.038725 – -0.009746 | 291,738 |
| 35 | Qwen3.8-Max alibaba | -0.028594 | -0.035402 – -0.021787 | 594,068 |
| 35 | Hy4 preview tencent | -0.030979 | -0.036006 – -0.025952 | 1,101,920 |
| 35 | Muse Spark 1.2 (xHigh) meta | -0.037642 | -0.048816 – -0.026468 | 212,382 |
| 36 | MiMo-V2.6-Flash xiaomi | -0.041424 | -0.050871 – -0.031977 | 379,502 |
| 38 | GLM-5.2 (Max) zai | -0.045286 | -0.053006 – -0.037566 | 468,965 |
| 40 | Kimi K3 (Max) moonshot | -0.046982 | -0.053399 – -0.040565 | 682,226 |
| 40 | MiniMax-M3.1-Flash minimax | -0.049565 | -0.059181 – -0.039948 | 310,814 |
| 40 | GLM-5.3 (Max) zai | -0.053902 | -0.06225 – -0.045554 | 521,367 |
| 40 | Gemini 4 Carbon (High) google | -0.054983 | -0.062424 – -0.047542 | 476,207 |
| 40 | Gemini 4 Argon (High) google | -0.056096 | -0.067569 – -0.044623 | 478,113 |
| 38 | Mistral Large 4 mistral | -0.058971 | -0.080303 – -0.037639 | 36,597 |
| 36 | Mistral Medium 3.5 mistral | -0.073185 | -0.112307 – -0.034063 | 32,021 |
| 47 | Spark-X2.6 synlan | -0.086609 | -0.109851 – -0.063366 | 119,972 |
| 49 | GLM-5.3-Flash zai | -0.112543 | -0.121052 – -0.104035 | 962,019 |
| 49 | Step 5 Preview stepfun | -0.116863 | -0.130816 – -0.10291 | 310,483 |
| 49 | Qwen3.8-27B alibaba | -0.117097 | -0.129487 – -0.104708 | 359,452 |
| 50 | NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4) nvidia | -0.127886 | -0.143494 – -0.112279 | 147,074 |
| 49 | Hy3 tencent | -0.136683 | -0.172148 – -0.101218 | 13,169 |
| 36 | Nemotron 3 Ultra nvidia | -0.15385 | -0.274069 – -0.033631 | 5,428 |
| 56 | Qwen3.8-Flash-Next alibaba | -0.217838 | -0.225425 – -0.210251 | 1,174,420 |
| 58 | Solar Pro 4 upstage | -0.448204 | -0.525002 – -0.371406 | 17,227 |

## What this ranking can tell you

 - The population is the models in this source board, including variants and models without an available API price.
- Ranks use the published confidence bounds: one plus the number of models whose lower bound exceeds this model’s upper bound. Shared ranks are not proof of equal performance.

## Check the source

The original evidence is retained byte for byte. The downloadable board includes the source URLs, revision and SHA-256 hashes.

[Download this board and provenance →](/data/rankings/arena-agent-bash-reliability.json)

 Source files and licences - [metadata · a7a159f557a1](/data/rankings/evidence/a7a159f557a1e594a5d41218fe8d53c63c63daa39b655e51d6fc5b482a17d059.json) · [upstream source ↗](https://huggingface.co/api/datasets/lmarena-ai/leaderboard-dataset)
- [licence · 88a46d472f35](/data/rankings/evidence/88a46d472f35274fb6b7a852c3d16c8ee8007d51b37f290161537bec26895ee4.txt) · [upstream source ↗](https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset/raw/8b6deb69ace01ee8dcda9e46eba7a5e8a808ec53/README.md)
- [rows · d1ac4050ea00](/data/rankings/evidence/d1ac4050ea00c1be6589371c86e83e5ecfe9ee016362dea6985b8e94703cc150.parquet) · [upstream source ↗](https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset/resolve/8b6deb69ace01ee8dcda9e46eba7a5e8a808ec53/agent_bash_reliability/latest-00000-of-00001.parquet)

[Explore other model types →](/rankings/)[Read the benchmark guide →](/benchmarks/)

## Continue your investigation

 - [Understand the measurements](/benchmarks/)
- [Compare task performance](/best/)
- [Compare embedding models](/embeddings/)
- [Inspect ranking rules](/methodology/)