--- configs: - config_name: agent data_files: - split: latest path: agent/latest-* - split: full path: agent/full-* default: true - config_name: agent_bash_recovery_steps data_files: - split: latest path: agent_bash_recovery_steps/latest-* - split: full path: agent_bash_recovery_steps/full-* - config_name: agent_bash_reliability data_files: - split: latest path: agent_bash_reliability/latest-* - split: full path: agent_bash_reliability/full-* - config_name: agent_praise_complaint data_files: - split: latest path: agent_praise_complaint/latest-* - split: full path: agent_praise_complaint/full-* - config_name: agent_steerability data_files: - split: latest path: agent_steerability/latest-* - split: full path: agent_steerability/full-* - config_name: agent_task_outcome_explicit data_files: - split: latest path: agent_task_outcome_explicit/latest-* - split: full path: agent_task_outcome_explicit/full-* - config_name: agent_tool_hallucination data_files: - split: latest path: agent_tool_hallucination/latest-* - split: full path: agent_tool_hallucination/full-* - config_name: document data_files: - split: latest path: document/latest-* - split: full path: document/full-* - config_name: document_style_control data_files: - split: latest path: document_style_control/latest-* - split: full path: document_style_control/full-* - config_name: image_edit data_files: - split: latest path: image_edit/latest-* - split: full path: image_edit/full-* - config_name: image_to_video data_files: - split: latest path: image_to_video/latest-* - split: full path: image_to_video/full-* - config_name: search data_files: - split: latest path: search/latest-* - split: full path: search/full-* - config_name: search_factuality data_files: - split: latest path: search_factuality/latest-* - split: full path: search_factuality/full-* - config_name: search_style_control data_files: - split: latest path: search_style_control/latest-* - split: full path: search_style_control/full-* - config_name: text data_files: - split: latest path: text/latest-* - split: full path: text/full-* - config_name: text_factuality data_files: - split: latest path: text_factuality/latest-* - split: full path: text_factuality/full-* - config_name: text_style_control data_files: - split: latest path: text_style_control/latest-* - split: full path: text_style_control/full-* - config_name: text_to_image data_files: - split: latest path: text_to_image/latest-* - split: full path: text_to_image/full-* - config_name: text_to_video data_files: - split: latest path: text_to_video/latest-* - split: full path: text_to_video/full-* - config_name: video_edit data_files: - split: latest path: video_edit/latest-* - split: full path: video_edit/full-* - config_name: vision data_files: - split: latest path: vision/latest-* - split: full path: vision/full-* - config_name: vision_style_control data_files: - split: latest path: vision_style_control/latest-* - split: full path: vision_style_control/full-* - config_name: webdev data_files: - split: latest path: webdev/latest-* - split: full path: webdev/full-* license: cc-by-4.0 dataset_info: - config_name: agent features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string - name: cost_per_task_p50 dtype: float64 - name: output_tokens_per_task_p50 dtype: float64 splits: - name: latest num_bytes: 8006 num_examples: 59 - name: full num_bytes: 318747 num_examples: 2366 download_size: 131015 dataset_size: 326753 - config_name: agent_bash_recovery_steps features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 6139 num_examples: 52 - name: full num_bytes: 186775 num_examples: 1583 download_size: 73940 dataset_size: 192914 - config_name: agent_bash_reliability features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: int64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 7046 num_examples: 59 - name: full num_bytes: 93230 num_examples: 783 download_size: 45063 dataset_size: 100276 - config_name: agent_praise_complaint features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 7046 num_examples: 59 - name: full num_bytes: 280299 num_examples: 2366 download_size: 107508 dataset_size: 287345 - config_name: agent_steerability features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 7046 num_examples: 59 - name: full num_bytes: 280299 num_examples: 2366 download_size: 109025 dataset_size: 287345 - config_name: agent_task_outcome_explicit features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 7046 num_examples: 59 - name: full num_bytes: 280299 num_examples: 2366 download_size: 108389 dataset_size: 287345 - config_name: agent_tool_hallucination features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: score dtype: float64 - name: score_ci_lower dtype: float64 - name: score_ci_upper dtype: float64 - name: observation_count dtype: float64 - name: session_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 7046 num_examples: 59 - name: full num_bytes: 280299 num_examples: 2366 download_size: 109507 dataset_size: 287345 - config_name: document features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5531 num_examples: 46 - name: full num_bytes: 71797 num_examples: 602 download_size: 39267 dataset_size: 77328 - config_name: document_style_control features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5531 num_examples: 46 - name: full num_bytes: 71797 num_examples: 602 download_size: 39219 dataset_size: 77328 - config_name: image_edit features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 19048 num_examples: 149 - name: full num_bytes: 451193 num_examples: 3608 download_size: 186481 dataset_size: 470241 - config_name: image_to_video features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5928 num_examples: 50 - name: full num_bytes: 191429 num_examples: 1636 download_size: 84646 dataset_size: 197357 - config_name: search features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5119 num_examples: 42 - name: full num_bytes: 107825 num_examples: 890 download_size: 52679 dataset_size: 112944 - config_name: search_factuality features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 4884 num_examples: 40 - name: full num_bytes: 30616 num_examples: 252 download_size: 24076 dataset_size: 35500 - config_name: search_style_control features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5119 num_examples: 42 - name: full num_bytes: 107825 num_examples: 890 download_size: 52738 dataset_size: 112944 - config_name: text features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: float64 - name: rank dtype: float64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 1469773 num_examples: 11175 - name: full num_bytes: 165521014 num_examples: 1268299 download_size: 64324826 dataset_size: 166990787 - config_name: text_factuality features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 537563 num_examples: 4079 - name: full num_bytes: 14320328 num_examples: 109165 download_size: 5418266 dataset_size: 14857891 - config_name: text_style_control features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: float64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 1469773 num_examples: 11175 - name: full num_bytes: 147020885 num_examples: 1126709 download_size: 57776740 dataset_size: 148490658 - config_name: text_to_image features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 89768 num_examples: 723 - name: full num_bytes: 2947504 num_examples: 23865 download_size: 1162711 dataset_size: 3037272 - config_name: text_to_video features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 5859 num_examples: 50 - name: full num_bytes: 174598 num_examples: 1511 download_size: 80641 dataset_size: 180457 - config_name: video_edit features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 1452 num_examples: 12 - name: full num_bytes: 14341 num_examples: 119 download_size: 15957 dataset_size: 15793 - config_name: vision features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 142895 num_examples: 1161 - name: full num_bytes: 6411481 num_examples: 52352 download_size: 2419045 dataset_size: 6554376 - config_name: vision_style_control features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 142895 num_examples: 1161 - name: full num_bytes: 5826586 num_examples: 47528 download_size: 2241753 dataset_size: 5969481 - config_name: webdev features: - name: model_name dtype: string - name: organization dtype: string - name: license dtype: string - name: rating dtype: float64 - name: rating_lower dtype: float64 - name: rating_upper dtype: float64 - name: variance dtype: float64 - name: vote_count dtype: int64 - name: rank dtype: int64 - name: category dtype: string - name: leaderboard_publish_date dtype: string splits: - name: latest num_bytes: 82580 num_examples: 672 - name: full num_bytes: 4155692 num_examples: 34222 download_size: 1601190 dataset_size: 4238272 --- # Arena Leaderboard Dataset Historical snapshots of the [Arena](https://arena.ai/leaderboard) leaderboard. ## Usage ```python from datasets import load_dataset # Load all historical text style control data ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full") # Load the current text style control leaderboard ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest") # Filter to overall category ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest", filters=[("category", "==", "overall")] ) # Track a specific model's rating over time ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full", filters=[("category", "==", "overall"), ("model_name", "==", "gpt-4o-2024-05-13")], columns=["model_name", "rating", "rank", "leaderboard_publish_date"] ) # Load raw (non-style-controlled) text ratings ds = load_dataset("lmarena-ai/leaderboard-dataset", "text", split="full") # Load the current Agent Arena leaderboard ds = load_dataset("lmarena-ai/leaderboard-dataset", "agent", split="latest") ``` ## Subsets Each arena is a separate subset, arenas with style control have an additional subset with a `_style_control` suffix . - `text`, `text_style_control` - `vision`, `vision_style_control` - `search`, `search_style_control` - `document`, `document_style_control` - `webdev` (Code Arena) - `text_to_image` - `image_edit` - `text_to_video` - `image_to_video` - `video_edit` - `agent` (Agent Arena aggregate), plus per-signal subsets: `agent_bash_recovery_steps`, `agent_bash_reliability`, `agent_praise_complaint`, `agent_steerability`, `agent_task_outcome_explicit`, `agent_tool_hallucination` ## Splits - `full`: All historically published leaderboards - `latest`: Only the most recently published leaderboards ## Notes - On January 9, 2024, the rating system was updated from Elo to [Bradley-Terry](https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model). - On May 16, 2025, style control was made the default for text and vision arenas and the offset was adjusted to put the style control leaderboard on the same rating scale as non style control. - On July 23, 2025, frequency-based re-weighting was implemented. - Search and Webdev Arena data starts from their releases on the arena.ai (then lmarena.ai) on August 7 and November 12 2025 respectively. - Agent Arena data starts from its release on June 4, 2026. - On October 12, 2026, the Agent Arena bash signal was switched from bash recovery to [bash reliability](https://arena.ai/resources/agent-arena-signals). For a complete record of all leaderboard methodology changes, see the [Leaderboard Changelog](https://arena.ai/blog/leaderboard-changelog/). ## Schema | Column | Type | Description | |--------|------|-------------| | model_name | string | Model identifier | | organization | string | Model creator/organization | | license | string | Model license | | rating | float | Arena Score | | rating_lower | float | Lower confidence bound | | rating_upper | float | Upper confidence bound | | variance | float | Rating variance | | vote_count | int | Number of battles for this model | | rank | int | Rank within this leaderboard | | category | string | Leaderboard category (e.g., overall, coding, math) | | leaderboard_publish_date | string | Date that this score was published (`YYYY-MM-DD`) | Agent Arena subsets use [IPS scores](https://arena.ai/blog/agent-arena-methodology/) instead of Bradley-Terry: | Column | Type | Description | |--------|------|-------------| | model_name | string | Model identifier | | organization | string | Model creator/organization | | license | string | Model license | | score | float | IPS score (τ̂) | | score_ci_lower | float | Lower 95% confidence bound | | score_ci_upper | float | Upper 95% confidence bound | | observation_count | int | Signal observations for this model | | session_count | int | Distinct sessions (aggregate subset only) | | rank | int | Rank within this leaderboard | | category | string | Leaderboard category | | leaderboard_publish_date | string | Date that this score was published (`YYYY-MM-DD`) |