Coding agents
Which models survive an agent loop
Writing a function is easy. Holding a plan across fifty tool calls, reading a failing test and fixing the right file is not. These are the measurements that separate the two.
Tool-capable models
330
Scored on agentic work
86
Best agentic score
58.4
Qwen3.8 Max
Best Terminal-Bench
66%
GPT-5.6 Sol
Indices
Coding and agentic ability
Two composites. Coding covers writing and fixing code; agentic covers planning, tool use and long-horizon work.
Coding Index
SciCode, LiveCodeBench and Terminal-Bench Hard combined · higher is better
- Claude Fable 5.1Anthropic81.6
- Claude Opus 5Anthropic78.0
- GPT-5.6 SolOpenAI77.4
- GPT-6 AstraOpenAI76.9
- Grok 4.6xAI76.8
- GPT-5.6 TerraOpenAI76.7
- Claude Fable 5Anthropic76.5
- Gemini 3.8 FlashGoogle76.3
- 76.2
- Kimi K3Moonshot AI76.2
View as table
| Model | Coding Index |
|---|---|
| Claude Fable 5.1 (Anthropic) | 81.6 |
| Claude Opus 5 (Anthropic) | 78.0 |
| GPT-5.6 Sol (OpenAI) | 77.4 |
| GPT-6 Astra (OpenAI) | 76.9 |
| Grok 4.6 (xAI) | 76.8 |
| GPT-5.6 Terra (OpenAI) | 76.7 |
| Claude Fable 5 (Anthropic) | 76.5 |
| Gemini 3.8 Flash (Google) | 76.3 |
| Qwen3.8 Max (0902) (Qwen) | 76.2 |
| Kimi K3 (Moonshot AI) | 76.2 |
Agentic Index
Tool use, planning and terminal work · higher is better
- Qwen3.8 MaxQwen58.4
- Claude Fable 5.1Anthropic57.9
- Claude Opus 5Anthropic56.5
- 56.0
- Muse Spark 1.3Meta55.5
- GLM 5.3Z.ai53.1
- Grok 4.6xAI53.0
- GPT-6 AstraOpenAI51.0
- GLM 5.3 FlashZ.ai50.9
- Claude Fable 5Anthropic50.7
View as table
| Model | Agentic Index |
|---|---|
| Qwen3.8 Max (Qwen) | 58.4 |
| Claude Fable 5.1 (Anthropic) | 57.9 |
| Claude Opus 5 (Anthropic) | 56.5 |
| Qwen3.8 Max (0902) (Qwen) | 56.0 |
| Muse Spark 1.3 (Meta) | 55.5 |
| GLM 5.3 (Z.ai) | 53.1 |
| Grok 4.6 (xAI) | 53.0 |
| GPT-6 Astra (OpenAI) | 51.0 |
| GLM 5.3 Flash (Z.ai) | 50.9 |
| Claude Fable 5 (Anthropic) | 50.7 |
Economics
Agentic ability against what it costs
Agent loops burn output tokens. A model that scores two points higher and costs four times as much is not obviously the right call.
Agentic Index vs cost per task
Estimated USD per task on a log scale · up and to the right is the better deal
- Proprietary
- Open weights
Labelled points are the Pareto frontier. Estimated from list pricing: 50K input tokens plus 80K output tokens for reasoning models (25K for non-reasoning).
View as table
| Model | Agentic | Cost / task |
|---|---|---|
| Qwen3.8 Max (Qwen) | 58.4 | $0.58 |
| Claude Fable 5.1 (Anthropic) | 57.9 | $4.50 |
| Claude Opus 5 (Anthropic) | 56.5 | $2.25 |
| Qwen3.8 Max (0902) (Qwen) | 56.0 | $0.58 |
| Muse Spark 1.3 (Meta) | 55.5 | $0.40 |
| GLM 5.3 (Z.ai) | 53.1 | $0.39 |
| Grok 4.6 (xAI) | 53.0 | $0.58 |
| GPT-6 Astra (OpenAI) | 51.0 | $4.50 |
| GLM 5.3 Flash (Z.ai) | 50.9 | $0.05 |
| Claude Fable 5 (Anthropic) | 50.7 | $4.50 |
| GPT-5.6 Sol (OpenAI) | 50.2 | $1.80 |
| Qwen3.8 2.4T A95B (Qwen) | 50.1 | $0.58 |
| Kimi K3 (Moonshot AI) | 50.0 | $1.22 |
| Qwen3.8 27B (Qwen) | 45.8 | $0.19 |
| Claude Sonnet 5 (Anthropic) | 43.6 | $0.90 |
| Muse Spark 1.2 (Meta) | 43.2 | $0.40 |
| GPT-5.6 Terra (OpenAI) | 43.2 | $1.06 |
| GPT-5.6 Luna (OpenAI) | 42.1 | $0.11 |
| Claude Opus 4.8 (Anthropic) | 41.9 | $2.25 |
| Grok 4.5 (xAI) | 41.2 | $0.58 |
| Gemini 3.8 Flash (Google) | 40.2 | $0.34 |
| Claude Opus 4.7 (Anthropic) | 38.6 | $2.25 |
| GLM 5.2 (Z.ai) | 38.4 | $0.56 |
| GPT-5.5 (OpenAI) | 36.4 | $2.65 |
| Gemini 3.7 Flash (Google) | 35.2 | $0.34 |
Underlying evaluations
The four that matter for agents
Terminal work, contest programming, research-grade scientific code, and tool-use discipline under a policy.
Terminal-Bench Hard
Percentage correct · higher is better
- GPT-5.6 SolOpenAI65.9%
- Claude Fable 5Anthropic62.9%
- GPT-5.5OpenAI60.6%
- Claude Opus 4.8Anthropic58.3%
- GPT-5.6 TerraOpenAI57.6%
- GPT-5.4OpenAI57.6%
- Gemini 3.1 Pro PreviewGoogle53.8%
- GPT-5.3-CodexOpenAI53.0%
- GPT-5.4 MiniOpenAI52.3%
- Claude Opus 4.7Anthropic51.5%
View as table
| Model | Score |
|---|---|
| GPT-5.6 Sol (OpenAI) | 65.9% |
| Claude Fable 5 (Anthropic) | 62.9% |
| GPT-5.5 (OpenAI) | 60.6% |
| Claude Opus 4.8 (Anthropic) | 58.3% |
| GPT-5.6 Terra (OpenAI) | 57.6% |
| GPT-5.4 (OpenAI) | 57.6% |
| Gemini 3.1 Pro Preview (Google) | 53.8% |
| GPT-5.3-Codex (OpenAI) | 53.0% |
| GPT-5.4 Mini (OpenAI) | 52.3% |
| Claude Opus 4.7 (Anthropic) | 51.5% |
τ²-bench (Telecom)
Percentage correct · higher is better
- GLM 5.2Z.ai99.1%
- GLM 4.7 FlashZ.ai98.8%
- Claude Fable 5Anthropic98.5%
- Step 3.7 FlashStepFun98.5%
- GLM 5V TurboZ.ai98.5%
- GLM 5 TurboZ.ai98.5%
- GLM 5Z.ai98.2%
- Grok 4.3xAI97.7%
- GLM 5.1Z.ai97.7%
- Qwen3.6 PlusQwen97.7%
View as table
| Model | Score |
|---|---|
| GLM 5.2 (Z.ai) | 99.1% |
| GLM 4.7 Flash (Z.ai) | 98.8% |
| Claude Fable 5 (Anthropic) | 98.5% |
| Step 3.7 Flash (StepFun) | 98.5% |
| GLM 5V Turbo (Z.ai) | 98.5% |
| GLM 5 Turbo (Z.ai) | 98.5% |
| GLM 5 (Z.ai) | 98.2% |
| Grok 4.3 (xAI) | 97.7% |
| GLM 5.1 (Z.ai) | 97.7% |
| Qwen3.6 Plus (Qwen) | 97.7% |
LiveCodeBench
Percentage correct · higher is better
- GLM 4.7Z.ai89.4%
- GPT-5.2OpenAI88.9%
- gpt-oss-120bOpenAI87.8%
- GPT-5.1OpenAI86.8%
- o4 Mini HighOpenAI85.9%
- o4 MiniOpenAI85.9%
- Kimi K2 ThinkingMoonshot AI85.3%
- GPT-5.1-CodexOpenAI84.9%
- GPT-5OpenAI84.6%
- GPT-5 MiniOpenAI83.8%
View as table
| Model | Score |
|---|---|
| GLM 4.7 (Z.ai) | 89.4% |
| GPT-5.2 (OpenAI) | 88.9% |
| gpt-oss-120b (OpenAI) | 87.8% |
| GPT-5.1 (OpenAI) | 86.8% |
| o4 Mini High (OpenAI) | 85.9% |
| o4 Mini (OpenAI) | 85.9% |
| Kimi K2 Thinking (Moonshot AI) | 85.3% |
| GPT-5.1-Codex (OpenAI) | 84.9% |
| GPT-5 (OpenAI) | 84.6% |
| GPT-5 Mini (OpenAI) | 83.8% |
SciCode
Percentage correct · higher is better
- Claude Opus 5.5Anthropic66.9%
- Claude Fable 5.1Anthropic63.1%
- Gemini 4 ArgonGoogle61.8%
- Claude Sonnet 5.5Anthropic61.0%
- Claude Fable 5Anthropic61.0%
- MiMo-V2.6-ProXiaomi60.9%
- Muse WonderMeta60.3%
- Kimi K3Moonshot AI59.5%
- GLM 5.3Z.ai59.0%
- Step 5 PreviewStepFun58.9%
View as table
| Model | Score |
|---|---|
| Claude Opus 5.5 (Anthropic) | 66.9% |
| Claude Fable 5.1 (Anthropic) | 63.1% |
| Gemini 4 Argon (Google) | 61.8% |
| Claude Sonnet 5.5 (Anthropic) | 61.0% |
| Claude Fable 5 (Anthropic) | 61.0% |
| MiMo-V2.6-Pro (Xiaomi) | 60.9% |
| Muse Wonder (Meta) | 60.3% |
| Kimi K3 (Moonshot AI) | 59.5% |
| GLM 5.3 (Z.ai) | 59.0% |
| Step 5 Preview (StepFun) | 58.9% |
Agent arena
Building whole applications
Elo from pairwise judgements on complete builds, not snippets — the closest proxy for what an agent is actually asked to do.
Full-stack apps
Arena Elo · dot plot on a stated scale, not bars from zero
- GPT-6 AstraOpenAI1,357
- Claude Fable 5.1Anthropic1,326
- Kimi K3Moonshot AI1,315
- Claude Opus 5Anthropic1,306
- 1,305
- Muse Spark 1.3Meta1,295
- Claude Fable 5Anthropic1,280
- GPT-5.6 SolOpenAI1,262
- Claude Sonnet 5Anthropic1,249
- Claude Opus 4.8Anthropic1,249
View as table
| Model | Elo | Win rate |
|---|---|---|
| GPT-6 Astra | 1357 | 61.1% |
| Claude Fable 5.1 | 1326 | 57.2% |
| Kimi K3 | 1315 | 61.8% |
| Claude Opus 5 | 1306 | 57.3% |
| Qwen3.8 Max (0902) | 1305 | 62.0% |
| Muse Spark 1.3 | 1295 | 56.2% |
| Claude Fable 5 | 1280 | 58.2% |
| GPT-5.6 Sol | 1262 | 54.3% |
| Claude Sonnet 5 | 1249 | 53.7% |
| Claude Opus 4.8 | 1249 | 55.3% |
Mobile apps
Arena Elo · dot plot on a stated scale, not bars from zero
- Claude Opus 5.5Anthropic1,327
- GPT-6 AstraOpenAI1,313
- Claude Fable 5.1Anthropic1,265
- Kimi K3Moonshot AI1,264
- Claude Fable 5Anthropic1,260
- GPT-5.6 SolOpenAI1,248
- Gemini 3.8 FlashGoogle1,233
- Gemini 3.7 FlashGoogle1,227
- Claude Opus 4.8Anthropic1,220
- Muse Spark 1.3Meta1,219
View as table
| Model | Elo | Win rate |
|---|---|---|
| Claude Opus 5.5 | 1327 | 64.2% |
| GPT-6 Astra | 1313 | 65.0% |
| Claude Fable 5.1 | 1265 | 56.3% |
| Kimi K3 | 1264 | 57.0% |
| Claude Fable 5 | 1260 | 60.4% |
| GPT-5.6 Sol | 1248 | 54.5% |
| Gemini 3.8 Flash | 1233 | 54.4% |
| Gemini 3.7 Flash | 1227 | 52.9% |
| Claude Opus 4.8 | 1220 | 55.5% |
| Muse Spark 1.3 | 1219 | 47.9% |
Full list
330 tool-capable models
Every model that can call a tool, which is the floor for running an agent at all.
| Qwen3.8 Max Qwen | 58.1 | $0.58 |
| Claude Fable 5.1 Anthropic | 53.4 | $4.50 |
| Claude Opus 5 Anthropic | 50.8 | $2.25 |
| Qwen3.8 Max (0902) Qwen | 45.4 | $0.58 |
| Muse Spark 1.3 Meta | 48.1 | $0.40 |
| GLM 5.3 Z.aiopen | 44.8 | $0.39 |
| Grok 4.6 xAI | 44.3 | $0.58 |
| GPT-6 Astra OpenAI | 52.7 | $4.50 |
| GLM 5.3 Flash Z.aiopen | 41.8 | $0.05 |
| Claude Fable 5 Anthropic | 49.6 | $4.50 |
| GPT-5.6 Sol OpenAI | 47.0 | $1.80 |
| Qwen3.8 2.4T A95B Qwenopen | 39.9 | $0.58 |
| Kimi K3 Moonshot AIopen | 43.6 | $1.22 |
| Qwen3.8 27B Qwenopen | 33.7 | $0.19 |
| Claude Sonnet 5 Anthropic | 38.2 | $0.90 |
| Muse Spark 1.2 Meta | 39.6 | $0.40 |
| GPT-5.6 Terra OpenAI | 42.1 | $1.06 |
| GPT-5.6 Luna OpenAI | 37.3 | $0.11 |
| Claude Opus 4.8 Anthropic | 41.8 | $2.25 |
| Grok 4.5 xAI | 38.8 | $0.58 |
| Gemini 3.8 Flash Google | 40.9 | $0.34 |
| Claude Opus 4.7 Anthropic | 40.7 | $2.25 |
| GLM 5.2 Z.aiopen | 33.7 | $0.56 |
| GPT-5.5 OpenAI | 38.4 | $2.65 |
| Gemini 3.7 Flash Google | 39.1 | $0.34 |
| Claude Sonnet 4.6 Anthropic | 24.7 | $1.35 |
| MiniMax M3 MiniMaxopen | 29.2 | $0.11 |
| Gemini 3.6 Flash Google | 34.0 | $0.34 |
| Ling 3.0 Flash VL InclusionAIopen | 24.6 | $0.02 |
| V4 Pro 0813 DeepSeekopen | 36.0 | $0.38 |
| V4 Flash 0731 DeepSeekopen | 34.3 | $0.05 |
| V4 Pro 0423 DeepSeekopen | 36.0 | $0.20 |
| V4 Flash 0423 DeepSeekopen | 34.3 | $0.02 |
| Gemini 3.5 Flash Google | 32.6 | $0.79 |
| Muse Spark 1.1 Meta | 33.7 | $0.40 |
| Hy3 Tencentopen | 25.3 | $0.05 |
| Hy3 preview Tencentopen | 22.7 | $0.06 |
| GLM 5.1 Z.aiopen | 26.1 | $0.29 |
| Inkling Small Thinking Machinesopen | 25.7 | $0.12 |
| Inkling Small (free) Thinking Machinesopen | 25.7 | Free |