Skip to content
llmwaves

Coding agents

Which models survive an agent loop

Writing a function is easy. Holding a plan across fifty tool calls, reading a failing test and fixing the right file is not. These are the measurements that separate the two.

Tool-capable models

330

Scored on agentic work

86

Best agentic score

58.4

Qwen3.8 Max

Best Terminal-Bench

66%

GPT-5.6 Sol

Indices

Coding and agentic ability

Two composites. Coding covers writing and fixing code; agentic covers planning, tool use and long-horizon work.

Coding Index

SciCode, LiveCodeBench and Terminal-Bench Hard combined · higher is better

View as table
ModelCoding Index
Claude Fable 5.1 (Anthropic)81.6
Claude Opus 5 (Anthropic)78.0
GPT-5.6 Sol (OpenAI)77.4
GPT-6 Astra (OpenAI)76.9
Grok 4.6 (xAI)76.8
GPT-5.6 Terra (OpenAI)76.7
Claude Fable 5 (Anthropic)76.5
Gemini 3.8 Flash (Google)76.3
Qwen3.8 Max (0902) (Qwen)76.2
Kimi K3 (Moonshot AI)76.2

Agentic Index

Tool use, planning and terminal work · higher is better

View as table
ModelAgentic Index
Qwen3.8 Max (Qwen)58.4
Claude Fable 5.1 (Anthropic)57.9
Claude Opus 5 (Anthropic)56.5
Qwen3.8 Max (0902) (Qwen)56.0
Muse Spark 1.3 (Meta)55.5
GLM 5.3 (Z.ai)53.1
Grok 4.6 (xAI)53.0
GPT-6 Astra (OpenAI)51.0
GLM 5.3 Flash (Z.ai)50.9
Claude Fable 5 (Anthropic)50.7

Economics

Agentic ability against what it costs

Agent loops burn output tokens. A model that scores two points higher and costs four times as much is not obviously the right call.

Agentic Index vs cost per task

Estimated USD per task on a log scale · up and to the right is the better deal

  • Proprietary
  • Open weights
01020304050$0.001$0.01$0.10$1.00$10.00Estimated cost per task, USD · better →Agentic IndexQwen3.8 MaxMuse Spark 1.3GLM 5.3GLM 5.3 FlashLing 3.0 Flash VLLing 3.0 Flash

Labelled points are the Pareto frontier. Estimated from list pricing: 50K input tokens plus 80K output tokens for reasoning models (25K for non-reasoning).

View as table
ModelAgenticCost / task
Qwen3.8 Max (Qwen)58.4$0.58
Claude Fable 5.1 (Anthropic)57.9$4.50
Claude Opus 5 (Anthropic)56.5$2.25
Qwen3.8 Max (0902) (Qwen)56.0$0.58
Muse Spark 1.3 (Meta)55.5$0.40
GLM 5.3 (Z.ai)53.1$0.39
Grok 4.6 (xAI)53.0$0.58
GPT-6 Astra (OpenAI)51.0$4.50
GLM 5.3 Flash (Z.ai)50.9$0.05
Claude Fable 5 (Anthropic)50.7$4.50
GPT-5.6 Sol (OpenAI)50.2$1.80
Qwen3.8 2.4T A95B (Qwen)50.1$0.58
Kimi K3 (Moonshot AI)50.0$1.22
Qwen3.8 27B (Qwen)45.8$0.19
Claude Sonnet 5 (Anthropic)43.6$0.90
Muse Spark 1.2 (Meta)43.2$0.40
GPT-5.6 Terra (OpenAI)43.2$1.06
GPT-5.6 Luna (OpenAI)42.1$0.11
Claude Opus 4.8 (Anthropic)41.9$2.25
Grok 4.5 (xAI)41.2$0.58
Gemini 3.8 Flash (Google)40.2$0.34
Claude Opus 4.7 (Anthropic)38.6$2.25
GLM 5.2 (Z.ai)38.4$0.56
GPT-5.5 (OpenAI)36.4$2.65
Gemini 3.7 Flash (Google)35.2$0.34

Underlying evaluations

The four that matter for agents

Terminal work, contest programming, research-grade scientific code, and tool-use discipline under a policy.

Terminal-Bench Hard

Percentage correct · higher is better

View as table
ModelScore
GPT-5.6 Sol (OpenAI)65.9%
Claude Fable 5 (Anthropic)62.9%
GPT-5.5 (OpenAI)60.6%
Claude Opus 4.8 (Anthropic)58.3%
GPT-5.6 Terra (OpenAI)57.6%
GPT-5.4 (OpenAI)57.6%
Gemini 3.1 Pro Preview (Google)53.8%
GPT-5.3-Codex (OpenAI)53.0%
GPT-5.4 Mini (OpenAI)52.3%
Claude Opus 4.7 (Anthropic)51.5%

τ²-bench (Telecom)

Percentage correct · higher is better

View as table
ModelScore
GLM 5.2 (Z.ai)99.1%
GLM 4.7 Flash (Z.ai)98.8%
Claude Fable 5 (Anthropic)98.5%
Step 3.7 Flash (StepFun)98.5%
GLM 5V Turbo (Z.ai)98.5%
GLM 5 Turbo (Z.ai)98.5%
GLM 5 (Z.ai)98.2%
Grok 4.3 (xAI)97.7%
GLM 5.1 (Z.ai)97.7%
Qwen3.6 Plus (Qwen)97.7%

LiveCodeBench

Percentage correct · higher is better

View as table
ModelScore
GLM 4.7 (Z.ai)89.4%
GPT-5.2 (OpenAI)88.9%
gpt-oss-120b (OpenAI)87.8%
GPT-5.1 (OpenAI)86.8%
o4 Mini High (OpenAI)85.9%
o4 Mini (OpenAI)85.9%
Kimi K2 Thinking (Moonshot AI)85.3%
GPT-5.1-Codex (OpenAI)84.9%
GPT-5 (OpenAI)84.6%
GPT-5 Mini (OpenAI)83.8%

SciCode

Percentage correct · higher is better

View as table
ModelScore
Claude Opus 5.5 (Anthropic)66.9%
Claude Fable 5.1 (Anthropic)63.1%
Gemini 4 Argon (Google)61.8%
Claude Sonnet 5.5 (Anthropic)61.0%
Claude Fable 5 (Anthropic)61.0%
MiMo-V2.6-Pro (Xiaomi)60.9%
Muse Wonder (Meta)60.3%
Kimi K3 (Moonshot AI)59.5%
GLM 5.3 (Z.ai)59.0%
Step 5 Preview (StepFun)58.9%

Agent arena

Building whole applications

Elo from pairwise judgements on complete builds, not snippets — the closest proxy for what an agent is actually asked to do.

Full-stack apps

Arena Elo · dot plot on a stated scale, not bars from zero

1,2251,375
View as table
ModelEloWin rate
GPT-6 Astra135761.1%
Claude Fable 5.1132657.2%
Kimi K3131561.8%
Claude Opus 5130657.3%
Qwen3.8 Max (0902)130562.0%
Muse Spark 1.3129556.2%
Claude Fable 5128058.2%
GPT-5.6 Sol126254.3%
Claude Sonnet 5124953.7%
Claude Opus 4.8124955.3%

Mobile apps

Arena Elo · dot plot on a stated scale, not bars from zero

1,2001,350
View as table
ModelEloWin rate
Claude Opus 5.5132764.2%
GPT-6 Astra131365.0%
Claude Fable 5.1126556.3%
Kimi K3126457.0%
Claude Fable 5126060.4%
GPT-5.6 Sol124854.5%
Gemini 3.8 Flash123354.4%
Gemini 3.7 Flash122752.9%
Claude Opus 4.8122055.5%
Muse Spark 1.3121947.9%

Full list

330 tool-capable models

Every model that can call a tool, which is the floor for running an agent at all.

Filter by creator
330 of 330 models
Qwen3.8 Max
Qwen
58.1$0.58
Claude Fable 5.1
Anthropic
53.4$4.50
Claude Opus 5
Anthropic
50.8$2.25
Qwen3.8 Max (0902)
Qwen
45.4$0.58
Muse Spark 1.3
Meta
48.1$0.40
GLM 5.3
Z.aiopen
44.8$0.39
Grok 4.6
xAI
44.3$0.58
GPT-6 Astra
OpenAI
52.7$4.50
GLM 5.3 Flash
Z.aiopen
41.8$0.05
Claude Fable 5
Anthropic
49.6$4.50
GPT-5.6 Sol
OpenAI
47.0$1.80
Qwen3.8 2.4T A95B
Qwenopen
39.9$0.58
Kimi K3
Moonshot AIopen
43.6$1.22
Qwen3.8 27B
Qwenopen
33.7$0.19
Claude Sonnet 5
Anthropic
38.2$0.90
Muse Spark 1.2
Meta
39.6$0.40
GPT-5.6 Terra
OpenAI
42.1$1.06
GPT-5.6 Luna
OpenAI
37.3$0.11
Claude Opus 4.8
Anthropic
41.8$2.25
Grok 4.5
xAI
38.8$0.58
Gemini 3.8 Flash
Google
40.9$0.34
Claude Opus 4.7
Anthropic
40.7$2.25
GLM 5.2
Z.aiopen
33.7$0.56
GPT-5.5
OpenAI
38.4$2.65
Gemini 3.7 Flash
Google
39.1$0.34
Claude Sonnet 4.6
Anthropic
24.7$1.35
MiniMax M3
MiniMaxopen
29.2$0.11
Gemini 3.6 Flash
Google
34.0$0.34
Ling 3.0 Flash VL
InclusionAIopen
24.6$0.02
V4 Pro 0813
DeepSeekopen
36.0$0.38
V4 Flash 0731
DeepSeekopen
34.3$0.05
V4 Pro 0423
DeepSeekopen
36.0$0.20
V4 Flash 0423
DeepSeekopen
34.3$0.02
Gemini 3.5 Flash
Google
32.6$0.79
Muse Spark 1.1
Meta
33.7$0.40
Hy3
Tencentopen
25.3$0.05
Hy3 preview
Tencentopen
22.7$0.06
GLM 5.1
Z.aiopen
26.1$0.29
Inkling Small
Thinking Machinesopen
25.7$0.12
Inkling Small (free)
Thinking Machinesopen
25.7Free