Methodology
What is measured, what is estimated
Some numbers here come straight from measurement. One is a derived estimate. Mixing the two silently is how comparison sites become useless, so this page keeps them apart.
Derived figures
Cost per task is an estimate
No source publishes a per-task cost, so this site computes one from list pricing against a single fixed token profile — the same profile for every model, published here so you can disagree with it.
The task profile
- Input tokens
- 50,000
- Output tokens · reasoning models
- 80,000
- Output tokens · other models
- 25,000
Reasoning models get the larger output allowance because their thinking tokens are billed as output. Cache-read pricing is ignored: it depends entirely on how you use the model, and assuming a hit rate would flatter models with aggressive caching.
cost = input_tokens / 1e6 * price.input
+ output_tokens / 1e6 * price.outputBlended price
A single price per model, weighting input and output at 3:1 — roughly the ratio of a realistic chat or agent workload. It exists so models with very different input/output splits can share one axis.
blended = (3 * price.input + 1 * price.output) / 4
Both figures use published list prices. Negotiated rates, committed-use discounts and free tiers are not modelled.
Measured figures
What the indices contain
Composite scores aggregate several evaluations. Aggregation hides shape — two models with the same index can be very different, which is why every model page breaks the score back apart.
GPQA Diamond
Graduate-level science questions written to resist search.
Humanity's Last Exam
A deliberately brutal cross-disciplinary exam; scores stay low.
SciCode
Research-grade scientific coding problems.
τ²-bench (Telecom)
Tool-use and policy adherence in a simulated telecom support agent.
Terminal-Bench Hard
Hard end-to-end tasks solved inside a real terminal.
LiveCodeBench
Contest programming on problems released after training.
MMLU-Pro
Broad multi-task knowledge, harder distractors than MMLU.
IFBench
Precise instruction following under explicit constraints.
AA-LCR (long context)
Long-context reasoning across large documents.
AIME 2025
Competition mathematics.
Speed and latency
Output speed is the median tokens per second across every provider serving that model; latency is the median time to first token. A model served by ten providers reports a blend, not a best case — check the provider pages if you are pinning to one endpoint.
Reasoning models spend the latency window thinking, so a slow first token there measures deliberation rather than a slow serving stack.
Arena Elo
Two models receive the same brief, a judge picks the better result, and Elo accumulates over many such matches. It measures preference, not correctness — a model can top the arena and still be wrong.
Elo is relative to the field a model actually faced, which is why win rate and Elo sometimes disagree.
Coverage
Where the gaps are
Not every model is measured on everything. A missing value is shown as an em dash and sorts to the bottom of any ranking — it is never treated as a zero, because those are very different claims.
Intelligence Index
161
50% of 323
Coding Index
84
26% of 323
Agentic Index
80
25% of 323
Arena Elo
100
31% of 323
Output speed
313
97% of 323
Latency
314
97% of 323
List pricing
323
100% of 323
Open weights
153
47% of 323
What is excluded from rankings
- 60 batch-pricing endpoints, which duplicate an existing model at a discount and would otherwise let one model occupy two rows.
- Free endpoints in price rankings — a $0 listing wins a cheapest-price board without saying anything useful.
- Models below Intelligence Index 30 in the cost-per-task board, so the ranking compares models you would plausibly choose between.
How to read the charts
- Every chart has a table view underneath it. Nothing is reachable only by hovering.
- Ranking bars use one colour: the categories are model names, which have no order, so bar length is the only value encoding.
- Colour carries identity only where series genuinely differ — scatter splits and multi-line trends.
- Scatter plots label the Pareto frontier rather than the top few, which would all pile into one corner.
323 models across 51 creators and 72 inference providers. Benchmarks and prices are point-in-time snapshots and move often — start from the full index if a figure here looks stale.