Objective
If you’re budgeting for an AI feature, you care about what it costs to get an answer that’s good enough to use. That cost can fall because tokens get cheaper, because the model needs fewer of them, or because both happen. I looked at benchmark and pricing data to see how much we can tell about each.
What the Capability Charts Show
Artificial Analysis’s Progress Index estimates model capability on a common scale across its evaluations. The leading open-weight models have earned higher scores over time. Some of those leaders have also grown in parameter count, which raises a question about how much progress is possible within a similar size class. Artificial Analysis open-weight model comparison

The best scores also rose among open-weight models with up to 4 billion parameters. The ceiling stays fixed, while the exact parameter count, training and effort at answer time can change. The size-band chart shows that more capable models became available within that ceiling. Artificial Analysis progress by model size

Comparable Capability Costs Less
Artificial Analysis tracks the lowest cost per Intelligence Index task among model settings that meet each Progress Index capability threshold. Those costs fell. This measures the cheapest qualifying option available at each point in time, so a different model can set the low at the next release. Artificial Analysis efficiency trends

The lowest price per million output tokens also fell at several capability thresholds. These two charts track distinct quantities. Price per token is the rate a provider charges. Tokens per task is how much a model generates to do the work, including reasoning tokens where reported. A lower cost per task can reflect either of those changes, as well as differences in input costs. Artificial Analysis efficiency trends

Where Token Efficiency Improved
OckBench reports accuracy and average output tokens per task on the same set of problems. Its combined metric, OckScore, rewards accuracy and penalizes token use. When OckScore goes up, the underlying figures show whether the model answered more questions correctly, generated fewer tokens, or did both. OckBench paper · OckBench leaderboard
In one comparison, Gemini 3 Flash Preview used 95.5% fewer output tokens per task than Gemini 2.5 Flash while scoring slightly higher (44.5% versus 43.5%). That’s an observed gain for the two settings OckBench tested. OckBench leaderboard data

Two other examples point in the same direction.
- FrontierMath. Epoch AI reported that GPT-5.2 at low reasoning effort needed about 5 million output tokens to reach roughly 27% accuracy. o4-mini at high effort took about 43 million to reach comparable accuracy. The models and effort settings differ, so the comparison shows what was possible on that benchmark. Epoch AI’s FrontierMath comparison
- GDPval v2. OpenAI reports that GPT-6.2 Astra scored 6.2 percentage points higher than GPT-6 Astra while using 22% fewer output tokens on its internal benchmark for professional workflows. OpenAI’s evaluation
The Chart I’d Like to See
These examples point to better token efficiency in some comparisons. How much it has improved across releases, and how often the pattern holds across providers, is still open.
It would be useful if Artificial Analysis added a release-by-release view of token efficiency. I’d plot benchmark scores against output tokens per task, including reasoning tokens, for successive releases in the same model family and size class. Keep the benchmark tasks and scoring fixed, and label the release date and effort setting at each point.
That would make the change easy to read. If a newer point moves left at about the same height, it reaches similar quality with fewer tokens. If it moves up and right, the score improved and the model generated more tokens to get there. For someone choosing a model, both are useful results. They describe different trade-offs.
The interactive companion brings the charts and sources together.
Acknowledgements
I used OpenAI Codex to help check the sources, prepare the charts and edit this draft.