When thinking becomes overthinking.
A visual tour of TRACE, a framework for seeing when a reasoning model is still solving the problem, and when it is just circling the runway.
≠
not always better
Reasoning is a tool, not a virtue.
Thinking models can be brilliant on hard problems, but they may burn a lot of tokens on easy ones without improving the answer.
Plain English
A model should spend enough effort to reach a reliable answer, then stop. TRACE helps you locate that stopping point from the shape of the reasoning, not just the number of tokens.
Longer is not the same as wiser.
The paper compares thinking and non-thinking modes across 14 models and six data domains, with a focus on simple queries.
For tasks like basic arithmetic, date arithmetic, and fact recall, extra reasoning often adds latency and cost after the answer is already within reach. The danger is operational: slower responses, higher inference spend, and more opportunities to talk yourself out of a correct answer.
What to watch
Overthinking is not “a response that looks long”. It is additional thought that fails to create a meaningful performance gain.
That distinction matters for evals: a token budget is a blunt proxy for effort; a utility curve is more informative.
There is a narrow sweet spot.
The paper’s vertical analysis suggests that useful thinking sits between trivial work and problems beyond a model’s representational capacity.
Two different failure modes
Too easy: the answer is already available, so extra thought mostly costs time.
Too hard: more tokens cannot repair a missing internal representation of the rules.
So “make the model think harder” is not a universal remedy.
Turn a monologue into a map.
TRACE stands for Thought-process Reconstruction and Automated Clustering Engine. Click each stage to see what it contributes.
01 / Response sampling
Collect reproducible reasoning traces from several thinking models, query types, and difficulty levels. The paper samples four large thinking models for TRACE and benchmarks 14 models overall.
Not every thought does the same job.
TRACE labels the relationship between one sub-thought and the next, so a graph can distinguish progress from motion.
Minimal sub-thoughts
A sub-thought should be self-contained, complete, and answer-bearing. That gives the analyser a stable unit to score for correctness and helpfulness.
Relational labels
Overthinking has a shape.
Explorer
The correct answer can appear early, but the model keeps branching into alternatives. It may backtrack, revisit an earlier answer, and distribute its probability of correctness across many nodes.
Typical waste: over-exploration, with verification as a secondary driver.
1
2
3
4
Stop measuring effort. Start measuring return.
The paper reframes overthinking as the point after which the marginal return drops below a threshold ε.
Explore the thought index
At sub-thought 1, the model is still establishing a path.
More compute can help, then mostly stops helping.
For Qwen3-235B-A22B, thinking adds substantial accuracy on harder math, but at a steep token cost. On easy levels, the gain is negligible.
97.44 → 100.00% accuracy, while inference cost grows from 46.5 → 320 words.
92.94 → 95.88% accuracy, while cost grows from 54.4 → 348.5 words.
74.75 → 91.50% accuracy, but thinking uses 1,021.7 words versus 118.1.
32.24 → 52.54% accuracy, but cost rises from 309.2 → 2,485.6 words.
Interpretation: the right policy is task- and model-aware. “Always think” and “never think” are both crude defaults.
Give the model a principled exit.
TRACE suggests proxies for the convergence point that can work without ground-truth answers at runtime.
Self-looping
Terminate after k consecutive self-verifications following an answer proposal. The paper’s starting case uses k = 2.
Backtrack
Terminate when the model revisits a previously proposed answer directly as the result of a backtrack action.
Use the lens anywhere inference has a budget.
Model evals
Measure the quality of thinking, not just the final answer.
- Build a golden set across easy, medium, and hard tasks.
- Track accuracy, helpfulness, tokens, latency, and cost together.
- Cluster failure traces into Explorer, Late Landing, sidetrack, and correction-heavy behaviours.
Build a stopping policy you can explain.
A practical TRACE-inspired loop: collect traces, label structure, estimate utility, and enforce an exit condition.
Start small. Make the stopping rule observable.
Define the workload
Choose one narrow workload where you can label success reliably. Mix easy, medium, and hard examples so your policy learns where more thought pays off.
workload = {
"domains": ["math", "temporal", "recall"],
"difficulty": ["easy", "medium", "hard"],
"metrics": ["accuracy", "latency", "tokens", "cost"]
}Put the exit logic beside the model.
The goal is not to expose chain-of-thought to users. The goal is to make inference behaviour measurable and controllable.
What to log
Design principle
Keep policy decisions inspectable. If you cannot explain why the model stopped, you cannot confidently tune the trade-off.
Good reasoning knows when it is done.
TRACE offers a sharper operational lens than “the answer got longer”.
One sentence to remember
Overthinking begins when the next unit of thought is costing more than it is returning.