Smaller models are not always faster.
That sentence broke a heuristic that lives in most engineering teams' heads. The story comes from Lucas's London talk on picking the right model.
An internal code fix pipeline started with Haiku 5, no thinking. Scored 92 percent. The team turned thinking on to hit 100 percent. Out of curiosity, they reran with Sonnet and Opus. Both scored 100. Both finished faster than Haiku with thinking on.
The reason matters. Larger models did not have higher per-token speed. They did the work in fewer turns. Less wandering, less re-validation, more direct execution. If your task is one prompt to one response, per-token speed dominates. If your task is an agent over ten turns, the model that finishes in three turns wins, even if each turn is slower.
The rule worth writing on your whiteboard: pick the model that is cheapest per successful outcome, not the one that is cheapest per token. A model that fails 30 percent of the time at half the per-token cost is more expensive than a reliable model at full cost, once you count retries and engineering time.
This is also why public benchmarks cannot decide your model choice. They give you direction. Your private eval gives you the answer. SWEBench is an average across public tasks. Your production workload is a specific distribution that almost certainly does not match it. Those averages cancel out the details that matter for your business.
The fix is unglamorous. Build a small private eval. Run it across models, thinking levels, and effort settings. Plot pass rate against tokens, cost, and latency. Pick your operating point on the resulting curve.
How recently did you sweep your own workload across model and thinking configurations, and did the results surprise you?
Full breakdown: [Medium article link]