Anthropic’s Claude Code drew sharp criticism after developers said the product seemed to get worse without warning, only for one user to uncover a hidden experiment in the way reasoning effort was represented in requests.
According to the source material, developer argofowl spent most of an afternoon trying to figure out why Claude Code was acting oddly. He first suspected that t3 code had failed, then thought his own code might contain a bug, and at one point even wondered whether he had somehow broken his Mac.
The picture changed when he opened the actual API request logs. There, he saw the number “10,” even though he had selected “high,” the top reasoning setting, in Claude Code. He also noted that the product changelog said nothing about such a change.
A “high” setting that showed up as 10
argofowl said that starting with Claude Code 2.1.237, the model was reading the “high” reasoning setting as 10 out of 100. That number, in earlier usage, matched what had been the “low” tier.
Further digging, as described in the source text, suggested that Anthropic had placed Fable 5 sessions in Claude Code 2.1.236 and later into an experiment that compressed the numeric effort scale. Older versions and Opus 5 were not affected by that experiment.
The source also said this was likely an A/B test, which would explain why not every user ran into the same behavior. That became a central complaint for developers: not simply that a model might feel stronger or weaker, but that some users could be placed into an experiment without being told, leaving them to spend hours debugging what they believed was their own environment or code.
After tech blogger Chubby reposted the issue, the discussion spread quickly across the AI community. Posts asking whether Claude had become “dumber” began appearing across X. Some users shared side-by-side outputs from identical prompts across versions. Others pulled up conversations from two weeks earlier and compared them line by line.
Anthropic engineer says the number had no standalone meaning
Claude Code engineer Thariq Shihipar responded soon after the complaints gained traction. He said Anthropic sometimes tests API service configurations inside Claude Code before deciding whether to push them more broadly.
In his explanation, the live experiment only changed the numeric mapping of effort values, which is why some users saw Claude identify itself as “10.” He added that the scale should not be interpreted as running from 0 to 100, so the number by itself carried no meaning. Users, he said, were still getting the effort level they had selected.
Shihipar also said the team had run deep evaluations and had confirmed that the change did not affect model performance.
Separate criticism focused on Opus 5
The Fable 5 explanation did not end the wider debate. Chubby then said Opus 5 felt like a significant downgrade. The source text described complaints that the model had become perfunctory, made frequent low-level mistakes, and often fell back on the same mechanical response when told it had failed to follow instructions: “You’re right, that was my oversight.”
It also said some users had already flagged what they saw as a clear drop in Opus 5’s capabilities several days earlier. Among the examples cited in the source were a tendency to introduce bugs, spend large amounts of time fixing those bugs, and repeatedly correct itself during the same task.
After more public questioning, Shihipar acknowledged that Opus 5 was an “extremely inconsistent” performer, with output quality swinging up and down. He said the team was working hard on the issue and that resolving it had become the highest priority.
Benchmarks up, user experience down
The Opus 5 dispute exposed a broader tension described in the source: benchmark performance and day-to-day user experience may be drifting apart.
The article cited benchmark figures for Opus 5 that looked strong on paper, including an overall score of 82.72, SWE-bench Pro at 79.2%, and Terminal-Bench at 86.7%. At the same time, users were describing the model in very different terms, calling it verbose, lazy, and argumentative.
That contrast stood out because both assessments were attached to the same model at the same time: high published scores on one side, and negative hands-on feedback on the other.
A transparency problem beyond one company
The source text argued that claims of models getting “dumber” are not unique to Anthropic. It framed the issue as part of a wider lack of transparency across the AI industry as large model services continue to change under the hood.
Traditional software usually comes with semantic versioning, changelogs, and rollback mechanisms, allowing developers to see what version they are using and what has changed. Large model products work differently. Under the same model name, providers may run A/B tests, switch quantization approaches, change model routing, or alter reasoning resources on the server side.
That leaves users relying heavily on feel. And as the source noted, intuition is easy to dismiss and hard to prove.
The episode’s value, in the source’s telling, was that it brought a long-running problem into open view. If models are becoming infrastructure, then stability is part of the trust contract. Benchmarks can support marketing claims. Stability has to be demonstrated in repeated use.
Referenced posts
The source cited public X posts tied to the debate, including Thariq Shihipar’s response and a post from kimmonismus.
- https://x.com/trq212/status/2091252347913773169?s=20
- https://x.com/kimmonismus/status/2091178321669198014