Anthropic's latest study of roughly 235,000 users and 400,000 Claude Code sessions upends a common assumption: the deciding factor in AI coding success is not how well you code, but how deeply you understand the problem you're solving.
An accountant can be an 'expert' on Claude — here's how
Anthropic built a five-level task-specific proficiency scale, from novice to expert. The definition of 'proficiency' is unconventional: it measures how well you grasp the problem, not your programming skill. A senior engineer writing Rust for the first time counts as a novice on that task; a Python-illiterate accountant who can precisely specify reconciliation rules and catch month-end logic errors is an expert on that task.
The numbers are stark. A novice session averages roughly 5 Claude actions and 600 characters per prompt; an expert session triggers 12 actions and 3,200 characters — 2x more actions, 5x more output. Regression analysis shows each proficiency level increase yields ~9% more Claude actions and ~13% more output, holding task type, value, month, occupation, and model version constant.
Who can pull the agent back when it goes wrong?
Success rates paint an even clearer picture. Anthropic defined two success tiers: 'judged success' (classifier determines goal met) and 'verified success' (hard evidence like tests passing, git commits, user confirmation). Overall, higher proficiency correlates with higher session success, with most gains concentrated at the lower end — the jump from novice to intermediate is larger than from intermediate to expert. Expert sessions achieve more than double the verified success rate of novices.
Recovery after failure is even more telling. Among sessions that hit trouble, verified success climbs from 4% (novice) to 15% (expert); partial success rates rise from 60% (novice) to 80-81% (intermediate through expert). Abandonment rates diverge sharply: novices quit 19% of the time (classified as fail with zero code changes) vs 5-7% for other levels. The insight: domain expertise matters most when the agent strays — the user knows where it's wrong and can steer it back.
Managers beat software engineers; occupational gaps nearly vanish
Another surprise: occupation matters less than expected. Software-related roles hit about 30% verified success, other occupations ~26%. Focusing on sessions with actual code output, the gap widens to 34% vs 29%, but at 'at least partial success' they converge: 89% vs 88%. All top-10 occupations fall within 7 percentage points of software engineers. Managers actually edge out engineers in verified success — possibly because their habit of delegating tasks and setting specs transfers well to directing AI agents.
Work patterns shifted rapidly over seven months. Bug-fixing sessions dropped from 33% to 19%, nearly halved. Operational tasks (deployments, configs, running pipelines) rose from 14% to 21%. Writing and data analysis roughly doubled from ~10% to 20%. Users are putting Claude Code to work on more code-adjacent tasks, not just coding itself. Task economic value also rose: estimated freelance market value per session climbed ~27% over seven months, with construction tasks up ~43%, operations ~34%, fixes ~32%.
Anthropic concludes with a framework: gains come from 'competence, not mastery' — basic-to-intermediate domain understanding captures most benefits; the slope flattens as you climb from intermediate to expert. As AI tools expand, they amplify not coding skill but problem comprehension. Those who don't understand what they're solving will just get lost faster with better models.