Experienced AI Users Get Better Answers. Anthropic Has the Data.

Does getting good at AI actually change what you get out of it? Or is it all down to the tool?

Anthropic’s March Economic Index gives the best answer I’ve seen, from a million real conversations.

Anthropic split Claude users into two groups. People who signed up at least six months earlier, and everyone else. Then it checked how often each group’s conversations got the job done.

The experienced group succeeded 73.1% of the time. Newer users managed 66.7%.

Both groups were using Claude in the same week in February, and the experienced group’s success rate was about 10% higher.

I train teams to use AI at work, so I’d love to tell you that settles the argument for training. It doesn’t quite, and I’ll show you why. But the detail underneath the headline shows exactly what good looks like.

What Anthropic Actually Measured

Every few months Anthropic publishes the Economic Index, a look at how people use Claude for real tasks. It’s built from privacy-preserving analysis of conversations. The March edition covered a million Claude.ai chats from one week in February.

Two definitions matter. “High tenure” means you signed up at least six months before the data was pulled. “Success” is Claude’s own judgement of whether it completed the task you gave it.

So what did the two groups do differently?

Isn’t This Just Clever People Signing Up Early?

That was my first thought too. Early adopters are more likely to be programmers and analysts, the kind of people who’d get more out of any tool.

Anthropic thought the same, so it ran the comparison again with people doing the same narrow task. One of its examples is a cluster called “Perform corporate financial analysis, valuation, and modeling for specific companies”. Experienced and newer users doing that exact work, side by side.

The gap shrank to about 3 percentage points. With task, model, country and use case all controlled for, it settled at about 4 points.

Experienced users also bring harder work. Their prompts need about 12.3 years of education to understand, against 11.5 for newer users. And harder work normally fails more often: in Anthropic’s January report, tasks needing less than a school-leaver’s education succeeded about 70% of the time against 66% for degree-level ones.

So the experienced group takes on tougher tasks and still gets more of them done.

Anthropic is careful about what it can’t prove yet. People who signed up a year ago and gave up aren’t in the data. Claude is marking its own homework on success. And February’s new users were swollen by sign-ups from Anthropic’s Super Bowl adverts.

What Experienced Users Do Differently

You’d expect the experienced crowd to fire off quick commands and let AI get on with it. Anthropic expected that too. In its own words, the result “pushes back against a hypothesis we made last year that automated use may be more typical of more experienced, sophisticated users”.

Experienced users give one-line orders in 29.4% of conversations, against 38.1% for newer users. They spend more of their time going back and forth on a draft, asking Claude to explain things and getting it to check their work.

They also use it for work more: 48.9% of their conversations against 41.6%.

What does that look like at a desk? Someone gives the context up front. They read the first draft, say what’s wrong with it and ask for another go, and when it makes a choice they ask why. They paste in their own work and ask what’s weak about it.

You can see the same habit elsewhere in Anthropic’s data. In the June report, the typical blog post written in chat took 13 rounds of back-and-forth.

That’s how you’d work with a good new starter.

Experienced users also pick the right model for the job. On Claude.ai, 34% of software developer tasks run on Anthropic’s top model, Opus, against 12% of tutoring tasks, and Opus use climbs with the hourly wage of the task. In my experience, plenty of people outside tech have never touched the model picker. That’s a ten-minute fix in a training session.

Your Brief Sets the Ceiling

Anthropic also measures how much education you’d need to understand each prompt, and each answer.

Across individual conversations the two line up closely, with a correlation of 0.87. Across countries it’s 0.925. Anthropic’s summary is blunt: “how users prompt Claude shapes how it responds.”

On average Claude answers about a year above the level of the prompt. For emails and blog posts that gap is close to nothing. Ask for a blog in a lazy sentence and you’ll get a blog written at the level of a lazy sentence.

If you lead a marketing or comms team, this is the useful bit: write the brief you’d give a good freelancer. Who it’s for, what it needs to do, what good looks like, what to avoid. The model meets you where you start.

Wiring It In and Walking Away

Plenty of businesses and developers skip the chat window altogether and plug AI straight into their systems through Anthropic’s API. That traffic looks very different.

API calls are single requests with no conversation, so there’s no chance to say “not quite, try again”. Anthropic suggests the higher Claude.ai success rate may reflect the benefits of multi-turn conversation, where people “can clarify, correct course, and iterate”.

We build these automations for clients. They work when someone who knows the job designs the checks, decides what gets reviewed by a person and tests it on real cases before it runs on its own. That’s the same skill the experienced chat users show, built into a process.

Where the UK Sits

Anthropic also tracks how much each country uses Claude against the size of its working-age population. A score of 1 means usage matches population. Higher means people use it more than you’d expect.

The UK went from 2.67 to 3.35, and France sits above both us and the US in the May snapshot, having been below both in August 2025.

Look at how Brits use it and there’s good news. In Anthropic’s published data for May, UK users hand over fewer one-line orders than the global average (about 28% of chats against 31%) and Anthropic’s estimate of the time they spend working on a task with Claude is higher (about 48 minutes against 40). That’s the collaborative style the experienced users show.

Now the catch. In the same data, about 39% of UK conversations are for work against roughly 43% globally, and a bigger share of ours are personal. Worldwide, the June report found personal use rises from around 35% of chats on weekdays to just under 50% at weekends.

So UK users work with it the way the experienced group does. Less of that is happening at work.

Six Months Is a Long Time to Wait

If experience is what makes the difference, can you speed it up? Two controlled studies say yes.

When Stanford and MIT researchers followed 5,172 customer support agents given an AI assistant, agents two months into the job with the AI assistant performed as well as agents more than six months in without it. The least experienced agents got the biggest lift, about 30% more issues resolved per hour.

In the Harvard and BCG study of 758 consultants, below-average performers improved by 43% with AI and above-average ones by 17%. But on a task outside what the AI was good at, consultants using it were 19 percentage points less likely to get the right answer. Knowing where it falls over is part of the skill.

And training hours matter. Here’s what BCG found when it asked more than 10,000 workers how much AI training they’d had.

Yet BCG’s 2026 survey found only 36% of workers feel properly trained, no better than the year before, while 88% expect to need major upskilling in the next five years.

Microsoft’s 2026 Work Trend Index points the same way. Among its most advanced AI users, 61% are on teams that share tips, agents and mistakes, against 36% of AI users overall. And 43% deliberately do some work without AI to keep their own skills sharp, against 30%.

What to Do With This

Anthropic will keep publishing these, and it says it hopes to separate learning from early-adopter effects in future editions. I’ll be reading.

In the meantime, how many of your team are six months in, and how many are still sending one-line orders?

Sources & References

Frequently Asked Questions

Roy Murphy is co-founder of G3NR8. This post draws on Anthropic’s Economic Index reports (February 2025 to June 2026) and independent research on AI skills and training. Published 24 October 2026.