Scovai Scovai
People Analytics 2026-10-23 1 min read

AI as a Tutor Builds Skill. AI as a Ghostwriter Doesn't โ€” and a Week Later You Can Tell Which One Your Team Used.

DSL

Dr. Sarah Liu

AI as a Tutor Builds Skill. AI as a Ghostwriter Doesn't โ€” and a Week Later You Can Tell Which One Your Team Used.

AI access raised test scores by 7.2 percentage points, and a week after the tool was taken away 3.9 points were still there โ€” but only for the people who used AI as a tutor. For those who had it write for them, the gain was gone (Contractor and Reyes, 2026).

Two people on your team can use the same model on the same task and produce work of the same quality. A week later, one of them is measurably better at the task and the other is back where they started. The deliverable in your hands cannot tell you which is which.

The experiment behind those numbers is a proctored, in-person randomised trial: undergraduates studied an unfamiliar technical topic under time pressure and wrote an analytical essay with or without AI access, then sat unaided assessments immediately and again about a week later. The authors' summary of the split is the whole finding โ€” gains remain for students who use AI as a tutor and fade for students who have AI write for them.

The Design Measured the Person, Not the Deliverable

Almost every study of AI and productivity measures output while the tool is in the room. This one does something harder and more decision-relevant: it takes the tool away and measures what the person retained.

Of 256 who signed up, 211 attended the first session and 205 came back for the second, a mean of 6.98 days later. Topics were deliberately unfamiliar โ€” blockchain, carbon capture, CRISPR. Session Two was proctored and unaided for everyone, treatment and control alike. Essays were scored by 309 graders holding a master's degree or PhD, with 23 excluded for failing a comprehension check.

That structure is what separates borrowed output from retained capability. The immediate test tells you whether the tool helped. The unaided follow-up tells you whether the person changed. Those are different questions, and nearly every adoption dashboard in use at a mid-market company answers only the first one โ€” if it answers either.

What the Numbers Actually Say

MeasureResult
Immediate knowledge test, AI access vs control+7.2 pp, 0.28 SD (p = 0.021)
Same test, treatment-on-treated+10.4 pp, 0.41 SD
Unaided test about one week later+3.9 pp, 0.21 SD
Session Two essay quality, written with no AI+0.24 SD
MechanismSame total time; time shifted from writing to reading and searching

The essay result deserves a second look. A week after the tool was removed, in a proctored room with no assistance, the people who had once had AI access wrote measurably better essays โ€” with the largest effects on use of evidence and examples, and on clarity of style. Something was transferred into the person, not just into the document.

Automation Users Won the First Session and Lost the Week

Here is the part that should change how you think about rollout.

Participants who used AI to write for them โ€” the authors call this automation โ€” submitted higher-rated essays in Session One. They looked like the top performers. A week later, in the unaided session, that advantage was simply gone. Participants who used AI as a tutor, asking it to explain rather than produce, kept theirs.

Two modes of use. Indistinguishable in the deliverable. Opposite in their effect on capability. And the mechanism is prosaic: both groups spent the same total time, but AI access shifted minutes away from writing and toward reading and searching for information. The tutor-mode users spent those minutes on comprehension. The ghostwriter-mode users spent them on nothing that stuck.

This is why "AI use" is the wrong adoption metric. It is not a weak metric or a lagging one. It is a metric that assigns the same value to the behaviour that builds your bench and the behaviour that hollows it out.

Your Adoption Dashboard Cannot See AI as a Tutor

If the mode of use is the variable that matters, the obvious question is whether anyone is measuring it. Broadly, no.

ActivTrak's Productivity Lab inferred AI adoption stages from behavioural telemetry across 120,620 workers rather than from licences, logins or self-report. It found 27% at research assistance, 14% at task execution, and 2% at genuine workflow integration (ActivTrak Productivity Lab, 2026). Seat counts and login dashboards measure distribution. They do not measure adoption, and they certainly do not measure mode.

Meanwhile the development capacity that would teach the better mode is contracting. PwC's 2026 workforce survey of 49,364 workers across 48 countries found that the share of workers saying they have access to the learning and development resources they need fell to 51%, down from 59% the previous year โ€” while daily generative-AI use rose from 14% to 22% (PwC, 2026). Access to the tool is scaling. Access to the instruction in how to use it is going the other way.

Workers notice. In GoTo's 2026 workforce study, 39% said their reliance on AI was eroding their skills, and 43% admitted shipping AI output they suspected was low quality (GoTo and Workplace Intelligence, 2026). Self-report is weak evidence on its own. Paired with a proctored experiment that finds the same shape, it stops being noise.

The Asymmetry That Makes This a Succession Problem

One more result, and it is the one that moves this out of L&D and into workforce planning.

A pre-registered three-month field experiment put an AI patent-drafting assistant in the hands of 133 practising patent lawyers at eleven US firms, with blinded expert attorneys scoring every submission. On assisted tasks, quality rose 0.38 SD at ninety days, with larger gains among junior lawyers. Then came an unassisted redlining task where AI was permitted to no one โ€” the skill measurement. Treated lawyers overall scored 0.32 SD higher, but the advantage was concentrated entirely among senior lawyers at 0.45 SD. Junior lawyers showed no average gain; their scores bifurcated, with sharply fewer mediocre results offset by more poor ones and more good ones (Autor et al., NBER, 2026).

The authors' own framing is the sentence to carry into your next planning meeting: the largest gains from AI accrued to the lawyers who retained the least, and foundational expertise may be a prerequisite for extracting durable skill from AI-assisted practice.

Read the two studies together. Mode of use decides whether capability accumulates, and existing expertise appears to decide which mode a person defaults to. Your most senior people are compounding. Your juniors are splitting into two groups, and the assisted output of both looks fine.

Worth stating plainly: the funding and authorship of the patent study sit partly with the tool's maker. The pre-registration and the blinded third-party grading are what carry its credibility, not the sponsor.

The Honest Counter

Four limits, before anyone builds a policy on this.

It is undergraduates on a one-hour task. Not employees on a quarter of real work. The mechanism โ€” time reallocation from production to comprehension โ€” plausibly travels. The effect sizes should not be quoted as workplace estimates, and I would not put 7.2 points in a business case.

It is a preprint. arXiv 2607.08849 was posted in July 2026 and revised in October 2026. It has not completed peer review.

The robustness checks move the number. Excluding participants who broke the study rules reduces the Session One learning effect to 0.24, 0.15 or 0.27 SD depending on which measure you use. The effect survives; its size is softer than the headline.

Mode was observed, not assigned. The tutor-versus-ghostwriter split is the most actionable result in the paper and the least experimentally clean part of it โ€” participants chose how to use the tool. People who reach for explanation may differ from people who reach for output in ways that also predict retention. The patent study's seniority asymmetry is consistent with exactly that reading.

What survives all four: the deliverable is not a read on whether the person is learning, and the gap between the two is a measurement problem you can fix without buying anything.

What to Change This Quarter

Four moves. None is a purchase.

  1. Stop reporting AI use as an adoption metric. Licences, logins and session counts describe distribution. Replace the headline with one number: how many people have demonstrated a named capability unaided after using AI to build it.
  2. Make tutor-mode prompting an explicit, taught behaviour. It is specific enough to put in an onboarding path and in a manager's one-to-one script: ask the model to explain its reasoning and check yours, not to produce the artefact. This is the cheapest intervention in the entire set and nobody is running it.
  3. Add one unaided task to the development cycle for roles where judgment is the deliverable. Not a test โ€” a real task, done without assistance, scored on the same rubric as the assisted version. The gap between the two is your skill-retention signal, and it does not exist in any output-quality report.
  4. Price the junior bifurcation into your succession plan. If assisted output is flat across seniority while retained capability is not, your two-year bench is thinner than your performance data says, and the error runs in one direction.

That fourth move is the one with no obvious home in most companies, because it needs competency measured per cycle against a stable framework rather than inferred from recent deliverables. Scovai's performance and succession layer is built on that separation โ€” competency scored deterministically per competency per cycle on the Great Eight framework, a 9-box computed from that competency data rather than from manager impressions, and succession readiness stated per critical seat as ready now or twelve to twenty-four months out. The relevance here is structural: if your potential axis is reading assisted output, it is reading the tool.

The uncomfortable implication of this research is that your highest-performing AI adopters and your fastest-decaying ones may be the same people, and that nothing you currently measure separates AI as a tutor from AI as a ghostwriter.

So pick one role this quarter, run one unaided task against the same rubric as the assisted version, and find out which direction the gap runs. Measure the person, not the document โ€” the document stopped telling you anything about either the day the tool arrived.

Ready to go beyond the CV?

Scovai's AI-powered Talent Passport reveals what resumes can't: personality, potential, and true job fit.