Twenty thousand UK civil servants were given a Microsoft 365 Copilot licence, and the cross-government findings report put the result at 26 minutes saved per person per day (Government Digital Service, 2025). The figure has been circulating in procurement decks ever since. Two details from the same report change what it can carry: participants produced it by estimating their own daily time savings from a set of ranges, and the conclusions section states that "due to experimental constraints it was not possible to identify how time saved was spent."
Then the Department for Work and Pensions โ one of the departments inside that same trial โ evaluated its own cohort against a comparison group of colleagues who never received a licence. Its estimate came in at 19 minutes a day across eight routine tasks (DWP, 2026).
Two measurements of one rollout, published by the same government. The distance between them is the distance between a time-savings headline and an AI pilot business case.
What 20,000 Licences Actually Measured
The cross-government experiment ran from 30 September to 31 December 2024, with each participating organisation committing at least 1,000 licences (Government Digital Service, 2025). By the standards of most AI productivity evidence, this is a good artefact: large, public sector, published with its methodology attached, and not a vendor case study with three flattering logos on it.
The method is where a Head of Operations should slow down. Participants were asked to estimate their average daily time saved by choosing from ranges spanning "none or less than 5 minutes" to "more than an hour." The 26-minute average was computed from the midpoints of those ranges, with the top band treated as 60 minutes. Seventeen percent noticed no clear time saving at all. More than 70% agreed Copilot reduced time spent searching for information โ which, as later evidence suggests, is the use case where the tool is most visibly useful and least economically consequential.
So the instrument is a recall question: how much time do you think this saved you? That question can be asked of 20,000 people cheaply, which is why it gets asked. It cannot produce the two numbers a business case actually runs on.
The two numbers a self-report cannot give you
The first is the counterfactual โ what would have happened to those tasks anyway, with the same staff, the same quarter and no licence. The second is the destination: where the recovered minutes went.
Without the first, you are measuring enthusiasm alongside effect. Without the second, you are measuring an input and calling it a return.
The Same Rollout, Run Against People Who Didn't Have It
DWP's evaluation supplies the first of those numbers. It used a Seemingly Unrelated Regression against a comparison group of non-users, controlling for demographic, occupational and AI-keenness variables, and estimated 19 minutes per day saved across eight routine tasks โ statistically significant across all specifications. The largest per-task reductions were searching for information (26 minutes) and writing emails (25 minutes). Ninety percent of users said Copilot helped them save time, and 65% reported feeling more fulfilled at work (DWP, 2026).
DWP is also explicit about its own ceiling: this was not a randomised controlled trial. Licences were allocated first-come, first-served, so self-selection and post-treatment bias are live concerns. The keenest people got the licences.
Resist the arithmetic here. The two figures are not apples to apples โ different populations, different task scope, different method โ and subtracting one from the other produces a number neither document supports. The finding is structural, not numerical: the department that added a comparison group reported a smaller figure than the self-estimate, in a programme where the enthusiasts had selected themselves into the treatment. That is the direction the correction runs, and it is the direction that matters when a vendor hands you a single number with no control group anywhere near it.
The Sentence That Should Stop the Budget Meeting
Of everything in the cross-government report, one line in the conclusions does the most damage to how the 26 minutes is used: "due to experimental constraints it was not possible to identify how time saved was spent."
A recovered minute has three possible fates. It becomes output. It gets absorbed by another task. Or it becomes slack. Only the first appears in a business case, and the trial could not distinguish between them.
This is not a public-sector quirk. The first federal, vendor-independent measurement of the same question found that among US workers who had used AI at work in the previous week, 56% put the additional time they would have needed without it at two hours a week or less; 10% reported no time saved, and 3% said AI required more time (U.S. Census Bureau, 2026).
Two hours a week, arriving as scattered minutes, does not consolidate into a reallocatable FTE. The operational test is blunt: does a whole task leave somebody's plate? If the answer is no, then hours multiplied by a loaded rate is a rhetorical device, not a forecast.
Why Per-User Minutes Don't Aggregate Into an AI Pilot Business Case
The participation problem compounds the destination problem. The ECB's Consumer Expectations Survey reports the same result at two altitudes: the median AI user saves around three hours a week, roughly 7.7% of median working time โ but only 48.8% of workers both use AI and save time with it, which puts the economy-wide efficiency gain at about 3.8% (ECB, 2026).
Same survey, same respondents, and the number halves when the denominator stops being "users" and starts being "everyone you are paying."
The ECB data also inverts the usage pattern in a way that should reorder a pilot. Code generation and debugging return close to eight hours a week โ and roughly 8% of users apply AI there. Research, information gathering, writing and editing are the most common uses and save considerably less. The cross-government trial's headline benefit, less time searching for information, sits squarely in the low-yield band.
Which means the business case needs three inputs where most decks have one: participation rate, task-level yield, and an explicit conversion factor for whether freed hours become output or slack. Each of those is observable before a renewal. None of them is in a 26-minute average.
The Honest Counter
Three limits, stated plainly.
Self-report can err in both directions โ but the measured direction has not been kind. METR's randomised trial on experienced open-source developers found early-2025 AI tools made them 19% slower while they believed they had sped up; its subsequent survey of 349 technical workers reports that participants in that study overestimated AI's effect on task time by 40 percentage points on average (METR, 2026). That is a narrow population working on repositories they know intimately, and the magnitude does not port to a mid-market back office. It establishes something smaller and still useful: a self-estimated saving is not a conservative input.
These evaluations were not failures. Both found real, statistically meaningful time savings, and DWP found fulfilment effects worth having on their own terms. The criticism here is not of the trials โ which are more transparent than most of what gets sold as evidence โ but of the work the headline number is asked to do once it leaves the report.
Public sector is not your sector. Civil service task mix, procurement constraints and baseline tooling are specific. The transferable asset is the evaluation design, not the minutes. Do not port 26 or 19 into your model; port the question that produced the second one.
What to Change in the Pilot Design Before the Next Renewal
Four moves, none of which requires a new budget line.
- Hold back a comparison group, and don't let the keen self-select. The cheapest way to make a pilot number defensible is a cohort inside the same function that doesn't get a licence for the same 90 days. DWP's own stated limitation โ first-come, first-served allocation โ is the part to fix: randomise licences within a team rather than handing them to whoever asks first.
- Name the destination before the pilot starts. Write down, per seat, which specific task is expected to leave whose plate, and what is expected to fill the space. If the answer is "people will have more time," the pilot is measuring satisfaction, and should be budgeted as such.
- Instrument one task, not the whole rollout. Pick two tasks with a system-observable cycle time and measure those end to end. A defensible delta on two tasks beats a survey average across forty.
- Score the seat, not the headcount. The destination question has to be answered at seat level, because that is the level at which work actually moves. Scovai's org-chart augmentability scoring is one worked form of that output โ each seat scored 0โ100% on how augmentable it is, with the uncovered roles and the per-seat plan falling out of the same chart.
The UK government did something unusual: it published both the flattering number and the method that undercuts it, and then let one of its departments mark its own homework with a comparison group. That is a higher evidentiary standard than most AI pilots inside private companies ever meet.
So before you renew a licence block this quarter, ask for one page: which tasks left which plates, and what arrived in the space. If nobody can write that page, you are not holding an AI pilot business case. You are holding a satisfaction survey with a licence fee attached.
