Scovai Scovai
AI & Operations 2026-10-28 1 min read

Ask the People Building Your Agents Which Assumptions Are Wrong โ€” 167 Technical Experts Just Ranked Them

DSL

Dr. Sarah Liu

Ask the People Building Your Agents Which Assumptions Are Wrong โ€” 167 Technical Experts Just Ranked Them

Almost every study that diagnoses why agentic AI underdelivers surveys the people who approved it. A paper written on 3 September 2026 surveyed the people who build it. Johannes Habel of the University of Houston and eight co-authors set out ten management illusions about agentic AI โ€” misconceptions about what it is, what it can accomplish, and how firms should implement it โ€” organised them along three stages of transformation, and put them to 167 technical experts, who validated both their prevalence and their negative impact. The illusion the builders rated most critical is managers' "tendency to overestimate the workforce reductions these agents can enable" (Habel et al., SSRN, 2026).

Treat that as a measurement rather than a complaint. The distance between your agentic AI assumptions and your engineers' expectations is a quantity you can collect in one meeting, and the people closest to the build have now told you which line of the business case is most likely to be wrong.

It is the headcount line. It is also, in most mid-market companies, the line the entire approval rested on.

The illusion they ranked first already has a scored forecast

The useful thing about the top-ranked illusion is that it is the one claim in agentic AI with a published track record. McKinsey's 2026 State of AI survey is the first edition in which the panel's own prior-year prediction could be graded against what actually happened. In the 2025 wave, 32% of respondents expected AI-related workforce reductions over the coming year. In the 2026 wave, "just 14 percent of respondents from organizations using AI report that AI contributed to an overall decline in workforce size in the past year" โ€” less than half the share that predicted it. Two-thirds reported little or no AI-related change in total employment (McKinsey, 2026).

The forecast overshot realised change by roughly a factor of two. The panel's response to being wrong was to raise the forecast: 39% now expect AI-related employment declines in the coming year, up from 32%.

So the builders in Habel's survey are not registering a vague cultural scepticism. They are ranking first the only assumption in the stack that has been scored, failed, and then repeated. If your agent business case contains a headcount saving, it is underwritten by an estimator with a documented one-year error rate of about 2x.

The headcount illusion has a measurable size โ€” and the average is the wrong number

How large is the real effect? The cleanest quantification comes from a counterfactual rather than a forecast. Colaiacovo and Koning recruited 199 founders of growth-oriented North American B2B ventures through Pitchbook, 176 of whom completed the survey in full, and asked how many additional employees they would need, in the absence of generative AI, to maintain the same output at the same quality (Colaiacovo & Koning, Harvard Business School, 2026).

The mean answer was 55% more employees. The median was 17%. The most common single answer was zero โ€” 30% of founders said they would need nobody extra.

MeasureValue
Mean counterfactual headcount increase without AI55%
Median17%
Share answering zero30%
Firms with AI formally integrated into workflows (mean)91.4%
Firms where AI use is informal (mean)24%

Two readings follow, and both matter more than the headline.

First, a vendor quoting "agents replace half a team" is quoting the mean of a sharply right-skewed distribution whose median firm reports 17%. Any plan built on the mean is a plan built on the tail.

Second, the variable that separates the tail from the middle is not belief about capability. Founders in the study were randomly shown optimistic or pessimistic information about AI task performance; beliefs moved, and willingness to delegate largely did not. What separated them was whether the work had been formally restructured around the tool. Formal integrators reported needing 91.4% more staff without AI โ€” nearly four times the 24% reported where use was informal.

That is not a procurement finding. It is an organisational design finding, which is exactly where Habel's second stage sits.

Stage two is where transformations stall: nobody owns the agent

Habel's framing puts the illusions about organisational structure between strategy and execution โ€” and that middle stage is where the external evidence is most alarming, because it measures accountability rather than intent.

IBM's Institute for Business Value surveyed 2,000 C-level technology executives. Two-thirds report being held accountable for AI systems they do not fully control. Only 11% say they are completely prepared for the scale of agent deployment expected in the coming year, against 80% operating under a CEO-driven AI mandate. Seventy percent say teams across the business deploy technology faster than IT can track it, and 77% say AI adoption already outpaces their governance capability. Surveyed organisations averaged 54 agent incidents in the preceding year โ€” unintended or harmful occurrences requiring human correction โ€” of which 17% were high-severity (IBM Institute for Business Value, 2026).

The same study prices the fix. Organisations that build control into the AI system itself, rather than governing it manually afterwards, experience 25% fewer incidents, deploy 16x more agents, report 18% higher operating margins, and spend 4x less of their AI budget.

Read those four numbers together. Control is not the brake on agent deployment; in this data it is the condition for it. The firms with the most agents running are the firms that made oversight a property of the system before they scaled.

Why this stage is the cheap one to fix

An illusion about strategy costs you a wrong slide. An illusion about organisational design costs you a deployment that cannot be unwound: no named owner for the agent's output, no defined boundary around what it may never do alone, no restructured role for the human who was supposed to be freed up. Those are decisions that are nearly free to make before procurement and expensive to retrofit once an agent is in production and the 54-incident clock is running.

Stage three: the pilot that is never killed and never scaled

Execution illusions show up as a specific pathology. Valliance surveyed 1,000 senior leaders at Europe's largest enterprises and found that 40% of AI initiatives remain pilots by design โ€” rising to 48% at mature organisations with established AI programmes. More maturity produced more permanent pilots, not fewer. Organisations stuck at pilot stage reported lower success rates (43% against 50% overall), longer time to value (6.6 months against 5.9), and strong ROI in only 20% of cases, against 76% where a pilot was scaled inside a mature organisation (Valliance via Consultancy.uk, 2026).

The firm's own gloss is worth borrowing: "experimentation itself isn't the problem." What fails is what happens next, or does not. Projects stall between proof of concept and production because nothing in the process forces a verdict.

Note the sequence this builds. The strategy illusion sets a headcount target that was never achievable. The organisational design illusion leaves the agent without an owner or a restructured role. Execution then produces a pilot that cannot be killed, because killing it would require admitting the first two.

What the paper does not give you

Be precise about the limits, because the paper is being quoted loosely.

It is a 14-page practitioner-facing article, not a field experiment. The published abstract reports that 167 technical experts validated the prevalence and negative impact of ten illusions and names the top-ranked one; it does not publish per-illusion percentages, response-rate details, or a sampling frame. Anyone citing a specific number against a specific illusion is inventing it. The expert panel is also self-selected and technical, which is the point of the design and simultaneously its bias: builders have a professional interest in the claim that management misunderstands the technology.

The corroborating sources each carry their own caveat. McKinsey's panel is global and enterprise-skewed. The founder counterfactuals are self-estimated beliefs from ventures averaging 16 employees, not audited payroll. Valliance covers Europe's largest enterprises at ยฃ39.2M average annual AI spend, nothing like a 200-person operation, and Bain has published a counternarrative arguing most use-case categories are in fact scaling to production. Both can hold if scaling is uneven and abandoned pilots simply accumulate alongside the successes.

What survives all of it is the direction and the ordering. Four independent instruments โ€” a builder survey, a graded forecast, a founder counterfactual and a CxO control audit โ€” point at the same conclusion: the binding constraint is organisational, it sits upstream of the tool, and the headcount assumption is the first thing to check.

Converting your agentic AI assumptions into a document before procurement

Five moves, none of which require a new vendor.

  1. Run the disagreement as a survey, not a conversation. Give your business case's three load-bearing claims to the engineers who would build or operate the agent, and ask each to rate prevalence and impact privately before anyone discusses it. You are measuring a gap, and a meeting averages it away.
  2. Strip the headcount saving out of the approval and make the case stand without it. If it does not stand, you have not found a use case; you have found a staffing argument wearing one. The realised-change baseline is roughly 14%, not the 32% that gets forecast.
  3. Name the owner and the restructured role before the contract. Habel's middle stage is the expensive one to retrofit. The question is not "which tasks can the agent do" but "whose seat is this, who approves its output, and what does the human in that workflow now do instead."
  4. Write down what stays permanently outside autonomy. Not a risk register โ€” an explicit list of action classes the agent may never execute alone, agreed before deployment, travelling with the assignment. This is the single cheapest artefact in the whole programme.
  5. Set the kill-or-scale gate with the budget, not after it. A fixed evaluation window, pre-committed success metrics, and a binary verdict. Forty percent of initiatives stay pilots by design because nobody ever has to decide.

Items 3 and 4 describe governance that has to live somewhere โ€” in your own process, or in the system you deploy. Scovai builds both into the seat: its Digital Employees are confirm-first by default, every externally visible action becomes an approval request with a named supervisor attached, autonomy is earned per kind of action from a logged ledger rather than granted at setup, specified action classes stay permanently outside autonomy, and each seat on the org chart carries an augmentability score before anything is deployed into it.

One decision for this quarter

You do not need to resolve whether agentic AI will reduce headcount. You need to decide whether your agentic AI assumptions are allowed to stand before the people building the thing have said so on the record.

So before the next agent contract goes to signature, put the three claims your business case depends on in front of your technical team, collect their ratings, and look at the headcount line first. If your engineers' estimate and your plan's estimate are far apart, you do not have a procurement decision in front of you. You have a forecast that has already failed once, and a free opportunity to not repeat it.

Ready to go beyond the CV?

Scovai's AI-powered Talent Passport reveals what resumes can't: personality, potential, and true job fit.