Scovai Scovai
Hiring 2026-11-02 1 min read

Discrimination Follows Discretion, Not the Job Title: a 36,880-Application Audit Locates the Gap in Subjective Evaluation

DSL

Dr. Sarah Liu

Discrimination Follows Discretion, Not the Job Title: a 36,880-Application Audit Locates the Gap in Subjective Evaluation

Nine economists sent 36,880 applications to 9,220 real job advertisements for new college graduates across the United States. In management occupations, callbacks came back 28 to 43 percent lower for Black men, Black women, White women and Hispanic men than for otherwise identical White men (Braun et al., 2026). That number will get quoted. The more useful finding is the one underneath it: the gap did not follow the job title. It followed how much of the hiring decision was left to unverifiable judgement.

That distinction changes what you do on Monday. If hiring discrimination were a property of occupations, the fix would be a policy applied uniformly across every requisition. If it is a property of evaluative discretion, uniform policy is the wrong instrument โ€” you are spreading a thin layer of structure across roles that do not need it while leaving the roles that do need it untouched.

The 36,880-Application Design, and Why It Can See What Others Can't

Rรฉsumรฉ audits are not new. What makes this one informative is the linkage. The authors took the task preferences firms express in their own advertisement text and linked them to occupation-level task measures from O\*NET and the American Community Survey. That gives every application a task profile rather than just an occupation code.

They then built a model in which discrimination increases with evaluative discretion โ€” defined as the share of the hiring decision driven by subjective rather than verifiable assessment. The audit tests that model against callbacks.

The headline management-occupation result is the familiar shape of the literature. The structural results are where the paper earns its keep:

  • Broad occupation categories conceal important variation in task demands. Grouping requisitions by job family averages away the signal you need.
  • When jobs are grouped by task intensity instead, discrimination concentrates in positions combining high analytical and interpersonal demands with low routine content.
  • Decomposing task content into subjective-evaluation and objective-precision components: subjective evaluation widens callback gaps, objective precision compresses them.
  • Customer contact amplifies the divergence, widening gaps in non-routine jobs but not in routine ones.

Read those four together and the picture is unambiguous. The exposure is not evenly distributed across your hiring. It sits on the judgement-heavy, client-facing, non-routine roles โ€” which, in most mid-market companies, are exactly the roles where hiring managers most resist a rubric.

For a mid-market operation this is the practically important part, because job-family reporting is how most companies already look at their funnel. A single "Sales" or "Operations" category can hold a tightly specified analyst requisition and a loosely specified client-facing manager requisition, and the paper's result is that those two carry materially different exposure. Averaged together, they report as fine. The reporting layer most teams trust is the layer that hides the problem.

One caveat belongs with every citation of this work: a callback audit measures the first gate, not hires, and the paper is a working paper under review (v1 April 2026, revised 23 July 2026). The direction and the mechanism are well-identified. The magnitudes are not yet peer-reviewed.

Hiring Discrimination Is a Property of the Process, Not the Rรฉsumรฉ

The result in this paper that should actually redirect budget is the credential experiment. The authors randomly assigned rรฉsumรฉ credentials and found that credentials which increase callbacks on average reduce gaps in low-discretion jobs but not in high-discretion jobs.

Sit with that asymmetry. A candidate who acquires a verifiable credential improves their odds overall. But that improvement only narrows the demographic gap where the evaluation was already relatively objective. Where the decision is mostly judgement, a stronger signal gets filtered through the same discretion that produced the gap in the first place.

The operational translation: candidate-side signalling cannot fix an employer-side discretion problem. Only constraining the evaluation can.

This is why "raise the bar on credentials" and "widen the sourcing funnel" both underdeliver on exactly the roles you most want to fix. Both interventions push more signal into a gate that is not reading signal reliably.

The same mechanism shows up in signals nobody defends

Discretion does not only admit demographic bias. It admits anything. In roughly 11.66 million real hiring conversations, a one-hour reply delay cut a candidate's hire odds by 46 percent โ€” and the effect persisted among managers who explicitly stated they did not expect a reply within an hour (Cornell SC Johnson, 2026). Reply latency correlates with time zone, seniority and caregiving load. It predicts nothing about job performance.

A gate loose enough to score people on response speed is loose enough to score them on much worse. The two findings are the same finding viewed from different angles.

Does Moving the Decision to a Model Close the Gap?

The obvious inference โ€” hand the judgement-heavy screening to a model โ€” is half right, and the half that is wrong is expensive.

The strongest recent audit of model-side screening ran 30,384 rรฉsumรฉ evaluations across three models, of which 29,734 returned valid scores. Under a structured, reference-anchored prompt, pooled demographic effects were statistically indistinguishable from zero: gender d = โˆ’0.00 (p = 0.97), Blackโ€“White d = +0.03 (p = 0.29). The positive control, mid-level versus junior seniority, came in at d = 3.40 โ€” roughly two orders of magnitude larger than any demographic effect (Pavlopoulos, 2026).

That looks like a clean win until you read the prompt-sensitivity result in the same paper. On age, the confounded estimate was d = 0.28. Holding experience identical, a structured prompt collapsed it to d = 0.02 (p = 0.90) โ€” while a naturalistic, gut-feel prompt retained d = 0.41 (p = 0.03). Same model. Same rรฉsumรฉs. The difference was how tightly the instruction was written.

And the null does not survive realistic deployment conditions. Adding company names, culture descriptions lifted from public careers pages, and selective constraints such as "only accept candidates in the top 10%" induced racial and gender gaps of up to 12 percentage points in interview rates across six models. Simple anti-bias prompt instructions, which work in controlled evaluations, stopped working once that context was present โ€” and the models inferred demographics from cues as subtle as college affiliation, with the bias invisible in the chain-of-thought (Karvonen & Marks, 2025).

There is a procurement consequence in that gap. A vendor's fairness evidence is almost always generated under controlled conditions โ€” clean factorial designs, neutral prompts, no client context. Those are the conditions under which the demographic effects vanish. The conditions under which they reappear are the conditions you will actually deploy in: your company name, your culture language, your top-10% constraint. Ask any screening vendor whether their bias testing was run with realistic context and selective constraints present. If the answer is no, the test did not measure the system you are buying.

So the model evidence does not contradict the audit. It replicates it. In human panels and in language models alike, the variance lives in how constrained the evaluation is โ€” not in the evaluator's priors, and not in the evaluator's species. An unstructured model is a fast hiring manager with worse documentation. A structured model is a constrained gate that happens to scale.

What a Head of Operations Should Do This Quarter

Four moves, in order of leverage.

1. Rank your open roles by discretion, not by seniority or spend. For each requisition, estimate the share of the decision that rests on verifiable assessment versus impression. Analytical-plus-interpersonal, low-routine, customer-facing roles go at the top of the list. That ranking is your remediation queue, and it will not match your headcount plan.

2. Put the written rubric on the top of that list first, not on the easy roles. Fixed dimensions, criticality weights set before you see a single candidate, and a written reason recorded per dimension. The discipline that matters is not scoring โ€” it is that the weights cannot move after the fact to justify a preferred candidate.

3. Cap the unstructured portion explicitly. The audit's finding is that objective precision compresses gaps and subjective evaluation widens them, so the ratio between them is a control you can set. One worked example of that ratio being fixed rather than left implicit: a weight budget split in half, with one half computed from the role's stated requirements under criticality weighting โ€” a blocker requirement counting three times an important one and ten times a nice-to-have โ€” and the other half returned as four separate judgement subscores, each carrying its own written reason and labelled as judgement rather than folded into a single total (Scovai's scoring model). Labelled judgement can be audited, challenged and tracked over time. Judgement buried in one number cannot be any of those things.

4. Give below-threshold candidates a review route the hiring manager cannot close alone. Discretion problems are self-sealing: the person exercising the judgement is the person who decides whether the judgement needs review. Break that loop procedurally, or the rubric becomes decoration.

None of this requires a diversity programme, and none of it depends on persuading a hiring manager that bias exists. It is process engineering applied where the measurement says the exposure is.

The Decision in Front of You

Most companies allocate hiring structure the way they allocate everything else: evenly, or by headcount. This audit says that is the one allocation guaranteed to miss, because hiring discrimination concentrates precisely where structure is hardest to impose and most strongly resisted.

So pick one requisition this quarter โ€” your most judgement-heavy, most client-facing, most senior-sounding open role โ€” and write the rubric before you open the pipeline. Not because it is the easiest place to start. Because it is the only place the evidence says it will move the number.

Ready to go beyond the CV?

Scovai's AI-powered Talent Passport reveals what resumes can't: personality, potential, and true job fit.