One vendor screened millions of applicants for an entire sector, and a Stanford-led team found clear racial disparities the vendor’s own testing had masked.
Researchers analyzed more than four million job applications across 156 employers, all screened by a single talent-assessment vendor. They found clear racial disparities in outcomes. The vendor had measured its own fairness by pooling every applicant across all employers and positions together, which hid disparities that surfaced only when each of the 1,746 positions was analyzed separately, the way anti-discrimination law actually requires. When one vendor’s model dominates a sector, its blind spots become the whole sector’s blind spots.
analyzed
vendor
affected
A vendor grading its own fairness in aggregate is the monoculture risk in miniature. AVAAS evaluates each deployment independently, with causal attribution applied per position rather than pooled, surfacing the disparate impact a vendor’s own averaged self-assessment is structurally unable to see.
This entry is one of 37 documented cases in the AVAAS evidence ledger, a public record of AI and automated-system failures with a verified source on every entry.
Every case here reached a person.
AVAAS certifies how AI systems behave at the decision point, with documented third-party evidence of conformity to a published standard.
Certify Your AI →