Protection against “algorithmic discrimination” is front and center in our federal government’s current framework for AI.¹ As noted in my Visual Guide to the U.S. Federal Regulation of AI, there’s seemingly no significant federal agency that does not have a regulatory dog in this fight.

Of all the intractable issues that AI gives rise to, algorithmic discrimination is in my view the most unsolvable. It resides at the convergence of:

1. civil rights and race, gender, and other relations—the biggest political football(s) of our times;² and
2. how AI works—e.g., it is trained on data that itself is biased.

Any hope for “solving” the latter would be through the application of data science and statistics. Which is made entirely impossible by the political forces at play.

Thankfully cutting this algorithmic Gordian knot is not the bar that companies providing or implementing generative AI services need clear. Practically speaking, all you will need to do is comply with the applicable laws and regulations to come.

This article focuses on one of the most important manifestations of algorithmic discrimination—the use of AI in automated employment decision tools

II. The law and regulation on disparate impact

All efforts to regulate against algorithmic discrimination stem from the disparate impact theory of discrimination. Congress, the courts, and regulatory agencies—namely the Equal Employment Opportunity Commission (established by Congress to enforce Title VII of the 1964 Civil Rights Act)—have incorporated disparate impact concepts into various laws and regulations.

EEOC laws cover most employers with at least 15 employees.³

A. The traditional disparate impact analysis

Disparate impact cases typically involve the following questions, based on the Supreme Court’s seminal opinion in Griggs v. Duke Power Co.:

• Does the employer use a particular employment practice that has a disparate impact on the basis of race, color, religion, sex, or national origin?
• If the selection procedure has a disparate impact based on race, color, religion, sex, or national origin, can the employer show that the selection procedure is job-related and consistent with business necessity?
• If the employer shows that the selection procedure is job-related and consistent with business necessity, is there a less discriminatory alternative available?⁴

B. The application of disparate impact to automated employment decision tools using AI

In May 2023, the EEOC published updated guidance on automated employment decision tools using AI.⁵ The EEOC guidance specifies that employers that use AI hiring decision tools can be liable for disparate impact violations, even if the tools are developed by third party vendors.⁶

The EEOC further notes: “One advantage of algorithmic decision-making tools is that the process of developing the tool may itself produce a variety of comparably effective alternative algorithms.”

And concludes: “The EEOC encourages employers to conduct self-analyses on an ongoing basis to determine whether their employment practices have a disproportionately large negative effect on a basis prohibited under Title VII or treat protected groups differently. Generally, employers can proactively change the practice going forward.”⁷

The EEOC’s analysis does not entail opening up of the black box of these AI employment decision tools. It does not directly assess how they work. It only looks solely to and proscribes any disparate impact in AI output.

This is both logically and deeply flawed. The old adage “the devil is in the details” comes to mind. As does “when you have a hammer, an awful lot of things start looking like a nail….”⁸

C. The NYC law on automated employment decision tools

The first proving ground for this modern version of MLK’s ideal arises not out of the federal or state levels. New York City’s Local Law 144 on automated employment decision tools was enacted in December 2021, with enforcement starting this past July.

It requires all NYC employers to disclose and audit any use and output of AI-based automated employment decision tools,⁹ echoing the EEOC’s May 2023 AI-output-focused guidance.

The apparent rationale is that “if employers are forced to measure and disclose the fairness of their algorithmic hiring and promotion systems, they will be incentivized to avoid building or buying biased systems.”¹⁰

The law is subject to numerous critiques.¹¹ This is no surprise and not only because of the politics involved. The issues are impossibly complex, starting with the technological equivalent of original sin embedded in AI itself.

III. The problems with algorithmic discrimination bias audits

Algorithmic discrimination will not be a stand-alone or primary claim in most cases. It will be a secondary claim, delved into primarily to support a broader claim of actual employment discrimination.

The EEOC is authorized to independently investigate possible discrimination (presumably now also including the algorithmic kind) under Title VII. But for anyone but for the really big fish, the most likely way an algorithmic discrimination bias audit might come up against your average company is if a former employee files:

• a federal claim of discrimination with the EEOC against you, and/or
• a (subsequent) federal private job discrimination lawsuit against your employer,^{12 13}

The problem of discrimination, algorithmic or otherwise, cannot be solved, by regulation or otherwise. It can only be mitigated against, and only in ways that will always be subject to fair criticism and then some.¹⁴ It is part and parcel of the broader Sisyphean struggle against discrimination itself.

There are a myriad reasons, starting with the two outlined below.

A. Bias in, bias out

AI replicates the human mind and analyzes data by classifying it. It does so but identifying similarities and differences between data, i.e., by discriminating.

If there is an overrepresentation of one group over another for a given factor that is deemed important for hiring decision-making in the real world, then this bias will be reflected in the data. And the use of such biased data to train the model will result in model outputs that also reflect such bias.

The main way to mitigate against bias in data is through the application of data science in the collection of data or the application of data science/statistics to “remove” bias during the processing of the data.

If such efforts to “de-discriminate” are not, however, standardized, then the outputs of individual efforts cannot be meaningfully compared.¹⁵

And the underlying politics will inevitably shed far more heat than light on the subject (presuming they let through any light to begin with).

B. And what exactly are we supposed to do with the results…?

The elephant in the room is: How much disparate impact is too much disparate impact? And worse, how much is acceptable?

The EEOC’s May 2023 Select Issues document provides guidance-less guidance on the “four-fifths rule,” a general rule of thumb for determining whether the selection rate for one group is “substantially” different than the selection rate for another group.¹⁶ It is a “practical and easy-to-administer” test that may be used to draw an initial inference that the selection rates for two groups may be substantially different, and to prompt employers to acquire additional information about the procedure in question.”¹⁷ The four-fifths rule, however, “is not always appropriate, especially where it is not a reasonable substitute for a test of statistical significance.”¹⁸ It further punts on anything else on this issue.¹⁹