Extended definition
An AI bias audit looks at what an automated hiring tool actually did, not what it was designed to do. Someone takes the records of who the tool advanced, rejected or scored, splits them by demographic group, and checks whether some groups came out worse. It’s the question you’d ask of any selection step: does this stage create adverse impact?
Anyone can run an informal version of that check. In New York City, though, the term has a legal definition, and that law is what most people searching it mean. Local Law 144 bars employers and employment agencies from using an automated employment decision tool (AEDT) unless an independent bias audit has been done within the past year, a summary of the results is public, and candidates have been told.
An audit measures outcomes. Where a skew comes from, whether training data or the features a model weighs, is the wider subject of algorithmic bias. The audit is the counting; that term is the diagnosis.
How an AI bias audit works
Under New York City’s rules, a bias audit must at minimum:
- Calculate selection rates — If 100 applicants from one group apply and the tool advances 40, that group’s selection rate is 40%.
- Calculate impact ratios — Divide each group’s selection rate by the rate of the most-selected group. Tools that score rather than select use a scoring rate instead: the share of each group scoring above the median.
- Cover three sets of categories — Sex, race/ethnicity, and intersectional groups such as Hispanic or Latino women.
- Count the unknowns — Report how many people the tool assessed whose demographic category isn’t known.
The impact ratio is the same comparison behind the four-fifths rule, so the arithmetic will look familiar if your team already runs adverse-impact analysis. What Local Law 144 doesn’t do is set a pass mark, or require any particular action based on the results. Federal, state and city anti-discrimination laws still apply, and that’s the part to keep in view.
Four more rules surround the audit in New York City:
- Who conducts it — An independent auditor. That excludes anyone involved in using, developing or distributing the tool, anyone employed by the employer or vendor during the audit, and anyone with a direct or material indirect financial interest in either. There’s no approved list.
- How often — Every year. An audit is valid for one year from the date it was conducted.
- Publishing — Before using the tool, the employer posts a summary of the results and the tool’s distribution date on the employment section of its website.
- Candidate notice — Candidates who live in the city get notice at least 10 business days before the tool is used, with instructions for requesting an alternative selection process or a reasonable accommodation.
Why AI bias audits matter
A tool applies the same logic to every applicant, so a small skew doesn’t stay small. It repeats across every requisition the tool touches, and nobody sees it in a single hiring decision. The audit is the moment someone finally counts.
It also changes the conversation with vendors. In New York City the obligation sits with the employer, not the vendor, though a vendor can commission an independent audit of its own tool. So ask for the latest audit summary, what data sat behind it, and whether your own data was included. A vendor who can’t answer those three questions quickly is telling you something.
The same questions apply to any tool that assesses candidates directly, such as an AI interviewer, and to an AI recruiting agent that screens as one step among many. Request the audit summary during procurement, and look at it again at renewal.
Common mistakes and misconceptions about AI bias audits
- Treating a passed audit as proof of fairness — An audit covers only the groups and data it used. New York City’s required categories don’t include age or disability, and an auditor can leave out any category under 2% of the data.
- Assuming only final decisions count — Screening is an employment decision under the city’s rules. A resume screener that only decides who gets an interview is covered.
- Auditing once — The annual requirement exists for a reason. Models get retrained and applicant pools change, so results drift between audits.
- Filling gaps with inferred demographics — The city’s rules don’t allow imputed or inferred demographic data. Where historical data is too thin, test data can be used if the summary explains why.
Frequently asked questions
What is an AI bias audit?
An AI bias audit measures whether an automated hiring tool treats demographic groups differently in practice, whatever it was designed to do. The auditor takes data on who the tool advanced or scored, works out a selection or scoring rate for each group, and divides each rate by the most-selected group's rate to get an impact ratio. Some places require one. In New York City, Local Law 144 requires an independent bias audit of covered tools every year.
What does NYC Local Law 144 require of AI hiring tools?
It requires employers and employment agencies using an automated employment decision tool in New York City to have had an independent bias audit within the past year, to publish a summary of the results, and to notify candidates who live in the city at least 10 business days before the tool is used. It applies to tools that substantially assist or replace decision-making, including screening. Outreach to people who haven't applied for a role isn't covered.
How is an impact ratio calculated in a bias audit?
Divide a group's selection rate by the selection rate of the most-selected group. If 50% of men and 40% of women advance, the impact ratio for women is 0.40 divided by 0.50, or 0.80. For tools that score rather than select, use the share of each group scoring above the median instead. In US adverse-impact analysis, a ratio below 0.80 is where the four-fifths rule says to look closer. It's a signal, not a verdict.
Who can conduct a bias audit?
Under New York City's rules, only an independent auditor: someone able to exercise objective and impartial judgment. That rules out anyone who helped use, develop or distribute the tool, anyone employed by the employer or the vendor during the audit, and anyone with a direct or material indirect financial interest in either. There's no approved list of auditors. A vendor can commission an audit of its own tool, but the employer still has to confirm the audit exists and is current.
How often does an AI hiring tool need a bias audit?
In New York City, every year. An employer can rely on a bias audit for one year from the date it was conducted, and can't use or keep using a covered tool once more than a year has passed since the latest one. Outside any legal requirement, the sensible habit is to re-test whenever the vendor retrains the model, and whenever you point the tool at a new type of role. The applicant pool changes, so the results can too.