What Is a New York City AI Bias Audit and When Do Employers Need One?

What a New York City AI bias audit actually is

A New York City AI bias audit is a documented examination of an automated employment decision tool used to assist or make hiring, promotion, termination, compensation, or other employment decisions. Local Law 144 of 2021, administered by the New York City Department of Consumer and Worker Protection, became effective on January 1, 2023, with enforcement beginning on July 5, 2023. It does not prohibit employers from using artificial intelligence in employment. Instead, it requires covered employers to test covered tools for discrimination, publish required information, give candidates advance notice, and provide a way to request information about the tool’s results.

Also worth reading: How Should Employers Run an HR Compliance Risk Audit in 2026? · AI Hiring Audit Checklist for 2026: How Should Employers Test Automated Hiring Decisions? · How Do Employers Audit AI in Employment in 2026?

The audit is not simply a general test of software quality, cybersecurity, or whether an algorithm contains offensive language. Its central purpose is to test whether the tool produces different results for protected groups without a job-related business justification. The comparison may examine selection or placement rates, ranks or scores, and other employment-related outputs. The familiar four-fifths rule—often expressed as an 80 percent ratio—is relevant to the analysis, but it is not a universal safe harbor. A difference below 80 percent generally requires explanation, while a difference above 80 percent can still require investigation depending on the number of observations, the size of the group, statistical uncertainty, and whether the tool creates a lawful employment practice.

New York City rules define an automated employment decision tool broadly enough to include many algorithmic screening systems, not only systems marketed as generative AI. A vendor’s description of a product as a recruiting assistant, interview guide, candidate-ranking system, résumé evaluator, or hiring prediction model does not determine whether it is covered. Employers should focus on what the system does in the employment process. Ordinary search, scheduling, and communications tools can fall outside the law when they do not assist or replace human judgment regarding the listed employment decisions. A narrow legal classification should nevertheless be supported by technical evidence rather than a product label.

Who must complete an audit under Local Law 144?

The core obligation generally applies to an employer, employment agency, or labor union that hires or employs workers in New York City and uses an automated employment decision tool for a covered purpose. A contracting vendor does not automatically take over the employer’s responsibilities. If the employer selects a tool to rank applicants, it should assume that a role in the audit and compliance process may be necessary even when the software, data, and model development are handled by a third party. Contract language should identify who supplies records, performance data, methodology, auditor access, and updates required after a model change.

The audit must be completed independently and at least once per year. A company employee who built or configured the system ordinarily cannot serve as the independent auditor. A qualified outside auditor can be a person, firm, or organization with the relevant expertise and sufficient independence. Some larger organizations may operate a properly structured internal audit function that satisfies the independence requirement, but having a separate team within the same business is not automatically enough. The agency’s rules address this independence question, so employers should document why the reviewer has no conflict of interest and why the audit can be considered objective.

The law also contains a small-employer exception. Covered employers with fewer than 20 employees in New York City are not required to conduct bias audits, although other obligations—including candidate notice and the right to request information—can still apply. Employers should be cautious with freelancers, subsidiaries, temporary workers, and staffing agencies. A company may have fewer than 20 direct employees while using hundreds of workers through a third party. The number of people used to calculate the exception and the entity operating the employment relationship must be evaluated carefully rather than inferred from the vendor contract alone.

Required audit content and employer evidence

A defensible audit is more than a vendor-generated PDF that reports a handful of fairness metrics. The auditor should understand the intended purpose of the tool, the populations in which it will be used, the decisions it influences, the data it receives, and the way its outputs affect people. The work should connect statistical results to New York City’s rules and applicable federal or state anti-discrimination law. It should also evaluate the agency’s own test design, including its choice of comparison metrics, subgroup definitions, minimum sample sizes, treatment of small populations, and treatment of unavailable demographic data.

The audit should evaluate historical employment data, test data, and operational data as available and appropriate. Historical data is necessary to understand how the tool performs in the employer’s actual context, while test data can help identify performance differences before deployment. Operational data can show whether outcomes differ from controlled testing, especially when the vendor changes the model, the employer changes thresholds, or certain applicant groups interact with the process differently. An audit based only on a generic demonstration dataset may not answer whether the system performs consistently in the employer’s recruiting environment.

The final report should include the auditor’s identity, the date of the audit, the tool and version tested, the test period, the categories of protected characteristics examined, the metrics and statistical methods used, the number of observations, relevant findings, and any limitation on the conclusions. A material change in the tool can create a need for additional testing, even before the next annual review. Employers should retain the report, supporting calculations, data dictionaries, auditor communications, remediation decisions, and evidence of corrective action. The audit results are not generally required to be posted publicly. A candidate may request a summary, and employers must respond to qualifying requests rather than treating an audit as a private exercise with no external accountability.

The audit does not certify that the tool is unbiased. No test can prove the absence of discrimination in every future case, particularly when sample sizes are small or outcomes depend on multiple systems. Its value is to establish whether the employer has tested the tool, considered group-level differences, identified limitations, and taken reasonable action when a problem appears. A report with dozens of favorable metrics but no discussion of data quality, uncertainty, job relevance, or corrective measures is weak evidence of a meaningful audit.

When an employer should act

An employer using a covered system in New York City should not wait for an agency investigation to begin the process. The sensible first review is whether the organization knows exactly which technologies influence employment decisions. Many companies use several tools that were never inventoried, including résumé screening, interview recording and transcription, candidate scoring, job matching, promotion recommendations, and internal mobility systems. Each system should be mapped to its owner, vendor, purpose, data sources, decision point, and level of human involvement. Products that merely summarize an interview may still affect a human decision even if they do not automatically reject an applicant.

A pre-deployment review is important because a new tool may process applicant data before the organization has established that a legal basis exists for that use. The employer should obtain the model card, intended-use statement, validation results, subgroup performance data, change history, and security documentation. The contract should state that the vendor will provide information reasonably needed for testing and compliance, disclose material model changes, and cooperate with an independent auditor. It should also address responsibility for correcting discriminatory outcomes rather than promising only a new software feature that does not change the underlying results.

Organizations should reassess the system when an acquisition, vendor replacement, algorithm update, score recalibration, or substantial shift in recruiting occurs. A change in a pass threshold can be as consequential as rewriting the model, while a new source of demographic data can change who is screened out even when the architecture remains unchanged. If an audit identifies a disadvantaged result, the employer should determine whether the difference is caused by a job-related factor, measurement error, proxy behavior, data limitations, or an actual exclusion. Remediating the process is not always legally equivalent to switching the vendor; the appropriate action depends on the tool, the job, and the affected candidates.

The timing question is especially important for smaller organizations. They may not face an annual audit obligation if they meet the fewer-than-20-employee exception, but they can still face discrimination claims, negligence claims, contract issues, or regulator questions about their tool. A documented review provides a defensible record even when it is not mandatory. Large employers and staffing firms have a stronger reason to create a recurring calendar that aligns vendor data delivery, independent review, management approval, and candidate-facing controls.

Practical steps for building a defensible process

Begin with a written inventory of employment technologies and distinguish tools that make or materially assist covered decisions from tools used only for administrative tasks. For every relevant product, collect the vendor agreement, product documentation, model version, score definition, intended purpose, feature list, and contact for incidents. Confirm whether the supplier’s terms prohibit the employer from sharing the information needed for an independent audit. If they do, legal review should occur before purchase, not after the system is embedded in a hiring workflow.

Next, identify the lawful purpose and job process. HR, recruiting, legal, data science, and the business owner should document whether a score is advisory, what a human decision-maker sees, how that person is expected to use it, and whether applicants can request an alternative process. A human-in-the-loop label does not automatically resolve the legal question. If the reviewer cannot reasonably disregard an automated recommendation, the system may still be functioning as a decision tool in substance. Candidate notice should explain the use of automation in clear language and provide the legally required alternative or process for submitting a résumé and applying for the role without the tool.

Data preparation deserves separate attention. Applicants may decline to provide race, sex, or disability information because voluntary self-identification in high-stakes employment can feel risky. An employer should use a legally appropriate collection method and protect the identity of respondents. It should not infer protected status through unsupported assumptions or use a proxy category as a substitute for properly obtained demographic data. Before calculating impact ratios, the auditor should explain how records with multiple selections, missing values, withdrawn applications, and differing job-related outcomes will be counted.

The organization should then select an independent auditor using technical, legal, and operational criteria. The auditor needs competence in employment discrimination, statistical analysis, and the specific type of system being tested, not merely access to an automated fairness dashboard. The final report should include limitations and recommended corrective measures. Management should assign an owner and deadline for each measure, retest after material remediation, and preserve evidence that the system was used consistently with the audit findings.

Comparing audit approaches

The requirement for an independent audit does not necessarily require the employer to buy a large suite of software. The right comparison is among methods that can produce credible evidence, depending on the employer’s size, data maturity, risk, and technical capability. Cost figures below are planning estimates rather than regulated fees; a quote can vary substantially based on model complexity, data quality, subgroup sample sizes, audit depth, and the number of tools involved.

A low-cost spreadsheet can help a small employer organize a limited review, but it should not be represented as an independent bias audit when the person preparing it built, selected, configured, or operates the tool. Generic bias-detection tools, including open-source projects and vendor products, can calculate useful metrics, but they cannot decide whether a metric is legally appropriate without understanding the hiring process. Similarly, buying compliance software is not a substitute for deciding which system is covered, what data is available, what the score means, and what the organization will do about a disparity.

Common mistakes and enforcement risk

A frequent mistake is treating “AI” as the only issue and overlooking ordinary algorithmic or rule-based tools. A system that ranks candidates based on features and historical data may be covered even if its vendor avoids the term artificial intelligence. Another mistake is assuming that a vendor’s statement that its product is “fair” resolves the employer’s responsibility. Product-wide fairness claims do not establish performance in a particular employer’s job, with its applicant mix, data quality, threshold settings, and decision process.

Organizations also make the mistake of using selection-rate ratios without investigating the larger decision process. New York City and federal enforcement are not limited to a single statistical threshold. A result under the 80 percent ratio may indicate a lawful reason to investigate, but a result above it does not automatically end the inquiry when statistical or practical concerns remain. Small samples can produce unstable ratios, and a tool may perform differently across combinations of race, sex, age, disability, or other relevant characteristics rather than only across one broad category at a time.

A major operational error is publishing a vague privacy notice instead of giving candidates the required employment-tool notice. Candidates need to know that an automated tool has been used for a relevant employment decision, what types of data and criteria the employer may use, and how to request an alternative process or additional information where required. The notice should be available before the candidate engages with the covered process and should avoid technical claims that obscure how the candidate’s information is used. Employers should not use a notice to discourage a lawful alternative, make unsupported promises about individual outcomes, or imply that the audit guarantees fairness.

The last common error is treating an audit as a one-time checkbox. The law requires an ongoing operational relationship between the report and the actual system. A vendor update, changed threshold, altered data pipeline, or new business unit can invalidate assumptions made during the last review. Employers should also investigate complaints, adverse-impact signals, drop-off rates, and deviations from validated use. A process that technically produces annual reports but ignores repeated warnings is unlikely to protect the organization from civil-rights exposure or agency scrutiny.

How audit results fit into broader HR compliance

An NYC AI bias audit is one part of a larger employment-compliance system. It should operate alongside the employer’s obligations under federal and New York anti-discrimination laws, privacy and data-security requirements, notice rules, records-retention practices, and any obligations related to applicants, employees, contractors, or vendors. The audit’s statistical conclusion is not a legal opinion on whether every employment decision was lawful. It is evidence about a tool’s performance that legal and HR teams can use in assessing risk and improving the process.

For an AI-powered labor-law compliance and HR regulatory-management program, the useful unit of control is not a model alone but a connected record covering the tool, job, data, decision-maker, candidate notice, audit, remediation, and approval history. That record allows a company to demonstrate that a recommendation was evaluated for a legitimate employment purpose, that protected-group outcomes were considered, and that corrective action was taken when needed. It can also make a review easier when a regulator, litigant, candidate, or internal audit team asks for evidence.

The operational cost should therefore be compared with the cost of the tool itself. A resume-ranking platform may cost little per month but create substantial legal, recruiting, and reputational exposure if its use is undocumented. A more expensive system with reliable data, configurable controls, audit access, and change notifications may have a higher total cost of ownership but can reduce the burden of reconstructing decisions. The most useful buying decision is based on documented capabilities, not a promise of automated compliance.

As of September 30, 2026, employers should treat Local Law 144 as part of the existing New York City employment environment, while separately monitoring proposed federal, state, and local changes because litigation, agency guidance, and enforcement practice can evolve. The practical standard is to verify the current rule text and agency materials rather than rely on a vendor blog. A compliance program that can identify the covered tool, produce a credible independent audit, provide candidate information, and respond to a disparity is better prepared than one that merely labels itself “AI compliant.”