The bias audit and its data

The bias audit is the condition on which every use of an automated employment decision tool in New York City rests. § 20-871(a)(1) makes use unlawful unless the tool "has been the subject of a bias audit conducted no more than one year prior to the use of such tool", and 6 RCNY §§ 5-301 and 5-302 say what the audit must calculate and what data it may rest on. MFF-27 carries the audit as MRF-477 and its data rules as MRF-478; both are backed by the new control MCF-684.

The core prohibition — § 20-871(a)

In the city, it shall be unlawful for an employer or an employment agency to use an automated employment decision tool to screen a candidate or employee for an employment decision unless: 1. Such tool has been the subject of a bias audit conducted no more than one year prior to the use of such tool; ...

The rules restate the clock from the other side (6 RCNY § 5-301(a)): an employer or employment agency may not use or continue to use an AEDT if more than one year has passed since the most recent bias audit of the AEDT. The window runs from the audit date to each use, not to the start of use. An audit that expires in the middle of a hiring cycle stops lawful use of the tool that day.

What a bias audit is

§ 20-870 defines a bias audit as "an impartial evaluation by an independent auditor", including at minimum "the testing of an automated employment decision tool to assess the tool’s disparate impact on persons of any component 1 category" reported under the federal EEO-1 framework. Two definitions from the rules carry the weight:

• Independent auditor (6 RCNY § 5-300): a person or group capable of exercising objective and impartial judgment on all issues within the scope of the audit. An auditor is not independent if they are or were involved in using, developing, or distributing the AEDT; if at any point during the audit they have an employment relationship with the deployer that seeks to use the tool or with a vendor that developed or distributes it; or if at any point during the audit they have a direct financial interest or a material indirect financial interest in either. DCWP keeps no list of approved auditors (FAQ IV).
• Category (6 RCNY § 5-300): any component 1 category required to be reported on the EEOC Employer Information Report EEO-1 (the rule reaches them through 42 U.S.C. § 2000e-8(c) as specified in 29 C.F.R. § 1602.7). The separate calculations for intersectional categories of sex, ethnicity, and race come from § 5-301(b)(3)(iii) and (c)(4)(iii), not from the definition.

DCWP's Statement of Basis and Purpose states that the required calculations are consistent with § 1607.4 of the EEOC Uniform Guidelines on Employee Selection Procedures (29 C.F.R. § 1607.4).

What the audit must calculate — 6 RCNY § 5-301

The calculation set depends on what the tool does with people. A tool can both select or classify and score: § 5-301(b) governs its selection or classification function and § 5-301(c) its scoring function. The rules do not separately specify how multiple distinct outputs within either function are treated; the control guidance asks the deployer to consider which simplified output actually drives the screening decision and whether the audit covers it.

Where the AEDT...	The audit must, at a minimum	Rule
selects candidates for employment or employees being considered for promotion to move forward, or classifies them into groups	calculate the selection rate for each category; calculate the impact ratio for each category; run both separately for sex categories, race/ethnicity categories, and intersectional categories of sex, ethnicity, and race; run them for each group if the tool classifies people into specified groups (for example, leadership styles); and indicate the number of individuals assessed that fall within an unknown category	§ 5-301(b)
scores candidates for employment or employees being considered for promotion	calculate the median score for the full sample of applicants; calculate the scoring rate for each category; calculate the impact ratio for each category; with the same category separation and the same unknown-category indication	§ 5-301(c)

The three rates are defined in 6 RCNY § 5-300:

• Selection rate: the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AEDT, computed by dividing the number in the category moving forward or assigned a classification by the total number in the category who applied for a position or were considered for promotion. The rules' own example: if 100 Hispanic women apply for a position and 40 are selected for an interview after use of an AEDT, the selection rate for Hispanic women is 40/100 or 40%.
• Scoring rate: the rate at which individuals in a category receive a score above the sample's median score, where the score has been calculated by an AEDT.
• Impact ratio: the selection rate for a category divided by the selection rate of the most selected category, or the scoring rate for a category divided by the scoring rate of the highest scoring category. The most selected or highest scoring category has an impact ratio of 1.00 whenever its rate is above zero, and every other category's ratio reads against it.

The adopted rules include full worked example tables for both a selection-rate audit and a scoring-rate audit, with sex, race/ethnicity, and intersectional categories and a note stating how many individuals fell within an unknown category. The audit must report the number of individuals omitted from the required calculations because they fall within an unknown category; the count is reported, not dropped.

The 2 percent exclusion — § 5-301(d)

An independent auditor may exclude a category that represents less than 2% of the data being used for the audit from the impact-ratio calculations only. Where a category is excluded, the summary of results must include the auditor's justification for the exclusion, as well as the number of applicants and the scoring rate or selection rate for the excluded category. The rule does not authorize dropping a category from the selection-rate or scoring-rate calculations altogether.

The data the audit may rest on — 6 RCNY § 5-302

A bias audit conducted pursuant to section 5-301 of this Chapter must use historical data of the AEDT. The historical data used to conduct a bias audit may be from one or more employers or employment agencies that use the AEDT. However, an individual employer or employment agency may rely on a bias audit of an AEDT that uses the historical data of other employers or employment agencies only in the following circumstances: if such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or if such employer or employment agency has never used the AEDT.

Historical data is data collected during an employer's or employment agency's use of an AEDT to assess candidates for employment or employees for promotion; test data is data used to conduct a bias audit that is not historical data (6 RCNY § 5-300). The rule then allows test data as a fallback (§ 5-302(b)): an employer or employment agency may rely on a bias audit that uses test data if insufficient historical data is available to conduct a statistically significant bias audit, and the summary of results must then explain why historical data was not used and describe how the test data used was generated and obtained.

The rules' three worked examples set the pattern:

Situation	May rely on
First-time user of the tool	An audit using other employers' or employment agencies' historical data, or an audit using test data
Six-month user whose own data is not sufficient for a statistically significant audit	An audit using pooled historical data, only if it provides its six months of data to the auditor; or an audit using test data
Three-year user with statistically significant data of its own	An audit using pooled historical data if it contributes its three years of data, or an audit using its own historical data alone; may no longer rely on test data

Four DCWP FAQ points sit alongside the rule:

• No imputation (FAQ III.6): imputed demographic data, or demographic attributes inferred by algorithmic software, cannot be used to conduct a bias audit.
• Statistical significance is the auditor's judgment (FAQ III.7): DCWP has set no specific significance requirement; the independent auditor determines whether the historical data suffices.
• No test-data specification (FAQ III.8): DCWP has deliberately set no requirements for test data, so that best practices can develop; the transparency in the published summary is the safeguard.
• Limited data should be explained (FAQ III.3): if the historical data was limited in any way, for instance to a region or a time period, DCWP says the audit should explain why. The binding duty is the published summary's source-and-explanation-of-data element (6 RCNY § 5-303(a)(1)); the limitation-specific explanation is DCWP's guidance on satisfying it. There is no requirement that employers contributing historical data used the tool for the same type of position (FAQ III.4).

What the results oblige you to do

Local Law 144 prescribes no remediation based on the audit results (FAQ II.2); what it requires is the audit itself and, under § 20-871(a)(2) and 6 RCNY § 5-303, the publication of its date and summary. Federal, state, and New York City anti-discrimination laws apply independently (FAQ II.2), and DCWP refers discrimination claims to the New York City Commission on Human Rights (FAQ VII.2). DCWP's Statement of Basis and Purpose says the required calculations are consistent with 29 C.F.R. § 1607.4; the four-fifths (80 percent) benchmark of that guideline is a federal selection-rate reference point, subject to the guideline's own statistical- and practical-significance caveats, and says nothing about the scoring-rate calculation. Local Law 144 sets no threshold of its own, and an impact ratio below 0.80 is not a Local Law 144 violation. Whether to act on an adverse ratio is a question under Title VII, the New York State Human Rights Law, and the New York City Human Rights Law, which apply independently.

Vendor audits and shared audits

A vendor may have an independent auditor audit its tool and coordinate data collection (FAQ V.1–V.2), and multiple employers may rely on the same audit under the pooled-data conditions of 6 RCNY § 5-302(a) (FAQ III.2, III.4). The deployer remains responsible for ensuring that a compliant audit exists before use. Where a deployer relies on a pooled audit and has used the tool before, it must be able to show it contributed its own usage data to the auditor.

How this maps in Modulos — MRF-477, MRF-478

MRF-477 (annual independent bias audit) requires the tool to have been the subject of a bias audit by an independent auditor no more than one year before each use, with the calculation set above, the category separation, the unknown-category count, and the justification of any category excluded from the impact-ratio calculations. MRF-478 (bias audit data requirements) requires the audit to rest on the right data: historical data by default, pooled data only under the rule's conditions, test data only where insufficient historical data exists for a statistically significant audit, and no imputed or inferred demographics.

Both are carried by the new control MCF-684 (Independent AEDT bias audit): the tool-specific duty of ensuring and relying on an annual audit by an independent auditor with this exact calculation set and data basis, whether the deployer commissions it or the vendor arranges it. Its evidence expectations are the audit report with the full category tables, the audit date tied to the period of use it covers, the auditor's written independence attestation, the audit's data-provenance section, the unknown-category count and any exclusion justifications, and, for pooled audits, the record that the deployer contributed its own data or had never used the tool.

Three reused controls support the audit without carrying any Local Law 144-specific wording:

• MCF-42 (Model Fairness Metrics) and MCF-43 (Model Bias Assessment) carry the deployer's own practice of defining fairness metrics and assessing model bias for protected groups. The independent audit complements that practice; it does not replace it, and adverse ratios route into these controls for whatever follow-up the deployer decides under other law.
• MCF-32 (Data Bias Assessment) carries the representativeness judgment behind the audit's data sufficiency.

Where to go next

• Publication and notices — where the audit summary is posted, and what candidates and employees are told.
• Operationalizing Local Law 144 in Modulos — the full MFF-27 rollout with the mapping table.

Disclaimer

This page is for general informational purposes and does not constitute legal advice. Always verify against the current published text of N.Y.C. Administrative Code §§ 20-870–874 and 6 RCNY §§ 5-300–5-304 and consult qualified advisers.