The bias audit and its data

Local Law 144 of New York City gives rules for automated employment decision tools (AEDTs). In the city, an employer or employment agency must not use an AEDT to screen a candidate or employee for an employment decision, unless all of these conditions apply (Administrative Code § 20-871(a)):

• Bias audit (§ 20-871(a)(1)): an independent auditor did a bias audit of the AEDT not more than one year before the use.
• Results on the website (§ 20-871(a)(2)): before the use, these items are publicly available on the website of the employer or employment agency:
  • A summary of the results of the most recent bias audit.
  • The distribution date of the AEDT.

6 RCNY § 5-303(a) adds the date of the most recent bias audit to these items. This page gives the bias audit and the rules of the Department of Consumer and Worker Protection (DCWP) for it (6 RCNY §§ 5-300 to 5-302). The page Publication and notices gives the results on the website. In Modulos, the MFF-27 Requirements MRF-477 and MRF-478 hold the bias audit.

The prohibition — § 20-871(a)

Administrative Code § 20-871(a) of Local Law 144 gives this rule for the employer or employment agency:

In the city, it shall be unlawful for an employer or an employment agency to use an automated employment decision tool to screen a candidate or employee for an employment decision unless: 1. Such tool has been the subject of a bias audit conducted no more than one year prior to the use of such tool; ...

6 RCNY § 5-301(a) of the DCWP rules also gives this limit. In this rule, an employer or employment agency "may not use or continue to use an AEDT if more than one year has passed since the most recent bias audit of the AEDT". The limit of one year applies to each use of the AEDT, not only to the first use:

• The date of a bias audit must be not more than one year before each use.
• A bias audit that an independent auditor does after a use does not apply to that use.
• After one year from the most recent bias audit, the employer or employment agency must have a new bias audit before it continues to use the AEDT.
• The limit also applies during a hiring process. When the year after the most recent bias audit ends, the employer or employment agency must stop the use of the AEDT until it has a new bias audit.

The definition of a bias audit

Local Law 144 gives the definition of a bias audit in Administrative Code § 20-870:

Bias audit. The term “bias audit” means an impartial evaluation by an independent auditor. Such bias audit shall include but not be limited to the testing of an automated employment decision tool to assess the tool’s disparate impact on persons of any component 1 category required to be reported by employers pursuant to subsection (c) of section 2000e-8 of title 42 of the United States code as specified in part 1602.7 of title 29 of the code of federal regulations.

Thus the bias audit is an impartial evaluation by an independent auditor. As a minimum, it must include a test of the AEDT for disparate impact on persons of each component 1 category. The DCWP rules in 6 RCNY § 5-300 give the definitions of two items that this definition uses, independent auditor and category:

• Independent auditor: a person or group that is "capable of exercising objective and impartial judgment on all issues within the scope of a bias audit of an AEDT". An auditor is not an independent auditor if one or more of these conditions apply:
  • The auditor "is or was involved in using, developing, or distributing the AEDT".
  • "At any point during the bias audit", the auditor has an employment relationship of one of these two types:
    • With an employer or employment agency that "seeks to use or continue to use the AEDT".
    • With a vendor that developed or distributes the AEDT.
  • "At any point during the bias audit", the auditor has a direct financial interest or a material indirect financial interest of one of these two types:
    • In an employer or employment agency that "seeks to use or continue to use the AEDT".
    • In a vendor that developed or distributed the AEDT.
• Category: a component 1 category that employers must report (42 U.S.C. § 2000e-8(c) and 29 C.F.R. § 1602.7). The categories are the ones on the Employer Information Report EEO-1 of the Equal Employment Opportunity Commission (EEOC). The rules in 6 RCNY § 5-301(b)(3)(iii) and (c)(4)(iii), not this definition, make the calculations for the intersectional categories of sex, ethnicity, and race necessary.

In DCWP FAQ IV.1, DCWP tells employers and employment agencies that it has no "list of approved independent auditors". It also gives this text: "The Law does not require independent auditors to be approved by DCWP." The DCWP FAQ of June 29, 2023 gives "general information and guidance", and it is not law.

In the Statement of Basis and Purpose of its rules, DCWP gives the selection-rate calculations. DCWP tells employers and employment agencies that a bias audit of an AEDT must calculate the selection rate for each race/ethnicity and sex category. DCWP also tells employers and employment agencies that the bias audit must compare these selection rates with the most selected category, to get an impact ratio. Then DCWP gives this text: "These calculations are consistent with Section 1607.4 of the EEOC Uniform Guidelines on Employee Selection Procedures." This text is about the selection-rate calculations only. It does not refer to the scoring-rate calculations.

The calculations of the bias audit — 6 RCNY § 5-301

The DCWP rule 6 RCNY § 5-301 gives the minimum calculations of the bias audit. The calculations are different for each function of the AEDT:

If the AEDT	The bias audit must, as a minimum	Rule
selects candidates for employment or employees being considered for promotion to move forward in the hiring process, or classifies them into groups	Calculate the selection rate for each category. Calculate the impact ratio for each category. Do these calculations "separately" for sex categories, race/ethnicity categories, and intersectional categories of sex, ethnicity, and race. If the AEDT classifies candidates or employees into specified groups (for example, "leadership styles"), do these calculations for each group. Give the number of individuals that the AEDT assessed and that are in an unknown category. The calculations do not include these individuals.	§ 5-301(b)
scores candidates for employment or employees being considered for promotion	Calculate the median score for the full sample of applicants. Calculate the scoring rate for individuals in each category. Calculate the impact ratio for each category. Do these calculations "separately" for sex categories, race/ethnicity categories, and intersectional categories of sex, ethnicity, and race. Give the number of individuals that the AEDT assessed and that are in an unknown category. The calculations do not include these individuals.	§ 5-301(c)

§ 5-301(b) and § 5-301(c) each apply when the AEDT does the function that they give. Thus, if the AEDT selects or classifies and also scores, the bias audit must include the two sets of calculations.

6 RCNY § 5-300 gives the definitions of the rates and of the impact ratio:

• Selection rate: the rate at which individuals in a category are selected to move forward in the hiring process, or are classified by an AEDT. You can calculate the selection rate from two numbers:

  • A: the number of individuals in the category who move forward or get a classification.
  • B: the total number of individuals in the category who applied for a position or were considered for promotion.

  The selection rate can be A divided by B. The rule gives this example: "If 100 Hispanic women apply for a position and 40 are selected for an interview after use of an AEDT, the selection rate for Hispanic women is 40/100 or 40%."

• Scoring rate: the rate at which individuals in a category get a score above the median score of the sample. The definition applies where an AEDT calculated the score.

• Impact ratio: one of these two values:

  • The selection rate for a category divided by the selection rate of the most selected category.
  • The scoring rate for a category divided by the scoring rate of the highest scoring category.

  Thus the most selected or highest scoring category has an impact ratio of 1.00 if its rate is more than zero.

The adopted rules contain two examples with full tables. One example is for a bias audit with selection rates, and the other is for a bias audit with scoring rates. Each example shows sex, race/ethnicity, and intersectional categories. Below the tables, each example gives the number of individuals in an unknown category. The calculations do not include these individuals.

The 2 percent exclusion — § 5-301(d)

In 6 RCNY § 5-301(d), an independent auditor is permitted to exclude a category from the impact-ratio calculations if the category is less than 2% of the data used for the bias audit. If the independent auditor excludes a category, the summary of results must include these items:

• The justification of the independent auditor for the exclusion.
• The number of applicants in the excluded category.
• The scoring rate or selection rate of the excluded category.

The exclusion applies to the impact-ratio calculations only. It does not let the independent auditor remove a category from the selection-rate or scoring-rate calculations.

The data for the bias audit — 6 RCNY § 5-302

The DCWP rule 6 RCNY § 5-302(a) tells the employer or employment agency which data the bias audit must use:

A bias audit conducted pursuant to section 5-301 of this Chapter must use historical data of the AEDT. The historical data used to conduct a bias audit may be from one or more employers or employment agencies that use the AEDT. However, an individual employer or employment agency may rely on a bias audit of an AEDT that uses the historical data of other employers or employment agencies only in the following circumstances: if such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or if such employer or employment agency has never used the AEDT.

6 RCNY § 5-300 gives the definitions of the two types of data. Historical data is "data collected during an employer or employment agency’s use of an AEDT to assess candidates for employment or employees for promotion". Test data is "data used to conduct a bias audit that is not historical data".

Thus the historical data can come from one employer or employment agency, or from more than one. An employer or employment agency can "rely on" a bias audit that uses the historical data of other employers or employment agencies only if one of these conditions applies:

• It "provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit".
• It "has never used the AEDT".

Because of the text "one or more" in § 5-302(a), this type of bias audit can use the historical data of one other employer or employment agency. The two conditions do not apply if the bias audit uses only the historical data of the employer or employment agency.

6 RCNY § 5-302(b) gives an exception to § 5-302(a). An employer or employment agency is permitted to "rely on" a bias audit that uses test data if "insufficient historical data is available to conduct a statistically significant bias audit". If a bias audit uses test data, the summary of results must "explain why historical data was not used and describe how the test data used was generated and obtained".

6 RCNY § 5-302 gives three examples:

Example in 6 RCNY § 5-302	The employer or employment agency is permitted to "rely on"
Example 1: an employer will use the AEDT for the first time.	A bias audit with the historical data of other employers or employment agencies, or a bias audit with test data.
Example 2: an employment agency used the AEDT for 6 months. Its data from these 6 months is not sufficient for a statistically significant bias audit.	A bias audit with the historical data of other employers and employment agencies, if the employment agency gives its 6 months of historical data to the independent auditor. A bias audit with test data is also permitted.
Example 3: an employer used the AEDT for 3 years. It has statistically significant data from these 3 years.	A bias audit with historical data from more than one employer, if the employer gives its 3 years of historical data to the independent auditor. A bias audit with only the historical data from the use of the AEDT by this employer is also permitted. The employer is not permitted to "rely on" a bias audit with test data.

The DCWP FAQ (not law) gives these points about the data:

• Imputed or inferred data (FAQ III.6): DCWP gives this text about imputed demographic information and demographic information that algorithmic software can infer: "No. Imputed or inferred data cannot be used to conduct a bias audit." The adopted rules do not give this rule about imputed or inferred data.
• Statistical significance (FAQ III.7): DCWP "has not set a specific requirement for statistical significance". If an independent auditor finds that the historical data is not sufficient for a statistically significant bias audit, test data is permitted.
• Test data (FAQ III.8): DCWP gives this text: "To allow for flexibility and development of best practices in this rapidly developing field, DCWP has not set requirements for test data." DCWP also tells employers and employment agencies that the summary of results must include the source and explanation of the data. If the bias audit used test data, the FAQ adds this text: "the summary should explain how the data was sourced or developed".
• Scope of the historical data (FAQ III.3): the FAQ gives this text: "The Law has no specific requirement about the historical data used for a bias audit." For historical data that "was limited in any way, including to a specific region or time period", the FAQ recommends that "the audit should explain why". 6 RCNY § 5-303(a)(1) is law. It gives this rule: the summary of results must include the source and explanation of the data used for the bias audit.
• Same type of position (FAQ III.4): other employers or employment agencies can give historical data. There is no condition that they used the AEDT for the same type of employment position.

The effect of the results

Local Law 144 makes the bias audit necessary for the employer or employment agency. DCWP FAQ II.2 (not law) tells employers and employment agencies that the law "does not require any specific actions based on the results of a bias audit". The law also has rules for the results on the website (§ 20-871(a)(2) and 6 RCNY § 5-303). The page Publication and notices gives these rules.

FAQ II.2 also gives this text: "Federal, state, and New York City laws prohibit discrimination." Employers and employment agencies "must comply with all relevant Anti-Discrimination laws and rules to determine any necessary actions based on the results of a bias audit". The FAQ refers to the New York City Human Rights Law for more information. FAQ VII.2 gives this text: "Claims of discrimination involving AEDT use should go to the NYC Commission on Human Rights". It also tells employers and employment agencies that "DCWP will refer any claims of discrimination to the Commission".

Administrative Code §§ 20-870 to 20-874 and 6 RCNY §§ 5-300 to 5-304 set no minimum impact ratio. 6 RCNY § 5-301(b) and (c) give calculations only. Thus Local Law 144 does not make a low impact ratio a violation.

Thus the four-fifths (80%) rule of the EEOC Uniform Guidelines (29 C.F.R. § 1607.4(D)) is not a minimum that Local Law 144 sets. Modulos reads the four-fifths rule as follows (MRF-477):

• The rule is a "rule of thumb" and a "federal adverse-impact reference point" for selection rates.
• The rule does not apply to the median score and the scoring rates of 6 RCNY § 5-301(c).

Vendor audits and one audit for more than one employer

Local Law 144 applies the bias audit condition of § 20-871(a) to the employer or employment agency that uses the AEDT. DCWP FAQ V.1 gives this text: "The vendor that created the AEDT is not responsible for a bias audit of the tool." The page Scope and applicability gives more about the vendor.

FAQ V.2 tells employers and employment agencies that a vendor can have an independent auditor do a bias audit of its AEDT. It also gives this text: "The Law does not prohibit a vendor from having a bias audit done or coordinating the collection of data to use to conduct a bias audit". But employers and employment agencies "are ultimately responsible for ensuring a bias audit was done before using an AEDT" (FAQ V.2).

More than one employer or employment agency can "rely on" the same bias audit (6 RCNY § 5-302(a)). If that bias audit uses the historical data of other employers or employment agencies, an employer or employment agency can "rely on" it only if one of the two conditions of § 5-302(a) applies. The data section of this page, above, gives these conditions. DCWP FAQ III.2 and III.4 (not law) also give them.

How this maps in Modulos

In Modulos, the Framework MFF-27 (NYC Local Law 144) has these Requirements for the bias audit: MRF-477 Annual independent bias audit and MRF-478 Bias audit data requirements. For the provisions of the law and the rules in each Requirement, see The Requirements. For the steps that Modulos recommends for the bias audit, see Steps that Modulos recommends.

Where to go next

• Go to Publication and notices for the summary of results on the website and the notices to candidates and employees.
• Go to Scope and applicability to find if a tool is an AEDT.
• Go to Operationalizing NYC Local Law 144 in Modulos for the Requirements of MFF-27 and the steps that Modulos recommends.

Disclaimer

This page gives general information. It is not legal advice. Always compare this page with the publicly available text of N.Y.C. Administrative Code §§ 20-870–874 and 6 RCNY §§ 5-300–5-304 that applies on the date of your decision. Before you make a decision, speak to a lawyer.