Docs

Appearance

Risk, evaluation, and monitoring ​

Clauses 8 and 9 of IEEE Std 7003-2024 (IEEE 7003) give the risk and impact assessment (RIA) for an autonomous intelligent system (AIS). They also give two evaluation procedures: the evaluation of the AIS design and outputs, and the ongoing evaluation (9.1). The RIA starts at the "system inception stage" and continues during the AIS life cycle (8.1, 8.4 a). Thus the RIA does not start only after the work of the other clauses. The evaluation of the design and outputs is part of the "design and development stages" (9.2.1). The ongoing evaluation continues during the operation of the AIS and during its decommissioning (9.3.1).

Each of these three gives an updated bias profile as an output (8.3 a, 9.2.3 a, 9.3.3 a). For the Modulos Requirements that hold Clauses 8 and 9 of IEEE 7003, see The Requirements.

Risk and impact assessment (Clause 8) ​

IEEE 7003 Clause 8 gives the tasks to identify and examine the bias risks and effects of an autonomous intelligent system (AIS) (8.1). This risk and impact assessment (RIA) continues during the life cycle of the AIS, and also when the context of use changes (8.1). In Clause 5.2.4, it is necessary that the RIA is "resourced" for the full life cycle. When requirements are set for stakeholder identification and data representation, the results are inputs to the requirements for the RIA (5.2.4).

IEEE 7003 requires an initial bias RIA at the "system inception stage" (8.1). At each subsequent step of the life cycle, after the evaluation of bias in the AIS, the RIA is examined again, and its risks and effects are "updated" (8.1). The problems that the RIA finds go into the bias profile, and the previous mitigations are examined again and "updated" (8.1). In Clause 8.1, it is very important to examine the RIA again at each step. This helps to "counter" unwanted systemic bias and an unwanted increase of bias.

Inputs (8.2). IEEE 7003 requires that, at each iteration, the inputs to the RIA are the "current version" of each of these items:

  • From the bias consideration process: the bias profile, the values statement for the AIS, stakeholder identification, data representation, and the evaluation report.
  • From the AIS life cycle: the business case, the AIS system design, the AIS deployment strategy, and the AIS retirement plan.

Outputs (8.3). The outputs are an updated bias profile and an initial or "updated" bias RIA.

Tasks (8.4). Clause 8.4 gives these tasks:

  • Start (8.4 a). IEEE 7003 requires that the assessment starts when the business case is made. It also requires that the risk and impact metrics are identified, and that the "rationale" for them is given.

  • Change (8.4 b). IEEE 7003 requires that the RIA is examined again and "updated" at each step in the life cycle of the AIS where one of these changes occurs:

    • The bias profile changes.
    • The AIS changes, or its context of use or its environment changes.

    The RIA is examined again for bias risks and effects that are new or that start to occur. It is also examined to find the "effectiveness" of the mitigation strategies and of the accepted levels of risk tolerance.

  • The RIA team (8.4 c). The procedure to examine the assessment again helps to make sure that the assessment can continue during the full AIS life cycle. IEEE 7003 recommends that the diversity of the persons in the RIA team agrees with the diversity of the identified stakeholders (also 1.6 e). These are the stakeholders that stakeholder identification found. IEEE 7003 also recommends that external stakeholders are "consulted" for input.

  • Two risk inventories (8.4 d, e). IEEE 7003 requires a risk inventory of the influencing stakeholders or internal stakeholders. It also requires a risk inventory of the effect on external stakeholders or impacted stakeholders. Informative Annex C (Guidance for Risk and Impact Assessment) has more information.

  • Accountability (8.4 f). IEEE 7003 requires that the assessment and each of its subsequent changes are signed, with accountability to the position that controls the AIS during the AIS life cycle. One more obligation applies when the RIA procedure finds that the risks are not removed or mitigated. Then IEEE 7003 requires that the "relevant" accountable entity also accepts accountability and responsibility for possible legal, regulatory, or other "harmful consequences".

Outcomes (8.5). The users of the standard identify the levels of the risk of bias and of the effects. From these levels, the tasks for risk treatment are identified from IEEE 7003. They are used "on an ongoing basis" to find if the identified risks and effects are mitigated. The owner of the AIS gives formal acceptance of the assessment. When the ownership of the algorithmic system changes, its environment changes, or the two change, a full reassessment and its formal acceptance are necessary.

Clause 8.4 f names the position that controls the AIS for the signed assessment and the accountability. Clause 8.5 names the owner of the AIS for the formal acceptance.

Evaluation of the AIS Design and Outputs (Clause 9.2) ​

IEEE 7003 Clause 9 gives two procedures for the evaluation of a complete autonomous intelligent system (AIS) for bias (9.1). The first procedure is the evaluation of the AIS design and outputs (9.2). The other procedure is the ongoing evaluation (9.3). IEEE 7003 requires that the evaluation of the design and outputs is part of the "design and development stages" (9.2.1). More than one evaluation during these steps is permitted, "as the product matures" (9.2.1).

In Clause 5.2.5, it is necessary that the evaluation is "resourced" during the full AIS life cycle. IEEE 7003 requires that the initial work for the evaluation examines the components of the AIS and how they use data. IEEE 7003 also requires that this work examines how the AIS can be a solution in its context of use (5.2.5). Personnel who know how to examine AIS bias can help (5.2.5).

Inputs (9.2.2). The inputs are these:

  • The "documentation" of the AIS requirements and of the AIS design.
  • The parts of the bias profile for stakeholder identification (Clause 6), data representation (Clause 7), and the risk and impact assessment (Clause 8).
  • The test datasets.
  • The "documentation" of the user interface (UI) design and the user experience (UX) design.

Outputs (9.2.3). The outputs are an updated bias profile and an evaluation document. IEEE 7003 gives nine items for the evaluation document (9.2.3 b):

  1. The selected metrics.
  2. The testing scenarios.
  3. The outputs of the tests.
  4. An assessment of these outputs with the selected methods, and the "rationale" for the selection of these methods.
  5. An assessment of the relation between the bias that the evaluation found and the business requirements.
  6. Where applicable, the AIS decisions compared with the decisions of an expert panel.
  7. Where applicable, an assessment of bias in the UI and UX design and of its effect on data bias.
  8. The biases that the evaluation examined and, where "feasible", their sources.
  9. The results of the evaluation of the mitigation procedure and of the performance optimization procedure for bias.

Tasks (9.2.4). Clause 9.2.4 gives these tasks:

  • Examine the records (9.2.4.1). Examine the AIS requirements, the data "documentation", and the design "documentation".

  • Data (9.2.4.2). Examine the data pre-processing for bias that it adds to the dataset, and for bias in the pre-processing. Do this step with the information in the data bias profile. IEEE 7003 requires that the pre-processing methods are examined for bias in the method and for bias that they add to the dataset. Clause 9.2.4.2 gives it as important to record the pre-processing, to make sure that "similar" procedures are followed when a model is retrained and changed. IEEE 7003 requires that these procedures are followed. It is also important to find how the pre-processing has an effect on the outcomes of the AIS.

  • Testing (9.2.4.3). Clause 9.2.4.3 gives these steps. Each step starts with the item of Clause 9.2.4.3 that gives it:

    1. (a) Select testing scenarios in which bias is a risk. Apply risk-based testing as ISO/IEC/IEEE 29119 gives it.
    2. (a) Examine the granularity of the testing, for example for each group or for each subgroup. Informative Annex D (Measuring Bias) has more information.
    3. (b) Select a metric for the evaluation of bias in the AIS, with the selected testing scenarios and reference values.
    4. (b, b 1) Record each test and its outputs. A bias evaluation method is a testing scenario and a metric.
    5. (b 2) Prepare and "justify" a method to examine the AIS outputs for each of these types of bias:
      • Biases that have an effect on the impacted stakeholders.
      • Biases that have an effect on the risk and impact assessment.
      • Biases that previous steps of the AIS life cycle identified.
    6. (b 3) Prepare the rules to "interpret" the method results, and "justify" these rules.
    7. (b 4) Test the AIS to get the outputs for the next step.
    8. (c) Examine the AIS outputs with the prepared method, and "interpret" the results.
    9. (c) Do this step "with respect to" the business requirements (Clause 4).
    10. (c) Record how you "interpret" the method results, and record their relation to the business requirements.
    11. (d) Recommend mitigations from the results of the evaluation.

    The specification of mitigations is out of the scope of IEEE 7003 (5.2). For this limit, see The bias profile (Clause 5).

  • End of the evaluation (9.2.4.4). Clause 9.2.4.4 gives these tasks:

    • Where "appropriate", compare the decisions that only persons make with the AIS decisions. An expert panel is permitted for this.
    • Examine the bias that procedures to help mitigate bias add. Examine and record each "trade off" where the mitigation of one type of bias adds a different type of bias. Record each new type of bias that the model outputs show after these procedures. Do these tasks again, as "appropriate".
    • Examine the bias that procedures for the optimization of computational performance add. Apply the metrics before the optimization. Apply the metrics after the optimization. Compare the results.
    • Compare the UI design and the UX design with the stakeholder identification of Clause 6. IEEE 7003 requires that the UI design "reflects" the attributes of all the identified stakeholders. Clause 9.2.4.4 gives two examples. One is a "drop-down list" without all the attributes of the impacted stakeholders. The other is a UI design without accessibility for persons with disabilities.

Outcome (9.2.5). IEEE 7003 requires that, after these tasks, the users of the standard know how to do an evaluation of bias in the AIS design.

Ongoing evaluation and drift monitoring (Clause 9.3) ​

IEEE 7003 Clause 9.3 gives an ongoing evaluation procedure for an autonomous intelligent system (AIS). This procedure examines the AIS for drift and for changes that have an effect on the AIS and that are related to bias (9.3.1). IEEE 7003 requires that continuous monitoring of the AIS helps to find drift during its operation and during its decommissioning (9.3.1). For example, drift can come from changes in the data sources or in "society" (9.3.1). It can also come from important changes that have an effect on the context of use of the AIS.

Clause 3.1 of IEEE 7003 gives this information:

  • Data drift — the "prediction accuracy" of a model decreases with time, because the statistical characteristics of the production data change.
  • Concept drift — the decision boundary moves, and the "accuracy of predictions" decreases, although the data does not change.
  • Model decay — the performance of a model decreases, as the specified performance metrics show. Model decay can be a symptom of drift of one or more types.

Inputs (9.3.2). The inputs are the "documentation" of the AIS requirements and of the AIS design. The parts of the bias profile for Clauses 6, 7, and 8 are also inputs. The records of the tasks of the evaluation of the design and outputs are also an input.

Outputs (9.3.3). The outputs are an updated bias profile and an "updated" evaluation document with these four items:

  1. The list of items to monitor (9.3.4.1).
  2. The process description (9.3.4.1).
  3. The "rationale for recurrence" (9.3.4.1).
  4. Each drift, and the "plans" for it, "if any" (9.3.4.2).

Set-up of the procedure (9.3.4.1). Clause 9.3.4.1 gives these tasks:

  • Prepare the list of items to monitor. The examples in Clause 9.3.4.1 are data, the decisions of the AIS, the effect of the AIS on the stakeholders, and "risks and impacts". Other examples are when persons accept AIS decisions without thought (complacency bias), feedback loops in recommender systems, and the user interface. The last examples are "the culture" (informative Annex E) and the effect of the AIS deployment on external real-world bias. These are only examples, and other items are possible.
  • Prepare and record a procedure for the ongoing evaluation of each item. IEEE 7003 requires that this procedure uses how the AIS operates to find which ongoing evaluation is "appropriate". Also examine the work that you did in the previous evaluation of the design and outputs.
  • Prepare and record a "rationale" for the recurrence of the ongoing evaluation. The recurrence is permitted to be by time or by other indications, as "appropriate" to the context of use. Examples of these indications are "transaction volume", throughput, "automatic alarms", and unusual conditions.
  • If mitigation steps occur in a part of the AIS, IEEE 7003 requires one more cycle of the evaluation procedure, as "appropriate".
  • If "appropriate", make mechanisms that collect the "feedback" of impacted stakeholders, and use it in the ongoing evaluation.

The NOTE to Clause 9.3.4.1 gives information only. In the NOTE, special conditions can make it necessary to do the evaluation again, not at the usual interval. Its examples are changes to the data representation or to the risk and impact assessment, and important changes to the life cycle of the system. Other examples are the "release of new versions" of parts of the AIS, and an external audit.

Clause 7.4.2.3 of IEEE 7003 also requires continuous monitoring of the data as Clause 9 gives it. It also requires that the bias profile is changed to agree with the results. For the data mapping, see Data exploration and data mapping (Clause 7).

Each iteration (9.3.4.2). IEEE 7003 requires that each iteration of the ongoing evaluation examines and records the types of drift that can cause bias. It also requires the "plans" for each type, "if any". Clause 9.3.4.2 gives these types:

  • Data drift. Check the AIS for data drift. Other names for data drift in Clause 9.3.4.2 are feature drift, "population" drift, and "covariate" drift. After this check, you can recommend that the model is retrained. You can also recommend changes to the "business metric", the bias metrics, or the two. You can also recommend a change to how frequently retraining and ongoing evaluation occur.
  • Concept drift. Check the AIS for concept drift. Concept drift can occur when the relation between inputs and outputs that the model got from its training changes. The model then does not show that relation correctly. After this check, you can recommend that the model is retrained and that the metrics of the design evaluation are changed. Or you can recommend a different algorithm for a new model with new metrics. A large external change can start this check.
  • Other drift. Identify and monitor the other types of drift of the AIS. IEEE 7003 requires that this drift is recorded, with a "plan" to "address" the bias. Also, if there is "evidence" of the "reinforcement" of external real-world bias, IEEE 7003 requires that this item is monitored. The mechanism is a loop. The AIS and its outputs have external effects, and these effects then go back into the AIS.

End of each iteration (9.3.4.3). At the end of each iteration of the ongoing evaluation, examine these five parts of the bias profile again, and change them if necessary:

  1. The requirements for bias consideration.
  2. Stakeholder identification (Clause 6), to "update" the identified stakeholders.
  3. Data representation (Clause 7).
  4. The risk and impact assessment (Clause 8).
  5. The evaluation procedure, which includes the list of items to monitor (Clause 9).

Outcome (9.3.5). IEEE 7003 requires that, after these tasks, the users of the standard know how to do the ongoing evaluation of the AIS for bias.

Other frameworks ​

To compare IEEE 7003 Clauses 8 and 9 with other frameworks, see How IEEE 7003 compares to other frameworks.

Where to go next ​

Stakeholders and data representation

Clauses 6 and 7: stakeholder identification and data representation

Bias requirements and the bias profile

Clauses 4 and 5: requirements for bias consideration and the bias profile

Operationalizing IEEE 7003 in Modulos

The Frameworks, the Requirements, and a sequence for Modulos

IEEE 7003 overview

IEEE 7003, where it applies, and conformance

Sources of this page ​

  • IEEE Std 7003-2024 — IEEE Standard for Algorithmic Bias Considerations, approved 11 December 2024, published 24 January 2025: Clauses 1.6, 1.8, 1.9, 3.1 (concept drift, data drift, model decay), 5.2, 5.2.4, 5.2.5, 7.4.2.3, 8, and 9, and Annexes C, D, and E. IEEE SA page.

Disclaimer

This page gives general information only. It is not legal advice. Use of IEEE 7003 is voluntary, but a procurer and a supplier can include it in their contract (1.8). Conformance to IEEE 7003 does not release a person or an organization from legal or other obligations (1.9). Before you make a decision, compare this page with IEEE Std 7003-2024.