Risk, evaluation, and monitoring

This page covers the back half of the IEEE 7003 process: risk and impact assessment (Clause 8), design and output bias evaluation (Clause 9.2), and ongoing evaluation and drift monitoring (Clause 9.3). It maps the per-system requirements MRF-445, MRF-446, and MRF-447.

These stages consume the footing established earlier — the boundaries of acceptability, the stakeholder reference set, and the data mapping — and turn it into assessed risk, evidence of evaluation, and a standing watch for drift. All three write back into the bias profile.

Risk and impact assessment (Clause 8)

Bias-related risks and impacts are identified and analyzed through an ongoing risk and impact assessment, not a one-time exercise:

• It commences when the business case is created, with the risk and impact metrics and their rationale set out from the start.
• It is reviewed and updated whenever the bias profile, the system, its context of use, or its environment changes, checking for new or emerging bias risks and whether existing mitigations and accepted risk-tolerance levels still hold.
• It maintains two separate inventories: one for influencing and internal stakeholders, and one for the impact on external and impacted stakeholders.
• The assessment team's makeup should reflect the identified stakeholders, and external stakeholders should be consulted.
• Each assessment and update is signed by, and accountable to, whoever controls the system across its lifecycle — and that accountability reaches the consequences of any risks left unmitigated.
• A change of ownership or environment requires a full reassessment with formal acceptance.

In Modulos, MRF-445 carries this through the new control MCF-667 (bias risk and impact register with owner acceptance), alongside reused risk-assessment controls (MCF-208, MCF-210). The accountable-owner signature — including responsibility for unmitigated consequences — is an explicit, falsifiable element of the control.

Design and output bias evaluation (Clause 9.2)

The design and outputs are assessed for bias through a process built into design and development that may be repeated as the product matures. It is broad by design, and it produces a single evaluation record. The work:

• reviews requirements, data, and design documentation;
• evaluates pre-processing for introduced bias and technique bias, repeating those procedures when the model is retrained or updated;
• chooses test scenarios in which bias is the hazard under examination, and tests them at a chosen granularity;
• defines and justifies output-bias metrics and methods against business requirements — the standard does not pick the metric for you;
• compares expert and system decisions where appropriate;
• assesses mitigation trade-offs, and checks post-mitigation bias, iterated as appropriate;
• checks optimization effects before and after;
• checks the UI and UX so the design reflects the attributes of all identified stakeholders, including accessibility.

The output is an evaluation record: chosen metrics and scenarios, each test's outputs and interpretation, explored biases and their sources, mitigation recommendations, post-mitigation bias, optimization results, UI/UX findings, and the resulting bias-profile updates.

In Modulos, MRF-446 carries this as the integrated new control MCF-665 (design and output bias evaluation record), which coordinates — rather than duplicates — reused testing, evaluation, and accessibility controls (MCF-35, MCF-42, MCF-43, MCF-44, MCF-58, MCF-152).

Ongoing evaluation and drift monitoring (Clause 9.3)

Bias is not settled at deployment. An ongoing evaluation process is set up and run to catch drift and change affecting bias through the system's operation, up to decommissioning.

• Context-specific monitored items. The team documents which items it will monitor and why. The standard's examples — data, decisions, stakeholder effects, complacency bias, feedback loops, interface, culture, real-world bias effects — are considerations to weigh, not a fixed checklist.
• A process per item. Every selected item gets its own documented evaluation process, shaped by the system's behavior and the preceding design-and-output evaluation, with recurrence documented and set by time or by triggers such as volume or alarms.
• Drift, recorded. Each iteration checks and records data drift, concept drift, and any system-specific drift, including evidence that outputs reinforce real-world bias, with plans to act.
• The profile stays current. After every iteration the relevant bias-profile parts are reviewed and updated; any mitigation triggers a further cycle as appropriate; and, where appropriate, impacted stakeholders' feedback feeds into the process.

In Modulos, MRF-447 carries this through the new control MCF-666 (ongoing bias evaluation program), alongside reused monitoring and feedback-loop controls (MCF-45, MCF-67, MCF-68, MCF-72). The feedback-loop control MCF-45 retains a named branch for EU AI Act Article 15(4) continuously-learning high-risk systems.

Cross-framework fit

Preview

• EU AI Act — the ongoing risk assessment and drift program align with the Article 9 risk-management-system duty and the Article 72 post-market monitoring expectation for high-risk systems; the feedback-loop treatment aligns with Article 15(4).
• ISO/IEC 42001 — the evaluation record and monitored-item program align with the performance-evaluation and continual-improvement clauses of the management system.
• NIST AI RMF — risk and impact assessment sits in Measure; the ongoing evaluation and drift program sits in Manage.

Related pages

Stakeholders and data representation

The prior stages: who is affected, and how the data represents them — MRF-442–444

Bias requirements and the bias profile

Setup, boundaries of acceptability, and the through-life profile — MRF-440, MRF-441

Operationalizing in Modulos

The OFF-25 / MFF-25 rollout, control library, and evidence model

IEEE 7003 overview

What the standard is, and the MFF-25 / OFF-25 split

Source attribution

The authoritative source is IEEE Std 7003-2024, IEEE Standard for Algorithmic Bias Considerations, published by IEEE. This page paraphrases Clauses 8 and 9 and references them by number and name; no text from the standard is reproduced, per IEEE licensing. Requirement and control codes are Modulos template identifiers, not IEEE references.

Disclaimer

This page is for general informational purposes and does not constitute legal advice. IEEE 7003 is a voluntary standard; conformance is not a legal obligation. Verify against the current published edition of IEEE Std 7003-2024.