Docs

Appearance

Make Humans Meaningfully Accountable ​

Dimension 2 of the IMDA Model AI Governance Framework for Agentic AI (the MGF) is about human accountability for agent actions. This page is about the agent value chain, teams, external parties, human oversight, and red teaming in the MGF.

Human accountability ​

The MGF for Agentic AI §2.2.1 tells organizations: "As deployers, organisations and humans remain accountable for the decisions and actions of agents." The MGF sets no obligations in law (see The MGF for Agentic AI). Thus, §2.2 does not set liability in law.

The MGF §2.2 identifies three problems for this accountability:

  • Agent actions "emerge dynamically and adaptively from interactions instead of fixed logic".
  • "Multiple stakeholders may also be involved in different parts of the agent lifecycle, diffusing accountability."
  • Automation bias is "the tendency to over-trust an automated system, especially when it has performed reliably in the past". This bias "becomes a bigger concern as humans supervise increasingly capable agents".

If human approvers monitor agents continuously, this can also cause automation bias and alert fatigue (MGF §1.2.3). For this risk, see Systemic risks.

For these problems, the MGF §2.2 tells organizations that they "should consider" two groups of measures:

  • "Clear allocation of responsibilities within and outside the organisation". The MGF gives two parts of this measure: "chains of accountability across the agent value chain and lifecycle", and adaptive governance. With adaptive governance, the organization can "quickly understand new developments and update their approach as the technology evolves".
  • "Measures to enable meaningful human oversight of agents". The examples in §2.2 are "requiring human approval at significant checkpoints, auditing the effectiveness of human approvals, and complementing these measures with automated monitoring".

Agent value chain ​

The MGF for Agentic AI §2.2.1 shows the "Simplified agentic AI value chain" with six roles:

  • Model developers
  • Tooling providers, for example MCP and APIs
  • Platform providers
  • System providers / app developers
  • Deployer
  • End users

The MGF tells organizations that they "should consider the allocation of responsibility both within their organisation, and vis-à-vis other organisations along the value chain". In §2.2.1, organizations "may play multiple overlapping roles across this value chain". For example, an organization "that develops its own agents and subsequently deploys them" has the role of the system provider and the role of the deployer.

The MGF §2.2.1 also recommends that an organization gives its users "sufficient information to hold the organisation accountable, as well as any information relating to the user’s own responsibilities" (see Enable End-User Responsibility).

Teams and accountability ​

The MGF for Agentic AI §2.2.1 tells organizations that they "should allocate responsibilities for different teams across the agent lifecycle". Each organization has a different structure. Thus, the MGF gives "an illustration of how such responsibilities may be allocated across different teams". In this example, the "Key responsibilities can include" these items:

Key decision makers

"Leaders who define strategic decisions and high-level policies for the organisation", for example "board members, C-suite executives, managing directors, or department leaders".

Their "Key responsibilities can include":

  • "Setting high-level goals for use of agents".
  • "Defining permitted operational use cases for agents, including limits on" the data access of agents.
  • "Setting the overall governance approach, including risk management frameworks and escalation processes".

Product teams

Persons who "oversee the translation of stakeholder needs or business goals into a technical agentic solution", for example "Product Managers, UI / UX Designers, AI Engineers, Software Engineers".

Their "Key responsibilities can include":

  • "Defining the design and requirements for agents, as well as any feature controls or phased rollouts".
  • "Reliable implementation of agents i.e. development, pre-deployment testing and post-deployment monitoring across the agent lifecycle".
  • "Educating users on responsible use of agentic product".

Cybersecurity teams

Persons who "oversee the protection of agentic systems from cyber threats", for example "Chief Security Officer, Cyber Security Specialist, Penetration Tester".

Their "Key responsibilities can include":

  • "Defining baseline security guardrails and secure-by-design templates" that "technical teams should implement or adapt to the agentic system being deployed".
  • "Conducting regular red teaming and threat modelling".

Users

"Any individual who utilises the output of the agents to contribute to an organisational goal", for example "company employees making decisions or automating workflows and practices".

Their "Key responsibilities can include":

  • "Ethical and responsible usage of agents".
  • "Attending required training, complying with usage policies, timely reporting of bugs or issues with agents".

These teams and their tasks are examples. The MGF does not give one structure for all organizations.

The MGF §2.2.1 also recommends: "All teams involved in agentic AI should also develop internal capabilities to understand the technology." In §2.2.1, organizations that are "aware of the improvements and limitations of agentic developments" can "quickly adapt their governance approach to new developments". Examples are "new modalities like computer use agents, or new evaluation frameworks".

Contracts and external agents ​

In the MGF for Agentic AI §2.2.1, organizations "may also need to work with external parties when deploying agents". Examples are "model developers, agentic AI providers, or hosts of external MCP servers or tools". For this work, the MGF recommends measures for the organization "to fulfil its own accountability". The MGF gives two "agent-specific considerations".

The first item is about contracts. The MGF recommends: "Clarify distribution of obligations in any terms and conditions or contracts between the organisation and the external party." For these contracts, the MGF tells organizations that they "should consider provisions on security arrangements, performance guarantees, or data protection". The MGF adds: "Where there are gaps, the organisation should reassess if the agentic deployment meets its risk tolerance."

The other item is about "third-party opacity". The MGF §2.2.1 identifies this problem: "Agents provided by external parties can be difficult to observe and control, making it challenging to understand what information they are grounded on, what they infer from conversations, or how data is stored and used." For this problem, the MGF recommends that organizations "take steps to maintain sufficient security and control". The examples in §2.2.1 are:

  • "Require transparency and accountability from external parties e.g. through disclosures on agent capabilities and data handling practices"
  • "Request and evaluate technical security and control features e.g. strong authentication measures such as scoped API keys, per-agent identity tokens, and observability such as the logging of tool calls and access history"

The MGF adds: "Where such features are lacking, organisations should consider alternative or in-house solutions, or scoping down the agentic use case, such as restricting access to sensitive data." The MGF gives these alternatives as possible results, not as necessary results.

Human oversight ​

The MGF for Agentic AI tells organizations that "a balance" is necessary because "continuous human oversight over all" agent workflows becomes "impractical at scale" (Executive Summary, p. 3).

The MGF for Agentic AI §2.2.2 gives organizations three parts of "a system for effective human supervision":

  1. "Significant" checkpoints or action boundaries for human approval.
  2. Training for human overseers to examine approval requests "effectively", and audits of these approvals.
  3. Automated monitoring mechanisms and "predefined" alert thresholds, which add to the first two parts.

For the levels of human involvement in agent actions (MGF §1.1.3), see Human approval.

Human approval ​

The MGF for Agentic AI §2.2.2 tells organizations that they "should define significant checkpoints or action boundaries that require human approval, especially before sensitive actions are executed". In §2.2.2, these checkpoints "can include":

  • "High-stakes actions and decisions e.g. editing of sensitive data, final decisions in high-risk domains (such as healthcare or legal), actions that may trigger liability"
  • "Irreversible actions e.g. permanently deleting data, sending communications, making payments"
  • "Outlier or atypical behaviour e.g. when agent accesses a system or database outside of its work scope, when agent selects a delivery route that is twice as long as the median distance."
  • "User-defined e.g. agents may act on behalf of users who have different risk appetites. Beyond organisation-defined boundaries, users may be given the option to define their own boundaries e.g. requiring approval for purchases above a certain amount."

These four types are examples.

Approval requests ​

The MGF for Agentic AI §2.2.2 also tells organizations that they "should also consider what form approvals should take". The MGF gives these two points:

  • "Keep approval requests contextual and digestible, while making the risk clear." The MGF continues: "When asking humans for approval, keep the request short and clear, instead of providing long logs or raw data that may be challenging to decipher. However, useful additional data, such as the associated risk with the action or a confidence score, should be included."
  • "Consider the form of human input required." The MGF gives these examples: "For straightforward actions such as accessing a database, the human user can simply approve or reject. For more complex cases, such as reviewing an agent’s plan before execution, it may be more productive for the human to edit the plan before giving the agent the go-ahead. For high-risk actions, human users can be required to provide an additional written justification before approving or rejecting."

Check human oversight ​

The MGF for Agentic AI §2.2.2 recommends that organizations "implement measures to ensure continued effectiveness of human oversight, particularly as humans remain susceptible to alert fatigue and automation bias and may be influenced by anthropomorphic design". In §2.2.2, these measures "can include" three items: regular audits, "Training human overseers", and domain expertise.

For regular audits of "the effectiveness of human oversight", the MGF gives these examples:

  • "Tracking measurable indicators":
    • The human override rate, "the frequency at which humans reject or modify agent actions". "A low rate may signal rubber-stamping behaviours."
    • "Human response times during review of agent actions." "A shorter time may signal automation bias or review fatigue."
  • Data analytics that find "outlier" human overseers, "whose decision patterns deviate significantly from the norm". "This may indicate compromised oversight."

In the MGF, these values "may signal" or "may indicate" a problem. The MGF does not give limit values for them.

Human overseers ​

The MGF for Agentic AI §2.2.2 recommends "Training human overseers to identify common failure modes or agent limitations". Examples are "inconsistent agent reasoning, agents referring to outdated policies". The MGF adds that chain-of-thought reasoning, "which is sometimes used for explainability, is not analogous to human reasoning and may not be a faithful explanation of the agent’s actions".

The MGF §2.2.2 also recommends "Ensuring that humans possess the domain expertise to evaluate agent actions". In the example of the MGF, "users using agents to “vibe code” may not have the software engineering expertise to review the robustness of generated code".

Automated monitoring ​

The MGF for Agentic AI §2.2.2 recommends that "human oversight should be complemented with automated real-time monitoring to escalate any unexpected or anomalous behaviour". The MGF gives these examples:

  • "Implementing alerts for certain logged events (e.g. attempted unauthorised access or multiple failed attempts to call a tool)"
  • "Using data science techniques to identify anomalous agent trajectories"
  • "Using agents to monitor other agents"
  • "Denying action by default when approval infrastructures fail (e.g. when human supervisors are unreachable or agents attempt new actions that do not have any established approval policies in place)."

The last item applies only "when approval infrastructures fail". The MGF does not recommend it for all agent actions. For monitoring during deployment, see Deployment and monitoring.

Red teaming and threat modeling ​

In the team example of the MGF for Agentic AI §2.2.1, the tasks of cybersecurity teams "can include" "Conducting regular red teaming and threat modelling". The MGF does not give a schedule for red teaming or threat modeling.

Other pages give more information about these tasks:

Use the framework in Modulos ​

In Modulos, the Singapore MGF for Agentic AI has two Frameworks: OFF-17 for the Organization and MFF-17 for each AI System. For the Requirements for human accountability and the steps that Modulos recommends, see Use the MGF for Agentic AI in Modulos.

Sources ​

  • MGF for Agentic AI — IMDA, Model AI Governance Framework for Agentic AI, Version 1.5 ("Published 20 May 2026", "Updated 5 June 2026"), sections 2.2, 2.2.1, and 2.2.2 (pages 25–30). Also the Executive Summary (page 3) and sections 1.1.3, 1.2.3, 2.1.1, 2.3.2, and 2.3.3. IMDA.

Disclaimer

This page gives general information. It is not legal advice.