Skip to content

Sources ​

A Source is a connection from an AI System to an external system, with its credentials. Modulos uses it to read data and context from that system. Metric Tests, Agent Tests, Scout, and the Risk Agent read the Sources of the AI System. The Source type sets which of them can read a Source.

Where in Modulos ​

  • AI System → Settings → Sources lists the Sources of the AI System and adds new Sources.
  • Organization → Source & Connectors → Sources makes each Source type available or unavailable for the organization.
AI System Settings, Sources tab, with the list of Sources and the Add Source action.

The Sources of an AI System, with their type, status, and connection.

  1. Sources tab The Sources of an AI System are in its Settings.
  2. Add Source Adds a Source to the AI System.
  3. Connection The result of the last connection check.

Who can do what ​

Permissions

  • Organization Admins make Source types available or unavailable for the organization.
  • Owners of an AI System add, change, and delete its Sources in AI System → Settings → Sources. Organization Admins, Risk Managers, and Policy Managers can do this in each AI System.
  • Editors and Viewers cannot open the Settings of an AI System and do not see its Sources there. Editors select Sources in Metric Tests, Agent Tests, and Scout.

Supported Source types ​

Source typeMetric TestsScout, Agent Tests, Risk AgentConnects to
AWS✓✓CloudWatch metrics and resources of an AWS account
Azure✓✓Metrics and resources of Azure subscriptions
Claude Console✓✓A Claude API organization on platform.claude.com
Claude Enterprise✓✓A Claude Enterprise organization on claude.ai
Datadog✓Metrics in a Datadog account
GitHub✓A GitHub repository
Langfuse✓✓A Langfuse project
Microsoft Copilot Studio✓The Copilot Studio agents of a Power Platform tenant
Modulos Client✓Metrics that your code sends to Modulos
OpenAI Platform✓✓An OpenAI API organization on platform.openai.com
Prometheus✓Metrics on a Prometheus server
Snowflake✓✓A Snowflake database
Vijil✓✓Evaluation scores of an agent in Vijil

Scout, Agent Tests, and the Risk Agent read the same Source types. The Risk Agent reads the active Sources of these types in the AI System. An Agent Test that has a Datadog or Prometheus Source in its Sources ends with an error.

For AWS, Azure, Claude Console, Claude Enterprise, Langfuse, OpenAI Platform, and Vijil, these AI agents read only one Source of each type in the AI System. Metric Tests can use each Source.

If a Source type is not available in AI System → Settings → Sources, the type is not available for the organization.

Make a Source type available ​

Each Source type is not available until an Organization Admin makes it available. See Source & Connectors, which also tells what happens to the Sources of a type that becomes unavailable.

Pull and push ​

  • A pull Source: Modulos connects to the external system and reads the data. All types in the Add Source dialog are pull Sources.
  • A push Source: your code sends data to Modulos. Modulos Client is the push Source. Each AI System shows it. It is active only when the organization activates Modulos Client.

Connection check ​

When you save a Source, Modulos tests the connection. If the test fails, Modulos does not save the Source. Most types then show a general error. The type pages tell you what Modulos checks.

Add Source dialog with the available Source types and the fields of the selected type.

Add a Source: select the type and give its credentials.

  1. Source type The external system to connect to the AI System.
  2. Fields The credentials and settings of the selected type.
  3. Save Modulos tests the connection before it saves the Source.

After you save, the Connection column of AI System → Settings → Sources shows Healthy, Broken, Unknown, or Not applicable. Modulos checks again when you open the page and after you add, change, or delete a Source. Put the pointer on the status to see the cause. Modulos Client always shows Not applicable, because it receives data.

Keys and secrets ​

  • Use a dedicated service account for each Source, with read access only.
  • Replace keys and secrets on a schedule.
  • After you save a Source, Modulos keeps its secrets and does not show them again.

Sources and Connectors ​

A Source belongs to an AI System. A Connector belongs to one user. The other differences come from this:

SourceConnector
Who uses itAll members of the AI SystemOnly the user who connected it
Where you add itAI System → Settings → Sources, or Connect Sources in the registration wizardUser Settings → Connectors
AccessThe credentials that you give, for example an API key of a service accountThe account of the user, through OAuth
What reads itMetric Tests, Agent Tests, Scout, and the Risk Agent, as the Source type permitsScout, in the chats of that user about an AI System or a Draft, when the user selects it with Context. Tests do not read Connectors.
TypesSee Supported Source typesGitHub, Bitbucket, Google Drive, Atlassian