Skip to main content

AI Agent

An AI Agent is a first-class catalog entity representing an AI agent — a system built on one or more LLMs that uses tools and skills to accomplish tasks (LangChain agents, Google ADK agents, Snowflake Cortex agents, DataHub's own "Ask DataHub" assistants, etc.). Cataloging agents lets teams discover what agents exist, understand what they can do and what they depend on, govern them, and track their lineage into the data they touch.

Agents come in three flavors, distinguished by the source.type of the aiAgentInfo aspect:

  • System (SYSTEM): bootstrapped at deployment time (e.g. via YAML) and not editable by end users.
  • Native (NATIVE): created and managed on DataHub by end users through the UI or API.
  • External (EXTERNAL): managed outside DataHub but cataloged here for discovery and governance.

This is an origin/governance axis, not a functional one — a single-agent, multi-agent, or copilot distinction (if needed) is layered on with the standard subTypes aspect.

Identity

AI Agents are identified by a single field:

  • id: A unique identifier for the agent, typically a human-readable slug such as ask-datahub or a generated UUID.

An example URN is urn:li:aiAgent:ask-datahub. The identifier is platform-agnostic; the platform, if any, is carried on the dataPlatformInstance aspect rather than in the key.

Important Capabilities

Agent Info

The core properties live on the aiAgentInfo aspect:

  • name: Display name, searchable with autocomplete.
  • tagline: A short (~120 char) one-line summary shown on agent cards, hero subtitles, and picker dropdowns. When absent, surfaces fall back to truncating the description.
  • description: What the agent does and when to route to it. The recommended style is "X does Y. Use when Z." — for router-style agents (like Ask DataHub) this text is read verbatim when deciding whether to delegate, so the "Use when…" tail is load-bearing rather than decorative.
  • instructions: Custom base instructions injected into the agent's system prompt, beyond the standard boilerplate.
  • source: The origin of the agent definition (see below).
  • created / lastModified: Audit stamps.

Source and Agent Families

The source field (an AIAgentSource) records the agent's origin type (SYSTEM / NATIVE / EXTERNAL) and, optionally, a clonedFrom reference to the agent this one was cloned from (the ClonedFromAgent relationship). Clones inherit their base's configuration verbatim — one frontend module, one tool palette, one set of instructions — and differ only in the artifacts they manage. Grouping a base agent and its clones this way forms an Agent Family.

Dependencies

The aiAgentDependencies aspect captures what the agent relies on, as typed relationships to other first-class entities:

  • skillsAgent Skill entities the agent adopts (AgentHasSkill).
  • toolsAPI entities the agent invokes (AgentUsesTool). This is a lineage edge: the tools are upstream of the agent, contributing to the repo → service → api → agent chain.
  • modelsmlModel entities (the LLMs) the agent runs on (AgentUsesModel).

Versioning, Governance, and Health

AI Agents support the standard governance aspects — ownership, tags, glossary terms, domains, structured properties, and institutional memory — as well as native versioning (versionProperties, so successive revisions of an agent group into a version set) and health via the shared incidents subsystem (incidentsSummary). Agents can also participate in lineage through upstreamLineage.

Technical Reference Guide

The sections above provide an overview of how to use this entity. The following sections provide detailed technical information about how metadata is stored and represented in DataHub.

Aspects are the individual pieces of metadata that can be attached to an entity. Each aspect contains specific information (like ownership, tags, or properties) and is stored as a separate record, allowing for flexible and incremental metadata updates.

Relationships show how this entity connects to other entities in the metadata graph. These connections are derived from the fields within each aspect and form the foundation of DataHub's knowledge graph.

Reading the Field Tables

Each aspect's field table includes an Annotations column that provides additional metadata about how fields are used:

  • ⚠️ Deprecated: This field is deprecated and may be removed in a future version. Check the description for the recommended alternative
  • Searchable: This field is indexed and can be searched in DataHub's search interface
  • Searchable (fieldname): When the field name in parentheses is shown, it indicates the field is indexed under a different name in the search index. For example, dashboardTool is indexed as tool
  • → RelationshipName: This field creates a relationship to another entity. The arrow indicates this field contains a reference (URN) to another entity, and the name indicates the type of relationship (e.g., → Contains, → OwnedBy)

Fields with complex types (like Edge, AuditStamp) link to their definitions in the Common Types section below.

Aspects

aiAgentInfo

Configuration and metadata for an AI agent.

Agents can be:

  1. System (SYSTEM): bootstrapped at deployment time via YAML, not editable
  2. Native (NATIVE): created by end users through the UI or API
  3. External (EXTERNAL): cataloged on DataHub but managed externally
FieldTypeRequiredDescriptionAnnotations
namestringDisplay name of the agentSearchable
taglinestringShort, user-facing tagline shown on agent cards, hero subtitles, and picker dropdowns. Should be ...Searchable
descriptionstringDescription of what this agent does AND when Ask DataHub should route to it. Style: "X does Y. Us...Searchable
instructionsstringCustom base instructions for the agent, beyond the standard DataHub boilerplate. Injected into th...
sourceAIAgentSourceThe source or origin of the agent definition. System agents (source.type = SYSTEM) are bootstrapp...
createdAuditStampWhen this agent was createdSearchable
lastModifiedAuditStampWhen this agent was last modifiedSearchable

aiAgentDependencies

Dependencies of an AI agent: the skills it adopts, the tools it invokes, and the models it runs on. These are governance/discovery relationships to first-class catalog entities.

FieldTypeRequiredDescriptionAnnotations
skillsstring[]High-level skills this agent adopts.Searchable, → AgentHasSkill
toolsstring[]APIs this agent invokes (cataloged API entities).Searchable, → AgentUsesTool
modelsstring[]Models (LLMs) this agent runs on.→ AgentUsesModel

dataPlatformInstance

The specific instance of the data platform that this entity belongs to

FieldTypeRequiredDescriptionAnnotations
platformstringData PlatformSearchable
instancestringInstance of the data platform (e.g. db instance)Searchable (platformInstance)

displayProperties

Properties related to how the entity is displayed in the Datahub UI

FieldTypeRequiredDescriptionAnnotations
colorHexstringThe color associated with the entity in Hex. For example #FFFFFF.
iconIconPropertiesThe icon associated with the entity

ownership

Ownership information of an entity.

FieldTypeRequiredDescriptionAnnotations
ownersOwner[]List of owners of the entity.
ownerTypesmapOwnership type to Owners map, populated via mutation hook.Searchable
lastModifiedAuditStampAudit stamp containing who last modified the record and when. A value of 0 in the time field indi...

status

The lifecycle status metadata of an entity, e.g. dataset, metric, feature, etc. This aspect is used to represent soft deletes conventionally.

FieldTypeRequiredDescriptionAnnotations
removedbooleanWhether the entity has been removed (soft-deleted). Kept for backward compatibility. When lifecyc...Searchable
lifecycleStagestringThe lifecycle stage of the entity, referencing a lifecycleStageType entity. When null, the entity...Searchable
lifecycleLastUpdatedAuditStampAttribution for the lifecycle stage transition ��� who moved the entity into its current stage an...

structuredProperties

Properties about an entity governed by StructuredPropertyDefinition

FieldTypeRequiredDescriptionAnnotations
propertiesStructuredPropertyValueAssignment[]Custom property bag.

upstreamLineage

Upstream lineage of a dataset

FieldTypeRequiredDescriptionAnnotations
upstreamsUpstream[]List of upstream dataset lineage information
fineGrainedLineagesFineGrainedLineage[]List of fine-grained lineage information, including field-level lineage→ DownstreamOf

globalTags

Tag aspect used for applying tags to an entity

FieldTypeRequiredDescriptionAnnotations
tagsTagAssociation[]Tags associated with a given entitySearchable, → TaggedWith

glossaryTerms

Related business terms information

FieldTypeRequiredDescriptionAnnotations
termsGlossaryTermAssociation[]The related business terms
auditStampAuditStampAudit stamp containing who reported the related business term

semanticContent

Semantic content for enabling vector similarity search.

This aspect stores chunked text and embedding vectors for any entity that supports semantic search. Generation of the data for this aspect should be built in tight collaboration with the embedding generator during semantic search query processing. Chunk determination and generation can happen somewhat independently The data in this aspect is directly passed along to the semantic search index.

Design notes:

  • Supports multiple embedding models (e.g., different providers or versions)
  • Supports chunked content for long documents
  • Text field is optional to support privacy-sensitive use cases where only embeddings (not source text) are shared with DataHub
FieldTypeRequiredDescriptionAnnotations
embeddingsmapMap of embedding model name to embedding data. Key is the model identifier (e.g., cohere_embed_v3...

institutionalMemory

Institutional memory of an entity. This is a way to link to relevant documentation and provide description of the documentation. Institutional or tribal knowledge is very important for users to leverage the entity.

FieldTypeRequiredDescriptionAnnotations
elementsInstitutionalMemoryMetadata[]List of records that represent institutional memory of an entity. Each record consists of a link,...

domains

Links from an Asset to its Domains

FieldTypeRequiredDescriptionAnnotations
domainsstring[]The Domains attached to an AssetSearchable, → AssociatedWith
domainAssociationsDomainAssociation[]Additional per-domain association metadata such as attribution and propagation source. A superset...

incidentsSummary

Summary related incidents on an entity.

FieldTypeRequiredDescriptionAnnotations
resolvedIncidentsstring[]Resolved incidents for an asset Deprecated! Use the richer resolvedIncidentsDetails instead.⚠️ Deprecated
activeIncidentsstring[]Active incidents for an asset Deprecated! Use the richer activeIncidentsDetails instead.⚠️ Deprecated
resolvedIncidentDetailsIncidentSummaryDetails[]Summary details about the set of resolved incidentsSearchable, → ResolvedIncidents
activeIncidentDetailsIncidentSummaryDetails[]Summary details about the set of active incidentsSearchable, → ActiveIncidents

versionProperties

Properties about a versioned asset i.e. dataset, ML Model, etc.

FieldTypeRequiredDescriptionAnnotations
versionSetstringThe linked Version Set entity that ties multiple versioned assets togetherSearchable, → VersionOf
versionVersionTagLabel for this versioned asset, is unique within a version setSearchable
aliasesVersionTag[]Associated aliases for this versioned assetSearchable
commentstringComment documenting what this version was created for, changes, or represents
sortIdstringSort identifier that determines where a version lives in the order of the Version Set. What this ...Searchable (versionSortId)
versioningSchemeVersioningSchemeWhat versioning scheme sortId belongs to. Defaults to a plain string that is lexicographically ...
sourceCreatedTimestampAuditStampTimestamp reflecting when this asset version was created in the source system.
metadataCreatedTimestampAuditStampTimestamp reflecting when the metadata for this version was created in DataHub
isLatestbooleanMarks whether this version is currently the latest. Set by a side effect and should not be modifi...Searchable

Common Types

These types are used across multiple aspects in this entity.

AuditStamp

Data captured on a resource/association/sub-resource level giving insight into when that resource/association/sub-resource moved into a particular lifecycle stage, and who acted to move it into that specific lifecycle stage.

Fields:

  • time (long): When did the resource/association/sub-resource move into the specific lifecyc...
  • actor (string): The entity (e.g. a member URN) which will be credited for moving the resource...
  • impersonator (string?): The entity (e.g. a service URN) which performs the change on behalf of the Ac...
  • message (string?): Additional context around how DataHub was informed of the particular change. ...

IncidentSummaryDetails

Summary statistics about incidents on an entity.

Fields:

  • urn (string): The urn of the incident
  • type (string): The type of an incident
  • createdAt (long): The time at which the incident was raised in milliseconds since epoch.
  • resolvedAt (long?): The time at which the incident was marked as resolved in milliseconds since e...
  • priority (int?): The priority of the incident

VersionTag

A resource-defined string representing the resource state for the purpose of concurrency control

Fields:

  • versionTag (string?):
  • metadataAttribution (MetadataAttribution?):

Relationships

Self

These are the relationships to itself, stored in this entity's aspects

  • ClonedFromAgent (via aiAgentInfo.source.clonedFrom)

Outgoing

These are the relationships stored in this entity's aspects

  • AgentHasSkill

    • AgentSkill via aiAgentDependencies.skills
  • AgentUsesTool

    • Api via aiAgentDependencies.tools
  • AgentUsesModel

    • MlModel via aiAgentDependencies.models
  • OwnedBy

    • Corpuser via ownership.owners.owner
    • CorpGroup via ownership.owners.owner
  • ownershipType

    • OwnershipType via ownership.owners.typeUrn
  • DownstreamOf

    • Dataset via upstreamLineage.upstreams.dataset
    • Dataset via upstreamLineage.fineGrainedLineages
    • SchemaField via upstreamLineage.fineGrainedLineages
  • TaggedWith

    • Tag via globalTags.tags
  • TermedWith

    • GlossaryTerm via glossaryTerms.terms.urn
  • AssociatedWith

    • Domain via domains.domains
  • ResolvedIncidents

    • Incident via incidentsSummary.resolvedIncidentDetails
  • ActiveIncidents

    • Incident via incidentsSummary.activeIncidentDetails
  • VersionOf

    • VersionSet via versionProperties.versionSet

Incoming

These are the relationships stored in other entity's aspects

  • IncidentOn

    • Incident via incidentInfo.entities

Global Metadata Model

Global Graph

💡 Contributing to this documentation

This page is auto-generated from the underlying source code. To make changes, please edit the relevant source files in the metadata-models directory.

Tip: For quick typo fixes or documentation updates, you can click the ✏️ Edit icon directly in the GitHub UI to open a Pull Request. For larger changes and PR naming conventions, please refer to our Contributing Guide.