Skip to main content

API

An API is a named callable with a typed input and output schema — an MCP tool, a REST endpoint, a gRPC method, a GraphQL operation, a function, and so on. Cataloging APIs as first-class entities lets both humans and agents discover existing callables before building new ones, and makes caller → API dependencies (the services that compose an API, the agents that invoke it) visible in the graph.

The kind of API is captured with the standard subTypes aspect. The canonical subtypes are MCP_TOOL, REST_ENDPOINT, GRPC_METHOD, GRAPHQL_OPERATION, and FUNCTION — each grounded in a concrete callable kind rather than in how the API happens to be used ("being an agent's tool" is a relationship, not an intrinsic kind — an MCP tool is just an API).

Identity​

APIs are identified by a single field:

  • id: A unique identifier for the API, such as order-lookup-mcp or a generated UUID.

An example URN is urn:li:api:order-lookup-mcp. For REST endpoints the id is minted from the owning service, HTTP method, and path (e.g. order-entry-api/GET/orders/{orderId}) so the same operation resolves to the same URN whether it is registered by hand or scraped from an OpenAPI spec. Because a (method, path) pair identifies one REST operation, an endpoint served under multiple methods (GET vs POST on the same path) is modeled as a distinct API entity per method.

Important Capabilities​

API Properties​

The apiProperties aspect holds catalog identity:

  • name: Display name, searchable with autocomplete.
  • description: What the API does and when to use it.
  • externalUrl: Optional link to the API's registry entry, documentation, or source.
  • sourceRepository: The Repository the API is produced from (SourcedFrom), lighting up the repo → service → api → app → dataset chain.
  • created / lastModified: Audit stamps.

Signature​

The input/output contract lives on a separate apiSignature aspect, kept apart from apiProperties because the signature is scraped from the endpoint (an OpenAPI/JSON-Schema doc, an MCP tool manifest, a function definition) and evolves on its own cadence — when the contract changes, only this aspect is re-ingested, leaving catalog identity untouched. It carries:

  • schemaDefinition: The full input/output schema as an opaque string (typically JSON Schema) — the source-of-truth representation that round-trips even constructs the structured fields can't capture.
  • inputFields / outputFields: A structured, typed view reusing DataHub's schema-field model, so nested/struct/array types, nullability, and field descriptions render with the standard schema components. Each field's fieldPath is the parameter name; nullable=false means a required argument.

REST Properties​

For APIs of subtype REST_ENDPOINT, the restApiProperties aspect records the method (HTTP verb) and path (route template relative to the owning service's base URL, e.g. /orders/{orderId}). These are kept on a subtype-specific aspect so callers can filter by method and group endpoints that share a path.

Relationships​

An API is connected to the rest of the software-to-data graph through incoming edges:

Governance and Versioning​

APIs support the standard governance aspects — ownership, tags, glossary terms, domains, structured properties, and institutional memory — plus native versioning (versionProperties).

Technical Reference Guide​

The sections above provide an overview of how to use this entity. The following sections provide detailed technical information about how metadata is stored and represented in DataHub.

Aspects are the individual pieces of metadata that can be attached to an entity. Each aspect contains specific information (like ownership, tags, or properties) and is stored as a separate record, allowing for flexible and incremental metadata updates.

Relationships show how this entity connects to other entities in the metadata graph. These connections are derived from the fields within each aspect and form the foundation of DataHub's knowledge graph.

Reading the Field Tables​

Each aspect's field table includes an Annotations column that provides additional metadata about how fields are used:

  • ⚠️ Deprecated: This field is deprecated and may be removed in a future version. Check the description for the recommended alternative
  • Searchable: This field is indexed and can be searched in DataHub's search interface
  • Searchable (fieldname): When the field name in parentheses is shown, it indicates the field is indexed under a different name in the search index. For example, dashboardTool is indexed as tool
  • → RelationshipName: This field creates a relationship to another entity. The arrow indicates this field contains a reference (URN) to another entity, and the name indicates the type of relationship (e.g., → Contains, → OwnedBy)

Fields with complex types (like Edge, AuditStamp) link to their definitions in the Common Types section below.

Aspects​

apiProperties​

Properties of an API ��� a named callable with a typed input and output schema (an MCP tool, REST endpoint, gRPC method, GraphQL operation, function, etc.). APIs are cataloged as first-class entities so both humans and agents can discover them before building new ones, and so caller -> API dependencies (services that compose APIs, agents that invoke them) are visible in the graph. The kind of API is captured via the standard subTypes aspect, and the input/output schema lives on the separate apiSignature aspect.

FieldTypeRequiredDescriptionAnnotations
namestring✓Display name of the API.Searchable
descriptionstringDescription of what the API does and when to use it.Searchable
externalUrlstringOptional link to the API's registry entry, documentation, or source.
sourceRepositorystringThe source-code repository this API is produced from (the SourcedFrom provenance edge). Lights up...Searchable, → SourcedFrom
createdAuditStampWhen this API was registered.Searchable
lastModifiedAuditStampWhen this API was last modified.Searchable

apiSignature​

The input/output signature of an API ��� the schema that defines what the callable accepts and returns. Kept separate from apiProperties because the signature is scraped from the endpoint (an OpenAPI/JSON-Schema doc, an MCP tool manifest, a function definition) and evolves on its own cadence: when the endpoint's contract changes, only this aspect is re-ingested, leaving the catalog identity (name, ownership, docs) on apiProperties untouched.

FieldTypeRequiredDescriptionAnnotations
schemaDefinitionstringThe input/output schema stored as an opaque string (typically JSON Schema). The source-of-truth r...
inputFieldsSchemaField[]Typed input parameters of the API's signature (the input schema), reusing DataHub's schema-field ...
outputFieldsSchemaField[]The output shape the API returns when invoked, modeled as schema fields (a scalar return is a sin...

restApiProperties​

REST-specific properties for APIs of subtype REST_ENDPOINT.

Only attached to API entities where subType = REST_ENDPOINT. The (method, path) pair identifies a REST operation, so an endpoint that serves the same path under multiple HTTP methods (e.g. GET vs POST /orders) is modeled as a distinct API entity per method, each carrying its own apiSignature. Kept on this subtype-specific aspect rather than the protocol-agnostic apiProperties so callers can filter by method and group endpoints that share a path.

FieldTypeRequiredDescriptionAnnotations
methodHttpMethod✓The HTTP method this endpoint responds to.Searchable
pathstring✓The route/path template of the endpoint, relative to the owning service's base URL, e.g. "/orders...Searchable

subTypes​

Sub Types. Use this aspect to specialize a generic Entity e.g. Making a Dataset also be a View or also be a LookerExplore

FieldTypeRequiredDescriptionAnnotations
typeNamesstring[]✓The names of the specific types.Searchable

dataPlatformInstance​

The specific instance of the data platform that this entity belongs to

FieldTypeRequiredDescriptionAnnotations
platformstring✓Data PlatformSearchable
instancestringInstance of the data platform (e.g. db instance)Searchable (platformInstance)

ownership​

Ownership information of an entity.

FieldTypeRequiredDescriptionAnnotations
ownersOwner[]✓List of owners of the entity.
ownerTypesmapOwnership type to Owners map, populated via mutation hook.Searchable
lastModifiedAuditStamp✓Audit stamp containing who last modified the record and when. A value of 0 in the time field indi...

status​

The lifecycle status metadata of an entity, e.g. dataset, metric, feature, etc. This aspect is used to represent soft deletes conventionally.

FieldTypeRequiredDescriptionAnnotations
removedboolean✓Whether the entity has been removed (soft-deleted). Kept for backward compatibility. When lifecyc...Searchable
lifecycleStagestringThe lifecycle stage of the entity, referencing a lifecycleStageType entity. When null, the entity...Searchable
lifecycleLastUpdatedAuditStampAttribution for the lifecycle stage transition ��� who moved the entity into its current stage an...

globalTags​

Tag aspect used for applying tags to an entity

FieldTypeRequiredDescriptionAnnotations
tagsTagAssociation[]✓Tags associated with a given entitySearchable, → TaggedWith

glossaryTerms​

Related business terms information

FieldTypeRequiredDescriptionAnnotations
termsGlossaryTermAssociation[]✓The related business terms
auditStampAuditStamp✓Audit stamp containing who reported the related business term

semanticContent​

Semantic content for enabling vector similarity search.

This aspect stores chunked text and embedding vectors for any entity that supports semantic search. Generation of the data for this aspect should be built in tight collaboration with the embedding generator during semantic search query processing. Chunk determination and generation can happen somewhat independently The data in this aspect is directly passed along to the semantic search index.

Design notes:

  • Supports multiple embedding models (e.g., different providers or versions)
  • Supports chunked content for long documents
  • Text field is optional to support privacy-sensitive use cases where only embeddings (not source text) are shared with DataHub
FieldTypeRequiredDescriptionAnnotations
embeddingsmap✓Map of embedding model name to embedding data. Key is the model identifier (e.g., cohere_embed_v3...

institutionalMemory​

Institutional memory of an entity. This is a way to link to relevant documentation and provide description of the documentation. Institutional or tribal knowledge is very important for users to leverage the entity.

FieldTypeRequiredDescriptionAnnotations
elementsInstitutionalMemoryMetadata[]✓List of records that represent institutional memory of an entity. Each record consists of a link,...

domains​

Links from an Asset to its Domains

FieldTypeRequiredDescriptionAnnotations
domainsstring[]✓The Domains attached to an AssetSearchable, → AssociatedWith
domainAssociationsDomainAssociation[]Additional per-domain association metadata such as attribution and propagation source. A superset...

structuredProperties​

Properties about an entity governed by StructuredPropertyDefinition

FieldTypeRequiredDescriptionAnnotations
propertiesStructuredPropertyValueAssignment[]✓Custom property bag.

versionProperties​

Properties about a versioned asset i.e. dataset, ML Model, etc.

FieldTypeRequiredDescriptionAnnotations
versionSetstring✓The linked Version Set entity that ties multiple versioned assets togetherSearchable, → VersionOf
versionVersionTag✓Label for this versioned asset, is unique within a version setSearchable
aliasesVersionTag[]✓Associated aliases for this versioned assetSearchable
commentstringComment documenting what this version was created for, changes, or represents
sortIdstring✓Sort identifier that determines where a version lives in the order of the Version Set. What this ...Searchable (versionSortId)
versioningSchemeVersioningScheme✓What versioning scheme sortId belongs to. Defaults to a plain string that is lexicographically ...
sourceCreatedTimestampAuditStampTimestamp reflecting when this asset version was created in the source system.
metadataCreatedTimestampAuditStampTimestamp reflecting when the metadata for this version was created in DataHub
isLatestbooleanMarks whether this version is currently the latest. Set by a side effect and should not be modifi...Searchable

Common Types​

These types are used across multiple aspects in this entity.

AuditStamp​

Data captured on a resource/association/sub-resource level giving insight into when that resource/association/sub-resource moved into a particular lifecycle stage, and who acted to move it into that specific lifecycle stage.

Fields:

  • time (long): When did the resource/association/sub-resource move into the specific lifecyc...
  • actor (string): The entity (e.g. a member URN) which will be credited for moving the resource...
  • impersonator (string?): The entity (e.g. a service URN) which performs the change on behalf of the Ac...
  • message (string?): Additional context around how DataHub was informed of the particular change. ...

SchemaField​

SchemaField to describe metadata related to dataset schema.

Fields:

  • fieldPath (string): Flattened name of the field. Field is computed from jsonPath field.
  • jsonPath (string?): Flattened name of a field in JSON Path notation.
  • nullable (boolean): Indicates if this field is optional or nullable
  • description (string?): Description
  • label (string?): Label of the field. Provides a more human-readable name for the field than fi...
  • created (AuditStamp?): An AuditStamp corresponding to the creation of this schema field.
  • lastModified (AuditStamp?): An AuditStamp corresponding to the last modification of this schema field.
  • type (SchemaFieldDataType): Platform independent field type of the field.
  • nativeDataType (string): The native type of the field in the dataset's platform as declared by platfor...
  • recursive (boolean): There are use cases when a field in type B references type A. A field in A re...
  • globalTags (GlobalTags?): Tags associated with the field
  • glossaryTerms (GlossaryTerms?): Glossary terms associated with the field
  • isPartOfKey (boolean): For schema fields that are part of complex keys, set this field to true We do...
  • isPartitioningKey (boolean?): For Datasets which are partitioned, this determines the partitioning key. Not...
  • jsonProps (string?): For schema fields that have other properties that are not modeled explicitly,...

VersionTag​

A resource-defined string representing the resource state for the purpose of concurrency control

Fields:

  • versionTag (string?):
  • metadataAttribution (MetadataAttribution?):

Relationships​

Outgoing​

These are the relationships stored in this entity's aspects

  • SourcedFrom

    • Repository via apiProperties.sourceRepository

  • SchemaFieldTaggedWith

    • Tag via apiSignature.inputFields.globalTags
    • Tag via apiSignature.outputFields.globalTags

  • TaggedWith

    • Tag via apiSignature.inputFields.globalTags.tags
    • Tag via apiSignature.outputFields.globalTags.tags
    • Tag via globalTags.tags

  • SchemaFieldWithGlossaryTerm

    • GlossaryTerm via apiSignature.inputFields.glossaryTerms
    • GlossaryTerm via apiSignature.outputFields.glossaryTerms

  • TermedWith

    • GlossaryTerm via apiSignature.inputFields.glossaryTerms.terms.urn
    • GlossaryTerm via apiSignature.outputFields.glossaryTerms.terms.urn
    • GlossaryTerm via glossaryTerms.terms.urn

  • OwnedBy

    • Corpuser via ownership.owners.owner
    • CorpGroup via ownership.owners.owner

  • ownershipType

    • OwnershipType via ownership.owners.typeUrn

  • AssociatedWith

    • Domain via domains.domains

  • VersionOf

    • VersionSet via versionProperties.versionSet

Incoming​

These are the relationships stored in other entity's aspects

  • Consumes

    • Application via applicationLineage.inputEdges

  • Produces

    • Application via applicationLineage.outputEdges

  • ServiceComposesApi

    • Service via serviceProperties.apis

  • AgentUsesTool

    • AiAgent via aiAgentDependencies.tools

Global Metadata Model​

Global Graph

💡 Contributing to this documentation

This page is auto-generated from the underlying source code. To make changes, please edit the relevant source files in the metadata-models directory.

Tip: For quick typo fixes or documentation updates, you can click the ✏️ Edit icon directly in the GitHub UI to open a Pull Request. For larger changes and PR naming conventions, please refer to our Contributing Guide.