Skip to main content
Version: Next

Application

The application entity represents software applications, services, or systems that produce or consume data assets in your data ecosystem. Applications serve as a way to organize and group related data assets by the software that creates, transforms, or uses them, enabling better understanding of data provenance and usage patterns.

Identity​

Applications are identified by a single piece of information:

  • The unique identifier (id) for the application: this is a string that uniquely identifies the application within your DataHub instance. It can be any meaningful identifier such as an application name, service name, or system identifier. Common patterns include using kebab-case names (e.g., customer-analytics-service), fully qualified service names (e.g., com.company.analytics.customer-service), or simple descriptive names (e.g., tableau-reports).

An example of an application identifier is urn:li:application:customer-analytics-service.

Unlike datasets which require platform and environment qualifiers, applications use a simpler URN structure with just the application identifier, making them flexible for representing any type of software application across your organization.

Important Capabilities​

Application Properties​

The core metadata about an application is stored in the applicationProperties aspect. This aspect contains:

  • Name: A human-readable display name for the application (e.g., "Customer Analytics Service", "Data Pipeline Orchestrator")
  • Description: Documentation describing what the application does, its purpose, and how it's used
  • Custom Properties: Key-value pairs for storing additional application-specific metadata
  • External References: Links to external documentation, source code repositories, dashboards, or other relevant resources

The following code snippet shows you how to create an application with basic properties.

Python SDK: Create an application
# Inlined from /metadata-ingestion/examples/library/application_create.py
# metadata-ingestion/examples/library/application_create.py
import os

from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import ApplicationPropertiesClass


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


gms_server = os.getenv("DATAHUB_GMS_URL", "http://localhost:8080")
token = os.getenv("DATAHUB_GMS_TOKEN")
emitter = DatahubRestEmitter(gms_server=gms_server, token=token)

application_urn = make_application_urn("customer-analytics-service")

application_properties = ApplicationPropertiesClass(
    name="Customer Analytics Service",
    description="A microservice that processes customer events and generates analytics insights for the marketing team",
    customProperties={
        "team": "data-platform",
        "language": "python",
        "repository": "https://github.com/company/customer-analytics",
    },
    externalUrl="https://wiki.company.com/customer-analytics",
)

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=application_properties,
)
emitter.emit(metadata_event)

print(f"Created application: {application_urn}")

Updating Application Properties​

You can update the properties of an existing application by emitting a new applicationProperties aspect. The SDK will merge the new properties with existing ones.

Python SDK: Update application properties
# Inlined from /metadata-ingestion/examples/library/application_update_properties.py
# metadata-ingestion/examples/library/application_update_properties.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import ApplicationPropertiesClass


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

updated_properties = ApplicationPropertiesClass(
    name="Customer Analytics Service v2",
    description="Updated: A microservice that processes customer events and generates real-time analytics insights. Now includes ML-based predictions.",
    customProperties={
        "team": "data-platform",
        "language": "python",
        "repository": "https://github.com/company/customer-analytics",
        "version": "2.0.0",
        "deployment": "kubernetes",
    },
    externalUrl="https://wiki.company.com/customer-analytics-v2",
)

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=updated_properties,
)
emitter.emit(metadata_event)

print(f"Updated properties for application: {application_urn}")

Tags and Glossary Terms​

Applications can have Tags or Terms attached to them, just like other entities. Read this blog to understand the difference between tags and terms so you understand when you should use which.

Adding Tags to an Application​

Tags are added to applications using the globalTags aspect. Tags are useful for categorizing applications by technology stack, team ownership, criticality level, or other organizational dimensions.

Python SDK: Add a tag to an application
# Inlined from /metadata-ingestion/examples/library/application_add_tag.py
# metadata-ingestion/examples/library/application_add_tag.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import GlobalTagsClass, TagAssociationClass


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


def make_tag_urn(tag_name: str) -> str:
    """Create a DataHub tag URN."""
    return f"urn:li:tag:{tag_name}"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

tag_to_add = make_tag_urn("production")

tags = GlobalTagsClass(
    tags=[
        TagAssociationClass(tag=tag_to_add),
    ]
)

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=tags,
)
emitter.emit(metadata_event)

print(f"Added tag {tag_to_add} to application {application_urn}")

Adding Glossary Terms to an Application​

Glossary terms are added using the glossaryTerms aspect. Terms are useful for associating applications with business concepts, data domains, or standardized terminology.

Python SDK: Add a term to an application
# Inlined from /metadata-ingestion/examples/library/application_add_term.py
# metadata-ingestion/examples/library/application_add_term.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import (
    AuditStampClass,
    GlossaryTermAssociationClass,
    GlossaryTermsClass,
)


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


def make_term_urn(term_name: str) -> str:
    """Create a DataHub glossary term URN."""
    return f"urn:li:glossaryTerm:{term_name}"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

term_to_add = make_term_urn("CustomerData")

terms = GlossaryTermsClass(
    terms=[
        GlossaryTermAssociationClass(urn=term_to_add),
    ],
    auditStamp=AuditStampClass(time=0, actor="urn:li:corpuser:datahub"),
)

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=terms,
)
emitter.emit(metadata_event)

print(f"Added term {term_to_add} to application {application_urn}")

Ownership​

Ownership is associated to an application using the ownership aspect. Owners can be of a few different types, TECHNICAL_OWNER, BUSINESS_OWNER, DATA_STEWARD, etc. See OwnershipType.pdl for the full list of ownership types and their meanings.

The following script shows you how to add an owner to an application using the low-level Python SDK.

Python SDK: Add an owner to an application
# Inlined from /metadata-ingestion/examples/library/application_add_owner.py
# metadata-ingestion/examples/library/application_add_owner.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import (
    OwnerClass,
    OwnershipClass,
    OwnershipTypeClass,
)


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


def make_user_urn(username: str) -> str:
    """Create a DataHub user URN."""
    return f"urn:li:corpuser:{username}"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

owner_to_add = make_user_urn("jdoe")

ownership = OwnershipClass(
    owners=[
        OwnerClass(
            owner=owner_to_add,
            type=OwnershipTypeClass.TECHNICAL_OWNER,
        )
    ]
)

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=ownership,
)
emitter.emit(metadata_event)

print(f"Added owner {owner_to_add} to application {application_urn}")

Domains​

Applications can be associated with domains to organize them by business area, department, or data domain. Domains are set using the domains aspect.

Python SDK: Add a domain to an application
# Inlined from /metadata-ingestion/examples/library/application_add_domain.py
# metadata-ingestion/examples/library/application_add_domain.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import DomainsClass


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


def make_domain_urn(domain_id: str) -> str:
    """Create a DataHub domain URN."""
    return f"urn:li:domain:{domain_id}"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

domain_to_add = make_domain_urn("marketing")

domains = DomainsClass(domains=[domain_to_add])

metadata_event = MetadataChangeProposalWrapper(
    entityUrn=application_urn,
    aspect=domains,
)
emitter.emit(metadata_event)

print(f"Added domain {domain_to_add} to application {application_urn}")

Application Assets (Membership)​

One of the most powerful features of applications is the ability to associate them with data assets. This creates a relationship between the application and the datasets, dashboards, charts, or other entities that the application produces or consumes.

The applications aspect is attached to the data assets (not the application entity itself) to indicate which applications are associated with those assets. This bidirectional relationship enables:

  • Finding all data assets associated with a specific application
  • Discovering which applications produce or consume a particular dataset
  • Tracking data lineage through application boundaries
  • Understanding the full scope of an application's data footprint
Python SDK: Associate assets with an application
# Inlined from /metadata-ingestion/examples/library/application_add_assets.py
# metadata-ingestion/examples/library/application_add_assets.py
from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import ApplicationsClass


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


def make_dataset_urn(platform: str, name: str, env: str) -> str:
    """Create a DataHub dataset URN."""
    return f"urn:li:dataset:(urn:li:dataPlatform:{platform},{name},{env})"


emitter = DatahubRestEmitter(gms_server="http://localhost:8080")

application_urn = make_application_urn("customer-analytics-service")

dataset_urns = [
    make_dataset_urn("snowflake", "prod.marketing.customer_events", "PROD"),
    make_dataset_urn("snowflake", "prod.marketing.customer_profiles", "PROD"),
    make_dataset_urn("kafka", "customer-events-stream", "PROD"),
]

for dataset_urn in dataset_urns:
    applications_aspect = ApplicationsClass(applications=[application_urn])

    metadata_event = MetadataChangeProposalWrapper(
        entityUrn=dataset_urn,
        aspect=applications_aspect,
    )
    emitter.emit(metadata_event)

    print(f"Associated {dataset_urn} with application {application_urn}")

print(f"\nSuccessfully associated {len(dataset_urns)} assets with application")

Querying Applications​

Applications can be queried using the standard DataHub REST API to retrieve all aspects and metadata.

Fetch application entity via REST API
# Inlined from /metadata-ingestion/examples/library/application_query_rest_api.py
# metadata-ingestion/examples/library/application_query_rest_api.py
import json
import os
from urllib.parse import quote

import requests


def make_application_urn(application_id: str) -> str:
    """Create a DataHub application URN."""
    return f"urn:li:application:{application_id}"


gms_server = os.getenv("DATAHUB_GMS_URL", "http://localhost:8080")
token = os.getenv("DATAHUB_GMS_TOKEN")
application_urn = make_application_urn("customer-analytics-service")

encoded_urn = quote(application_urn, safe="")

headers = {}
if token:
    headers["Authorization"] = f"Bearer {token}"

response = requests.get(f"{gms_server}/entities/{encoded_urn}", headers=headers)

if response.status_code == 200:
    entity_data = response.json()
    print(f"Application: {application_urn}")
    print(json.dumps(entity_data, indent=2))

    if "aspects" in entity_data and "applicationProperties" in entity_data["aspects"]:
        props = entity_data["aspects"]["applicationProperties"]["value"]
        print(f"\nApplication Name: {props.get('name')}")
        print(f"Description: {props.get('description')}")
        if "customProperties" in props:
            print(
                f"Custom Properties: {json.dumps(props['customProperties'], indent=2)}"
            )
else:
    print(f"Failed to fetch application: {response.status_code} - {response.text}")

Or using curl:

curl 'http://localhost:8080/entities/urn%3Ali%3Aapplication%3Acustomer-analytics-service'

You can also query for all assets associated with an application using the search API:

Find all assets associated with an application
# Using the search API to find entities with a specific application
curl -X POST 'http://localhost:8080/entities?action=search' \
  -H 'Content-Type: application/json' \
  -d '{
    "input": "*",
    "entity": "dataset",
    "start": 0,
    "count": 10,
    "filter": {
      "or": [
        {
          "and": [
            {
              "field": "applications",
              "value": "urn:li:application:customer-analytics-service",
              "condition": "EQUAL"
            }
          ]
        }
      ]
    }
  }'

Integration Points​

Relationships with Other Entities​

Applications can be associated with multiple types of entities in DataHub:

  1. Datasets: The most common use case is associating applications with the datasets they produce or consume. This helps track data provenance and understand which applications are upstream or downstream of specific datasets.

  2. Dashboards and Charts: Applications that generate reports or visualizations can be linked to dashboard and chart entities, making it clear which application produces which analytics artifacts.

  3. Data Jobs and Data Flows: ETL pipelines and data processing applications can be associated with the dataJob and dataFlow entities they orchestrate or execute.

  4. ML Models and ML Features: Machine learning applications can be linked to the models and features they train or use for inference.

  5. Domains: Applications can belong to business domains, helping organize them by organizational structure or business function.

  6. Ownership: Applications have owners (technical owners, business owners) who are responsible for maintaining and operating them.

Common Usage Patterns​

Pattern 1: Service-to-Dataset Mapping​

The most common pattern is mapping microservices or applications to the datasets they produce:

Customer Service Application → produces → customer_events dataset
                             → produces → customer_profiles dataset
                             → consumes → user_authentication dataset

This pattern is typically implemented during ingestion by:

  • Adding the applications aspect to each dataset
  • Referencing the application URN in the aspect

Pattern 2: Reporting Application Organization​

Business intelligence and reporting tools can organize their dashboards and charts by application:

Tableau Application → contains → Sales Dashboard
                    → contains → Marketing Dashboard
Power BI Application → contains → Finance Dashboard

Pattern 3: Data Pipeline Grouping​

Data pipeline orchestration systems can group related jobs under an application:

Airflow Application → orchestrates → daily_etl_pipeline
                    → orchestrates → hourly_refresh_pipeline
                    → orchestrates → weekly_aggregation_pipeline

GraphQL API​

Applications are fully integrated with DataHub's GraphQL API, enabling:

  • Creating applications via createApplication mutation
  • Updating application properties
  • Deleting applications via deleteApplication mutation
  • Batch setting application assets via batchSetApplication mutation
  • Querying applications and their relationships

The GraphQL resolvers for applications are located in datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/resolvers/application/.

Notable Exceptions​

Application vs Data Product​

Applications and Data Products are related but distinct concepts:

  • Applications represent software systems that produce or consume data. They are technical entities representing code and infrastructure.
  • Data Products represent curated data assets packaged for consumption. They are business entities representing data as a product.

An application might produce multiple data products, and a data product might be produced by multiple applications working together.

Application vs Platform​

Applications should not be confused with Data Platforms:

  • Data Platform (e.g., Snowflake, BigQuery, Kafka) represents the technology or system that hosts data assets.
  • Application represents the specific software application that creates or uses those data assets.

For example:

  • Platform: snowflake
  • Dataset: urn:li:dataset:(urn:li:dataPlatform:snowflake,database.schema.table,PROD)
  • Application: urn:li:application:customer-analytics-service (which writes to that Snowflake table)

UUID-based Identifiers​

While the application ID can be any string, using UUIDs is supported but not required. If you don't provide an ID when creating an application through the ApplicationService, a UUID will be automatically generated. However, meaningful, human-readable identifiers are generally preferred for better discoverability and maintenance.

Future Enhancements​

The application entity is relatively new and may be expanded in the future to include:

  • Service mesh integration for automatic discovery
  • API specifications and endpoint documentation
  • Deployment and version tracking
  • Resource consumption metrics
  • Dependency graph visualization

Technical Reference Guide​

The sections above provide an overview of how to use this entity. The following sections provide detailed technical information about how metadata is stored and represented in DataHub.

Aspects are the individual pieces of metadata that can be attached to an entity. Each aspect contains specific information (like ownership, tags, or properties) and is stored as a separate record, allowing for flexible and incremental metadata updates.

Relationships show how this entity connects to other entities in the metadata graph. These connections are derived from the fields within each aspect and form the foundation of DataHub's knowledge graph.

Reading the Field Tables​

Each aspect's field table includes an Annotations column that provides additional metadata about how fields are used:

  • ⚠️ Deprecated: This field is deprecated and may be removed in a future version. Check the description for the recommended alternative
  • Searchable: This field is indexed and can be searched in DataHub's search interface
  • Searchable (fieldname): When the field name in parentheses is shown, it indicates the field is indexed under a different name in the search index. For example, dashboardTool is indexed as tool
  • → RelationshipName: This field creates a relationship to another entity. The arrow indicates this field contains a reference (URN) to another entity, and the name indicates the type of relationship (e.g., → Contains, → OwnedBy)

Fields with complex types (like Edge, AuditStamp) link to their definitions in the Common Types section below.

Aspects​

applicationProperties​

The main properties of an Application

FieldTypeRequiredDescriptionAnnotations
customPropertiesmap✓Custom property bag.Searchable
externalUrlstringURL where the reference existSearchable
namestringDisplay name of the ApplicationSearchable
descriptionstringDocumentation of the applicationSearchable

ownership​

Ownership information of an entity.

FieldTypeRequiredDescriptionAnnotations
ownersOwner[]✓List of owners of the entity.
ownerTypesmapOwnership type to Owners map, populated via mutation hook.Searchable
lastModifiedAuditStamp✓Audit stamp containing who last modified the record and when. A value of 0 in the time field indi...

glossaryTerms​

Related business terms information

FieldTypeRequiredDescriptionAnnotations
termsGlossaryTermAssociation[]✓The related business terms
auditStampAuditStamp✓Audit stamp containing who reported the related business term

globalTags​

Tag aspect used for applying tags to an entity

FieldTypeRequiredDescriptionAnnotations
tagsTagAssociation[]✓Tags associated with a given entitySearchable, → TaggedWith

domains​

Links from an Asset to its Domains

FieldTypeRequiredDescriptionAnnotations
domainsstring[]✓The Domains attached to an AssetSearchable, → AssociatedWith

institutionalMemory​

Institutional memory of an entity. This is a way to link to relevant documentation and provide description of the documentation. Institutional or tribal knowledge is very important for users to leverage the entity.

FieldTypeRequiredDescriptionAnnotations
elementsInstitutionalMemoryMetadata[]✓List of records that represent institutional memory of an entity. Each record consists of a link,...

status​

The lifecycle status metadata of an entity, e.g. dataset, metric, feature, etc. This aspect is used to represent soft deletes conventionally.

FieldTypeRequiredDescriptionAnnotations
removedboolean✓Whether the entity has been removed (soft-deleted).Searchable

structuredProperties​

Properties about an entity governed by StructuredPropertyDefinition

FieldTypeRequiredDescriptionAnnotations
propertiesStructuredPropertyValueAssignment[]✓Custom property bag.

forms​

Forms that are assigned to this entity to be filled out

FieldTypeRequiredDescriptionAnnotations
incompleteFormsFormAssociation[]✓All incomplete forms assigned to the entity.Searchable
completedFormsFormAssociation[]✓All complete forms assigned to the entity.Searchable
verificationsFormVerificationAssociation[]✓Verifications that have been applied to the entity via completed forms.Searchable

testResults​

Information about a Test Result

FieldTypeRequiredDescriptionAnnotations
failingTestResult[]✓Results that are failingSearchable, → IsFailing
passingTestResult[]✓Results that are passingSearchable, → IsPassing

subTypes​

Sub Types. Use this aspect to specialize a generic Entity e.g. Making a Dataset also be a View or also be a LookerExplore

FieldTypeRequiredDescriptionAnnotations
typeNamesstring[]✓The names of the specific types.Searchable

Common Types​

These types are used across multiple aspects in this entity.

AuditStamp​

Data captured on a resource/association/sub-resource level giving insight into when that resource/association/sub-resource moved into a particular lifecycle stage, and who acted to move it into that specific lifecycle stage.

Fields:

  • time (long): When did the resource/association/sub-resource move into the specific lifecyc...
  • actor (string): The entity (e.g. a member URN) which will be credited for moving the resource...
  • impersonator (string?): The entity (e.g. a service URN) which performs the change on behalf of the Ac...
  • message (string?): Additional context around how DataHub was informed of the particular change. ...

FormAssociation​

Properties of an applied form.

Fields:

  • urn (string): Urn of the applied form
  • incompletePrompts (FormPromptAssociation[]): A list of prompts that are not yet complete for this form.
  • completedPrompts (FormPromptAssociation[]): A list of prompts that have been completed for this form.

TestResult​

Information about a Test Result

Fields:

  • test (string): The urn of the test
  • type (TestResultType): The type of the result
  • testDefinitionMd5 (string?): The md5 of the test definition that was used to compute this result. See Test...
  • lastComputed (AuditStamp?): The audit stamp of when the result was computed, including the actor who comp...

Relationships​

Outgoing​

These are the relationships stored in this entity's aspects

  • OwnedBy

    • Corpuser via ownership.owners.owner
    • CorpGroup via ownership.owners.owner

  • ownershipType

    • OwnershipType via ownership.owners.typeUrn

  • TermedWith

    • GlossaryTerm via glossaryTerms.terms.urn

  • TaggedWith

    • Tag via globalTags.tags

  • AssociatedWith

    • Domain via domains.domains

  • IsFailing

    • Test via testResults.failing

  • IsPassing

    • Test via testResults.passing

Incoming​

These are the relationships stored in other entity's aspects

  • AssociatedWith

    • Dataset via applications.applications
    • DataJob via applications.applications
    • DataFlow via applications.applications
    • Chart via applications.applications
    • Dashboard via applications.applications
    • Notebook via applications.applications
    • Container via applications.applications
    • GlossaryTerm via applications.applications
    • MlModel via applications.applications
    • MlModelGroup via applications.applications
    • MlFeatureTable via applications.applications
    • MlFeature via applications.applications
    • MlPrimaryKey via applications.applications
    • DataProduct via applications.applications

Global Metadata Model​

Global Graph