# v2.3.0 ## Release Changelog ### v2.3.1 This release rolls up hotfixes on top of v2.3.0. #### Release Availability Date TBD #### Recommended Versions - **CLI/SDK**: 1.7.0.14 - **Remote Executor**: - **Image**: v2.3.1-cloud, v2.3.0-cloud, v2.2.3-cloud, v2.2.2-cloud, v2.2.1-cloud, v2.2.0-cloud, v2.1.5-cloud, v2.1.4-cloud, v2.1.3-cloud, v2.1.2-cloud, v2.1.1-cloud, v2.1.0-cloud - **Helm chart**: 0.0.70 - **On-Prem Versions**: - **Helm**: 2.0.78 - **API Gateway**: v0.7.3 #### New Feature Highlights - **Ask DataHub @-mentions** — type `@` in the Ask DataHub composer to pick a DataHub asset and send the agent its URN. - **AI credits by feature** — the billing dashboard can split AI credit usage by consumer, in billed credits per day. - **Eval Workbench** — an opt-in **Fix** button on the Evals page hands you a prompt to run the local curator against this DataHub. #### Product - **Ask DataHub @-mentions** — typing `@` opens a picker. Search by name from 3 characters, narrow by type, domain, platform, container, owner, or tag, and insert the asset as a mention so the agent gets the exact URN. The change is on for everyone; it is not behind a flag. - **Generated Context documents are locked** — a semantic-anchor document's contents, title, and type can no longer be edited. Publish and unpublish, comments, owners, tags, move, and delete still work. The document and Context Hub review show a lock banner plus Evidence and Advanced sections. Review of a generated document is approve/reject, or the publish toggle, only. - **Explicit document save** — On for Context public beta customers. When on, document body edits stay local until **Save**, with **Cancel / Propose / Save** and a prompt if you navigate away with unsaved edits. - **Billing AI credits by consumer** — the AI Credits chart's "Group by: Consumer" view shows which features used credits (Ask DataHub chat, custom agents, memory synthesis, description generation, and the other shipped modules), per day, in credits. Amounts come from priced invoice line items, so they match the burn-down. The consumer list is `BILLING_DASHBOARD_AI_MODULES` (default: every AI module the product ships). Amounts under 0.01 display as "<0.01" instead of zero. - **Billing dashboard corrections** — burn-down charts draw a point when credits or commitments are granted or expire, including a renewal on a day with no usage, and mid-day grant timestamps line up with the chart's UTC days. A contract that prices AI usage without an included credits pool shows credits consumed (card and chart, no burn-down or allowance), using `BILLING_DASHBOARD_AI_CREDITS_CREDIT_UNITS` (default `AI Credits`) as the invoice credit unit. Usage-over-time charts appear for the standard categories whether or not the contract covers them. The chart view offers every granularity the selected span supports (daily, weekly, monthly). The usage-card health bar turns red only at or over the limit, and orange while approaching. Chart tooltips no longer flicker at the edge of a chart. - **Stats tab scope** — a sample or partition profile is labeled with that scope and the report date, instead of reading as whole-table stats. Row count, storage, and query-count charts widen to include the latest profile or query when it is older than the default window, instead of saying the asset was never profiled or queried. See [datahub-project/datahub#20002](https://github.com/datahub-project/datahub/pull/20002). - **Data product lineage** — members with no upstream or downstream lineage are hidden, so the view shows assets that are connected. - **Select Assets lists** — in the view builder's Select Assets tab and the asset collection module, the selected-assets list no longer flickers empty on each select or deselect, and the result lists scroll inside the modal instead of being clipped on short screens. #### AI / Ask DataHub - **Trust-tiered SQL retrieval** — off by default (`SQL_CONTEXT_TRUST_TIERING_ENABLED`). When on, a high-scoring tableset can be nominated on the SQL routing card, published anchors are treated as reviewed, and the card gains a trust-policy table filter plus `scope_locked` and `complexity`. With the flag off, the card keeps its previous shape. - **Eval Workbench** — Shows the **Fix** button on the Evals page, and the Eval Workbench guide in the Help Center (which also needs `showContextHub`). The button stays disabled until the instance has at least one eval. It opens a prompt to paste into Claude Code, which installs the curator with `acryl-datahub-cloud evals workbench install` and runs the curation-assistant workflow. Turn it on only where the integrations service serves the curator and the LLM gateway is enabled. - **Evals: Score an answer without recording a run** — `acryl-datahub-cloud evals judge URN --answer TEXT|-` scores an answer against a saved eval's question, reference answers, and conditions, using the same judge as native runs, and prints the verdict. It records nothing, so the answer does not appear in run history or the pass rate. Keep using `evals report` for a real external eval run. `judge` calls `POST /openapi/v1/eval/judge` and needs the Manage Evals or Manage Agents privilege. - **Thinking while an agent turn runs** — `get_agent_response` returns the agent's thinking messages as they stream, in `thinking`, with a `thinking_cursor`. Pass the cursor back as `thinking_after` on the next poll. Only the replica running the turn can report them. A poll returns at most `AGENT_RUN_MAX_THINKING_PER_POLL` messages (default 20), the newest ones. - **Search tool time fields** — the search tool documents `lastModifiedAt`, `createdAt`, and dataset `lastOperationTime` for filters and sorts (for example `lastModifiedAt >= 2026-01-31`), so an agent can answer "what changed since yesterday" without guessing field names. #### Platform - **Documentation text in the search index** — `documentation` aspect entries are indexed as `attributedDescriptions`. Default and UI search still ignore that field. Entries written before the upgrade are backfilled by a non-blocking system-update step that reindexes `documentation` one entity type at a time (`BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_ENABLED`, default `true`; `BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_BATCH_SIZE`, `BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_DELAY_MS`, and `BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_LIMIT` throttle it). The backfill only rewrites search documents: it emits no entity change events and does not trigger documentation propagation. `schemaField` is skipped. Until the step finishes, agents find only documentation written after the upgrade. **Action:** none by default. If you disable the step, or you add `schemaField` to the default search entity types, run a one-time `RestoreIndices` scoped to `aspectNames=documentation` after the upgrade. See [Search and Graph Reindexing](../../how/restore-indices.md). - **Document usage is collected by default** — the usage reporter records how often documents are read, by people and by agents, and uses that to rank documents in search. If a usage index is missing, or the check for that index fails, that part of the job is skipped and the rest of usage reporting still runs. To turn it off, set `document_usage_stats_enabled` to the string `"false"` in `DATAHUB_USAGE_REPORTING_BOOTSTRAP_VALUES`. A JSON boolean `false` is treated as unset, and the feature stays on. The usage-reporting bootstrap recipe moves to v7, which replaces the managed usage-reporting ingestion source on the next system update. Edits made in the UI that are not in that env var are overwritten; values in the env var are kept. - **Frontend health probes and upstream cap** — the Play frontend caps concurrent requests that wait on an upstream (`DATAHUB_FRONTEND_PROXY_MAX_IN_FLIGHT`, default 1024). GMS (`/api`, `/openapi`), login and signup, SSO, and in-flight OTEL forwards share that budget. Browser trace export keeps its own cap of 100. Above the cap, those requests return **503** with `Retry-After: 1` instead of holding a Play connection. The management listener (`MANAGEMENT_SERVER_PORT`, default 4319) serves `GET /health/live` (200 while the process can answer) and `GET /health/ready` (503 while starting, shutting down, or while in-flight upstream calls are above 90% of the cap; ready again at or below 70%). Docker `HEALTHCHECK` tries `/health/live` and falls back to `GET /admin` on the Play port. `GET /admin` and `GET /health` on port 9002 still return `200 GOOD` (or `503` during graceful shutdown) and are not subject to the cap. **Action:** none for current Helm charts, which still probe `/admin` on 9002. Point liveness at `/health/live` and readiness at `/health/ready` on port 4319 when the chart is updated. Clients that send large bursts through the frontend may see 503 until in-flight calls drain; raise `DATAHUB_FRONTEND_PROXY_MAX_IN_FLIGHT` if 1024 is too low. See **Deprecations**. - **Active user counts** — weekly and monthly active-user highlights count distinct users again, and exclude the `admin` user, instead of distinct browsers. #### Ingestion **New ingestion sources:** - **Context document extraction (Onto Docs)** — `datahub-onto-docs` reads Context documents, decides which tables each section describes, and attaches that section's text to the table as a `documentation` entry. The entry is the document's own words under a short header naming the document and section. It needs an executor that can install `acryl-datahub-integrations`. - **Scope:** by default it processes every document that is published or has no lifecycle stage. System documents, semantic anchors, AI Memory, and bridge documents are never processed. `document_subtypes`, `document_platform_filter`, `document_source_types`, and `document_lifecycle_stages` narrow this; a document that falls outside them, or is deleted, has its entries removed. - **Re-runs:** re-running the source replaces a document's entries and removes them from tables it no longer describes. Runs skip documents whose content is unchanged, so most runs make no LLM calls. Remove every entry from one document with `python -m datahub_integrations.gen_ai.onto_docs --rollback --write`. - **Enabling:** the source is registered as the hidden system ingestion source `urn:li:dataHubIngestionSource:datahub-onto-docs` and is not scheduled until you opt in: set `DATAHUB_ONTO_DOCS_BOOTSTRAP_VALUES` to `{"enabled": true}` (hourly by default; `schedule.interval`, `ingestion.platform_filter`, `ingestion.llm_model`, and `ingestion.max_documents` are optional). System update applies these values once per template version. To enable, disable, or reconfigure later, also set `DATAHUB_ONTO_DOCS_REVISION` to a new value (for example `{"version":"v1-2"}`) and rerun system update. Otherwise the new values are ignored and the existing schedule keeps running. - **Action:** none unless you want to enable it. To turn it off before a downgrade, run that rollback for each processed document first: older versions do not recognize these entries and would show them as table descriptions. **Ingestion infrastructure:** - **Managed CLI/SDK** in the executor and integrations service is **1.7.0.14** (previously 1.7.0.10). **Executor:** - **Connection tests** run on the Kafka worker, on the same path as ingestion. - **pycurl** is **7.48.0**, installed from the published wheel. The executor image no longer builds pycurl from source. The wheel carries its own OpenSSL and does not load the host OpenSSL config, which is what previously forced the source build. - **OpenSSL 3 hold** — the actions, executor, and integrations images pin Python, and the executor pins libcurl, to their last OpenSSL 3 builds. That keeps Snowflake ingestion working (its client imports a TLS constant removed from the OpenSSL 4 Python build) and keeps the executor's curl packages installable. #### Breaking Changes - **OAuth2 refresh tokens for public clients** — a public OAuth client (PKCE, no client secret) now has one session per login across refresh-token rotation, and only the newest refresh token works. Using an older refresh token from the same login is treated as token theft: the request fails with `invalid_grant` and the login is revoked, so the user must sign in again. Clients that store and use the refresh token returned by each refresh are not affected. Refresh tokens issued before this release are not tracked and expire as before. The token endpoint also no longer accepts an access token as `refresh_token`. #### Deprecations - **`GET /admin` and `GET /health` on the Play port (9002) are deprecated as health checks.** They still return `200 GOOD` (or `503` during graceful shutdown) so existing monitors keep working, but they run on the Play connection table and can time out while the process is up. Use `GET /health/live` and `GET /health/ready` on `MANAGEMENT_SERVER_PORT` (default 4319). **Action:** point Kubernetes liveness and readiness, Docker healthchecks, and any other prober at those management paths. Do not add new checks against `/admin`. The route will be removed in a later release after charts and monitors have moved. - **`SMART_SEARCH_POOL_SATURATION_LOG_INTERVAL_S` is no longer read.** Excess SQL-context retrieval work runs inline instead of waiting in the previous expansion queue. #### Security / Dependencies - **setuptools**: floor raised to 83.0.0 (locked at 84.0.0 in the executor and integrations images) to address CVE-2026-59890. - **PyJWT**: 2.13.0 → 2.15.1 in the executor and integrations images. - **urllib3**: 2.7.0 → 2.8.0 in the executor and integrations locks. #### Bug Fixes - Fixed the Stats tab treating a sample or partition profile as whole-table stats, and claiming an asset was never profiled when its only profiles were older than the chart window. - Fixed data product lineage drawing members that have no lineage. - Fixed the Select Assets list flickering on every selection change, and the last rows being clipped on short screens. - Fixed weekly and monthly active-user highlights counting browsers instead of users. - Fixed document usage reporting creating a stub document for a read of a deleted or never-saved document, which then blocked creating a document with that id. Usage is written only for indexed documents that have `documentInfo`. - Fixed per-user AI Memory documents appearing as instructions on the SQL routing card. - Fixed description cleanup deleting comparison text such as `a < b` inside SQL embedded in a description. Embedded data-URI images are still removed. - Fixed an agent request for an asset's `description` returning a 5,000-character cut. The full text is returned up to 20,000 characters, then a `[+N chars]` marker. - Fixed large imported model definitions pushing `find_sql_context` responses past client limits. - Fixed GMS debug logging building large strings even when debug logging was off, which could stall the process. - Fixed keyword search on structured property values. v2.3.0 stopped copying those values into the full-text fields, so a search for a property value matched nothing. **Action:** a mapping change does not fill in values already indexed. With structured-property system update enabled, a `copy_to` mismatch reindexes those indexes. Where that system update is off, reindex the affected indexes yourself. Do not run `restoreIndices` with `clean` on them. - Fixed search-filter usage events failing to index because their `values` field collided with structured-property values. The field is `filterValues`. - Fixed connection tests (`TEST_CONNECTION`) not running on the executor's Kafka worker. - Fixed actions, executor, and integrations images failing Snowflake ingestion after the base Python build moved to OpenSSL 4. #### Known Issues - None known at release time. #### Environment Variables - **`SHOW_EVAL_WORKBENCH`** (GMS, default `false`) — shows the **Fix** button on the Evals page and the Eval Workbench guide in the Help Center. Also needs `showContextHub`. The button is disabled until the instance has at least one eval. - **`TEXT_TO_SQL_ENABLED`** (integrations service, default `false`) — registers `text_to_sql` and selects it for `datahub agent create snowflake --sql-mode auto`. - **`SQL_CONTEXT_TRUST_TIERING_ENABLED`** (integrations service, default `false`) — trust-tiered SQL retrieval. Off keeps the previous routing-card shape. - **`DOCUMENT_EXPLICIT_SAVE_ENABLED`** (GMS, default `false`) — document body edits stay local until the user clicks **Save**. - **`MCP_RETRIEVAL_POOL_WORKERS`** (integrations service, default 64) — worker threads shared by SQL-context retrieval, including keyword expansion. If unset, `SMART_SEARCH_EXPANSION_BASE_POOL_WORKERS` supplies the size. - **`MCP_RETRIEVAL_REQUEST_MAX_WORKERS`** (integrations service, default 16) — maximum pool workers for one `find_sql_context` call, including nested retrieval. When the pool is full, work runs on the caller. These limits do not cap caller threads. - **`INSPECT_EXTRACTED_DOC_CHAR_CAP`** (integrations service, default 3000) — maximum characters of extracted Context-document text returned per table by `inspect_tables_for_sql`. - **`DATAHUB_ONTO_DOCS_BOOTSTRAP_VALUES`** — set to `{"enabled": true}` to schedule the Context document extraction source. Optional keys: `schedule.interval`, `ingestion.platform_filter`, `ingestion.llm_model`, `ingestion.max_documents`. - **`DATAHUB_ONTO_DOCS_REVISION`** — set to a new value, such as `{"version":"v1-2"}`, and rerun system update whenever you change `DATAHUB_ONTO_DOCS_BOOTSTRAP_VALUES` after the first apply. - **`BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_ENABLED`** (default `true`) — system update reindexes `documentation` aspects so agent search can see text written before the upgrade. Set `false` to skip it, then run `RestoreIndices` yourself if agents should see that text. - **`BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_BATCH_SIZE`** (default `1000`), **`BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_DELAY_MS`** (default `30000`), **`BOOTSTRAP_SYSTEM_UPDATE_DOCUMENTATION_LIMIT`** (default `0`, no limit) — throttle that reindex. - **`document_usage_stats_enabled`** — inside `DATAHUB_USAGE_REPORTING_BOOTSTRAP_VALUES`. The string `"false"` turns off document usage collection. A JSON boolean `false` does not. - **`BILLING_DASHBOARD_AI_MODULES`** — comma-separated AI modules shown in the AI Credits "Group by: Consumer" chart. Default is every module the product ships. - **`BILLING_DASHBOARD_AI_CREDITS_CREDIT_UNITS`** (default `AI Credits`) — invoice credit unit used when a contract prices AI usage without an included credits pool. - **`DATAHUB_FRONTEND_PROXY_MAX_IN_FLIGHT`** (frontend, default `1024`) — cap on Play requests waiting on an upstream. Above the cap, those requests return 503. - **`AGENT_RUN_MAX_THINKING_PER_POLL`** (integrations service, default `20`) — newest thinking messages returned on one `get_agent_response` poll while a turn is running. - **`ENABLE_STRUCTURED_PROPERTIES_COPY_TO_MISMATCH_REINDEX`** (default `true`) — when structured-property system update is also enabled, a structured-property field whose `copy_to` does not match the target mapping is fully reindexed. Set `false` to leave those indexes until you reindex them yourself. ### v2.3.0 #### Release Availability Date 30-SEP-2026 #### Recommended Versions - **CLI/SDK**: 1.7.0.12 - **Remote Executor**: v2.3.0-cloud, v2.2.2-cloud, v2.2.1-cloud, v2.2.0-cloud, v2.1.5-cloud, v2.1.4-cloud, v2.1.3-cloud, v2.1.2-cloud, v2.1.1-cloud, v2.1.0-cloud - **On-Prem Versions**: - **Helm**: 2.0.62 - **API Gateway**: v0.7.3 #### New Feature Highlights - **Context (Public Beta)** — the Context module is now in public beta, enabling you to build data agents on top of your DataHub context. - **Context Evals** — part of the Context module: root cause analysis, generation of evals, domain-oriented eval grouping, support for externally run evals, and improved resolution workflows in the product. Available under **Context → Validation**. - **Context Generation** — part of the Context module: an improved assistant experience for configuring the generation of context from BI dashboards, query history, and more. - **Context Feedback (Private Beta)** — part of the Context module: surfaces feedback collected through user conversations about missing, incorrect, or invalid context, so you can improve your DataHub context going forward. Available under **Context → Validation**. - **Dark Mode (Public Beta)** — a dark color theme across the app. Turn it on under **Settings → Appearance**. - **Ask DataHub Charts & Graphs** — data retrieved in Ask DataHub can be visualized as bar charts, line graphs, and metric highlights. - **Ask DataHub Proposal Support** — Ask DataHub can now propose changes to descriptions, tags, domains, structured properties, glossary terms, and documents. - **Ask DataHub & Custom Agents (Context) available in the MCP Server** — ask agent-to-agent questions to Ask DataHub or Custom Agents via the MCP server. Manage under **Settings → AI → MCP Servers**. - **Metrics as first-class lineage citizens** — Metrics and Semantic Models are now discoverable in global search and fully participate in lineage: charts, dashboards, and datasets can consume a Metric as an upstream, with column-level lineage modeled through metric upstreams. - **Two new ingestion sources** — [IBM/HCL Informix](/docs/generated/ingestion/sources/informix) and [Monte Carlo](/docs/generated/ingestion/sources/montecarlo). - **Structured Properties management** — creating, editing, and viewing a structured property moves from a side drawer to a dedicated page. Allowed values can be reordered by drag and drop, each with an optional description. Leaving the page with unsaved edits prompts before discarding them. The properties table adds **Type** and **Platforms** columns. #### Product - **Data products in search** — data products are now filterable in global search. - **Note on Data Product events** - upon upgrade there is a one-time search update so Data Product filters work on member assets. This update triggers change log event (RESTATE) on data products themselves, and possibly writes on member assets (dataProducts). There is no user action required. - **Custom Branding** — logo drag-and-drop upload, light brand colors updated for accessible contrast, and a dedicated **Favicon** field in Appearance settings. The logo is only reused as the browser-tab favicon when it is a standalone symbol; upload a square favicon under **Settings → Appearance → Branding** to keep a custom tab icon when **"Logo includes organization name"** is enabled. - **Compliance forms** — forms can be assigned to owners of a specific ownership type. - **Deprecation dates** — the deprecation date input is now editable. - **Internationalization** - Simplified Chinese and Russian languages supported in public beta. #### AI / Ask DataHub - **Agent visibility** — agents marked "Show in Ask DataHub Chat" are now visible to all users, not only admins. - **New Manage Evals platform privilege** — gates managing all evals and eval actions. - **Semantic anchors** — the Context module now supports creating Semantic Anchor documents that incorporate LookML definitions, with incremental runs enabled by default. #### Platform - **OpenSearch 3.x** — supported via a unified OpenSearch client shim. - **Metrics and semantic models in global search** — both entity types are now searchable. - **Native semantic content** — an MCL hook maintains semantic content for bridge-covered entities. - **Structured properties in search** — oversized structured property values can optionally be omitted from search documents. - **GraphQL worker pool** — pinned to 8-core defaults, with optional processor-scaled concurrency and a configurable queue size. - **AWS IRSA** — GMS owns a single process-wide default credentials provider, constructed and closed once per process. - **Timeseries read authorization** — dataset profile, usage and operations timeseries now enforce the matching View privileges on Rest.li and OpenAPI as well as GraphQL. See **Breaking Changes**. - **Support login** — `ticket_id` is required when using sudo on support login. - **SPARQL endpoint** — unanchored closures, built-in predicate IRIs, default prefixes and friendlier error messages. - **Configuration** — per-operation configuration reads via config enrichment, and per-tenant configuration resolved from the config service. - **Multi-tenancy flags split** — propagation, database and search flags are now independent. - **Access tokens** — never-expiring tokens are disabled by default and allowed durations are configurable. See **Breaking Changes**. #### Ingestion **New ingestion sources:** - **IBM/HCL Informix** — new connector. - **Monte Carlo** — new connector. **Connector improvements:** - **Snowflake** - password login is deprecated: the CLI and UI now warn when a recipe uses a username and password, and there's a guide for moving to key-pair auth. - **BigQuery** — support for BigQuery Sharing linked datasets and multi-column partition info; the profiler moves into its own package with added SQL and identifier guards. - **Cube** — Cube views are ingested as Semantic Models. - **Dataplex** — supports the Dataplex Metadata Export API extraction method. - **Metabase** — column-level lineage, model ingestion, collection tags and containers, plus general hardening. - **MicroStrategy** — project schema is ingested as a Semantic Model. - **Power BI** — resolves BigQuery `EXTERNAL_QUERY` federation lineage. - **SAP Analytics Cloud** — column-level lineage to SAP Datasphere for DWC live models, and SAC → SAP Datasphere live-model lineage. - **SAP Datasphere** — calculated column formulas are surfaced in the column description. - **Sigma** — Sigma Dataset lineage is now recovered from Sigma's connection metadata rather than SQL. See **Breaking Changes**. - **Kafka Connect** — SQL Server is treated as a three-level platform. See **Breaking Changes**. **Ingestion infrastructure:** - **Connector support statuses** have been updated and refreshed; each source has been assigned as Alpha, Beta, or GA. - **Configurable SQL lineage timeout.** - **Structured properties settings** can be ingested from the Python SDK. - **Secret masking** — stdin envelope secrets are registered on receipt. #### Breaking Changes - **MCP server: the `?token=` query parameter no longer works** — every MCP endpoint takes the access token in the `Authorization: Bearer ` header only, and a request with a `token` query parameter gets a `400 Bad Request` explaining the header form. A token in a URL persists in proxy and CDN logs, browser history and `Referer` headers. **Action:** point MCP clients at the plain URL and send the header. Clients that cannot set headers can use `mcp-remote` with `--header "Authorization: Bearer "`. - **Metric upstreams and lineage privileges** — charts, dashboards and datasets may take a Metric as an upstream, stored on the shared `upstreamMetrics` aspect. OpenAPI and Rest.li writes of `upstreamMetrics` require **Edit Lineage** or **Edit Entity** on the consumer and on each destination Metric. A dataset cannot both consume a Metric via `upstreamMetrics` and appear on that Metric's `metricUpstreams.datasetUpstreams`. **Action:** automation that sent unsupported `updateLineage` pairs now receives an error instead of a silent no-op. Use `metricUpstreams` to attach a dataset to a Metric, and grant **Edit Lineage** to any client writing `upstreamMetrics` outside the UI. - **Sigma ingestion: dataset lineage route changed** ([#19815](https://github.com/datahub-project/datahub/pull/19815)) — Sigma ended dataset-as-data-source support on 2026-09-15 and dataset-backed workbook elements no longer return SQL, so Sigma Dataset lineage URNs are built from `connection_to_platform_map` rather than `chart_sources_platform_mapping`. `env` and `platform_instance` now come from the recipe unless a mapping entry supplies them, adding an entry makes its `env` authoritative (defaulting to `PROD`), and identifier casing changes on platforms other than Snowflake. **Action:** add the Sigma `connectionId` to `connection_to_platform_map` with the `env` and `platform_instance` your warehouse connector used, plus `convert_urns_to_lowercase: true` if you want the previous spelling for those table names. The connector warns when a mapping's `env` or `platform_instance` no longer applies. - **Structured property hard-delete requires a soft-delete first** ([#19445](https://github.com/datahub-project/datahub/pull/19445)) — hard-deleting an active structured property, or deleting its `propertyDefinition` aspect directly, is now rejected across the UI, GraphQL, OpenAPI, Rest.li and the CLI. Hard deletion left the property's qualified name in the search index mappings, so the name could not be reused until affected indices were reindexed. The UI delete flow performs both steps automatically. **Action:** where automation issued a single hard delete, soft-delete then hard-delete. To reuse a name burned by an earlier hard delete, run SystemUpdate with `ELASTICSEARCH_INDEX_BUILDER_MAPPINGS_REINDEX=true`. - **Timeseries read authorization** — dataset profile, usage and operations timeseries (and dashboard/chart usage statistics) require the matching **View Dataset Profile / Usage / Operations** privileges on Rest.li and OpenAPI as well as GraphQL. Dedicated timeseries APIs also require **Get Timeseries Aspect API** when REST API authorization is enabled. Single-aspect GET/HEAD returns 403 without the privilege; assembled entity GET omits the aspect. **Action:** grant the View Dataset Profile / Usage / Operations privileges, or rely on the default `view-dataset-sensitive` policy, for any API client that previously read these aspects with only entity GET. - **Role and group membership writes are authorized at the aspect layer** — adding a role to a user or group (`roleMembership`) requires **Manage Policies**; adding a user to a group requires **Edit Group Members**; adding a corpGroup owner requires **Edit Owners**. When the acting user is the one being added, both require **Manage Users & Groups**. Only additions are checked. Previously **Edit Entity** on a user was enough to grant that user the Admin role. **Action:** grant **Manage Policies** to automation that writes `roleMembership` outside the UI. Group-membership sync (LDAP, Okta, Azure AD, `datahub user upsert`) needs **Edit Group Members** or **Manage Users & Groups**, not Manage Policies. Group owners can no longer add themselves to groups they own. - **Analytics API requires its own privilege** — `POST /openapi/v2/analytics/datahub_usage_events/_search` now requires **Analytics API access** or **Manage System Operations**. It previously accepted **View Analytics**, which every user holds by default, even though the endpoint forwards a caller-supplied search request to the search engine. **Action:** grant **Analytics API access** to any service account or script calling this endpoint. The in-app analytics dashboard still works with **View Analytics**. - **Compliance form assignment requires Manage Compliance Forms** — `batchAssignForm`, `batchRemoveForm` and `createDynamicFormAssignment`, and any write to a form's `forms` or `dynamicFormAssignment` aspect, now require the **Manage Compliance Forms** platform privilege. Writes that only change prompt completion or verification state are unaffected, so assignees can still complete forms. These mutations previously performed no authorization check. **Action:** grant **Manage Compliance Forms** to automation that assigns or unassigns forms outside the UI. - **Asset summary settings writes are authorized** ([#19745](https://github.com/datahub-project/datahub/pull/19745)) — `updateAssetSettings` and any write to the `assetSettings` aspect require **Edit Entity** or **Manage Asset Summary** on the target asset. The GraphQL mutation previously performed no authorization check, so any authenticated user could change any asset's summary template. **Action:** grant **Manage Asset Summary** or **Edit Entity** to automation writing `assetSettings` outside the UI. Note that OpenAPI and Rest.li still authorize this as a generic entity update, so **Manage Asset Summary** alone is not sufficient there. - **Never-expiring access tokens are disabled by default** — allowed finite durations are configured as a comma-separated ISO-8601 list (`ACCESS_TOKEN_ALLOWED_DURATIONS`, default `PT1H,P1D,P7D,P30D,P90D,P180D,P365D`). Exactly one of GraphQL `duration` or `durationIso` must be provided on create. **Already-issued tokens, including never-expiring ones, keep working** — this is create-time policy only. **Action:** set `ACCESS_TOKEN_ALLOW_NO_EXPIRY=true` to allow never-expire tokens again, or set `ACCESS_TOKEN_ALLOWED_DURATIONS` to customize available TTLs. - **OpenAPI entity create authorization is per-URN** ([#18944](https://github.com/datahub-project/datahub/pull/18944)) — `createEntity` / `createGenericEntities` and aligned batch paths authorize each URN after building the MCP batch, using entity existence to choose Create vs Edit privileges. A principal with only **Create Entity** can no longer overwrite an existing entity via create/UPSERT. **Action:** grant **Edit Entity** to callers that update existing entities or use default `createAspect`; keep create-only policies for true creates. - **Domain-scoped create and domain writes** ([#18944](https://github.com/datahub-project/datahub/pull/18944)) — ingest-time authorization evaluates domain-scoped **Create Entity** / **Edit Entity** policies against _proposed_ `domains` when establishing domains on a new entity. A `domains` PATCH always requires **Edit Entity**, and domain-scoped Edit must allow both the before and after domain membership. **Action:** domain-separated writers must include matching `domains` in the create; grant Edit on both source and destination domains before moving assets between domains with a domains PATCH. - **Group member removal now removes unmigrated members** ([#18982](https://github.com/datahub-project/datahub/pull/18982)) — `removeGroupMembers` removes members holding only the legacy `groupMembership` aspect. Previously the mutation stripped only `nativeGroupMembership`, reported success, and left the user in the group with access intact. **Action:** none required. - **MySQL, MariaDB, Doris and TiDB profiling limits are enforced** ([#18699](https://github.com/datahub-project/datahub/pull/18699)) — `profiling.profile_table_row_limit` and `profiling.profile_table_size_limit` were previously accepted and ignored on these sources. **Action:** re-check any value already set in your recipe before upgrading, as it will start filtering profiles. Both default to `null` on these sources, so nothing changes unless you set them. - **`sql-queries` reports total failure** ([#19099](https://github.com/datahub-project/datahub/pull/19099)) — a run where every input line is unparseable or every query fails now reports failure and exits non-zero instead of completing successfully with nothing to show. Individual bad rows are still skipped and counted. `enable_lazy_schema_loading` is removed, and two report fields are renamed: `num_queries_processed_sequential` → `num_queries_processed`, `num_temp_tables_detected` → `num_temp_table_matches`. **Action:** remove `enable_lazy_schema_loading` from your recipe. - **Python SDK circuit breakers move to `DataHubGraph`** ([#18548](https://github.com/datahub-project/datahub/pull/18548)) — `BaseApi` and its `Assertion` / `Operation` subclasses query GMS through `DataHubGraph` instead of a `gql` client, gaining OAuth, PAT, system auth and mTLS. The `transport=` constructor argument is replaced by `graph=`, `.client` by `.graph`, and GraphQL errors raise `GraphError` instead of `TransportQueryError`. **Action:** update constructor calls and error handling. - **Kafka Connect treats SQL Server as three-level** — JDBC sink connectors default the schema to `dbo` when `schema.name` is unset, so sink URNs change from `database.table` to `database.dbo.table`. JDBC source connectors drop a dataset with a warning when the schema cannot be resolved, rather than emitting a two-level URN. Oracle JDBC sinks now emit `schema.table`. **Action:** none if your SQL Server assets were ingested with schema in the URN. If a source connector starts warning about a missing schema, set `schema.name` on the connector or use `generic_connectors` to force the mapping. - **Elasticsearch ingestion requires `opensearch-py` 3.x** — `acryl-datahub[elasticsearch]` now requires `opensearch-py>=3.0.0,<4.0.0`. The 3.x client still talks to OpenSearch 2 clusters. **Action:** none if you install via the extra; if you pin `opensearch-py` yourself, raise the floor. - **The legacy Great Expectations SQL profiler is removed** — the SQLAlchemy profiler, default since v1.1.0 and at feature parity, is now the only SQL profiler. The `profiling.method` option and the `acryl-datahub[profiling-ge]` extra no longer exist; recipes that still set `profiling.method` emit a deprecation warning and profile with SQLAlchemy. For Unity Catalog, `profiling.method: ge` is rejected. **Action:** remove `profiling.method` from recipes and drop the `profiling-ge` extra. This does not affect the separate Great Expectations integration that ingests GX validation results into DataHub. #### Deprecations - **The operational dashboard under `/admin/dashboard` is being removed.** It proxied an internal Grafana dashboard and has no equivalent after the migration to a new observability stack. Operational signals remain available through the DataHub public APIs, with published SLOs as the forward-looking replacement. Behavior is controlled per instance by `GRAFANA_DASHBOARD_MODE`: | Release | Mode | What customers see | | ------- | ----------- | ---------------------------------------------------------------- | | 2.2 | `TOMBSTONE` | `410 Gone` with a deprecation notice pointing at the public APIs | | 2.3.0 | `DISABLED` | `404` — the path no longer responds | | 2.4.0 | — | the proxy and its configuration are deleted | The default remains `ENABLED` in this release, so nothing changes until an instance is flipped. #### Security / Dependencies - **transformers**: bumped to 5.17.0 to address CVE-2026-9856. - **Apache Parquet**: bumped to 1.18.1 to address CVE-2026-73334. - **mariadb-java-client**: bumped to 2.7.14 to address CVE-2026-55856, CVE-2026-55857 and CVE-2026-55858. - **cryptography**: bumped to 50.0.1 in the integrations service to address CVE-2026-69247. - **cryptography / pyOpenSSL**: ingestion floors raised to address CVE-2026-69247 and CVE-2026-27459. - **Spring Framework / Boot / Security**: bumped to 7.0.9, 4.0.8 and 7.0.7 respectively. - **httpclient5, Spring Boot and the OpenTelemetry agent**: bumped for reported CVEs. - **nltk**: bumped 3.10.0 → 3.10.3. - **gitpython**: bumped to 3.1.61 in the Cloud image lockfiles. - **setuptools**: the Python package layer is refreshed in the executor and actions images to address CVE-2025-47273. - **curl**: base image packages refreshed to address reported curl CVEs. - **mcp and json-repair**: bumped for security fixes. - **Executor CVE gating**: the executor image now gates and tracks CVEs as part of the build. ## Known Issues - None known at release time.