Skip to content

feat: Add invocation_id to the default query comment #1377

@GuyEshdat

Description

@GuyEshdat

Describe the feature

The default query comment (DATABRICKS_QUERY_COMMENT in connections.py) currently includes metadata like dbt_version, profile_name, target_name, and node_id, but does not include invocation_id.

The invocation_id is a UUID generated for each dbt command invocation and is already available in the query-comment compilation context. Including it in the default comment would allow users to correlate all queries from a single dbt run / dbt build / dbt test invocation directly in the Databricks SQL Query History, without needing to override the default query comment macro.

Who would benefit?

Any team debugging or auditing dbt runs on Databricks. Today, if multiple models fail in a run, you can see each query's node_id but can't easily group them by the invocation that triggered them. Adding invocation_id makes it straightforward to filter Query History by a single dbt invocation.

Proposed change

In dbt/adapters/databricks/connections.py, add invocation_id=invocation_id to the comment_dict.update() call in DATABRICKS_QUERY_COMMENT. This would produce query comments like:

{"app": "dbt", "dbt_version": "1.11.0", "dbt_databricks_version": "1.11.6", "databricks_sql_connector_version": "...", "profile_name": "my_profile", "target_name": "dev", "invocation_id": "01234567-abcd-...", "node_id": "model.my_project.my_model"}

Additional context

  • invocation_id is already available in the query-comment Jinja context per the dbt docs
  • The BigQuery adapter already includes invocation_id as a job label
  • Users currently need a custom query_comment macro override just to get this, which is unnecessary boilerplate

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions