Skip to content

fix: enforce 255-character identifier length limit for Databricks relations#1391

Open
psaikaushik wants to merge 1 commit intodatabricks:mainfrom
psaikaushik:fix/1309-relation-max-name-length
Open

fix: enforce 255-character identifier length limit for Databricks relations#1391
psaikaushik wants to merge 1 commit intodatabricks:mainfrom
psaikaushik:fix/1309-relation-max-name-length

Conversation

@psaikaushik
Copy link
Copy Markdown

Summary

Enforces the 255-character identifier length limit for Databricks relations at relation creation time, preventing a cryptic runtime DatabricksExecutionError when store_failures generates overly-long table names.

Closes #1309

Problem

When store_failures: true is enabled, dbt generates table names by concatenating the test name, model name, and arguments. For deep schema tests with verbose names, this can exceed Databricks' 255-character limit for table names, causing:

[RequestId=...] Invalid input: RPC CreateStagingTable Field
managedcatalog.StagingTableInfo.name: name "accepted_map_keys_stg_product_search_create_sizes__relax__..." too long.
Maximum length is 255 characters.

This error is unhelpful — it doesn't explain what the 255-character limit is, which test/model caused it, or how to fix it.

Fix

Added relation_max_name_length() and __post_init__ validation to DatabricksRelation, following the exact same pattern used by the Postgres adapter:

MAX_CHARACTERS_IN_IDENTIFIER = 255

def relation_max_name_length(self):
    return MAX_CHARACTERS_IN_IDENTIFIER

def __post_init__(self):
    if (
        self.identifier is not None
        and self.type is not None
        and len(self.identifier) > self.relation_max_name_length()
    ):
        raise DbtRuntimeError(
            f"Relation name '{self.identifier}' "
            f"is longer than {self.relation_max_name_length()} characters. ..."
        )

Before:

[RequestId=...] Invalid input: RPC CreateStagingTable ... name "accepted_map_keys_stg_..." too long.

After:

Relation name 'accepted_map_keys_stg_product_search_create_sizes__...'
is longer than 255 characters. Databricks has a maximum identifier
length of 255 characters. If this is a store_failures table, consider
using a shorter test name or setting a custom alias.

Checklist

  • Follows the same pattern as the Postgres adapter
  • Catches the error at relation creation time, not at SQL execution
  • Error message includes actionable guidance

…ations

Add relation_max_name_length() returning 255 and a __post_init__
validation to DatabricksRelation, following the same pattern used by
the Postgres adapter. This catches overly-long table names (commonly
generated by store_failures for tests with verbose names) at relation
creation time with a clear error message instead of a cryptic runtime
DatabricksExecutionError from the SQL engine.

Closes databricks#1309
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] DatabricksExecutionError when store_failures creates table names > 255 characters

1 participant