ALTER TABLE SET TBLPROPERTIES on UniForm tables triggers DELTA_FEATURE_DROP_DEPENDENT_FEATURE

[mehul-gemini]

Describe the bug

On incremental runs against UniForm tables (icebergCompatV3 enabled), the adapter's config reconciliation step runs ALTER TABLE SET TBLPROPERTIES even when the property already matches the table state. On UniForm tables, any ALTER TABLE SET TBLPROPERTIES — even setting an unrelated property like delta.feature.allowColumnDefaults — triggers an internal Delta feature reconciliation that attempts to drop rowTracking, which fails because icebergCompatV3 depends on it:

[DELTA_FEATURE_DROP_DEPENDENT_FEATURE] Cannot drop table feature `rowTracking`
because some other features (icebergCompatV3) in this table depends on `rowTracking`.
Consider dropping them first before dropping this feature.

Two separate issues contribute to this bug

1. Unnecessary ALTER TABLE: The tblproperties comparison logic detects a mismatch and generates ALTER TABLE SET TBLPROPERTIES ('delta.feature.allowColumnDefaults' = 'supported') even though the table already has this property set. The ALTER should not be generated at all.

2. Databricks platform bug: Even if the ALTER TABLE were necessary, setting an unrelated property like allowColumnDefaults should not trigger a rowTracking feature drop on a UniForm table. This appears to be a Delta Lake / Databricks platform issue.

How we isolated the root cause

We overrode the apply_config_changeset macro with var-controlled flags to skip individual operations:

• --vars '{skip_tblproperties: true}' → 2nd incremental run passes ✅
• --vars '{skip_column_comments: true}' → 2nd incremental run fails ❌ (same error)

This proves the apply_tblproperties call is the sole trigger. Column comments, tags, constraints, and all other config reconciliation operations work fine.

Steps to reproduce

1. Configure a project with UniForm (icebergCompatV3) enabled via dbt_project.yml:

   +tblproperties:
     delta.feature.allowColumnDefaults: 'supported'
     delta.enableIcebergCompatV3: 'true'
     delta.universalFormat.enabledFormats: 'iceberg'
     delta.enableDeletionVectors: 'true'

2. Create an incremental model with merge strategy
3. Run dbt build -s model_name — succeeds (table created with correct tblproperties)
4. Run dbt build -s model_name again — fails with DELTA_FEATURE_DROP_DEPENDENT_FEATURE

First runs and --full-refresh always work because tblproperties are applied during CREATE TABLE, which does not trigger the feature reconciliation.

Observed SQL from dbt logs

On the 2nd run, the adapter issues:

ALTER TABLE `dev_mart`.`marketing`.`fact_braze_sends` SET
    tblproperties ('delta.feature.allowColumnDefaults' = 'supported')

This ALTER is unnecessary — the table already has this property from creation. The adapter's tblproperties comparison logic incorrectly detects a mismatch.

Expected behavior

1. The adapter should not generate ALTER TABLE SET TBLPROPERTIES when the table properties already match the declared config.
2. If an ALTER is generated, setting an unrelated property should not trigger a feature drop on UniForm tables.

Current workaround

We override apply_tblproperties as a no-op at the project level:

{% macro apply_tblproperties(relation, tblproperties) -%}
  {{ log("Skipping apply_tblproperties for " ~ relation ~ " (UniForm workaround)") }}
{%- endmacro -%}

tblproperties are still applied correctly at CREATE TABLE time. To apply new tblproperties changes, affected models must be run with --full-refresh.

System information

dbt version: 2.0.0 (dbt Cloud)
dbt-databricks adapter: bundled with dbt Cloud 2.0
Databricks Runtime: Latest (Unity Catalog, managed tables)
Table configuration: Delta with icebergCompatV3, UniForm, liquid clustering, deletion vectors
Materialization: Incremental with merge strategy