Add support for Vector Search Endpoint (direct only) by janniklasrose · Pull Request #4887 · databricks/cli

janniklasrose · 2026-04-02T11:38:26Z

Changes

Adds vector_search_endpoints as a first-class resource type, using the direct deployment engine (only, no TF support).

New configuration surface

resources:
  vector_search_endpoints:
    my_endpoint:
      name: my-endpoint
      endpoint_type: STANDARD
      min_qps: 1
      budget_policy_id: my-policy
      permissions:
        - level: CAN_USE
          group_name: data-team

Required fields: name, endpoint_type. Optional: min_qps, budget_policy_id, permissions.

Key points to note

State ID = endpoint name. The CRUD API identifies endpoints by name; the UUID
(endpoint_uuid) is stored separately in the refresh output for use by the permissions API.

endpoint_type is immutable. Changing it triggers delete + recreate (resources.yml).

Two separate update APIs. DoUpdate dispatches to:

UpdateEndpointBudgetPolicy when budget_policy_id changes
PatchEndpoint when min_qps changes

These can fire in the same deploy if both fields change.

budget_policy_id drift is suppressed. The API returns effective_budget_policy_id
(which includes inherited workspace policies), not the user-set value. Until the SDK
exposes budget_policy_id separately, remote changes to this field are ignored
(reason: effective_vs_requested in resources.yml). See TODO in
bundle/direct/dresources/vector_search_endpoint.go:53.

Permissions use UUID, not name. The PreparePermissionsInputConfig function uses
${...endpoint_uuid} as the object ID when constructing the permissions API path for
vector search endpoints.

Direct-only validation. ValidateDirectOnlyResources (bundle/config/mutator/) emits
an error at plan/deploy time if vector_search_endpoints are present in a non-direct bundle.
Vector Search Endpoints have no Terraform provider.

No dev-mode name prefix. Like UC resources, vector search endpoint names are NOT
prefixed with the dev user name in development mode.

Tests

Acceptance & Unit tests.
Tested e2e with CLI build.

andrewnester

Could you please add tests for bind / unbind for this resources before merging?
https://github.com/databricks/cli/tree/main/acceptance/bundle/deployment/bind

andrewnester · 2026-04-13T14:08:58Z

+		}
+	}
+	info := remote.EndpointInfo
+	budgetPolicyId := info.EffectiveBudgetPolicyId // TODO: use info.BudgetPolicyId when available


Why this TODO though? Shouldn't we alway use EffectiveBudgetPolicyId anyway?

EffectiveBudgetPolicyId is potentially unreachable by setting BudgetPolicyId (depending how backend resolves this given workspace policies etc), leading to permanent drift. Once the API (next SDK release) is declarative friendly, we can do what we do for other resources too (use remote .BudgetPolicyId or "" if not available)

andrewnester · 2026-04-13T14:11:51Z

+}
+
+func (r *ResourceVectorSearchEndpoint) DoUpdate(ctx context.Context, id string, config *vectorsearch.CreateEndpoint, entry *PlanEntry) (*VectorSearchRefreshOutput, error) {
+	if entry.Changes.HasChange(pathBudgetPolicyId) {


Did you consider sending these two requests in parallel in separate goroutines? pros / cons

Don't expect this to be much of a bottleneck, but they're independent so we could. There's precedent in model_serving_endpoint.go to make them sequential on purpose (though granted it's following Terraform there, which only does 1 call for vector search since they're missing the min_qps path).

I will double-check with the VS team if the APIs are safe to call in parallel and update in a follow-up.

No need to be blocked by this though, just an observation for later improvements if needed

denik · 2026-04-16T10:31:47Z

+  vector_search_endpoints:
+    foo:
+      name: test-endpoint-$UNIQUE_NAME
+      endpoint_type: STANDARD


Could you also add another config with permissions?

denik · 2026-04-16T10:33:36Z

+trace $CLI vector-search-endpoints patch-endpoint "${endpoint_name}" --min-qps 5
+
+title "Plan detects drift and proposes update"
+trace $CLI bundle plan


you can also add contains.py to encode expectation

trace $CLI bundle plan | contains.py "Plan: 0 to add, 0 to delete, 1 to update"

denik · 2026-04-16T10:36:39Z

 	}

+	// Vector search endpoints use the endpoint name as deployment id; the permissions API uses endpoint UUID.
+	if strings.HasPrefix(baseNode, "resources.vector_search_endpoints.") {


nit: the above if chain should be if/else chain.

denik · 2026-04-16T10:44:37Z


+# Regenerate refschema for new resources
+.PHONY: generate-refschema
+generate-refschema:


can be skipped, you get it automatically when running test-update (unless you reference it in some skill / doc, but then it should be part of that PR).

denik · 2026-04-16T10:45:50Z

                  "description": "The alert_id is the canonical identifier of the alert.",
-                  "$ref": "#/$defs/string"
+                  "$ref": "#/$defs/string",
+                  "x-since-version": "v0.296.0"


Seems unrelated to your changes, ideally should go to separate PR.

Will rebase on SDK bump, which auto-gen'ed these

denik · 2026-04-16T10:48:11Z

 	"resources.synced_database_tables.*.spec.scheduling_policy":                                                                                 {"CONTINUOUS", "SNAPSHOT", "TRIGGERED"},
 	"resources.synced_database_tables.*.unity_catalog_provisioning_state":                                                                       {"ACTIVE", "DEGRADED", "DELETING", "FAILED", "PROVISIONING", "UPDATING"},

+	"resources.vector_search_endpoints.*.endpoint_type":        {"STANDARD"},


Should not we have public preview one there? STORAGE_OPTIMIZED

https://docs.databricks.com/api/workspace/vectorsearchendpoints/createendpoint

Waiting for SDK bump

denik · 2026-04-16T10:49:23Z

+}
+
+func (*ResourceVectorSearchEndpoint) RemapState(remote *VectorSearchRefreshOutput) *vectorsearch.CreateEndpoint {
+	if remote == nil || remote.EndpointInfo == nil {


The framework will not pass nil into any of the methods (RemapState, DoCreate etc), so we can drop nil checks from there.

denik · 2026-04-16T10:52:10Z

+
+// VectorSearchRefreshOutput is remote state for a vector search endpoint. It embeds API response
+// fields for drift comparison and adds endpoint_uuid for permissions; deployment state id remains the endpoint name.
+type VectorSearchRefreshOutput struct {


Let's call this VectorSearchRemote? This is the most common convention:

~/work/cli/bundle/direct/dresources % git grep -w type | grep Remote app.go:type AppRemote struct { job.go:// JobRemote is the return type for DoRead. It embeds JobSettings so that all job.go:type JobRemote struct { model.go:type MlflowModelRemote struct { pipeline.go:// PipelineRemote is the return type for DoRead. It embeds CreatePipeline so that all pipeline.go:type PipelineRemote struct {

We'll fix this one later as well:

~/work/cli/bundle/direct/dresources % git grep -w type | grep RefreshOut model_serving_endpoint.go:type RefreshOutput struct {

Sounds good! Renamed single occurrence of RefreshOut in #4989

denik · 2026-04-16T10:55:44Z

+		_, err := r.client.VectorSearchEndpoints.PatchEndpoint(ctx, vectorsearch.PatchEndpointRequest{
+			EndpointName:    id,
+			MinQps:          config.MinQps,
+			ForceSendFields: utils.FilterFields[vectorsearch.PatchEndpointRequest](config.ForceSendFields, "MinQps"),


if you're excluding MinQps here and there is nothing else, can we just set ForceSendFields to nil?

janniklasrose added 20 commits March 30, 2026 14:04

Add testserver

21f14c2

Add bundle/config/resources

0d8d10a

[autogen] make schema

7709fa9

[autogen] make schema-for-docs

1373144

[autogen] make generate-validation

114911e

Fix endpoint URL

f252c6d

Add bundle/direct/dresources

86a7ffe

Add generate-refschema target

be3589d

[autogen] make generate-refschema

c2692ec

Add TODO for budgetPolicyId

f31ca1e

[autogen] make generate-direct-resources

f10545a

Add resources.yml override

3603137

Ignore remote changes for budget_policy_id

0533a01

Add acceptance tests

ca87557

Add invariant tests

e700521

Exclude from migrate invariant test

885309a

Fix tests

8d8f392

Appease linters

7a95921

Add direct-only validation

63532ee

Handle 404 in Exists

56c99d9

janniklasrose had a problem deploying to test-trigger-is April 2, 2026 11:39 — with GitHub Actions Failure

janniklasrose added 9 commits April 10, 2026 15:54

[WIP] Add permissions, some gen'ed files

c95e932

Comments

0b615d5

Use wrapper for uuid for permissions

08c6565

Fix apply-bundle-permissions

e5cbab8

Add permissions acceptance test

08b590e

Fix tests & schemas

097a2bf

Missing out.test.toml for permission test

bf43e56

Capture badness regarding permissions update if endpoint is updated

bbe139f

Add minQps to test server

5bfac54

janniklasrose and others added 4 commits April 13, 2026 15:20

Missing acceptance test files

733688f

Merge branch 'main' into janniklasrose/vs-endpoint

6a153df

Lint & generate-validation

b526c7f

Appease exhaustruct

484814f

janniklasrose marked this pull request as ready for review April 13, 2026 14:02

janniklasrose requested review from andrewnester and denik April 13, 2026 14:02

Add changelog entry

908066b

andrewnester approved these changes Apr 13, 2026

View reviewed changes

janniklasrose and others added 3 commits April 13, 2026 16:27

Skip terraform-direct diff in permissions suite

3e736c0

Add bind/unbind test

4226c79

Merge branch 'main' into janniklasrose/vs-endpoint

fff9fdc

janniklasrose changed the title ~~Vector Search endpoint support (direct only)~~ Add support for Vector Search Endpoint (direct only) Apr 15, 2026

Update changelog for Vector Search Endpoints support

81394d1

denik reviewed Apr 16, 2026

View reviewed changes

Conversation

janniklasrose commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

New configuration surface

Key points to note

Tests

Uh oh!

andrewnester left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

janniklasrose commented Apr 2, 2026 •

edited

Loading