Skip to content

[CRD] Add KVCachePool API and runtime connector specs#607

Open
YouNeedCryDear wants to merge 3 commits into
mainfrom
feat/kv-cache-pool-crd-specs
Open

[CRD] Add KVCachePool API and runtime connector specs#607
YouNeedCryDear wants to merge 3 commits into
mainfrom
feat/kv-cache-pool-crd-specs

Conversation

@YouNeedCryDear
Copy link
Copy Markdown
Collaborator

What this PR does

  • Adds the namespace-scoped KVCachePool API for provider-neutral KV cache pool configuration.
  • Adds InferenceService.spec.kvCachePool as a reference-only binding to an existing pool.
  • Adds ServingRuntime.spec.kvCacheConnectors for runtime-side KV cache connector metadata.
  • Adds generated API helpers for deepcopy, defaulting, clientset, informers, and listers.
  • Updates OEP-0010 to align the design with the implemented alpha API shape.

Why we need it

OME needs a first-class API for independently managed KV cache pools so serving endpoints can attach to compatible pool implementations without treating KV cache infrastructure as an InferenceService-owned component. Runtime connector metadata lets the selected runtime describe how engine, decoder, router, or predictor components should connect to a referenced pool.

Fixes #

How to test

Not run by Codex while creating this PR; the branch and commits were already pushed.

Checklist

  • Tests added/updated (if applicable)
  • Docs updated (if applicable)
  • make test passes locally

@github-actions github-actions Bot added documentation Documentation changes api API/Types changes in pkg/apis oep OME Enhancement Proposal labels May 11, 2026
@YouNeedCryDear YouNeedCryDear marked this pull request as ready for review May 11, 2026 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api API/Types changes in pkg/apis documentation Documentation changes oep OME Enhancement Proposal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant