feat(retrieval): Phase 1 — typed schema and FT.CREATE translation

[jamby77]

Summary

Phase 1 of the Retrieval SDK plan, stacked on #234 (Phase 0). Adds the first code of @betterdb/retrieval plus one deferred Phase 0 review item in the kit.

• Scaffold packages/retrieval (@betterdb/retrieval 0.1.0), mirroring valkey-search-kit, with a workspace dep on the kit
• Pure buildFtCreateArgs(name, schema, capabilities?) translating the typed index schema (text / tag+separator / numeric+sortable fields; HNSW|FLAT vector as a discriminated union) into the full FT.CREATE argument vector — HNSW defaults M=16 / EF_CONSTRUCTION=200 / EF_RUNTIME=10 always emitted; exported indexName() / keyPrefix() naming helpers
• TEXT field emission gated via FtCapabilities.textFields — valkey-search < 1.2 rejects TEXT, so callers on older modules get an actionable error instead of a server failure
• Tighten isIndexNotFoundError in valkey-search-kit: the broad 'not found' substring match is now scoped to index errors ('not found' + 'index' co-occurrence). Verified against live engines — valkey-search 1.2 emits Index with name '…' not found in database 0, Redis 8 emits No such index …; both stay matched, generic key not found-style messages no longer misclassify. The semantic-cache characterization lock was deliberately split into positive + negative cases for this.

Test Plan

• @betterdb/retrieval unit tests: 32/32 (table-driven, full-vector deep equality)
• @betterdb/valkey-search-kit unit tests: 32/32 (incl. empirically captured engine phrasings)
• @betterdb/semantic-cache suite: 191/191 (characterization net intact)
• semantic-cache integration suite vs live valkey-bundle (valkey-search 1.2, port 6384): 13/13
• tsc builds clean across the three packages

Stacked PR: base is feature/retrieval-sdk-valkey-search-kit (#234). After #234 merges, this will be rebased onto master and retargeted — do not delete the base branch before that.

---

Note

Medium Risk
The tightened index-not-found heuristic changes runtime error handling for semantic-cache initialization and any other kit consumers; behavior is well-tested but mis-tuned matching could still mis-route FT.INFO failures.

Overview
Introduces @betterdb/retrieval (0.1.0) as Phase 1 of the retrieval SDK: typed index schema (text / tag / numeric fields plus HNSW|FLAT vector specs) and pure buildFtCreateArgs that emits the full FT.CREATE argument vector, with indexName / keyPrefix helpers and FtCapabilities.textFields to fail fast when TEXT fields are used on valkey-search < 1.2.

isIndexNotFoundError in @betterdb/valkey-search-kit no longer treats every 'not found' substring as a missing index; it now requires both 'not found' and 'index' (plus existing phrasings), so messages like key not found are not misclassified. Semantic-cache characterization tests were split into positive index-missing cases and a negative case where generic not-found errors surface as ValkeyCommandError instead of triggering index creation.

^{Reviewed by Cursor Bugbot for commit ab188f1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[KIvanow]

Minor DX note: fresh-checkout tests fail until the kit is built

Running pnpm --filter @betterdb/semantic-cache test on a clean checkout currently fails with:

Error: Failed to resolve entry for package "@betterdb/valkey-search-kit".
The package may have incorrect main/module/exports specified in its package.json.
  ❯ src/utils.ts:8:1

Vitest resolves the workspace symlink through the kit's main: ./dist/index.js, so the kit's dist/ has to exist before semantic-cache (and soon retrieval, once it actually imports the kit) tests can run. A turbo test → ^build dependency, or pointing the kit's dev-time resolution at src/ (e.g. via publishConfig), would make pnpm test work out of the box.

Totally fine to leave as-is if this is already planned for one of the next PRs in the stack — just flagging it so it doesn't get lost.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ee9fb6a. Configure here.}

[cursor]

HNSW tuning params unvalidated

Medium Severity

buildVectorArgs applies ?? for m, efConstruction, and efRuntime, so explicit NaN or non-finite values are forwarded into the FT.CREATE vector attribute list (e.g. M becomes the string NaN). dims is validated via requireDims, but HNSW tuning fields are not, so invalid schemas can produce server-rejected commands instead of a clear client error.

 

^{Reviewed by Cursor Bugbot for commit ee9fb6a. Configure here.}

[cursor]

Whitespace-only schema field names

Low Severity

validateFieldNames treats only zero-length keys as invalid, while index names and vector fieldName reject whitespace-only strings via trim(). A schema field key consisting only of spaces is accepted and emitted in the FT.CREATE SCHEMA section, which is inconsistent validation and can yield confusing server failures.

 

^{Reviewed by Cursor Bugbot for commit ee9fb6a. Configure here.}