Cache current benchmark version lookups

[olearycrew]

Problem

• Production traffic showed 912 visitors producing 76K requests, 77K API calls, 299K D1 queries, and 548M rows read.
• Every unversioned API request resolved the current benchmark versions by querying benchmark_versions, even though the table has only a few rows and changes weekly.
• That created roughly 77K unnecessary D1 version-resolution queries per day before the endpoint-specific queries ran.

Root Cause

• resolveBenchmarkVersions() in src/utils/query.ts performed SELECT id FROM benchmark_versions WHERE current = 1 AND hidden = 0 on every request without a version or benchmark_version query param.
• The helper is shared by /api/submissions, /api/model-submissions, /api/me/submissions, /api/users/:github_username/submissions, /api/leaderboard, /api/models, /api/stats, and /api/providers/:provider/models.

Solution

• Added a 60s in-memory, per-isolate TTL cache for the unversioned current-version lookup.
• Added in-flight request coalescing so concurrent cold misses share one D1 lookup instead of fanning out.
• Preserved explicit version/benchmark_version behavior: requested versions are still validated against D1 and hidden versions still resolve to an empty filter.
• Return defensive array copies so downstream route code cannot mutate cached state.
• Added GitHub Actions test CI and synchronized/upgraded the test dependency stack; npm audit now reports zero vulnerabilities.

Test Coverage

• Added src/utils/query.test.ts with a production-equivalent benchmark version fixture containing current, historical, hidden, and semver/prerelease IDs.
• Parity test recreates the pre-cache resolver and asserts identical benchmark response fields for every affected endpoint/query shape.
• Regression tests assert repeated unversioned requests hit D1 once, the cache refreshes after 60s, concurrent cold misses coalesce to one D1 query, explicit version validation remains uncached, and cached arrays are defensively copied.
• CI runs npm ci and npm test on pushes and pull requests.

How to run:

npm ci
npm test
npm audit --audit-level=moderate

Metrics Impact

• Expected current-version lookup queries drop from approximately 77K/day to approximately 1,440/day per warm isolate at a 60s TTL.
• For the 5-row benchmark_versions table, this reduces rows read for that hot path from roughly 385K/day to roughly 7.2K/day per warm isolate, about a 98% reduction.
• Total D1 query volume should drop by about one query for every unversioned affected API request; explicit version requests intentionally keep their validation query.

Risks & Mitigations

• Risk: current-version changes can take up to 60s to appear in a warm isolate. Mitigation: benchmark versions change weekly, and the TTL is intentionally short.
• Risk: Cloudflare Workers isolate-local memory means multiple isolates can each perform one lookup per TTL window. Mitigation: this still bounds the lookup rate dramatically versus one query per request.
• Risk: cache stampede on cold starts. Mitigation: in-flight promise coalescing ensures concurrent cold misses share one D1 query within an isolate.
• Risk: dependency updates for CI/test hardening. Mitigation: validated with npm ci && npm test; npm audit --audit-level=moderate reports zero vulnerabilities.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
❌ Deployment failed View logs	pinchbench-api	cce0303	Jun 08 2026, 06:35 PM