Skip to content

Fix submissions count token join#54

Merged
olearycrew merged 1 commit into
mainfrom
fix/d1-disparity-submissions-count
Jun 8, 2026
Merged

Fix submissions count token join#54
olearycrew merged 1 commit into
mainfrom
fix/d1-disparity-submissions-count

Conversation

@olearycrew

Copy link
Copy Markdown
Member

Problem

The public /api/submissions paginated list runs a data query plus a pagination COUNT(*) query. The COUNT path reused the data query's tokens join even when the request was not filtering by verified=true, adding unnecessary D1 read work on a hot read-heavy endpoint.

The observed production pattern was heavily read-biased: 912 visitors produced 76K requests, 77K API calls, 299K D1 queries, and 548M rows read, with only 104 rows written.

Root Cause

src/routes/submissions.ts always joined tokens in the /api/submissions COUNT query:

  • The data query needs tokens to project claimed.
  • The COUNT query only needs tokens when applying AND t.claimed_at IS NOT NULL for verified=true.
  • For default and verified=false list requests, the join was redundant because submissions.token_id is non-null and backed by the token FK.

Solution

  • Make the COUNT query append JOIN tokens t ON s.token_id = t.id only when verified=true.
  • Keep the data query unchanged so response shape and claimed projection stay identical.
  • Add an inline SQL-builder comment explaining why the join is intentionally conditional.
  • Sync package-lock.json with the existing vitest dev dependency so clean CI installs can run tests.
  • Add a minimal GitHub Actions workflow running npm ci and npm test for PRs.

Test Coverage

Added src/routes/submissions.test.ts with a production-equivalent D1 fixture covering:

  • Before/after parity snapshot for default /api/submissions response shape and total count against the previous joined-count behavior.
  • Verified-filter behavior proving the COUNT query still joins tokens when t.claimed_at is required.
  • Regression coverage proving default, verified=false, official-only, and model/provider filtered unverified COUNT variants do not reintroduce a token join.
  • Lightweight row-read estimate assertion showing the optimized COUNT avoids token-table reads in the fixture.

Validated locally:

  • npm ci && npm test
  • npm test
  • git diff --check

Metrics Impact

Expected impact is scoped to /api/submissions pagination COUNT queries:

  • Removes one tokens join from every default or otherwise unverified /api/submissions COUNT request.
  • The fixture benchmark reduces estimated COUNT rows read from 8 to 5 for the representative current-version list case, a 37.5% reduction for that COUNT query.
  • Real production savings scale with token-table size and the share of /api/submissions traffic that is not verified=true.

Risks & Mitigations

  • Risk: Counts could diverge if orphaned submissions.token_id rows exist.
  • Mitigation: Schema defines token_id TEXT NOT NULL with FOREIGN KEY (token_id) REFERENCES tokens(id), and submission creation validates tokens before insert.
  • Risk: Verified requests still require token data.
  • Mitigation: Regression tests assert the join remains present when verified=true.
  • Risk: Broader D1 disparity may have additional sources.
  • Mitigation: Opened follow-up issue Audit D1 token joins across read endpoints #53 to audit similar unconditional token joins across leaderboard/model/provider read endpoints.

@cloudflare-workers-and-pages

Copy link
Copy Markdown
Contributor

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
pinchbench-api 23cbcbc Commit Preview URL

Branch Preview URL
Jun 08 2026, 02:20 PM

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes the hot /api/submissions pagination COUNT(*) query by avoiding an unnecessary tokens join when the request is not filtering for verified=true, reducing D1 read work while preserving response shape for the data query.

Changes:

  • Make the /api/submissions COUNT query conditionally join tokens only when verified=true requires t.claimed_at.
  • Add regression tests covering COUNT SQL shape (join/no-join) and parity vs the legacy joined-count behavior.
  • Add CI coverage via a minimal GitHub Actions workflow and synchronize package-lock.json to include the existing vitest dev dependency.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File Description
src/routes/submissions.ts Skips tokens join in the COUNT query unless verified=true requires t.claimed_at.
src/routes/submissions.test.ts Adds fixture-backed tests asserting COUNT query join behavior and result parity/regressions.
package-lock.json Updates lockfile to align with vitest in package.json for clean npm ci installs.
.github/workflows/test.yml Adds CI workflow to run npm ci and npm test on PRs and main pushes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@olearycrew olearycrew merged commit 7ae80ec into main Jun 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants