Skip to content

perf(db): add published_at composite index for frontend listing queries#333

Merged
ascorbic merged 2 commits intoemdash-cms:mainfrom
mvanhorn:perf/277-published-at-index
Apr 11, 2026
Merged

perf(db): add published_at composite index for frontend listing queries#333
ascorbic merged 2 commits intoemdash-cms:mainfrom
mvanhorn:perf/277-published-at-index

Conversation

@mvanhorn
Copy link
Copy Markdown
Contributor

@mvanhorn mvanhorn commented Apr 6, 2026

What does this PR do?

Closes #277

Adds a composite index (deleted_at, published_at DESC, id DESC) to all ec_* content tables. The frontend listing query orders by published_at, but no existing index covers it. On a site with ~16k posts this causes ~15,883 rows read per query (full table scan).

Also registers migration 033 (optimize_content_indexes) which existed on disk but was missing from the StaticMigrationProvider.

Type of change

  • Bug fix
  • Feature (requires approved Discussion)
  • Refactor (no behavior change)
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm --silent lint:json | jq '.diagnostics | length' returns 0
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/...

AI-generated code disclosure

  • This PR includes AI-generated code

This contribution was developed with AI assistance (Codex).

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 6, 2026

🦋 Changeset detected

Latest commit: beefc8a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/plugin-ai-moderation Patch
@emdash-cms/plugin-atproto Patch
@emdash-cms/plugin-audit-log Patch
@emdash-cms/plugin-color Patch
@emdash-cms/plugin-embeds Patch
@emdash-cms/plugin-forms Patch
@emdash-cms/plugin-webhook-notifier Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Apr 6, 2026

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@333

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@333

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@333

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@333

emdash

npm i https://pkg.pr.new/emdash@333

create-emdash

npm i https://pkg.pr.new/create-emdash@333

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@333

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@333

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@333

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@333

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@333

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@333

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@333

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@333

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@333

commit: beefc8a

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

This may cause merge conflicts or duplicated work. A maintainer will coordinate.

@ascorbic
Copy link
Copy Markdown
Collaborator

Could you fix the conflicts? It'd be good to get this in

Adds idx_{table}_deleted_published_id on (deleted_at, published_at DESC, id DESC)
to all ec_* content tables. The frontend listing query orders by published_at but
no existing index covers it, causing full table scans on D1 (~15,883 rows per query
on a 16k post site).

Also registers migration 033 (optimize_content_indexes) which existed on disk but
was missing from the StaticMigrationProvider.

Fixes emdash-cms#277
@mvanhorn mvanhorn force-pushed the perf/277-published-at-index branch from bd3e886 to beefc8a Compare April 11, 2026 13:35
@mvanhorn
Copy link
Copy Markdown
Contributor Author

@ascorbic rebased on main (beefc8a) and resolved conflicts:

  • packages/core/src/database/migrations/runner.ts: kept the new MIGRATIONS constant from main, added "034_published_at_index": m034 to it, and removed the old inline dict in getMigrations() (now just return MIGRATIONS)
  • packages/core/tests/database/migrations.test.ts, tests/integration/database/migrations.test.ts, tests/integration/database/dialect-compat.test.ts: swapped my hardcoded migration count (33) for the new MIGRATION_COUNT constant exported from runner.ts

All 2252 tests pass locally after the rebase. Let me know if there's anything else you'd like me to tweak.

Copy link
Copy Markdown
Collaborator

@ascorbic ascorbic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ascorbic ascorbic merged commit dd708b1 into emdash-cms:main Apr 11, 2026
25 checks passed
This was referenced Apr 11, 2026
fmhall pushed a commit to fmhall/emdash that referenced this pull request Apr 13, 2026
…es (emdash-cms#333)

* perf(db): add published_at composite index for frontend listing queries

Adds idx_{table}_deleted_published_id on (deleted_at, published_at DESC, id DESC)
to all ec_* content tables. The frontend listing query orders by published_at but
no existing index covers it, causing full table scans on D1 (~15,883 rows per query
on a 16k post site).

Also registers migration 033 (optimize_content_indexes) which existed on disk but
was missing from the StaticMigrationProvider.

Fixes emdash-cms#277

* docs: add changeset for published_at index

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: frontend listing query still full-scans — missing published_at index

2 participants