feat(engine): Phase 11.9 WAL log-record durability + crash recovery (SQLR-22)

[joaoh82]

Summary

• Closes the long-running plan-doc Phase 10.5 crash-recovery half. Pre-11.9 BEGIN CONCURRENT commits durably persisted table state through the legacy save_database mirror, but MvStore's version index was reborn empty on every reopen — fine for single-session workloads, but breaks the visibility rule once cross-process MVCC is in scope.
• Adds a typed MvccCommitBatch WAL frame (sentinel page_num = u32::MAX) appended before the legacy save's commit-frame fsync, so one fsync covers both writes — torn-write atomicity for the whole transaction.
• On reopen, the WAL replay decodes every MVCC frame and re-pushes the row versions into MvStore, then seeds MvccClock past max(header.clock_high_water, max(commit_ts among replayed batches)) so post-restart transactions can never regress.

Architecture

                           Phase 11.9 commit path
                           ─────────────────────────────────
                                       │
                          ┌────────────┴───────────────┐
                          │   commit_concurrent        │
                          │                            │
                          │ 1. validate write-set      │
                          │ 2. push_committed → MvStore│
                          │ 3. apply diff to db.tables │
                          │ 4. ⮕ append_mvcc_batch     │   ← buffered (no fsync)
                          │ 5. ⮕ save_database         │   ← commit-frame fsync
                          └────────────────────────────┘     covers both writes

                           WAL frame stream after the commit:
                           ┌──────────────┬──────────────┬──────────────┐
                           │ page-N dirty │ MVCC marker  │ page-0 COMMIT│
                           │ (legacy)     │ (u32::MAX)   │ + fsync      │
                           └──────────────┴──────────────┴──────────────┘

Changes

• src/mvcc/log.rs: MvccCommitBatch + MvccLogRecord types and codec (MVCC0001 magic + commit_ts + record stream, fits one frame body). 6 codec tests.
• src/sql/pager/wal.rs: WAL_FORMAT_VERSION 2 → 3; MVCC_FRAME_MARKER = u32::MAX as the page-number discriminator; replay branches the frame stream into a pending_mvcc list that promotes onto recovered_mvcc on each commit barrier; Wal::append_mvcc_batch + Wal::recovered_mvcc_commits accessors.
• src/sql/pager/pager.rs: Pager proxies (append_mvcc_batch, recovered_mvcc_commits, clock_high_water, observe_clock_high_water).
• src/sql/pager/mod.rs: replay_mvcc_into_db drains recovered batches into Database::mv_store and seeds MvccClock at open time.
• src/connection.rs: commit_concurrent encodes the resolved write-set into an MvccCommitBatch, appends it pre-save_database, and bumps the in-memory WAL header's clock_high_water. 6 durability tests cover round-trip persistence, multi-row batches, ROLLBACK-no-frame, legacy-commit-no-frame, multi-commit replay after unclean close, and clock seeding past the last commit_ts.

Docs

• docs/roadmap.md: Phase 11.9 promoted to shipped; remaining checkpoint-drain half scoped as a follow-up.
• docs/file-format.md: WAL header v3 row + MVCC log-record body diagram + frame-marker convention.
• docs/concurrent-writes-plan.md: plan-doc §10.5 annotated with what shipped vs. what's parked.
• docs/design-decisions.md: new §12g covering the piggyback-fsync rationale, sentinel choice, and clock-seed correctness argument.
• docs/_index.md: phase-summary bullet refreshed.

What 11.9 deferred

The other half of plan-doc §10.5 — extending the checkpointer to drain MVCC log records into pager-level updates, and re-enabling the Mvcc → Wal set_journal_mode downgrade — stays parked. Durability on the read path already works through the legacy save_database mirror, so this gap is foundation work for cross-process MVCC, not a correctness regression. The existing per-commit GC bounds in-memory chain growth.

Test plan

• cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets
• cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks — 599 / 599 (was 587, +12 = 6 codec + 6 durability)
• cargo fmt --all -- --check
• cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets
• cargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps
• CI green
• Smoke-test by hand: open existing v0.9.1 database files (v1 WAL, v2 WAL with clock high-water set) and confirm the version bump still accepts them.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
rust-sqlite	Ready	Preview, Comment	May 11, 2026 7:39am