feat(retention): [retention] manifest section + verisimiser gc subcommand#87
Merged
Conversation
…ommand Closes #50. Sidecar tables grew unboundedly. README mentioned retention; the data model didn't. Manifest: - New `[retention]` section with three `*-days` fields (`provenance-days`, `temporal-days`, `lineage-days`). `0` means "keep forever" — the default. Modelled as `RetentionConfig` with serde renames so the TOML field names stay kebab-case. - `init` template emits the section with explanatory comments, pulling defaults from `RetentionConfig::default()` (continues the V-L2-O1 "no string drift between code defaults and template" invariant). New module `src/gc.rs`: - `run_gc(&Manifest, dry_run: bool) -> Result<GcReport>` opens the SQLite sidecar via rusqlite and DELETEs rows whose timestamp is older than `now - retention_days` per dimension. - `dry_run = true` only counts candidates (no writes), so users can audit before applying. - `temporal_versions` purge is scoped to `valid_to IS NOT NULL` so the *current* version is never deleted regardless of age. - Non-SQLite sidecar backends fail loudly with a typed error ("only supports the SQLite sidecar backend") rather than silently no-op'ing. - `GcReport` is `Serialize` so `--json` can emit a structured report (sidecar path, dry-run flag, per-dimension counts, total). New CLI: `verisimiser gc [--manifest <path>] [--dry-run] [--json]`. New dep: `rusqlite = { version = "0.32", features = ["bundled"] }`. The `bundled` feature compiles SQLite from source so users don't need a system libsqlite3 install. Adds ~1MB to the binary; outweighed by deployment simplicity. Tests in `gc::tests` (require rusqlite at compile time but use only in-tempdir SQLite files): - `gc_dry_run_counts_but_does_not_delete` — dry-run reports correct counts; row count in DB is unchanged. - `gc_apply_deletes_old_rows` — real purge deletes the right rows; crucially, the current (`valid_to IS NULL`) temporal version survives even though its `valid_from` is old. - `gc_retention_zero_is_forever` — every retention=0 means total=0 deletions. - `gc_rejects_non_sqlite_backend` — non-SQLite storage returns the typed error. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Per V-L2-P1: sidecar tables grew unboundedly. Adds retention bounds + a purge subcommand.
[retention]section in manifest withprovenance-days/temporal-days/lineage-days(0= forever, default).inittemplate emits the section with the same Default-driven approach as the rest of V-L2-O1.src/gc.rsmodule:run_gc(&Manifest, dry_run) -> Result<GcReport>. Opens SQLite sidecar via rusqlite, DELETEs rows older thannow - retention_daysper dimension. Crucially, temporal purge is scoped tovalid_to IS NOT NULLso the current version is never deleted regardless of age.verisimiser gc [--manifest <path>] [--dry-run] [--json].rusqlite = { version = "0.32", features = ["bundled"] }. Bundled SQLite avoids system-library dependency at the cost of ~1MB binary size.Closes
Test plan
cargo clippy --all-targets -- -D warningsclean on Linux CI (local Windows build blocked by AppControl/WDAC on libsqlite3-sys build script — Windows-specific local issue)gc::tests:gc_dry_run_counts_but_does_not_deletegc_apply_deletes_old_rows(verifies current temporal version survives)gc_retention_zero_is_forevergc_rejects_non_sqlite_backendNote on local build
Windows Application Control policy is blocking the freshly-compiled
libsqlite3-sysbuild script on my dev box (os error 4551). Linux CI builds and tests should be unaffected; please flag if CI reports the same.