Feature: Time-decay and usage-weighted relevance scoring for search

[bm-clawd]

Motivation

Currently all notes/entities are weighted equally in search results. A stale observation from three weeks ago ranks the same as a correction that saved 30 minutes yesterday. As knowledge bases grow, retrieval quality degrades because there's no signal for which memories are actually useful.

Inspired by memelord's approach (Turso blog), which combines cosine similarity with time decay and usage weighting:

ORDER BY (1.0 - vector_distance_cos(embedding, vector32(?)))
  * POWER(0.995, julianday('now') - julianday(created_at))

Proposed Changes

Phase 1: Time-decay scoring (low effort, high impact)

• Add a time-decay factor to search result ranking
• Recent notes score higher than old ones at equal similarity
• Configurable decay rate (default ~0.995/day means half-life of ~138 days)
• This is purely a scoring change — no schema or storage changes needed

Phase 2: Usage-weighted scoring (medium effort)

• Track when a note/entity is retrieved via search or build_context
• Track when retrieved notes are actually used in the conversation (accessed via read_note after appearing in search results)
• Boost frequently-useful notes, decay ignored ones
• Could be as simple as a last_accessed_at + access_count on entities

Phase 3: Feedback loop (higher effort, ties into SPEC-28)

• Allow agents to explicitly rate memory usefulness (upvote/downvote)
• Automatic contradiction detection — when an agent stores a correction, the corrected memory gets downweighted
• Self-cleaning: memories below a weight threshold get flagged for review/archival

Why This Matters

BM's differentiator is human-readable plain text. But search ranking doesn't have to be naive just because storage is simple. We can have sophisticated retrieval on top of transparent storage.

The memelord approach validates this — they use SQLite + vector search + weighted scoring. We already have the first two. Adding weighted scoring closes the gap while keeping our plain-text advantage.

References

• Turso/memelord blog: https://turso.tech/blog/giving-openclaw-a-memory-that-actually-works
• memelord repo: https://github.com/glommer/memelord
• Letta sleep-time compute (similar concept): memory consolidation + re-ranking during idle time
• Our memory-reflect skill already does manual consolidation — this would automate the ranking part

[claude]

Claude finished @bm-clawd's task in 1m 32s —— View job

---

Issue Triage

• Analyze issue and classify
• Check for duplicate issues
• Review current search implementation
• Apply labels
• Provide triage summary

---

Triage Summary

Type: Enhancement / Feature Request
Priority: Medium
Component: Database (search), MCP tools
Complexity: Complex (multi-phase)
Labels applied: enhancement

---

Analysis

No duplicate issues found.

What the issue proposes: Three phases of smarter search ranking:

1. Phase 1 (Time-decay) — Multiply BM25 score by a time-decay factor based on created_at/updated_at. The issue calls this "low effort, high impact" since no schema changes are needed.
2. Phase 2 (Usage tracking) — Add last_accessed_at + access_count to entities and boost frequently-retrieved notes.
3. Phase 3 (Feedback loop) — Explicit agent ratings, contradiction detection, auto-archival. Ties into SPEC-28.

Current search implementation: Looking at sqlite_search_repository.py:819, the FTS query currently uses bm25(search_index) as score with no temporal weighting. The entity model already has created_at and updated_at fields with indexes (ix_entity_created_at, ix_entity_updated_at). The search index also has created_at and updated_at columns, making Phase 1 feasible without schema changes.

Phase 1 feasibility: Essentially straightforward — the search SQL at line 802-825 could incorporate a decay multiplier like the Turso blog example:

bm25(search_index) * POWER(0.995, julianday('now') - julianday(search_index.updated_at)) AS score

The tricky part is:

• BM25 in SQLite FTS5 returns negative values (lower = more relevant), so the formula needs adjustment
• The decay should be optional/configurable, not always-on
• The Postgres search repository would need equivalent logic

Phase 2 feasibility: Requires schema changes — adding last_accessed_at and access_count columns to the entity or search_index table, plus Alembic migrations. Medium effort, touches MCP tools (search, read_note, build_context all need to update these counters).

Phase 3 feasibility: High complexity — requires explicit agent APIs, contradiction detection logic, and memory consolidation. Should be a separate issue/SPEC.

Recommendations

1. Phase 1 should be scoped as a standalone PR — it's genuinely low-effort and high-value. The decay factor could be a configurable setting (default off, or a very mild default like 0.999).
2. Phase 2 warrants its own issue since it requires schema migrations and coordination across multiple MCP tools.
3. Phase 3 is a longer-term research item, best tracked via SPEC-28 reference.

The Turso/memelord approach validates the pattern. BM25 note: SQLite FTS5's bm25() returns negative values where more negative = better match, so the decay formula needs to account for the sign when combining with time decay. A rank normalization step would be needed for a clean multiplicative combination.

[bm-clawd]

Config note: Time-decay scoring should be opt-in via project config, not default behavior. Some use cases (research archives, documentation, permanent knowledge bases) need stable ranking regardless of age. Others (agent memory, daily workflows, conversation context) benefit from recency bias.

Proposed config shape:

[search]
time_decay = true          # enable/disable
time_decay_rate = 0.995    # per-day decay factor (0.995 = ~138 day half-life)

Could also support per-note opt-out via frontmatter (e.g. pinned: true or decay: false) for notes that should never fade regardless of project setting.