Skip to content

feat: Document Router Layer #100

@zTgx

Description

@zTgx

Problem

Workspace-scoped queries send all DocCards to the Orchestrator's LLM analysis phase in a single prompt. At 1000+ documents this exceeds token budgets, is expensive, and LLM selection quality degrades.

Proposal

Add a Router layer between Engine.query() and the Orchestrator that pre-filters documents using compile-stage artifacts (DocCard, ReasoningIndex, DocumentGraph) before LLM analysis.

  • BM25 on DocCard text for lexical matching
  • Keyword overlap using QueryPlan concepts vs ReasoningIndex topics
  • Graph boost from cross-document relationships
  • Optional LLM-assisted re-ranking of top candidates
  • Only activates when workspace exceeds configurable threshold (default: 20 docs)

Full RFC

See docs/rfc/router

Acceptance Criteria

  • rust/src/router/ module with DocumentRouter, RouteResult, RouterConfig
  • Three-signal scoring (BM25 + keyword + graph)
  • Incremental index maintenance on document add/remove
  • Integration into Engine.query() with activation threshold
  • RouterConfig exposed to Python SDK
  • Unit tests + 1000-document simulation test

Open Questions

  • Score normalization strategy across different corpus sizes
  • Cold start handling for new documents (no graph edges)
  • Thread safety model for mutable BM25 index (RwLock vs rebuild-on-query)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions