Skip to content

docs(skills): SQL-first contract with 4-call ceiling for pipeline audits#4

Merged
alexskatell merged 1 commit into
mainfrom
feat/sql-first-skills
May 13, 2026
Merged

docs(skills): SQL-first contract with 4-call ceiling for pipeline audits#4
alexskatell merged 1 commit into
mainfrom
feat/sql-first-skills

Conversation

@alexskatell
Copy link
Copy Markdown
Contributor

Summary

Brings the bundled skills/hermes/SKILL.md (0.2.0 → 1.2.0) and skills/claude-code/SKILL.md in line with the SQL-first contract validated in production this week. Any agent that installs os-cli will now get the same playbook our internal agents are using, instead of the older REST-first guidance.

The contract change

For any "what happened in pipeline X over window W" question, the agent runs exactly:

  1. date (only if window is custom/relative)
  2. topline --agent query sql --sql '<freshness check>' — one call covering messages, opportunities, call_events, appointments
  3. topline --agent query sql --sql '<composite audit>' — one call returning open count/value, stage rollup, activity by channel/direction, moved/created/closed deals, top active deals
  4. Answer

Hard ceiling: 4 tool calls after skills load. No pipeline audit, no opportunities search, no conversations search, no Python verification loops.

Why this matters

Last week's qualified-pipeline test ran 15+ tool calls because the previous skill said "3–5 target" (soft), told the agent to use SQL-first and verify-the-negative with pipeline audit, and auto-fallback to REST when SQL was "fresh but incomplete". Three contradictory instructions = a thrashy agent. This PR makes the contract hard and removes the auto-fallback so warehouse coverage gaps surface as os-mcp bugs instead of getting papered over.

What changed

  • skills/hermes/SKILL.md v0.2.0 → v1.2.0
    • New ## Standard pipeline audit contract (hard limit — 4 calls) section
    • topline pipeline audit demoted to "diagnostic / one-off only — NOT the analytics default"
    • TOPLINE_QUERY_TOKEN documented (connection-bound, from https://os-mcp.topline.com/connect; raw PITs intentionally rejected for SQL)
    • New pitfalls: starting analytics with REST when SQL is available; running both SQL and pipeline audit "to verify"; auto-falling back to REST when SQL is partial; mislabeling SQL/native disagreements as sync lag
  • skills/claude-code/SKILL.md: mirrors the same contract in shorter form for Claude Code consumers

What didn't change

  • No code changes. No new CLI surface. No new env vars beyond documenting TOPLINE_QUERY_TOKEN, which already shipped in Add hosted warehouse query commands #3.
  • topline pipeline audit still exists and works; only its role in the default audit flow changed.

Test plan

  • Fresh agent thread asks "What activity happened this week in our qualified pipeline?"
  • Trace shows ≤4 tool calls after skills load (date, freshness SQL, composite SQL, answer)
  • No pipeline audit call in the default flow
  • If SQL coverage is partial, answer discloses the gap and stops (no silent REST fallback)
  • Drilldown still works when explicitly requested ("drill into deal X" → opportunities/conversations/messages calls)

Follow-ups (separate PRs)

Tracked in docs/plans/2026-05-13-sql-first-retrieval-improvements.md:

  • Phase 1: topline query doctor — token/base-URL/schema reachability probe
  • Phase 5.1: scripts/install-local.sh + README — fixes the wrapper-drift footgun where installers have to hand-edit the launcher to allowlist TOPLINE_QUERY_TOKEN
  • os-mcp views (separate repo): pipeline_activity_window, pipeline_snapshot, pipeline_movement_window, warehouse_freshness, plus updating contact_timeline to UNION appointments and call_events

Bumps bundled skills to match the SQL-first contract validated in
production this week. Default analytics path is now the hosted
warehouse via `topline --agent query`; `pipeline audit` is demoted
to diagnostic / one-off use only.

skills/hermes/SKILL.md (0.2.0 -> 1.2.0):
- Adds "Standard pipeline audit contract" with hard 4-call ceiling:
  date -> freshness SQL -> composite SQL -> answer.
- Demotes `topline pipeline audit` to diagnostic/one-off, not the
  default for "what happened this week in pipeline X" questions.
- Adds `TOPLINE_QUERY_TOKEN` (connection-bound, from
  https://os-mcp.topline.com/connect) as the auth surface for SQL.
- Adds pitfalls: starting analytics with REST when SQL is available;
  running both SQL and pipeline audit "to verify"; auto-falling back
  to REST when SQL is partial; mislabeling SQL/native disagreements
  as sync lag.

skills/claude-code/SKILL.md: mirrors the same contract in shorter
form for Claude Code consumers.
@alexskatell alexskatell merged commit 4c00e60 into main May 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant