Skip to content

docs(analytics): fix dimension availability and label resolution drift#51

Closed
jtcies wants to merge 2 commits into
mainfrom
devin/1781445812-fix-analytics-schema-drift
Closed

docs(analytics): fix dimension availability and label resolution drift#51
jtcies wants to merge 2 commits into
mainfrom
devin/1781445812-fix-analytics-schema-drift

Conversation

@jtcies

@jtcies jtcies commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Two drift items found during daily sync against openrouter-web query builder:

  1. workspace and app dimensions were incorrectly categorized under "Limited to 31-day time ranges." Both have availableIn: ['mv', 'generations'] with MV columns (workspace_id, app_id) since migration 106 — they support up to 365 days. Moved them to the "Available with all time ranges" category.

  2. model label resolution was missing from all three skills. dimension-labels.ts resolves model permaslugs to display names via getModelNamesByPermaslugs (e.g., openai/gpt-4o-2024-08-06GPT-4o), but the skills said model was "returned as-is without resolution." Added model to label resolution tables in all three skill files.

Link to Devin session: https://openrouter.devinenterprise.com/sessions/68368ec02ac04a0c875a6b288a2b19b5
Requested by: @jtcies


Open in Devin Review

- Move workspace and app from 31-day to all-time-ranges category
  (both have MV columns since migration 106)
- Add model to label resolution tables (permaslugs are resolved
  to display names via getModelNamesByPermaslugs)

Co-Authored-By: Joseph Ciesielski <joseph.ciesielski@openrouter.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor
Original prompt from Joseph

# Daily Analytics Skills Sync

You are responsible for keeping the analytics skills in OpenRouterTeam/skills in sync with the analytics query builder in OpenRouterTeam/openrouter-web. Post your findings to the Slack channel #proj-activity-observability (C0AKMN11K7G).

#``# Step 1: Check for recent query builder changes

Clone both repos:

  • OpenRouterTeam/openrouter-web (query builder source of truth)
  • OpenRouterTeam/skills (analytics skills that document the query builder)

Check git log for the last 24 hours of changes in the query builder directory:

cd openrouter-web
git log --since="24 hours ago" --oneline -- packages/clickhouse/analytics/

Also check the analytics route handlers:

git log --since="24 hours ago" --oneline -- services/cfw-api/src/routes/analytics/

#``# Step 2: Read the current query builder state

Read these files to understand the current state of the query builder:

  • packages/clickhouse/analytics/schemas.ts — MetricNameSchema, DimensionNameSchema, Granularity, FilterOperator definitions
  • packages/clickhouse/analytics/metric-registry.ts — full metric definitions (name, displayLabel, availability, expressions, isRate, displayFormat)
  • packages/clickhouse/analytics/dimension-registry.ts — full dimension definitions (name, displayLabel, mvColumn, generationsColumn, availableIn, clickhouseType)
  • packages/clickhouse/analytics/operator-registry.ts — filter operator definitions
  • packages/clickhouse/analytics/table-resolver.ts — table resolution logic
  • packages/clickhouse/analytics/filter-builder.ts — filter building logic
  • packages/clickhouse/analytics/enrichment/ — label resolution (which dimensions get labels)
  • packages/clickhouse/analytics/trends/ — trends/leaderboard capabilities
  • services/cfw-api/src/routes/analytics/get-meta.ts — what the meta endpoint exposes

#``# Step 3: Read the current analytics skills

Read these skill files:

  • skills/openrouter-analytics/SKILL.md — main analytics ski... (2439 chars truncated...)

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Model dimension description says 'permaslug' while label resolution says 'display name'

At skills/openrouter-analytics-schema/SKILL.md:131, the dimension description reads model — the OpenRouter model ID (permaslug), while skills/openrouter-analytics-schema/SKILL.md:118 now says the dimension value is resolved to a display name (e.g., openai/gpt-4o-2024-08-06GPT-4o). These are not strictly contradictory — the dimension concept is the model (identified by permaslug) but the response value is a display name — but the juxtaposition could confuse readers. The description at line 131 was not updated to note that the returned value is the resolved display name, not the raw permaslug.

(Refers to line 131)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4de2c30 — clarified the dimension description to note that model is "identified by permaslug; resolved to display name in responses" with a cross-reference to the Label Resolution section above.

Comment thread skills/openrouter-analytics-query/SKILL.md
…lay names

- Response examples now show 'Claude Sonnet 4' / 'GPT-4o' instead of
  permaslugs, matching the label resolution behavior
- Clarify model dimension description: identified by permaslug but
  resolved to display name in responses

Co-Authored-By: Joseph Ciesielski <joseph.ciesielski@openrouter.ai>

@perry-the-pr-reviewer perry-the-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Adds model to the label-resolved dimensions list across three analytics skill docs, and fixes the time-range category for workspace and app (all-time, not 31-day limited).

Verdict: 🔁 Needs changes

Details

Risk: 🟢 Low

CI: ⏳ 1 pending: perry/review (this review — in progress)

Findings (see inline comment for full context):

  • 🟡 skills/openrouter-analytics-schema/SKILL.md:132 — dimension description says "permaslug" but label resolution table now says display name

Research: N/A — documentation-only change, trivial tier

Security: no concerns — pure documentation update

Test coverage: N/A — documentation only

Unresolved threads: 2 open Devin threads covering the same inconsistencies; Devin pushed a fix in commit 4de2c30 (on branch, not yet in PR head)

Scope: first review (full)
Review: tier=trivial · model=claude-sonnet-latest · score=0.1

Comment thread skills/openrouter-analytics-schema/SKILL.md
@perry-the-pr-reviewer perry-the-pr-reviewer Bot dismissed their stale review June 14, 2026 14:28

Superseded by updated Perry review

@perry-the-pr-maintainer perry-the-pr-maintainer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Fixes model dimension documentation inconsistency — response examples and label resolution text now consistently show display names, and the dimension description clarifies the permaslug/display-name duality.

Verdict: ✅ LGTM

Details

Risk: 🟢 Low — documentation-only PR, no runtime code changes

CI: ⚠️ 1 pending: the perry/review check (this review run — not a blocker)

Findings: none

Security: no concerns — documentation only, none of OpenRouter's 8 security categories touched

Test coverage: N/A — SKILL.md documentation, no executable code

Unresolved threads: Devin's thread (unresolved in GitHub UI) confirmed fixed in 4de2c30 by Devin's own reply; Perry's own prior REQUEST_CHANGES thread addressed by this push

Scope: incremental — 4 new lines since prior review

Review: tier=small · model=claude-sonnet-latest · score=0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant