feat(agents-server-ui): show per-response token usage in meta row#4502
feat(agents-server-ui): show per-response token usage in meta row#4502kevin-dp wants to merge 4 commits into
Conversation
Sums input/output tokens across every step of the run and renders them
next to the elapsed-time ticker (e.g. `Thinking · 12s · 1.2k ↑ 412 ↓`).
Counter updates at step boundaries — the LLM SDK only reports `usage`
at end-of-step, so within a single text stream the value stays flat;
tool-using runs see jumps as each step settles.
Token plumbing (additive, no migration):
- `StepValue` Zod + TS gains optional `input_tokens` / `output_tokens`
- `outbound-bridge.ts:onStepEnd` now persists the `tokenInput` /
`tokenOutput` values it was already receiving but dropping
- `IncludesStep` / `EntityTimelineStepItem` and the three step
`.select()` blocks surface the new fields
- The cached `agent_response` section gets a summed `tokens?: { input?,
output? }`, and the section-cache fingerprint includes per-step token
deltas so a late `onStepEnd` invalidates a stale section
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4502 +/- ##
=======================================
Coverage ? 56.56%
=======================================
Files ? 359
Lines ? 39243
Branches ? 11028
=======================================
Hits ? 22198
Misses ? 16974
Partials ? 71
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Electric Agents Mobile BuildLocal mobile checks ran for commit The EAS Android preview build was skipped because the |
samwillis
left a comment
samwillis
left a comment
There was a problem hiding this comment.
Interactive review with GPT.
Thanks for wiring this through. I traced the token usage path end-to-end:
pi-adapter message_end.usage
→ bridge.onStepEnd({ tokenInput, tokenOutput })
→ steps.update({ input_tokens, output_tokens })
→ timeline step rows
→ UI meta row.
Overall the stream write looks sound: token usage is attached to the step completion update, so it appears once at the end of a pure text generation step, and jumps at step boundaries for tool-using runs.
A couple of suggestions/questions:
- Can we compute the per-run total in the timeline query/view model?
Right now AgentResponseLive subscribes to run.steps and sums input_tokens / output_tokens in React, while buildAgentSection separately performs the same aggregation for materialized sections.
If TanStack DB supports this shape cleanly, I think it would be better to expose a single per-run tokens field from createEntityTimelineQuery / the includes query, e.g. via a scoped aggregate over steps for the run. Then the UI only renders run.tokens / section.tokens, and the aggregation logic lives in one layer.
Since usage only lands at step end, not token-by-token, updating the parent run/timeline row at those boundaries seems acceptable to me.
- Missing usage fields currently become real zeroes
In pi-adapter, when msg.usage exists, missing sides are coerced to 0:
...(usage && {
tokenInput: usage.input ?? usage.inputTokens ?? 0,
tokenOutput: usage.output ?? usage.outputTokens ?? 0,
}),Now that these values are persisted and displayed, that can make an unknown side look like a real 0. If pi-ai guarantees both input and output are always present, a small regression test would be useful. Otherwise I’d preserve undefined for missing sides and only write numeric values.
- Test coverage
I’d like to see a targeted regression test that proves token usage reaches the steps.update event, and another around the timeline/view-model total if we move the aggregation there. That would lock down the important stream contract introduced by this PR.
|
👍 yeah just showing total tokens for the run at the bottom makes sense to me |
… layer
Move the input/output token summation out of `AgentResponseLive` and
`buildAgentSection` into a single `leftJoin` against a per-run aggregate
subquery (groupBy run_id, sum + count) in both `createEntityTimelineQuery`
and `createEntityIncludesQuery`. Consumers now read `run.tokens` directly
without re-summing step rows.
- `IncludesRun` and `EntityTimelineRunRow` gain `tokens?: { input?, output? }`.
- `buildIncludesRuns` (in-memory builder) computes the same shape from
materialized steps.
- `fingerprintRun` hashes the resolved tokens instead of per-step deltas.
- `AgentResponseLive` drops the React-side step reducer; coerces
TanStack DB's SQL-style `null` for absent sides to `undefined` so
`TokenUsage`'s `!= null` checks stay correct.
- Adds 7 regression tests covering the live query path, in-memory
builder, and section plumbing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-adapter `pi-adapter`'s `message_end` handler was coercing a missing `usage.input` / `usage.output` to `0` via `?? 0`, making an unreported side indistinguishable from a real zero-token step in the rendered meta row. With the per-run aggregate now landing in the query layer (see prior commit), a fabricated `0` would also poison `count(input_tokens)` / `count(output_tokens)`, marking absent sides as present. Forward `undefined` for any side that doesn't arrive as a `number`; `onStepEnd` already conditionally writes those columns, so the `steps` row stays null on the missing side. Adds regression tests for both ends of the contract: - `outbound-bridge.test.ts`: `onStepEnd` with token fields produces a `steps.update` event whose `input_tokens` / `output_tokens` match, and omits a column when the corresponding arg is undefined. - `pi-adapter.test.ts`: a synthetic `message_end` with a `Usage` payload routes through to the step update; deleting a side from the payload omits that column instead of writing `0`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pi-ai's `Usage` splits the input side across three counters: `input` (new uncached tokens), `cacheRead` (prompt-cache hits — the system prompt + history once the cache is warm) and `cacheWrite` (tokens added to the cache this turn). The adapter was reading only `usage.input`, which on second+ turns of any cache-using provider (Anthropic, etc.) collapses to a handful of tokens because everything else hits the cache. The meta row was showing "3 ↑" regardless of how large the actual prompt was. Sum all three input-side counters into `tokenInput` so the displayed total reflects the prompt volume the model actually saw. `inputTokens` remains as a legacy flat-field fallback for non-pi-ai providers that don't split the cache columns. Adds a regression test that a `Usage` payload with all three counters populated yields the sum on the `steps.update` event. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for electric-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
3 participants
Summary
Adds a token-usage label to the agent response meta row, e.g.
Thinking · 12s · 1.2k ↑ 412 ↓while streaming and✓ done in 12s · 1.2k ↑ 412 ↓ · 14:18once settled. Counter updates atstep boundaries — for a single-turn LLM call it lands once at done;
for tool-using runs it jumps as each step completes (the LLM SDK only
emits
usageat end-of-step, so we can't tick smoothly betweenstreamed tokens — the elapsed-time ticker still ticks every second
alongside it).
Plumbing
The runtime already had the token data —
pi-adapter.ts:358-359extracts
tokenInput/tokenOutputfrom the provider's per-stepusagepayload — but the bridge silently dropped them beforepersistence. This PR closes that gap and surfaces them all the way to
the UI:
StepValuegains optionalinput_tokens/output_tokenscolumns(Zod + TS). Strictly additive: events recorded before this change
still validate (both fields optional), so no migration is needed.
outbound-bridge.ts:onStepEndnow persists the values it wasalready receiving from
pi-adapter.ts.IncludesStep/EntityTimelineStepItemsurface the new fields,and the three
.select()blocks that materialize step rows includethem.
agent_responsesection grows atokens?: { input?, output? }summed across the run's steps atsection-build time, and the
fingerprintRuncache key includesper-step token deltas so a late-arriving
onStepEndinvalidates astale cached section.
<TokenUsage>component inagents-server-uiwithtabular-numsso digits don't jitter, locale-aware compactformatting via
Intl.NumberFormat. Renders next to<ElapsedTime>in both the live and cached meta rows.
Test plan
pnpm typecheckclean inagents-runtime+agents-server-uipnpm testinagents-server-ui(66 passed)pnpm test outbound-bridge use-chat entity-timelineinagents-runtime(74 passed)agents-runtimetest suite: my branch matches the samepre-existing 401 failures observed on clean
main(unrelatedpermission-system breakage in the test harness, not introduced
by this PR)
jump at each step boundary
Notes
persisted (older
stepsrows lack the columns). Thetokensfieldis conditional on at least one step reporting a number, so those
sections continue to render with no token row instead of "0 / 0".
1.2k ↑ 412 ↓chosen for compactness in the metarow. Open to changing to
1.2k in / 412 outor similar if thearrow direction is unclear — input goes up to the model, output
comes down.
🤖 Generated with Claude Code