Skip to content

feat(market): yfinance alignment + per-card tooltips + auto-refresh#138

Merged
luokerenx4 merged 10 commits intomasterfrom
dev
Apr 23, 2026
Merged

feat(market): yfinance alignment + per-card tooltips + auto-refresh#138
luokerenx4 merged 10 commits intomasterfrom
dev

Conversation

@luokerenx4
Copy link
Copy Markdown
Contributor

@luokerenx4 luokerenx4 commented Apr 22, 2026

Summary

Six commits addressing three fronts surfaced while shipping the equity workbench:

yfinance field alignment (opentypebb)

  • toSnakeCase was inserting underscores before every uppercase letter, so EBITDA became e_b_i_t_d_a and NetPPE became net_p_p_e. Switched to a two-pass regex that preserves acronym runs (ebitda, net_ppe, ebitda_margin).
  • Each yfinance fetcher (quote, key-metrics, balance, income, cash-flow) had a partial alias dict canonicalizing to its own names. Reworked all five to emit the same snake_case canonical fields FMP emits: price_to_earnings / ev_to_ebitda / ev_to_sales / price_to_sales / gross_profit_margin / operating_profit_margin / net_profit_margin / ma50 / ma200 / market_cap / revenue / total_operating_income / income_tax_expense / consolidated_net_income / basic_earnings_per_share / diluted_earnings_per_share / short_term_debt / total_current_liabilities / total_liabilities / plant_property_equipment_net / net_cash_from_operating_activities / investing_activities / financing_activities / cash_at_end_of_period.
  • Quote fetcher was also missing change / change_percent / last_timestamp aliases (→ null values despite Yahoo returning them).
  • Quote schema was .strip()ing marketCap because neither the alias dict nor the schema extension declared it.

UI panel chrome

  • Each market Card now carries a free-form info prop surfaced as a hover tooltip on a small circled "i" next to the title. Profile / Key Metrics / Financial Statements fill it with: data source (provider name), endpoint path, conventions in play (TTM, annual, B/M scaling), and provider-specific caveats (ratios not reported on yfinance). First cut used the native title= attribute but its ~700ms browser delay reads as "nothing is happening"; switched to a CSS-only group-hover tooltip with zero delay, multi-line rendering, and app-consistent styling.
  • KeyMetricsPanel used Promise.all for metrics + ratios, which fails fast — a 500 on ratios discarded the successful metrics response along with it. Switched to Promise.allSettled so a single rejection doesn't nuke the other source.

Auto-refresh

  • A tab left open overnight was showing yesterday's last print, then jumping when the user triggered any refetch. Polling added where it matters: QuoteHeader 60s, KlinePanel 60s intraday / 5min daily, KeyMetricsPanel 5min. Profile and Financial Statements don't poll — they don't change on that timescale.
  • Refreshes are silent (no spinner flicker); initial load still shows loading. clearInterval + a cancelled flag handle unmount and symbol changes without double-applying stale state.

Additional work accumulated on dev

Three more commits landed on dev between the original push and merge; rolling them into the same PR rather than fragmenting.

Event system docs (6c12cc1)

docs/event-system.md — recipe-oriented walkthrough of how events, Listeners, and Producers fit together, with checklists for each common change (new event type, new Listener, new Producer, opening an event to HTTP) and a common-pitfalls section covering the load-bearing wiring AI sessions have historically half-done (forgetting as const on emit tuples, calling eventLog.append directly instead of declaring a Producer, opening an event to HTTP without extending webhook-ingest's tuple, registry name collisions, wildcard emit misunderstandings). CLAUDE.md gains a "Subsystem guides" pointer so future sessions read it before touching the area.

Shared connector producer (68b8e3e)

Per-connector producers (web-chat, telegram-connector) were declaring identical event tuples (['message.received', 'message.sent']) with identical payload shapes — the channel field already carried source attribution, so the two producer nodes on the Flow graph were indistinguishable. Moved the pump up to ConnectorCenter: one connectors producer declared on construction (conditional on ListenerRegistry availability, mirroring the interaction-tracker pattern), typed emitMessageReceived / emitMessageSent helpers, and symmetric stop() teardown. WebPlugin and TelegramPlugin dropped their own producer wiring and call the center directly. main.ts shutdown order: connectorCenter.stop() before listenerRegistry.stop() so the producer name is released before the registry tears down. Event-system guide updated with a special-case note so future connector authors don't reinvent per-plugin message producers.

Chat history — infinite scroll up (5519b84)

Chat had a hard-cap of 100 messages, so conversations longer than that lost earlier context — scrolling up hit the top of the loaded window but never fetched the rest. Replaced with cursor-based pagination:

  • BackendGET /api/chat/history now accepts before=<uuid> and returns { messages, hasMore }. toChatHistory attaches a cursor (= first-entry uuid) to every ChatHistoryItem — including merged tool_calls items — so the client can pass the oldest one back as before for the next page.
  • FrontenduseChat exposes loadMore(), hasMore, isLoadingMore. Channel switch resets pagination state. Uses refs internally so loadMore is referentially stable (prevents ChatPage's scroll listener from re-registering on every stream tick).
  • ChatPage — fires loadMore when scrollTop < 200, hasMore, not already loading. Captures scrollHeight - scrollTop before fetch; after prepend lays out, useLayoutEffect restores scrollTop so the viewport stays anchored to the same visual position. If the user returned to the bottom during the fetch, restoration is skipped — auto-scroll-to-bottom wins. A separate effect clears the captured offset if the fetch returned empty, so the next unrelated messages change doesn't use a stale offset. Top-of-container status line: "Loading older messages…" / "— beginning of history —".

Initial page size dropped 100 → 50; subsequent pages 50 each.

Test plan

  • npx tsc --noEmit clean
  • pnpm test1097 tests / 56 files pass (1088 → 1097 after the shared connector producer added 10 cases; chat pagination adjusted one existing cursor assertion)
  • yfinance equity endpoints all return canonical field names (P/E, ROE, gross_profit_margin, revenue, total_operating_income, ebitda, ebit, net_cash_from_*, ma50, market_cap, change, change_percent, last_timestamp)
  • Non-FMP users see Key Metrics populate (was showing error due to Promise.all fail-fast on the missing ratios endpoint)
  • Hover on ⓘ icon shows multi-line tooltip instantly (Profile / Key Metrics / Financials)
  • QuoteHeader and KlinePanel pick up fresh data without manual refresh
  • /api/topology producers list: 3 (was 4) — cron-engine, connectors, webhook-ingest
  • POST /api/chat fires message.received via shared producer, payload.channel = 'web', seq chain intact; connector-interaction-tracker still updates lastInteraction
  • Chat: long conversations (>50 entries) scroll up progressively — older pages prepend without viewport jump; reach "— beginning of history —" at the active-window boundary

🤖 Generated with Claude Code

Ame and others added 10 commits April 22, 2026 09:39
OpenBB's point is one schema across providers, but the TypeScript port
had drifted: FMP canonicalized things like price_to_earnings / ev_to_ebitda
/ gross_profit_margin / revenue / total_operating_income while yfinance
emitted pe_ratio / enterprise_to_ebitda / gross_margin / total_revenue /
total_operating_income_as_reported, so any UI built against one
provider's output saw dashes on the other.

Two roots of the drift:

- `toSnakeCase` used a naive `replace(/([A-Z])/g, '_$1')` that inserted a
  separator before every uppercase letter, so `EBITDA` became
  `e_b_i_t_d_a` and `NetPPE` became `net_p_p_e`. Switched to the standard
  two-pass regex that respects acronym runs — acronym boundaries only
  split when a lowercase follows (EBITDAMargin → ebitda_margin).
- Each yfinance fetcher (quote, key-metrics, balance, income, cash-flow)
  had a partial alias dict using its own canonical names. Reworked to
  mirror FMP's: price_to_earnings / ev_to_ebitda / ev_to_sales /
  price_to_sales / gross_profit_margin / operating_profit_margin /
  net_profit_margin / ma50 / ma200 on metrics and quote; revenue /
  total_operating_income / income_tax_expense / consolidated_net_income
  / basic_earnings_per_share / diluted_earnings_per_share on income;
  short_term_debt / total_current_liabilities / total_liabilities /
  plant_property_equipment_net on balance; net_cash_from_operating_activities
  / investing_activities / financing_activities / cash_at_end_of_period on
  cash-flow.

Genuine gaps (yfinance never returns them regardless of aliasing) stay
as dashes, not zeros — that distinction was already correct in format.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two small UX fixes after aligning yfinance to the canonical schema.

Each Card now accepts a `source` prop that renders a subtle ⓘ next to
the title with a native tooltip exposing which provider produced the
data. Quote header and K-line already surfaced the provider as a badge;
the fundamentals cards were silent until now. Profile, Key Metrics, and
Financial Statements thread their response's `provider` into this slot.

KeyMetricsPanel used to treat any sub-endpoint error as fatal, even
when only one of the two merged sources (metrics + ratios) failed. That
over-eager error path hid valid yfinance data because yfinance hasn't
implemented FinancialRatios and always 404s it. Treat the sources
independently: if at least one returns a row, render; surface an error
only when both are empty.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the yfinance alignment.

yfinance quote was dropping marketCap on the floor — the schema used
.strip() and the alias dict didn't map it, so Yahoo's marketCap field
passed through extraction but then got stripped by Zod. Added the
alias and a schema extension, matching how FMP's fetcher handles it.

KeyMetricsPanel fired fetchJson for both metrics and ratios inside
Promise.all. opentypebb returns 500 when a fetcher doesn't exist, so
ratios on yfinance always rejected — and Promise.all's fail-fast
discarded the successful metrics response along with it. Switched to
Promise.allSettled so a 500 on one source doesn't nuke the other.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The first attempt hooked the info icon up to the native HTML title
attribute, which carries a ~700ms browser-level delay that reads as
"nothing is happening" — and the OS help cursor visible during that
window looks like a bare question mark floating next to the pointer,
amplifying the confusion.

Replaced with a Tailwind group/group-hover tooltip: zero delay, styled
to match the app chrome, whitespace-pre-line for the multi-line info
content, pointer-events-none so it doesn't eat hover from neighbours,
w-max + max-w-sm so it sizes to content without running off screen.

Also thickened the info content itself — each card now explains where
the data came from (provider + endpoint path), what convention applies
(TTM vs annual, scaling), and flags provider-specific caveats (ratios
missing on yfinance). Room to grow; future notes just push onto the
info line list in each panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…timestamp

yahoo-finance2 returns these under regularMarket* keys but the fetcher's
ALIAS_DICT only covered price/volume/MA/market_cap — so UI consumers
reading the canonical snake_case fields saw null change and no
timestamp. Added the three missing aliases; change_percent comes back
in decimal form (matches canonical), last_timestamp already arrives as
a date string so no post-processing needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A tab left open overnight happily showed yesterday's last print as if
it were live until the user switched provider and saw the number jump.
The mismatch wasn't between providers — it was between what the page
had fetched at open time and what the market had done since.

Polling added where it matters:

- QuoteHeader: 60s, last_price and change move constantly.
- KlinePanel: 60s for intraday intervals (1m / 5m / 1h — a tick
  is a fresh bar), 5min for daily (same-day refresh is cosmetic).
- KeyMetricsPanel: 5min — market cap and EV track price but users
  don't watch them tick-by-tick.
- ProfilePanel, FinancialStatementsPanel: no polling. Company
  overview and quarterly financials don't change on that timescale.

Each refresh is silent (no loading flicker), and the initial fetch
stays as-is so the first render still shows a spinner. clearInterval
+ a cancelled flag handle unmount and symbol changes without
double-applying state from stale requests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds docs/event-system.md — a recipe-oriented walkthrough of how
events, Listeners, and Producers fit together, with checklists for
each common change (new event type, new Listener, new Producer,
opening an event to HTTP) and a common-pitfalls section covering the
load-bearing wiring AI sessions have historically half-done:

- Forgetting `as const` on emit tuples
- Calling eventLog.append directly instead of declaring a Producer
- Opening an event to HTTP without extending webhook-ingest's tuple
- Registry name collisions between Listeners and Producers
- Wildcard emit misunderstandings

CLAUDE.md gains a "Subsystem guides" section that directs future
sessions to read the guide before touching this area.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-connector producers were duplicative by construction — `web-chat`
and `telegram-connector` both declared `['message.received', 'message.sent']`
with identical payload shapes, and the `channel` field already carried
source attribution. The structure forced every new connector to reinvent
the declare/emit/dispose wiring and rendered two indistinguishable
producer nodes on the Flow graph.

This moves the pump to the Connector abstraction layer:

- ConnectorCenter declares a single `connectors` producer on
  construction (conditional on ListenerRegistry availability, mirroring
  the existing interaction-tracker pattern), and exposes typed
  `emitMessageReceived` / `emitMessageSent` methods that throw loudly if
  the center was built without a registry.
- `stop()` disposes the producer and unregisters the
  `connector-interaction-tracker` listener — symmetric teardown.
- WebPlugin drops its `chatProducer` field + declaration + dispose;
  chat.ts drops `producer` from ChatDeps and calls
  `ctx.connectorCenter.emitMessage*` directly.
- TelegramPlugin drops its `producer` field, declaration, and dispose;
  handleMessage calls `engineCtx.connectorCenter.emitMessage*`.
- main.ts shutdown calls `connectorCenter.stop()` before
  `listenerRegistry.stop()` so the producer's name is released before
  the registry is torn down (avoids leaks in test suites that spin
  registries up/down).
- Event system guide updated: canonical producer list swaps the old
  per-plugin entries for the shared `connectors` producer, and the
  "add a new Producer" recipe gains a special-case note telling future
  connector authors to use the ConnectorCenter API instead of declaring
  their own message producer.

Connector interface itself is unchanged. Individual Connector classes
(WebConnector, TelegramConnector, McpAskConnector, MockConnector) are
unchanged — they never emitted events, that was always plugin-level
responsibility.

End-to-end verification against running dev server:
- /api/topology producers: 3 (was 4) — cron-engine, connectors, webhook-ingest
- POST /api/chat fires `message.received` via the new producer,
  payload.channel = 'web', seq chain intact
- connector-interaction-tracker still picks up the event and updates
  `lastInteraction`

Tests: 1097 pass (10 new cases covering the shared producer — list
membership, typed emit success paths, no-registry error paths, stop()
semantics).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Chat page has always hard-capped history at the last 100 messages,
so conversations longer than that lost everything earlier — scrolling
up hit the top of the loaded window but never fetched the rest. Replace
the cap with cursor-based pagination.

Backend
- GET /api/chat/history now accepts `before=<uuid>` and returns
  `{ messages, hasMore }`. When `before` is set, the server slices
  entries appearing BEFORE that uuid and takes the last `limit` of those.
- toChatHistory now attaches a `cursor` field (= first-entry uuid) on
  every ChatHistoryItem so the client can pass the oldest one back as
  `before` for the next page. Cursor is preserved even for tool_calls
  items that merge tool_use + tool_result entries.

Frontend
- useChat exposes loadMore(), hasMore, isLoadingMore. Channel switch
  resets pagination state. loadMore reads the oldest loaded item's
  cursor, fetches the next 50, prepends. Uses refs internally so the
  callback is referentially stable — prevents ChatPage's scroll
  listener from re-registering on every stream tick.
- ChatPage fires loadMore when scrollTop < 200px, hasMore, and not
  already loading. Captures `scrollHeight - scrollTop` before the
  fetch; after prepend lays out, useLayoutEffect restores scrollTop so
  the viewport stays anchored to the same visual position. If the user
  returned to the bottom during the fetch, restoration is skipped —
  auto-scroll-to-bottom wins. A separate effect clears the captured
  offset if the fetch returned empty, so the next unrelated messages
  change doesn't use a stale offset.
- Top-of-container status line: "Loading older messages…" during fetch,
  "— beginning of history —" when the server reports no more.

Initial page size dropped 100 → 50; subsequent pages also 50. Readable
on first paint, progressive load on demand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The component had `const setInterval = (iv: Interval) => ...` as a
URL-sync setter. In the useEffect below it, `setInterval(() => fetch(false), pollMs)`
resolved to this local setter instead of the global timer, so tsc
reported "Expected 1 arguments, but got 2" and clearInterval saw the
setter's `void` return. Renamed the local setter to `selectInterval`.
@luokerenx4 luokerenx4 merged commit 9d9a5a9 into master Apr 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant