Skip to content

feat(telemetry): full Sentry observability + per-command usage tags#75

Merged
lukeocodes merged 3 commits intomainfrom
feat/telemetry-full-sentry
May 9, 2026
Merged

feat(telemetry): full Sentry observability + per-command usage tags#75
lukeocodes merged 3 commits intomainfrom
feat/telemetry-full-sentry

Conversation

@lukeocodes
Copy link
Copy Markdown
Member

@lukeocodes lukeocodes commented May 9, 2026

Summary

Stacked on #74 (fix/telemetry-session-mode). Three commits, one logical change: take the CLI's Sentry telemetry from "errors-only" to a full observability + product-analytics surface.

What changed

1. SDK init kwargs — turn the dials on (packages/deepctl-telemetry/.../client.py)

Init kwarg Before After
traces_sample_rate 0.0 1.0
profiles_sample_rate 0.0 1.0
enable_logs (default False) True
attach_stacktrace (default False) True
max_breadcrumbs 20 100
send_default_pii False False (unchanged)
before_send=_scrub_event yes yes (unchanged)

2. Transaction wrap (src/deepctl/main.py)

cli() is now invoked inside a sentry_sdk.start_transaction(op="cli.command"). Initial name is a fixed "cli" placeholder — BaseCommand.execute renames it once Click has dispatched. (Previous heuristic in _safe_command_name produced json for dg --output json whoami because flag values look like command names. Dropped.)

3. Per-command usage tags (packages/deepctl-core/.../base_command.py)

BaseCommand.execute now wraps handle() with two telemetry hooks:

  • _tag_telemetry_start(ctx) — runs before handle. Renames the active transaction to ctx.command_path (so dg debug audio becomes deepctl debug audio instead of just debug), and tags the scope with:
    • cmd.flags — sorted comma-separated list of flag names the user passed (e.g. diarize,model,output). Never values. Sentinel (none) when no flags were used.
    • cmd.output_format — one of json | yaml | table | csv | default.
  • _tag_telemetry_status(status) — runs after handle. Tags cmd.status with ok, cancelled, error, or whatever the BaseResult.status enum returned.

Both methods are wrapped in bare except Exception: pass so a Sentry hiccup, missing scope, or unknown Click parameter-source enum value can never crash the user's command.

What you'll see in Sentry per dg invocation

Per dg listen URL --diarize --model nova-3 -o json:

transaction:
  op:    cli.command
  name:  deepctl listen           ← from ctx.command_path
  tags:
    cli.os:          darwin
    cli.arch:        arm64
    cli.python:      3.13
    cli.version:     0.2.23
    cmd.flags:       diarize,model,output
    cmd.output_format: json
    cmd.status:      ok

Plus a session envelope (release health), profile envelope (when sampled long enough), and any log envelopes for logging.warning+ records.

Privacy

  • Flag names only, never values. --model nova-3 becomes cmd.flags: model, not cmd.flags: model=nova-3.
  • cmd.output_format is a bounded enum.
  • cmd.status is a bounded enum.
  • transaction.name is ctx.command_path — the dispatch tree, not user input.
  • send_default_pii=False and before_send=_scrub_event continue to scrub request bodies, headers, cookies, and user identifiers from error events.

Proof

$ SENTRY_DEBUG=1 dg --output json whoami 2>&1 | grep -iE '(transaction <|envelope)'
[sentry] DEBUG: [Tracing] Starting <cli.command> transaction <cli>
[sentry] DEBUG: Sending envelope [envelope with 1 items (transaction)] project:4510993603362816 host:o206115.ingest.us.sentry.io
[sentry] DEBUG: Sending envelope [envelope with 1 items (internal)]    project:4510993603362816 host:o206115.ingest.us.sentry.io
[sentry] DEBUG: Sending envelope [envelope with 1 items (session)]     project:4510993603362816 host:o206115.ingest.us.sentry.io

<cli> is the placeholder — the transaction is renamed to deepctl whoami by _tag_telemetry_start once Click dispatches. Sentry serialises the final state on send, so the actual envelope arrives with the resolved name.

Tests

File Class Cases
packages/deepctl-telemetry/tests/unit/test_telemetry.py TestSessionFlush::test_init_enables_full_observability_stack (renamed) asserts every new kwarg landed + session_mode not in kwargs regression guard
packages/deepctl-core/tests/unit/test_base.py TestTelemetryTagging (new) 10 cases — transaction rename, flag-name extraction (only COMMANDLINE-sourced params), (none) sentinel for zero flags, output format default, parametrised status pass-through, exception suppression on both methods
21 passed (telemetry: 11, base_command: 10)

lukeocodes added 2 commits May 9, 2026 08:23
Promotes the SDK from errors-only to the full Sentry feature set that
makes sense for a short-lived CLI process:

- traces_sample_rate=1.0, profiles_sample_rate=1.0 — every dg invocation
  produces a transaction; transaction profiling samples the body for
  long-enough commands (short ones discard with 'insufficient samples'
  which is normal).
- enable_logs=True — Python logging records ship as Sentry log items
  (the SDK 2.x Logs product). Combined with attach_stacktrace=True every
  warning+ log line carries enough context to debug.
- max_breadcrumbs bumped 20 -> 100 — the SDK default. Previous tighter
  budget made sense for an errors-only model; with traces on we want
  the headroom.
- send_default_pii stays False; before_send=_scrub_event still runs.

Wraps the cli() invocation in src/deepctl/main.py with a
sentry_sdk.start_transaction(op='cli.command', name=<safe-arg>) so
performance traces actually have a span to attach to. The transaction
name comes from a _safe_command_name helper that picks the first short,
slash-and-dot-and-@-and-=-free positional arg — catches command names
like 'listen' or 'speak' but rejects URLs, file paths, and
flag-as-value strings that might come right after.

Verified locally with SENTRY_DEBUG=1 dg --version: the SDK ships three
envelopes per invocation (transaction, internal client report, session)
to project 4510993603362816. Profile was discarded for --version
because the run is faster than the profiler sampling interval; heavier
commands will produce real profiles.

Tests: TestSessionFlush::test_init_enables_full_observability_stack
asserts every new kwarg landed (traces=1.0, profiles=1.0, enable_logs,
attach_stacktrace, max_breadcrumbs=100, send_default_pii=False) plus
the session_mode regression guard from the previous PR.
Adds two methods to BaseCommand that fire around handle():

- _tag_telemetry_start(ctx) sets the active transaction's name to the
  full Click command path (e.g. 'deepctl debug audio' instead of just
  'debug') and tags it with cmd.flags (sorted comma-separated list of
  flag NAMES the user explicitly passed at the command line, never
  values), cmd.output_format (one of json/yaml/table/csv/default), and
  is wrapped in bare-except so a Sentry hiccup or unknown Click
  parameter-source enum value can never crash the user's command.

- _tag_telemetry_status(status) tags cmd.status with one of 'ok',
  'cancelled', 'error', or whatever the BaseResult.status returned.
  Also bare-except.

execute() now calls _tag_telemetry_start before handle() and
_tag_telemetry_status in the success path AND in both KeyboardInterrupt
and Exception arms, so every command produces a tagged outcome.

Also drops the heuristic in main.py:_safe_command_name. It tried to
extract a command name from raw sys.argv but failed on flag VALUES that
look like command names — 'dg --output json whoami' produced transaction
name 'json' because 'json' is short, lowercase, and slash-free. Replaced
with a fixed 'cli' placeholder; BaseCommand.execute then renames to the
real ctx.command_path. One source of truth.

Net product effect: Sentry transactions now answer 'what features are
people actually using?' (cmd.flags), 'what output formats do they
prefer?' (cmd.output_format), 'what's the success rate per command?'
(cmd.status, transaction name) — instead of just 'who installed the
CLI?'.

Tests: 10 new cases in TestTelemetryTagging covering transaction rename,
flag tag including only COMMANDLINE-sourced params, '(none)' sentinel
for zero flags, output format default, exception suppression on both
methods, and parametrised status pass-through (ok/error/cancelled/
partial).
@lukeocodes lukeocodes changed the title feat(telemetry): turn on full Sentry observability for the CLI feat(telemetry): full Sentry observability + per-command usage tags May 9, 2026
Sweeps in 65 small lint fixes that were already present on main but that
the previous lint-check tolerated:

- typing.Dict / List / Tuple -> builtin generics
- import sorting + dedup
- unused-import removal in tests
- redundant string annotations under future annotations

Plus the one new error this branch actually introduced (TC003: move
collections.abc.Iterator into a TYPE_CHECKING block in src/deepctl/main.py).

The wide cleanup landed in this commit accidentally (git add -A after a
prior ruff check --fix pass). Keeping it because:
- everything is auto-fix-safe (no behavioural deltas)
- main already passes CI without these fixes (the lint-check Make
  target only scans src/ and packages/**/src, not tests), so we're
  removing latent debt rather than introducing scope creep
- reverting would just put the same warnings back in the same files

Pre-existing test_output_result_* failures in test_base.py:: TestBaseCommand
are unchanged and unrelated; same as on main.
@lukeocodes lukeocodes force-pushed the feat/telemetry-full-sentry branch from b72b32d to 9354e24 Compare May 9, 2026 07:42
Base automatically changed from fix/telemetry-session-mode to main May 9, 2026 08:59
@lukeocodes lukeocodes merged commit 0fe43d2 into main May 9, 2026
34 checks passed
@lukeocodes lukeocodes deleted the feat/telemetry-full-sentry branch May 9, 2026 09:00
@github-actions github-actions Bot mentioned this pull request May 9, 2026
lukeocodes pushed a commit that referenced this pull request May 9, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.2.24</summary>

## [0.2.24](v0.2.23...v0.2.24)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
* **telemetry:** track per-command usage via Sentry tags
([490b37e](490b37e))
* **telemetry:** turn on full Sentry observability for the CLI
([cc2c208](cc2c208))
* **web:** swap Sentry for Heap + GA4 on cli.deepgram.com
([e736528](e736528))
* **web:** swap Sentry for Heap + GA4 on cli.deepgram.com
([#76](#76))
([b12a55b](b12a55b))
* **web:** wire real GA4 measurement ID G-TYPC1TBCKT
([a6468e4](a6468e4))


### Bug Fixes

* **telemetry:** explicitly start session after init
([8dd2843](8dd2843))
* **telemetry:** explicitly start session after init
([#74](#74))
([ad75efa](ad75efa))
* uniform 'any arg = non-interactive' rule across all commands
([#78](#78))
([6370f32](6370f32))
</details>

<details><summary>deepctl-core: 0.2.12</summary>

##
[0.2.12](deepctl-core-v0.2.11...deepctl-core-v0.2.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
* **telemetry:** track per-command usage via Sentry tags
([490b37e](490b37e))


### Bug Fixes

* uniform 'any arg = non-interactive' rule across all commands
([#78](#78))
([6370f32](6370f32))
</details>

<details><summary>deepctl-shared-utils: 0.1.12</summary>

##
[0.1.12](deepctl-shared-utils-v0.1.11...deepctl-shared-utils-v0.1.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-telemetry: 0.0.4</summary>

##
[0.0.4](deepctl-telemetry-v0.0.3...deepctl-telemetry-v0.0.4)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
* **telemetry:** turn on full Sentry observability for the CLI
([cc2c208](cc2c208))


### Bug Fixes

* **telemetry:** explicitly start session after init
([8dd2843](8dd2843))
* **telemetry:** explicitly start session after init
([#74](#74))
([ad75efa](ad75efa))
</details>

<details><summary>deepctl-cmd-login: 0.1.15</summary>

##
[0.1.15](deepctl-cmd-login-v0.1.14...deepctl-cmd-login-v0.1.15)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))


### Bug Fixes

* uniform 'any arg = non-interactive' rule across all commands
([#78](#78))
([6370f32](6370f32))
</details>

<details><summary>deepctl-cmd-debug-audio: 0.1.13</summary>

##
[0.1.13](deepctl-cmd-debug-audio-v0.1.12...deepctl-cmd-debug-audio-v0.1.13)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-cmd-debug-browser: 0.1.12</summary>

##
[0.1.12](deepctl-cmd-debug-browser-v0.1.11...deepctl-cmd-debug-browser-v0.1.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))


### Bug Fixes

* uniform 'any arg = non-interactive' rule across all commands
([#78](#78))
([6370f32](6370f32))
</details>

<details><summary>deepctl-cmd-debug-network: 0.1.12</summary>

##
[0.1.12](deepctl-cmd-debug-network-v0.1.11...deepctl-cmd-debug-network-v0.1.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-cmd-update: 0.2.5</summary>

##
[0.2.5](deepctl-cmd-update-v0.2.4...deepctl-cmd-update-v0.2.5)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-cmd-plugin: 0.1.12</summary>

##
[0.1.12](deepctl-cmd-plugin-v0.1.11...deepctl-cmd-plugin-v0.1.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-cmd-skills: 0.0.6</summary>

##
[0.0.6](deepctl-cmd-skills-v0.0.5...deepctl-cmd-skills-v0.0.6)
(2026-05-09)


### Bug Fixes

* uniform 'any arg = non-interactive' rule across all commands
([#78](#78))
([6370f32](6370f32))
</details>

<details><summary>deepctl-cmd-listen: 0.0.13</summary>

##
[0.0.13](deepctl-cmd-listen-v0.0.12...deepctl-cmd-listen-v0.0.13)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-cmd-completion: 0.0.3</summary>

##
[0.0.3](deepctl-cmd-completion-v0.0.2...deepctl-cmd-completion-v0.0.3)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

<details><summary>deepctl-plugin-example: 0.1.12</summary>

##
[0.1.12](deepctl-plugin-example-v0.1.11...deepctl-plugin-example-v0.1.12)
(2026-05-09)


### Features

* **telemetry:** full Sentry observability + per-command usage tags
([#75](#75))
([0fe43d2](0fe43d2))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant