Skip to content

fix(deep): error prefix, semantic pattern_rule lineage, category serialization#23

Merged
boorad merged 3 commits into
mainfrom
fix/deep-mode-shakedown
Apr 29, 2026
Merged

fix(deep): error prefix, semantic pattern_rule lineage, category serialization#23
boorad merged 3 commits into
mainfrom
fix/deep-mode-shakedown

Conversation

@boorad
Copy link
Copy Markdown
Contributor

@boorad boorad commented Apr 29, 2026

Summary

Three small bugs caught during an end-to-end shakedown of --deep across all three transports (HTTP via Ollama, subprocess via claude -p + codex exec, and MCP via Claude Code).

Fixes

1. DeepError::Config display prefixsrc/deep/error.rs
Said missing config: even when the variant fired for validation errors with values present (ftp:// scheme, --agent-cmd + --base-url mutex, NaN cost rates). Renamed to deep config error:.

2. Semantic findings inheriting structural pattern_rule verbatimsrc/deep/finding.rs
A model-recategorized finding (ownership seed → feature_gate verdict) was still displaying Rule: ts-ownership-check. The model can re-categorize, drop, or re-scope a seed; the bare structural rule lies about provenance. Now:

  • if the model's range overlaps the seed's range → {rule}-semantic (genuine re-evaluation, lineage preserved)
  • if disjoint (incidental finding from the surrounding context window) → semantic-{category}
  • cold-region candidates (no seed) → semantic-{category}

The range-aware branch was caught during the subprocess walkthrough: one ownership escalation's expanded window covered an unrelated checkPermission function and that feature_gate finding was getting stamped ts-ownership-check-semantic.

3. summary.by_category keys disagreeing with findings[].categorysrc/output/json.rs, src/types.rs, src/mcp/resources.rs
Summary used Display ("Business Rule" → lowercased to "business rule" with a space), findings used serde rename_all="snake_case" ("business_rule"). Two spellings of the same enum in one JSON document. Added AuthCategory::slug() as the single source of truth for the snake_case wire form, swapped output/json.rs and mcp/resources.rs over to it, deleted two duplicate private category_slug helpers.

Test plan

  • All 310 unit + integration tests pass (was 308; +2 for ranges_overlap helper and disjoint-range lineage)
  • cargo clippy --all-targets -- -D warnings clean
  • cargo fmt --check clean
  • Manual: HTTP transport via Ollama qwen2.5-coder:14b — escalation + cold-region paths both verified
  • Manual: MCP transport via Claude Code (Opus 4.7) — host-driven full tool sequence verified
  • Manual: subprocess transport via claude -p (Opus 4.7) and codex exec — both produced 3 findings on the same fixture
  • Manual: confirmed range-aware lineage post-refinement — disjoint-range feature_gate finding now correctly tags semantic-feature_gate instead of ts-ownership-check-semantic

Follow-ups (not in this PR)

  • MCP server has no cold-region discovery tool; Tier 1 hosts can only reach files surfaced via existing structural findings. Worth a discover_cold_regions tool to bring Tier 1 into parity with Tier 2/3.
  • Typo'd --agent-cmd (exit 127) currently per-candidate-skips with BadResponse warnings; arguably should fail fast at first invocation. Design tradeoff between best-effort enrichment and clear misconfiguration feedback.
  • TS structural matcher gaps unrelated to deep mode: req.user.role !== 'admin' (negated form) and req.user.hasRole(...) (member-expression call) didn't match the existing rules in a hand-written fixture.

Summary by CodeRabbit

  • Bug Fixes

    • Configuration error text made clearer to indicate validation failures and malformed inputs.
    • JSON report category summary keys now use canonical snake_case for consistent grouping.
  • New Features / Behavior

    • Semantic findings now assign synthetic pattern/rule identifiers to preserve lineage and avoid empty IDs; overlap logic refined.
    • Category identifiers now expose stable snake_case slugs for canonical wire keys.
  • Documentation

    • Added a deep-mode walkthrough and README callout explaining transports and behavior.
  • Tests

    • Added tests covering range-overlap and boundary behavior.

boorad added 2 commits April 29, 2026 16:18
…alization

Three small bugs surfaced during a manual end-to-end shakedown of --deep
across all three transports (HTTP/Ollama, subprocess/claude -p + codex,
MCP/Claude Code).

1. DeepError::Config display prefix said "missing config:" even when the
   variant fired for validation errors with values present (ftp:// scheme,
   --agent-cmd + --base-url mutex, NaN cost rates). Renamed to
   "deep config error:" so the message is accurate in both cases.

2. Semantic findings were inheriting the structural seed's pattern_rule
   verbatim, so a model-recategorized finding (e.g. ownership escalation
   coming back as feature_gate) would still display
   "Rule: ts-ownership-check". The model can re-categorize, drop, or
   re-scope a seed; surfacing the bare structural rule lies about
   provenance. Now:
     - if the model's range overlaps the seed's range → genuine
       re-evaluation, tag {rule}-semantic (preserves lineage);
     - if disjoint (e.g. an incidental finding from the surrounding
       context window) → fall through to semantic-{category};
     - cold-region candidates (no seed) → always semantic-{category}.
   The range-aware branch was caught during the subprocess walkthrough:
   one ownership escalation's expanded window covered an unrelated
   checkPermission function and that feature_gate finding was getting
   stamped ts-ownership-check-semantic.

3. summary.by_category map keys disagreed with findings[].category in the
   same JSON document. The summary used Display ("Business Rule" → space
   lowercased), the findings used serde rename_all=snake_case
   ("business_rule" → underscore). Consumers grouping summary by category
   couldn't round-trip. Added AuthCategory::slug() as the single source of
   truth for the snake_case wire form, swapped output/json.rs and
   mcp/resources.rs over to it, and deleted two private category_slug
   helpers in deep/finding.rs and mcp/resources.rs.

Tests: +5 (range overlap helper, range-aware lineage, disjoint-range
fallback, plus existing-test updates). 313 total passing, clippy + fmt
clean.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 29, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3cf4c849-8ee6-4e2a-a836-9eb46fb15515

📥 Commits

Reviewing files that changed from the base of the PR and between fec06e2 and f04a998.

📒 Files selected for processing (2)
  • README.md
  • docs/DEEP_MODE_WALKTHROUGH.md
✅ Files skipped from review due to trivial changes (1)
  • README.md

📝 Walkthrough

Walkthrough

Introduces AuthCategory::slug() and replaces local slug helpers across modules; changes semantic finding translation to synthesize pattern_rule based on inclusive line-range overlap; generalizes DeepError::Config message and docs to a broader "deep config error" classification.

Changes

Cohort / File(s) Summary
Category slug consolidation
src/types.rs, src/mcp/resources.rs, src/output/json.rs
Add AuthCategory::slug(); remove local category_slug helper; use cat.slug() for resource payloads and use f.category.slug() as JSON summary keys (canonical snake_case).
Semantic finding rule ID logic
src/deep/finding.rs
Change semantic-finding pattern_rule synthesis: append -semantic to seed.pattern_rule when seed and semantic ranges overlap, otherwise use semantic-{model_category_slug}; add inclusive ranges_overlap helper and tests.
Error message generalization
src/deep/error.rs
Update DeepError::Config display text from "missing config: {0}" to "deep config error: {0}" and add public doc comment clarifying absent-field and validation-failure cases.
Documentation / Walkthrough
README.md, docs/DEEP_MODE_WALKTHROUGH.md
Add first-time --deep callout and a detailed DEEP_MODE_WALKTHROUGH covering transports, examples, MCP workflow, and behavioral notes (doc-only changes).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Possibly related PRs

Poem

🐰 I stitched a slug in snake_case style,
Rules that bloom when ranges reconcile.
Configs now speak in kinder tone,
Findings find their rightful home—
Hops of joy across each file.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the three main fixes in the changeset: error prefix correction, semantic pattern_rule lineage behavior, and category serialization standardization. It is concise, specific, and clearly conveys the primary changes.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Review rate limit: 4/5 reviews remaining, refill in 12 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@amazon-q-developer amazon-q-developer Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR successfully addresses three distinct bugs discovered during end-to-end testing of the --deep mode. The changes are well-structured, properly tested, and maintain backward compatibility.

Key fixes validated:

  1. Error message prefix now accurately reflects both missing and malformed config scenarios
  2. Range-aware semantic finding lineage prevents false attribution when model identifies incidental findings outside seed range
  3. Category serialization consistency achieved through centralized AuthCategory::slug() method

All 310 tests pass, manual testing across three transport mechanisms completed successfully, and the implementation demonstrates careful attention to edge cases. The changes are ready to merge.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/mcp/resources.rs (1)

45-46: Centralize slug parsing alongside emission.

AuthCategory::slug() now owns the write-side contract, but category_from_slug() below still hand-maintains the reverse mapping. That leaves list_resources() and read_resource() able to drift on the next category add/rename. Consider moving the parse side into impl FromStr for AuthCategory (or equivalent) and reusing it here.

Also applies to: 83-83

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/mcp/resources.rs` around lines 45 - 46, The code duplicates slug logic:
AuthCategory::slug() defines the write-side format while category_from_slug()
manually reverses it, risking drift; implement parsing centrally by adding impl
FromStr for AuthCategory (or a parse_slug method) and replace manual parsing
calls in category_from_slug(), list_resources(), and read_resource() to use
AuthCategory::from_str (or the shared parse_slug) so both emission
(AuthCategory::slug()) and parsing share the same contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/mcp/resources.rs`:
- Around line 45-46: The code duplicates slug logic: AuthCategory::slug()
defines the write-side format while category_from_slug() manually reverses it,
risking drift; implement parsing centrally by adding impl FromStr for
AuthCategory (or a parse_slug method) and replace manual parsing calls in
category_from_slug(), list_resources(), and read_resource() to use
AuthCategory::from_str (or the shared parse_slug) so both emission
(AuthCategory::slug()) and parsing share the same contract.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 477e4e7e-5731-4cab-b644-74e333f1eefd

📥 Commits

Reviewing files that changed from the base of the PR and between e30b892 and fec06e2.

📒 Files selected for processing (6)
  • plans/done/00-deep-mode-overview.md
  • src/deep/error.rs
  • src/deep/finding.rs
  • src/mcp/resources.rs
  • src/output/json.rs
  • src/types.rs

coderabbitai[bot]
coderabbitai Bot previously approved these changes Apr 29, 2026
Captures the manual shakedown session that produced the fixes in this
branch. Six runs against the same fixture (or a paired cold-region-only
fixture) across HTTP/Ollama, MCP/Claude Code, and subprocess/claude-p +
subprocess/codex-exec. Real commands, real outputs, real timing/cost
numbers.

Sections:
- TL;DR matrix mapping each mode to "when to use it"
- Fixture rationale (one escalation candidate + one cold-region candidate
  in a single TS file)
- Static baseline vs each transport, with the actual JSON they returned
- Comparison table across runs
- The pattern_rule lineage rules (escalation+overlap, escalation+disjoint,
  cold region) as a table — reproduces the truth table that drove this
  PR's Fix #2 refinement
- Surprises encountered during the shakedown that aren't bugs but are
  worth knowing (Ollama's lax json_schema support, escalation suppression
  of overlapping cold regions, claude -p cold-start timeouts)

README points to the new doc from the deep-mode section.
@boorad boorad merged commit 3d116f6 into main Apr 29, 2026
2 checks passed
@boorad boorad deleted the fix/deep-mode-shakedown branch April 29, 2026 20:40
@boorad boorad self-assigned this May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant