Skip to content

fix(ai-eval): remove untestable pipeline assertions, fix Slack channel ID#197

Merged
janhesters merged 1 commit intomainfrom
fix/ai-eval-pipeline-assertions-and-slack
Apr 16, 2026
Merged

fix(ai-eval): remove untestable pipeline assertions, fix Slack channel ID#197
janhesters merged 1 commit intomainfrom
fix/ai-eval-pipeline-assertions-and-slack

Conversation

@janhesters
Copy link
Copy Markdown
Collaborator

Summary

  • Remove 3 pipeline skill test assertions that can't be verified from agent output:
    • Subagent type delegation (internal dispatch detail, not visible in output)
    • Task prompt construction (intermediate artifact, not shown in output)
    • Narrative text filtering (fixture has no narrative text to test against)
  • Fix Slack notification channel ID (C0A5ZRP7XR5C0ASZRP7XRS)

Subagent delegation testing requires new RITEway AI tooling (paralleldrive/riteway#437). The remaining 7 assertions cover all observable pipeline behavior.

Test plan

  • Trigger AI eval workflow manually and verify pipeline tests pass (7/7)
  • Trigger a deliberate failure and verify Slack notification arrives in #prism-reports

…l ID

Remove 3 pipeline skill assertions that can't be verified from output:
- subagent type delegation (internal dispatch detail)
- self-contained Task prompt construction (intermediate artifact)
- narrative text filtering (fixture doesn't contain narrative text)

The remaining 7 assertions cover all observable behavior. Subagent
delegation testing requires new RITEway AI tooling (riteway#437).

Also fix Slack notification channel ID (C0A5ZRP7XR5 → C0ASZRP7XRS).
@janhesters janhesters merged commit 877831c into main Apr 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant