Skip to content

Decouple CancellationToken from HTTP request for streaming support#440

Open
rido-min wants to merge 3 commits intonext/corefrom
fix/decouple-cancellation-token
Open

Decouple CancellationToken from HTTP request for streaming support#440
rido-min wants to merge 3 commits intonext/corefrom
fix/decouple-cancellation-token

Conversation

@rido-min
Copy link
Copy Markdown
Member

Summary

  • Decouples the CancellationToken passed to bot handlers from the incoming HTTP request's HttpContext.RequestAborted token, which fires when Teams closes the connection (~15s) — before streaming handlers finish their work.
  • Adds BotApplicationOptions.ProcessActivityTimeout (default: 5 minutes) as a configurable timeout that replaces the HTTP-bound token.
  • Handles OperationCanceledException gracefully when the timeout fires, logging a warning instead of throwing BotHandlerException.

Problem

Streaming bots send responses via the Bot Connector API (ConversationClient.SendActivityAsync), not through the original HTTP response. The handler legitimately outlives the HTTP request. Previously, the HTTP disconnection would cancel in-flight LLM streaming calls, causing TaskCanceledExceptionBotHandlerException → 500 errors.

Design document

See docs/design-decouple-cancellation-token.md for the full design proposal, alternatives considered, and impact analysis.

Test plan

  • Verify StreamingBot sample streams full LLM responses without TaskCanceledException
  • Verify non-streaming bots (e.g. CoreBot, EchoBot) are unaffected by the 5-minute default
  • Verify ProcessActivityTimeout can be configured to a custom value
  • Verify timeout fires and logs warning (not exception) when a handler exceeds the configured duration
  • Verify Debugger.IsAttached still bypasses the timeout with CancellationToken.None

🤖 Generated with Claude Code

…time

The CancellationToken passed to bot handlers was tied to HttpContext.RequestAborted,
which fires when Teams closes the HTTP connection (~15s). For streaming bots that
send responses via the Bot Connector API, this caused TaskCanceledException during
legitimate long-running operations like LLM streaming.

Replace the HTTP-bound token with a configurable timeout (default 5 min) via
BotApplicationOptions.ProcessActivityTimeout. Add graceful OperationCanceledException
handling so timeouts log a warning instead of throwing BotHandlerException.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rido-min rido-min requested a review from MehakBindra April 22, 2026 16:08
@rido-min rido-min added the CORE label Apr 22, 2026
rido-min and others added 2 commits April 22, 2026 11:12
…ion token

The test was asserting the middleware receives the caller's exact token,
but ProcessAsync now creates its own timeout-based token. Updated to
verify the middleware receives a valid, non-cancelled token instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant