Decouple CancellationToken from HTTP request for streaming support#440
Open
Decouple CancellationToken from HTTP request for streaming support#440
Conversation
…time The CancellationToken passed to bot handlers was tied to HttpContext.RequestAborted, which fires when Teams closes the HTTP connection (~15s). For streaming bots that send responses via the Bot Connector API, this caused TaskCanceledException during legitimate long-running operations like LLM streaming. Replace the HTTP-bound token with a configurable timeout (default 5 min) via BotApplicationOptions.ProcessActivityTimeout. Add graceful OperationCanceledException handling so timeouts log a warning instead of throwing BotHandlerException. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ion token The test was asserting the middleware receives the caller's exact token, but ProcessAsync now creates its own timeout-based token. Updated to verify the middleware receives a valid, non-cancelled token instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CancellationTokenpassed to bot handlers from the incoming HTTP request'sHttpContext.RequestAbortedtoken, which fires when Teams closes the connection (~15s) — before streaming handlers finish their work.BotApplicationOptions.ProcessActivityTimeout(default: 5 minutes) as a configurable timeout that replaces the HTTP-bound token.OperationCanceledExceptiongracefully when the timeout fires, logging a warning instead of throwingBotHandlerException.Problem
Streaming bots send responses via the Bot Connector API (
ConversationClient.SendActivityAsync), not through the original HTTP response. The handler legitimately outlives the HTTP request. Previously, the HTTP disconnection would cancel in-flight LLM streaming calls, causingTaskCanceledException→BotHandlerException→ 500 errors.Design document
See
docs/design-decouple-cancellation-token.mdfor the full design proposal, alternatives considered, and impact analysis.Test plan
TaskCanceledExceptionProcessActivityTimeoutcan be configured to a custom valueDebugger.IsAttachedstill bypasses the timeout withCancellationToken.None🤖 Generated with Claude Code