Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .changeset/turn-level-retry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
'@openrouter/agent': minor
---

Add turn-level retry and hang detection to the callModel tool loop, plus a tool-level retry helper.

**`retryTurn` option on `callModel`** — when a turn (one provider request + stream consumption) fails, the turn is re-sent with the full accumulated conversation intact instead of aborting the whole loop. Tool results gathered in prior turns are never discarded and tools are not re-executed. Covers all turn sites: the initial request (send and consume phases), follow-up requests after tool execution, the forced final response, and state resume.

- `limit` — max retries per turn (default 2)
- `idleTimeoutMs` — converts silently-hung streams (no events, no terminal frame, connection left open) into retryable `TurnIdleTimeoutError` failures; the hung connection is cancelled
- `isRetryable` — custom retryability policy; the default (`defaultIsTurnRetryable`) retries idle timeouts, streams that ended without a terminal event, network errors, and HTTP 408/429/5xx, and does not retry `response.failed` terminal events (e.g. refusals) or other 4xx
- `backoffMs` — fixed or per-attempt delay between retries

Mid-turn retries emit a new `turn.retry` event on `getFullResponsesStream()` (`isTurnRetryEvent` guard exported); events already received for that turn should be treated as void since the retried attempt re-streams the turn from the start. Failure classification is now typed: `TurnIdleTimeoutError`, `TurnStreamEndedError`, `TurnResponseFailedError` (messages unchanged).

**`withToolRetry(tool, options)`** — wrap a tool so its `execute` function is automatically re-run when it throws (transient network failures inside tools no longer burn a model round trip). Supports regular and generator tools, preserves tool typing and type-guard classification, with `limit`, `backoffMs`, `isRetryable`, and `onRetry` observability hook. Only wrap idempotent tools.
13 changes: 13 additions & 0 deletions packages/agent/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,9 @@ export type { ContextInput } from './lib/tool-context.js';
export { buildToolExecuteContext, ToolContextStore } from './lib/tool-context.js';
// Real-time tool event broadcasting
export { ToolEventBroadcaster } from './lib/tool-event-broadcaster.js';
// Tool-level retry
export type { ToolRetryContext, ToolRetryOptions } from './lib/tool-retry.js';
export { withToolRetry } from './lib/tool-retry.js';
export type {
ChatStreamEvent,
ClientTool,
Expand Down Expand Up @@ -178,6 +181,7 @@ export type {
ToolWithGenerator,
TurnContext,
TurnEndEvent,
TurnRetryEvent,
TurnStartEvent,
TypedToolCall,
TypedToolCallUnion,
Expand All @@ -198,11 +202,20 @@ export {
isToolPreliminaryResultEvent,
isToolResultEvent,
isTurnEndEvent,
isTurnRetryEvent,
isTurnStartEvent,
ToolType,
toolHasApprovalConfigured,
} from './lib/tool-types.js';
// Turn context helpers
export { buildTurnContext, normalizeInputToArray } from './lib/turn-context.js';
// Turn-level retry
export type { RetryTurnOptions, TurnRetryContext } from './lib/turn-retry.js';
export {
defaultIsTurnRetryable,
TurnIdleTimeoutError,
TurnResponseFailedError,
TurnStreamEndedError,
} from './lib/turn-retry.js';
export type { Hook, OpenRouterOptions, SDKOptions } from './openrouter.js';
export { OpenRouter } from './openrouter.js';
4 changes: 4 additions & 0 deletions packages/agent/src/inner-loop/call-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ export function callModel<
onTurnStart,
onTurnEnd,
allowFinalResponse,
retryTurn,
...apiRequest
} = request;

Expand Down Expand Up @@ -165,5 +166,8 @@ export function callModel<
...(allowFinalResponse !== undefined && {
allowFinalResponse,
}),
...(retryTurn !== undefined && {
retryTurn,
}),
} as GetResponseOptions<TTools, TShared>);
}
18 changes: 18 additions & 0 deletions packages/agent/src/lib/async-params.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import type {
ToolContextMapWithShared,
TurnContext,
} from './tool-types.js';
import type { RetryTurnOptions } from './turn-retry.js';

// Re-export Tool type for convenience
export type { Tool } from './tool-types.js';
Expand Down Expand Up @@ -98,6 +99,22 @@ type BaseCallModelInput<
* (HITL pause, approval pause, interruption, or natural completion).
*/
allowFinalResponse?: boolean | string;
/**
* Retry a failed turn (one provider request + stream consumption) instead
* of aborting the whole tool loop. The accumulated conversation —
* including all tool results gathered in prior turns — is preserved
* across retries, so a single dead stream no longer discards the loop's
* gathered context.
*
* `idleTimeoutMs` additionally converts silently-hung streams (no events,
* no terminal frame, connection left open) into retryable failures.
*
* Mid-turn retries emit a `turn.retry` event on
* `getFullResponsesStream()`; events already received for that turn
* should be treated as void since the retried attempt re-streams the
* turn from the start.
*/
retryTurn?: RetryTurnOptions;
};

/**
Expand Down Expand Up @@ -199,6 +216,7 @@ export async function resolveAsyncFunctions<TTools extends readonly Tool[] = rea
'onTurnStart', // Client-side turn start callback
'onTurnEnd', // Client-side turn end callback
'allowFinalResponse', // Client-side: triggers no-tools final turn when stopWhen breaks the loop
'retryTurn', // Client-side turn retry configuration
]);

// Iterate over all keys in the input
Expand Down
Loading
Loading