A retry policy can be attached to any client<llm> and will attempt to retry requests that fail due to a network error.

BAML
1retry_policy MyPolicyName {
2 max_retries 3
3}

Usage:

BAML
1client<llm> MyClient {
2 provider anthropic
3 retry_policy MyPolicyName
4 options {
5 model "claude-3-sonnet-20240229"
6 api_key env.ANTHROPIC_API_KEY
7 }
8}

Fields

max_retries
intRequired

Number of additional retries to attempt after the initial request fails.

strategy
Strategy

The strategy to use for retrying requests. Default is constant_delay(delay_ms=200).

StrategyDocsNotes
constant_delayDocs
exponential_backoffDocs

Example:

BAML
1retry_policy MyPolicyName {
2 max_retries 3
3 strategy {
4 type constant_delay
5 delay_ms 200
6 }
7}

Strategies

constant_delay

type
constant_delayRequired

Configures to the constant delay strategy.

delay_ms
int

The delay in milliseconds to wait between retries. Default: 200

exponential_backoff

type
exponential_backoffRequired

Configures to the exponential backoff strategy.

delay_ms
int

The initial delay in milliseconds to wait between retries. Default: 200

multiplier
float

The multiplier to apply to the delay after each retry. Default: 1.5

max_delay_ms
int

The maximum delay in milliseconds to wait between retries. Default: 10000