BAML Snippets LLM Clients provider strategies

retry_policy

A retry policy can be attached to any client<llm> and will attempt to retry requests that fail due to a network error.

BAML

1 retry_policy MyPolicyName {
2   max_retries 3
3 }

Usage:

BAML

1 client<llm> MyClient {
2   provider anthropic
3   retry_policy MyPolicyName
4   options {
5     model "claude-3-sonnet-20240229"
6     api_key env.ANTHROPIC_API_KEY
7   }
8 }

Fields

max_retries

intRequired

Number of additional retries to attempt after the initial request fails.

strategy

Strategy

The strategy to use for retrying requests. Default is constant_delay(delay_ms=200).

Strategy	Docs	Notes
`constant_delay`	Docs
`exponential_backoff`	Docs

Example:

BAML

1 retry_policy MyPolicyName {
2   max_retries 3
3   strategy {
4     type constant_delay
5     delay_ms 200
6   }
7 }

Strategies

constant_delay

type

constant_delayRequired

Configures to the constant delay strategy.

delay_ms

int

The delay in milliseconds to wait between retries. Default: 200

exponential_backoff

type

exponential_backoffRequired

Configures to the exponential backoff strategy.

delay_ms

int

The initial delay in milliseconds to wait between retries. Default: 200

multiplier

float

The multiplier to apply to the delay after each retry. Default: 1.5

max_delay_ms

int

The maximum delay in milliseconds to wait between retries. Default: 10000

1	client<llm> MyClient {
2	provider anthropic
3	retry_policy MyPolicyName
4	options {
5	model "claude-3-sonnet-20240229"
6	api_key env.ANTHROPIC_API_KEY
7	}
8	}

1	retry_policy MyPolicyName {
2	max_retries 3
3	strategy {
4	type constant_delay
5	delay_ms 200
6	}
7	}