fallback

You can use the fallback provider to add more resilience to your application.

A fallback will attempt to use the first client, and if it fails, it will try the second client, and so on.

You can nest fallbacks inside of other fallbacks.

BAML

1 client<llm> SuperDuperClient {
2   provider fallback
3   options {
4     strategy [
5       ClientA
6       ClientB
7       ClientC
8     ]
9   }
10 }

Options

strategy

List[string]Required

The list of client names to try in order. Cannot be empty.

retry_policy

Like any other client, you can specify a retry policy for the fallback client. See retry_policy for more information.

The retry policy will test the fallback itself, after the entire strategy has failed.

BAML

1 client<llm> SuperDuperClient {
2   provider fallback
3   retry_policy MyRetryPolicy
4   options {
5     strategy [
6       ClientA
7       ClientB
8       ClientC
9     ]
10   }
11 }

Nesting multiple fallbacks

You can nest multiple fallbacks inside of each other. The fallbacks will just chain as you would expect.

BAML

1 client<llm> SuperDuperClient {
2   provider fallback
3   options {
4     strategy [
5       ClientA
6       ClientB
7       ClientC
8     ]
9   }
10 }
11 
12 client<llm> MegaClient {
13   provider fallback
14   options {
15     strategy [
16       SuperDuperClient
17       ClientD
18     ]
19   }
20 }

1	client<llm> SuperDuperClient {
2	provider fallback
3	options {
4	strategy [
5	ClientA
6	ClientB
7	ClientC
8	]
9	}
10	}

1	client<llm> SuperDuperClient {
2	provider fallback
3	retry_policy MyRetryPolicy
4	options {
5	strategy [
6	ClientA
7	ClientB
8	ClientC
9	]
10	}
11	}