Switching LLMs

BAML Supports getting structured output from all major providers as well as all OpenAI-API compatible open-source models. See LLM Providers Reference for how to set each one up.

BAML can help you get structured output from any Open-Source model, with better performance than other techniques, even when it’s not officially supported via a Tool-Use API (like o1-preview) or fine-tuned for it! Read more about how BAML does this.

Using `client "<provider>/<model>"`

Using openai/model-name or anthropic/model-name will assume you have the ANTHROPIC_API_KEY or OPENAI_API_KEY environment variables set.

BAML

1 function MakeHaiku(topic: string) -> string {
2   client "openai-responses/gpt-5-mini" // or anthropic/claude-sonnet-4-20250514
3   prompt #"
4     Write a haiku about {{ topic }}.
5   "#
6 }

Using a named client

Use this if you are using open-source models or need customization

The longer form uses a named client, and supports adding any parameters supported by the provider or changing the temperature, top_p, etc.

BAML

1 client<llm> MyClient {
2   provider "openai"
3   options {
4     model "gpt-5-mini"
5     api_key env.OPENAI_API_KEY
6     // other params like temperature, top_p, etc.
7     temperature 0.0
8     base_url "https://my-custom-endpoint.com/v1"
9     // add headers
10     headers {
11       "anthropic-beta" "prompt-caching-2024-07-31"
12     }
13   }
14 
15 }
16 
17 function MakeHaiku(topic: string) -> string {
18   client MyClient
19   prompt #"
20     Write a haiku about {{ topic }}.
21   "#
22 }

Consult the provider documentation for a list of supported providers and models, the default options, and setting retry policies.

If you want to specify which client to use at runtime, in your Python/TS/Ruby code, you can use the client registry to do so.

This can come in handy if you’re trying to, say, send 10% of your requests to a different model.

1	function MakeHaiku(topic: string) -> string {
2	client "openai-responses/gpt-5-mini" // or anthropic/claude-sonnet-4-20250514
3	prompt #"
4	Write a haiku about {{ topic }}.
5	"#
6	}

1	client<llm> MyClient {
2	provider "openai"
3	options {
4	model "gpt-5-mini"
5	api_key env.OPENAI_API_KEY
6	// other params like temperature, top_p, etc.
7	temperature 0.0
8	base_url "https://my-custom-endpoint.com/v1"
9	// add headers
10	headers {
11	"anthropic-beta" "prompt-caching-2024-07-31"
12	}
13	}
14
15	}
16
17	function MakeHaiku(topic: string) -> string {
18	client MyClient
19	prompt #"
20	Write a haiku about {{ topic }}.
21	"#
22	}