Switching LLMs

BAML Supports getting structured output from all major providers as well as all OpenAI-API compatible open-source models. See LLM Providers Reference for how to set each one up.

BAML can help you get structured output from any Open-Source model, with better performance than other techniques, even when it’s not officially supported via a Tool-Use API (like o1-preview) or fine-tuned for it! Read more about how BAML does this.

Using client "<provider>/<model>"

Using openai/model-name or anthropic/model-name will assume you have the ANTHROPIC_API_KEY or OPENAI_API_KEY environment variables set.

BAML
1function MakeHaiku(topic: string) -> string {
2  client "openai/gpt-4o" // or anthropic/claude-3-5-sonnet-20240620
3  prompt #"
4    Write a haiku about {{ topic }}.
5  "#
6}

Using a named client

Use this if you are using open-source models or need customization
The longer form uses a named client, and supports adding any parameters supported by the provider or changing the temperature, top_p, etc.

BAML
1client<llm> MyClient {
2  provider "openai"
3  options {
4    model "gpt-4o"
5    api_key env.OPENAI_API_KEY
6    // other params like temperature, top_p, etc.
7    temperature 0.0
8    base_url "https://my-custom-endpoint.com/v1"
9    // add headers
10    headers {
11      "anthropic-beta" "prompt-caching-2024-07-31"
12    }
13  }
14
15}
16
17function MakeHaiku(topic: string) -> string {
18  client MyClient
19  prompt #"
20    Write a haiku about {{ topic }}.
21  "#
22}

Consult the provider documentation for a list of supported providers and models, the default options, and setting retry policies.

If you want to specify which client to use at runtime, in your Python/TS/Ruby code, you can use the client registry to do so.

This can come in handy if you’re trying to, say, send 10% of your requests to a different model.