anthropic

Will be passed as a bearer token. Default: env.ANTHROPIC_API_KEY

Authorization: Bearer $api_key

The base URL for the API. Default: https://api.anthropic.com

Additional headers to send with the request.

Unless specified with a different value, we inject in the following headers:

"anthropic-version" "2023-06-01"

Example:

1
client<llm> MyClient {
2
  provider anthropic
3
  options {
4
    api_key env.MY_ANTHROPIC_KEY
5
    model "claude-3-5-sonnet-20240620"
6
    headers {
7
      "X-My-Header" "my-value"
8
    }
9
  }
10
}

The role to use if the role is not in the allowed_roles. Default: "user" usually, but some models like OpenAI’s gpt-4o will use "system"

Picked the first role in allowed_roles if not “user”, otherwise “user”.

Which roles should we forward to the API? Default: ["system", "user", "assistant"] usually, but some models like OpenAI’s o1-mini will use ["user", "assistant"]

When building prompts, any role not in this list will be set to the default_role.

Which role metadata should we forward to the API? Default: []

For example you can set this to ["cache_control"] to forward the cache policy to the API.

If you do not set allowed_role_metadata, we will not forward any role metadata to the API even if it is set in the prompt.

Then in your prompt you can use something like:

1
client<llm> ClaudeWithCaching {
2
  provider anthropic
3
  options {
4
    model claude-3-haiku-20240307
5
    api_key env.ANTHROPIC_API_KEY
6
    max_tokens 1000
7
    allowed_role_metadata ["cache_control"]
8
    headers {
9
      "anthropic-beta" "prompt-caching-2024-07-31"
10
    }
11
  }
12
}
13
14
client<llm> FooWithout {
15
  provider anthropic
16
  options {
17
  }
18
}
19
20
template_string Foo() #"
21
  {{ _.role('user', cache_control={"type": "ephemeral"}) }}
22
  This will be cached for ClaudeWithCaching, but not for FooWithout!
23
  {{ _.role('user') }}
24
  This will not be cached for Foo or FooWithout!
25
"#

You can use the playground to see the raw curl request to see what is being sent to the API.

Whether the internal LLM client should use the streaming API. Default: true

Then in your prompt you can use something like:

1
client<llm> MyClientWithoutStreaming {
2
  provider anthropic
3
  options {
4
    model claude-3-haiku-20240307
5
    api_key env.ANTHROPIC_API_KEY
6
    max_tokens 1000
7
    supports_streaming false
8
  }
9
}
10
11
function MyFunction() -> string {
12
  client MyClientWithoutStreaming
13
  prompt #"Write a short story"#
14
}

1
# This will be streamed from your python code perspective, 
2
# but under the hood it will call the non-streaming HTTP API
3
# and then return a streamable response with a single event
4
b.stream.MyFunction()
5
6
# This will work exactly the same as before
7
b.MyFunction()

Which finish reasons are allowed? Default: null

Will raise a BamlClientFinishReasonError if the finish reason is not in the allow list. See Exceptions for more details.

Note, only one of finish_reason_allow_list or finish_reason_deny_list can be set.

For example you can set this to ["stop"] to only allow the stop finish reason, all other finish reasons (e.g. length) will treated as failures that PREVENT fallbacks and retries (similar to parsing errors).

Then in your code you can use something like:

1
client<llm> MyClient {
2
  provider "openai"
3
  options {
4
    model "gpt-4o-mini"
5
    api_key env.OPENAI_API_KEY
6
    // Finish reason allow list will only allow the stop finish reason
7
    finish_reason_allow_list ["stop"]
8
  }
9
}

Which finish reasons are denied? Default: null

Will raise a BamlClientFinishReasonError if the finish reason is in the deny list. See Exceptions for more details.

Note, only one of finish_reason_allow_list or finish_reason_deny_list can be set.

For example you can set this to ["length"] to stop the function from continuing if the finish reason is length. (e.g. LLM was cut off because it was too long).

Then in your code you can use something like:

1
client<llm> MyClient {
2
  provider "openai"
3
  options {
4
    model "gpt-4o-mini"
5
    api_key env.OPENAI_API_KEY
6
    // Finish reason deny list will allow all finish reasons except length
7
    finish_reason_deny_list ["length"]
8
  }
9
}

BAML will auto construct this field for you from the prompt, if necessary. Only the first system message will be used, all subsequent ones will be cast to the assistant role.

BAML will auto construct this field for you from the prompt

BAML will auto construct this field for you based on how you call the client in your code

The model to use.

See anthropic docs for the latest list of all models. You can pass any model name you wish, we will not check if it exists.

The maximum number of tokens to generate. Default: 4069