azure-openai

Will be injected via the header API-KEY. Default: env.AZURE_OPENAI_API_KEY

API-KEY: $api_key

The base URL for the API. Default: https://${resource_name}.openai.azure.com/openai/deployments/${deployment_id}

May be used instead of resource_name and deployment_id.

See the base_url field.

See the base_url field.

The default role for any prompts that don’t specify a role. Default: system

We don’t have any checks for this field, you can pass any string you wish.

Will be passed via a query parameter api-version.

Additional headers to send with the request.

Example:

1
client<llm> MyClient {
2
  provider azure-openai
3
  options {
4
    resource_name "my-resource-name"
5
    deployment_id "my-deployment-id"
6
    api_version "2024-02-01"
7
    api_key env.AZURE_OPENAI_API_KEY
8
    headers {
9
      "X-My-Header" "my-value"
10
    }
11
  }
12
}

Which role metadata should we forward to the API? Default: []

For example you can set this to ["foo", "bar"] to forward the cache policy to the API.

If you do not set allowed_role_metadata, we will not forward any role metadata to the API even if it is set in the prompt.

Then in your prompt you can use something like:

1
client<llm> Foo {
2
  provider openai
3
  options {
4
    allowed_role_metadata: ["foo", "bar"]
5
  }
6
}
7
8
client<llm> FooWithout {
9
  provider openai
10
  options {
11
  }
12
}
13
template_string Foo() #"
14
  {{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15
  This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16
  {{ _.role('user') }}
17
  This will have none of the role metadata for Foo or FooWithout.
18
"#

You can use the playground to see the raw curl request to see what is being sent to the API.

Whether the internal LLM client should use the streaming API. Default: true

Then in your prompt you can use something like:

1
client<llm> MyClientWithoutStreaming {
2
  provider anthropic
3
  options {
4
    model claude-3-haiku-20240307
5
    api_key env.ANTHROPIC_API_KEY
6
    max_tokens 1000
7
    supports_streaming false
8
  }
9
}
10
11
function MyFunction() -> string {
12
  client MyClientWithoutStreaming
13
  prompt #"Write a short story"#
14
}

1
# This will be streamed from your python code perspective, 
2
# but under the hood it will call the non-streaming HTTP API
3
# and then return a streamable response with a single event
4
b.stream.MyFunction()
5
6
# This will work exactly the same as before
7
b.MyFunction()

BAML will auto construct this field for you from the prompt

BAML will auto construct this field for you based on how you call the client in your code