azure-openai

For azure-openai, we provide a client that can be used to interact with the OpenAI API hosted on Azure using the /chat/completions endpoint.

Example:

BAML
1client<llm> MyClient {
2  provider azure-openai
3  options {
4    resource_name "my-resource-name"
5    deployment_id "my-deployment-id"
6    // Alternatively, you can use the base_url field
7    // base_url "https://my-resource-name.openai.azure.com/openai/deployments/my-deployment-id"
8    api_version "2024-02-01"
9    api_key env.AZURE_OPENAI_API_KEY
10  }
11}

api_version is required. Azure will return not found if the version is not specified.

The options are passed through directly to the API, barring a few. Here’s a shorthand of the options:

Non-forwarded options

api_key
string

Will be injected via the header API-KEY. Default: env.AZURE_OPENAI_API_KEY

API-KEY: $api_key

base_url
string

The base URL for the API. Default: https://${resource_name}.openai.azure.com/openai/deployments/${deployment_id}

May be used instead of resource_name and deployment_id.

deployment_id
stringRequired

See the base_url field.

resource_name
stringRequired

See the base_url field.

default_role
string

The default role for any prompts that don’t specify a role. Default: system

We don’t have any checks for this field, you can pass any string you wish.

api_version
stringRequired

Will be passed via a query parameter api-version.

headers
object

Additional headers to send with the request.

Example:

BAML
1client<llm> MyClient {
2  provider azure-openai
3  options {
4    resource_name "my-resource-name"
5    deployment_id "my-deployment-id"
6    api_version "2024-02-01"
7    api_key env.AZURE_OPENAI_API_KEY
8    headers {
9      "X-My-Header" "my-value"
10    }
11  }
12}
allowed_role_metadata
string[]

Which role metadata should we forward to the API? Default: []

For example you can set this to ["foo", "bar"] to forward the cache policy to the API.

If you do not set allowed_role_metadata, we will not forward any role metadata to the API even if it is set in the prompt.

Then in your prompt you can use something like:

1client<llm> Foo {
2  provider openai
3  options {
4    allowed_role_metadata: ["foo", "bar"]
5  }
6}
7
8client<llm> FooWithout {
9  provider openai
10  options {
11  }
12}
13template_string Foo() #"
14  {{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15  This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16  {{ _.role('user') }}
17  This will have none of the role metadata for Foo or FooWithout.
18"#

You can use the playground to see the raw curl request to see what is being sent to the API.

Forwarded options

messages
DO NOT USE

BAML will auto construct this field for you from the prompt

stream
DO NOT USE

BAML will auto construct this field for you based on how you call the client in your code

For all other options, see the official Azure API documentation.