openai — Boundary Documentation

The openai provider supports the OpenAI /chat endpoint, setting OpenAI-specific default configuration options.

For Azure, we recommend using azure-openai instead.

For all other OpenAI-compatible API providers, such as Groq, HuggingFace, Ollama, OpenRouter, Together AI, and others, we recommend using openai-generic instead.

Example:

BAML

1 client<llm> MyClient {
2   provider "openai"
3   options {
4     api_key env.MY_OPENAI_KEY
5     model "gpt-3.5-turbo"
6     temperature 0.1
7   }
8 }

The options are passed through directly to the API, barring a few. Here’s a shorthand of the options:

Non-forwarded options

api_key

stringDefaults to env.OPENAI_API_KEY

Will be used to build the Authorization header, like so: Authorization: Bearer $api_key

Default: env.OPENAI_API_KEY

base_url

string

The base URL for the API.

Default: https://api.openai.com/v1

default_role

string

The default role for any prompts that don’t specify a role.

We don’t do any validation of this field, so you can pass any string you wish.

Default: system

headers

object

Additional headers to send with the request.

Example:

BAML

1 client<llm> MyClient {
2   provider openai
3   options {
4     api_key env.MY_OPENAI_KEY
5     model "gpt-3.5-turbo"
6     headers {
7       "X-My-Header" "my-value"
8     }
9   }
10 }

allowed_role_metadata

string[]

Which role metadata should we forward to the API? Default: []

For example you can set this to ["foo", "bar"] to forward the cache policy to the API.

If you do not set allowed_role_metadata, we will not forward any role metadata to the API even if it is set in the prompt.

Then in your prompt you can use something like:

1 client<llm> Foo {
2   provider openai
3   options {
4     allowed_role_metadata: ["foo", "bar"]
5   }
6 }
7 
8 client<llm> FooWithout {
9   provider openai
10   options {
11   }
12 }
13 template_string Foo() #"
14   {{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15   This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16   {{ _.role('user') }}
17   This will have none of the role metadata for Foo or FooWithout.
18 "#

You can use the playground to see the raw curl request to see what is being sent to the API.

supports_streaming

boolean

Whether the internal LLM client should use the streaming API. Default: <auto>

Model	Supports Streaming
`o1-preview`	false
`o1-mini`	false
`o1-*`	false
`gpt-4o`	true
`gpt-4o-mini`	true
`*`	true

Then in your prompt you can use something like:

1 client<llm> MyClientWithoutStreaming {
2   provider openai
3   options {
4     model gpt-4o
5     api_key env.OPENAI_API_KEY
6     supports_streaming false 
7   }
8 }
9 
10 function MyFunction() -> string {
11   client MyClientWithoutStreaming
12   prompt #"Write a short story"#
13 }

1 # This will be streamed from your python code perspective, 
2 # but under the hood it will call the non-streaming HTTP API
3 # and then return a streamable response with a single event
4 b.stream.MyFunction()
5 
6 # This will work exactly the same as before
7 b.MyFunction()

Forwarded options

messages

DO NOT USE

BAML will auto construct this field for you from the prompt

stream

DO NOT USE

BAML will auto construct this field for you based on how you call the client in your code

model

string

The model to use.

Model	Description
`gpt-3.5-turbo`	Fastest
`gpt-4o`	Fast + text + image
`gpt-4-turbo`	Smartest + text + image + code
`gpt-4o-mini`	Cheapest + text + image

See openai docs for the list of openai models. You can pass any model name you wish, we will not check if it exists.

For all other options, see the official OpenAI API documentation.

1	client<llm> MyClient {
2	provider "openai"
3	options {
4	api_key env.MY_OPENAI_KEY
5	model "gpt-3.5-turbo"
6	temperature 0.1
7	}
8	}

1	client<llm> MyClient {
2	provider openai
3	options {
4	api_key env.MY_OPENAI_KEY
5	model "gpt-3.5-turbo"
6	headers {
7	"X-My-Header" "my-value"
8	}
9	}
10	}

1	client<llm> Foo {
2	provider openai
3	options {
4	allowed_role_metadata: ["foo", "bar"]
5	}
6	}
7
8	client<llm> FooWithout {
9	provider openai
10	options {
11	}
12	}
13	template_string Foo() #"
14	{{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15	This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16	{{ _.role('user') }}
17	This will have none of the role metadata for Foo or FooWithout.
18	"#

1	client<llm> MyClientWithoutStreaming {
2	provider openai
3	options {
4	model gpt-4o
5	api_key env.OPENAI_API_KEY
6	supports_streaming false
7	}
8	}
9
10	function MyFunction() -> string {
11	client MyClientWithoutStreaming
12	prompt #"Write a short story"#
13	}

1	# This will be streamed from your python code perspective,
2	# but under the hood it will call the non-streaming HTTP API
3	# and then return a streamable response with a single event
4	b.stream.MyFunction()
5
6	# This will work exactly the same as before
7	b.MyFunction()