google-ai — Boundary Documentation

The google-ai provider supports the https://generativelanguage.googleapis.com/v1beta/models/{model_id}/generateContent and https://generativelanguage.googleapis.com/v1beta/models/{model_id}/streamGenerateContent endpoints.

The use of v1beta rather than v1 aligns with the endpoint conventions established in Google’s SDKs and offers access to both the existing v1 models and additional models exclusive to v1beta.

BAML will automatically pick streamGenerateContent if you call the streaming interface.

Example:

BAML

1 client<llm> MyClient {
2   provider google-ai
3   options {
4     model "gemini-1.5-flash"
5   }
6 }

The options are passed through directly to the API, barring a few. Here’s a shorthand of the options:

Non-forwarded options

api_key

string

Will be passed as the x-goog-api-key header. Default: env.GOOGLE_API_KEY

x-goog-api-key: $api_key

base_url

string

The base URL for the API. Default: https://generativelanguage.googleapis.com/v1beta

default_role

string

The default role for any prompts that don’t specify a role. Default: user

We don’t have any checks for this field, you can pass any string you wish.

model

string

The model to use. Default: gemini-1.5-flash

We don’t have any checks for this field, you can pass any string you wish.

Model	Input(s)	Optimized for
`gemini-1.5-pro`	Audio, images, videos, and text	Complex reasoning tasks such as code and text generation, text editing, problem solving, data extraction and generation
`gemini-1.5-flash`	Audio, images, videos, and text	Fast and versatile performance across a diverse variety of tasks
`gemini-1.0-pro`	Text	Natural language tasks, multi-turn text and code chat, and code generation

See the Google Model Docs for the latest models.

headers

object

Additional headers to send with the request.

Example:

BAML

1 client<llm> MyClient {
2   provider google-ai
3   options {
4     model "gemini-1.5-flash"
5     headers {
6       "X-My-Header" "my-value"
7     }
8   }
9 }

allowed_role_metadata

string[]

Which role metadata should we forward to the API? Default: []

For example you can set this to ["foo", "bar"] to forward the cache policy to the API.

If you do not set allowed_role_metadata, we will not forward any role metadata to the API even if it is set in the prompt.

Then in your prompt you can use something like:

1 client<llm> Foo {
2   provider openai
3   options {
4     allowed_role_metadata: ["foo", "bar"]
5   }
6 }
7 
8 client<llm> FooWithout {
9   provider openai
10   options {
11   }
12 }
13 template_string Foo() #"
14   {{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15   This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16   {{ _.role('user') }}
17   This will have none of the role metadata for Foo or FooWithout.
18 "#

You can use the playground to see the raw curl request to see what is being sent to the API.

supports_streaming

boolean

Whether the internal LLM client should use the streaming API. Default: true

Then in your prompt you can use something like:

1 client<llm> MyClientWithoutStreaming {
2   provider anthropic
3   options {
4     model claude-3-haiku-20240307
5     api_key env.ANTHROPIC_API_KEY
6     max_tokens 1000
7     supports_streaming false
8   }
9 }
10 
11 function MyFunction() -> string {
12   client MyClientWithoutStreaming
13   prompt #"Write a short story"#
14 }

1 # This will be streamed from your python code perspective, 
2 # but under the hood it will call the non-streaming HTTP API
3 # and then return a streamable response with a single event
4 b.stream.MyFunction()
5 
6 # This will work exactly the same as before
7 b.MyFunction()

Forwarded options

contents

DO NOT USE

BAML will auto construct this field for you from the prompt

For all other options, see the official Google Gemini API documentation.

1	client<llm> MyClient {
2	provider google-ai
3	options {
4	model "gemini-1.5-flash"
5	}
6	}

1	client<llm> Foo {
2	provider openai
3	options {
4	allowed_role_metadata: ["foo", "bar"]
5	}
6	}
7
8	client<llm> FooWithout {
9	provider openai
10	options {
11	}
12	}
13	template_string Foo() #"
14	{{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15	This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16	{{ _.role('user') }}
17	This will have none of the role metadata for Foo or FooWithout.
18	"#

1	client<llm> MyClientWithoutStreaming {
2	provider anthropic
3	options {
4	model claude-3-haiku-20240307
5	api_key env.ANTHROPIC_API_KEY
6	max_tokens 1000
7	supports_streaming false
8	}
9	}
10
11	function MyFunction() -> string {
12	client MyClientWithoutStreaming
13	prompt #"Write a short story"#
14	}

1	# This will be streamed from your python code perspective,
2	# but under the hood it will call the non-streaming HTTP API
3	# and then return a streamable response with a single event
4	b.stream.MyFunction()
5
6	# This will work exactly the same as before
7	b.MyFunction()