Client Registry

If you need to modify the model / parameters for an LLM client at runtime, you can modify the ClientRegistry for any specified function.

1import os
2from baml_py import ClientRegistry
3
4async def run():
5    cr = ClientRegistry()
6    # Creates a new client
7    cr.add_llm_client(name='MyAmazingClient', provider='openai', options={
8        "model": "gpt-4o",
9        "temperature": 0.7,
10        "api_key": os.environ.get('OPENAI_API_KEY')
11    })
12    # Sets MyAmazingClient as the primary client
13    cr.set_primary('MyAmazingClient')
14
15    # ExtractResume will now use MyAmazingClient as the calling client
16    res = await b.ExtractResume("...", { "client_registry": cr })

ClientRegistry Interface

Note: ClientRegistry is imported from baml_py in Python and @boundaryml/baml in TypeScript, not baml_client.

As we mature ClientRegistry, we will add a more type-safe and ergonomic interface directly in baml_client. See Github issue #766.

Methods use snake_case in Python and camelCase in TypeScript.

add_llm_client / addLlmClient

A function to add an LLM client to the registry.

name
stringRequired

The name of the client.

Using the exact same name as a client also defined in .baml files overwrites the existing client whenever the ClientRegistry is used.

provider
stringRequired

This configures which provider to use. The provider is responsible for handling the actual API calls to the LLM service. The provider is a required field.

The configuration modifies the URL request BAML runtime makes.

Provider NameDocsNotes
anthropicAnthropic
aws-bedrockAWS Bedrock
azure-openaiAzure OpenAI
google-aiGoogle AI
openaiOpenAI
openai-genericOpenAI (generic)Any model provider that supports an OpenAI-compatible API
vertex-aiVertex AI

We also have some special providers that allow composing clients together:

Provider NameDocsNotes
fallbackFallbackUsed to chain models conditional on failures
round-robinRound RobinUsed to load balance
options
dict[str, Any]Required

These vary per provider. Please see provider specific documentation for more information. Generally they are pass through options to the POST request made to the LLM.

retry_policy
string

The name of a retry policy that is already defined in a .baml file. See Retry Policies.

set_primary / setPrimary

This sets the client for the function to use. (i.e. replaces the client property in a function)

name
stringRequired

The name of the client to use.

This can be a new client that was added with add_llm_client or an existing client that is already in a .baml file.