Client Registry
If you need to modify the model / parameters for an LLM client at runtime, you can modify the ClientRegistry
for any specified function.
Python
TypeScript
Ruby
OpenAPI
ClientRegistry Interface
Note: ClientRegistry
is imported from baml_py
in Python and @boundaryml/baml
in TypeScript, not baml_client
.
As we mature ClientRegistry
, we will add a more type-safe and ergonomic interface directly in baml_client
. See Github issue #766.
Methods use snake_case
in Python and camelCase
in TypeScript.
add_llm_client / addLlmClient
A function to add an LLM client to the registry.
The name of the client.
Using the exact same name as a client also defined in .baml files overwrites the existing client whenever the ClientRegistry is used.
This configures which provider to use. The provider is responsible for handling the actual API calls to the LLM service. The provider is a required field.
The configuration modifies the URL request BAML runtime makes.
We also have some special providers that allow composing clients together:
These vary per provider. Please see provider specific documentation for more information. Generally they are pass through options to the POST request made to the LLM.
The name of a retry policy that is already defined in a .baml file. See Retry Policies.
set_primary / setPrimary
This sets the client for the function to use. (i.e. replaces the client
property in a function)
The name of the client to use.
This can be a new client that was added with add_llm_client
or an existing client that is already in a .baml file.