client<llm>
Clients are used to configure how LLMs are called, like so:
This is <provider>/<model>
shorthand for:
Consult the provider documentation for a list of supported providers and models, and the default options.
If you want to override options like api_key
to use a different environment
variable, or you want to point base_url
to a different endpoint, you should use
the latter form.
If you want to specify which client to use at runtime, in your Python/TS/Ruby code, you can use the client registry to do so.
This can come in handy if you’re trying to, say, send 10% of your requests to a different model.
Fields
This configures which provider to use. The provider is responsible for handling the actual API calls to the LLM service. The provider is a required field.
The configuration modifies the URL request BAML runtime makes.
We also have some special providers that allow composing clients together:
These vary per provider. Please see provider specific documentation for more information. Generally they are pass through options to the POST request made to the LLM.
The name of the retry policy. See Retry Policy.