A client is the mechanism by which an impl is executed (can be an LLM or otherwise).

Syntax

client<CLIENT_TYPE> Name {
    provider ProviderName
    options {
      // ...
    }
}
  • CLIENT_TYPE: What type of AI model will be used. Currently must be llm
  • Name: The name of the client (can be any [a-zA-Z], numbers or _). Must start with a letter.

Properties

PropertyTypeDescriptionRequired
providername of the providerThe provider to use.Yes
optionskey-value pairThese are passed through directly to the provider.No
retry_policyname of the policyLearn moreNo

Providers

BAML ships with the following providers (you can can also write your own!):

  • LLM client providers
    • baml-anthropic
    • baml-azure-chat
    • baml-azure-completion
    • baml-openai-chat
    • baml-openai-completion
    • baml-ollama-chat
    • baml-ollama-completion (only for python)
  • Composite client providers
    • baml-fallback
    • baml-round-robin

There are two primary types of LLM clients: chat and completion. BAML abstracts away the differences between these two types of LLMs by putting that logic in the clients.

You can call a chat client with a single completion prompt and it will automatically map it to a chat prompt. Similarly you can call a completion client with multiple chat prompts and it will automatically map it to a completion prompt.

OpenAI/Azure

Provider names:

  • baml-openai-chat
  • baml-openai-completion
  • baml-azure-chat
  • baml-azure-completion

You must pick the right provider for the type of model you are using. For example, if you are using a GPT-3 model, you must use a chat provider, but if you’re using the instruct model, you must use a completion provider.

You can see all models supported by OpenAI here.

Accepts any options as defined by OpenAI/Azure SDK

See Azure Docs to learn how to get your Azure API key.

// A client that uses the OpenAI chat API.
client<llm> MyGPT35Client {
  // Since we're using a GPT-3 model, we must use a chat provider.
  provider baml-openai-chat
  options {
    model gpt-3.5-turbo
    // Set the api_key parameter to the OPENAI_API_KEY environment variable
    api_key env.OPENAI_API_KEY
  }
}

// A client that uses the OpenAI chat API.
client<llm> MyAzureClient {
  // I configured the deployment to use a GPT-3 model,
  // so I must use a chat provider.
  provider baml-azure-chat
  options {
        api_key env.AZURE_OPENAI_KEY
        api_base env.AZURE_OPENAI_ENDPOINT
        // This may change in the future
        api_version '2023-05-15'
        api_type azure
        engine REPLACE_WITH_YOUR_DEPLOYMENT_NAME
    }
}

BAML uses openai <= 0.28.1! We’re working on migrating our providers over and ensuring you can use the latest version of openai regardless. The benefit of BAML is that even if openai’s SDK changes, you can still use the same BAML code.

Anthropic

Provider names:

  • baml-anthropic

Accepts any options as defined by Anthropic SDK

client<llm> MyClient {
  provider baml-anthropic
  options {
    model claude-2
    max_tokens_to_sample 300
  }
}

Ollama

  • BAML Python Client >= 0.18.0
  • BAML Typescript Client >= 0.0.6

Provider names:

  • baml-ollama-chat
  • baml-ollama-completion (only for python)

Accepts any options as defined by Ollama SDK.

client<llm> MyClient {
  provider baml-ollama-chat
  options {
    model mistral
    options {
      temperature 0
    }
  }
}

Fallback

The baml-fallback provider allows you to define a resilient client, by specifying strategies for re-running failed requests. See Fallbacks/Redundancy for more information.

Round Robin

The baml-round-robin provider allows you to load-balance your requests across multiple clients. Here’s an example:

client<llm> MyClient {
  provider baml-round-robin
  options {
    strategy [
      MyGptClient
      MyAnthropicClient
    ]
  }
}

This will alternate between MyGptClient and MyAnthropicClient for successive requests, starting from a randomly selected client (so that if you run multiple instances of your application, they won’t all start with the same client).

If you want to control which client is used for the first request, you can specify a start index, which tells BAML to start with the client at index start, like so:

client<llm> MyClient {
  provider baml-round-robin
  options {
    start 1
    strategy [
      MyGptClient
      MyAnthropicClient
    ]
  }
}

Creating your own (Advanced)

Creating a provider is requires a bit more deeper understanding. You can see our source code for any of our providers here:

You can add your own provider by creating a new file anywhere in your application logic, then setting the provider name in .baml to name you registered it with in your implementation.

We are actively working on how to make custom providers better / easier. If you have ideas, please reach out to us on discord!