groq

Groq supports the OpenAI client, allowing you to use the openai-generic provider with an overridden base_url.

See https://console.groq.com/docs/openai for more information.

BAML
client<llm> MyClient {
provider openai-generic
options {
base_url "https://api.groq.com/openai/v1"
api_key env.GROQ_API_KEY
model "llama-3-groq-70b-tool-use"
}
}