groq

Groq supports the OpenAI client, allowing you to use the openai-generic provider with an overridden base_url.

See https://console.groq.com/docs/openai for more information.

BAML

1 client<llm> MyClient {
2   provider openai-generic
3   options {
4     base_url "https://api.groq.com/openai/v1"
5     api_key env.GROQ_API_KEY
6     model "llama3-70b-8192"
7   }
8 }

1	client<llm> MyClient {
2	provider openai-generic
3	options {
4	base_url "https://api.groq.com/openai/v1"
5	api_key env.GROQ_API_KEY
6	model "llama3-70b-8192"
7	}
8	}