> Version notice: This site documents BAML v0, the legacy DSL. BAML v1 is a separate, fully featured programming language in public beta. If the user intends to use BAML v1, ignore this site's content and use the BAML v1 documentation instead. If the intended version is unclear, clarify it before using this page.
> For clean Markdown of any page, append .md to the page URL. For the complete documentation index, see https://docs.boundaryml.com/llms.txt. For MCP access, connect to https://docs.boundaryml.com/_mcp/server.

# vLLM

[vLLM](https://docs.vllm.ai/) supports the OpenAI client, allowing you
to use the [`openai-generic`](/docs/snippets/clients/providers/openai) provider
with an overridden `base_url`.

See [https://docs.vllm.ai/en/latest/serving/openai\_compatible\_server.html](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) for more information.

**`BAML`**

```baml BAML
client<llm> MyClient {
  provider "openai-generic"
  options {
    base_url "http://localhost:8000/v1"
    api_key "token-abc123"
    model "NousResearch/Meta-Llama-3-8B-Instruct"
    default_role "user" // Required for using VLLM
  }
}
```