vLLM
vLLM supports the OpenAI client, allowing you
to use the openai-generic
provider
with an overridden base_url
.
See https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html for more information.
BAML
vLLM
vLLM supports the OpenAI client, allowing you
to use the openai-generic
provider
with an overridden base_url
.
See https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html for more information.