Tools / Function Calling

“Function calling” is a technique for getting an LLM to choose a function to call for you.

The way it works is:

1. You define a task with certain function(s)
2. Ask the LLM to choose which function to call
3. Get the function parameters from the LLM for the appropriate function it choose
4. Call the functions in your code with those parameters

It’s common for people to think of “function calling” or “tool use” separately from “structured outputs” (even OpenAI has separate parameters for them), but at BAML, we think it’s simpler and more impactful to think of them equivalently. This is because, at the end of the day, you are looking to get something processable back from your LLM. Whether it’s extracting data from a document or calling the Weather API, you need a standard representation of that output, which is where BAML lives.

In BAML, you can get represent a tool or a function you want to call as a BAML class, and make the function output be that class definition.

class WeatherAPI {
  // we can use literals to denote the name of the tool
  // the field can be named anything we want! "api_name" "tool" "function_name"
  // whatever you feel the LLM would understand best
  api_name "weather_request"
  city string @description("the user's city")
  timeOfDay string @description("As an ISO8601 timestamp")
}
function UseTool(user_message: string) -> WeatherAPI {
  client "openai/gpt-5-mini"
  prompt #"
    Given a message, extract info.
    {# special macro to print the functions return type. #}
    {{ ctx.output_format }}
    {{ _.role('user') }}
    {{ user_message }}
  "#
}

Call the function like this:

import asyncio
import datetime
from baml_client import b
from baml_client.types import WeatherAPI
def get_weather(city: str, time_of_day: datetime.date):
    ...
def main():
    weather_info = b.UseTool("What's the weather like in San Francisco?")
    print(weather_info)
    assert isinstance(weather_info, WeatherAPI)
    print(f"City: {weather_info.city}")
    print(f"Time of Day: {weather_info.time_of_day}")
    weather = get_weather(city=weather_info.city, time_of_day=weather_info.timeOfDay)
if __name__ == '__main__':
    main()

Choosing multiple Tools

To choose ONE tool out of many, you can use a union:

function UseTool(user_message: string) -> WeatherAPI | MyOtherAPI {
  .... // same thing
}

Call the function like this:

import asyncio
from baml_client import b
from baml_client.types import WeatherAPI, MyOtherAPI
async def main():
    tool = b.UseTool("What's the weather like in San Francisco?")
    print(tool)
    
    if isinstance(tool, WeatherAPI):
        print(f"Weather API called:")
        print(f"City: {tool.city}")
        print(f"Time of Day: {tool.timeOfDay}")
    elif isinstance(tool, MyOtherAPI):
        print(f"MyOtherAPI called:")
        # Handle MyOtherAPI specific attributes here
if __name__ == '__main__':
    main()

Choosing N Tools

To choose many tools, you can use a union of a list:

function UseTool(user_message: string) -> (WeatherAPI | MyOtherAPI)[] {
  client "openai/gpt-5-mini"
  prompt #"
    Given a message, extract info.
    {# special macro to print the functions return type. #}
    {{ ctx.output_format }}
    {{ _.role('user') }}
    {{ user_message }}
  "#
}

Call the function like this:

import asyncio
from baml_client import b
from baml_client.types import WeatherAPI, MyOtherAPI
async def main():
    tools = b.UseTool("What's the weather like in San Francisco and New York?")
    print(tools)  
    
    for tool in tools:
        if isinstance(tool, WeatherAPI):
            print(f"Weather API called:")
            print(f"City: {tool.city}")
            print(f"Time of Day: {tool.timeOfDay}")
        elif isinstance(tool, MyOtherAPI):
            print(f"MyOtherAPI called:")
            # Handle MyOtherAPI specific attributes here
if __name__ == '__main__':
    main()

Disambiguating Between Similar Tools

When building functions that can call multiple tools (represented as BAML classes), you might encounter situations where different tools accept arguments with the same name. For instance, consider GetWeather and GetTimezone classes, both taking a city: string argument. How does the system determine whether a user query like “What’s the time in London?” corresponds to GetTimezone or potentially GetWeather?

You can use string literals to solve this problem:

class GetWeather {
  tool_name "get_weather" @description("Use this tool to get the current weather forecast for a specific city.")
  city string @description("The city for which to get the weather.")
}
class GetTimezone {
  tool_name "get_timezone" @description("Use this tool to find the current timezone of a specific city.")
  city string @description("The city for which to find the timezone.")
}
function ChooseTool(query: string) -> GetWeather | GetTimezone {
  client "openai/gpt-5"
  prompt #"
    Given the user query, determine the primary intent and select the appropriate tool to call.
    {# special macro to add tool structures + descriptions here #}
    {{ ctx.output_format }} 
    {{ _.role('user') }}
    {{ query }}
  "#
}

Dynamically Generate the tool signature

It might be cumbersome to define schemas in baml and code, so you can define them from code as well. Read more about dynamic types here

class WeatherAPI {
  @@dynamic // params defined from code
}
function UseTool(user_message: string) -> WeatherAPI {
  client "openai/gpt-5-mini"
  prompt #"
    Given a message, extract info.
    {# special macro to print the functions return type. #}
    {{ ctx.output_format }}
    {{ _.role('user') }}
    {{ user_message }}
  "#
}

Call the function like this:

import asyncio
import inspect
from baml_client import b
from baml_client.type_builder import TypeBuilder
from baml_client.types import WeatherAPI
async def get_weather(city: str, time_of_day: str):
    print(f"Getting weather for {city} at {time_of_day}")
    return 42
async def main():
    tb = TypeBuilder()
    type_map = {int: tb.int(), float: tb.float(), str: tb.string()}
    signature = inspect.signature(get_weather)
    for param_name, param in signature.parameters.items():
        tb.WeatherAPI.add_property(param_name, type_map[param.annotation])
    tool = b.UseTool("What's the weather like in San Francisco this afternoon?", { "tb": tb })
    print(tool)
    weather = await get_weather(**tool.model_dump())
    print(weather)
if __name__ == '__main__':
    asyncio.run(main())

Function-calling APIs vs Prompting

Injecting your function schemas into the prompt, as BAML does, outperforms function-calling across all benchmarks for major providers (see our Berkeley FC Benchmark results with BAML).

Amongst other limitations, function-calling APIs will at times:

1. Return a schema when you don’t want any (you want an error)
2. Not work for tools with more than 100 parameters.
3. Use many more tokens than prompting.

Keep in mind that “JSON mode” is nearly the same thing as “prompting”, but it enforces the LLM response is ONLY a JSON blob. BAML does not use JSON mode since it allows developers to use better prompting techniques like chain-of-thought, to allow the LLM to express its reasoning before printing out the actual schema. BAML’s parser can find the json schema(s) out of free-form text for you. Read more about different approaches to structured generation here

BAML will still support native function-calling APIs in the future (please let us know more about your use-case so we can prioritize accordingly)

Create an Agent that utilizes these Tools

We can create an Agent or an “agentic loop” that continuously uses tools in a program simply by adding a while loop in our code. In this example, we’ll have two tools:

1. An API that queries the weather.
2. An API that does basic calculations on numbers.

This is what it looks in the BAML file:

class WeatherAPI {
  intent "weather_request"
  city string
  time string @description("Current time in ISO8601 format")
}
class CalculatorAPI {
  intent "basic_calculator"
  operation "add" | "subtract" | "multiply" | "divide"
  numbers float[]
}
function SelectTool(message: string) -> WeatherAPI | CalculatorAPI {
  client "openai/gpt-5"
  prompt #"
    Given a message, extract info.
    {{ ctx.output_format }}
    {{ _.role("user") }} {{ message }}
  "#
}

In our agent code, we’ll:

1. Implement our APIs
2. Implement our Agent that continuously will use different tools

from baml_client import b
from baml_client.types import WeatherAPI, CalculatorAPI
def handle_weather(weather: WeatherAPI):
    # Simulate weather API call, but you can implement this with a real API call
    return f"The weather in {weather.city} at {weather.time} is sunny."
def handle_calculator(calc: CalculatorAPI):
    numbers = calc.numbers
    if calc.operation == "add":
        result = sum(numbers)
    elif calc.operation == "subtract":
        result = numbers[0] - sum(numbers[1:])
    elif calc.operation == "multiply":
        result = 1
        for n in numbers:
            result *= n
    elif calc.operation == "divide":
        result = numbers[0]
        for n in numbers[1:]:
            result /= n
    return f"The result is {result}"
def main():
    print("Agent started! Type 'exit' to quit.")
    
    while True:
        # Get user input
        user_input = input("You: ")
        if user_input.lower() == 'exit':
            break
        # Call the BAML function to select tool
        tool_response = b.SelectTool(user_input)
        # Handle the tool response
        if isinstance(tool_response, WeatherAPI):
            result = handle_weather(tool_response)
            print(f"Agent (Weather): {result}")
        
        elif isinstance(tool_response, CalculatorAPI):
            result = handle_calculator(tool_response)
            print(f"Agent (Calculator): {result}")
if __name__ == "__main__":
    main()

We can test this by asking things like:

1. What is the weather in Seattle?
2. What’s 5+2?

This is the output:

Agent started! Type 'exit' to quit.
You: What's the weather in Seattle
Agent (Weather): The weather in Seattle at 2023-10-02T12:00:00Z is sunny.
You: What's 5+2
Agent (Calculator): The result is 7.0