Tools / Function Calling

“Function calling” is a technique for getting an LLM to choose a function to call for you.

The way it works is:

  1. You define a task with certain function(s)
  2. Ask the LLM to choose which function to call
  3. Get the function parameters from the LLM for the appropriate function it choose
  4. Call the functions in your code with those parameters

It’s common for people to think of “function calling” or “tool use” separately from “structured outputs” (even OpenAI has separate parameters for them), but at BAML, we think it’s simpler and more impactful to think of them equivalently. This is because, at the end of the day, you are looking to get something processable back from your LLM. Whether it’s extracting data from a document or calling the Weather API, you need a standard representation of that output, which is where BAML lives.

In BAML, you can get represent a tool or a function you want to call as a BAML class, and make the function output be that class definition.

BAML
1class WeatherAPI {
2  city string @description("the user's city")
3  timeOfDay string @description("As an ISO8601 timestamp")
4}
5
6function UseTool(user_message: string) -> WeatherAPI {
7  client "openai/gpt-4o-mini"
8  prompt #"
9    Given a message, extract info.
10    {# special macro to print the functions return type. #}
11    {{ ctx.output_format }}
12
13    {{ _.role('user') }}
14    {{ user_message }}
15  "#
16}

Call the function like this:

1import asyncio
2import datetime
3from baml_client import b
4from baml_client.types import WeatherAPI
5
6def get_weather(city: str, time_of_day: datetime.date):
7    ...
8
9def main():
10    weather_info = b.UseTool("What's the weather like in San Francisco?")
11    print(weather_info)
12    assert isinstance(weather_info, WeatherAPI)
13    print(f"City: {weather_info.city}")
14    print(f"Time of Day: {weather_info.time_of_day}")
15    weather = get_weather(city=weather_info.city, time_of_day=weather_info.timeOfDay)
16
17if __name__ == '__main__':
18    main()

Choosing multiple Tools

To choose ONE tool out of many, you can use a union:

BAML
1function UseTool(user_message: string) -> WeatherAPI | MyOtherAPI {
2  .... // same thing
3}
If you use VSCode Playground, you can see what we inject into the prompt, with full transparency.

Call the function like this:

1import asyncio
2from baml_client import b
3from baml_client.types import WeatherAPI, MyOtherAPI
4
5async def main():
6    tool = b.UseTool("What's the weather like in San Francisco?")
7    print(tool)
8    
9    if isinstance(tool, WeatherAPI):
10        print(f"Weather API called:")
11        print(f"City: {tool.city}")
12        print(f"Time of Day: {tool.timeOfDay}")
13    elif isinstance(tool, MyOtherAPI):
14        print(f"MyOtherAPI called:")
15        # Handle MyOtherAPI specific attributes here
16
17if __name__ == '__main__':
18    main()

Choosing N Tools

To choose many tools, you can use a union of a list:

BAML
1function UseTool(user_message: string) -> (WeatherAPI | MyOtherAPI)[] {
2  client "openai/gpt-4o-mini"
3  prompt #"
4    Given a message, extract info.
5    {# special macro to print the functions return type. #}
6    {{ ctx.output_format }}
7
8    {{ _.role('user') }}
9    {{ user_message }}
10  "#
11}

Call the function like this:

1import asyncio
2from baml_client import b
3from baml_client.types import WeatherAPI, MyOtherAPI
4
5async def main():
6    tools = b.UseTool("What's the weather like in San Francisco and New York?")
7    print(tools)  
8    
9    for tool in tools:
10        if isinstance(tool, WeatherAPI):
11            print(f"Weather API called:")
12            print(f"City: {tool.city}")
13            print(f"Time of Day: {tool.timeOfDay}")
14        elif isinstance(tool, MyOtherAPI):
15            print(f"MyOtherAPI called:")
16            # Handle MyOtherAPI specific attributes here
17
18if __name__ == '__main__':
19    main()

Dynamically Generate the tool signature

It might be cumbersome to define schemas in baml and code, so you can define them from code as well. Read more about dynamic types here

BAML
1class WeatherAPI {
2  @@dynamic // params defined from code
3}
4
5function UseTool(user_message: string) -> WeatherAPI {
6  client "openai/gpt-4o-mini"
7  prompt #"
8    Given a message, extract info.
9    {# special macro to print the functions return type. #}
10    {{ ctx.output_format }}
11
12    {{ _.role('user') }}
13    {{ user_message }}
14  "#
15}

Call the function like this:

Python
1import asyncio
2import inspect
3
4from baml_client import b
5from baml_client.type_builder import TypeBuilder
6from baml_client.types import WeatherAPI
7
8async def get_weather(city: str, time_of_day: str):
9    print(f"Getting weather for {city} at {time_of_day}")
10    return 42
11
12async def main():
13    tb = TypeBuilder()
14    type_map = {int: tb.int(), float: tb.float(), str: tb.string()}
15    signature = inspect.signature(get_weather)
16    for param_name, param in signature.parameters.items():
17        tb.WeatherAPI.add_property(param_name, type_map[param.annotation])
18    tool = b.UseTool("What's the weather like in San Francisco this afternoon?", { "tb": tb })
19    print(tool)
20    weather = await get_weather(**tool.model_dump())
21    print(weather)
22
23if __name__ == '__main__':
24    asyncio.run(main())
Note that the above approach is not fully generic. Recommended you read: Dynamic JSON Schema

Function-calling APIs vs Prompting

Injecting your function schemas into the prompt, as BAML does, outperforms function-calling across all benchmarks for major providers (see our Berkeley FC Benchmark results with BAML).

Amongst other limitations, function-calling APIs will at times:

  1. Return a schema when you don’t want any (you want an error)
  2. Not work for tools with more than 100 parameters.
  3. Use many more tokens than prompting.

Keep in mind that “JSON mode” is nearly the same thing as “prompting”, but it enforces the LLM response is ONLY a JSON blob. BAML does not use JSON mode since it allows developers to use better prompting techniques like chain-of-thought, to allow the LLM to express its reasoning before printing out the actual schema. BAML’s parser can find the json schema(s) out of free-form text for you. Read more about different approaches to structured generation here

BAML will still support native function-calling APIs in the future (please let us know more about your use-case so we can prioritize accordingly)

Built with