Function Calling / Tools

“Function calling” is a technique for getting an LLM to choose a function to call for you.

The way it works is:

1. You define a task with certain function(s)
2. Ask the LLM to choose which function to call
3. Get the function parameters from the LLM for the appropriate function it choose
4. Call the functions in your code with those parameters

In BAML, you can get represent a tool or a function you want to call as a BAML class, and make the function output be that class definition.

1
class WeatherAPI {
2
  city string @description("the user's city")
3
  timeOfDay string @description("As an ISO8601 timestamp")
4
}
5
6
function UseTool(user_message: string) -> WeatherAPI {
7
  client GPT4Turbo
8
  prompt #"
9
    Extract the info from this message
10
    ---
11
    {{ user_message }}
12
    ---
13
14
    {# special macro to print the output schema. #}
15
    {{ ctx.output_format }}
16
17
    JSON:
18
  "#
19
}

Choosing multiple Tools

To choose ONE tool out of many, you can use a union:

1
function UseTool(user_message: string) -> WeatherAPI | MyOtherAPI {
2
  .... // same thing
3
}

Choosing N Tools

To choose many tools, you can use a union of a list:

1
function UseTool(user_message: string) -> (WeatherAPI | MyOtherAPI)[] {
2
  .... // same thing
3
}

Function-calling APIs vs Prompting

Injecting your function schemas into the prompt, as BAML does, outperforms function-calling across all benchmarks for major providers (see Berkeley’s Function-calling Leaderboard, where “Prompt” outperforms “FC”).

Keep in mind that “JSON mode” is nearly the same thing as “prompting”, but it enforces the LLM response is ONLY a JSON blob. BAML does not use JSON mode since it allows developers to use better prompting techniques like chain-of-thought, to allow the LLM to express its reasoning before printing out the actual schema. BAML’s parser can find the json schema(s) out of free-form text for you.

BAML may support native function-calling APIs in the future (please let us know more about your use-case so we can prioritize accordingly)