Dynamic Types - TypeBuilder

Sometimes you have output schemas that change at runtime — for example if you have a list of Categories that you need to classify that come from a database, or your schema is user-provided.

TypeBuilder is used to create or modify dynamic types at runtime to achieve this.

Dynamic BAML Enums

Imagine we want to make a categorizer prompt, but the list of categories to output come from a database.

1. Add @@dynamic to the class or enum definition to mark it as dynamic in BAML.

1
enum Category {
2
  VALUE1 // normal static enum values that don't change
3
  VALUE2
4
  @@dynamic // this enum can have more values added at runtime
5
}
6
7
// The Category enum can now be modified at runtime!
8
function DynamicCategorizer(input: string) -> Category {
9
  client GPT4
10
  prompt #"
11
    Given a string, classify it into a category
12
    {{ input }}
13
14
    {{ ctx.output_format }}
15
  "#
16
}

2. Import the TypeBuilder from baml_client in your runtime code and modify Category. All dynamic types you define in BAML will be available as properties of TypeBuilder. Think of the typebuilder as a registry of modified runtime types that the baml function will read from when building the output schema in the prompt.

1
from baml_client.type_builder import TypeBuilder
2
from baml_client import b
3
4
async def run():
5
  tb = TypeBuilder()
6
  tb.Category.add_value('VALUE3')
7
  tb.Category.add_value('VALUE4')
8
  # Pass the typebuilder in the baml_options argument -- the last argument of the function.
9
  res = await b.DynamicCategorizer("some input", { "tb": tb })
10
  # Now res can be VALUE1, VALUE2, VALUE3, or VALUE4
11
  print(res)

Dynamic BAML Classes

Now we’ll add some properties to a User class at runtime using @@dynamic.

1
class User {
2
  name string
3
  age int
4
  @@dynamic
5
}
6
7
function DynamicUserCreator(user_info: string) -> User {
8
  client GPT4
9
  prompt #"
10
    Extract the information from this chunk of text:
11
    "{{ user_info }}"
12
13
    {{ ctx.output_format }}
14
  "#
15
}

We can then modify the User schema at runtime. Since we marked User with @@dynamic, it’ll be available as a property of TypeBuilder.

1
from baml_client.type_builder import TypeBuilder
2
from baml_client import b
3
4
async def run():
5
  tb = TypeBuilder()
6
  tb.User.add_property('email', tb.string())
7
  tb.User.add_property('address', tb.string()).description("The user's address")
8
  res = await b.DynamicUserCreator("some user info", { "tb": tb })
9
  # Now res can have email and address fields
10
  print(res)

Creating new dynamic classes or enums not in BAML

The previous examples showed how to modify existing types. Here we create a new Hobbies enum, and a new class called Address without having them defined in BAML.

Note that you must attach the new types to the existing Return Type of your BAML function(in this case it’s User).

1
from baml_client.type_builder import TypeBuilder
2
from baml_client.async_client import b
3
4
async def run():
5
  tb = TypeBuilder()
6
  hobbies_enum = tb.add_enum("Hobbies")
7
  hobbies_enum.add_value("Soccer")
8
  hobbies_enum.add_value("Reading")
9
10
  address_class = tb.add_class("Address")
11
  address_class.add_property("street", tb.string()).description("The user's street address")
12
13
  tb.User.add_property("hobby", hobbies_enum.type().optional())
14
  tb.User.add_property("address", address_class.type().optional())
15
  res = await b.DynamicUserCreator("some user info", {"tb": tb})
16
  # Now res might have the hobby property, which can be Soccer or Reading
17
  print(res)

TypeBuilder provides methods for building different kinds of types:

Adding descriptions to dynamic types

1
tb = TypeBuilder()
2
tb.User.add_property("email", tb.string()).description("The user's email")

Creating dynamic classes and enums at runtime with BAML

The TypeBuilder has a higher level API for creating dynamic types at runtime. Here’s an example:

1
tb = TypeBuilder()
2
tb.add_baml("""
3
  // Creates a new class Address that does not exist in the BAML source.
4
  class Address {
5
    street string
6
    city string
7
    state string
8
  }
9
10
  // Modifies the existing @@dynamic User class to add the new address property.
11
  dynamic class User {
12
    address Address
13
  }
14
15
  // Modifies the existing @@dynamic Category enum to add a new variant.
16
  dynmic enum Category {
17
    VALUE5
18
  }
19
""")

Building dynamic types from JSON schema

We have a working implementation of this, but are waiting for a concrete use case to merge it. Please chime in on the GitHub issue if this is something you’d like to use.

1
import pydantic
2
from baml_client import b
3
4
class Person(pydantic.BaseModel):
5
    last_name: list[str]
6
    height: Optional[float] = pydantic.Field(description="Height in meters")
7
8
tb = TypeBuilder()
9
tb.unstable_features.add_json_schema(Person.model_json_schema())
10
11
res = await b.ExtractPeople(
12
    "My name is Harrison. My hair is black and I'm 6 feet tall. I'm pretty good around the hoop. I like giraffes.",
13
    {"tb": tb},
14
)

Testing dynamic types in BAML

When testing dynamic types there are two different cases:

1. Injecting properties into dynamic types returned by the tested function.
2. Injecting values into dynamic types received as arguments by the tested function.

The first case requires using the type_builder and dynamic blocks in the test, whereas the second case only requires specifying the values in the args block.

Testing return types

Dynamic classes

Suppose we have a dynamic class Resume and we want to add a property that stores the user’s work experience when we testing a specific function. We can do that by specifying the types and properties that we need in the type_builder block.

1
class Resume {
2
  name string
3
  skills string[]
4
  @@dynamic // Marked as @@dynamic.
5
}
6
7
// Function that returns a dynamic class.
8
function ExtractResume(from_text: string) -> Resume {
9
  // Prompt
10
}
11
12
test ReturnDynamicClassTest {
13
  functions [ExtractResume]
14
  type_builder {
15
    // Defines a new type available only within this test block.
16
    class Experience {
17
      title string
18
      company string
19
      start_date string
20
      end_date string
21
    }
22
23
    // Injects new properties into the `@@dynamic` part of the Resume class.
24
    dynamic class Resume {
25
      experience Experience[]
26
    }
27
  }
28
  args {
29
    from_text #"
30
      John Doe
31
32
      Experience
33
      - Software Engineer, Boundary, Sep 2022 - Sep 2023
34
35
      Skills
36
      - Python
37
      - Java
38
    "#
39
  }
40
}

The rendered prompt for ExtractResume will now include the experience field defined in the dynamic block and the LLM will correctly extract the experience in the input text.

Dynamic enums

Dynamic enums can be included in the type_builder block just like classes. The only difference is that we inject new variants in the dynamic block instead of properties.

1
enum Category {
2
  Refund
3
  CancelOrder
4
  TechnicalSupport
5
  AccountIssue
6
  Question
7
  @@dynamic // Marked as @@dynamic.
8
}
9
10
// Function that returns a dynamic enum.
11
function ClassifyMessage(message: string) -> Category {
12
  // Prompt
13
}
14
15
test ReturnDynamicEnumTest {
16
  functions [ClassifyMessage]
17
  type_builder {
18
    // Injects new variants into the `@@dynamic` part of the Category enum.
19
    dynamic enum Category {
20
      Feedback
21
    }
22
  }
23
  args {
24
	  message "I think the product is great!"
25
  }
26
}

The Feedback variant will be rendered in the prompt for ClassifyMessage during the test execution.

Testing parameter types

When a dynamic type is used as an input parameter of a function, we can simply pass any value in the args block of the test and the value will be rendered in the prompt.

Dynamic classes

1
class Resume {
2
  name string
3
  skills string[]
4
  @@dynamic // Marked as @@dynamic.
5
}
6
7
function WriteResume(resume: Resume) -> string {
8
  // Prompt
9
}
10
11
test DynamicClassAsInputTest {
12
  functions [WriteResume]
13
  args {
14
    resume {
15
      name "John Doe"
16
      skills ["C++", "Java"]
17
      experience [
18
        {
19
          title "Software Engineer"
20
          company "Boundary"
21
          start_date "2023-09-01"
22
          end_date "2024-09-01"
23
        }
24
      ]
25
    }
26
  }
27
}

Dynamic enums

Enums work the same way, any variant defined in the args block will be rendered normally.

1
enum Category {
2
  Refund
3
  CancelOrder
4
  TechnicalSupport
5
  AccountIssue
6
  Question
7
  @@dynamic // Marked as @@dynamic.
8
}
9
10
function WriteCustomerMessage(category: Category) -> string {
11
  // Prompt
12
}
13
14
test DynamicEnumAsInputTest {
15
  functions [WriteCustomerMessage]
16
  args {
17
    category Feedback // The enum is dynamic so it accepts a new variant.
18
  }
19
}

For more information about dynamic types, see Type Builder.