test

Tests are first-class citizens in BAML, designed to make testing AI functions straightforward and robust. BAML tests can be written anywhere in your codebase and run with minimal setup.

Overview

A BAML test consists of:

  • Test name and metadata
  • Functions under test
  • Input arguments
  • Optional testing configuration
  • Optional assertions
  • Optional type builders
1test TestName {
2    functions [FunctionName]
3    args {
4        paramName "value"
5    }
6}

Test Declaration

Basic Syntax

1test name {
2    functions [function_list]
3    args {
4        parameter_assignments
5    }
6}

Optional Features

1test name {
2    functions [function_list]
3    type_builder {
4        class NewType {
5            // Props
6        }
7        dynamic class ExistingDynamicType {
8            new_prop NewType
9            // Inject Props Here
10        }
11    }
12    args {
13        parameter_assignments
14    }
15    @@check( check_length, {{ this.prop|length > 0 }} )
16    @@assert( {{ this.prop|length < 255 }})
17}

Components

  • name: Test identifier (unique per function)
  • functions: List of functions to test
  • args: Input parameters for the test case
  • type_builder: Block used to inject values into dynamic types
  • @@check: Conditional check for test validity
  • @@assert: Assertion for test result

Input Types

Basic Types

Simple values are provided directly:

1test SimpleTest {
2    functions [ClassifyMessage]
3    args {
4        input "Can't access my account"
5    }
6}

Complex Objects

Objects are specified using nested structures:

1test ComplexTest {
2    functions [ProcessMessage]
3    args {
4        message {
5            user "john_doe"
6            content "Hello world"
7            metadata {
8                timestamp 1234567890
9                priority "high"
10            }
11        }
12    }
13}

Arrays

Arrays use bracket notation:

1test ArrayTest {
2    functions [BatchProcess]
3    args {
4        messages [
5            {
6                user "user1"
7                content "Message 1"
8            }
9            {
10                user "user2"
11                content "Message 2"
12            }
13        ]
14    }
15}

Media Inputs

Images

Images can be specified using three methods:

  1. File Reference
1test ImageFileTest {
2    functions [AnalyzeImage]
3    args {
4        param {
5            file "../images/test.png"
6        }
7    }
8}
  1. URL Reference
1test ImageUrlTest {
2    functions [AnalyzeImage]
3    args {
4        param {
5            url "https://example.com/image.jpg"
6        }
7    }
8}
  1. Base64 Data
1test ImageBase64Test {
2    functions [AnalyzeImage]
3    args {
4        param {
5            base64 "a41f..."
6            media_type "image/png"
7        }
8    }
9}

Audio

Similar to images, audio can be specified in three ways:

  1. File Reference
1test AudioFileTest {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            file "../audio/sample.mp3"
6        }
7    }
8}
  1. URL Reference
1test AudioUrlTest {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            url "https://example.com/audio.mp3"
6        }
7    }
8}
  1. Base64 Data
1test AudioBase64Test {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            base64 "..."
6            media_type "audio/mp3"
7        }
8    }
9}

Multi-line Strings

For long text inputs, use the block string syntax:

1test LongTextTest {
2    functions [AnalyzeText]
3    args {
4        content #"
5            This is a multi-line
6            text input that preserves
7            formatting and whitespace
8        "#
9    }
10}

Testing Multiple Functions

This requires each function to have the exact same parameters:

1test EndToEndFlow {
2    functions [
3        ExtractInfo
4        ProcessInfo
5        ValidateResult
6    ]
7    args {
8        input "test data"
9    }
10}

Testing Dynamic Types

Dynamic types can be tested using type_builder and dynamic blocks:

1class DynamicClass {
2    static_prop string
3    @@dynamic
4}
5
6function ReturnDynamicClass(input: string) -> DynamicClass {
7    // ...
8}
9
10test DynamicClassTest {
11    functions [ReturnDynamicClass]
12    type_builder {
13        dynamic class DynamicClass {
14            new_prop_here string
15        }
16    }
17    args {
18        input "test data"
19    }
20}

Integration with Development Tools

VSCode Integration

  • Tests can be run directly from the BAML playground
  • Real-time syntax validation
  • Test result visualization
Built with