test

Tests are first-class citizens in BAML, designed to make testing AI functions straightforward and robust. BAML tests can be written anywhere in your codebase and run with minimal setup.

Overview

A BAML test consists of:

  • Test name and metadata
  • Functions under test
  • Input arguments
  • Optional testing configuration
  • Optional assertions
1test TestName {
2    functions [FunctionName]
3    args {
4        paramName "value"
5    }
6}

Test Declaration

Syntax

1test name {
2    functions [function_list]
3    args {
4        parameter_assignments
5    }
6}

Components

  • name: Test identifier (unique per function)
  • functions: List of functions to test
  • args: Input parameters for the test case

Input Types

Basic Types

Simple values are provided directly:

1test SimpleTest {
2    functions [ClassifyMessage]
3    args {
4        input "Can't access my account"
5    }
6}

Complex Objects

Objects are specified using nested structures:

1test ComplexTest {
2    functions [ProcessMessage]
3    args {
4        message {
5            user "john_doe"
6            content "Hello world"
7            metadata {
8                timestamp 1234567890
9                priority "high"
10            }
11        }
12    }
13}

Arrays

Arrays use bracket notation:

1test ArrayTest {
2    functions [BatchProcess]
3    args {
4        messages [
5            {
6                user "user1"
7                content "Message 1"
8            }
9            {
10                user "user2"
11                content "Message 2"
12            }
13        ]
14    }
15}

Media Inputs

Images

Images can be specified using three methods:

  1. File Reference
1test ImageFileTest {
2    functions [AnalyzeImage]
3    args {
4        param {
5            file "../images/test.png"
6        }
7    }
8}
  1. URL Reference
1test ImageUrlTest {
2    functions [AnalyzeImage]
3    args {
4        param {
5            url "https://example.com/image.jpg"
6        }
7    }
8}
  1. Base64 Data
1test ImageBase64Test {
2    functions [AnalyzeImage]
3    args {
4        param {
5            base64 "a41f..."
6            media_type "image/png"
7        }
8    }
9}

Audio

Similar to images, audio can be specified in three ways:

  1. File Reference
1test AudioFileTest {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            file "../audio/sample.mp3"
6        }
7    }
8}
  1. URL Reference
1test AudioUrlTest {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            url "https://example.com/audio.mp3"
6        }
7    }
8}
  1. Base64 Data
1test AudioBase64Test {
2    functions [TranscribeAudio]
3    args {
4        audio {
5            base64 "..."
6            media_type "audio/mp3"
7        }
8    }
9}

Multi-line Strings

For long text inputs, use the block string syntax:

1test LongTextTest {
2    functions [AnalyzeText]
3    args {
4        content #"
5            This is a multi-line
6            text input that preserves
7            formatting and whitespace
8        "#
9    }
10}

Testing Multiple Functions

This requires each function to have teh exact same parameters:

1test EndToEndFlow {
2    functions [
3        ExtractInfo
4        ProcessInfo
5        ValidateResult
6    ]
7    args {
8        input "test data"
9    }
10}

Integration with Development Tools

VSCode Integration

  • Tests can be run directly from the BAML playground
  • Real-time syntax validation
  • Test result visualization