test | Boundary Documentation

Tests are first-class citizens in BAML, designed to make testing AI functions straightforward and robust. BAML tests can be written anywhere in your codebase and run with minimal setup.

Overview

A BAML test consists of:

Test name and metadata
Functions under test
Input arguments
Optional testing configuration
Optional assertions
Optional type builders

test TestName {
    functions [FunctionName]
    args {
        paramName "value"
    }
}

Test Declaration

Basic Syntax

test name {
    functions [function_list]
    args {
        parameter_assignments
    }
}

Optional Features

test name {
    functions [function_list]
    type_builder {
        class NewType {
            // Props
        }
        dynamic class ExistingDynamicType {
            new_prop NewType
            // Inject Props Here
        }
    }
    args {
        parameter_assignments
    }
    @@check( check_length, {{ this.prop|length > 0 }} )
    @@assert( {{ this.prop|length < 255 }})
}

Components

name: Test identifier (unique per function)
functions: List of functions to test
args: Input parameters for the test case
type_builder: Block used to inject values into dynamic types
@@check: Conditional check for test validity
@@assert: Assertion for test result

Input Types

Basic Types

Simple values are provided directly:

test SimpleTest {
    functions [ClassifyMessage]
    args {
        input "Can't access my account"
    }
}

Complex Objects

Objects are specified using nested structures:

test ComplexTest {
    functions [ProcessMessage]
    args {
        message {
            user "john_doe"
            content "Hello world"
            metadata {
                timestamp 1234567890
                priority "high"
            }
        }
    }
}

Arrays

Arrays use bracket notation:

test ArrayTest {
    functions [BatchProcess]
    args {
        messages [
            {
                user "user1"
                content "Message 1"
            }
            {
                user "user2"
                content "Message 2"
            }
        ]
    }
}

Media Inputs

Images

Images can be specified using three methods:

File Reference

test ImageFileTest {
    functions [AnalyzeImage]
    args {
        param {
            file "../images/test.png"
        }
    }
}

URL Reference

test ImageUrlTest {
    functions [AnalyzeImage]
    args {
        param {
            url "https://example.com/image.jpg"
        }
    }
}

Base64 Data

test ImageBase64Test {
    functions [AnalyzeImage]
    args {
        param {
            base64 "a41f..."
            media_type "image/png"
        }
    }
}

Audio

Similar to images, audio can be specified in three ways:

File Reference

test AudioFileTest {
    functions [TranscribeAudio]
    args {
        audio {
            file "../audio/sample.mp3"
        }
    }
}

URL Reference

test AudioUrlTest {
    functions [TranscribeAudio]
    args {
        audio {
            url "https://example.com/audio.mp3"
        }
    }
}

Base64 Data

test AudioBase64Test {
    functions [TranscribeAudio]
    args {
        audio {
            base64 "..."
            media_type "audio/mp3"
            }
  }
}

Pdfs

Unlike images and audio, Pdfs cannot be supplied via URL. They must be provided either as a local file reference or as Base64 data.

File Reference

test PdfFileTest {
    functions [AnalyzePdf]
    args {
        pdf {
            file "../documents/report.pdf"
        }
    }
}

Base64 Data

test PdfBase64Test {
    functions [AnalyzePdf]
    args {
        pdf {
            base64 "JVBERi0K..."
            media_type "application/pdf"
        }
    }
}

Videos

Similar to other media types, videos can be specified in three ways:

File Reference

test VideoFileTest {
    functions [AnalyzeVideo]
    args {
        video {
            file "../videos/sample.mp4"
        }
    }
}

URL Reference

test VideoUrlTest {
    functions [AnalyzeVideo]
    args {
        video {
            url "https://example.com/video.mp4"
        }
    }
}

Base64 Data

test VideoBase64Test {
    functions [AnalyzeVideo]
    args {
        video {
            base64 "AAAAGGZ0eXBpc29t..."
            media_type "video/mp4"
        }
    }
}

Multi-line Strings

For long text inputs, use the block string syntax:

test LongTextTest {
    functions [AnalyzeText]
    args {
        content #"
            This is a multi-line
            text input that preserves
            formatting and whitespace
        "#
    }
}

Template Strings

Share string data between tests by putting it into a template_string, then invoke the template_string in the test args:

function Echo(input: string) -> string {
  client GPT4o
  prompt #"
    Echo the input string back to me verbatim:
    {{ input }}
  "#
}
template_string EchoTestData(arg: int) ##"
  The number is {{ arg }}
"##
test TemplateTest {
  functions [Echo]
  args {
   input: EchoTestData()
  }
  @@assert( {{ this == "The number is 43" }} )
}

Testing Multiple Functions

This requires each function to have the exact same parameters:

test EndToEndFlow {
    functions [
        ExtractInfo
        ProcessInfo
        ValidateResult
    ]
    args {
        input "test data"
    }
}

Testing Dynamic Types

Dynamic types can be tested using type_builder and dynamic blocks:

class DynamicClass {
    static_prop string
    @@dynamic
}
function ReturnDynamicClass(input: string) -> DynamicClass {
    // ...
}
test DynamicClassTest {
    functions [ReturnDynamicClass]
    type_builder {
        dynamic class DynamicClass {
            new_prop_here string
        }
    }
    args {
        input "test data"
    }
}

Integration with Development Tools

VSCode Integration

Tests can be run directly from the BAML playground
Real-time syntax validation
Test result visualization