> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.boundaryml.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.boundaryml.com/_mcp/server.

The `test` command runs BAML function tests defined in your BAML files. It provides comprehensive testing capabilities including filtering, parallel execution, and various output formats.

## Usage

```
baml-cli test [OPTIONS]
```

## Options

| Option                    | Description                                                      | Default       |
| ------------------------- | ---------------------------------------------------------------- | ------------- |
| `--from <PATH>`           | Path to the `baml_src` directory                                 | `./baml_src`  |
| `--list`                  | Only list selected tests without running them                    | `false`       |
| `-i, --include <PATTERN>` | Include specific functions or tests (can be used multiple times) | Run all tests |
| `-x, --exclude <PATTERN>` | Exclude specific functions or tests (can be used multiple times) | None          |
| `--parallel <NUM>`        | Number of tests to run in parallel                               | `10`          |
| `--pass-if-no-tests`      | Pass if no tests are selected                                    | `false`       |
| `--require-human-eval`    | Fail if any tests need human evaluation                          | `true`        |
| `--dotenv`                | Load environment variables from .env file                        | `true`        |
| `--dotenv-path <PATH>`    | Path to custom .env file                                         | None          |

## Description

The `test` command performs the following actions:

1. Discovers and parses all test cases defined in BAML files
2. Applies include/exclude filters to select which tests to run
3. Executes tests in parallel (configurable concurrency)
4. Reports results with detailed output and assertions
5. Supports various output formats and CI integration

## Test Filtering

The `--include` and `--exclude` options support powerful pattern matching:

### Pattern Syntax

* `*` matches any characters within a name
* `FunctionName::TestName` matches a specific test in a specific function
* `FunctionName::` matches all tests in a function
* `::TestName` matches a test name across all functions
* Multiple patterns can be combined

### Examples

```bash
# Run all tests
baml-cli test

# List all available tests
baml-cli test --list

# Run tests for a specific function
baml-cli test -i "ExtractResume::"

# Run a specific test
baml-cli test -i "ExtractResume::TestBasicResume"

# Run all tests matching a pattern
baml-cli test -i "Extract*::"

# Run tests with multiple include patterns
baml-cli test -i "Extract*::" -i "Classify*::"

# Exclude specific tests
baml-cli test -x "ExtractResume::TestComplexResume"

# Combine include and exclude (exclude takes precedence)
baml-cli test -i "Extract*::" -x "*::TestSlow*"
```

## Parallel Execution

Control the number of tests running simultaneously:

```bash
# Run tests with default parallelism (10)
baml-cli test

# Run tests sequentially
baml-cli test --parallel 1

# Run with high parallelism
baml-cli test --parallel 20
```

## Environment Variables

The test command automatically loads environment variables:

```bash
# Use default .env file loading
baml-cli test

# Disable .env file loading
baml-cli test --no-dotenv

# Use custom .env file
baml-cli test --dotenv-path .env.test
```

Environment variables can also be set directly:

```bash
# Set API keys for testing
OPENAI_API_KEY=... ANTHROPIC_API_KEY=... baml-cli test
```

## Exit Codes

The `test` command returns different exit codes based on results:

| Exit Code | Meaning                        |
| --------- | ------------------------------ |
| `0`       | All tests passed               |
| `1`       | Test failures occurred         |
| `2`       | Tests require human evaluation |
| `3`       | Test execution was cancelled   |
| `4`       | No tests were found to run     |

## Examples

### Basic Testing

```bash
# Run all tests in the project
baml-cli test

# Run tests from a specific directory
baml-cli test --from /path/to/my/baml_src
```

### Development Workflow

```bash
# Run tests for a function you're developing
baml-cli test -i "MyNewFunction::"

# Run specific test while debugging
baml-cli test -i "MyFunction::TestEdgeCase"

# List tests to see what's available
baml-cli test --list -i "Extract*::"
```

### CI/CD Integration

```bash
# Fail fast on first assertion failure
baml-cli test --require-human-eval

# Allow tests that need human evaluation to pass
baml-cli test --no-require-human-eval

# Run with controlled parallelism for CI
baml-cli test --parallel 5
```

### Test Discovery

```bash
# See all available tests
baml-cli test --list

# See tests for specific functions
baml-cli test --list -i "ClassifyMessage::"

# See what tests would run with filters
baml-cli test --list -i "Extract*::" -x "*::TestSlow*"
```

## Test Definition

Tests are defined in BAML files using the `test` block syntax:

```baml
function ExtractResume(resume: string) -> Resume {
  client GPT4
  prompt #"Extract resume information: {{resume}}"#
}

test TestBasicResume {
  functions [ExtractResume]
  args {
    resume "John Doe\njohn@example.com\nSoftware Engineer"
  }
  @@assert({{ this.name == "John Doe" }})
  @@assert({{ this.email == "john@example.com" }})
}
```

## Related Commands

* [`baml dev`](./dev) - Development server with hot reload for interactive testing
* [`baml serve`](./serve) - Production server for HTTP API testing
* [`baml generate`](./generate) - Generate client code before running tests