BAML Basics

AbortSignal / Timeouts

Overview

Abort controllers allow you to cancel ongoing LLM operations, which is essential for:

  • User-initiated cancellations (e.g., “Stop generating” buttons)
  • Implementing timeouts for long-running operations
  • Cleaning up resources when components unmount
  • Managing multiple parallel requests

Quick Start

import { b } from '../../baml_client'
// TypeScript uses AbortSignal for cancellation
// No additional imports needed - it's built into the runtime
// Modern approach: Use AbortSignal.timeout() for automatic timeout
try {
  const result = await b.ExtractResume(text, {
    signal: AbortSignal.timeout(5000) // 5 second timeout
  })
} catch (error) {
  if (error.name === 'BamlAbortError') {
    console.log('Operation was cancelled')
  }
}
// Manual approach: Create controller and cancel later
const controller = new AbortController()
const promise = b.ExtractResume(text, {
  signal: controller.signal
})
// Cancel after 5 seconds
setTimeout(() => controller.abort(), 5000)
try {
  const result = await promise
} catch (error) {
  if (error.name === 'BamlAbortError') {
    console.log('Operation was cancelled')
  }
}

Basic Examples

Implementing Timeouts

Automatically cancel operations that take too long:

// Modern approach using AbortSignal.timeout()
async function extractWithTimeout(text: string, timeoutMs: number = 30000) {
  try {
    const result = await b.ExtractResume(text, {
      signal: AbortSignal.timeout(timeoutMs)
    })
    return result
  } catch (error) {
    if (error.name === 'BamlAbortError') {
      throw new Error(`Operation timed out after ${timeoutMs}ms`)
    }
    throw error
  }
}
// Manual implementation (for when you need more control)
async function extractWithManualTimeout(text: string, timeoutMs: number = 30000) {
  const controller = new AbortController()
  
  // Set up automatic timeout
  const timeoutId = setTimeout(() => {
    controller.abort('timeout')
  }, timeoutMs)
  try {
    const result = await b.ExtractResume(text, {
      signal: controller.signal
    })
    clearTimeout(timeoutId)
    return result
  } catch (error) {
    clearTimeout(timeoutId)
    if (error.name === 'BamlAbortError') {
      throw new Error(`Operation timed out after ${timeoutMs}ms`)
    }
    throw error
  }
}

User-Initiated Cancellation

Build responsive backend services that allow users to cancel long-running operations:

import express from 'express'
import { b } from '../../baml_client'
const app = express()
const activeControllers = new Map<string, AbortController>()
app.post('/extract/:requestId', async (req, res) => {
  const { requestId } = req.params
  const { text } = req.body
  
  const controller = new AbortController()
  activeControllers.set(requestId, controller)
  
  try {
    const result = await b.ExtractResume(text, {
      signal: controller.signal
    })
    res.json({ result })
  } catch (error) {
    if (error.name === 'BamlAbortError') {
      res.json({ status: 'cancelled' })
    } else {
      res.status(500).json({ error: error.message })
    }
  } finally {
    activeControllers.delete(requestId)
  }
})
app.post('/cancel/:requestId', (req, res) => {
  const { requestId } = req.params
  const controller = activeControllers.get(requestId)
  
  if (controller) {
    controller.abort()
    res.json({ status: 'cancellation requested' })
  } else {
    res.status(404).json({ status: 'request not found' })
  }
})

Streaming with Abort Controllers

Abort controllers work seamlessly with streaming responses:

const controller = new AbortController()
const stream = b.stream.GenerateStory(prompt, {
  signal: controller.signal
})
let wordCount = 0
try {
  for await (const chunk of stream) {
    wordCount += chunk.split(' ').length
    
    // Stop if we've generated enough
    if (wordCount > 1000) {
      controller.abort('word limit reached')
      break
    }
    
    // Process chunk
    console.log(chunk)
  }
} catch (error) {
  if (error instanceof BamlAbortError) {
    console.log('Stream cancelled:', error.reason)
  }
}

Error Handling

Properly handle abort errors to distinguish cancellations from other failures:

import { BamlAbortError } from '../../baml_client'
try {
  const result = await b.ExtractResume(text, {
    signal: controller.signal
  })
  return { success: true, data: result }
} catch (error) {
  if (error instanceof BamlAbortError) {
    // User cancelled - this is expected
    return { success: false, cancelled: true }
  }
  
  if (error.name === 'BamlValidationError') {
    // Schema validation failed
    return { success: false, validationError: error.message }
  }
  
  // Unexpected error
  console.error('Extraction failed:', error)
  throw error
}

Best Practices

When to Use Each Pattern

// ✅ Use AbortSignal.timeout() for simple timeouts
const result = await b.ExtractResume(text, {
  signal: AbortSignal.timeout(30000)
})
// ✅ Use manual AbortController when you need to cancel conditionally
const controller = new AbortController()
const promise = b.ExtractResume(text, {
  signal: controller.signal
})
// Cancel based on user action or business logic
if (shouldCancel) {
  controller.abort('cancelled by user')
}
// ✅ Combine both patterns for timeout + manual control
const controller = new AbortController()
const timeoutId = setTimeout(() => controller.abort('timeout'), 30000)
const result = await b.ExtractResume(text, {
  signal: controller.signal
})
clearTimeout(timeoutId)

Key Benefits

  • AbortSignal.timeout(): Cleaner code for simple timeout scenarios
  • Manual AbortController: More control over cancellation logic and reasons
  • Better Error Handling: Clear distinction between timeouts and user cancellations
  • Standards Compliance: Uses modern web standards that work across different environments

Advanced Patterns

For more advanced abort controller patterns including:

  • Cancelling parallel operations - Cancel multiple concurrent calls at once or individually
  • Fastest request wins - Race multiple LLM providers and cancel slower ones
  • Implementing timeouts for parallel operations - Set automatic timeouts for batches of operations
  • Batching with cancellation support - Process items in batches with cancellation

See the Concurrent Calls guide for detailed examples and implementations.