aws-bedrock

The aws-bedrock provider supports all text-output models available via the Converse API.

Quick Start

1
client<llm> MyClient {
2
  provider aws-bedrock
3
  options {
4
    model "anthropic.claude-3-sonnet-20240229-v1:0"
5
    inference_configuration {
6
      max_tokens 100
7
      temperature 0.7
8
    }
9
    // Pass any other parameters that are model specific, 
10
    // like with claude thinking models.
11
    additional_model_request_fields {
12
      thinking {
13
        type "enabled"
14
        budget_tokens 1030
15
      }
16
    }
17
  }
18
}

Authentication

AWS Bedrock uses standard AWS authentication methods. We recommend using AWS profiles in development and AWS services’ IAM roles in production, but all of the following are supported:

1
[default]
2
sso_start_url = https://your-sso-start-url.awsapps.com/start
3
sso_region = us-west-2
4
sso_account_id = 123456789012
5
sso_role_name = YourSSORole
6
region = us-west-2
7
output = json

$
export AWS_PROFILE=staging-profile

$
# First, login with SSO
$
aws sso login --profile staging-profile
$
$
# Then use the profile in your BAML config
$
client<llm> MyClient {
>
  provider aws-bedrock
>
  options {
>
    profile "staging-profile"
>
    model "anthropic.claude-3-sonnet-20240229-v1:0"
>
  }
>
}

Credential Resolution

BAML follows a specific order when resolving AWS credentials:

1. Explicit BAML Configuration

   BAML

   1
   client<llm> MyClient {
   2
     provider aws-bedrock
   3
     options {
   4
       access_key_id env.MY_ACCESS_KEY      // Highest precedence
   5
       secret_access_key env.MY_SECRET_KEY
   6
       region "us-east-1"
   7
     }
   8
   }

2. Environment Variables

   $
   AWS_ACCESS_KEY_ID
   $
   AWS_SECRET_ACCESS_KEY
   $
   AWS_SESSION_TOKEN    # Optional
   $
   AWS_REGION
   $
   AWS_PROFILE

3. AWS Configuration Files

   1
   # ~/.aws/credentials
   2
   [default]
   3
   aws_access_key_id = ...
   4
   aws_secret_access_key = ...
   5
   6
   # ~/.aws/config
   7
   [default]
   8
   region = us-east-1

4. Instance Metadata (EC2/ECS only)

   • IAM Role credentials
   • Instance profile credentials

Important Rules

1. All or Nothing

   • If you provide any credential explicitly, you must provide all required credentials
   • This won’t work:
     BAML

     1
     client<llm> MyClient {
     2
       provider aws-bedrock
     3
       options {
     4
         access_key_id env.AWS_ACCESS_KEY_ID
     5
         // Error: secret_access_key is required when access_key_id is provided
     6
         model "anthropic.claude-3-sonnet-20240229-v1:0"
     7
       }
     8
     }

2. Session Token Requirements

   • When using session_token, you must provide all three:
     • access_key_id
     • secret_access_key
     • session_token

3. Profile Exclusivity

   • When using profile, you cannot specify other credentials:
     BAML

     1
     client<llm> MyClient {
     2
       provider aws-bedrock
     3
       options {
     4
         profile "my-profile"
     5
         access_key_id env.AWS_ACCESS_KEY_ID  // Error: Cannot mix profile with explicit credentials
     6
         model "anthropic.claude-3-sonnet-20240229-v1:0"
     7
       }
     8
     }

4. Environment Variable Override

   • Explicit values in BAML always override environment variables:
     BAML

     1
     client<llm> MyClient {
     2
       provider aws-bedrock
     3
       options {
     4
         access_key_id "AKIAXXXXXXXX"  // This will be used even if AWS_ACCESS_KEY_ID exists
     5
         secret_access_key env.AWS_SECRET_ACCESS_KEY
     6
         model "anthropic.claude-3-sonnet-20240229-v1:0"
     7
       }
     8
     }

5. AWS Lambda/ECS/EC2

   • In AWS services, credentials are automatically provided by the runtime
   • Explicitly provided credentials will override the automatic ones
   • Best practice: Don’t specify credentials in AWS environments, use IAM roles instead

Using Custom Environment Variables

You can map your own environment variable names:

1
client<llm> MyClient {
2
  provider aws-bedrock
3
  options {
4
    access_key_id env.MY_CUSTOM_AWS_KEY_ID
5
    secret_access_key env.MY_CUSTOM_AWS_SECRET
6
    session_token env.MY_CUSTOM_AWS_SESSION  // Optional
7
    region env.MY_CUSTOM_AWS_REGION
8
    model "anthropic.claude-3-sonnet-20240229-v1:0"
9
  }
10
}

Cross-Account Access

To use Bedrock from a different AWS account:

1. Set up the target account role (where Bedrock is):

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Principal": {
7
        "AWS": "arn:aws:iam::SOURCE_ACCOUNT_ID:root"
8
      },
9
      "Action": "sts:AssumeRole",
10
      "Condition": {
11
        "StringEquals": {
12
          "sts:ExternalId": "YOUR_EXTERNAL_ID"
13
        }
14
      }
15
    }
16
  ]
17
}

2. Configure the source account (where your application runs):

1
# ~/.aws/config
2
[profile target-role]
3
role_arn = arn:aws:iam::TARGET_ACCOUNT_ID:role/ROLE_NAME
4
source_profile = default
5
region = us-east-1

1
client<llm> MyClient {
2
  provider aws-bedrock
3
  options {
4
    profile "target-role"
5
    model "anthropic.claude-3-sonnet-20240229-v1:0"
6
  }
7
}

IAM Permissions

Basic Permissions

The following IAM permissions are required for basic Bedrock access:

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Action": [
7
        "bedrock:InvokeModel",
8
        "bedrock:InvokeModelWithResponseStream"
9
      ],
10
      "Resource": "arn:aws:bedrock:*:*:model/*"
11
    }
12
  ]
13
}

Additional Permissions

Depending on your setup, you might need additional permissions:

Best Practices

• Follow the principle of least privilege
• Use resource-based policies when possible
• Consider using AWS Organizations SCPs for enterprise-wide controls
• Regularly audit IAM permissions using AWS IAM Access Analyzer

Configuration Options

BAML-specific request options

These unique parameters (aka options) are modify the API request sent to the provider.

You can use this to modify the region, access_key_id, secret_access_key, and session_token sent to the provider.

1
{
2
  "user": "human",
3
  "assistant": "ai",
4
}

1
client<llm> Foo {
2
  provider openai
3
  options {
4
    allowed_role_metadata: ["foo", "bar"]
5
  }
6
}
7
8
client<llm> FooWithout {
9
  provider openai
10
  options {
11
  }
12
}
13
template_string Foo() #"
14
  {{ _.role('user', foo={"type": "ephemeral"}, bar="1", cat=True) }}
15
  This will be have foo and bar, but not cat metadata. But only for Foo, not FooWithout.
16
  {{ _.role('user') }}
17
  This will have none of the role metadata for Foo or FooWithout.
18
"#

1
client<llm> MyClientWithoutStreaming {
2
  provider anthropic
3
  options {
4
    model claude-3-5-haiku-20241022
5
    api_key env.ANTHROPIC_API_KEY
6
    max_tokens 1000
7
    supports_streaming false
8
  }
9
}
10
11
function MyFunction() -> string {
12
  client MyClientWithoutStreaming
13
  prompt #"Write a short story"#
14
}

1
# This will be streamed from your python code perspective, 
2
# but under the hood it will call the non-streaming HTTP API
3
# and then return a streamable response with a single event
4
b.stream.MyFunction()
5
6
# This will work exactly the same as before
7
b.MyFunction()

1
client<llm> MyClient {
2
  provider "openai"
3
  options {
4
    model "gpt-5-mini"
5
    api_key env.OPENAI_API_KEY
6
    // Finish reason allow list will only allow the stop finish reason
7
    finish_reason_allow_list ["stop"]
8
  }
9
}

1
client<llm> MyClient {
2
  provider "openai"
3
  options {
4
    model "gpt-5-mini"
5
    api_key env.OPENAI_API_KEY
6
    // Finish reason deny list will allow all finish reasons except length
7
    finish_reason_deny_list ["length"]
8
  }
9
}

Modular API

• b.request returns a fully signed SigV4 HTTPRequest pointing at the Converse API.
• Forward the request as-is. Do not mutate the headers; they already include Authorization, X-Amz-Date, and (if needed) X-Amz-Security-Token.
• Send the request immediately after building it. The signature is computed at request time, so rebuilding gives you a fresh signature.
• Streaming modular calls are not yet supported for Bedrock.

1
import { SignatureV4 } from "@aws-sdk/signature-v4"
2
import { defaultProvider } from "@aws-sdk/credential-provider-node"
3
import { HttpRequest } from "@aws-sdk/protocol-http"
4
import { b } from 'baml_client'
5
6
async function callBedrock() {
7
  const req = await b.request.ExtractResume("John Doe | Software Engineer | BSc in CS")
8
9
  const body = req.body.json() as any
10
  const bodyString = JSON.stringify(body)
11
  const url = new URL(req.url)
12
  const region = (req.client_details.options?.region as string) ?? process.env.AWS_REGION ?? "us-east-1"
13
14
  const signer = new SignatureV4({
15
    service: "bedrock",
16
    region,
17
    credentials: defaultProvider(),
18
  })
19
20
  const unsigned = new HttpRequest({
21
    protocol: url.protocol,
22
    hostname: url.hostname,
23
    path: url.pathname,
24
    method: req.method,
25
    headers: {
26
      ...req.headers,
27
      host: url.host,
28
      "content-type": "application/json",
29
    },
30
    body: bodyString,
31
  })
32
33
  const signed = await signer.sign(unsigned)
34
35
  const res = await fetch(req.url, {
36
    method: req.method,
37
    headers: signed.headers as Record<string, string>,
38
    body: bodyString,
39
  })
40
41
  if (!res.ok) {
42
    throw new Error(`Bedrock request failed: ${res.status}`)
43
  }
44
45
  const payload = await res.json()
46
  const message = payload.output.message.content.find((block: any) => block.text)?.text ?? ''
47
48
  return b.parse.ExtractResume(message)
49
}

1
import json
2
import requests
3
from botocore.auth import SigV4Auth
4
from botocore.awsrequest import AWSRequest
5
import boto3
6
from baml_client import b
7
8
def call_bedrock():
9
  req = b.request.ExtractResume("John Doe | Software Engineer | BSc in CS")
10
11
  body = req.body.json()
12
  body_bytes = json.dumps(body).encode("utf-8")
13
14
  session = boto3.Session()
15
  credentials = session.get_credentials().get_frozen_credentials()
16
  region = req.client_details.options.get("region") or session.region_name or "us-east-1"
17
18
  aws_request = AWSRequest(
19
    method=req.method,
20
    url=req.url,
21
    data=body_bytes,
22
    headers=dict(req.headers),
23
  )
24
  SigV4Auth(credentials, "bedrock", region).add_auth(aws_request)
25
26
  response = requests.post(
27
    req.url,
28
    headers=dict(aws_request.headers.items()),
29
    data=body_bytes,
30
  )
31
  response.raise_for_status()
32
33
  payload = response.json()
34
  message = payload["output"]["message"]["content"][0]["text"]
35
  return b.parse.ExtractResume(message)

media_url_handler

Controls how media URLs are processed before sending to the provider. This allows you to override the default behavior for handling images, audio, PDFs, and videos.

1
client<llm> MyClient {
2
  provider openai
3
  options {
4
    media_url_handler {
5
      image "send_base64"                    // Options: send_base64 | send_url | send_url_add_mime_type | send_base64_unless_google_url
6
      audio "send_url"
7
      pdf "send_url_add_mime_type"
8
      video "send_url"
9
    }
10
  }
11
}

Options

Each media type can be configured with one of these modes:

• send_base64 - Always download URLs and convert to base64 data URIs
• send_url - Pass URLs through unchanged to the provider
• send_url_add_mime_type - Ensure MIME type is present (may require downloading to detect)
• send_base64_unless_google_url - Only process non-gs:// URLs (keep Google Cloud Storage URLs as-is)

Provider Defaults

If not specified, each provider uses these defaults:

When to Use

• Use send_base64 when your provider doesn’t support external URLs and you need to embed media content
• Use send_url when your provider handles URL fetching and you want to avoid the overhead of base64 conversion
• Use send_url_add_mime_type when your provider requires MIME type information (e.g., Vertex AI)
• Use send_base64_unless_google_url when working with Google Cloud Storage and want to preserve gs:// URLs

Provider request parameters

These are other options that are passed through to the provider, without modification by BAML. For example if the request has a temperature field, you can define it in the client here so every call has that set.

Consult the specific provider’s documentation for more information.

1
client<llm> MyClient {
2
  provider aws-bedrock
3
  options {
4
    inference_configuration {
5
      max_tokens 1000
6
      temperature 1.0
7
      top_p 0.8
8
    }
9
  }
10
}

Troubleshooting

Common Errors

1
{
2
  "Error": "AccessDeniedException",
3
  "Message": "User is not authorized to perform: bedrock:InvokeModel"
4
}

1
{
2
  "Error": "UnrecognizedClientException",
3
  "Message": "The security token included in the request is invalid"
4
}

1
{
2
  "Error": "ValidationException",
3
  "Message": "Model is not supported in this Region"
4
}

1
{
2
  "Error": "ValidationException",
3
  "Message": "Account is not authorized to use model"
4
}

Environment-Specific Setup