Chain-of-Thought Prompting

Chain-of-thought prompting is a technique that asdf encourages the language model to think step by step, reasoning through the problem before providing an answer. This can improve the quality of the response and make it easier to understand.

Chain-of-Thought Prompting

Chain-of-Thought Prompting Wei et al. (2022)

There are a few different ways to implement chain-of-thought prompting, especially for structured outputs.

  1. Require the model to reason before outputting the structured object.
    • Bonus: Use a template_string to embed the reasoning into multiple functions.
  2. Require the model to flexibly reason before outputting the structured object.
  3. Embed reasoning in the structured object.
  4. Ask the model to embed reasoning as comments in the structured object.

Let’s look at an example of each of these.

We recommend Technique 2 for most use cases. But each technique has its own trade-offs, so please try them out and see which one works best for your use case.

Since BAML leverages Schema-Aligned Parsing (SAP) instead of JSON.parse or LLM modification (like constrained generation or structured outputs), we can do all of the above techniques with any language model!

Technique 1: Reasoning before outputting the structured object

In the below example, we use chain of thought prompting to extract information from an email.

1function GetOrderInfo(email: Email) -> OrderInfo {
2  client "openai/gpt-4o-mini"
3  prompt #"
4    extract everything from this email.
5
6
7    {{ ctx.output_format }}
8
9    Before you answer, please explain your reasoning step-by-step. 
10    
11    For example:
12    If we think step by step we can see that ...
13
14    Therefore the output is:
15    {
16      ... // schema
17    }
18
19    {{ _.role('user') }}
20
21    Sender: {{email.from_address}}
22    Email Subject: {{email.subject}}
23    Email Body: {{email.body}}
24  "#
25}
26
27class Email {
28    subject string
29    body string
30    from_address string
31}
32
33
34class OrderInfo {
35    order_status "ORDERED" | "SHIPPED" | "DELIVERED" | "CANCELLED"
36    tracking_number string?
37    estimated_arrival_date string?
38}
39
40test Test1 {
41  functions [GetOrderInfo]
42  args {
43    email {
44      from_address "hello@amazon.com"
45      subject "Your Amazon.com order of 'Wood Dowel Rods...' has shipped!"
46      body #"
47        Hi Sam, your package will arrive:
48        Thurs, April 4
49        Track your package:
50        www.amazon.com/gp/your-account/ship-track?ie=23&orderId123
51
52        On the way:
53        Wood Dowel Rods...
54        Order #113-7540940
55        Ship to:
56            Sam
57            SEATTLE, WA
58
59        Shipment total:
60        $0.00
61    "#
62
63    }
64  }
65}

Reusable Chain-of-Thought Snippets

You may want to reuse the same technique for multiple functions. Consider template_string!

1template_string ChainOfThought(action: string?) #"
2    Before you answer, please explain your reasoning step-by-step.
3    {% if action %}{{ action }}{% endif %}
4    
5    For example:
6    If we think step by step we can see that ...
7
8    Therefore the output is:
9    {
10      ... // schema
11    }
12"#
13
14function GetOrderInfo(email: Email) -> OrderInfo {
15  client "openai/gpt-"
16  prompt #"
17    Extract everything from this email.
18
19    {{ ctx.output_format }}
20
21    {{ ChainOfThought("focus on things related to shipping") }}
22
23    {{ _.role('user') }}
24
25    Sender: {{email.from_address}}
26    Email Subject: {{email.subject}}
27    Email Body: {{email.body}}
28  "#
29}

Technique 2: Allowing for flexible reasoning

This is one we recommend for most use cases.

1function GetOrderInfo(email: Email) -> OrderInfo {
2  client "openai/gpt-"
3  prompt #"
4    extract everything from this email.
5
6
7    {{ ctx.output_format }}
8
9    Outline some relevant information before you answer.
10    Example:
11    - ...
12    - ...
13    ...
14    {
15      ... // schema
16    }
17
18    {{ _.role('user') }}
19
20    Sender: {{email.from_address}}
21    Email Subject: {{email.subject}}
22    Email Body: {{email.body}}
23  "#
24}

The benefit of using - ... is that we allow the model to know it needs to output some information, but we don’t limit it to a specific format or inject any bias by adding example text that may not be relevant.

Similarly, we use ... after two - ... to indicate that we don’t mean to limit the number of items to 2.

1template_string ChainOfThought() #"
2    Outline some relevant information before you answer.
3    Example:
4    - ...
5    - ...
6    ...
7    {
8      ... // schema
9    }
10"#
11
12function GetOrderInfo(email: Email) -> OrderInfo {
13  client "openai/gpt-"
14  prompt #"
15    extract everything from this email.
16
17    {{ ctx.output_format }}
18
19    {{ ChainOfThought() }}
20
21    {{ _.role('user') }}
22
23    Sender: {{email.from_address}}
24    Email Subject: {{email.subject}}
25    Email Body: {{email.body}}
26  "#
27}

Technique 3: Embed reasoning in the structured object

1class OrderInfo {
2    clues string[] @description(#"
3      relevant quotes from the email related to shipping
4    "#)
5    order_status "ORDERED" | "SHIPPED" | "DELIVERED" | "CANCELLED"
6    tracking_number string?
7    estimated_arrival_date string?
8}
9
10function GetOrderInfo(email: Email) -> OrderInfo {
11  client "openai/gpt-"
12  prompt #"
13    extract everything from this email.
14
15    {{ ctx.output_format }}
16
17    {{ _.role('user') }}
18
19    Sender: {{email.from_address}}
20    Email Subject: {{email.subject}}
21    Email Body: {{email.body}}
22  "#
23}

Technique 4: Ask the model to embed reasoning as comments in the structured object

1class OrderInfo {
2    order_status "ORDERED" | "SHIPPED" | "DELIVERED" | "CANCELLED"
3      @description(#"
4        before fields, in comments list out any relevant clues from the email
5      "#)
6    tracking_number string?
7    estimated_arrival_date string?
8}
9
10function GetOrderInfo(email: Email) -> OrderInfo {
11  client "openai/gpt-"
12  prompt #"
13    extract everything from this email.
14
15    {{ ctx.output_format }}
16
17    {{ _.role('user') }}
18
19    Sender: {{email.from_address}}
20    Email Subject: {{email.subject}}
21    Email Body: {{email.body}}
22  "#
23}
Built with