Now that we’ve built a basic classifier in Level 1, we can start to improve our results.

A common problem with LLMs is that they over-index on the name of the actual category names you declare. For example, in our enum we had TechnicalSupport and AccountIssue, which are closely related. We should likely disambiguate between the two by adding descriptions to each.

Using descriptions in .baml files

Add a @description attribute to each enum value we declared earlier like so:

enum Category {
    Refund @description("Customer wants to refund a product")
    CancelOrder @description("Customer wants to cancel an order")
    TechnicalSupport @description("Customer needs help with a technical issue unrelated to account creation or login")
    AccountIssue @description("Specifically relates to account-login or account-creation")
    Question @description("Customer has a question")
}

Since we used print_enum, we don’t need to change the prompt. As you’ll see in the playground, the prompt will automatically update to include the descriptions.

Classify the following INPUT into ONE
of the following Categories:

Category
---
Refund: Customer wants to refund a product
CancelOrder: Customer wants to cancel an order
TechnicalSupport: Customer needs help with a technical issue unrelated to account creation or login
AccountIssue: Specifically relates to account-login or account-creation
Question: Customer has a question

INPUT: {arg}

Respond only with the name / identifier. Not any other description.
Category:

We don’t need to change your actual application logic in the Python code at all.

Renaming our classes

A common problem with LLMs is that they tend to over-index on the class names you declare. For example, depending on the training data the LLM saw, it may understand “troubleshooting” better than “TechnicalSupport”. Even using ALLCAPS may improve results.

In our case, we will actually rename all the classes to “symbols” (k1, k2, k3…) so that the LLM focuses only on the actual description. This is a technique developed in this research paper called symbol tuning.

To do this, we will use the @alias attribute. Add the following to your enum:

enum Category {
    Refund
    @alias("k1")
    @description("Customer wants to refund a product")

    CancelOrder
    @alias("k2")
    @description("Customer wants to cancel an order")

    TechnicalSupport
    @alias("k3")
    @description("Customer needs help with a technical issue unrelated to account creation or login")

    AccountIssue
    @alias("k4")
    @description("Specifically relates to account-login or account-creation")

    Question
    @alias("k5")
    @description("Customer has a question")
}

The rendered prompt will now be:

Classify the following INPUT into ONE
of the following Categories:

Category
---
k1: Customer wants to refund a product
k2: Customer wants to cancel an order
k3: Customer needs help with a technical issue unrelated to account creation or login
k4: Specifically relates to account-login or account-creation
k5: Customer has a question

INPUT: {arg}

Respond only with the name / identifier. Not any other description.
Category:

At this point, your python code remains the same. When the LLM outputs k1, the BAML deserializer will automatically recognize the alias and map it back to Intent.Refund.

Here is the same python code as before for reference:

from baml_client import baml as b
# The Category type is generated from your BAML code.
from baml_client.baml_types import Category
import asyncio


async def main():
  category = await b.ClassifyMessage("I want to cancel my order")
  if category == Category.CancelOrder:
    print("Customer wants to cancel order")
  else:
    print("Customer wants to {}".format(category))


if __name__ == "__main__":
    asyncio.run(main())

Skipping classes

You can also skip a class entirely by using the @skip attribute. For example, if you wanted to remove the Question class, you could do so like this:

enum Category {
    ...

    Question
    @skip
    @alias("k5")
    @description("Customer has a question")
}

The @skip makes print_enum remove the class from the prompt entirely, so the LLM is only working on a subset of the classes you declared.

  Classify the following INPUT into ONE
  of the following Categories:

  Category
  ---
  k1: Customer wants to refund a product
  k2: Customer wants to cancel an order
  k3: Customer needs help with a technical issue unrelated to account creation or login
  k4: Specifically relates to account-login or account-creation
- k5: Customer has a question


  INPUT: {arg}

  Respond only with the name / identifier. Not any other description.
  Category:

Using impl-level overrides (alpha)

If you want to have two different versions of a prompt — each with different descriptions, you can do so by using impl-level overrides.

For example, say you don’t want TechnicalSupport to be aliased to “k3” and you also want a different description for it, but only in prompt version2. You can declare a version2 impl with the same prompt, but with an override on the Category enum like so:

impl<llm, ClassifyMessage> version2 {
  client GPT4

  override Category {
    TechnicalSupport
    @alias("technical-support")
    @description("Customer needs help with a technical issue unrelated to account creation")

    AccountIssue
    @alias("account-issue")
    @description("Specifically relates to account-creation")

    Question
    @skip
  }

  prompt #"
    Classify the following INPUT into ONE
    of the following Categories:
    {#print_enum(Category)}

    INPUT: {#input}

    Respond only with the name / identifier. Not any other description.
    Category:
  "#
}

Once you declare this, the BAML compiler will error with:

error: Error validating: ClassifyMessage has multiple impls(2).
Add a `default_impl your_impl` field to the function

this happens because in your python code we choose a default version for you to call when you call:

b.ClassifyMessage("I want to cancel my order")

To fix this, add default_impl to the BAML function declaration:

  function ClassifyMessage {
    input string
    output Category
+   default_impl version2
  }

You can always call a specific impl in python using this syntax:

b.ClassifyMessage.get_impl("version1").run("I want to cancel my order")

The impl version2 will now render this prompt:

Classify the following INPUT into ONE
of the following Categories:

Category
---
k1: Customer wants to return a product
k2: Customer wants to cancel an order
technical-support: Customer needs help with a technical issue unrelated to account creation
account-issue: Specifically relates to account-creation

INPUT: {arg}

Respond only with the name / identifier. Not any other description.
Category:

Conclusion

Congrats! Now you should be familiar with

  • @description
  • @alias
  • override MyEnum
  • @skip
  • Using multiple versions (impls) of your prompt

In the next level we will implement a different prompting strategy to generate better results, while keeping our business logic the same.