Creating a Classification Function with Symbol Tuning

Aliasing field names to abstract symbols like “k1”, “k2”, etc. can improve classification results. This technique, known as symbol tuning, helps the LLM focus on your descriptions rather than being biased by the enum or property names themselves.

See the paper Symbol Tuning Improves In-Context Learning in Language Models for more details.

1enum MyClass {
2    Refund @alias("k1")
3    @description("Customer wants to refund a product")
4
5    CancelOrder @alias("k2")
6    @description("Customer wants to cancel an order")
7
8    TechnicalSupport @alias("k3")
9    @description("Customer needs help with a technical issue unrelated to account creation or login")
10
11    AccountIssue @alias("k4")
12    @description("Specifically relates to account-login or account-creation")
13
14    Question @alias("k5")
15    @description("Customer has a question")
16}
17
18function ClassifyMessageWithSymbol(input: string) -> MyClass {
19  client GPT4o
20
21  prompt #"
22    Classify the following INPUT into ONE
23    of the following categories:
24
25    INPUT: {{ input }}
26
27    {{ ctx.output_format }}
28
29    Response:
30  "#
31}
32
33test Test1 {
34  functions [ClassifyMessageWithSymbol]
35  args {
36    input "I can't access my account using my login credentials. I havent received the promised reset password email. Please help."
37  }
38}