Comparing Marvin
Marvin lets developers do extraction or classification tasks in Python as shown below (TypeScript is not supported):
You can also provide instructions:
or using enums to classify
For enum classification, you can add more instructions to each enum, but then you don’t get fully typed outputs, nor can reuse the enum in your own code. You’re back to working with raw strings.
Marvin has some inherent limitations for example:
- How to use a different model?
- What is the full prompt? Where does it live? What if I want to change it because it doesn’t work well for my use-case? How many tokens is it?
- How do I test this function?
- How do I visualize results over time in production?
Using BAML
Here is the BAML equivalent of this classification task based off the prompt Marvin uses under-the-hood. Note how the prompt becomes transparent to you using BAML. You can easily make it more complex or simpler depending on the model.
And you can call this function in your code
The prompt string may be more wordy, but with BAML you now have
- Fully typed responses, guaranteed
- Full transparency and flexibility of the prompt string
- Full freedom for what model to use
- Helper functions to manipulate types in prompts (print_enum)
- Testing capabilities using the VSCode playground
- Analytics in the Boundary Dashboard
- Support for TypeScript
- A better understanding of how prompt engineering works
Marvin was a big source of inspiration for us — their approach is simple and elegant. We recommend checking out Marvin if you’re just starting out with prompt engineering or want to do a one-off simple task in Python. But if you’d like a whole added set of features, we’d love for you to give BAML a try and let us know what you think.
Limitations of BAML
BAML does have some limitations we are continuously working on. Here are a few of them:
- It is a new language. However, it is fully open source and getting started takes less than 10 minutes. We are on-call 24/7 to help with any issues (and even provide prompt engineering tips)
- Developing requires VSCode. You could use vim and we have workarounds but we don’t recommend it.