TypeScript support for streaming is still in closed alpha - please contact us if you would like to use it!

Streaming partial objects

The following returns an object that slowly gets filled in as the response comes in. This is useful if you want to start processing the response before it’s fully complete. You can stream anything from a string output type, to a complex object.

Example:

{"prop1": "hello"}
{"prop1": "hello how are you"}
{"prop1": "hello how are you", "prop2": "I'm good, how are you?"}
{"prop1": "hello how are you", "prop2": "I'm good, how are you?", "prop3": "I'm doing great, thanks for asking!"}
async def main():
    async with baml.MyFunction.stream(MyInput(...)) as stream:
        async for output in stream.parsed_stream:
           
            if output.is_parseable:
              assert output.parsed.my_property is not None
              print("my property is present", output.parsed.my_property)
              print(f"streaming: {output.parsed.model_dump_json()}")
            
            # You can also get the current delta. This will always be present.
            print(f"streaming: {output.delta}")

        final_output = await stream.get_final_response()
        if final_output.has_value:
            print(f"final response: {final_output.value}")
        else:
            # A deserialization error likely occurred.
            print(f"final resopnse didnt have a value")

You can also get the deltas from the output using output.delta

Stream a specific impl

The following returns a stream of a specific impl. This is useful if you want to process the response as it comes in, but don’t want to deal with the object being partially filled in.

async def main():
    async with baml.MyFunction.get_impl("v1").stream(...) as stream:
        async for chunk in stream.parsed_stream:
            print(f"streaming: {chunk.delta}")

        final_output = await stream.get_final_response()
        if final_output.has_value:
            print(f"final response: {final_output.value}")

Caveats

Not supported with:

  1. Fallback clients
  2. Retry policies (it may work but there may be unknown behaviors)
  3. Output adapters
  4. Union types
  5. TypeScript