Overview

You can configure the global EventBus with EventListeners to listen for various framework events. See Event Listener Drivers for examples on forwarding events to external services.

Specific Event Types

You can listen to specific event types:

All Event Types

Or listen to all events:

Stream Iterator

You can use Structure.run_stream() for streaming Events from the Structure in the form of an iterator.

Tip

Set stream=True on your Prompt Driver in order to receive completion chunk events.

Context Managers

You can also use EventListeners as a Python Context Manager. The EventListener will automatically be added and removed from the EventBus when entering and exiting the context.

Streaming

You can use the BaseChunkEvent to stream the completion results from Prompt Drivers.

You can also use the TextChunkEvent and ActionChunkEvent to further differentiate the different types of chunks for more customized output.

If you want Gen AI Builder to handle the chunk events for you, use the Stream utility to automatically wrap BaseChunkEvents in a Python iterator.

The Stream utility does not automatically enable streaming on the Drivers that produce BaseChunkEvents. Make sure to enable streaming on the Drivers or else Stream will yield no iterations.

Sometimes, streaming can be too verbose. You can use Stream.event_types to only listen to specific event types. A good example is to remove the ActionChunkEvent from the stream if you don't need to see events related to Tool usage.

Counting Tokens

To count tokens, you can use Event Listeners and the TokenCounter util:

Inspecting Payloads

You can use the StartPromptEvent to inspect the Prompt Stack and final prompt string before it is sent to the LLM.

...
Prompt Stack Messages:
system:
user: Write me a poem.
Final Prompt String:


User: Write me a poem.

Assistant:
...

EventListenerDriver.on_event Return Value Behavior

The value that gets returned from the EventListener.on_event will determine what gets sent to the event_listener_driver.

EventListener.on_event is None

By default, the EventListener.on_event function is None. Any events that the EventListener is listening for will get sent to the event_listener_driver as-is.

Return BaseEvent or dict

You can return a BaseEvent or dict object from EventListener.on_event, and it will get sent to the event_listener_driver.

Return None

You can return None in the on_event function to prevent the event from getting sent to the event_listener_driver.


Could this page be better? Report a problem or suggest an addition!