Gen AI Builder: A Python Framework for Building Generative AI Applications

Gen AI Builder is a Python framework designed to simplify the development of generative AI applications. It offers a set of straightforward, flexible abstractions for working with areas such as Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and much more.

  1. Technology-agnostic: Gen AI Builder is designed to work with any capable LLM, data store, and backend through the abstraction of Drivers.
  2. Minimal prompt engineering: It’s much easier to reason about code written in Python, not natural languages. Gen AI Builder aims to default to Python in most cases unless absolutely necessary.

Why Consider a Framework?

If you're just beginning to experiment with generative AI, it's often best to start by building simple applications from scratch. However, as your project grows in complexity, a framework like Gen AI Builder can provide structure and best practices that help you maintain clarity and scalability.

Gen AI Builder strives to be transparent about the underlying AI techniques it uses. Nevertheless, having some foundational knowledge of LLMs and related technology will be helpful when building more sophisticated applications.

Prerequisites

1. OpenAI API Key

By default, Gen AI Builder uses OpenAI for LLM interactions. We’ll learn how to change this via Prompt Drivers later. For now:

  1. Obtain an OpenAI API key.
  2. Set it as an environment variable:
    export OPENAI_API_KEY="your-api-key"
Info

Drivers are a primary building block of Gen AI Builder and Prompt Drivers handle all the details when prompting the model with input.

Prompt Drivers are model agnostic, letting you switch out LLM providers without changing application code.

Tip

To securely and easily manage environment variables, check out tools like python-dotenv or mise.

2. Install Gen AI Builder

Gen AI Builder can be installed via multiple methods. Below are two popular approaches:

Using Uv

uv is a fast, user-friendly dependency manager. If you've used poetry, uv is a similar project but with a focus on simplicity and speed.

  1. Install uv.

  2. Initialize a new project:

    uv init griptape-quickstart
  3. Change into the new directory:

    cd griptape-quickstart
  4. Add Gen AI Builder as a dependency:

    uv add "griptape[all]"
  5. A virtual environment will be automatically created. Activate it:

    source .venv/bin/activate
    Note

    If you use another shell (like fish), this command may differ.

    Tip

    If you’re using mise, you can configure it to automatically activate your virtual environment.

Using Pip

If uv isn’t your style, you can always install Gen AI Builder using pip:

pip install "griptape[all]" -U

Installing Optional Dependencies

Gen AI Builder provides optional dependencies that enable specific features and third-party integrations. If you're new to Gen AI Builder, consider installing [all] to unlock its full functionality. This ensures you have everything you need right from the start.

However, for a more streamlined setup or if you only need certain features, you have two primary options:

  1. Core Dependencies: These provide the minimal, foundational set of libraries for Gen AI Builder to run most default features.

  2. Optional Dependencies: These are additional, vendor-specific drivers (e.g., for Anthropic or Pinecone). Any driver requiring an extra will indicate this in the documentation.

For a core-only installation:

uv add griptape

To install specific optional dependencies such as Anthropic and Pinecone drivers—use:

uv add "griptape[drivers-prompt-anthropic,drivers-vector-pinecone]"

To see all available optional dependencies, see Gen AI Builder's pyproject.toml.

Prompt Task

With everything set up, let’s start building! One of Gen AI Builder’s core abstractions is a Task. We'll start with a Prompt Task to prompt the LLM with some text.

Create a file named app.py and add:

Then run:

python app.py
Tip

Whenever you see a “Logs” tab in the Gen AI Builder documentation, you can click it to view logs and outputs.

If you got a response from the model, congrats! You’ve created your first generative AI application with Gen AI Builder. This introductory guide sticks to the Prompt Task, but Gen AI Builder supports a wide range of Tasks. Explore them to see what else you can do.

Changing Models

Info

This example briefly demonstrates swapping out the underlying model. If you’d prefer to continue using the default OpenAI integration, feel free to skip ahead.

Gen AI Builder’s “Prompt Drivers” make switching model providers simple. For instance, to use Anthropic, follow the same steps you used for the OpenAI key, but this time set an Anthropic API key:

export ANTHROPIC_API_KEY="your-api-key"

Then import Anthropic Prompt Driver and pass it to your Task:

Run the script again to see a similar output. Notice you didn’t have to change any of the application-level logic; you only changed the driver.

For the remainder of this guide, we’ll revert to the OpenAI Chat Prompt Driver:

If you’d like to keep using Anthropic, be sure to adjust code accordingly.

Templating

Gen AI Builder uses the popular Jinja templating engine to help create dynamic prompts. Let’s adapt our Prompt Task to utilize a Jinja template:

Here, {{ args }} is rendered at runtime, allowing you to dynamically insert data into the prompt.

Info

args is a special attribute added to the Task’s context. Depending on the type of Task, it may also have access to additional context variables.

For static, key-value data, we can add context to our Task:

These two methods allow you to build more versatile, data-driven prompts. Next, we’ll discuss steering the LLM’s output using Gen AI Builder’s Rule system.

Rules

LLMs can be unpredictable. Getting them to follow specific instructions can be challenging. Gen AI Builder addresses this by providing Rules and Rulesets, which offer a structured approach to steering the model’s output.

For example, we can create a simple Rule that gives the LLM a name:

Info

Rules are placed in the LLM’s system prompt, which typically yields the highest likelihood that the model will adhere to them. even so, LLMs still sometimes have minds of their own and will not follow Rules 100% of time

Tip

While we typically recommend you stick to Rule-based steering, you can always override the system prompt with a custom one. Though pay close attention to the discussed tradeoffs that come with this approach.

Multiple rules can be grouped into Rulesets. Let’s refactor our code to use more than one Ruleset:

Info

Providing a rules list automatically creates a default Ruleset. This is great for getting started, but we recommend explicitly defining Rulesets for more complex applications.

Gen AI Builder also includes a specialized Rule, JsonSchemaRule, which inserts a JSON Schema into the system prompt, ensuring the LLM attempts to return structured, machine-readable data:

Note

The typing.cast usage is optional, but helps satisfy our type-checker, pyright. We’re actively working to improve Gen AI Builder’s type hints to avoid clunky casts like this.

Now let’s explore an even more robust way to enforce structured outputs.

Structured Output

While a JsonSchemaRule can encourage the LLM to return structured data, it doesn’t provide guarantees, especially for complex schemas. For stronger enforcement, Gen AI Builder’s Prompt Drivers support Structured Output.

Structured Output uses the LLM provider’s built-in API features to produce outputs in a well-defined schema. It also simplifies parsing, so you can work directly with Python objects rather than manual JSON parsing.

Let’s update our Task to use Structured Output:

The LLM response is not only formatted correctly, but automatically parsed into a Python object.

Tip

You can use either pydantic or schema to define your schemas. We recommend pydantic for its ease of use.

Info

If your chosen model provider doesn’t natively support structured output, Gen AI Builder employs multiple fallback strategies. The final fallback is the JsonSchemaRule, which you saw earlier.

Now that we can generate structured responses, let’s make our Task more conversational using Gen AI Builder’s memory features.

Conversation Memory

By default, each LLM call is stateless. If you want the model to recall previous interactions, use Conversation Memory. It tracks Task run history, providing context for the current run.

Tip

You can customize where the conversation is stored by providing a Conversation Memory Driver.

With that set up, the LLM will now remember our previous exchanges. Let’s move on to enabling the model to perform external actions through Tools.

Tools

LLMs themselves cannot browse the internet or perform external actions. You can enable these abilities via Tools. A “Tool” is simply a Python class, and any method marked with the @activity decorator becomes available to the LLM.

Gen AI Builder comes with a collection of built-in Tools. Below is an example that uses a web search and scrape Tool:

Info

Many built-in Tools either provide general-purpose functions (e.g., CalculatorTool) or are Driver-based to support exchanging external providers (e.g., WebSearchTool).

Tools are a powerful method for extending the LLM’s capabilities. Consider writing your own Tools to enable more specialized functionality.

At this point, we have a Task that can handle real-world operations. Now let’s look at orchestrating multiple Tasks.

Structures

So far, we’ve only worked with individual Tasks. As your app becomes more complex, you may want to organize multiple Tasks that execute in sequence or in parallel.

Gen AI Builder provides three Structures:

  • Agent: A lightweight wrapper around a single PromptTask. Great for quick usage but still under the hood is just a PromptTask.
  • Pipeline: Executes Tasks in a specific, linear order.
  • Workflow: Creates highly parallel DAGs, allowing you to fan out and converge Tasks.

Of the three, Workflow is generally the most versatile. Agent and Pipeline can be handy in certain scenarios but are less frequently needed if you’re comfortable just orchestrating Tasks directly.

Let’s place our project-research Task into a Workflow so multiple Tasks can run in parallel:

Warning

Unlike PromptTask.run(), calling Workflow.run() returns the Workflow instance itself. To see final results, use Workflow.output.

Info

Rulesets applied to a Structure are inherited by all child Tasks.

Note

Notice that we've removed the explicit Conversation Memory setup. Structures automatically handle memory for their child Tasks.

A Workflow can handle complex branching and merging of Tasks. Next, we’ll look at automatically summarizing large amounts of text with minimal code changes.

Info

Much of Gen AI Builder's current documentation uses Structures. In particular, the Agent Structure. We are in the process of updating the documentation to include more Task-centric examples. Until then, know that an Agent is nothing more than a Structure with a single Task, usually a PromptTask.

Engines

Gen AI Builder’s Engines provide higher-level patterns for common generative AI tasks. A typical example is summarizing text that may exceed the LLM’s input limits. Gen AI Builder’s PromptSummaryEngine automatically chunks text and merges the partial summaries into a final result.

!!! question

*Couldn’t we just feed all the text into the LLM and say “summarize”?*

Yes, but LLMs have a token limit. `PromptSummaryEngine` helps you handle text that goes beyond that limit by processing it in smaller parts.

The role of an Engine is to provide you with a higher-level abstraction that simplifies common tasks ([summary](./engines/summary-engines.md), [RAG](./engines/rag-engines.md), [evals](./engines/eval-engines.md), [extraction](./engines/extraction-engines.md)).

Directly Using PromptSummaryEngine

Engines can be used standalone or integrated into Tasks. To illustrate, let’s summarize the outputs of our Workflow:

This produces a concise summary. But one strength of a Workflow is its ability to fan out and converge results. Let’s remove the direct PromptSummaryEngine usage and use a built-in summary Task instead.

TextSummaryTask

The TextSummaryTask leverages the PromptSummaryEngine behind the scenes. We’ll have our project-research Tasks converge into one summary Task:

Info

{{ parents_output_text }} is an automatic context variable that contains the text output of all parent Tasks in the Workflow.

Info

Notice we assign child_ids to our parallel PromptTasks and set an id on the TextSummaryTask. This ensures the Workflow correctly connects their outputs.

Finally, let’s visualize our Workflow:

Visit the printed URL in your logs to view a diagram illustrating the Workflow’s structure:

Workflow Diagram

Conclusion and Next Steps

That’s it—you’ve built your first generative AI application with Gen AI Builder, exploring:

  • Simple and advanced Task usage
  • Dynamic prompt creation with Jinja templating
  • Structured output for more reliable data handling
  • Conversation memory for stateful interactions
  • Tools for extending LLM capabilities
  • Parallel Task orchestration with Workflows
  • Text summarization using Engines

This is just the tip of the iceberg. For topic-based guides, continue exploring the Gen AI Builder documentation. For more detailed guides, demos, and advanced techniques, check out Gen AI Builder Tradeschool.


Could this page be better? Report a problem or suggest an addition!