Vector Store

Suggest edits

Overview

Gen AI Builder provides a way to build drivers for vector DBs where embeddings can be stored and queried. Every Vector Store Driver implements the following methods:

  • upsert() for updating or inserting new text, TextArtifacts or ImageArtifacts into vector DBs. The method will automatically generate embeddings for a given value.
  • upsert_collection() for performing an upsert() in parallel.
  • upsert_vector() for updating new vectors directly.
  • query() for querying vector DBs.

Each Vector Store Driver takes a BaseEmbeddingDriver used to dynamically generate embeddings for strings.

Info

When working with vector database indexes with Gen AI Builder Drivers, make sure the number of dimensions is equal to 1536. Nearly all embedding models create vectors with this number of dimensions. Check the documentation for your vector database on how to create/update vector indexes.

Info

More Vector Store Drivers are coming soon.

Vector Store Drivers

Local

The LocalVectorStoreDriver can be used to load and query data from memory. Here is a complete example of how the Driver can be used to load a webpage into the Driver and query it later:

Gen AI Builder Knowledge Base

The GriptapeCloudVectorStoreDriver can be used to query data from a Gen AI Builder Knowledge Base. Loading into Knowledge Bases is not supported at this time, only querying. Here is a complete example of how the Driver can be used to query an existing Knowledge Base:

Pinecone

Info

This Driver requires the drivers-vector-pinecone extra.

The PineconeVectorStoreDriver supports the Pinecone vector database.

Here is an example of how the Driver can be used to load and query information in a Pinecone cluster:

Marqo

Info

This Driver requires the drivers-vector-marqo extra.

The MarqoVectorStoreDriver supports the Marqo vector database.

Here is an example of how the Driver can be used to load and query information in a Marqo cluster:

Mongodb Atlas

Info

This Driver requires the drivers-vector-mongodb extra.

The MongodbAtlasVectorStoreDriver provides support for storing vector data in a MongoDB Atlas database.

Here is an example of how the Driver can be used to load and query information in a MongoDb Atlas Cluster:

The format for creating a vector index should look similar to the following:

{
  "fields": [
    {
      "numDimensions": 1536,
      "path": "<path_to_vector>",
      "similarity": "euclidean",
      "type": "vector"
    },
    {
      "path": "namespace",
      "type": "filter"
    }
  ]
}

Replace path_to_vector with the expected field name where the vector content will be.

Azure MongoDB

Info

This Driver requires the drivers-vector-mongodb extra.

The AzureMongoDbVectorStoreDriver provides support for storing vector data in an Azure CosmosDb database account using the MongoDb vCore API

Here is an example of how the Driver can be used to load and query information in an Azure CosmosDb MongoDb vCore database. It is very similar to the Driver for MongoDb Atlas:

Redis

Info

This Driver requires the drivers-vector-redis extra.

The RedisVectorStoreDriver integrates with the Redis vector storage system.

Here is an example of how the Driver can be used to load and query information in a Redis Cluster:

The format for creating a vector index should be similar to the following:

FT.CREATE idx:griptape ON hash PREFIX 1 "griptape:" SCHEMA namespace TAG vector VECTOR FLAT 6 TYPE FLOAT32 DIM 1536 DISTANCE_METRIC COSINE

OpenSearch

Info

This Driver requires the drivers-vector-opensearch extra.

The OpenSearchVectorStoreDriver integrates with the OpenSearch platform, allowing for storage, retrieval, and querying of vector data.

Here is an example of how the Driver can be used to load and query information in an OpenSearch Cluster:

The body mappings for creating a vector index should look similar to the following:

{
    "mappings": {
        "properties": {
            "vector": {"type": "knn_vector", "dimension": 1536},
            "namespace": {"type": "keyword"},
            "metadata": {"type": "object", "enabled": true}
        }
    }
}

PGVector

Info

This Driver requires the drivers-vector-pgvector extra.

The PGVectorVectorStoreDriver integrates with PGVector, a vector storage and search extension for Postgres. While Gen AI Builder will handle enabling the extension, PGVector must be installed and ready for use in your Postgres instance before using this Vector Store Driver.

Here is an example of how the Driver can be used to load and query information in a Postgres database:

Qdrant

Info

This Driver requires the drivers-vector-qdrant extra.

The QdrantVectorStoreDriver supports the Qdrant vector database.

Here is an example of how the Driver can be used to query information in a Qdrant collection:

Astra DB

Info

This Driver requires the drivers-vector-astra-db extra.

The AstraDbVectorStoreDriver supports DataStax Astra DB.

The following example shows how to store vector entries and query the information using the driver:

PG.AI

Info

This Driver requires the drivers-vector-pgai extra.

The PgAiKnowledgeBaseVectorStoreDriver integrates with PG.AI, a managed postgres platform from EnterpriseDB.

Here is an example of how the Driver can be used to load and query information in a PG.AI Knowledge Base:

← Prev

Rerank

↑ Up

Prompt

Next →

File Manager

Could this page be better? Report a problem or suggest an addition!