Pipelines models reference Innovation Release

This reference documentation for EDB Postgres AI - AI Accelerator Pipelines models includes information on the functions and views available in the aidb extension for working with models.

Tables

aidb.model_providers

The aidb.model_providers table stores information about the model providers that are available.

ColumnTypeDescription
server_namenameName for the model server
server_descriptiontextDescription of the model server
server_optionstext[]Options for the model server

aidb.models

Returns a list of all models in the registry and their configured options, including predefined models and user-created models.

ColumnTypeDescription
nametextUser-defined name for the model
providertextName of the model provider
optionsjsonbOptional configuration for the model provider

Example

SELECT * FROM aidb.models;
Output
 name  |  provider  |    options
-------+------------+---------------
 bert  | bert_local | {"config={}"}
 clip  | clip_local | {"config={}"}
 t5    | t5_local   | {"config={}"}

Functions

aidb.create_model

Creates a new model in the system by saving its name, provider, and optional configuration.

Parameters

ParameterTypeDefaultDescription
nametextUser-defined name for the model.
providertextName of the model provider (as found in aidb.model_providers).
configjsonb'{}'::jsonbOptional configuration for the model provider. May include model-specific parameters such as model, url, and TLS options (e.g., tls_config).
credentialsjsonb'{}'::jsonbOptional credentials for the model provider.
replace_credentialsbooleanfalseIf true, replaces the credentials for the model provider. If false, the credentials aren't overwritten.

Example

SELECT aidb.create_model(
               name => 'my_t5'::text,
               provider => 't5_local'::character varying,
               config => '{"param1": "value1", "param2": "value2"}'::jsonb,
               credentials => '{"token": "abcd"}'::jsonb
           );

or equivalently, using default values:

SELECT aidb.create_model('my_t5', 't5_local');

or if updating the credentials of a model's provider, which has already been created.

SELECT aidb.create_model(
               name => 'my_t5'::text,
               provider => 't5_local'::character varying,
               config => '{"param1": "value1", "param2": "value2"}'::jsonb,
               credentials => '{"token": "abcd"}'::jsonb,
               replace_credentials => true
           );

TLS Configuration (optional)

To securely connect to HTTPS-based model endpoints, the config object can include a tls_config field:

"tls_config": {
  "insecure_skip_verify": true,  // (optional) disables certificate validation
  "ca_path": "/etc/aidb/myCA.pem"  // (optional) path to a trusted CA certificate
}

aidb.get_model

Returns the configuration for a model in the registry.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model

Returns

ColumnTypeDescription
nametextUser-defined name for the model
providertextName of the model provider
optionsjsonbOptional configuration for the model provider

Example

SELECT * FROM aidb.get_model('t5');
Output
 name | provider |    options
------+----------+---------------
 t5   | t5_local | {"config={}"}
(1 row)

aidb.delete_model

Deletes a model from the registry.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model

Example

SELECT aidb.delete_model('t5');
Output
     delete_model
---------------------------------
 (t5,t5_local,"{""config={}""}")
(1 row)

Returns

ColumnTypeDescription
delete_modeljsonbThe name, provider, and options of the deleted model

aidb.list_hcp_models

Gets models running on the Hybrid Manager.

Returns

ColumnTypeDescription
nametextThe name of the model instance running on the HCP (Hybrid Manager)
urltextThe API URL of the model running on the HCP (Hybrid Manager)
modeltextThe name the model running on the HCP (Hybrid Manager)

Example

SELECT * FROM  aidb.get_hcp_models();
                 name                  |                                       url                                        |               model
---------------------------------------+----------------------------------------------------------------------------------+------------------------------------
 llama-3-1-8b-instruct-1xgpu-g6        | http://llama-3-1-8b-instruct-1xgpu-g6-predictor.default.svc.cluster.local        | meta/llama-3.1-8b-instruct
 llama-3-2-nv-embedqa-1b-v2            | http://llama-3-2-nv-embedqa-1b-v2-predictor.default.svc.cluster.local            | nvidia/llama-3.2-nv-embedqa-1b-v2
 meta-nim-llama3-70b-instruct-8xgpu-g5 | http://meta-nim-llama3-70b-instruct-8xgpu-g5-predictor.default.svc.cluster.local | meta/llama3-70b-instruct
(3 rows)

aidb.create_hcp_model

Creates a new model in the system by referencing a running instance in the HCP (Hybrid Manager)

Parameters

ParameterTypeDefaultDescription
nametextUser-defined name of the model
hcp_model_nametextName of the model instance running on HCP (Hybrid Manager)

aidb.sync_hcp_models

Creates models in the HCP (Hybrid Manager) and sets is_hcp_model=true; deletes models with that setting if not found in the HCP (Hybrid Manager).

Returns

ColumnTypeDescription
statustextSynchronization status; either deleted, unchanged, created, or skipped.
modeltextThe name the synchronized HCP (Hybrid Manager) model.

aidb.encode_text

Encodes text using a model, generating a vector representation of a given text input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to encode with
texttextText to encode

aidb.encode_text_batch

Encodes a batch of text using a model, generating a vector representation of a given text input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to encode with
texttext[]Array of text to encode

aidb.decode_text

Decodes text using a model, generating a text response from a given text input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to decode with
texttextText prompt/input to decode
inference_configjsonNULLOptional runtime inference settings (use aidb.inference_config() helper)

Returns

ColumnTypeDescription
decode_texttextThe decoded text

Example

-- Basic usage
SELECT aidb.decode_text('t5', 'translate to german: hello world');

-- With inference configuration
SELECT aidb.decode_text(
    'llama',
    'Explain quantum computing.',
    aidb.inference_config(
        system_prompt => 'Be concise and factual.',
        temperature => 0.7,
        max_tokens => 100
    )::json
);

aidb.decode_text_batch

Decodes a batch of text using a model, generating text responses for each input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to decode with
inputtext[]Array of text inputs/prompts to decode
inference_configjsonNULLOptional runtime inference settings (use aidb.inference_config() helper)

Returns

ColumnTypeDescription
decode_text_batchtextThe decoded text

Example

-- Basic usage
SELECT * FROM aidb.decode_text_batch('t5', ARRAY[
    'translate to german: hello',
    'translate to german: goodbye'
]);

-- With inference configuration
SELECT * FROM aidb.decode_text_batch(
    't5',
    ARRAY['summarize: a very long text goes here', 'summarize: another text'],
    aidb.inference_config(
        temperature => 0.3,
        max_tokens => 50,
        seed => 42
    )::json
);

aidb.encode_image

Encodes an image using a model, generating a vector representation of a given image input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to encode with
imagebyteaImage to encode

Returns

ColumnTypeDescription
encode_imagebyteaThe encoded image

aidb.rerank_text

Reranks text using a model, generating a vector representation of a given text input.

Parameters

ParameterTypeDefaultDescription
model_nametextName of the model to rerank with
querytextQuery based on which the input will be ranked
inputtext[][]Inputs to be ranked

Returns

ColumnTypeDescription
texttextThe text from "input"
logit_scoredouble precisionScore/rank of this text
idintindex that the text had in the input array

Inference configuration helper function

aidb.inference_config

Creates a configuration JSON object for runtime inference settings. This helper allows you to override model defaults when calling aidb.decode_text(), aidb.decode_text_batch(), or aidb.summarize_text().

Parameters

ParameterTypeDefaultDescription
system_promptTEXTNULLSystem prompt to control model behavior (overrides model's configured system prompt). Not supported by T5 models.
temperatureDOUBLE PRECISIONNULLSampling temperature. 0.0 is deterministic, higher values increase variety. Typical range: 0.0 to 2.0.
max_tokensINTEGERNULLMaximum number of tokens to generate.
top_pDOUBLE PRECISIONNULLTop-p (nucleus) sampling. Samples only from the smallest set of tokens whose cumulative probability exceeds p.
seedBIGINTNULLSeed for random number generation to ensure reproducibility.
repeat_penaltyREALNULLPenalty for repeating tokens. 1.0 applies no penalty, higher values increase the penalty. Not supported by NIM and OpenAI Completions.
repeat_last_nINTEGERNULLNumber of most recent tokens to consider for repeat_penalty. Not supported by NIM and OpenAI Completions.
thinkingBOOLEANNULLWhether to show <think>...</think> tags in the response. true and NULL (default) retain the tags (shows reasoning), false strips them.
extra_argsJSONBNULLExtra arguments to pass to the LLM API, allowing provider-specific parameters. For example, '{"stop": ["\n"], "logprobs": true}'.

Returns

JSONB configuration object for use with aidb.decode_text(), aidb.decode_text_batch(), or aidb.summarize_text_config().

Examples

-- Basic inference configuration
SELECT aidb.decode_text(
    'llama',
    'Explain relativity.',
    aidb.inference_config(
        temperature => 0.7,
        max_tokens => 200
    )::json
);

-- Full configuration with system prompt
SELECT aidb.decode_text(
    'llama',
    'What is machine learning?',
    aidb.inference_config(
        system_prompt => 'You are a computer science professor. Explain concepts clearly.',
        temperature => 0.5,
        max_tokens => 150,
        top_p => 0.9,
        seed => 42,
        repeat_penalty => 1.1,
        repeat_last_n => 64
    )::json
);

-- With thinking tags control (for reasoning models)
SELECT aidb.decode_text(
    'reasoning_model',
    'Solve this step by step: 2x + 5 = 15',
    aidb.inference_config(
        thinking => false,  -- Hide internal reasoning
        temperature => 0.3
    )::json
);

-- With provider-specific extra arguments
SELECT aidb.decode_text(
    'openai_model',
    'List three colors.',
    aidb.inference_config(
        max_tokens => 50,
        extra_args => '{"stop": ["\n\n"], "logprobs": true}'::jsonb
    )::json
);

-- Use with summarize_text_config
SELECT aidb.summarize_text(
    'Long text to summarize...',
    aidb.summarize_text_config(
        't5',
        inference_config => aidb.inference_config(
            temperature => 0.3,
            max_tokens => 100
        )
    )::json
);

Model configuration helper functions

These helper functions simplify creating JSONB configuration objects for aidb.create_model(). They provide proper parameter types, defaults, and validation.

aidb.embeddings_config

Creates configuration for embeddings model providers (embeddings, openai_embeddings, nim_embeddings).

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
api_keyTEXTNULLAPI key (can use credentials instead)
basic_authTEXTNULLBasic auth credentials
urlTEXTNULLCustom API URL
max_concurrent_requestsINTEGERNULLMaximum concurrent requests
max_batch_sizeINTEGERNULLMaximum batch size
input_typeTEXTNULLInput type for the model
input_type_queryTEXTNULLInput type for queries
tls_configJSONBNULLTLS configuration
is_hcp_modelBOOLEANNULLWhether this is an HCP (Hybrid Manager) model

Example

SELECT aidb.create_model(
  'my_model',
  'openai_embeddings',
  aidb.embeddings_config(model => 'text-embedding-3-small'),
  '{"api_key": "sk-..."}'::JSONB
);

aidb.completions_config

Creates configuration for completions model providers (completions, openai_completions, nim_completions).

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
api_keyTEXTNULLAPI key (can use credentials instead)
basic_authTEXTNULLBasic auth credentials
urlTEXTNULLCustom API URL
max_concurrent_requestsINTEGERNULLMaximum concurrent requests
max_tokensJSONBNULLMax tokens config (use aidb.max_tokens_config())
is_hcp_modelBOOLEANNULLWhether this is an HCP (Hybrid Manager) model
system_promptTEXTNULLSystem prompt to control model behavior. Not supported by T5 models.
temperatureDOUBLE PRECISIONNULLSampling temperature (0.0 = deterministic, higher = more random). Typical range: 0.0 to 2.0
top_pDOUBLE PRECISIONNULLTop-p (nucleus) sampling. Only samples from the smallest set of tokens whose cumulative probability exceeds p.
seedBIGINTNULLSeed for random number generation (for reproducibility)
thinkingBOOLEANNULLWhether to show <think>...</think> tags in responses. true keeps thinking tags (shows reasoning), false strips tags and their contents, NULL keeps tags.
extra_argsJSONBNULLExtra arguments to pass to the LLM API, allowing provider-specific parameters. Example: '{"stop": ["\n"], "logprobs": true}'

Example

-- Basic configuration
SELECT aidb.create_model(
  'my_model',
  'openai_completions',
  aidb.completions_config(model => 'gpt-4o'),
  '{"api_key": "sk-..."}'::JSONB
);

-- Configuration with default inference settings
SELECT aidb.create_model(
  'my_assistant',
  'openai_completions',
  aidb.completions_config(
    model => 'gpt-4o',
    system_prompt => 'You are a helpful assistant.',
    temperature => 0.7,
    top_p => 0.9
  ),
  '{"api_key": "sk-..."}'::JSONB
);

aidb.max_tokens_config

Creates the max_tokens configuration for completions models.

Parameters

ParameterTypeDefaultDescription
sizeINTEGERRequiredMaximum number of tokens
formatTEXTNULLFormat: 'default', 'legacy', or 'both'

Example

aidb.completions_config(
  model => 'gpt-4o',
  max_tokens => aidb.max_tokens_config(size => 1024, format => 'default')
)

aidb.bert_config

Creates configuration for the bert_local model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
revisionTEXTNULLModel revision
cache_dirTEXTNULLCache directory path

aidb.clip_config

Creates configuration for the clip_local model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
revisionTEXTNULLModel revision
cache_dirTEXTNULLCache directory path
image_sizeINTEGERNULLImage size

aidb.llama_config

Creates configuration for the llama_instruct_local model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
revisionTEXTNULLModel revision
cache_dirTEXTNULLCache directory path
system_promptTEXTNULLSystem prompt
use_flash_attentionBOOLEANNULLUse flash attention
model_pathTEXTNULLLocal model path
seedBIGINTNULLRandom seed
temperatureDOUBLE PRECISIONNULLSampling temperature
top_pDOUBLE PRECISIONNULLTop-p sampling
sample_lenINTEGERNULLMax tokens to generate
use_kv_cacheBOOLEANNULLUse KV cache
repeat_penaltyREALNULLRepetition penalty
repeat_last_nINTEGERNULLTokens for repetition penalty

aidb.t5_config

Creates configuration for the t5_local model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier.
revisionTEXTNULLModel revision.
temperatureDOUBLE PRECISIONNULLSampling temperature.
seedBIGINTNULLRandom seed.
max_tokensINTEGERNULLMax tokens to generate.
repeat_penaltyREALNULLRepetition penalty.
repeat_last_nINTEGERNULLTokens for repetition penalty.
model_pathTEXTNULLLocal model path.
cache_dirTEXTNULLCache directory path.
top_pDOUBLE PRECISIONNULLTop-p (nucleus) sampling. Only samples from tokens whose cumulative probability exceeds p.

aidb.gemini_config

Creates configuration for the gemini model provider.

Parameters

ParameterTypeDefaultDescription
api_keyTEXTRequiredThe Gemini API key
modelTEXTNULLThe model identifier
urlTEXTNULLCustom API URL
max_concurrent_requestsINTEGERNULLMaximum concurrent requests
thinking_budgetINTEGERNULLThinking tokens budget

aidb.nim_clip_config

Creates configuration for the nim_clip model provider.

Parameters

ParameterTypeDefaultDescription
api_keyTEXTNULLAPI key (can use credentials)
basic_authTEXTNULLBasic auth credentials
modelTEXTNULLThe model identifier
urlTEXTNULLCustom API URL
is_hcp_modelBOOLEANNULLWhether this is an HCP (Hybrid Manager) model

aidb.nim_ocr_config

Creates configuration for the nim_paddle_ocr model provider.

Parameters

ParameterTypeDefaultDescription
api_keyTEXTNULLAPI key (can use credentials)
basic_authTEXTNULLBasic auth credentials
urlTEXTNULLCustom API URL
modelTEXTNULLThe model identifier
is_hcp_modelBOOLEANNULLWhether this is an HCP (Hybrid Manager) model

aidb.nim_reranking_config

Creates configuration for the nim_reranking model provider.

Parameters

ParameterTypeDefaultDescription
api_keyTEXTNULLAPI key (can use credentials)
basic_authTEXTNULLBasic auth credentials
modelTEXTNULLThe model identifier
urlTEXTNULLCustom API URL
is_hcp_modelBOOLEANNULLWhether this is an HCP (Hybrid Manager) model

aidb.openrouter_chat_config

Creates configuration for the openrouter_chat model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
api_keyTEXTNULLAPI key (can use credentials instead)
urlTEXTNULLCustom API URL
max_concurrent_requestsINTEGERNULLMaximum concurrent requests
max_tokensJSONBNULLMax tokens config (use aidb.max_tokens_config())

aidb.openrouter_embeddings_config

Creates configuration for the openrouter_embeddings model provider.

Parameters

ParameterTypeDefaultDescription
modelTEXTRequiredThe model identifier
api_keyTEXTNULLAPI key (can use credentials instead)
urlTEXTNULLCustom API URL
max_concurrent_requestsINTEGERNULLMaximum concurrent requests
max_batch_sizeINTEGERNULLMaximum batch size