Models reference v7

Reference for all model-related functions and views in AIDB. For guide-style documentation, see Integrating models.

Configuration model

A model's behavior is controlled two ways, and it matters which one you use for a given setting:

• Creation-time config — set when you call aidb.create_model(), using a provider-specific config helper (aidb.bert_config(), aidb.completions_config(), and so on). This is the model's persistent, stored configuration.
• Call-time inference_config — an optional per-call override, built with aidb.inference_config(), passed directly to an inference function. It overrides the model's stored config for that one call only; anything you don't override falls back to the value set at creation time.

Only some inference functions accept a call-time override — the rest are fixed entirely by the model's creation-time config:

This asymmetry follows from what each capability needs: text generation has runtime knobs worth tuning per call (temperature, system prompt, and so on); embeddings, reranking, and OCR don't, so there's nothing to override.

Catalog views

aidb.model_providers

Lists all available model providers registered in the system.

aidb.models

Lists all models in the registry, including default and user-created models.

Example

SELECT * FROM aidb.models;

  name  |  provider  |    options
--------+------------+---------------
 bert   | bert_local | {"config={}"}
 clip   | clip_local | {"config={}"}
 t5     | t5_local   | {"config={}"}
(3 rows)

Model management functions

aidb.create_model

Registers a new model in the AIDB model registry.

Parameters

TLS configuration

To connect to HTTPS model endpoints, include a tls_config field inside config:

"tls_config": {
  "insecure_skip_verify": true,
  "ca_path": "/etc/aidb/myCA.pem"
}

Validation

By default (validate => true), aidb.create_model() runs a minimal probe inference before the registration commits, to confirm the model is functional:

• Local models are downloaded and loaded, and a test inference is run — so a bad model identifier or unloadable weights fail at creation time rather than on first use.
• External models receive a small test request — so an invalid API key, wrong URL, or unreachable endpoint is caught right away.

If the probe fails, aidb.create_model() raises an error and registers nothing. Pass validate => false to skip the probe — for example, to defer a large download to first use, or to register a model before its endpoint or credentials are available. You can validate a model later with aidb.validate_model().

Examples

-- Minimal registration
SELECT aidb.create_model('my_t5', 't5_local');

-- With config and credentials
SELECT aidb.create_model(
    name        => 'my_t5',
    provider    => 't5_local',
    config      => '{"param1": "value1"}'::JSONB,
    credentials => '{"token": "abcd"}'::JSONB
);

-- Rotate credentials without re-creating
SELECT aidb.create_model(
    name                => 'my_openai',
    provider            => 'openai_embeddings',
    config              => aidb.embeddings_config(model => 'text-embedding-3-small'),
    credentials         => '{"api_key": "<new-key>"}'::JSONB,
    replace_credentials => true
);

aidb.get_model

Returns the configuration for a registered model.

Parameters

Returns

Example

SELECT * FROM aidb.get_model('t5');

 name | provider |    options
------+----------+---------------
 t5   | t5_local | {"config={}"}
(1 row)

aidb.delete_model

Removes a model from the registry. Doesn't affect pipelines or knowledge bases that reference it until they are next executed.

Parameters

Returns

The name, provider, and options of the deleted model as JSONB.

Example

SELECT aidb.delete_model('t5');

     delete_model
---------------------------------
 (t5,t5_local,"{""config={}""}")
(1 row)

aidb.validate_model

Probes a registered model to confirm it's functional, running a minimal test inference. Called automatically by aidb.create_model() unless validate => false; you can also call it directly to re-check an existing model.

Parameters

Returns

TEXT — the model capability that was probed (for example, text embedding, language, reranking, image embedding, or OCR). Raises an error if the probe fails.

Example

SELECT aidb.validate_model('my_bert');

 validate_model
----------------
 text embedding
(1 row)

HCP model functions

These functions manage models running on EDB Hybrid Manager (HCP).

aidb.list_hcp_models

Lists models currently running on the Hybrid Manager.

Returns

Example

SELECT * FROM aidb.list_hcp_models();

             name              |                         url                          |            model
-------------------------------+------------------------------------------------------+------------------------------
 llama-3-1-8b-instruct-1xgpu   | http://llama-3-1-8b-predictor.default.svc.local      | meta/llama-3.1-8b-instruct
(1 row)

aidb.create_hcp_model

Registers an HCP-hosted model by referencing its running instance name.

Parameters

aidb.sync_hcp_models

Synchronizes the AIDB model registry with models currently running on HCP. Creates entries for new HCP models and deletes entries for models no longer running there.

Returns

Inference functions

aidb.encode_text

Encodes a single text string into a vector using the specified model.

Parameters

Returns

real[] — the embedding for the input text. Cast to vector to use with pgvector operators.

Example

SELECT aidb.encode_text('bert_local', 'The quick brown fox');

aidb.encode_text_batch

Encodes an array of text strings into vectors in a single call.

Parameters

Returns

SETOF real[] — one row per input, in input order. The result column is named encode_text_batch.

aidb.encode_image

Encodes a binary image into a vector using a multimodal model (for example, CLIP).

Parameters

Returns

real[] — the embedding for the input image. Cast to vector to use with pgvector operators.

Example

SELECT aidb.encode_image('clip_local', pg_read_binary_file('/tmp/photo.jpg')::BYTEA);

aidb.decode_text

Generates a text response from a prompt using the specified model.

Parameters

Returns

TEXT — the generated response.

Examples

-- Basic usage
SELECT aidb.decode_text('t5_local', 'translate to French: Hello, world.');

-- With inference configuration
SELECT aidb.decode_text(
    'my_llama',
    'Explain quantum computing in one sentence.',
    aidb.inference_config(
        system_prompt => 'Be concise and factual.',
        temperature   => 0.7,
        max_tokens    => 100
    )::json
);

aidb.decode_text_batch

Generates text responses for an array of prompts in a single call.

Parameters

Returns

SETOF TEXT — one row per input prompt, in input order. The result column is named decode_text_batch.

Example

SELECT aidb.decode_text_batch('t5_local', ARRAY[
    'translate to German: hello',
    'translate to German: goodbye'
]);

aidb.rerank_text

Scores and ranks a set of text inputs against a query using a reranking model.

Parameters

Returns

Example

SELECT * FROM aidb.rerank_text(
    'my_reranker',
    'How do I configure AIDB?',
    ARRAY[
        'AIDB requires shared_preload_libraries.',
        'Postgres supports JSON natively.',
        'Run CREATE EXTENSION aidb CASCADE to install.'
    ]
) ORDER BY logit_score DESC;

Inference configuration helper

aidb.inference_config

Builds a JSONB configuration object for runtime inference settings. Pass the result (cast to json) to aidb.decode_text(), aidb.decode_text_batch(), or use it with aidb.summarize_text_config().

All parameters are optional — omit any you don't need.

Parameters

Returns

JSONB — cast to ::json before passing to inference functions.

Examples

-- With temperature and token limit
SELECT aidb.decode_text(
    'my_llama',
    'What is machine learning?',
    aidb.inference_config(temperature => 0.5, max_tokens => 150)::json
);

-- Full configuration
SELECT aidb.decode_text(
    'my_llama',
    'Explain relativity.',
    aidb.inference_config(
        system_prompt  => 'You are a physics professor. Be precise.',
        temperature    => 0.3,
        max_tokens     => 200,
        top_p          => 0.9,
        seed           => 42,
        repeat_penalty => 1.1
    )::json
);

-- Hide reasoning tags from a thinking model
SELECT aidb.decode_text(
    'my_reasoning_model',
    'Solve: 2x + 5 = 15',
    aidb.inference_config(thinking => false)::json
);

Tool calling and structured output

Three more inference settings exist but aren't exposed as named parameters of aidb.inference_config() — build them as raw JSON instead, merged with the helper's output using || if you also need other settings. Only the llamacpp_generate provider honors these; every other provider rejects them.

-- Tool calling
SELECT aidb.decode_text(
    'my_llamacpp_chat',
    'What is the weather in Paris?',
    (aidb.inference_config(temperature => 0.2) || jsonb_build_object(
        'tools', jsonb_build_array(
            jsonb_build_object(
                'type', 'function',
                'function', jsonb_build_object(
                    'name', 'get_weather',
                    'description', 'Get current weather for a city',
                    'parameters', jsonb_build_object(
                        'type', 'object',
                        'properties', jsonb_build_object('city', jsonb_build_object('type', 'string')),
                        'required', jsonb_build_array('city')
                    )
                )
            )
        ),
        'tool_choice', 'auto'
    ))::json
);

-- Structured output
SELECT aidb.decode_text(
    'my_llamacpp_chat',
    'Extract the city from: I live in Paris.',
    jsonb_build_object(
        'response_format', jsonb_build_object(
            'type', 'json_schema',
            'json_schema', jsonb_build_object(
                'schema', jsonb_build_object(
                    'type', 'object',
                    'properties', jsonb_build_object('city', jsonb_build_object('type', 'string')),
                    'required', jsonb_build_array('city')
                )
            )
        )
    )::json
);

Model config helpers

These functions return a JSONB configuration object for the config parameter of aidb.create_model(). Each helper corresponds to a specific model provider.

aidb.embeddings_config

For openai_embeddings and any OpenAI-compatible embeddings endpoint.

Example

SELECT aidb.create_model(
    'my_embedder',
    'openai_embeddings',
    config => aidb.embeddings_config(
        model   => 'text-embedding-3-small',
        api_key => 'sk-...',
        url     => 'https://api.openai.com/v1'
    )
);

aidb.completions_config

For openai_completions and any OpenAI-compatible completions endpoint.

Example

SELECT aidb.create_model(
    'my_llm',
    'openai_completions',
    config => aidb.completions_config(
        model       => 'gpt-4o',
        api_key     => 'sk-...',
        temperature => 0.2
    )
);

aidb.max_tokens_config

Builds the max_tokens object for use with aidb.completions_config().

Example

SELECT aidb.completions_config(
    model      => 'gpt-4o',
    max_tokens => aidb.max_tokens_config(size => 1024, format => 'default')
);

aidb.bert_config

For the bert_local provider.

Example

SELECT aidb.create_model('my_bert', 'bert_local',
    config => aidb.bert_config('sentence-transformers/all-MiniLM-L6-v2'));

aidb.clip_config

For the clip_local provider.

Example

SELECT aidb.create_model('my_clip', 'clip_local',
    config => aidb.clip_config('openai/clip-vit-base-patch32'));

aidb.llama_config

For the llama_instruct_local provider.

Example

SELECT aidb.create_model('my_llama', 'llama_instruct_local',
    config => aidb.llama_config(
        'meta-llama/Llama-3.2-3B-Instruct',
        temperature => 0.5
    )
);

aidb.t5_config

For the t5_local provider.

Example

SELECT aidb.create_model('my_t5', 't5_local',
    config => aidb.t5_config('google/flan-t5-base'));

aidb.gemini_config

For the gemini provider.

Example

SELECT aidb.create_model('my_gemini', 'gemini',
    config => aidb.gemini_config('AIza...', model => 'gemini-2.0-flash'));

aidb.nim_clip_config

For the nim_clip provider (NVIDIA NIM multimodal embeddings).

Example

SELECT aidb.create_model('my_nim_clip', 'nim_clip',
    config => aidb.nim_clip_config(api_key => 'nvapi-...', model => 'nvidia/nvclip'));

aidb.nim_ocr_config

For the nim_ocr provider (NVIDIA NIM OCR).

Example

SELECT aidb.create_model('my_nim_ocr', 'nim_ocr',
    config => aidb.nim_ocr_config(api_key => 'nvapi-...'));

aidb.nim_reranking_config

For the nim_reranking provider.

Example

SELECT aidb.create_model('my_reranker', 'nim_reranking',
    config => aidb.nim_reranking_config(
        api_key => 'nvapi-...',
        model   => 'nvidia/nv-rerankqa-mistral-4b-v3'
    )
);

aidb.openrouter_chat_config

For the openrouter_chat provider.

Example

SELECT aidb.create_model('my_or_chat', 'openrouter_chat',
    config => aidb.openrouter_chat_config(
        'anthropic/claude-3-5-haiku',
        api_key => 'sk-or-...'
    )
);

aidb.openrouter_embeddings_config

For the openrouter_embeddings provider.

Example

SELECT aidb.create_model('my_or_embedder', 'openrouter_embeddings',
    config => aidb.openrouter_embeddings_config(
        'mistral/mistral-embed',
        api_key => 'sk-or-...'
    )
);