Operations reference v7

Reference for pipeline error log functions. For guide-style documentation, see Pipeline error log.

Error log functions

Each pipeline maintains an error log table at {source_schema}.pipeline_{pipeline_name}_errors. These functions provide a consistent interface for querying and managing that log.


aidb.get_error_logs

Returns error log entries for a pipeline, with optional filters.

Parameters

ParameterTypeDefaultDescription
p_pipeline_nameTEXTRequiredName of the pipeline.
p_source_idTEXTNULLFilter to a specific source record ID.
p_pipeline_stepSMALLINTNULLFilter to a specific pipeline step number.
p_error_categoryTEXTNULLFilter by category: RecordTemporary, RecordPermanent, PipelineTemporary, or PipelinePermanent.
p_limitINTEGERNULLMaximum number of rows to return.
p_offsetINTEGERNULLNumber of rows to skip (for pagination).

Returns

ColumnTypeDescription
idbigintError log entry ID
source_idtextSource record that failed
part_idsbigint[]Part identifiers (for chunked records)
pipeline_stepsmallintStep number where the failure occurred
step_operationtextOperation name (for example, ParsePdf, ChunkText)
error_messagetextError description
error_categoryaidb.ErrorBlockingError category
failed_attimestamptzTimestamp of the failure
retry_countintegerNumber of retry attempts
last_retry_attimestamptzTimestamp of the last retry

Examples

-- All errors for a pipeline
SELECT * FROM aidb.get_error_logs('my_pipeline');

-- Filter by source record
SELECT * FROM aidb.get_error_logs('my_pipeline', p_source_id => 'doc_42');

-- Filter by step and category
SELECT * FROM aidb.get_error_logs(
    'my_pipeline',
    p_pipeline_step  => 1::SMALLINT,
    p_error_category => 'RecordPermanent'
);

-- Paginate
SELECT * FROM aidb.get_error_logs('my_pipeline', p_limit => 20, p_offset => 40);

aidb.get_error_log_summary

Returns aggregated error counts for a single pipeline, grouped by step, operation, and category.

Parameters

ParameterTypeDescription
p_pipeline_nameTEXTName of the pipeline.

Returns

ColumnTypeDescription
pipeline_stepsmallintStep number
step_operationtextOperation name
error_categorytextError category
error_countbigintNumber of errors in this group
latest_failed_attimestamptzMost recent failure timestamp

Example

SELECT * FROM aidb.get_error_log_summary('my_pipeline');
Output
 pipeline_step | step_operation | error_category  | error_count | latest_failed_at
---------------+----------------+-----------------+-------------+----------------------------
             1 | ParsePdf       | RecordPermanent |           2 | 2026-04-10 14:31:58.456+00
             2 | ChunkText      | RecordPermanent |           1 | 2026-04-10 14:32:01.123+00
(2 rows)

aidb.get_all_error_summaries

Returns the same rollup as aidb.get_error_log_summary() but across all pipelines at once.

Returns

ColumnTypeDescription
pipeline_nametextPipeline name
pipeline_stepsmallintStep number
step_operationtextOperation name
error_categorytextError category
error_countbigintNumber of errors in this group
latest_failed_attimestamptzMost recent failure timestamp

Example

SELECT * FROM aidb.get_all_error_summaries();
Output
 pipeline_name | pipeline_step | step_operation | error_category    | error_count | latest_failed_at
---------------+---------------+----------------+-------------------+-------------+----------------------------
 my_pipeline   |             1 | ParsePdf       | RecordPermanent   |           2 | 2026-04-10 14:31:58.456+00
 pdf_ingester  |             1 | ParsePdf       | PipelinePermanent |           1 | 2026-04-10 15:00:12.000+00
(2 rows)

aidb.clear_error_logs

Deletes specific error log entries by ID. Review entries with aidb.get_error_logs() before clearing.

Parameters

ParameterTypeDescription
p_pipeline_nameTEXTName of the pipeline.
p_error_idsBIGINT[]Array of error log entry IDs to delete.

Returns

BIGINT — the number of rows deleted.

Example

SELECT aidb.clear_error_logs('my_pipeline', ARRAY[1, 3]);
Output
 clear_error_logs
------------------
                2
(1 row)

aidb.requeue_pipeline_errors

Introduced in 7.5.0. Deletes the targeted record-level error log entries and marks their source records dirty on the pipeline's state surface. The next normal run re-processes them; the call itself does no processing. See Requeuing failed records for the workflow.

Parameters

ParameterTypeDescription
p_pipeline_nameTEXTName of the pipeline.
p_error_idsBIGINT[]Array of error log entry IDs to requeue.

Returns

One row per input ID, ordered by ascending error_id.

ColumnTypeDescription
error_idbigintError log entry ID from the input array.
actiontextrequeued, skipped_pipeline_level, or not_found (see below).
  • requeued: record-level row deleted and source marked dirty. Table sources enqueue a state event; volume sources set a one-shot retry_requested_at marker.
  • skipped_pipeline_level: entry has a NULL source_id and is left in place. Fix the underlying issue and re-run, or dismiss with aidb.clear_error_logs.
  • not_found: ID not present in the error log, or the pipeline has no error log table.

Concurrent requeues on overlapping IDs serialize on per-row FOR UPDATE locks; non-overlapping ID sets proceed in parallel. Background pipelines drain requeued rows automatically on the next worker tick. Disabled or Live pipelines need a follow-up aidb.run_pipeline call.

Example

SELECT * FROM aidb.requeue_pipeline_errors('my_pipeline', ARRAY[1, 2, 99]);
Output
 error_id | action
----------+------------------------
        1 | requeued
        2 | requeued
       99 | not_found
(3 rows)

Error categories

CategoryScopeBlocks pipeline?Description
RecordTemporarySingle recordNoTransient failure for one record. May succeed on retry.
RecordPermanentSingle recordNoPermanent failure for one record (for example, corrupt data).
PipelineTemporaryEntire stepYesTransient failure that stopped the step (for example, service outage). May resolve on retry.
PipelinePermanentEntire stepYesPermanent failure that stopped the step (for example, invalid model config).