Data Processing-PostgreSQL Issues

Where

Why

How to resolve

Error message

Error Location

Error type

Error identification (Cause)

Probable resolution

ERROR: duplicate key value violates unique constraint "_pkey"

Postgres logs in console

Duplicates record are in Input file even after validation (Primary key constraint violation of table XYZ)

When the primary key has same set of values for more than 2 records, the error will occur.

The error mostly occurs while updating to aggregation tables, due to inconsitency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.

ERROR: null value in column "XYZ" violates not-null constraint

Postgres logs in console

Null value records are in Input file even after NULL validation for column XYZ

When an required field has null value in the data file.

Null values need to be removed from the data file

ERROR: deadlock detected: DETAILS: Process 21601 waits for ShareLock on transaction 89371; blocked by process 21603

Postgres logs in console

DEADLOCK situation because process 21601 is waiting until process 21603 is completed

When simultaneous update is happening to the same record by two process.

Check if there are two Job running simultaneously, if so stop one of them.

ERROR: could not extend file "base/16384/75771.2": No space left on device

Postgres logs in console

Not enough disk space in system

When an operation performed doesn't have enough disk space to accomplish the task due to lack of memory.

Check if there is enough disk space to accomplish the task.

"ERROR: out of memory DETAIL: Failed on request of size 324589128."

Postgres logs in console

Not enough main memory

When we consume more memory than available on our machine we can see out of memory errors within our Postgres logs, or in worse cases the OOM killer can start to randomly kill running processes to free up memory. An out of memory error in Postgres simply errors on the query you’re running, where as the the OOM killer in linux begins killing running processes which in some cases might even include Postgres itself

Reduce the emission file size to avoid the issue.

ERROR: value too long for type character varying(100)

Postgres logs in console

Data field exceeds size exceeds the column size

When the field has too long to accommodate within a column, we get this error

Correct the data field which exceeds the size.

ERROR: value too long for type integer

Postgres logs in console

Data field exceeds size exceeds the column size

When the field has too long to accommodate within a column, we get this error

Correct the data field which exceeds the size.

ERROR: column "" does not exist at character

Postgres logs in console

Data field doesn't exist

When the data fields are not according to the emission specification

Correct the header in the emitted file

ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification

Postgres logs in console

Inconsistency/mapping issues in the data

When mapping of the data has issues, we may get this issue

The error mostly occurs while updating to aggregation tables, due to inconsistency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.

FATAL: terminating connection due to administrator command

Postgres logs in console

When the data base was restarted

If the data base restarts during the running process, we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.

LOG: received fast shutdown request

Postgres logs in console

When the data base was restarted

If the data base restarts during the running process, we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.

ERROR: relation "" does not exist at character

Postgres logs in console

When the relation doesn't exist

If the relation doesn't exist, will get this issue

Last updated