Sunbird cQube
  • πŸ“–KNOW ABOUT CQUBE
    • What is cQube & what does it solve
      • Business case
      • cQube ed
      • Design Principles
    • cQube adoptions
    • Discuss more about cQube
  • πŸ‘TRY CQUBE
    • cQube on Gitpod
  • πŸŒ…WHAT IS NEXT IN CQUBE
    • cQube Roadmap
  • πŸ’»TECHANICAL OVERVIEW
    • Architecture
    • Design Principles
    • Key Components
    • Details of microservices
  • πŸ‘©β€πŸ’»Get started on cQube
    • Suggested Team Structure
    • Hardware Requirements
    • Prerequisites Checklist
    • Checking pre existing ports
    • Instance Creation
    • Copying SSL Certificate
  • πŸ›ƒUSE CQUBE
    • How can I install cQube?
      • High level understanding of deployment
      • Oracle Installation
      • AWS Installation
      • SDC Installation
      • Azure Installation
    • How to prepare the data
      • Schema
      • How to prepare schemas for dimension files
      • How to prepare schemas for event files
      • Examples of dimension & event files
      • All cQube schemas used for VSK
    • Ingesting the data
      • High-level understanding of ingestion
      • Steps to ingest schema
      • Steps to ingest data files
        • Upload the .csv data file for state specific programs using ingestion API
        • API to upload starter pack data files for NVSK programs
        • Get file status API
        • Scheduled API
      • Error during ingestion
        • Error Monitoring
        • Common errors in data files during ingestion
      • Processor group name
    • Adapter details
    • Postman details
    • Processing of data
      • Data Processing using CLI command
      • API Details for Nifi-Rest
      • Nifi section
    • Visualizing the data
      • High level understanding of how visualizations work in cQube
      • Programs and reports out-of-the-box
      • Enhance /Customize cQube
        • Available customizations
          • Changing Dashboard Logos and Headers
          • Changing Program Name, Icon and Side Menu Sequence
          • Adding a new KPI
          • Adding a Map KPI into dashboard ms
          • Table Drill Down Customization
          • Adding a Scatter Plot KPI into dashboard ms
          • Configure default date range across app/specific report
        • How to add a New Indicator
        • How to add a new report in an existing program
        • How to add a new program (end to end)
    • Additional Features
      • Public/Private dashboards
      • Role based access control
      • Saving geographical preferences
      • Admin Panel
        • Data Debugger
        • Schema Generator
        • System Monitoring
    • Adding Users
      • Adding an individual user
      • Adding bulk users
  • πŸ–₯️MONITOR cQUBE
    • Infra health monitoring
    • Usage monitoring
  • πŸ”ŽQA testing
    • Testing approaches & activities
    • Manual & Automated testing
    • Functional Testing
      • Smoke Testing
      • Functional tests
      • Regression Testing
      • System Testing
    • Non Functional Testing
      • Performance Testing
        • Load Testing
        • Volume Testing
        • Performance testing results
    • Test for One-Step Installation
    • Test for Ingestion
    • Test for nifi processing
    • Test for UI Application
    • Test for KPIs
  • β˜€οΈDEPLOYMENT PROCESS
    • State List
    • AWS Deployment
    • SDC Deployment
    • Adapter Details During the Processing
  • 🈴UPGRADING TO LATEST VERSION
    • How can I upgrade cQube to the latest release
  • πŸ†˜Common issues and their solutions
    • Deployment & ingestion related issues & their solutions
  • ⏱️Standard Operating Procedure
    • Reporting a Bug
    • Protocol for issue reporting & resolution
    • Suggesting Enhancements
    • Raising a PR
  • ❓Frequently Asked Questions
    • Running List
  • πŸ§‘β€πŸ«πŸ§‘πŸ« Recording of trainings
    • Link to the training videos
  • 🧠Key Terms & Concepts
    • Definitions
  • πŸš€cQube Release Notes
    • cQube - Release V 5.0.5
    • cQube - Release V 5.0.3
    • cQube - Release V 5.0.2
    • cQube - Release V 5.0.1
    • cQube - Release V 5.0
    • cQube - Release V 4.1-beta
    • cQube - Release V 4.0-beta
    • cQube - Release V 4.0-alpha
    • cQube - Release V 3.7
    • cQube - Release V 3.6
    • cQube - Release V 3.5
    • cQube - Release V 3.4
    • cQube - Release V 3.3
    • cQube - Release V 3.2
    • cQube - Release V 3.1
    • cQube - Release V 3
    • cQube - Release V 2
    • cQube - Release V 1.13 and V 1.13.1
    • cQube - Release V 1.12 and V 1.12.1
    • cQube - Release V 1.11
    • cQube - Release V 10 and V 10.1
    • cQube - Release V 1.9
    • cQube - Release V 1.8 and V 1.8.1
    • cQube - Release Notes V 1.7
    • cQube - Release Notes V 1.6 and V 1.6.1
    • cQube - Release Notes V 1.5
    • cQube - Release Notes V 1.4
    • cQube - Release Notes V 1.3
    • cQube - Release Notes V 1.2 and V 1.2.1
    • cQube - Release Notes V 1.1
    • cQube - Release Notes V 1.0
  • πŸ“‚cQube V 4.1 - Beta
    • Sunbird cQube Overview
    • cQube Product Description
    • Listen to Experts (Youtube)
    • Software Requirements
    • Acronyms
    • cQube Software Architecture
    • AWS - Network Architecture
      • Hardware requirements
      • Data Storage Locations
    • Security Implementations
    • Prerequisites for Installation process
    • New Use-Case Creation
    • cQube Setup & configuration
    • Base Installation steps
    • Base Upgradation steps
    • Workflow Installation steps
    • Workflow Upgradation steps
    • Laptop/Desktop Installation
      • Base Installation
      • Workflow Installation
      • Mock Data Processing
    • Ad-hoc analysis
    • Workflow process
    • Emission Process
    • cQube ER Diagrams
    • Data Validation after Ingestion
    • User Authentication Process
    • Admin Login Process
    • Admin Features
    • cQube Datasource Configuration
    • cQube data replay process
    • S3 Partitioning
    • Reports
    • Troubleshooting Issues
      • Data Processing-NIFI Issues
      • Data Processing-PostgreSQL Issues
      • Data Emission Issues
      • Angular & Node Issues
    • FAQs
    • Discuss
    • Report
    • Source Code
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. cQube V 4.1 - Beta
  2. Troubleshooting Issues

Data Processing-PostgreSQL Issues

Where

Why

How to resolve

Error message

Error Location

Error type

Error identification (Cause)

Probable resolution

ERROR: duplicate key value violates unique constraint "_pkey"

Postgres logs in console

Duplicates record are in Input file even after validation (Primary key constraint violation of table XYZ)

When the primary key has same set of values for more than 2 records, the error will occur.

The error mostly occurs while updating to aggregation tables, due to inconsitency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.

ERROR: null value in column "XYZ" violates not-null constraint

Postgres logs in console

Null value records are in Input file even after NULL validation for column XYZ

When an required field has null value in the data file.

Null values need to be removed from the data file

ERROR: deadlock detected: DETAILS: Process 21601 waits for ShareLock on transaction 89371; blocked by process 21603

Postgres logs in console

DEADLOCK situation because process 21601 is waiting until process 21603 is completed

When simultaneous update is happening to the same record by two process.

Check if there are two Job running simultaneously, if so stop one of them.

ERROR: could not extend file "base/16384/75771.2": No space left on device

Postgres logs in console

Not enough disk space in system

When an operation performed doesn't have enough disk space to accomplish the task due to lack of memory.

Check if there is enough disk space to accomplish the task.

"ERROR: out of memory DETAIL: Failed on request of size 324589128."

Postgres logs in console

Not enough main memory

When we consume more memory than available on our machine we can see out of memory errors within our Postgres logs, or in worse cases the OOM killer can start to randomly kill running processes to free up memory. An out of memory error in Postgres simply errors on the query you’re running, where as the the OOM killer in linux begins killing running processes which in some cases might even include Postgres itself

Reduce the emission file size to avoid the issue.

ERROR: value too long for type character varying(100)

Postgres logs in console

Data field exceeds size exceeds the column size

When the field has too long to accommodate within a column, we get this error

Correct the data field which exceeds the size.

ERROR: value too long for type integer

Postgres logs in console

Data field exceeds size exceeds the column size

When the field has too long to accommodate within a column, we get this error

Correct the data field which exceeds the size.

ERROR: column "" does not exist at character

Postgres logs in console

Data field doesn't exist

When the data fields are not according to the emission specification

Correct the header in the emitted file

ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification

Postgres logs in console

Inconsistency/mapping issues in the data

When mapping of the data has issues, we may get this issue

The error mostly occurs while updating to aggregation tables, due to inconsistency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.

FATAL: terminating connection due to administrator command

Postgres logs in console

When the data base was restarted

If the data base restarts during the running process, we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.

LOG: received fast shutdown request

Postgres logs in console

When the data base was restarted

If the data base restarts during the running process, we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.

ERROR: relation "" does not exist at character

Postgres logs in console

When the relation doesn't exist

If the relation doesn't exist, will get this issue

PreviousData Processing-NIFI IssuesNextData Emission Issues

Last updated 3 years ago

Was this helpful?

πŸ“‚