Error monitoring post ingestion
Describes how the errors during data ingestion in cQube V 5.0 can be monitored and rectified
There may be some errors in the data that is being ingested by the cQube adopter into cQube. All the errors during ingestion can be monitored in cQube.
An error file will be created during ingestion, which will store all the error records during the ingestion process and will be uploaded to the appropriate cloud storage. The user can login to the respective cloud storage and download the file to take a look at the error records present in the CSV.
Following steps need to be followed to access the error file:
Step-1: Go to the cloud storage and locate the bucket / container named as cQube-edu.
Minio Storage (On-premise)
Azure Storage
AWS Storage
Step-2: Inside the bucket / container, there will be multiple folders for different processing stages. In order to access the ingestion error files, select the ingestion_error folder.
Step-3: The ingestion_error folder will contain folders for each program, the name of the folder will be same as the <program_name>.
Minio Storage (On-premise)
Azure Storage
AWS Storage
Step-4: If there are errors during ingestion, the <program_name> folder will have a folder named as the current date.
Minio Storage (On-premise)
Azure Storage
AWS Storage
Step - 5: The user can access this folder to see the error files and download the same to view it in the format of CSV.
Minio Storage (On-premise)
Azure Storage
AWS Storage