# Data Processing-NIFI Issues



|                                                                                                                                    | **Where**                                        |                                                      | **Why**                                                    | **How to resolve**                                                                                                                                                                                                                                                                                                                         |
| ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ---------------------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Error message**                                                                                                                  | **Error Location**                               | **Error type**                                       | **Error identification (Cause)**                           | **Probable resolution**                                                                                                                                                                                                                                                                                                                    |
| Error student\_attendance.txt is not valid zip format.                                                                             | Installation/logs/nifi-app.log                   | checking the extension                               | If file is not valid zip                                   | <p>"1. Check the file is emitted in the .Zip format or not<br> 2. Check the file is in rar or any other then zip format.<br> 3. If the above 2 are correct then contact cQube team."</p>                                                                                                                                                   |
| Error while extracting the header/columns from csv file (error)                                                                    | Installation/logs/nifi-app.log                   | Data validation                                      | Error while extracting columns from file.                  | <p>1. Check the delimiter of the csv file, delimiter should be <br> "|" except for Diksha Datasource. Diksha delimiter " , "</p>                                                                                                                                                                                                           |
| Error - columns did not match                                                                                                      | Installation/logs/nifi-app.log                   | Data validation                                      | If the columns do not match with the requirement           | <p>"1. Check the column names are accoding to the cQube doc.<br> 2. Rename the columns as per the cQube doc. <br> 3. Clear the Nifi Queue [ Righ click on canvas , then click Empty all queues]<br> 4. Re emit the file with correct columns."</p>                                                                                         |
| Error -  filename has invalid data type: invalid record=, Total records=                                                           | Installation/logs/nifi-app.log                   | Data validation                                      | If the columns datatype doesn't match with the requirement | <p>"1. Check the column datatypes are accoding to the cQube doc.<br>2. Rename the columns as per the cQube doc.<br> 3. Clear the Nifi Queue [ Righ click on canvas , then click Empty all queues] <br>4. Re emit the file with correct columns."</p>                                                                                       |
| Error while checking mirror data                                                                                                   | Installation/logs/nifi-app.log                   | Data validation                                      | when unexpected characters found in data                   | <p>"1. Check for the special characters or invalid csv format file. <br>2. If special characters or invalid csv format exists, then clear the Nifi Queue [ Righ click on canvas , <br>then click Empty all queues] and delete the staging / temp tables. <br>3. If the above steps doesn't solve the issue, then contact cQube team."</p>  |
| Error while selecting unique data                                                                                                  | Installation/logs/nifi-app.log                   | Data validation                                      | when unexpected characters found in data                   | <p>"1. check for the special characters or invalid csv format file.<br> 2. If special characters or invalid csv format exists, then clear the Nifi Queue<br> [ Righ click on canvas , then click Empty all queues] and delete the staging / temp tables.<br> 3. If the above steps doesn't solve the issue, then contact cQube team."</p>  |
| Error while getting null values                                                                                                    | Installation/logs/nifi-app.log                   | Data validation                                      | when unexpected characters found in data                   | <p>"1. Check for the special characters or invalid csv format file <br>.2. If special characters or invalid csv format exists, then clear the Nifi Queue <br>[ Righ click on canvas , then click Empty all queues] and delete the staging / temp tables.<br>3. If the above steps doesn't solve the issue, then contact cQube team."</p>   |
| Error while fetching the year and month from temp table.                                                                           | Installation/logs/nifi-app.log                   | Run time Error                                       | When DB is down/ not in running state                      | <p>"1. Check the Db status.<br> 2. If it is stopped , start it .<br> 3. If the problem persist after doing above steps , Contact cQube team."</p>                                                                                                                                                                                          |
| Error while executing  query.                                                                                                      | Installation/logs/nifi-app.log                   | Run time Error                                       | When DB is down/ not in running state                      | <p>"1. Check the Db status. <br>2. If it is stopped , start it .<br> 3. If the problem persist after doing above steps , Contact cQube team."</p>                                                                                                                                                                                          |
| <p>Failed to perform validation due to <br>java.lang.OutOfMemoryError: Java heap space</p>                                         | Installation/logs/nifi-app.log                   | Run time Error - Out of Heap Memory                  | when Heap memory is consumed full.                         | 1. Contact cQube team.                                                                                                                                                                                                                                                                                                                     |
| Amazon S3 due to Please reduce your request rate. (Service: Amazon S3; Status Code: 503; Error Code: SlowDown;                     | Installation/logs/nifi-app.log                   | Run time Error                                       | When large number of file writes happens at same time      | 1. Contact cQube team.                                                                                                                                                                                                                                                                                                                     |
| Error while checking diksha summary rollup request status from table, Error=                                                       | Installation/logs/nifi-app.log                   | Run time Error                                       | When DB is down/ not in running state                      | <p>"1. Check the Db status. <br>2. If it is stopped , start it . <br>3. If the problem persist after doing above steps , Contact cQube team."</p>                                                                                                                                                                                          |
| Invoke HTTP Routing to Failure due to exception: Failed to connect to :8082: java.net.ConnectException: Failed to connect to :8082 | Installation/logs/nifi-app.log                   | Configuration Error                                  | Diksha API Call                                            | <p>"1. Check whether API is working ,test it  using curl or postman.<br> 2. If it is not working , contact the respective API team. <br>3.If the above steps doesn't solve the issue, contact cQube team."</p>                                                                                                                             |
| InvokeHTTP  Error codes 4XX                                                                                                        | Installation/logs/nifi-app.log                   | Configuration Error                                  | Diksha API Call                                            | <p>"1. Check whether ur able make the api call using curl or postman. <br>2. If it is not working , contact the respective API team.<br> 3.If the above steps doesn't solve the issue, contact cQube team."</p>                                                                                                                            |
| "Error while downloading the file.File url = "                                                                                     | Installation/logs/nifi-app.log                   | Run time Error                                       | Diksha API Call                                            | <p>"1. Check the file expiry time from diksha_api_meta table.<br> 2. If file ur is expired , then clear the Nifi queues. <br> 3. delete the today's request and re run the diksha processor.<br> 4. If problem persist after performing above 3 steps, Contact cQube team."</p>                                                            |
| NO DATA FOUND - DATASET (PROGRESS-EXHAUST) for Request:                                                                            | Installation/logs/nifi-app.log                   | Configuration Error /  API                           | Diksha API Call                                            | <p>"1. Check whether you have emitted batch ids file.<br> 2. Check whether Diksha API is returning data using CURL / POSTMAN. <br> 3. If problem persist after performing above 2 steps, Contact cQube team."</p>                                                                                                                          |
| NO DATA FOUND - DATASET (SUMMARY-ROLLUP) for Request:                                                                              | Installation/logs/nifi-app.log                   | Configuration Error /  API                           | Diksha API Call                                            | <p>"1. Check whether Diksha API is returning data using CURL / POSTMAN. <br>2. If problem persist after performing above the step 1, Contact cQube team."</p>                                                                                                                                                                              |
| Routing to failure : unable to unmarshal JSON to an Object                                                                         | Installation/logs/nifi-app.log                   | Run time Error                                       | when unexpected characters found in output data            | <p>"1. check for the special characters or invalid csv format file.<br> 2. If special characters or invalid csv format exists, then clear the Nifi Queue <br> [ Righ click on canvas , then click Empty all queues] and delete the staging / temp tables. <br>3. If the above steps doesn't solve the issue, then contact cQube team."</p> |
