Data Debugger
It analyzes the data and reports the user about the errored records in the raw data file of an event or dimension. User has to select an event or dimension schema which is needed to validate the records of uploaded events or dimension files. After that debugger will analyze the uploaded data against the selected schema and it reports back to the user with the errored records. This feature helps the user to analyze the data before ingesting the data for data processing.
Data Debugger analyzes the fed data and reports and identifies the errors in them, if any, in an event or dimension files.
How it works:
To validate the raw data of either dimension or an event, first:
Upload the grammar file for the dimension or event that you want to validate.
Validate the grammar file.
Validate the raw data against the uploaded grammar file.
Steps to validate the Raw Data file:
(Please note - The raw data file should be in CSV format.)
Check 'Dimension Grammar' is present in the dropdown menu.
Upload the dimension file.
Click on the 'Validate Grammar' Tab.
Now, the system will show errors if any, in the screen for the uploaded file. If the file does not contain any errors, the user will be directed to the next screen.
Here are more information about each step above.
Upload the grammar file for the dimension or event that you want to validate.
The grammar file is a CSV file that defines the structure of the data that you want to validate. It contains information about the different fields in the data, such as the field name, data type, and constraints.
Following screenshot of validate grammar of data debugger
Following is the sample grammar file block dimension:
Note: Include primary key if it is a dimension grammar file.
Validate the grammar file.
Once you have uploaded the grammar file, you need to validate it to make sure that it is correct. The validation process will check for errors in the grammar file, such as missing fields or invalid data types.
Here is a sample screenshot of a table that might be displayed to the user:
If the uploaded grammar file is valid, then a Validate Data button will be enabled for the user. Using the Validate Data button, the user can go into the validation step.
Following screenshot will be displayed to the user
Validate the raw data against the uploaded grammar file.
Once the grammar file has been validated, you can validate the raw data against it. The validation process will check for errors in the raw data, such as missing fields or invalid values.
If there are no errors in the raw data file, then the validation will be successful. Otherwise, the validation will fail, and you will be given a report of the errors.
Following screenshot for validate the data
If there are any errors in the uploaded data, then the errors will be reported to the user in the form of a table. Following is a screenshot of the error screen.
If the file is valid, then a alert message will be displayed to the user, which means that the data is ready to be ingested. Following is the screenshot:
Steps:
Grammar Validation
The user must first choose the grammar file type to validate, either dimension or event. Then, the user must upload the grammar file. After uploading the file, the debugger analyzes the grammar and reports any errors to the user. If there are no errors, the debugger shows a message indicating that the schema is valid.
Data Validation
The user must first choose the data file type to validate, either dimension or event. Then, the user must upload the raw data file. After uploading the file, the debugger analyzes the data against the grammar uploaded in the previous step and reports any errors to the user. If there are no errors, the debugger shows a message indicating that the schema is valid.
Last updated
