How to add a new program (end to end)
Note: These steps are for adding a new program
Last updated
Note: These steps are for adding a new program
Last updated
Design the event and dimension grammar files for your program. This grammar defines the structure and relationships between different data elements, such as event attributes and dimensions.
Event grammar: The event grammar defines the structure of the events to be ingested into the system. It includes information such as the event name, its attributes (e.g. event timestamp, user ID, etc.), and data types.
Dimension grammar: The dimension grammar defines the dimensions, or attributes, that will be used to group and filter events. It includes information such as the dimension name, its attributes (e.g. dimension value, category, etc.), and data types.
Config.json provides directive to processing engine on how to process the grammar and data files, where they are stored, etc. it should be placed in /ingest/<VSK or NVSK>/
globals : global settings
dimensions : It includes, file format, location and namespace
programs : program specific settings like namespace, description, input location, output location, dimensions which should be considered etc.
Only programs mentioned under config.json will be considered for ingestion.
Convert the Event and Dimension Grammar to JSON format as given in the below example
Example: if there is a event with two columns
school_id and school_name
Below would be the JSON object
{
"program": "school-attendance",
"input": {
"type": "object",
"properties": {
"school_id": {
"type": "string",
"shouldNotNull": true
},
"school_name": {
"type": "string",
"shouldNotNull": true
}
}
}
Dimension grammar : Use the /spec/dimension API to upload the dimension grammar
Place the dimension grammar file as -dimension.grammar.csv in processing-ms/impl/c-qube/ingest/VSK or NVSK/dimensions/ folder as seen below or as per the config.json settings
Contents of the district-dimension.grammar.csv
Event schema can be ingested in 2 ways:
Though API Method
Event spec API: spec/event
Through YARN CLI
YARN CLI INGEST
This command is used to ingest Event and Dimension Spec and Dimension Data based on the configuration we have
Create a <new-program> folder in processing-ms/impl/c-qube/ingest/<state_code>/programs\
Place the event grammar file as <xyz>-event.grammar.csv in processing-ms/impl/c-qube/ingest/<state_code>/programs/<new-program> folder as seen below (example for diksha program)
This would be based on default config.json or else it should be as per config.json
Contents of the district-dimension.grammar.csv
This process may involve uploading the CSV file or connecting to the database directly. Following links will provide more details on the process.
Add event data on a regular frequency: To keep your program up-to-date, continuously add new event data to the platform. Determine an appropriate frequency for updating the data, such as daily, weekly, or monthly.w Ensure that the new data is compatible with the existing event and dimension schema.
Create a new module manually or using cli common ng g m <module-name> in the below folder in the angular code base: src/app/views
Add the folders config and pages, where config holds a configuration file for the reports in this program and pages contains the code of all the tabs/reports.
Add the routing to the program component in the app-routing module using lazy-loading
Add the configuration of this program to the main_matric.csv present in the root folder of query builder.( To add this program to side nav and summary statistics) as shown below.
Next add the programId filled in the above file to another file Program Selector in the query Builder root folder itself. The Program Selector file takes input from the ansible during installation to update which programs to be visible in the dashboard. If a new program is added after installation manually fill the programId and set the status true for that program to be visible in the dashboard.
Configuration file of the newly added program should be added to the below file:
src/app/core/config/configMapping.ts
All the configuration files are mapped at a place for easy access , This object will be used to map each row in main_metric with program wise configuration files with the respective programIds.
Later one can add the desired KPI to the above program created following the below steps
