How do you document and maintain data load script vs data model?

1 Data load script basics

The data load script is the code that you write in the script editor to load data from various sources into QlikView. It can include commands, functions, variables, expressions, and comments. The data load script defines the structure and logic of your data model, which is the representation of the data in QlikView's memory. You can access and modify the data load script by pressing Ctrl+E or clicking on the Edit Script button in the toolbar.

2 Data model basics

The data model is the result of executing the data load script. It consists of tables, fields, and associations that store and relate your data in QlikView. You can view and analyze the data model by pressing Ctrl+T or clicking on the Table Viewer button in the toolbar. The data model determines how you can use the data in your charts, tables, and other objects in your QlikView application.

3 Documenting your data load script

Documenting your data load script is essential for understanding, debugging, and maintaining your code. It can also help you communicate your logic and decisions to other developers or users. You can document your data load script by using comments, variables, and labels. Comments are lines of text that start with // or /* and end with */. They are ignored by QlikView but can explain the purpose or function of a code block. Variables are placeholders for values or expressions that you can reuse or change throughout your script. They can make your code more readable and flexible. Labels are names that you assign to tables or fields in your script. They can help you identify and reference them in your data model.

4 Documenting your data model

Documenting your data model is also important for understanding, validating, and optimizing your data structure and relationships. It can also help you design and create your QlikView objects and expressions. You can document your data model by using labels, aliases, and annotations. Labels are the same as in the data load script, but you can also change them in the table viewer by double-clicking on a table or field name. Aliases are alternative names that you can assign to fields in your script. They can help you avoid conflicts or confusion between fields with the same name from different sources. Annotations are notes that you can add to tables or fields in the table viewer by right-clicking on them and selecting Add Annotation. They can provide additional information or context about the data.

5 Maintaining your data load script

Maintaining your data load script means keeping it up to date, error-free, and efficient. You can use various tools and techniques to achieve this, such as debugging, incremental loading, and modularization. Debugging is the process of finding and fixing errors or bugs in your script. You can use the debug mode, the syntax check, and the log file to help you with this. Incremental loading is the method of loading only new or changed data instead of reloading the entire data set. This can improve the performance and speed of your script. Modularization is the practice of dividing your script into smaller and reusable parts that you can store in separate files and call from the main script. This can make your script more organized and manageable.

6 Maintaining your data model

Maintaining your data model means ensuring that it is accurate, consistent, and optimal. You can use various tools and techniques to achieve this, such as data quality checks, data transformation, and data modeling best practices. Data quality checks are the steps that you take to verify and correct the quality of your data, such as missing values, duplicates, outliers, or inconsistencies. You can use the data profiling tool, the information density tool, or custom expressions to help you with this. Data transformation is the process of modifying or enhancing your data to suit your needs, such as concatenating, joining, splitting, renaming, or deriving fields. You can use the script editor, the table viewer, or the expression editor to help you with this. Data modeling best practices are the guidelines that you follow to create a robust and efficient data model, such as avoiding synthetic keys, circular references, or unnecessary fields. You can use the table viewer, the system fields, or the document analyzer to help you with this.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?