ç™»å½•æŸ¥çœ‹æ›´å¤šå†…å®¹

å…¨éƒ¨

What are some best practices or tips for writing clear and concise data transformation formulas?

ç”±äººå·¥æ™ºèƒ½å’Œé¢†è‹±ç¤¾åŒºæä¾›æŠ€æœ¯æ”¯æŒ

Data transformation formulas are essential for manipulating and analyzing data in various formats and sources. They allow you to perform calculations, filter, join, aggregate, and reshape data according to your needs and goals. However, writing clear and concise data transformation formulas can be challenging, especially when dealing with complex or large datasets. In this article, we will share some best practices or tips for writing data transformation formulas that are easy to read, understand, and maintain.

æ¤æ–‡ç« ä¸çš„ä¸šç•Œè¾¾äºº

ç”±ç¤¾åŒºä»Ž 8 æ¡å†…å®¹ä¸ç²¾é€‰ã€‚äº†è§£æ›´å¤š

1 Use consistent naming conventions

One of the first steps to writing clear and concise data transformation formulas is to use consistent naming conventions for your variables, columns, tables, and functions. Naming conventions help you avoid confusion, errors, and duplication in your formulas. They also make your formulas more readable and searchable for yourself and others. For example, you can use camelCase, snake_case, or PascalCase for your variables, and prefix your tables with a source name or abbreviation.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

åŠ è½½æ›´å¤šå†…å®¹

2 Comment your formulas

Another best practice for writing clear and concise data transformation formulas is to comment your formulas with explanatory notes or descriptions. Comments help you document the logic, purpose, and assumptions behind your formulas. They also make your formulas easier to debug, modify, and reuse in the future. For example, you can use // or /* */ to comment your formulas in SQL, or # or ''' ''' to comment your formulas in Python.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Michael Huang

Crafty STEM-ist | Secretly a Consultant | Data Analyst, Sheets & Queries | IT Support/Automation | SaaS Lover | Supply Chain Logistics | Six Sigma / Lean | Etcetera!
ä¸¾æŠ¥å†…å®¹
The clearer and more useful the comment, the merrier the design! However, when commenting or adding explanatory notes, it is highly advisable to add notes, where a particular section of code needs remarks or attention. One might start with outline first, using comments. Then, as the code progresses, clean up codes and ensure the work is clearly presented. That will reduce redundancy and amount of work needed.

å·²ç¿»è¯‘

èµž

3 Break down complex formulas

Sometimes, you may need to write complex formulas that involve multiple steps, conditions, or calculations. In such cases, it is advisable to break down your formulas into simpler and smaller parts. This way, you can improve the readability, accuracy, and performance of your formulas. You can also test and troubleshoot each part separately before combining them into a final formula. For example, you can use intermediate variables, subqueries, or functions to break down your complex formulas.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Vamsee Narahari

Problem Solver | Data
ä¸¾æŠ¥å†…å®¹
There are a couple of things in terms of the language you are using 1. SQL: Use CTEs to break down complex formulas, Most of the users utilize subqueries but CTEs make life easier to understand the complex calculations and structure of the code. 2. Python: Utilize UDFs to break down the complex code, and finally use the pipeline features (sklearn transform) to collect the operations and apply them to the dataframe. It helps in reuse and easy way of finding out bugs in the production environment.

å·²ç¿»è¯‘

èµž
L Gee

Associate Director at Raytheon Technologies
ä¸¾æŠ¥å†…å®¹
What I find helpful, is being mindful of the â€œas-isâ€ reference frame and â€œto-beâ€ reference frame. Once you account for your initial domain then you can identify the transformation steps in a piecewise fashion to arrive at the future target domain.

å·²ç¿»è¯‘

èµž

4 Use indentation and spacing

Another tip for writing clear and concise data transformation formulas is to use indentation and spacing to organize and format your formulas. Indentation and spacing help you create a visual hierarchy and structure for your formulas. They also make your formulas more readable and consistent. For example, you can use tabs or spaces to indent your formulas, and add spaces before and after operators, parentheses, commas, and keywords.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Katarzyna (Kate) Beben

Artificial Intelligence and Data Manager at EY
ä¸¾æŠ¥å†…å®¹
Properly formatting your formulas with whitespace and indentation can greatly improve readability. Use line breaks and indentation to separate logical sections or group related operations together.

å·²ç¿»è¯‘

èµž

5 Follow the order of operations

One of the most important rules for writing clear and concise data transformation formulas is to follow the order of operations. The order of operations determines the sequence and priority of the calculations or transformations in your formulas. It also affects the results and outcomes of your formulas. Therefore, you should always follow the order of operations and use parentheses to group or clarify your formulas. For example, you can use the PEMDAS acronym to remember the order of operations: parentheses, exponents, multiplication and division, addition and subtraction.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Michael Huang

Crafty STEM-ist | Secretly a Consultant | Data Analyst, Sheets & Queries | IT Support/Automation | SaaS Lover | Supply Chain Logistics | Six Sigma / Lean | Etcetera!
(å·²ç¼–è¾‘)
ä¸¾æŠ¥å†…å®¹
Like working with arithmetic PEMDAS, it is suggested to note the order of logic operations (i.e. AND, OR) for a target language. However, it is also suggested to read and understand their usages by reading documentations. For instance, if suppose we have a binary operator OR, then this takes two elements within the parentheses or brackets. Some languages take adjacent OR without brackets. Because languages are complex that do not have particular rules for including brackets and such, there is no one fixated method to creating formulas. It is possible that the series of OR can work without brackets (though not technically practical)

å·²ç¿»è¯‘

èµž

6 Review and test your formulas

The final best practice for writing clear and concise data transformation formulas is to review and test your formulas before applying them to your data. Reviewing and testing your formulas help you check for errors, typos, or inconsistencies in your formulas. They also help you verify the accuracy, validity, and reliability of your formulas. For example, you can use sample data, dummy values, or validation tools to review and test your formulas.

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Katarzyna (Kate) Beben

Artificial Intelligence and Data Manager at EY
ä¸¾æŠ¥å†…å®¹
Before deploying your data transformation formulas, thoroughly test them using sample data or known inputs. Verify that the outputs match the expected results. This helps catch any errors or inconsistencies early on.

å·²ç¿»è¯‘

èµž

7 Hereâ€™s what else to consider

This is a space to share examples, stories, or insights that donâ€™t fit into any of the previous sections. What else would you like to add?

æ·»åŠ æ‚¨çš„è§‚ç‚¹

Katarzyna (Kate) Beben

Artificial Intelligence and Data Manager at EY
ä¸¾æŠ¥å†…å®¹
Additional tips: 1. Use descriptive/meaningful variable names: Choose variable names that accurately represent the data or calculations they refer to 2. Avoid repetition: Look for opportunities to reuse calculations or intermediate results to minimize redundancy, improves clarity and optimize performance. 3. Consider error handling and edge cases: Anticipate errors or edge cases in your data and implement appropriate error handling mechanisms. 4. Document your transformations: Maintain documentation that describes the purpose, inputs, and outputs of each transformation formula. 5. Iterate and refactor: As your data transformation requirements evolve or become more complex, be prepared to iterate on your formulas and refactor them as needed.

å·²ç¿»è¯‘

èµž

Data Analytics

+ å…³æ³¨

ç»™æ–‡ç« è¯„åˆ†

å¾ˆæ£’ ä¸å¤ªå¥½

ä¸¾æŠ¥æ¤æ–‡ç«

What are some best practices or tips for writing clear and concise data transformation formulas?

1

2

3

4

5

6

7

1 Use consistent naming conventions

2 Comment your formulas

3 Break down complex formulas

4 Use indentation and spacing

5 Follow the order of operations

6 Review and test your formulas

7 Hereâ€™s what else to consider

Data Analytics

ç»™æ–‡ç« è¯„åˆ†

æ„Ÿè°¢æ‚¨çš„åé¦ˆ

æ›´å¤šData Analyticsç›¸å…³æ–‡ç«

æ›´å¤šç›¸å…³é˜…è¯»å†…å®¹

What are some best practices or tips for writing clear and concise data transformation formulas?

1

2

3

4

5

6

7

1 Use consistent naming conventions

2 Comment your formulas

3 Break down complex formulas

4 Use indentation and spacing

5 Follow the order of operations

6 Review and test your formulas

7 Hereâ€™s what else to consider

Data Analytics

ç»™æ–‡ç« è¯„åˆ†

æ„Ÿè°¢æ‚¨çš„åé¦ˆ

æ„Ÿè°¢æ‚¨çš„åé¦ˆ