How do you plan and scope data analysis with quality assessment in mind?

Scope is the most defining option. If the plan is not structured, very well defined, then the it can go off the rails and fail before it has a chance to achieve anything

I always ask myself three things before planning a report or analysis when thinking about the objectives. 1) "What is the purpose of this data in relation to the task?" 2) "What does this data help communicate to me or the intended audience?" 3) "How can I make this unfamiliar data (to the audience) meaningful, visually simplified, and easily understood?" It's literally more important for me to scrap any side mission I may be tempted to dive into, and simply follow the instruction from a manager or other fellow employee. My best advice is not to over think it and go back to the request or email and keep going with your plan until it starts to grow.

Once scope is defined, then source. What data will be used, how will it be gathered, how deep of a dive will it take to get enough, but avoid data overload.

Source data doesn't always come formatted! This will wreak havoc on your pivot tables so check the formatting. There can be formatting relative to different date formats as well that need adjusted. It is of the utmost importance to rename any data that your audience will not understand. Rename it in the source data or pivot table fields. Skip acronyms that are not understood by the audience. Review data and the outputs. Does the math makes make sense if formulas are not listed? Complete some random checks and question anything to your data provider that is incomplete or does not make sense. Never feel afraid to ask another person to data review if they are more familiar with the data's meaning.

Exactly, key benefit of data profiling is its ability to reveal relationships, dependencies, and correlations among data elements. This interconnectedness often holds vital insights, such as causative factors or variable interactions. Understanding these relationships allows you to design more accurate models and draw meaningful conclusions. There are various methods to perform data profiling. Descriptive statistics provide a preliminary overview of the central tendencies and variations present in the data. Visualizations, including histograms, scatter plots, and box plots, offer a graphical representation that can highlight patterns and outliers. Also, specialized data quality tools automate the profiling process, making it more efficient.

Don't be afraid to slow down and take appropriate time to understand the data. If a co-worker or manager has a question on the data it is best that you know the answer from understanding the file and it's data meanings. In addition, understanding the date ranges and auditing the date range to what you expect is very important. I also prefer to think about the cardinalities of the data, or if I have a primary key that helps guide my steps to examining data. I've encountered data that is confusing and lists two primary keys in the same column which skews say the final billed costs since they are not truly isolated. It's also may be important to think about what columns may have no values and why that is true and not true in other cases.

This is when it can be confirmed that the data is right and if not, anything missing can be corrected and going back to the first step, scope can be redefined and the project kept on track.

Figure out what your primary key is in this data. What identifier in the data is the step 1 to figure out your objective? How many relationships in that data is linked to this primary key? Check for duplicates and understand the relationships to ensure you are not missing any sum or average of numbers. The worst thing you can do is report wrong numbers where say costs are split into two different identifiers, but yet you reported a cost as one instance of a cost when it should be two split out. Test the data with pivot tables. Does the seasonality of your business make sense with the trends that you find in a simple line graph? Pivot out the months and other relevant data to test your knowledge of the business against the data.

A service provider may (deep breathe) provide an egregious data file that has too much information, requires breaking down & cleansing by removing unnecessary data, or adding your own source data. It's your job to understand the file so get a meeting to review it from the provider with any questions you have. Then think about adding your own columns to source data if you need to use formulas to sum up certain charges or calculate certain number of days from "this date" to "that date". Make sure no provider is discounting certain timelines or adding charges that may not be applicable to your contracts. If you aren't familiar with the contracts get with someone who is so you can win on the task and win on saving your company money when able.

Scrubbing unnecessary data. This can avoid paralysis by over abundance of data. Data overload is a real thing. Maintaining data integrity is the key to keeping the on point for the project.

It's time to contrast the visualization you have put your data into, or provide a secondary data visualization that shows a comparative analysis for change or other. Depending on the data it's best to get some more pivot tables going and even insert charts linked to those pivot tables. I recommend looking at averages and totals, even percentages to gain insight. Excel wise 3D maps is an amazing easy tool that functions say a location's pattern if related to shipping and customer base (heat maps). This is the best time to sit back and think outside the box on what you have discovered insight on to your company's patterns relative to the data.

Proof of concept. Fitting the data to the expected conclusion. Both to be avoided when performing analysis. Following the data not manipulation of the data to fit the expectations. This is the payoff of all the work. Dig into what has been found, but it is not the end. A good exploration of the data will create a direction for the next project. Sometimes success is not just in the current project, but the next steps that it leads too

Time to ensure your tables have simple things that are correct such as; properly spelled titles, right formatting, legends with data labels when needed, and are not so complex that the data is visually disgusting and requires readers with 100 x to read. Do not overload the presentation. Using more PowerPoint slides or scrolling over left to right is best vs visual overload where data enthusiasts take it in others will give up and become uninterested no matter the relevance. Think on combo charts to use your main feature as say a bar graph and corresponding data with costs or totals as line graphs. Again, do not be afraid to segment out data vs a death by chart or line graph visual.

From my experience and industry finite billing details can help identify hard or soft savings opportunities. Being able to sum up for example transportation costs simply from the net billed invoice doesn't always tell you how the service provider got to that cumulative net bill. Digging deeper into requesting more detailed reports that segregate each charge out in this case will help anyone make more insights to patterns outbound from distribution centers or inbound to distribution centers. But the task is not done yet the pricing needs understood for these charges from contracts otherwise the number being billed is taken as fact. I was once told, "Inspect what you expect."