How do you develop and test hypotheses using data?

Something to keep in mind when formulating your question is personal and professional biases. These can seriously impact what data is collected and how it is used.

When there are so much data, you can approach data in so many ways that it can be overwhelming. One thing that helps me to start investigating on a problem is identifying my parameters. For example: - What is my timeline? Who is involved and what piece does each person own? Is there an existing company standard or rules and regulation? Do we have historical data to help set a baseline, ect. When you can start narrowing down even a little bit, this will give you some direction to start trying out what doesn't work or identify who to ask if you don't know certain things. Eventually after testing and feedback, you'll find a road that will help answer your question or questions if maybe digging help open up more roads to investigate!

When approaching any problem, it's essential to define the objective clearly. By doing so, we can avoid ambiguity and ensure that our efforts are focused on addressing the core issue. Additionally, by formulating the problem specifically, we can determine the necessary data and resources needed for an effective investigation. Once the problem is well-defined, we need to find multiple approaches to see the problem from different lenses.

A great resource would be to look at data that has been peer reviewed. This adds to the validity and credibility of the data as it removes bias and provides support from the professionals within the field that the data derives from.

To further enhance the process of hypothesis formulation, I'd like to add a framework and a point to remember: 1. Framework for Hypothesis Formulation: a. Identify the Variables. b. Determine the Relationship. c. Specify the Directionality. d. Define the Scope. 2. Point to Remember: Avoid Causal Claims without Experimental Design. By following this framework and keeping the point about causality in mind, we can ensure that our hypotheses are well-structured, testable, and aligned with the scientific method. This approach enhances the credibility of our investigations and contributes to the advancement of knowledge in our respective fields.

Too often experiments are presented with key data and outcomes, yet the researchers have forgotten to include their study design and test type together. This makes it difficult for other researchers to complete the same experiment to test on new assumptions. Controlling confounding factors can be done in a number of ways, and aside from using randomisation, another popular way is to restrict a treatment group by only including subjects with the same values of potential confounding factors. (Thomas, 2022) Sometimes bias cannot be minimised, however it should always be reported, so that other researchers or end users can understand the relevance and context.

When you are designing an experiment or an analysis from scratch, think about who you are presenting this data. Data is compiled to tell a story so I would always build it in a way that I can start macro and drill down to the micro. For example, in an excel analytics, there would always be a summary tab, detail tabs maybe group by certain criterias, data dump tab with timestamps of data or any kind of backup attached to it. When I do this, I can tell my story to whoever need to see the data high level, and if there are questions, we can always drill down as needed.

Additional considerations 1. Ensure data quality: use reliable methods, minimize errors, and consider inaccuracies. Validate data, check outliers, and maintain integrity for accurate conclusions. 2. Ethical Considerations: respect privacy, obtain consent, and follow guidelines. Beware of biases for integrity and objectivity. 3. Interpreting Results- Consider the context: study limitations, confounding variables, and generalizability. Responsible for data interpretation. 4. Communicate findings effectively: with clear language, visualizations, and storytelling. Drive informed decisions, understand implications, and facilitate actions. 5. Iterative Process: Iterative data analysis: learn, refine, and solve complex problems.