What is Hypothesis Testing?
Hypothesis testing is a statistical method used to make decisions or inferences about population parameters based on sample data. It involves making an assumption, known as a hypothesis, and then determining whether this assumption is supported by the data.
Example
Imagine you want to prove that the average height of students in a class is 30 inches, or that boys are generally taller than girls. These statements are assumptions or hypotheses that need to be tested statistically to determine their validity. Hypothesis testing provides a mathematical framework to draw conclusions about these assumptions.
Key Concepts in Hypothesis Testing
Null Hypothesis (??0)
The null hypothesis represents a statement of no effect or no difference. It is the default assumption that there is no significant change or relationship.
- Example: The average height in the class is 30 inches (??0: ?? = 30)
Alternative Hypothesis (??1)
The alternative hypothesis is a statement that contradicts the null hypothesis. It represents the effect or difference we are testing for.
- Example: The average height in the class is not 30 inches (??1: ?? ≠ 30)
Key Terms in Hypothesis Testing
- Level of Significance (??): It is like a threshold for deciding whether the results of a test are significant. It represents the probability of incorrectly rejecting a true null hypothesis (a false positive). Common values are 0.05, 0.01, or 0.10, which means there is a 5%, 1%, or 10% risk of concluding that there is an effect when there actually isn't.
- P-value: The p-value tells us how likely it is to see our test results, or something more extreme, if the null hypothesis is true. A small p-value (usually less than the ??) suggests that the observed data is unlikely under the null hypothesis, thus providing strong evidence against it and leading us to reject the null hypothesis.
- Test Statistic: A test statistic is a number calculated from the sample data that helps decide whether to reject the null hypothesis. It's a way to standardize the data so that we can compare it against a known distribution (like the normal distribution). Examples include t-statistics, z-statistics, and chi-square statistics, depending on the type of test being performed.
- Critical Value: The critical value is the cut-off point that our test statistic needs to exceed for us to reject the null hypothesis. It is determined based on the chosen significance level (??) and the type of statistical test we are using. If the test statistic exceeds this value, we conclude that the results are statistically significant.
- Degrees of Freedom: Degrees of freedom refer to the number of independent values in a calculation that are free to vary. They are crucial for determining the exact shape of the distribution we use to interpret the test statistic. In simple terms, it's like the number of options or choices we have left after applying some constraints during our calculations.
Why Do We Use Hypothesis Testing?
Hypothesis testing is essential for making data-driven decisions. It helps to determine whether observed data deviates significantly from what is expected under the null hypothesis. This process is critical in fields such as medicine, finance, social sciences, and more, where decisions must be based on empirical evidence.
When to Use Hypothesis Testing in Data Science?
Hypothesis testing is used in data science to validate assumptions and models, test the effectiveness of algorithms, and make inferences about population parameters based on sample data. It is crucial for ensuring the reliability and validity of analytical results.
Real-Life Example
- Hypothesis: The coin is fair (50% heads, 50% tails).
- Null Hypothesis: The coin is fair.
- Alternative Hypothesis: The coin is not fair.
- Test: Toss the coin 100 times and see if the heads/tails ratio deviates significantly from 50/50.
- Hypothesis: A new recipe is better than the old one.
- Null Hypothesis: The new recipe is not better.
- Alternative Hypothesis: The new recipe is better.
- Test: Conduct a taste test with 30 people and compare their ratings for both recipes.
- Hypothesis: A new training program improves performance.
- Null Hypothesis: The new training program does not improve performance.
- Alternative Hypothesis: The new training program improves performance.
- Test: Measure the performance of athletes before and after the new training program.
Steps to Perform Hypothesis Testing:
- Formulate Hypotheses: Define the null hypothesis (??0) and the alternative hypothesis (??1).
- Choose a Significance Level: Select the level of significance (??).
- Collect Data: Obtain a sample from the population.
- Calculate the Test Statistic: Compute the test statistic based on the sample data.
- Determine the Critical Value or P-value: Find the critical value or calculate the p-value.
- Make a Decision: Compare the test statistic with the critical value or the p-value with ?? to decide whether to reject ??0.
One-Tailed and Two-Tailed Tests
- One-Tailed Test: Evaluates whether the sample mean is significantly greater than or less than a specified value.
Example: Testing if boys are taller than girls (??1: ??boys > ??girls).
- Two-Tailed Test: Assesses whether the sample mean is significantly different from a specified value in either direction.
Example: Testing if the average height in the class is not equal to 30 inches (??1: ?? ≠ 30).
Hypothesis Testing in different domain
Example 1: Healthcare A pharmaceutical company tests a new drug to determine if it lowers blood pressure more effectively than an existing drug.
- ??0: The new drug is not more effective than the existing drug.
- ??1: The new drug is more effective than the existing drug.
Example 2: Education A school tests a new teaching method to see if it improves student performance compared to the traditional method.
- ??0: The new teaching method does not improve performance.
- ??1: The new teaching method improves performance.
Example 3: Manufacturing A factory tests whether a new machine produces fewer defective items than the old machine.
- ??0: The new machine does not produce fewer defective items.
- ??1: The new machine produces fewer defective items.
