Analysis Of Variance (ANOVA)

In this article, I am going to explain to you the?core concept of Analysis of Variance. We deep dive into concept of Hypothesis Testing.

What is ANOVA?

A common approach to figure out a reliable treatment method would be to analyse the days it took the patients to be cured. We can use a statistical technique which can compare these three treatment samples and depict how different these samples are from one another. Such a technique, which compares the samples on the basis of their means, is called ANOVA.

Analysis of variance (ANOVA) is a statistical technique that is used to check if the means of two or more groups are significantly different from each other. ANOVA checks the impact of one or more factors by comparing the means of different samples.

Analyzes the variance of groups to assess differences in means between groups. It is an extension of regression (the General Linear Model) or (GLM1). The General linear model is a special form regression. So in reality everything we do from with hypothesis testing from z-tests to t-tests, to correlation, to anova is regression.

Why ANOVA?

When it comes to achieving the mean of two or more population groups, ANOVA (Analysis of variance) and t-test are the two best practices preferred. Although there is a thin line between both of them. The t-test is conducted when you have to find the population mean between two groups, when there are three or more than three groups you go for ANOVA test.

T-tests are used for pure hypothesis testing purposes whereas ANOVA is used to examine standard deviations. T-test is used when the population is less than 30 and ANOVA is used for larger populations.

ANOVA tests the hypothesis:

• Null hypothesis H0: All population means are the same
• Alternative hypothesis H1: At least one population mean is different

Working of ANOVA

Before we get started with the applications of ANOVA, I would like to introduce some common terminologies used in the technique.

Grand Mean: Mean is a simple or arithmetic average of a range of values. There are two kinds of means that we use in ANOVA calculations, which are separate sample means ?and the grand mean ?. The grand mean is the mean of sample means or the mean of all observations combined, irrespective of the sample.

Hypothesis: Just like any other kind of hypothesis that you might have studied in statistics, ANOVA also uses a Null hypothesis and an Alternate hypothesis. TheNull hypothesis in ANOVA is valid when all the sample means are equal, or they don’t have any significant difference.

Between Group Variability: Consider the distributions of the below two samples. As these samples overlap, their individual means won’t differ by a great margin. Hence the difference between their individual means and grand mean won’t be significant enough.

Within Group Variability: We can measure Within-group variability by looking at how much each value in each sample differs from its respective sample mean. So first, we’ll take the squared deviation of each value from its respective sample mean and add them up.

Like between-group variability, we then divide the sum of squared deviations by the?degrees of freedom??to find a less-biased estimator for the average squared deviation (essentially, the average-sized square from the figure above). Again, this quotient is called the mean square, but for within-group variability.

This time, the degrees of freedom is the sum of the sample sizes (N) minus the number of samples (k). Another way to look at degrees of freedom is that we have the total number of values (N), and subtract 1 for each sample:

F-Statistic: The statistic which measures if the means of different samples are significantly different or not is called the F-Ratio. Lower the F-Ratio, more similar are the sample means. In that case, we cannot reject the null hypothesis.

F = Between group variability / Within group variability

This above formula is pretty intuitive. The numerator term in the F-statistic calculation defines the between-group variability. As we read earlier, as between group variability increases, sample means grow further apart from each other. In other words, the samples are more probable to be belonging to totally different populations.

This F-statistic calculated here is compared with the F-critical value for making a conclusion. In terms of our medication example, if the value of the calculated F-statistic is more than the F-critical value (for a specific α/significance level), then we reject the null hypothesis and can say that the treatment had a significant effect.

Unlike the z and t-distributions, the F-distribution does not have any negative values because between and within-group variability are always positive due to squaring each deviation.

Therefore, there is only one critical region, in the right tail (shown as the blue shaded region above). If the F-statistic lands in the critical region, we can conclude that the means are significantly different and we reject the null hypothesis. Again, we have to find the critical value to determine the cut-off for the critical region. We’ll use the?F-table for this purpose.

One Way ANOVA

As we now understand the basic terminologies behind ANOVA, let’s dive deep into its implementation using a few examples.

A recent study claims that using music in a class enhances the concentration and consequently helps students absorb more information. As a teacher, your first reaction would be skepticism.

What if it affected the results of the students in a negative way? Or what kind of music would be a good choice for this? Considering all this, it would be immensely helpful to have some proof that it actually works.

To figure this out, we decided to implement it on a smaller group of randomly selected students from three different classes. The idea is similar to conducting a survey. We take three different groups of ten randomly selected students (all of the same age) from three different classrooms. Each classroom was provided with a different environment for students to study. Classroom A had constant music being played in the background, classroom B had variable music being played and classroom C was a regular class with no music playing. After one month, we conducted a test for all the three groups and collected their test scores.

Limitations of one-way ANOVA: A one-way ANOVA tells us that at least two groups are different from each other. But it won’t tell us which groups are different. If our test returns a significant f-statistic, we may need to run a post-hoc test to tell us exactly which groups have a difference in means. Below I have mentioned the steps to perform one-way ANOVA in Excel along with a post-hoc test.

Another measure for ANOVA is the p-value. If the p-value is less than the alpha level selected (which it is, in our case), we reject the Null Hypothesis.

Two-Way ANOVA:

Using one-way ANOVA, we found out that the music treatment was helpful in improving the test results of our students. But this treatment was conducted on students of the same age. What if the treatment was to affect different age groups of students in different ways? Or maybe the treatment had varying effects depending upon the teacher who taught the class.

Moreover, how can we be sure as to which factor(s) is affecting the results of the students more? Maybe the age group is a more dominant factor responsible for a student’s performance than the music treatment.

For such cases, when the outcome or dependent variable (in our case the test scores) is affected by two independent variables/factors we use a slightly modified technique called two-way ANOVA.

In the one-way ANOVA test, we found out that the group subjected to ‘variable music’ and ‘no music at all’ performed more or less equally. It means that the variable music treatment did not have any significant effect on the students.

So, while performing two-way ANOVA we will not consider the “variable music” treatment for simplicity of calculation. Rather a new factor, age, will be introduced to find out how the treatment performs when applied to students of different age groups.

Two-way ANOVA tells us about the main effect and the interaction effect. The main effect is similar to a one-way ANOVA where the effect of music and age would be measured separately. Whereas, the interaction effect is the one where both music and age are considered at the same time.

That’s why a two-way ANOVA can have up to three hypotheses, which are as follows:

Two null hypotheses will be tested if we have placed only one observation in each cell. For this example, those hypotheses will be:

H1: All the music treatment groups have equal mean score.

H2: All the age groups have equal mean score.

For multiple observations in cells, we would also be testing a third hypothesis:

H3: The factors are independent?or?the interaction effect does not exist.

An?F-statistic?is computed for each hypothesis we are testing.

Multi-variate ANOVA (MANOVA):

Until now, we were making conclusions on the performance of students based on just one test. Could there be a possibility that the music treatment helped improve the results of a subject like mathematics but would affect the results adversely for a theoretical subject like history?

How can we be sure that the treatment won’t be biased in such a case? So again, we take two groups of randomly selected students from a class and subject each group to one kind of music environment, i.e., constant music and no music. But now we thought of conducting two tests (maths and history), instead of just one. This way we can be sure about how the treatment would work for different kind of subjects.

We can say that one IDV/factor (music) will be affecting two dependent variables (maths scores and history scores) now. This kind of a problem comes under a multivariate case and the technique we will use to solve it is known as MANOVA. Here, we will be working on a specific case called one factor MANOVA.

RealStats add-on shows us the results by different methods. Each one of them denotes the same p-value. As the p-value is less than the alpha value, we will reject the null hypothesis. Or in simpler terms, it means that the music treatment did have a significant effect on the test results of students. But we still cannot tell which subject was affected by the treatment and which was not. This is one of the limitations of MANOVA; even if it tells us whether the effect of a factor on a population was significant or not, it does not tell us which dependent variable was actually affected by the factor introduced.

Conculsion:

I hope this article was helpful and now you’d be comfortable in solving similar problems using Analysis of Variance. I suggest you take different kinds of problem statements and take your time to solve them using the above-mentioned techniques.