How can you remove outliers in a box plot without distorting data?

1 What are trimming and winsorizing?

Trimming and winsorizing are two techniques to deal with outliers in a box plot. Trimming means removing the outliers from the data set, while winsorizing means replacing the outliers with the closest non-outlier values. Both methods reduce the influence of outliers on the box plot, but they have different effects on the summary statistics and the shape of the distribution.

2 How to trim outliers in a box plot?

To trim outliers in a box plot, you need to define a criterion for identifying and excluding them. One common criterion is to use the interquartile range (IQR), which is the difference between the third quartile (Q3) and the first quartile (Q1) of the data. The IQR measures the spread of the middle 50% of the data. Any value that is more than 1.5 times the IQR below Q1 or above Q3 is considered an outlier. To trim these outliers, you can use a filter or a conditional statement in your data analysis software. For example, in R, you can use the subset() function to select only the values that are within the IQR criterion. Here is an example of how to trim outliers in a box plot using R:

# Generate some data with outliers

set.seed(123)

data <- rnorm(100, mean = 0, sd = 1)

data <- c(data, -5, 5)

# Calculate the IQR and the lower and upper bounds

iqr <- IQR(data)

lower <- quantile(data, 0.25) - 1.5 * iqr

upper <- quantile(data, 0.75) + 1.5 * iqr

# Trim the outliers

data_trimmed <- subset(data, data > lower & data < upper)

# Plot the original and trimmed data

par(mfrow = c(1, 2))

boxplot(data, main = "Original data")

boxplot(data_trimmed, main = "Trimmed data")

3 How to winsorize outliers in a box plot?

To winsorize outliers in a box plot, you need to define a criterion for identifying and replacing them. The same IQR criterion as above can be used, or you can use a percentage of the data as the cutoff point. For example, you can winsorize the top and bottom 5% of the data by replacing them with the 5th and 95th percentiles, respectively. To winsorize outliers, you can use a function or a loop in your data analysis software. For example, in R, you can use the winsorize() function from the DescTools package. Here is an example of how to winsorize outliers in a box plot using R:

# Load the DescTools package

library(DescTools)

# Generate some data with outliers

set.seed(123)

data <- rnorm(100, mean = 0, sd = 1)

data <- c(data, -5, 5)

# Winsorize the outliers at 5%

data_winsorized <- winsorize(data, probs = c(0.05, 0.95))

# Plot the original and winsorized data

par(mfrow = c(1, 2))

boxplot(data, main = "Original data")

boxplot(data_winsorized, main = "Winsorized data")

4 What are the pros and cons of trimming and winsorizing?

Trimming and winsorizing are both useful methods to remove outliers in a box plot without distorting data, however, they come with their own set of advantages and disadvantages. Trimming reduces the sample size and the variance of the data, which can affect the statistical inference and the power of the analysis. On the other hand, winsorizing preserves the sample size and the variance of the data, which can improve robustness and efficiency. Trimming eliminates outliers completely, resulting in potential loss of information and bias in estimation. In contrast, winsorizing retains outliers partially, allowing for a more accurate reflection of population distribution. Although trimming simplifies data to make box plots easier to interpret and compare, winsorizing complicates them and makes them harder to explain.

5 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?