How do you use scatter plots to explore correlations and outliers in your data?

In addition, scatter plots can also reveal patterns, trends, and relationships between the two variables, such as positive or negative correlation, outliers, clusters, or nonlinearities. Scatter plots can also be enhanced with additional information, such as color, size, or shape of the dots, to represent a third variable or category. They are commonly used in exploratory data analysis, hypothesis testing, and regression analysis to visualize and understand the distribution and association of the data.

When using scatter plots to explore correlations, I often like to use a Deming regression instead of a simple linear regression, because it accounts for errors in observations on both the x- and the y- axis. And for the the case when δ = 1, the Deming regression is called an orthogonal regression.

Additionally, it is important to choose the appropriate scale for each axis to ensure that the data is clearly displayed and easy to interpret. It is also important to consider any outliers or unusual observations and how they may affect the overall interpretation of the plot. Adding trend lines or regression lines can help to identify any patterns or relationships between the variables, and including error bars or confidence intervals can provide a sense of the variability or uncertainty in the data. Finally, it is important to carefully consider the message that the scatter plot is intended to convey and to ensure that the graph effectively communicates this message to the intended audience.

Additionally, it's important to look for outliers or any patterns or clusters in the data that could indicate subgroups or relationships between the variables. Other factors to consider when interpreting a scatter plot include the scale and range of the axes, as well as any potential confounding variables that could affect the relationship between the variables of interest. Finally, it's essential to remember that correlation does not imply causation, and further analysis is often needed to establish a causal relationship between the variables.

In my past experiences of using Scatter Plots for the retail domain: Scatter plots have been really helpful in showcasing price elasticity of a product. Without needing to add a regression line, one can visually find whether: 1) Price Elasticity exists - Inverted curve with peak close to y axis 2) If there is a step change in reaction - The line will break and it will appear to be 2 lines cut in between and shifted up. It is important to know here that this indicates a potential factor outside of X axis that provides this shift.

To identify outliers in a scatter plot, you can also use robust regression techniques that are less sensitive to outliers than traditional regression methods. One such technique is the Theil-Sen estimator, which computes the median slope among all possible pairs of data points and is resistant to up to 29% of contaminated data. Another technique is the Least Trimmed Squares (LTS) estimator, which finds the subset of data points that minimize the sum of squared errors and is resistant to up to 50% of outliers. These techniques can help improve the accuracy and robustness of your analysis when dealing with data that contains outliers.

In addition to identifying correlations and outliers, scatter plots can also be used for hypothesis generation and testing. By visually exploring the data and identifying patterns and relationships, you can generate hypotheses about underlying factors and test them using statistical methods. Scatter plots can also be used to identify potential data quality issues, such as missing or incorrect values, or to detect nonlinear relationships that may require more advanced modeling techniques. Overall, scatter plots are a powerful tool for data exploration and can reveal important insights and trends that may not be apparent from summary statistics alone.

One of the beautiful attributes of scatter plots is that they can reveal big picture patterns yet still allow you to see detail in the individual data points. With thoughtful design, they can deliver the story and the nuance and the minutia all at once.