Statistical inference vs Machine learning inference vs Deep learning Inference

Background

I created this for my students. If you understand this post, you are a long way towards understanding the mathematical foundations of data science than most people!?

The term ‘inference’ means to infer new information from some existing information.?

Inference is the process of drawing conclusions or making decisions based on evidence, reasoning, or data.?

The challenge is:? The word inference has a somewhat different meaning in the context of statistics and machine learning.?

I shared previously that the biggest misconception in learning the mathematical foundations of data science which no one tells you is. that- statistical inference is not the same as machine learning inference???

In this post, we expand on the ideas of statistical inference, machine learning inference and deep learning inference - and why their understanding is significant.?

First, we loosely define three terms:??

1. Statistical Inference

Statistical inference is the process of using data from a sample to draw conclusions or make predictions about a larger population. It involves estimating population parameters (e.g., mean, variance) or testing hypotheses using statistical methods. Key techniques include:

• Confidence intervals
• Hypothesis testing
• Regression analysis

Example: Using a sample of survey data to estimate the average income of a population.

2. Machine Learning Inference

Machine learning inference refers to the process of using a trained machine learning model to make predictions or decisions on new, unseen data. The model applies the patterns and relationships it learned during training to generate outputs such as classifications, regressions, or clustering.

Example: A trained spam detection model labeling incoming emails as "spam" or "not spam."

???

3. Deep Learning Inference

Deep learning inference is a subset of machine learning inference, specifically involving deep neural networks. It refers to the process of running a trained deep learning model (e.g., CNNs, RNNs, Transformers) on new data to make predictions, such as object detection, text generation, or speech recognition.

Example: A trained convolutional neural network (CNN) identifying objects in an image as "cat" or "dog."

When to use each type of inference

The choice between statistical inference and machine learning inference depends on the problem at hand, the type of data available, and the goals of the analysis.??

Statistical inference is typically used when the primary goal is to understand relationships, test hypotheses, or make population-level conclusions.

Statistical inference is used?

1. When you want to test specific hypotheses (e.g., "Does smoking increase the risk of lung cancer?").?
2. When data is relatively small and manageable,?
3. Assumptions about the data like normality or independence are reasonable.
4. When interpretability is a priority, such as understanding the impact of specific variables or testing causality.?

Sampling is central to statistical inference. Statistical inference is ideal for drawing population-level conclusions from a representative sample.???

Machine Learning Inference

Machine learning inference is used when the focus is on predicting outcomes or automating tasks, often without requiring a deep understanding of underlying relationships.? We use machine learning when? the primary goal is to predict outcomes based on patterns in data, rather than to explain why. Example: Predicting house prices based on features like size, location, and condition.

Machine learning is useful when dealing with large datasets with many features or unstructured data (e.g., images, text) and when the relationships between variables are complex, nonlinear, or unknown, and you want a flexible model to capture patterns. Machine learning models prioritize predictive accuracy, often at the cost of interpretability. Thus, machine learning relies on algorithms to learn patterns from data, with minimal focus on underlying theoretical distributions.

Deep learning inference

Deep learning inference is a subset of machine learning inference that specializes in complex, large-scale problems, particularly involving unstructured data.??

Deep learning is typically used when the problem involves highly complex patterns or relationships, typically for unstructured data,? that are difficult to model with traditional statistical or machine learning methods.? Deep learning models automatically learn feature representations from raw data, which is ideal when feature engineering is challenging or infeasible.

Example:?

Extracting features from audio signals for speech recognition.

Training a neural network on millions of medical images to detect anomalies.

Combined Use Cases

In the machine learning pipeline,? statistical and machine learning inferences are used together:

• Exploratory Phase: Use statistical inference to identify significant variables and their relationships
• Predictive Phase: Apply machine learning to build predictive models.
• For example, in healthcare, statistical inference might identify risk factors for a disease, while machine learning builds a model to predict individual patient risk. Leverage deep learning for processing medical images or unstructured text in patient records.

????????

Conclusions

1. Sampling plays a central role in statistical inference, as it enables researchers to draw conclusions about a larger population based on a smaller, manageable subset of data.?
2. In contrast, for machine learning inference, we have the test - train split concept - and no sampling as in the case of statistical inference.? This is the key difference
3. Statistical inference and machine learning inference are thus different - but in the machine learning pipeline, they can be used together as we have seen before.?
4. Deep learning Inference is a special case of machine learning inference and is most commonly used in case of unstructured data or highly parameterised models. ,?

Image source: HERE

Baker street - home of Sherlock Holmes. I am often at Baker street for business meetings. Sherlock Holmes is a good example of inference in the broad sense - in the context of inferring new information from existing information.?