Early adopter version of my book - explaining machine learning algorithms as a hidden function that maps x and y

Background

I shared before about the ?early adopter version of my forthcoming book - mathematical foundations of data science. If you have messaged me - I will come back to you soon.

To put the idea of the book in context

I am creating a small community for my book - mathematical foundations of data science. You get pdf when released but you also chapters as they are released and you get to engage and ask questions. The price is a one off 40 USD. If you are interested please DM me. I am trying to keep spaces limited since I want to learn from feedback so that's an important criterion as well.

The idea of the book is simple: In an age when a majority of the code could be LLM generated, its very useful to approach AI from first principles i.e. from the maths. The good news is .. there are only four things to know: linear algebra, statistics, optimization and probability theory. The bad news is: its not easy to tie these four ideas to every machine learning and deep learning algorithm considering that the field itself is rapidly evolving. In this sense, the book helps by creating a concise structure. Since these ideas are known to many people at A levels (around age 18) - the book creates a foundation to know AI based on ideas that you already know - even if you have studied them years ago!

In this post, I explain the idea of a hidden/ mapping function and how this idea can be used to understand all machine learning and deep learning algorithms.??

I have discussed this idea before? ?Mathematical foundations of Data Science: Machine learning and Deep Learning algorithms expressed as a mapping of a hidden function?

In machine learning and deep learning, a hidden function refers to a mathematical function that transforms the? input data into intermediate representations that contribute to the final output. For example, we can think of the "hidden layers" of a deep neural network as playing the role of a hidden function. We can think of the hidden function as a process that transforms inputs (like x) into outputs (like y) through some internal computation that isn't directly observable, hence the name "hidden." i deep learning, this function could involve a combination of weights, biases, and activation functions that are applied to the inputs in the hidden layers.

One of the key benefits of hidden functions is their ability to model complex and nonlinear relationships between x and y. Simple models like linear regression only capture linear relationships, but with hidden functions, neural networks can model complex, nonlinear relationships that exist in real-world data.??

Hidden functions are key to how machine learning models generalize. By learning useful representations of the data through the hidden functions, a model can make accurate predictions not just on the training data but also on unseen data.

From a learning standpoint, we can extrapolate this idea further and view any machine learning and deep learning algorithm as being modelled by a hidden function or rule as below.

I used a combination of chatGPT and personal insights for this.?

1. Supervised Learning Algorithms

• Linear Regression: The hidden rule is the linear relationship between the input features and the target variable (how a change in the input affects the output in a straight-line manner).
• Logistic Regression: The hidden rule is the probability boundary between two classes, learned through the input features, which separates the outcomes into "yes" or "no."
• Decision Trees: The hidden rule is the sequence of decision boundaries (questions) that best splits the data into different classes or predictions.
• Random Forest: The hidden rule is the collective decision-making process of multiple decision trees, where each tree contributes to a more reliable prediction by finding its own decision boundaries.
• Support Vector Machines (SVM): The hidden rule is the optimal hyperplane (or boundary) that maximizes the margin between different classes in the data.
• k-Nearest Neighbors (k-NN): The hidden rule is that similar data points are close to each other. The algorithm tries to learn that nearby data points share similar outcomes.
• Gradient Boosting Machines (GBM): The hidden rule is to learn how to improve from mistakes by sequentially building models that correct errors from previous ones.
• XGBoost: The hidden rule is similar to GBM, but it also tries to learn how to balance accuracy and simplicity by preventing the model from becoming too complex.
• AdaBoost: The hidden rule is to learn to focus more on difficult cases, improving predictions by giving more weight to examples that were previously misclassified.
• Naive Bayes: The hidden rule is the probabilistic relationship between features and the target, assuming that all features are independent. It calculates how likely an outcome is based on the occurrence of features.

2. Unsupervised Learning Algorithms

• K-Means Clustering: The hidden rule is to learn the groupings (clusters) of data points that are most similar to each other.
• Hierarchical Clustering: The hidden rule is to learn how to group similar data points into a hierarchy, starting from smaller groups and combining them into larger clusters.
• DBSCAN: The hidden rule is to identify dense regions in the data and group points that are close together, while ignoring isolated points (outliers).
• Principal Component Analysis (PCA): The hidden rule is to learn the main directions of variation in the data, so that it can reduce the complexity while retaining the most important information.
• t-SNE: The hidden rule is to learn how to preserve the structure of relationships between data points when visualizing high-dimensional data in a lower-dimensional space.
• Autoencoders: The hidden rule is to learn a compressed representation of the data that still contains the most important features, which can be used to reconstruct the original data.?

3. Reinforcement Learning Algorithms

• Q-Learning: The hidden rule is to learn the value of taking specific actions in different situations to maximize the total reward over time.
• Deep Q-Networks (DQN): The hidden rule is to learn how to approximate the best actions in complex environments using a neural network to estimate future rewards.

4. Deep Learning Algorithms

• Multilayer Perceptron (MLP): The hidden rule is to learn complex, nonlinear relationships between inputs and outputs by passing data through layers of neurons.
• Convolutional Neural Networks (CNN): The hidden rule is to learn how to recognize patterns in the data, such as edges, shapes, or objects in images, by applying filters at different levels.
• Recurrent Neural Networks (RNN): The hidden rule is to learn how to capture dependencies over time, processing sequential data by remembering information from previous inputs.
• LSTM Networks: The hidden rule is to learn which parts of the sequence are important to remember or forget over time, making it effective at handling long-term dependencies in data like text or time-series.
• Generative Adversarial Networks (GANs): The hidden rule is for the generator to learn how to create data that is indistinguishable from real data, while the discriminator learns to detect fake data.
• Transformer Networks: The hidden rule is to learn how to pay attention to important parts of the input data, like focusing on key words or parts of a sentence in natural language processing.
• BERT: The hidden rule is to learn how to understand the context of a word in a sentence by looking at both the words before and after it, improving its understanding of language.
• Variational Autoencoders (VAEs): The hidden rule is to learn how to encode data into a simpler latent space and then generate new data from that space, while ensuring the generated data is similar to the original.

?If you want to be part of the early adopter version of the book as above, please DM me here.