Beyond Basics: Advancing from Single to Multiple Perceptrons in Deep Learning

In our last exploration, "Understanding and Applying a Perceptron in a Real-Life Scenario," we demystified the fundamental building block of neural networks: the perceptron. We discussed its real-world applications and how it forms the crux of more complex models. Building on that foundation, let's delve deeper into the capabilities and limitations of a single perceptron and discover how we can transcend these boundaries by introducing multiple perceptrons into our neural network.

In deep learning, it's crucial to recognize that a perceptron is often referred to as a neuron. While neurons and perceptrons share similarities, they are also significantly different. Neurons are exceedingly more complex than perceptrons. Yet, it was the biological neuron that inspired computer scientists to create the perceptron.

This inspiration is why we use the term "neuron" and refer to a network of perceptrons as an Artificial Neural Network (ANN).

Having clarified this, let's delve into the advantages and limitations of a single perceptron model.

Advantages of a Perceptron:

• Simplicity: Perceptrons are straightforward to understand and implement, making them accessible to beginners.
• Binary Classification: They perform well on tasks that require binary classification.
• Foundational: Perceptrons serve as the building blocks for more complex algorithms, such as the Multilayer Perceptron (MLP).

Limitations of a Perceptron:

• Linear Separability: Perceptrons cannot solve problems where data is not linearly separable, such as the XOR problem.
• Overfitting: They are prone to overfitting, especially with noisy datasets.
• Classification Scope: Their capabilities are limited strictly to binary classification tasks.

The Core Issue with Single Perceptrons

Consider a scenario with non-linearly separable data. A single perceptron model can attempt to separate two classes—depicted as green and red points—only in a limited manner, leading to inaccurate predictions.

Let's assume our data is as below

The decision boundary drawn by a single perceptron often fails to provide an adequate separation, rendering the single perceptron model ineffective for complex datasets.

Let's build a single perceptron model and show the output

And if we draw the decision boundary that separates features we see, here a single perceptron is not able to make a good separation, hence we can not use this single perceptron model.

Enhancing the Model with Multiple Perceptrons

To address this shortcoming, we might employ two perceptrons. Each perceptron would receive inputs, perform calculations, and produce outputs X and Y, respectively. These outputs then act as inputs to a third perceptron, which, after further calculation, yields the final output. This layered approach forms the basic premise of an MLP.

It simplifies the idea, however, there are details to it as to what will be the calculation and output.

This is how Neural Network would like

Here is a simple code

Here is the output after 2 Neurons.

When we use more than 1 neuron in our network it is called MLP, a Multi-layer perceptron.

Expanding the Network

Expanding the network by increasing the number of perceptrons enhances the model's performance. For instance, upgrading to three perceptrons:

mlp = MLPClassifier(hidden_layer_sizes=(3,), activation='tanh', solver='lbfgs', random_state=42)        

Further increasing to four perceptrons:

mlp = MLPClassifier(hidden_layer_sizes=(4,), activation='tanh', solver='lbfgs', random_state=42)        

This iterative process of adding neurons or layers enables the model to handle more complex relationships and a greater number of features.

Conclusion

In summary, while the single perceptron model is a great starting point, its real-world applications are limited due to its simplicity. By expanding into a multilayer network, we can significantly improve the model's ability to capture and learn from data complexity. The MLP, or Multilayer Perceptron, thus represents a more powerful and adaptable solution for a wide array of problems in deep learning.