Neural Networks Made Fun With TensorFlow Playground!

Introduction

When we embark on Deep Learning, we are inevitably confronted with the difficulty of understanding the very concept of the neural network but also its relation to its configuration.?Indeed, it is difficult to fully understand each adjustment lever of such a mechanism which seems so abstract!

Personally, I have always found the idea that we can model almost anything with such a seemingly simple system (from a unitary, neuronal point of view) magical.?But what is the real impact on the whole system if:

• We add hidden layers & neurons on a particular layer
• The learning rate changes
• We change the number of epochs or cycle of propagation-back-propagation
• The activation functions changes
• Regulation functions are used
• We change the size of the batches
• etc.

Interface presentation

The interface is rather simple and uncluttered, let’s detail it in part.

The upper strip allows you:

• To start learning or to reset it
• Owe the number of epochs performed (once we have started learning)
• View the learning rate (for gradient descent)
• See the activation functions of our neurons
• View the regularization functions (only L1 & L2)
• Change the type of problem (is it a classification or a regularization problem?)

Then we have the neural network inputs:

• We can choose the type of data distribution
• The ratio of training data to test data.
• Add outliers (noises) to disrupt the learning of the network a little by inserting more realistic data.
• The size of the neural network learning batches.

Note: Orange points have a value of -1 while blue points have a value of +1        

We then have the output:

The output graph shows the result of the neural network.?What is particularly interesting at this level is that we have above the result the curves of the cost functions (loss) by epoch (epochs).?This is one of the most important things to watch out for when you go to start learning to see if your network is performing or not.?The training cost curve is gray while the test one is black.

Most interesting (for the end) maintaining the network design graphically in the middle:

The network is presented of course by vertical layers.?You can add up to 8 hidden layers by clicking on the + (or removing it with them -).?On each layer, you can also add or remove neurons with the + and – above the layers.

What to note here (once you have started your learning):

• You will see by neuron how at this stage the result begins to materialize.?You will notice that the more layers you add the more the network refines its result: by roughing in the first hidden layers to do more specialized work in the last ones.
• Hover the mouse over the links between the neurons, and you can also see the weight values and why not detect dead neurons for example.?The thickness of the line also gives you an indication of the importance of the weight.

Some tests

We’re ready to go for a few tries now.?Try it out several times, changing parameter by parameter to see its impact.?Look particularly at the cost curves, it is necessary of course that they are as low as possible together.?Be careful that it does not diverge, which would mean that you are for example over-learning (training curve at the bottom and test curve higher).

To go more law and for example detect a disappearance of gradient, try the distribution of the data in the form of a spiral:

You will already see that the training takes a lot longer.?It will undoubtedly be very unstable insofar as it will sometimes have stagnation (plateaus) then recovery, etc.?If you have put more than 3 hidden layers, also notice the evolution of the weights on the layers on the left compared to those on the right.

Conclusion

This tool is above all a fun tool but it allows you to feel the importance of the major adjustment levers of a neural network.?So if you are embarking on this Deep Learning adventure I highly recommend that you play with this simulator.

go to a ready-made instance here!

#deeplearning #neuralnetworks #machinelearning #artificialintelligence #learning