Deep Learning in Python with TensorFlow and Keras API for creating AI algorithms/models. Sequential models.

Hello! In part 2 of this series of tutorials, model architecture and weights were created for Artificial Neural network(ANN) and this time, i will show how to visualize the basic structure an ANN. Visualization is a very important step into understanding functions of different layers and what's actually happening inside the Artificial Neural Network. Additionally, methods of saving this Neural Network architecture and neuron weights will be discussed. This is a vital part of working with neural networks and deploying them. Before i start, let me make a reminder that TensorFlow[1], Keras[2] for Python[3] were used t for creating Sequential model, compiling the model and fitting it were written in previous tutorials...You may read about these in Parts 1 and 2.

For this tutorial i will change the dense layers, so this is the code for a Sequential model...

#Architecture of ANN

model = Sequential([

?Dense(632, activation='relu', input_shape=(1,)),

?Dense(8, activation='relu'),

?Dense(16, activation='relu'),

?Dense(40, activation='relu'),

?Dense(40, activation='relu'),

?Dense(8, activation='relu'),

?Dense(5, activation='relu'),

?Dense(3, activation='relu'),

?Dense(10, activation='relu'),

?Dense(7, activation='relu'),

?Dense(2, activation='softmax'),

])

As it can be seen i made some layers and few with less complexity in terms of neuron number in the second half of the Neural Network.

I will now continue with visualization of the model.

As it can be seen, i imported Dense() layer from keras layers beofre, but now also defaultdic() from collections to be able to easily set the color for the dense layer which will be used later. So i just specified the color to be grey in the cm object.

Now, to be able to visualize the network i will import visualkeras Python library as v_k. So using the graph_view() fucntion, ANN can be visualized.

As it can bee seen, the number of hidden layers can be assessed, se 11 hidden layers are present in the Network. But the ANN actually has 1 layers, as input and output layers should be added to those 11 hidden layers and this represents the depth of the Neural Network. Depth adds huge amount of complexity for data structuring and is one of the reasons why Deep Learning methods are so successful in the real world settings...

We can also analyze the width of the hidden layers... So layers 1,3,4,5 are wide layers and 6 (medium) and are represented with those 3 dots, meaning that number of neurons is too large to present. Other layers have less neurons and we can see their exact number on the graph. It should be noted that all neurons from one Dense layer are being connected to all neurons from next Dense layer and this can now be seen on the graph instead of just having a theoretical sense. This is very important as we will also talk about layers where some neurons are favored and some connection might not even exist between layers. More on this in next articles.

To fit the model to the data, model should now be compiled. Then mode.fit() function can be run. Here is the code...

#Compile structure of neural network, optimizer, loss and metrics

model.compile(

??optimizer=Adam(learning_rate=0.005),

??loss='sparse_categorical_crossentropy' ,

??metrics=['binary_crossentropy', 'accuracy']

)

#Train a model in 200 epochs

model.fit(x=train_S, y=train_L, epochs=200, verbose=2, shuffle=True)

Now the model is fit and it has around 0.975 or 97.5% percentage of accurate predictions on the training set. So both architecture and learned, fit model were created. The next important step is to learn how to save these in different formats.

Simple statement like model.save() for can be used to save the model. Of course the statement will follow a working directory path and will save it there. The name of the model must be between the quote signs to differentiate future model name from the objects in the enviroment. If .h5 is added after the name, the model will save it in the most optimal format in my opinion, saving both the architecture of the neural network and the weights of the neurons for a fitted model. Additionally, hyperparameters can be saved too, so i saved the optimizer parameters by using include_optimizer=True. Another scenario is to save the model as a .pb format file, which is another useful format for ANN models. If nothing is specified after the model name, it will be saved as a .pb file by default.

As it can be seen, i specified a custom name to be able to perform the procedure just by calling it to perform model saving using keras. So after i run MSave1 it will save the model as .h5 model. Model can be saved using direct TensorFlow code, so using a statement tf.saved_model.save() i wrote another line which would save the model using just MSave2. Now, i can efficiently implement model saving or model updating using just Msave1 or Msave2, which will call those TensorFlow/Keras functions...Similar procedure can be done for loading the model using load_model from keras.models. This would complete the circle of saving and loading model architecture and weights...

Another important way of working with ANNs is saving the models in parts. The rchitecture might be saved using model.to_json(), without saving the weights. This json file can also now easily be implemented other software like JavaScript based programs...

The weights can be saved without the architecture too using model.save_weights(), but this would not be advised in most situations since accurate architecture is needed in most deployment types (Deep Learning).

In next tutorial, i will show how to add more layer types with different functions and think like a Deep Learning engineer... Thanks for reading!

The tutorial was created by:

Darko Medin, Data Science/AI Expert at Edanz Group

Spyder IDE was used to implement the code [4].

References :

1.Martín Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. www.tensorflow.org.

2.Chollet Francois et al. Keras. 2015. https://keras.io

3.Python Software Foundation. https://www.python.org

4.Raybaut, P. (2009). Spyder-documentation. Available Online at: Pythonhosted. Org.