How to build a simple Neural Network with Keras

Keras is an open source neural network library written in Python. It is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, or Theano. Designed to enable fast experimentation, it focuses on being user-friendly, modular, and extensible. It was developed as part of the research effort of project ONEIROS (Open-ended Neuro-Electronic Intelligent Robot Operating System), and its primary author and maintainer is Fran?ois Chollet, a Google engineer.

If you want to quickly build and test a neural network with minimal lines of code, choose Keras. This post will show how fast and easy it is to build a DNN in Keras. If this looks a bit to complicated you can always check Keras documentation for more information.

import numpy as np
import matplotlib.pyplot as plt
# importing datasets from sklearn
from sklearn import datasets
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam

Building our data set and assigning values to X and labels to y.

• noise = 0.15 standard deviation of Gaussian noise, if noise level is high it is harder to classify generated data.
• n_samples = number of points (pts) to generate random_state = 321 to generate the same number to reproduce errors if they occurs.
• factor = 0.3 represents smaller circle diameter it is 30 % of larger circle.

np.random.seed(0)
pts = 500
X, y = datasets.make_circles(n_samples=pts, random_state=321, noise=0.15 factor=0.3)

Next we will grab all the points that has label of 0 and selecting 0 column to select X coordinates, doing the same for Y coordinates label 0 column 1.

The same is done for the points that has a label value of 1.

plt.scatter(X[y==0,0],[X[y==0,1]])

plt.scatter(X[y==1,0],[X[y==1,1]])
# saving the plot for this article used plt.savefig()
plt.savefig('scatter.png')

Next we defining the DNN as sequential model. Basically we are implementing the same DNN model that you can play with TensorFlow playground website. This is one amazing site to play and visualize how DNN work.

Our DNN will be with 2 input nodes 4 hidden layer nodes and one output node. We will use Sigmoid activation function, and Adam optimizer learning rate 0.01, and finally we will use binary_crossentrophy because we have only two classes to separate. The metric we will use, is accuracy witch will evaluate data how often it matches the given labels. This is only 4 line of codes, that is why Keras is one of the best.

model = Sequential()
model.add(Dense(4,input_shape=(2,), activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))
model.compile(Adam(lr=0.01), 'binary_crossentropy', metrics=['accuracy'])

Next step we will train our model. Training our model with fit function, passing out data x=X and labels y=y, verbose=1 shows the progress bar on each epoch. batch_size and epochs depends on a data on witch you are working, lastly we use shuffle to shuffle our data to not get stuck in a local minimum.

h = model.fit(x=X,y=y, verbose=1, batch_size=30, epochs=100, shuffle='true')

Epoch 1/100
500/500 [==============================] - 0s 782us/step - loss: 0.7057 - acc: 0.46
-------------------------------------\\--------------------------------------------
Epoch 100/100
500/500 [==============================] - 0s 42us/step - loss: 0.1331 - acc: 0.956

Next step we will plot results from training our model, we will display accuracy and loss results of our DNN.

plt.plot(h.history['acc'])

plt.xlabel('epoch')

plt.legend(['Accuracy'])

plt.title('Accuracy')

plt.plot(h.history['loss'])

plt.xlabel('epoch')

plt.legend(['loss'])

plt.title('Loss')

Lastly we will build a function for decision boundary, with 3 arguments. 1st. argument for data 2nd. argument for labels and last is DNN model. In this function we will produce grid that spans perfectly our data from minimum values and maximum values of x, y coordinates, and also adding a tolerance for better visual effect. We will use numpy library to manipulate the data to best meet our needs.

def des_bound(X,y,model):

   x_span = np.linspace(min(X[:,0])-0.25, max(X[:,0])+0.25)
   y_span = np.linspace(min(X[:,1])-0.25, max(X[:,1])+0.25)
   xx, yy = np.meshgrid(x_span,y_span)
   xx_, yy_ = xx.ravel(), yy.ravel()
   grid = np.c_[xx_,yy_]

   pf = model.predict(grid)

   z = pf.reshape(xx.shape)
   plt.contourf(xx,yy,z)

Now lest produce a plot with decision boundary to visualize the separation between two classes also creating a new point and feeding to our DNN to test the probability of being a 1 or a 0.

des_bound(X,y,model)
plt.scatter(X[y==0,0],[X[y==0,1]])
plt.scatter(X[y==1,0],[X[y==1,1]])
# creating an array to our new point to classify it 
x = 0.4
y = 0
point = np.array([[x,y]])
# feeding a new point to our model
prediction = model.predict(point)
# marker = 'o' means dot, other self explainetory
plt.plot([x],[y], marker='o', markersize=10, color='red')
print('The prediction is: ',prediction)

The prediction is:[[0.9387336]]

We used Binary Classification to predict if our new point is 1 or 0, the results were 0.9387336 in a plot with decision boundary we can see what it looks like. As we can see our model made a prediction on a new point quite well.

Problems like this generally fall into the domain of Supervised Learning since the training data-set has labels. And as the name suggests it is simply a special case in which there are only two classes. 

As you can see it is quite easy to get started with DNN with Keras. Thanks for reading.