How to Classify the paintings of an artist using Convolutional Neural?Network

Original Article was published on Data Spoof Blog

In this article, I will explore how we can use Pytorch to solve an image classification problems of multiple classes. Pytorch comes with a lot of tools and libraries that help in solving our problem. Pytorch provides modules in the range from a high level like torch.nn module( It is used for creating neural networks) to low-level autograd functions. Most of the deep learning researchers use the PyTorch framework to do their tasks. Pytorch is written in c++, Python, and Scala language to make our life easier.

Some other blog post that you may want to read is

• Time Series analysis in python part-1
• K-Means Clustering in Python using scikit-learn
• Implementation of K- Nearest Neighbors from scratch in python
• Chapter 1- PyTorch for Beginners: Basics
• Building a logistic regression in Python

Recently I have visited an art gallery where 100 paintings are hanging on their wall. It is very difficult for normal human beings to classify paintings of different artists. So there comes deep learning technology to help in the identification of paintings by different artists.

The article is divided into seven sub-articles and each sub-article will focus on a particular aspect of this workflow including:

1- Data Exploration and it is objective

2- Data visualization

3- Preparing an artist classification Convolutional Neural Network(CNN) and train on the following architecture

A. ) Using pre-trained networks like ResNet18, VGG19, Alexnet, and many more.

4- Testing on the test dataset

5- Evaluating the classification model on the test set using traditional methods like classification reports, confusion matrix, precision, recall.

Our multi-class classification dataset

The dataset that will be using in today’s PyTorch multi-class classification tutorial is we will be classified paintings of different painters.

The dataset is publicly available on Kaggle as Impressionist_Classifier_Data. You can download the dataset and proceed further.

Our dataset consists of 5000 images across 10 classes.

The goal of our convolutional neural network will be to predict which painter painting is it.

It will take you approx 5–10 minutes based on your internet speed.

Configure your Virtual environment

To Configure your environment, I will recommend you to follow these steps

How to install Pytorch on Windows

PyTorch without CUDA:

conda install pytorch torchvision cpuonly -c pytorch

PyTorch with CUDA 10.1:

conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

How to install Pytorch on macOS

pip install torch torchvision

You can use any of the methods based on your operating system.

Preparing an image classification Convolutional Neural Network (CNN) and train on the following architecture

A) Using pre-trained networks like ResNet18, VGG19, Alexnet, and many more.

The CNN architecture that we are using in this tutorial is ResNet18. There are many variants of ResNet models like ResNet36, ResNet34, and ResNet 54. Each ResNet block is either 2 layer deep(ResNet18, ResNet34) or 3 layer deep (ResNet50, ResNet101, ResNet152). The reason for choosing ResNet is deeper networks start converging due to the depth in the network the accuracy gets saturated and degrades rapidly.

So the first step is to import all the required libraries:

So the first step is to import all the required libraries:

And then second step is to load our dataset:

Let us see few of the images in our dataset

After that we apply the Data Transformation on training and testing folder, so that the images have the right shape, for that we need to define transfomer. We use torchvision.trasnform class to convert the datatset.

1- transforms.RandomResizedCrop(): crops the images in a particular size.

2- transforms.RandomRotation(): we will rotate the image randomly by 15 degrees.

3- transforms.RandomHorizontalFlip(). It will flip the image horizontally with a given probability.

4- transforms.CenterCrop(): It will crop the given image at 224*224 pixels about the center.

5- transforms.ToTensor(): It transforms the dataset into the PyTorch tensor.

6- transforms.Normalize(): We will normalize the pixel values. It contains parameters like mean and standard deviation. In this case, we are passing 3 values of mean and 3 value of standard deviation because the image is in RGB format.

Now we will define a function plot_Images which is used to display five sample images.

The next step is to load the data from the training and test folder and then calculate the size of the training and testing data.

The next step is to prepare a train and test loader. We further set the argument batch_size=64, so that in the training this will fetch the data in batches of 64.

For the train and test loader, we set the argument shuffle= True, so that in training biases can be removed and data became more generalized. Now we set pin_memenory= True to the train and test data loader will automatically fetch the data tensors in pinned memory, and this enables faster data transfer to Cuda GPU.

Now the set the last argument which is drop_last-=True, by this, we drop the last incomplete batch if the dataset size is not divisible by the batch size.

After that, we print the target class name. Now we Set the default device as GPU if available else CPU.

Now we will the load the pretrained model which is ResNet18 and pass an argument that the function can downlaod the weights of the resnet model.

Now we modify fully connected layers to match num_classes.

Once the resnet weight is downloaded, we can proceed with the other steps. If you want you can also check the model summary you can torch summary function as follows and then we load the model to GPU.

Now we Set Loss Criteria, Optimizer and Learning rate decay

As we know that this is a multiclass problem so we use CrossEntropyLoss().

We also need to define an optimizer, in our case, we will be using Stochastic Gradient Descent (SGD) optimizer with a learning rate =0.001 and momentum=0.9.

Additionally, we need to define Learning rate decay with parameters like step_size=7 and gamma=0.1.

Now we define the train_model function in which we pass num_epochs=10 and print the loss and accuracy.

And save the model as .pth file.

As you can see we trained the network for 10 epochs, achieving:

1- 87.96% multi-class classification on the training set

2- 77.17% multi-class classification on the testing set

Applying Pytorch multi-class classification to new images

Now that our multi-class classification PyTorch model is trained, let us apply it to new images of the painting.

On the first five lines, we import the necessary packages for the script.

Now we load the image and preprocess the input image for classification.

Now we load the saved model which is .pth.

Next we pre-process the image and prepare a batch of size 8 to be passed through the network.

And then we classify the painting.

The output is shown below

Conclusion

In this tutorial, we learn about how to classify paintings using CNN in the PyTorch framework. You can download this notebook on my GitHub.

If you like the article please clap on the article and if you have any problem regarding implementation feel free to comment.

Some of the related research works are

Artist Identification with Convolutional Neural Networks

Art Painting Identification using Convolutional Neural Network

#machinelearning

#datascience

#deeplearning #ai #python #pytorch