PyTorch For Deep Learning: Quick Start ( Installation & Coding)

What is PyTorch?

PyTorch is an open-source machine-learning library for Python. It is used for developing and training deep learning models. It is developed by Facebook's artificial intelligence research group. PyTorch is known for its efficiency, flexibility, and dynamic nature. It allows developers to perform computations with dynamic graphs, which means the user can change the graph on the fly during runtime. This is in contrast to static computation graphs, which are used in other deep learning libraries such as TensorFlow. PyTorch is widely used in the research community and is also gaining popularity in the industry.

More Details

PyTorch is a powerful open-source deep learning framework built on top of the popular numerical computation library, NumPy. It is used for applications such as natural language processing and computer vision. One of the key features of PyTorch is its ability to compute gradients automatically, making it very well-suited for training deep neural networks.

In addition to the PyTorch core library, there are several popular packages that are built on top of PyTorch and provide additional functionality. One such package is torchvision, which is a computer vision library that makes it easy to train models in PyTorch. It includes a wide range of datasets, model architectures, and common image transformations, making it a useful tool for quickly building and evaluating deep learning models for tasks such as image classification and object detection.

Another popular PyTorch package is torchtext, a natural language processing library that provides easy-to-use datasets and data loaders for NLP tasks. It also includes popular word embedding models such as word2vec and GloVe, making it easy to incorporate these models into your NLP projects. Overall, PyTorch and its various packages offer a strong and flexible toolkit for building and training deep learning models.

PyTorch Installation

To install PyTorch, you can follow the steps below:

1. First, make sure that you have the latest version of pip, the package manager for Python. You can check the version of pip that you have installed by running the following command:

pip --version        

2. If you do not have the latest version of pip, you can update it by running the following command:

pip install --upgrade pip        

3. Once you have the latest version of pip installed, you can install PyTorch by running the following command:

pip install torch        

4. If you want to install a specific version of PyTorch, you can specify the version number by running the following command:

pip install torch==1.7.0        

5. If you want to install PyTorch with GPU support, you will need to install the GPU version of PyTorch. This can be done by running the following command:

pip install torch-gpu        

6. Once PyTorch is installed, you can verify the installation by running the following Python code:

import torch
print(torch.__version__)        

This should print the version number of PyTorch that you have installed.

Creating a Tensor

It is very simple to create Tensors using PyTorch. To create a tensor in PyTorch, you can use the torch.tensor function. This function takes in a list or an array and returns a tensor.

For example, to create a tensor from a list:

import torch
	

# create a tensor 1D
torch.rand(10)
	

> tensor([0.5998, 0.7840, 0.1017, 0.5188, 0.7417, 0.3671, 0.7304, 0.4467, 0.7782, 0.0533])
	

# create a tensor 2D
torch.rand(4,4)
	

> tensor([[0.8032, 0.8408, 0.6130, 0.8030],
	        [0.8343, 0.0078, 0.9008, 0.8328],
	        [0.7299, 0.3207, 0.2760, 0.9779],
	        [0.5821, 0.5505, 0.0618, 0.4833]])
	

# create a tensor integer
torch.randint(low=0, high=100, size=(4,))
	

> tensor([63, 74, 75, 71])
	

# create a tensor from list
torch.Tensor([1,2,3])
	

> tensor([1., 2., 3.])
	

# create a Long Tensor
torch.LongTensor([1,2,3])
	

> tensor([1, 2, 3])
	

torch.FloatTensor([1.1, 1.2, 1.3])
	

> tensor([1.1000, 1.2000, 1.3000])        

Activation Functions

Activation functions are a key component of neural networks, as they introduce non-linearity to the model and allow it to learn more complex patterns. In PyTorch, there are several built-in activation functions that you can use in your neural network layers.

One popular activation function is the softmax function, which is often used in classification tasks. It maps the output of the layer to a probability distribution over the classes, such that the predicted class is the one with the highest probability. The softmax function is defined as:

softmax(x_i) = exp(x_i) / sum_j(exp(x_j))         

where x_i is the ith element of the input tensor and sum_j(exp(x_j)) is the sum of the exponentiated elements of the tensor.

Another common activation function is the ReLU (rectified linear unit) function, which is defined as:

relu(x) = max(0, x)         

The ReLU function outputs the input if it is positive, and outputs 0 if it is negative. It is widely used because it is simple and has been shown to work well in many cases.

Finally, the sigmoid function is often used in binary classification tasks. It maps the input to a value between 0 and 1, making it easy to interpret as a probability. The sigmoid function is defined as:

sigmoid(x) = 1 / (1 + exp(-x))         

These are just a few of the many activation functions available in PyTorch. You can find more information about activation functions and how to use them in the PyTorch documentation.

The following code will show us how we can implement the activation functions to a Tensor.

import torch
	

x = torch.rand(10)
print(x)
	

> tensor([0.2791, 0.7676, 0.5146, 0.5865, 0.5029, 0.5618, 0.2659, 0.9412, 0.4960, 0.1228])
	        
# apply softmax
torch.softmax(x, dim=0)
	

> tensor([0.0778, 0.1268, 0.0984, 0.1058, 0.0973, 0.1032, 0.0768, 0.1508, 0.0966, 0.0665])
	        
# apply sigmoid
torch.sigmoid(x)
	

> tensor([0.5693, 0.6830, 0.6259, 0.6426, 0.6231, 0.6369, 0.5661, 0.7193, 0.6215, 0.5307])
	        
# apply relu
torch.relu(x)
	

> tensor([0.2791, 0.7676, 0.5146, 0.5865, 0.5029, 0.5618, 0.2659, 0.9412, 0.4960, 0.1228])
        

Build the Architecture Neural Network (UNet For Image Segmentation)

The U-Net architecture is a popular choice for image segmentation tasks, as it is able to handle small and large objects and has a good balance between performance and speed. Here is one way to build a U-Net for image segmentation using PyTorch:

import torch
import torch.nn as nn


class UNet(nn.Module):
? ? def __init__(self, in_channels, out_channels):
? ? ? ? super().__init__()
? ? ? ??
? ? ? ? # Encoder layers
? ? ? ? self.conv1 = nn.Conv2d(in_channels, 64, 3, padding=1)
? ? ? ? self.conv2 = nn.Conv2d(64, 64, 3, padding=1)
? ? ? ? self.pool1 = nn.MaxPool2d(2)
? ? ? ??
? ? ? ? self.conv3 = nn.Conv2d(64, 128, 3, padding=1)
? ? ? ? self.conv4 = nn.Conv2d(128, 128, 3, padding=1)
? ? ? ? self.pool2 = nn.MaxPool2d(2)
? ? ? ??
? ? ? ? self.conv5 = nn.Conv2d(128, 256, 3, padding=1)
? ? ? ? self.conv6 = nn.Conv2d(256, 256, 3, padding=1)
? ? ? ? self.pool3 = nn.MaxPool2d(2)
? ? ? ??
? ? ? ? self.conv7 = nn.Conv2d(256, 512, 3, padding=1)
? ? ? ? self.conv8 = nn.Conv2d(512, 512, 3, padding=1)
? ? ? ? self.pool4 = nn.MaxPool2d(2)
? ? ? ??
? ? ? ? self.conv9 = nn.Conv2d(512, 1024, 3, padding=1)
? ? ? ? self.conv10 = nn.Conv2d(1024, 1024, 3, padding=1)
? ? ? ??
? ? ? ? # Decoder layers

? ? ? ? self.up1 = nn.ConvTranspose2d(1024, 512, 2, stride=2)
? ? ? ? self.conv11 = nn.Conv2d(1024, 512, 3, padding=1)
? ? ? ? self.conv12 = nn.Conv2d(512, 512, 3, padding=1)
? ? ? ??
? ? ? ? self.up2 = nn.ConvTranspose2d(512, 256, 2, stride=2)
? ? ? ? self.conv13 = nn.Conv2d(512, 256, 3, padding=1)
? ? ? ? self.conv14 = nn.Conv2d(256, 256, 3, padding=1)
? ? ? ??
? ? ? ? self.up3 = nn.ConvTranspose2d(256, 128, 2, stride=2)
? ? ? ? self.conv15 = nn.Conv2d(256, 128, 3, padding=1)
? ? ? ? self.conv16 = nn.Conv2d(128, 128, 3, padding=1)
? ? ? ??
? ? ? ? self.up4 = nn.ConvTranspose2d(128, 64, 2, stride=2)
? ? ? ? self.conv17 = nn.Conv2d(128, 64, 3, padding=1)
? ? ? ? self.conv18 = nn.Conv2d(64, 64, 3, padding=1)
    
    ? ? self.conv19 = nn.Conv2d(64, out_channels, 3, padding=1)
        self.out = nn.Sigmoid()
    
    
    def forward(self, x):
    ? ? 
        # Encoder

    ? ? x = self.conv1(x)
    ? ? x = self.conv2(x)
    ? ? x1 = self.pool1(x)
    ? ??
    ? ? x = self.conv3(x1)
    ? ? x = self.conv4(x)
    ? ? x2 = self.pool2(x)
    ? ??
    ? ? x = self.conv5(x2)
    ? ? x = self.conv6(x)
    ? ? x3 = self.pool3(x)
    ? ??
    ? ? x = self.conv7(x3)
    ? ? x = self.conv8(x)
    ? ? x4 = self.pool4(x)
    ? ??
    ? ? x = self.conv9(x4)
    ? ? x = self.conv10(x)
    ? ??
    ? ? # Decoder

    ? ? x = self.up1(x)
    ? ? x = torch.cat((x, x4), dim=1)
    ? ? x = self.conv11(x)
    ? ? x = self.conv12(x)
    ? ??
    ? ? x = self.up2(x)
    ? ? x = torch.cat((x, x3), dim=1)
    ? ? x = self.conv13(x)
    ? ? x = self.conv14(x)
    ? ??
    ? ? x = self.up3(x)
    ? ? x = torch.cat((x, x2), dim=1)
    ? ? x = self.conv15(x)
    ? ? x = self.conv16(x)
    ? ??
    ? ? x = self.up4(x)
    ? ? x = torch.cat((x, x1), dim=1)
    ? ? x = self.conv17(x)
    ? ? x = self.conv18(x)
    ? ??
    ? ? x = self.conv19(x)
    ? ? out = self.out(x)
    ? ??
    ? ? return out        

This code defines the U-Net architecture, with the encoder part consisting of a series of convolutional and max pooling layers, and the decoder part consisting of a series of convolutional and transpose convolutional layers. The output of the network is passed through a sigmoid activation function, which maps the output to a range between 0 and 1.

You can then create an instance of the U-Net by calling UNet(in_channels, out_channels), where in_channels is the number of channels in the input image and out_channels is the number of classes in the segmentation task.

To create an instance of the above UNet model, you can use the following code:

# Create an instance of the UNet model

model = UNet(in_channels=3, out_channels=2)
print(model)        

This will create an instance of the UNet model with 3 input channels (assuming the input images are color images with 3 channels: red, green, and blue) and 2 output classes. The print(model) statement will output a summary of the model, including the number of trainable parameters.

Build Dataset Class

To build a custom dataset class in PyTorch, you can subclass the torch.utils.data.Dataset class and override the following methods:

• __init__: this method is used to initialize the dataset and can be used to store the data in memory or on disk.
• __len__: this method should return the length of the dataset (i.e., the number of samples).
• __getitem__: this method should return a sample from the dataset given an index.

Here is an example of a custom dataset class for a dataset containing images and labels:

import torch
from torch.utils.data import Dataset


class ImageLabelDataset(Dataset):
? ? def __init__(self, images, labels):
? ? ? ? self.images = images
? ? ? ? self.labels = labels
? ? ? ??
? ? def __len__(self):
? ? ? ? return len(self.images)
? ??
? ? def __getitem__(self, idx):
? ? ? ? image = self.images[idx]
? ? ? ? label = self.labels[idx]
? ? ? ? return image, label        

To use this dataset class, you can create an instance of it and pass it to a PyTorch data loader. For example:

# Create an instance of the dataset
dataset = ImageLabelDataset(images, labels)


# Create a data loader
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32, shuffle=True)        

This will create a data loader that iterates over the dataset in batches of size 32, shuffling the samples each epoch. You can then use the data loader to train a model by looping over the batches and passing the data to the model.

Optimizer & Loss Function

Optimizers and loss functions are two important components in training a machine learning model. The optimizer is responsible for updating the model parameters based on the computed gradients, and the loss function is used to compute the difference between the predicted output and the true output, which is used to compute the gradients.

Optimizers

In PyTorch, there are several built-in optimizers and loss functions that you can use in your projects. Some popular optimizers include:

• Stochastic gradient descent (SGD): a simple and widely used optimizer that updates the parameters using the gradient of the loss function with respect to the parameters.
• Adam: a more advanced optimizer that uses an adaptive learning rate and momentum to improve convergence.

To use an optimizer in PyTorch, you can create an instance of the optimizer and pass it to the model parameters that you want to optimize.

Loss Function

Some popular loss functions in PyTorch include:

• Cross-entropy loss: a widely used loss function for classification tasks. It measures the difference between the predicted probability distribution over the classes and the true probability distribution.
• Mean squared error (MSE): a loss function commonly used for regression tasks. It measures the difference between the predicted and true outputs by computing the squared difference between them.

Here is the example code for multiple optimizer and loss functions in PyTorch.

# import modules
import torch.nn as nn
import torch.optim as optim
	

# define optimizer

# using Adam optimizer and learning rate = 0.001
optimizer = optim.Adam(model.parameters(), lr=1e-3)
	

print(optimizer)
	

# define loss function
# if the task is regression probem we can use Mean Squared Error
loss_fn = nn.MSELoss()
	

# if the task is binary classification problem, we can use BCELoss
loss_fn = nn.BCELoss()
	

# if the task is multiclass classification problem, we can use CrossEntropy
loss_fn = nn.CrossEntropyLoss()        

Training Routine

To create the training routine for a model in PyTorch, you can follow these steps:

1. Define the model, optimizer, and loss function.
2. Loop over the number of epochs.
3. Loop over the data in the training dataset using the data loader.
4. Pass the data through the model to get the predicted output.
5. Compute the loss using the loss function.
6. Backpropagate the loss to compute the gradients.
7. Use the optimizer to update the model parameters.
8. Optionally, track the loss and other metrics for each epoch.

Here is an example of a training routine for a PyTorch model:

# Loop over the number of epochs

for epoch in range(num_epochs):

    # Loop over the data in the training dataset

    for data, target in dataloader:

        # Pass the data through the model
        output = model(data)
        
        # Compute the loss
        loss = loss_fn(output, target)
        
        # Backpropagate the loss and update the model parameters

        optimizer.zero_grad()

        loss.backward()
        optimizer.step()
        
    # Print the loss for each epoch
    print(f"Epoch {epoch}: Loss = {loss.item():.4f}")        

Conclusion

PyTorch is a very extensive library that researchers are using to build and configure deep learning models to solve real work problems. I have tried to provide an overview of how we can install, set up, and create models using PyTorch. Each section contains an independent example code block just to introduce you to the coding part of PyTorch. I hope this article helps you in creating the models for your own projects and research.

Please feel free to contact me if you have any questions or need any help.