Pytorch

Deep Learning and Artificial Intelligence have been one of the hottest topics of 21st century. While there are many out there like TensorFlow, PyTorch has always stood out as one of the best open source machine learning framework for research prototyping. To put it formally, PyTorch is a scientific computing framework with wide support for machine learning algorithms that put GPUs first. It was primarily developed by Facebook’s artificial intelligence research group.

If you are new to PyTorch, this is the prefect blog for you to get started (provided you have a basic understanding of Python) as we will cover the very fundamentals of PyTorch here.

PyTorch and Imperative programming

If you have some background in coding, imperative programming may not be a new concept to you. To distinguish imperative programming from its counterpart symbolic programming we will look at some examples.

import torch
a = torch.tensor(1.0)
b = torch.tensor(1.0)
c = a + b
print('c:'c)        

The above code indicates Imperative programming paradigm of computer programming where the program itself describes steps that change the state of the computer. Here the computation is defined as we type it.

import tensorflow as tf
sess = tf.Session()
a = tf.placeholder(tf.int8)
b = tf.placeholder(tf.int8)
sess.run(a+b, feed_dict={a: 1, b: 1})        

This code snippet portrays symbolic programming paradigm where the computation is defined first and then executed.

As you might have noticed, PyTorch uses imperative programming paradigm which may be relatively easier to follow.

PyTorch Tensors

To give you an overview of tensors let me use the same explanation as provided in the official documentation page of PyTorch.

A PyTorch Tensor is basically the same as a numpy array: it does not know anything about deep learning or computational graphs or gradients, and is just a generic n-dimensional array to be used for arbitrary numeric computation.

If you did not understand what a numpy array is, don’t worry. It is nothing more than an array data structure but more computationally efficient. PyTorch Tensors are similar and can be used as a replacement for numpy arrays.

To declare a Pytorch tensor first we should import the torch module. After that we can provide a python list as an input to the torch.tensor() method

import torch
a = torch.tensor([1,2,3])        

This would declare an tensor ‘a’ of type ‘torch.LongTensor ’ with a data type ‘torch.int64’. You can verify this by typing the following command:

print(a.type())
print(a.dtype)        

You can also explicitly define a Tensor of a certain type with the following command:

b = torch.FloatTensor([1,2,3])
print(b.type())        

This would return a ‘torch.FloatTensor’ type. You can learn more about Tensor types and datatypes here.

You can also declare a Tensor with the torch.linspace() method

c = torch.linspace(-2,2,steps=4)
print(c)#Output
tensor([-2.0000, -0.6667,  0.6667,  2.0000])        

Tensor dimension and size

A Tensor may be uni-dimensional or multidimensional. To see the number of dimensions in a Tensor we could use the ndimension() method :

a = torch.tensor([[1,2],[3,4]])
print(a.ndimension())#Output
2        

Unlike ndimension(), the size() method returns the shape of the tensor :

a = torch.tensor([[1,2],[3,4]])
print(a.size())#Output
torch.Size([2, 2])        

To reshape a Tensor we can use the view() method and pass the required shape of the tensor as input to the same.

a = torch.tensor([1,2,3,4,5,6])
print(a.view(6,1))#Output
tensor([[1],
        [2],
        [3],
        [4],
        [5],
        [6]])        

As stated earlier a PyTorch tensor is similar to a numpy array and can be used as a replacement for the same but that doesn’t restrict you from converting a Torch tensor to numpy array and vice versa. To convert a numpy array to a tensor and vice versa you can use the following code.

import numpy as np
import torch
np_ar = np.array([0.0,1.0,2.0,3.0])
torch_tensor = torch.from_numpy(np_ar)
back_to_numpy = torch_tensor.numpy()
        

A pandas series object can also be converted to a torch tensor similarly.

Indexing and Slicing

Indexing and slicing in torch tensors can be done similar to that in regular python lists and numpy arrays. The following code snippet would give you an overview of indexing and slicing in torch tensors

a = torch.tensor([[1,2],[3,4]])
print(a[0])
print(a[0][1])
print(a[0,1])
print(a[:,0])
print(a[:,1])
print(a[0:][1])#Output
tensor([1, 2])
tensor(2)
tensor(2)
tensor([1, 3])
tensor([2, 4])
tensor([3, 4])        

Basic Operations in PyTorch

To perform basic tensor operations like addition, multiplication, and subtraction we can do the following:

a = torch.tensor([[1,2],[3,4]])
b = torch.tensor([[1,1],[1,1]])print(a+b)
print(a*b)
print(a-b)#Output
tensor([[2, 3],         
        [4, 5]]) 
tensor([[1, 2],
        [3, 4]])
tensor([[0, 1],
        [2, 3]])        

For one-dimensional tensors a dot operation can be performed using the following code:

a = torch.tensor([1,2,3])
b = torch.tensor([4,5,6])
print(torch.dot(a,b))#Output
tensor(32)        

In the above example a dot operation does the following :

1*4 + 2*5 + 3*6 = 32

Two matrices can also be multiplied using the following code:

a = torch.tensor([[1,2],[3,4],[5,6]])
b = torch.tensor([[1,1,1],[1,1,1]])
print(a.size())print(b.size())
print(torch.mm(a,b))#Output
torch.Size([3, 2])
torch.Size([2, 3])
tensor([[ 3,  3,  3],
        [ 7,  7,  7],
        [11, 11, 11]])        

Derivatives in PyTorch

We can also apply derivatives on functions and yield corresponding results using PyTorch. We can start off by declaring a tensor variable x with requires_grad property of tensor() method set to True

x= torch.tensor(2.0 ,requires_grad=True)        

The requires_grad is set to True to store all operations associated with that variable.

Then we define our function, say y=x**2

y = x**2        

Now we use the backward() method on y to find the derivative of y with respect to x.

y.backward()        

Once the derivative is calculated we then find the value of derivative with respect to x at x = 2 by applying grad on x.

x.grad#Output
tensor(4.)        

Conclusion

Congratulation, now that you have covered the basics you are all set to dive into the world of Deep Learning with PyTorch.