How To Augment Video Data?

Today, data is the key soul of the era, and as the use of the internet grows, so will the amount of data generated. In today’s world, video action recognition has become a hot research topic; nevertheless, constructing video action recognition models requires a large amount of labeled video data. There is a lot of video data available online, but it is dispersed and unlabeled. Hence one of the solution to this is that we use augmentation to improve the quality and quantity of your labeled video data.

What is Data Augmentation?

Data Augmentation is a technique that can be used to artificially expand the size of a training set by creating modified data from the existing one. It is a good practice to use Data Augmentation if you want to prevent overfitting, or the initial dataset is too small to train on, or even if you want to squeeze better performance from your model. Having large datasets is crucial for developing good deep learning models, hence data augmentation is widely used while developing deep learning models.

Data Augmentation can be applied on various datasets like image, text, audio and video. In this blog we will be learning about how to augment the video data.

Augmenting Video Data

For data augmentation for video dataset we will use an amazing library by?AugLy?by Facebook.

Let’s begin by installing the dependencies

!pip install -U augly[video]

!sudo apt-get install python3-magic
!sudo add-apt-repository ppa:jonathonf/ffmpeg-4
!apt install ffmpeg
import augly.utils as utils
import augly.video as vidaugs        

Note-These dependences are specific to google colab. If you are trying to use this library on your local machine then there might be some minute changes that you need to do while installing the dependences. You can take a look at the documentation and install accordingly link to?Augly?documentation

And that’s it! we are now good to go!

For this example I am using a video from the UCF101 video dataset. This is the original video from pizza tossing class on which we will perform augmentation.

Let’s define the input and output paths for our video.

input_video_path=”/content/sample_data/v_PizzaTossing_g10_c02.avi”

output_video_path=”/content/sample_data/out_video.avi”        

Next, we will add Noise to the video.

aug0 = vidaugs.AddNoise()

aug0(input_video_path, output_video_path)        

Here in this video you can clearly see the difference between the original video and the augmented video. Similarly there are many augmentation function provided by AugLy, like:-

Brightness- This function has 3 parameters input video path, output video path and the brightness level which should be in between [-1.0, 1.0].

aug1 = vidaugs.Brightness(0.4)

aug1(input_video_path, output_video_path)        

ColorJitter- This augmentation helps you to augment the video at 3 different levels, brightness, contrast and saturation. Let’s see how we can do it.

aug2 = vidaugs.ColorJitter(0.2,100,1.0)

aug2(input_video_path, output_video_path)        

Rotate- As the name says it rotates your input video. In this example I have set the rotation to 60 degrees you can change it according to your requirements.

aug3 = vidaugs.Rotate(60)

aug3(input_video_path, output_video_path)        

You can also apply multiple augmenters to on a single video for that we need to use compose method. In the below example I have used two augmentations OverlayDots and Rotate.

aug4 = vidaugs.Compose(

[
vidaugs.Rotate(45),
vidaugs.OverlayDots(),
]
)
aug4(input_vid_path, out_vid_path)        

These are some of the augmentations provided by Augly library; there is a wide range of augmentations available. You can learn more about them by visiting their github repo.

I hope now you have an idea of how we can use Augly Library to augment our video dataset. Augmentation, particularly on video data, is incredibly helpful in training your deep learning classification models because it helps to increase your video dataset.

If you found my explanation about video augmentation useful, please share it with your friends and colleagues :).

#machinelearning #machinelearningengineer #machinelearningalgorithms #deeplearning #neuralnetworks #deepneuralnetworks #videos #dataset #data #datascience #dataanalytics #growth #learn #learneveryday #augmentation #research