Kornia (PyTorch)

 – Computer Vision Library for Deep Learning

Computer vision has driven a lot of advances in modern society in various industries such as self-driving cars, industrial robotics, visual effects, image search, etc resulting in a wide field of applications. One of the key components of this achievement is due to the open-source software and the community which provides open-source implementations in main computer vision algorithms.

OpenCV is a library of programming function that helps to provide a common interface between human and machine perception. OpenCV has more than 2500 optimized algorithms that include classic, state of the art computer vision and machine learning algorithms.

We all know about OpenCV and its applications for the past three decades. Out of its vast applications, a very few problems have also occurred. One of the major issues is that OpenCV is not differentiable, like functions of computer vision are operators placed as layers in the neural networks for training via backpropagation.

Kornia is an advanced computer vision library especially for deep learning, it can be also called as differentiable computer vision library. It provides high-level interfaces to computer vision algorithms computed on tensors. 

By comparing different computer vision libraries with their main features, Kornia and TensorFlow. Image is the only frameworks that implements their functionalities in the GPU, using differentiable batched data, with the ability to be distributed. By using our library, classical vision problems such as image registration by holography, depth estimation, or local features matching can be very easily solved with high performance existing libraries.

What is differentiable computer vision library?

 Computer vision library contains a set of differentiable routines and modules to solve generic problems. Auto-differentiation defines and calculates the gradient complex functions. Inspired by OpenCV, Kornia has a set of modules with operators in neural networks to train models to do image transformations, camera calibration, epipolar geometry, and in low level image processing techniques, such as filtering and edge detection that works on high dimensional tensors.

Image processing algorithms can be defined as Direct Acyclic Graph (DAG) which can be inserted in a neural network and can be optimized during training, by making use of the reverse-mode auto-differentiation, compute gradients via backpropagation. In simpler words, computer vision functions are operators that can be placed as layers in the neural networks for training via backpropagation.

The main features that distincts kornia from OpenCV are:-

Parallel programming: -Parallel batch processing is an important aspect that runs operator using parallelism of data. The assumed input from the operators receives N-channel image tensors in batches, which opposes standard libraries with 1-3 channel images. Hence, multi hyperspectral or huge volumes of images can be done directly using Kornia.

Distributed:- It provides support for communication operators in many nodes that run on one or many groups of local devices. The library is designed to allow users to run their applications in varying distributed systems, and even it can process large vision data in an efficient way.

Is Kornia, a rival to OpenCV?

Kornia seeks in implementating OpenCV in research purposes which means that it mimics the main functionalities and adds the ability to backpropagate through the different operators. However, note that Kornia cannot act as a replacement for OpenCV, since it cannot be optimized in the process of production to achieve high-performances in embedded devices. As we know Kornia is inspired by OpenCV, it is not a rival to OpenCV. By considering the widespread use of OpenCV in multiple languages (c++, java, python etc.) kornia along with OpenCV can solve problems of both the packages.

Thus, implementation of new library must be kept on the conquest of optimizing and in making computer vision algorithms more effectual.

-Meghraj Arli, MBA(BA, AI & ML) Student, Woxsen University, Hyderabad &

Dr. Hemachandran K, Professor, School of Business (AI & ML), Woxsen University, Hyderabad