Kornia: an Open Source Differentiable Computer Vision Library for PyTorch

By Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, Gary R. Bradski. 2019

459 citations recorded by the source corpus.

This work presents Kornia – an open source computer vision library which consists of a set of differentiable routines and modules to solve generic computer vision problems. The package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by OpenCV, Kornia is composed of a set of modules containing operators that can be inserted inside neural networks to train models to perform image transformations, camera calibration, epipolar geometry, and low level image processing techniques, such as filtering and edge detection that operate directly on high dimensional tensor representations. Examples of classical vision problems implemented using our framework are provided including a benchmark comparing to existing vision libraries.

Subfields: Computer Vision and Pattern Recognition, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In ML Systems & Hardware

Citations in the corpus

Towards Robust Vision Transformer (2021, neutral): “Other hyper-parameters are consistent with the implementation in Ko-rnia [32].”
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation (2019, methodology): “ library. We transplanted the operations described in section 3.1 to PyTorch [22], a tensor computation library with automatic differentiation. For geometric operations, we extend functions in kornia [25]. For color-enhancing operations, sample pairing [13] and cutout [6], we implement them using Py”
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance (2022, methodology): “We take random crops of the candidate image and then apply further augmentations such as flipping, color jitter, noising, etc. [42].”
TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning (2020, background): “, 2019], Kornia [Riba et al., 2019], or imgaug [Jung et al.”
Data-Efficient Reinforcement Learning with Self-Predictive Representations (2020, methodology): “We use Kornia (Riba et al., 2020) for efﬁcient GPU-based data augmentations.”
Nerfstudio: A Modular Framework for Neural Radiance Field Development (2023, methodology): “Similarly, frameworks such as PyTorch3D [Ravi et al. 2020] and Kornia [Riba et al. 2020]”
Dataset Distillation by Matching Training Trajectories (2022, methodology): “For KIP, instead of the Kornia [32] ZCA implementation, we use the authors’ custom ZCA implementation for evaluation of their method.”