Deep Nets: What have They Ever Done for Vision?

By Alan L. Yuille, Chenxi Liu. 2018

107 citations recorded by the source corpus.

This is an opinion paper about the strengths and weaknesses of Deep Nets for vision. They are at the heart of the enormous recent progress in artificial intelligence and are of growing importance in cognitive science and neuroscience. They have had many successes but also have several limitations and there is limited understanding of their inner workings. At present Deep Nets perform very well on specific visual tasks with benchmark datasets but they are much less general purpose, flexible, and adaptive than the human visual system. We argue that Deep Nets in their current form are unlikely to be able to overcome the fundamental problem of computer vision, namely how to deal with the combinatorial explosion, caused by the enormous complexity of natural images, and obtain the rich understanding of visual scenes that the human visual achieves. We argue that this combinatorial explosion takes us into a regime where “big data is not enough” and where we need to rethink our methods for benchmarking performance and evaluating vision algorithms. We stress that, as vision algorithms are increasingly used in real world applications, that performance evaluation is not merely an academic exercise but has important consequences in the real world. It is impractical to review the entire Deep Net literature so we restrict ourselves to a limited range of topics and references which are intended as entry points into the literature. The views expressed in this paper are our own and do not necessarily represent those of anybody else in the computer vision community.

Subfields: Biophysics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Local Relation Networks for Image Recognition (2019, background): “1, since filter learning could potentially face a combinatorial explosion of different valid compositional possibilities [24, 34, 23].”
Relational inductive biases, deep learning, and graph networks (2018, background): “…however, important critiques (Marcus, 2001; Shalev-Shwartz et al., 2017; Lake et al., 2017; Lake and Baroni, 2018; Marcus, 2018a,b; Pearl, 2018; Yuille and Liu, 2018) have highlighted key challenges it faces in complex language and scene understanding, reasoning about structured data,…”
The language of proteins: NLP, machine learning & protein sequences (2021, background): “Deep-learning models also easily overﬁt (even on random noise), and may not necessarily generalize to new, unseen data [120,118].”
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion (2020, background): “The large variability of occluders in terms of their shape, appearance and position introduces an exponential complexity in the data distribution [67] that is unfeasible to be exhaustively represented in finite training data.”