Learning and the Unknown: Surveying Steps toward Open World Recognition

By Terrance E. Boult, Steve Cruz, Akshay Raj Dhamija, Manuel Günther, James Henrydoss, Walter J. Scheirer. 2019

159 citations recorded by the source corpus.

As science attempts to close the gap between man and machine by building systems capable of learning, we must embrace the importance of the unknown. The ability to differentiate between known and unknown can be considered a critical element of any intelligent self-learning system. The ability to reject uncertain inputs has a very long history in machine learning, as does including a background or garbage class to account for inputs that are not of interest. This paper explains why neither of these is genuinely sufficient for handling unknown inputs – uncertain is not unknown, and unknowns need not appear to be uncertain to a learning system. The past decade has seen the formalization and development of many open set algorithms, which provably bound the risk from unknown classes. We summarize the state of the art, core ideas, and results and explain why, despite the efforts to date, the current techniques are genuinely insufficient for handling unknown inputs, especially for deep networks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Few-Shot Continual Active Learning by a Robot (2022, background): “Another related field to active learning is open-set recognition (or out of the distribution detection (OOD)) [37, 38, 38, 39], in which a model that is trained on a set of tasks/classes might face samples belonging previously unknown classes in the test set.”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “However, paying homage to the detailed argumentation of the recent review by Boult et al. (2019), any machine learning model is still trained in a closed world scenario, independently of whether e.g. a Bayesian formalism is employed to obtain uncertainties.”
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition (2019, methodology): “[31] and corresponding follow-up literature [8], [9], [32], [33]: For any recognition function f over an input space X , the open space O is defined as O ⊂= X − SK , where SK is a union of balls of radius ro including all of the training examples for known classes x ∈ K.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “…(Ruff et al., 2021; Pang et al., 2020; Bulusu et al., 2020; Chalapathy & Chawla, 2019), novelty detection (ND) (Pimentel et al., 2014; Miljkovi´c, 2010; Markou & Singh, 2003a; Markou & Singh, 2003b), and open set recognition (OSR) (Boult et al., 2019; Huang & Chen, 2020; Mahdavi & Carvalho, 2021).”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “In [115], a semi-supervised Bayesian AL model, which is a deep generative-based model that uses BNNs to give discriminative component, was developed.”
Learning Bounds for Open-Set Learning (2021, background): “While many OSL algorithms can be roughly interpreted as minimizing the open-space risk or using the extreme value theory, several disconnections still form non-negligible gaps between the theories and algorithms (Boult et al., 2019; Geng et al., 2018).”