Learning Concept Embeddings with Combined Human-Machine Expertise

By Michael J. Wilber, Iljung S. Kwak, David J. Kriegman, Serge J. Belongie. 2015

40 citations recorded by the source corpus.

This paper presents our work on "SNaCK," a low-dimensional concept embedding algorithm that combines human expertise with automatic machine similarity kernels. Both parts are complimentary: human insight can capture relationships that are not apparent from the object's visual similarity and the machine can help relieve the human from having to exhaustively specify many constraints. We show that our SNaCK embeddings are useful in several tasks: distinguishing prime and nonprime numbers on MNIST, discovering labeling mistakes in the Caltech UCSD Birds (CUB) dataset with the help of deep-learned features, creating training datasets for bird classifiers, capturing subjective human taste on a new dataset of 10,000 foods, and qualitatively exploring an unstructured set of pictographic characters. Comparisons with the state-of-the-art in these tasks show that SNaCK produces better concept embeddings that require less human supervision than the leading methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization (2020, methodology): “…embedding from triples of points, with comparisons between points in the form of “a is more similar to b than to c.” Approaches using triplets include DrLIM (Hadsell et al., 2006), t-STE (van der Maaten and Weinberger, 2012), SNaCK (Wilber et al., 2015) and TriMAP (Amid and Warmuth, 2019).”
Probabilistic End-To-End Noise Correction for Learning With Noisy Labels (2019, background): “CUB-200: We tested the robustness of our framework in a fine-grained classification dataset CUB-200.”
Disentangling Nonlinear Perceptual Embeddings With Multi-Query Triplet Networks (2016, background): “[27], but this work did not include joint feature and embedding learning.”
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition (2018, background): “[26] and [27] incorporate human knowledge and various types of computer vision algorithms into a human-in-the-loop framework for the complementary strengths of both ends.”
Fine-Grained Image Classification by Exploring Bipartite-Graph Labels (2015, background): “[55] developed hybrid systems to introduce human in the loop for localizing discriminative regions for computing features.”
Integrating Scene Text and Visual Appearance for Fine-Grained Image Classification (2017, background): “ive similarity. Xie et al. [18] proposed a novel data augmentation approach to address the scarcity of data and a novel regularization technique to improve the generalization performance. In [11] and [19], the authors proposed to include humans in loop to select discriminative features. The most rel”