Measuring Dataset Granularity

By Yin Cui, Zeqi Gu, Dhruv Kumar Mahajan, L. V. D. van der Maaten, Serge J. Belongie, Ser-Nam Lim. 2019

14 citations recorded by the source corpus.

Despite the increasing visibility of fine-grained recognition in our field, "fine-grained'' has thus far lacked a precise definition. In this work, building upon clustering theory, we pursue a framework for measuring dataset granularity. We argue that dataset granularity should depend not only on the data samples and their labels, but also on the distance function we choose. We propose an axiomatic framework to capture desired properties for a dataset granularity measure and provide examples of measures that satisfy these properties. We assess each measure via experiments on datasets with hierarchical labels of varying granularity. When measuring granularity in commonly used datasets with our measure, we find that certain datasets that are widely considered fine-grained in fact contain subsets of considerable size that are substantially more coarse-grained than datasets generally regarded as coarse-grained. We also investigate the interplay between dataset granularity with a variety of factors and find that fine-grained datasets are more difficult to learn from, more difficult to transfer to, more difficult to perform few-shot learning with, and more vulnerable to adversarial attacks.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

Grafit: Learning fine-grained image representations with coarse labels (2020, background): “The impact of pretraining granularity is discussed in prior works [15, 67].”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “, the designation lacks a precise definition [189], [190].”