Real-World Image Datasets for Federated Learning

By Jiahuan Luo, Xueyang Wu, Yu Luo, Anbu Huang, Yunfeng Huang, Yang Liu, Qiang Yang. 2019

111 citations recorded by the source corpus.

Federated learning is a new machine learning paradigm which allows data parties to build machine learning models collaboratively while keeping their data secure and private. While research efforts on federated learning have been growing tremendously in the past two years, most existing works still depend on pre-existing public datasets and artificial partitions to simulate data federations due to the lack of high-quality labeled data generated from real-world edge applications. Consequently, advances on benchmark and model evaluations for federated learning have been lagging behind. In this paper, we introduce a real-world image dataset. The dataset contains more than 900 images generated from 26 street cameras and 7 object categories annotated with detailed bounding box. The data distribution is non-IID and unbalanced, reflecting the characteristic real-world federated learning scenarios. Based on this dataset, we implemented two mainstream object detection algorithms (YOLO and Faster R-CNN) and provided an extensive benchmark on model performance, efficiency, and communication in a federated learning setting. Both the dataset and algorithms are made publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bo Du
2. Guodong Long
3. Mang Ye
4. Bingsheng He
5. Qinbin Li
6. Jing Jiang
7. Chengqi Zhang
8. Tianyi Zhou

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Bo Du
3. Mang Ye
4. Bingsheng He
5. Qinbin Li
6. Pong Chi Yuen
7. Ming Ding
8. Tzu-Ming Harry Hsu

In Evaluation & Benchmarks

1. Dacheng Tao
2. Hangyu Zhu
3. Tao Shen
4. Jun Li
5. Ming Ding
6. Bo Du
7. Jing Jiang
8. Dinh C. Nguyen

In Federated & Distributed Learning

1. Bingsheng He
2. Qinbin Li
3. Matthew Brown
4. Tzu-Ming Harry Hsu
5. Hang Qi
6. Hangyu Zhu
7. Dacheng Tao
8. Ming Ding

Citations in the corpus

• The Non-IID Data Quagmire of Decentralized Machine Learning (2019, background): “18) provides datasets that are partitioned in various ways. Luo et al. release 900 images collected from cameras in different locations, and they show severe skewed label distribution across cameras (Luo et al., 2019). Our study on geo-tagged mammals on Flickr shows the same problem at a much larger”
• Federated Visual Classification with Real-World Data Distribution (2020, background): “[18] propose a dataset containing 900 images from 26 street-level cameras, which they use to train object detectors.”
• Collaborative Unsupervised Visual Representation Learning from Decentralized Data (2021, background): “Federated learning is an emerging distributed training technique [21] that have empowered multiple parties to collaboratively learn computer vision models, such as segmentation [18, 28], object detection [20], and person re-identification [42].”
• Federated Learning for Internet of Things: A Comprehensive Survey (2021, background): “clients to accomplish an FL task, such as object detection [35], while keeping training data secure and private.”
• Federated Learning on Non-IID Data: A Survey (2021, background): “• Although two real image dataset are introduced in [91, 54], a universal homogeneous and heterogeneous benchmark dataset still lacks for FL.”
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “[124] present real-world image datasets which are collected from 26 different street cameras.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “[138] introduce a federated dataset for object detection.”
• FedProto: Federated Prototype Learning across Heterogeneous Clients (2021, methodology): “2020) and street image object detection (Luo et al. 2019).”
• Multi-center federated learning: clients clustering for better personalization (2020, background): “[40] proposed an object detection-based dataset for FL. Heterogeneity is a core challenge in the federated setting and has been widely studied from various perspectives.”