Functional Map of the World

By Gordon A. Christie, Neil Fendley, James Wilson, Ryan Mukherjee. 2017

530 citations recorded by the source corpus.

We present a new dataset, Functional Map of the World (fMoW), which aims to inspire the development of machine learning models capable of predicting the functional purpose of buildings and land use from temporal sequences of satellite images and a rich set of metadata features. The metadata provided with each image enables reasoning about location, time, sun angles, physical sizes, and other features when making predictions about objects in the image. Our dataset consists of over 1 million images from over 200 countries. For each image, we provide at least one bounding box annotation containing one of 63 categories, including a "false detection" category. We present an analysis of the dataset along with baseline approaches that reason about metadata and temporal views. Our data, code, and pretrained models have been made publicly available.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, background): “We examine the conditional and unconditional generation qualities of D2C over CIFAR-10 [53], CIFAR-100 [53], fMoW [21], CelebA-64 [58], CelebA-HQ-256 [48], and FFHQ-256 [49].”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, background): “1: Model soups improve accuracy when fine-tuning on the diverse classification tasks WILDS-FMoW (Koh et al., 2021; Christie et al., 2018) and WILDS-iWildCam (Koh et al.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “We study this problem through a variant of the Functional Map of the World dataset (Christie et al., 2018).”
Robust fine-tuning of zero-shot models (2021, methodology): “In addition to the aforementioned ImageNet distribution shifts, WiSE-FT consistently improves robustness on a diverse set of six further distribution shifts including: (i) geographic shifts in satellite imagery and wildlife recognition (WILDS-FMoW, WILDS-iWildCam) (Koh et al., 2021; Christie et al.,”
Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning (2022, methodology): “Our analysis uses eight different land-use classification datasets: RESISC-45 [11], the UC Merced Land Use Dataset [62], AiRound and CV-BrCT [42], MLRSNet [46], EuroSAT [28], Optimal-31 [53], WHU-RS19 [14], SpaceNet v1 and v2 [51], and Functional Map of the World [12].”
Satlas: A Large-Scale, Multi-Task Dataset for Remote Sensing Image Understanding (2022, methodology): “S AT N ET is inspired by recent work that employ task-speciﬁc output heads [20, 27, 32], as well as methods that synthesize features across remote sensing image time series [21,25].”
Datamodels: Predicting Predictions from Training Data (2022, background): “FMoW (Christie et al., 2018) is a land use classification dataset based on satellite imagery.”
DataComp: In search of the next generation of multimodal datasets (2023, background): “85,054 2 accuracy FMoW [25, 76] Satellite imagery recognition 22,108 62 worst-region acc.”
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution (2022, unknown): “• FMoW Geo-shift is adapted from the satellite remote sensing dataset Functional Map of the World (Christie et al., 2018; Koh et al., 2021).”
SpaceNet: A Remote Sensing Dataset and Challenge Series (2018, methodology): “The second SpaceNet challenge aimed to extract building footprints from the DigitalGlobe WorldView 3 satellite in a continuous image strip.”
An Empirical Study of Remote Sensing Pretraining (2022, background): “overlapping scenes, exceeding the competitive counterpart fMoW [54] and BigEarthNet [55], which separately includes 132,716 and 590,326 scenes.”
Visual Prompting: Modifying Pixel Space to Adapt Pre-trained Models (2022, methodology): “training distribution di ers from the test distribution, we evaluate on three image classi cation datasets in WILDS [35]: Camelyon17 [36], FMoW [37], and iWildCAM [38].”
Exploring Visual Prompts for Adapting Large-Scale Models (2022, methodology): “We also measure robustness to distribution shift, i.e., training distribution differs from the test distribution, by evaluating on three image classiﬁcation datasets in WILDS [23]: Camelyon17 [1], FMoW [8], and iWildCAM [2].”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, unknown): “Dataset #Domain #Class #Sample Description Reference Office-Caltech 4 10 2,533 Caltech, Amazon, Webcam, DSLR [230] Office-31 3 31 4,110 Amazon, Webcam, DSLR [230] PACS 4 7 9,991 Art, Cartoon, Photos, Sketches [2] VLCS 4 5 10,729 Caltech101, LabelMe, SUN09, VOC2007 [231] Office-Home 4 65 15,588 Art, ”
Geography-Aware Self-Supervised Learning (2020, methodology): “Our experiments on the functional Map of the World [4] dataset consisting of high spatial resolution satellite images show that we improve MoCo-v2 baseline significantly.”
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022, background): “Since our data has much shorter temporal sequence lengths [16], we let PT = 1 such that LT = L · T .”
FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding (2020, methodology): “A collection of images of buildings and lands named Functional Map of the World (fMoW) was introduced by Christie et al. in [14].”