Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping

By Sang Michael Xie, Neal Jean, Marshall Burke, David B. Lobell, Stefano Ermon. 2015

458 citations recorded by the source corpus.

The lack of reliable data in developing countries is a major obstacle to sustainable development, food security, and disaster relief. Poverty data, for example, is typically scarce, sparse in coverage, and labor-intensive to obtain. Remote sensing data such as high-resolution satellite imagery, on the other hand, is becoming increasingly available and inexpensive. Unfortunately, such data is highly unstructured and currently no techniques exist to automatically extract useful insights to inform policy decisions and help direct humanitarian efforts. We propose a novel machine learning approach to extract large-scale socioeconomic indicators from high-resolution satellite imagery. The main challenge is that training data is very scarce, making it difficult to apply modern techniques such as Convolutional Neural Networks (CNN). We therefore propose a transfer learning approach where nighttime light intensities are used as a data-rich proxy. We train a fully convolutional CNN model to predict nighttime lights from daytime imagery, simultaneously learning features that are useful for poverty prediction. The model learns filters identifying different terrains and man-made structures, including roads, buildings, and farmlands, without any supervision beyond nighttime lights. We demonstrate that these learned features are highly informative for poverty mapping, even approaching the predictive performance of survey data collected in the field.

Subfields: Computer Vision and Pattern Recognition, Global and Planetary Change, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Global and Planetary Change

In Media Technology

Citations in the corpus

WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “However, ground truth measurements of poverty are lacking for much of the developing world, since gathering them requires conducting expensive surveys in the field (Blumenstock et al., 2015; Xie et al., 2016; Jean et al., 2016; Yeh et al., 2020).”
Learning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification (2017, background): “Recently, transferring pretrained deep learning network has shown its potential in remote-sensed scene classification [41]–[43].”
A Survey on Negative Transfer (2020, methodology): “[53] proposed a feature-based method to handle daytime scarce satellite image data by transferring knowledge learned from an object classification task with the help of nighttime light intensity information as a bridge.”
Deep Visual Domain Adaptation: A Survey (2018, methodology): “Multi-step Approaches Brief Description Hand-crafted users determine the intermediate domains based on experience [129]”
Applications of Satellite Remote Sensing of Nighttime Light Observations: Advances, Challenges, and Perspectives (2019, background): “For example, with a novel machine learning approach [108,242], the combined information from NTL and high-resolution daytime images have been demonstrated to be helpful for estimating socioeconomic activities (e.”
In-domain representation learning for remote sensing (2019, background): “Xie et al. (2015) developed a transfer learning pipeline for mapping poverty distribution based on transfering the nighttime lights prediction task.”
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness (2020, background): “This is an especially salient obstacle in remote sensing applications, which includes central problems such as predicting poverty, crop type, and land cover from satellite imagery for downstream use in climate change mitigation [52, 21, 49, 40, 38].”
A Decade Survey of Transfer Learning (2010–2020) (2020, methodology): “Moreover, another feature-based method [62] was proposed to deal with scarce satellite image data.”
Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community (2017, background): “Xie et al.310 used transfer learning to avoid training issues due to scarce training data, transfer learning is used.”
Towards better exploiting convolutional neural networks for remote sensing scene classification (2016, background): “Methods based on deep learning have obtained state-of-the-art results in many diﬀerent remote sensing applications, such as image classiﬁcation [15], oil spill [17], poverty mapping [18], and urban planning [19].”
Machine Learning for the Geosciences: Challenges and Opportunities (2017, background): “been explored in [97], [98] for mapping poverty using high-”
Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data (2017, background): “…sustainability applications, such as species distribution modeling (Fink, Damoulas, and Dave 2013; Kelling et al. 2012), poverty mapping (Xie et al. 2015; Ermon et al. 2015), climate modeling (Ristovski et al. 2013), and preventing natural disasters (Boulton, Shotton, and Williams 2016).”
Combining satellite imagery and machine learning to predict poverty (2016, unknown): “We overcome this challenge through a multistep “transfer learning” (14) approach (see supplementary materials section 1), whereby a noisy but easily obtained proxy for poverty is used to train a deep learning model (15).”
ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data (2019, methodology): “…DST 5) in the ISPRS competition used networks with pre-trained weights on external large data sets (e.g. Ima-geNet, Deng et al., 2009) and ﬁne-tuning, i.e. a methodology called transfer learning (Pan and Yang, 2010, see also Penatti et al. (2015), Xie et al. (2015) for remote sensing applications).”
Distant Domain Transfer Learning (2017, background): “Transfer Learning In this category, we choose five transfer learning algorithms including Adaptive SVM (ASVM) (Aytar and Zisserman 2011), Geodesic Flow Kernel (GFK) (Gong et al. 2012), Landmark (LAN) (Gong, Grauman, and Sha 2013), Deep Transfer Learning (DTL) (Yosinski et al. 2014), and Transitive T”