Deep Learning the City: Quantifying Urban Perception at a Global Scale

By Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, César A. Hidalgo. 2016

540 citations recorded by the source corpus.

Computer vision methods that quantify the perception of urban environment are increasingly being used to study the relationship between a city’s physical appearance and the behavior and health of its residents. Yet, the throughput of current methods is too limited to quantify the perception of cities across the world. To tackle this challenge, we introduce a new crowdsourced dataset containing 110,988 images from 56 cities, and 1,170,000 pairwise comparisons provided by 81,630 online volunteers along six perceptual attributes: safe, lively, boring, wealthy, depressing, and beautiful. Using this data, we train a Siamese-like convolutional neural architecture, which learns from a joint classification and ranking loss, to predict human judgments of pairwise image comparisons. Our results show that crowdsourcing combined with neural networks can produce urban perception data at the global scale.

Subfields: Computer Vision and Pattern Recognition, Global and Planetary Change, Transportation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Nam N. Vo
2. Nathan Jacobs
3. James Hays
4. Yi Yang
5. Yunchao Wei
6. Vladlen Koltun
7. Ming-Hsuan Yang
8. Hongdong Li

In Transportation

1. James Hays

Citations in the corpus

• Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “Deep learning has been applied successfully to location prediction [36] and other tasks related to our problem: predicting scene type [40], perceptual attributes [6] such as safety, liveliness and geo-informative attributes [15] like GDP, elevation.”
• A Survey on Mobile Crowdsensing Systems: Challenges, Solutions, and Opportunities (2019, methodology): “Crowdsensed data analyzed with neural networks techniques permits to predict human perceptual response to images [310].”
• Improving Fine-Grained Visual Classification using Pairwise Confusion (2017, unknown): “Subsequently, pairwise neural network models have become common for attribute modeling [24,25,26,27].”
• Subjective and Objective Measures of Streetscape Perceptions: Relationships with Property Value in Shanghai (2021, background): “On the other hand, well-designed and maintained street environment increases residents’ physical activities, lessen their stress, improve their health (Brownson et al., 2004; Jackson, 2003) and perceived safety (Dubey et al., 2016).”