Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning

By Mohammad Sadegh Norouzzadeh, Anh Totti Nguyen, Margaret Kosmala, A. Swanson, Meredith S. Palmer, Craig Packer, Jeff Clune. 2017

1,085 citations recorded by the source corpus.

Significance Motion-sensor cameras in natural habitats offer the opportunity to inexpensively and unobtrusively gather vast amounts of data on animals in the wild. A key obstacle to harnessing their potential is the great cost of having humans analyze each image. Here, we demonstrate that a cutting-edge type of artificial intelligence called deep neural networks can automatically extract such invaluable information. For example, we show deep learning can automate animal identification for 99.3% of the 3.2 million-image Snapshot Serengeti dataset while performing at the same 96.6% accuracy of crowdsourced teams of human volunteers. Automatically, accurately, and inexpensively collecting such data could help catalyze the transformation of many fields of ecology, wildlife biology, zoology, conservation biology, and animal behavior into “big data” sciences. Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would improve our ability to study and conserve ecosystems. We investigate the ability to automatically, accurately, and inexpensively collect such data, which could help catalyze the transformation of many fields of ecology, wildlife biology, zoology, conservation biology, and animal behavior into “big data” sciences. Motion-sensor “camera traps” enable collecting wildlife pictures inexpensively, unobtrusively, and frequently. However, extracting information from these pictures remains an expensive, time-consuming, manual task. We demonstrate that such information can be automatically extracted by deep learning, a cutting-edge type of artificial intelligence. We train deep convolutional neural networks to identify, count, and describe the behaviors of 48 species in the 3.2 million-image Snapshot Serengeti dataset. Our deep neural networks automatically identify animals with >93.8% accuracy, and we expect that number to improve rapidly in years to come. More importantly, if our system classifies only images it is confident about, our system can automate animal identification for 99.3% of the data while still performing at the same 96.6% accuracy as that of crowdsourced teams of human volunteers, saving >8.4 y (i.e., >17,000 h at 40 h/wk) of human labeling effort on this 3.2 million-image dataset. Those efficiency gains highlight the importance of using deep neural networks to automate data extraction from camera-trap images, reducing a roadblock for this widely used technology. Our results suggest that deep learning could enable the inexpensive, unobtrusive, high-volume, and even real-time collection of a wealth of information about vast numbers of animals in the wild.

Subfields: Computer Vision and Pattern Recognition, Ecology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Ecology

Citations in the corpus

Convolutional neural network: a review of models, methodologies and applications to object detection (2019, background): “Finally, some areas of applications are discussed, which mainly embrace wild animal detection [34, 35], arm detection [36, 37] and miscellaneous object detection.”
Perspectives in machine learning for wildlife conservation (2021, background): “the purpose of population estimates is becoming increasingly popular for images (58, 59), acoustic spectrograms (101), and videos (102).”
Efficient Pipeline for Camera Trap Image Review (2019, result): “Previous work has shown good results on automated species classification in camera trap data [8], but further analysis has shown that these results do not generalize to new cameras or new geographical regions [3].”
Identifying animal species in camera trap images using deep learning and citizen science (2018, methodology): “Norouzzadeh et al. (2018) demonstrated, however, that categorical (nonlocalized) la - bels can be sufficient to train a model to jointly identify species and other labels provided by citizen scientists, such as count or animal behaviour categories.”
Applications for deep learning in ecology (2018, methodology): “Beyond species identification, the timing of species presence in any given site can also be measured with time labels tailored to species lifecycles 20 .”
Ultragentle manipulation of delicate structures using a soft robotic gripper (2019, methodology): “For example, in situ characterization of physiological and genomic properties of animals could be achieved through integration of sensors into these ultragentle actuators (52, 53) and using deep neural networks to automate tasks such as species identification (54).”
Deep learning and computer vision will transform entomology (2020, background): “undergone rapid development in the past decade (14, 16-18).”
Tackling Climate Change with Machine Learning (2019, methodology): “[535] Mohammad Sadegh Norouzzadeh, Anh Nguyen, Margaret Kosmala, Alexandra Swanson, Meredith S Palmer, Craig Packer, and Jeff Clune.”
Machine learning and deep learning—A review for ecologists (2022, methodology): “…architecture that uses kernels (convolution matrices) to detect certain shapes and which is used, for example, in identifying species (Ferreira et al., 2020 ), automatic monitoring of species (Norouzzadeh et al., 2018; Tuia et al., 2022 ), or landscape classification (Stupariu et al., 2022 ).”