Automated Inference on Criminality using Face Images

By Xiaolin Wu, Xi Zhang. 2016

118 citations recorded by the source corpus.

We study, for the first time, automated inference on criminality based solely on still face images. Via supervised machine learning, we build four classifiers (logistic regression, KNN, SVM, CNN) using facial images of 1856 real persons controlled for race, gender, age and facial expressions, nearly half of whom were convicted criminals, for discriminating between criminals and non-criminals. All four classifiers perform consistently well and produce evidence for the validity of automated face-induced inference on criminality, despite the historical controversy surrounding the topic. Also, we find some discriminating structural features for predicting criminality, such as lip curvature, eye inner corner distance, and the so-called nose-mouth angle. Above all, the most important discovery of this research is that criminal and non-criminal face images populate two quite distinctive manifolds. The variation among criminal faces is significantly greater than that of the non-criminal faces. The two manifolds consisting of criminal and non-criminal faces appear to be concentric, with the non-criminal manifold lying in the kernel with a smaller span, exhibiting a law of normality for faces of non-criminals. In other words, the faces of general law-biding public have a greater degree of resemblance compared with the faces of criminals, or criminals have a higher degree of dissimilarity in facial appearance than normal people.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

LOGAN: Membership Inference Attacks Against Generative Models (2017, background): “) Similarly, if images from a database of criminals are used to train a face generation algorithm [67], membership inference may expose an individual’s criminal history.”
Designing a Moral Compass for the Future of Computer Vision Using Speculative Analysis (2017, unknown): “With computer vision research emerging that attempts to predict criminality of a face [60], and the many advances of object detection and”
The Profiling Potential of Computer Vision and the Challenge of Computational Empiricism (2019, methodology): “Most telling is the authors’ acknowledgment that the variance between criminal and non-criminal populations is not evident from visual assessments or simple Euclidean measurements [82].”
How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, background): “For example, the perceived gender (e.g., [136, 141, 147]) or ethnicity (e.g., [60, 100, 103]) of a face; whether the face is considered beautiful or not (e.g., [41, 170]); and even what the face can tell us about a person’s criminality, sexuality, or intelligence (e.g., [107, 168, 173]).”
Goal-Oriented Design for Ethical Machine Learning and NLP (2017, background): “Here are two conclusions from an abstract on automated infer-88 ence of criminality using faces (Wu and Zhang, 2016): All four classifiers perform consistently well and produce evidence for the validity of automated face-induced inference on criminality… Also, we find some discriminating structural…”
Pitfalls in Machine Learning Research: Reexamining the Development Cycle (2020, unknown): “Recent examples of this include algorithms for determining criminal tendencies [53, 22] and detecting sexual orientation [51] from photos of faces, as well as commercial services that automatically determine suitable job candidates for hiring managers [12].”
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (2018, background): “And others such as (Wu and Zhang, 2016) and Israeli based company Faception (Faception) have developed software that purports to determine an individual’s characteristics (e.g. propensity towards crime, IQ, terrorism) solely from their faces.”