Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks
By Songül Tolan, Annarosa Pesole, Fernando Martínez-Plumed, Enrique Fernández-Macías, José Hernandez-Orallo, Emilia G'omez. 2020
112 citations recorded by the source corpus.
In this paper we develop a framework for analysing the impact of Artificial Intelligence (AI) on occupations. This framework maps 59 generic tasks from worker surveys and an occupational database to 14 cognitive abilities (that we extract from the cognitive science literature) and these to a comprehensive list of 328 AI benchmarks used to evaluate research intensity across a broad range of different AI areas. The use of cognitive abilities as an intermediate layer, instead of mapping work tasks to AI benchmarks directly, allows for an identification of potential AI exposure for tasks for which AI applications have not been explicitly created. An application of our framework to occupational databases gives insights into the abilities through which AI is most likely to affect jobs and allows for a ranking of occupations with respect to AI exposure. Moreover, we show that some jobs that were not known to be affected by previous waves of automation may now be subject to higher AI exposure. Finally, we find that some of the abilities where AI research is currently very intense are linked to tasks with comparatively limited labour input in the labour markets of advanced economies (e.g., visual and auditory processing using deep learning, and sensorimotor interaction through (deep) reinforcement learning). This article appears in the special track on AI and Society.
Subfields: Industrial and Manufacturing Engineering, Safety Research, Sociology and Political Science, Evaluation & Benchmarks, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Industrial and Manufacturing Engineering
In Safety Research
In Sociology and Political Science
- Robert C. Seamans
- Daniel Rock
- Pamela Mishkin
- Tyna Eloundou
- Sam Manning
- Manav Raj
- Edward W. Felten
- David H. Autor
In Evaluation & Benchmarks
In AI Ethics, Fairness & Society
Citations in the corpus
- GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (2023, background): “(Tolan et al., 2021) • Expert labeling of automation potential for a set of O*NET occupations where experts had high confidence, combined with a probabilistic classifier to estimate automation potential for the remainder of O*NET occupations.”
- The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review (2024, background): “Further arguments suggest that while humans perceive benefits and time savings from integrating AI into decision-making, this reliance may diminish cognitive capabilities by over-shadowing human biological processing abilities (Politanskyi & Klymash, 2019; Tolan et al., 2021).”
- Occupational, industry, and geographic exposure to artificial intelligence: A novel dataset and its potential uses (2021, methodology): “As another example, data from the Equal Employment Opportunity Commission (EEOC) allow researchers to study occupational-level details within firms (e.g., Ferguson & Koning, 2018; Koning & Ferguson, 2019; Tomaskovic-Devey et al., 2016).”
- How will Language Modelers like ChatGPT Affect Occupations and Industries? (2023, background): “Second, the article builds on and extends a set of papers that provide systematic methodologies for studying how AI affects occupations (e.g., Brynjolfsson et al, 2018; Frey & Osborne, 2017; Tolan et al., 2021; Webb, 2020).”