Human-in-the-loop machine learning: a state of the art
By Eduardo Mosqueira-Rey, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, Á. Fernández-Leal. Artificial Intelligence Review, 2022
865 citations recorded by the source corpus.
Researchers are defining new types of interactions between humans and machine learning algorithms generically called human-in-the-loop machine learning. Depending on who is in control of the learning process, we can identify: active learning, in which the system remains in control; interactive machine learning, in which there is a closer interaction between users and learning systems; and machine teaching, where human domain experts have control over the learning process. Aside from control, humans can also be involved in the learning process in other ways. In curriculum learning human domain experts try to impose some structure on the examples presented to improve the learning; in explainable AI the focus is on the ability of the model to explain to humans why a given solution was chosen. This collaboration between AI models and humans should not be limited only to the learning process; if we go further, we can see other terms that arise such as Usable and Useful AI. In this paper we review the state of the art of the techniques involved in the new forms of relationship between humans and ML algorithms. Our contribution is not merely listing the different approaches, but to provide definitions clarifying confusing, varied and sometimes contradictory terms; to elucidate and determine the boundaries between the different methods; and to correlate all the techniques searching for the connections and influences between them.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/62cadbc4fcc73204a72847300cb2214f4401efad
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Structured information extraction from scientific text with large language models (2024, methodology): “The primary advantage of this method is its accessibility and ease-of-use, as LLM-NERRE requires only Like many others, we have found using a human-in-the-loop process can help decrease the time required to collect a full training set 52 .”
- Agentic AI: Autonomous Intelligence for Complex Goals—A Comprehensive Survey (2025, positive): “Such frameworks as Human-in-the-loop (HITL) [68] and human-on-the-loop (HOTL) [69] have been designed with the idea of maintaining an equilibrium between the extent of autonomy experienced and the necessity of human control.”
- Toward Adaptive BCIs: Enhancing Decoding Stability via User State-Aware EEG Filtering (2025, unknown): “However, BCIs continue to face persistent obstacles that limit their long-term adaptability [2].”
- Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Expert reviewers can guide taxonomy refinement, while crowd-sourcing methods help maintain data relevance over time [176].”