Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher-AI Complementarity
By Kenneth Holstein, Bruce M. McLaren, Vincent A. W. M. M. Aleven. Journal of Learning Analytics, 2019
365 citations recorded by the source corpus.
Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article reports on the iterative co-design, development, and classroom evaluation of Konscia, a wearable, real-time awareness tool for teachers working in AI-enhanced K-12 classrooms. In the process, we argue that the co-design of LA systems requires new kinds of prototyping methods. We introduce one of our own prototyping methods, REs, to address unique challenges of co-prototyping LA tools. This work presents the first end-to-end demonstration of how non-technical stakeholders can participate throughout the whole design process for a complex LA system—from early generative phases to the selection and tuning of analytics to evaluation in real-world contexts. We conclude by providing methodological recommendations for future LA co-design efforts.
Subfields: Computer Science Applications, Developmental and Educational Psychology, Information Systems
View Paper: https://www.semanticscholar.org/paper/de56e7764dc481c803027a9627cebf05fc6137c3
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Science Applications
In Developmental and Educational Psychology
In Information Systems
Citations in the corpus
- Leading teachers' perspective on teacher-AI collaboration in education (2023, neutral): “The human (teacher and student)-AI interaction in the realm of AIED has mainly driven from two different perspectives: (1) the replacement perspective and (2) the augmentation perspective (Holstein et al., 2019a, b; Cukurova et al., 2019; Mavrikis et al., 2021).”
- The promise and challenges of generative AI in education (2024, neutral): “…teaching in the same way; some domains and teaching aspects will require hybrid intelligence approaches (e.g. support teacher – AI complementarity; Holstein, McLaren, and Aleven 2019), where these broader domain objectives are met by systems that synergistically combine the complementary…”
- Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “Forest inventory also plays a crucial role in forest engineering, as it provides critical information on forest characteristics, such as tree species, size, and density, which can inform forest management decisions [102,103].”
- Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT (2023, background): “In addition to these theoretical explorations, some, albeit limited, empirical research demonstrated how human teachers can actually complement AI (e.g., Holstein et al, 2019).”
- SAGE-RAI: Design Patterns for Transparent RAG Systems (2026, unknown): “The assistance-independence tension engages Holstein et al.’s [13] identification of tensions between AI support and learning autonomy.”