Evaluating the Quality of Learning Resources: A Learnersourcing Approach

By Solmaz Abdi, Hassan Khosravi, Shazia Wasim Sadiq, Gianluca Demartini. 2021

41 citations recorded by the source corpus.

Learnersourcing is emerging as a viable approach for mobilizing the learner community and harnessing the intelligence of learners as creators of learning resources. Previous works have demonstrated that the quality of resources developed by students is quite diverse with some resources meeting rigorous judgmental criteria, whereas other resources are ineffective, inappropriate, or incorrect. Consequently, to effectively utilize these large repositories of resources in student learning, there is a need for a selection and moderation process to separate high-quality resources from low-quality ones in such repositories. Instructors and domain experts are potentially the most reliable source for doing this task; however, their availability is often quite limited. This article explores whether and how learnersourcing, as an alternative approach, can be used for evaluating the quality of learning resources. To do so, we first follow a data-driven approach to explore students’ ability in judging the quality of learning resources. Results from this study suggest that, overall, ratings provided by students strongly correlate with ratings from experts; however, students’ ability in evaluating learning resources can also vary significantly. We then present a consensus approach based on matrix factorization and indicate how it can be used for improving the accuracy of aggregating learnersourced decisions. In this article, we also demonstrate how utilizing information on student performance and incorporating ratings from domain experts on a limited number of learning resources can be leveraged to further improve the accuracy of the results.

Subfields: Computer Science Applications, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Human-Computer Interaction

Citations in the corpus

Comparing Code Explanations Created by Students and Large Language Models (2023, background): “While learnersourcing techniques could be used to generate code explanations efficiently by directly involving students in their creation [23, 34], there are known issues relating to quality when learning content is sourced from students [1, 9].”
Robosourcing Educational Resources - Leveraging Large Language Models for Learnersourcing (2022, background): “With respect to utilising the resources produced by other learners, one of the widely cited issues with learnersourcing is the low quality of some of the content generated by novices [12, 13, 14].”