Quality Control in Crowdsourcing Systems: Issues and Directions

By Mohammad Allahbakhsh, Boualem Benatallah, A. Ignjatović, H. M. Nezhad, Elisa Bertino, Schahram Dustdar. 2013

396 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “Several surveys [82]–[84] cover these issues in detail.”
Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning (2019, methodology): “This is also a common practice in other crowdsourcing platforms (e.g., AMT) where often acceptance rates are used in an attempt to obtain high-quality responses [1].”
Crowdsourcing, Citizen Science or Volunteered Geographic Information? The Current State of Crowdsourced Geographic Information (2016, background): “, [23,78–80], as well as the development of credibility, reputation and trust methodologies for crowdsourced geographic information such as discussed in Reference [81].”
Shading Annotations in the Wild (2017, background): “It is important to control quality when collecting crowdsourcing data [1, 15].”
Crowdsourcing Content Creation for SQL Practice (2020, background): “The crowd can be pre-selected based on their applications or self-reported expertise on the area of the task [6], but this can be time and resource consuming, taking away from the advantages crowdsourcing has over outsourcing, mainly flexibility and swift-ness.”