Examination of the Quality of Multiple-choice Items on Classroom Tests

By D. Dibattista, Laura Kurzawa. 2011

126 citations recorded by the source corpus.

Because multiple-choice testing is so widespread in higher education, we assessed the quality of items used on classroom tests by carrying out a statistical item analysis. We examined undergraduates’ responses to 1198 multiple-choice items on sixteen classroom tests in various disciplines. The mean item discrimination coefficient was +0.25, with more than 30% of items having unsatisfactory coefficients less than +0.20. Of the 3819 distractors, 45% were flawed either because less than 5% of examinees selected them or because their selection was positively rather than negatively correlated with test scores. In three tests, more than 40% of the items had an unsatisfactory discrimination coefficient, and in six tests, more than half of the distractors were flawed. Discriminatory power suffered dramatically when the selection of one or more distractors was positively correlated with test scores, but it was only minimally affected by the presence of distractors that were selected by less than 5% of examinees. Our findings indicate that there is considerable room for improvement in the quality of many multiple-choice tests. We suggest that instructors consider improving the quality of their multiple-choice tests by conducting an item analysis and by modifying distractors that impair the discriminatory power of items. Etant donne que les examens a choix multiple sont tellement generalises dans l’enseignement superieur, nous avons effectue une analyse statistique des items utilises dans les examens en classe afin d’en evaluer la qualite. Nous avons analyse les reponses des etudiants de premier cycle a 1198 questions a choix multiples dans 16 examens effectues en classe dans diverses disciplines. Le coefficient moyen de discrimination de l’item etait +0.25. Plus de 30 % des items avaient des coefficients insatisfaisants inferieurs a + 0.20. Sur les 3819 distracteurs, 45 % etaient imparfaits parce que moins de 5 % des etudiants les ont choisis ou a cause d’une correlation negative plutot que positive avec les resultats des examens. Dans trois examens, le coefficient de discrimination de plus de 40 % des items etait insatisfaisant et dans six examens, plus de la moitie des distracteurs etait imparfaits. Le pouvoir de discrimination etait considerablement affecte en cas de correlation positive entre un distracteur ou plus et les resultatsde l’examen, mais la presence de distracteurs choisis par moins de 5 % des etudiants avait une influence minime sur ce pouvoir. Nos resultats indiquent que les examens a choix multiple peuvent etre considerablement ameliores. Nous suggerons que les enseignants procedent a une analyse des items et modifient les distracteurs qui compromettent le pouvoir de discrimination des items.

Subfields: Education, Media Technology

View Paper: https://www.semanticscholar.org/paper/6cbdd3f3138f47f156ec0e59fe0528ec88235d9e