Rating scale impact on EFL essay marking: A mixed-method study
By Khaled Barkaoui. 2007
122 citations recorded by the source corpus.
Abstract Educators often have to choose among different types of rating scales to assess second-language (L2) writing performance. There is little research, however, on how different rating scales affect rater performance. This study employed a mixed-method approach to investigate the effects of two different rating scales on EFL essay scores, rating processes, and raters’ perceptions. Four EFL teachers in Tunisia rated a set of 24 EFL essays silently and two subsets of four essays while thinking aloud using a holistic scale and then a multiple-trait rating scale. The essay scores were analyzed using G-theory while the think-aloud protocols were coded in terms of Cuming, Kantor, and Power's (Cumming, A., Kantor, R., & Powers, D. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86 (1), 67–96.) rater decision-making scheme. The holistic scale resulted in higher inter-rater agreement. Raters employed similar processes with both rating scales. Raters were the main source of variability in terms of scores and decision-making behavior. These findings have implications for writing assessment practices and for further research.
Subfields: Education, Language and Linguistics, Literature and Literary Theory
View Paper: https://www.semanticscholar.org/paper/6f35326bc4e8b2e2b72b1c85b658cce227830c22
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Education
In Language and Linguistics
In Literature and Literary Theory
Citations in the corpus
- Variability in ESL Essay Rating Processes: The Role of the Rating Scale and Rater Experience (2010, background): “I am not aware of any study that has examined the effects of different types of rating scales on L2 essay rating processes (but see Barkaoui, 2007b ).”