The use of scoring rubrics: Reliability, validity, and educational consequences

By A. Jonsson, Gunilla Svingby. 2007

1,377 citations recorded by the source corpus.

Subfields: Education, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/1d6ddb6e4951eb46e1af75aa9cfab807231d7808

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

In Management Science and Operations Research

Citations in the corpus

Reliability and validity of rubrics for assessment through writing (2010, neutral): “The intra-rater reliability (consistency of grading a given writing by the same rater twice) is reported to be higher (Cronbach’s alpha about .70) than the inter-rater reliability when a rubric is used (Jonsson & Svingby, 2007).”
Rubrics to assess critical thinking and information processing in undergraduate STEM courses (2020, methodology): “Jonsson and Svingby (2007) found that most of the rubrics they reviewed had adjacent agreement scores of 90% or greater.”
Experimental Evidence on Teachers’ Racial Bias in Student Evaluation: The Role of Grading Scales (2020, result): “However, this would require that the vague grade-level measure has higher reliability than the more clearly defined rubric, which contradicts past research (e.g., Jonsson & Svingby, 2007) and intuition.”
The relation between content-specific and general teacher knowledge and skills (2016, neutral): “00; Jonsson & Svingby, 2007).”
Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students (2017, methodology): “…only (e.g. scoring the activity) the aim is no longer the students’ learning, but there can still be an additional positive effect by enhancing the inter-rater and the rater (when only one teacher is scoring) reliability which results in more solid educational evaluation (Jonsson & Svingby, 2007).”
Scaffolding Self-Regulated Learning Through Self-Assessment and Peer Assessment: Guidelines for Classroom Implementation (2016, background): “A particular branch of AfL research emphasizing the need for clear criteria is research on scoring rubrics used by students (Jonsson and Svingby 2007; Panadero and Jonsson 2013).”
Rubric‐referenced self‐assessment and middle school students’ writing (2010, background): “Rubrics have become popular with teachers as a means of communicating expectations for an assignment, providing focused feedback on works in progress, and grading final products (Andrade 2000 ; Jonsson and Svingby 2007 ; Moskal 2003 ; Popham 1997 ).”
Rubrics and Self-Assessment Scripts Effects on Self-Regulation, Learning and Self-Efficacy in Secondary Education. (2012, background): “Similar results have been found in previous research on script effects (Alonso-Tapia & Panadero, 2010; Bannert, 2009; Kostons et al., 2009; Kramarski & Michalsky, 2009, 2010; Montague, 2007) and mixed results on rubrics (Andrade et al., 2009; Jonsson & Svingby, 2007; Schafer et al., 2001).”
Motivations for choosing teaching as a career: effects on general pedagogical knowledge during initial teacher education (2012, methodology): “Two raters coded the answers independently of one another; as a measure of consensus and internal consistency Cohen's Kappa was estimated (Jonsson & Svingby, 2007 ), ranging from .”
A review of rubric use in higher education (2010, background): “As with any form of assessment, the clarity of the language in a rubric is a matter of validity because an ambiguous rubric cannot be accurately or consistently interpreted by instructors, students or scorers (Payne 2003).”
The Use of Scoring Rubrics for Formative Assessment Purposes Revisited: A Review. (2013, background): “In one review of research on the use of rubrics, Jonsson and Svingby (2007) noted that it was not possible to draw any conclusions about student improvement since the results were mixed.”
Rubrics vs. self-assessment scripts effect on self-regulation, performance and self-efficacy in pre-service teachers (2013, methodology): “These two tools have proved to have some positive effects on self-regulation and learning (Alonso-Tapia & Panadero, 2010; Bannert, 2009; Jonsson & Svingby, 2007).”
Assessment rubrics: towards clearer and more replicable design, research and practice (2017, methodology): “Jonsson and Svingby’s (2007) review of 75 studies identifies two classification dimensions: whether a rubric is analytic or holistic, and if it is generic or task-specific.”