Peer and Teacher Assessments of Oral Presentation Skills: How reliable are they?

By D. Magin, P. Helmore. 2001

174 citations recorded by the source corpus.

This article reports findings on the reliabilities of peer and teacher summative assessments of engineering students' oral presentation skills in a fourth year communications subject. The context of the study is unusual, in that each oral presentation was subject to multiple ratings by teams of students and teams of academic staff. Analysis of variance procedures were used to provide separate estimates of inter-rater reliability of assessments by peers and teachers for classes in four succeeding years. Teacher ratings were found to have substantially higher levels of inter-rater agreement than peer ratings. Generalising over the four years, it would require the averaging of between two and four peer ratings to match the reliability of single teacher assessments. However, the estimates of individual rater reliability for teachers, based on the intra-class correlation coefficient, were moderately low (0.40 to 0.53). It is concluded that the reliability of summative assessments of oral presentations can be improved by combining teacher marks with the averaged marks obtained from multiple peer ratings.

Subfields: Education, Media Technology

View Paper: https://www.semanticscholar.org/paper/39ba6c47b1c1f1064824cd38300a9dc9888b1c19

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

In Media Technology

Citations in the corpus

‘Only true friends could be cruelly honest’: cognitive scaffolding and social-affective support in teacher feedback literacy (2017, background): “Although most of the scholarly writing on this topic have focused on reliability and fairness (e.g. Cheng and Warren 1999; Magin and Helmore 2001; Suñol et al. 2016), the real reasons for the deviation of grades may lie in a lack of assessment experience and observation skills (Suñol et al., 2016).”
Peer and self‐assessment in the first year of university (2011, background): “(3) Correlations between student assessments and ratings given by academic staff (Cassidy 2007; Falchikov and Goldfinch 2000; Gruppen et al. 1997; Hafner and Hafner 2003; Lejk and Wyvill 2001a, 2001b; Magin 1993, 2001a, 2001b; Magin and Churches 1988; Magin and Helmore 2001).”
Goals of peer assessment and their associated quality concepts (2011, background): “Falchikov and Goldfinch 2000 ; Magin and Helmore 2001 ), as well as to peers providing a qualitative appreciation of the behaviour of a peer (e.”
How effective are self- and peer assessment of oral presentation skills compared with teachers’ assessments? (2012, background): “Analysis of the assessment of oral presentation skills mainly results in an overview of studies about self- and peer assessment of individual (oral) presentation skills (AlFallay, 2004; Campbell et al., 2001; Cheng and Warren, 2005; Hafner and Hafner, 2003; Hughes and Large, 1993; Langan et al., 200”
Can Charisma Be Taught? Tests of Two Interventions (2011, background): “…communicated in the introduction); framework (beginning and end well-connected); logic of the speech; simple and easy-to-understand language; nonlexical utterances; clear pronunciation; correct English; nervousness; and tempo of speech (see Frese et al., 2003; Magin & Helmore, 2001; Towler, 2003).”
Peer assessment of oral presentations: effects of student gender, university affiliation and participation in the development of assessment criteria (2005, background): “There are bene®ts in assessing the reliability of such schemes by examining how well student marks correspond to tutor grades in parallel with a consideration of variation of tutor marks (see Magin & Helmore, 2001).”
An inventory of peer assessment diversity (2011, background): “Magin and Helmore ( 2001 ) and Robinson ( 2002 ) studied the effect of additional peer assessors on the reliability of an assessment, and on its perceived reliability.”