Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness

By Anne Boring, Kellie Ottoboni, Philip B. Stark. 2016

450 citations recorded by the source corpus.

Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. We show: SET are biased against female instructors by an amount that is large and statistically significant the bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded the bias varies by discipline and by student gender, among other things it is not possible to adjust for the bias, because it depends on so many factors SET are more sensitive to students' gender bias and grade expectations than they are to teaching effectiveness gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors. These findings are based on nonparametric statistical tests applied to two datasets: 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sections of an online course in a randomized, controlled, blind experiment at a US university.

Subfields: Education, Social Psychology

View Paper: https://www.semanticscholar.org/paper/f56dd58b0938d601e3e28f065e05dc3793765ee0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

In Social Psychology

Citations in the corpus

Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform (2021, background): “For example, men are more likely to be seen as adept, brilliant, or organized instructors across a range of institutional settings and research methodologies (Arbuckle & Williams, 2003; Abel & Meltzer, 2007; Boring et al., 2016; MacNell et al., 2015; McPherson et al., 2009; Mengel et al., 2018; Ridg”
Teaching Computational Reproducibility for Neuroimaging (2018, background): “Student ratings have little or no relationship to the success of the course in conveying the material (Boring et al., 2016; Uttl et al., 2017), but they do appear to measure a variety of unfortunate and irrelevant factors such as the gender and race of the teacher (Boring et al., 2016), the subject…”
Gender, ethnicity and teaching evaluations: Evidence from mixed teaching teams (2016, background): “While more and more women and teachers of different ethnic backgrounds enter academia, white male professors are still the norm and tend to achieve better teaching evaluations (Boring et al., 2016; Basow and Silberg, 1987).”
Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis (2020, methodology): “But they are not (e.g., Boring et al., 2016; Uttl et al., 2017).”
The L2 motivational self system: A meta-analysis (2018, background): “The presence of all of these biases has led some researchers to cast serious doubt on the value of course evaluation, with some considering any attempt to statistically adjust for the many biases involved to be practically “impossible” (Ottoboni et al., 2016, p. 10).”
Gender biases in student evaluations of teaching (2017, methodology): “13 Other studies find that SETs are unrelated to teaching effectiveness (Boring et al., 2016).”
Does the Flipped Classroom Improve Student Learning and Satisfaction? A Systematic Review and Meta-Analysis (2019, background): “This is an important result because it illustrates a point made by several commentators, that student satisfaction is related to a number of factors that may be irrelevant to student learning (Boring et al., 2016; Spooren, Brockx, & Mortelmans, 2013; Uttl et al., 2017).”
Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance (2017, methodology): “Yet in the almost 20 years that have passed since his paper was published, SETs remain the main tool utilized in the assessment of instructor teaching competence, promotion and tenure (Boring et al., 2016; Kelly, 2012; Spooren et al., 2013) in both the United States and Canada notwithstanding the no”
Scaling Down Inequality: Rating Scales, Gender Bias, and the Architecture of Evaluation (2019, background): “…that teaching evaluations also contribute to gender inequalities within the profession (Abel and Meltzer 2007; Arbuckle and Williams 2003; Baldwin and Blattner 2003; Boring et al. 2016; MacNell et al. 2015; McPherson et al. 2009; Mengel et al. 2017; Sidanius and Crane 1989; Wagner et al. 2016).”