An Evaluation of Course Evaluations

By Philip B. Stark, Richard L. Freishtat. 2014

236 citations recorded by the source corpus.

Student ratings of teaching have been used, studied, and debated for almost a century. This article examines student ratings of teaching from a statistical perspective. The common practice of relying on averages of student teaching evaluation scores as the primary measure of teaching effectiveness for promotion and tenure decisions should be abandoned for substantive and statistical reasons: There is strong evidence that student responses to questions of “effectiveness” do not measure teaching effectiveness. Response rates and response variability matter. And comparing averages of categorical responses, even if the categories are represented by numbers, makes little sense. Student ratings of teaching are valuable when they ask the right questions, report response rates and score distributions, and are balanced by a variety of other sources and methods to evaluate teaching.

Subfields: Accounting, Education

View Paper: https://www.semanticscholar.org/paper/9b9d5bf621cb7437c196678fd4294b9086840c36

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Accounting

In Education

Citations in the corpus

Sexism, racism, prejudice, and bias: a literature review and synthesis of research surrounding student evaluations of courses and teaching (2021, background): “However, this review’s findings also demonstrate that the methodology surrounding SETs is inherently flawed because the data being input into the system is influenced by biases and prejudices that are invisible in the data’s results (marsh 2007; Osoian et al. 2010; Stark and Freishtat 2014).”
Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness (2016, background): “measure student satisfaction and grade expectations more than they measure teaching effectiveness [4,7].”
Gender biases in student evaluations of teaching (2017, methodology): “17 Alternative methods for evaluating teaching effectiveness include syllabi, course content and exam content examinations by peers, peer evaluations, evaluations by trained observers, instructor self-evaluations, evaluations from past students, and measures of student performance such as test score”
The L2 motivational self system: A meta-analysis (2018, background): “(Beleche, Fairris, & Marks, 2012, p. 709) Other potential factors shown to confound course evaluations include the teacher’s age, ethnicity, gender, and even clothes and attractiveness (for reviews, see Ottoboni, Boring, & Stark, 2016; Stark & Freishtat, 2014).”
Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform (2021, background): “…exists across institutions despite the abundance of research demonstrating ways in which student-driven evaluation can be problematic (Spooren et al., 2013; Stark & Freishtat, 2014; Wachtel, 1998), particularly for women and other marginalized groups who are fewer in number as rank increases1.”
Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance (2017, background): “Stark and Freishtat (2014) suggest that reliably and routinely measuring teaching effectiveness will never happen because it does not seem possible to effectively define it.”