Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes

By James Carifio, Rocco J. Perla. 2007

1,180 citations recorded by the source corpus.

A recent article by Jamieson in Medical Education outlined some of the (alleged) abuses of "Likert scales" with suggestions about how researchers can overcome some of the (alleged) methodological pitfalls and limitations (1) . However, many of the ideas advanced in the Jamison article, as well as a great many of articles it cited, and similar recent articles in medical, health, psychology, and educational journals and books, are themselves common misunderstandings, misconceptions, conceptual errors, persistent myths and "urban legends" about "Likert scales" and their characteristics and qualities that have been propagated and perpetuated across six decades, for a variety of different reasons. This article identifies, analyses and traces many of these aforementioned problems and presents the arguments, counter arguments and empirical evidence that show these many persistent claims and myths about "Likert scales" to be factually incorrect and untrue. Many studies have shown that Likert Scales (as opposed to single Likert response format items) produce interval data and that the F-test is very robust to violations of the interval data assumption and moderate skewing and may be used to analyze "Likert data" (even if it is ordinal), but not on an item-by-item "shotgun" basis, which is simply a current research and analysis practice that must stop. After sixty years, it is more than time to dispel these particular research myths and urban legends as well as the various damage and problems they cause, and put them to bed and out of their misery once and for all.

Subfields: Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/04170c8b29af1cf1bf6eecd936a70ca9481a6c4d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Motivating young language learners: A longitudinal model of self-determined motivation in elementary school foreign language classes. (2017, methodology): “As Likert data may be considered ordered categorizations rather than truly continuous (Carifio & Perla, 2007), we used weighted least squares estimation for its ability to model nonnormal ordered categorical data (Muthén & Muthén, 2012).”
Role of Parent Literacy and Numeracy Expectations and Activities in Predicting Early Numeracy Skills (2015, methodology): “Children enter kindergarten with a large variety in early numeracy skills, building on innate mathematical intuitions (number sense; Dehaene, 2011). Early numeracy entails a broad set of skills that has great predictive value for later mathematical achievement (Watts, Duncan, Siegler, & Davis-Kean, ”
Investigating the Long-Term Impacts of Climate Change Communications on Individuals’ Attitudes and Behavior (2014, methodology): “The comparison of means may therefore be problematic and there is controversy over whether parametric tests may be used with this type of data (e.g., Carifio & Perla, 2007; de Winter & Dodou, 2010; Knapp, 1990).”
How to analyze Likert and other rating scale data (2015, methodology): “”(18) (p390) Many of these “ordinalist” views seem to stem from overlooking the difference between individual items and overall Likert scales as pointed out by those holding the contrasting “intervalist” viewpoint.(8,10,11,23) This intervalist vs.”
Likert Data: What to Use, Parametric or Non-Parametric? (2013, neutral): “Carifio and Perla(2007) believe that the lack of understanding of the difference between Likert scales and Likert response formats is the root of the confusion.”
Equidistance of Likert-Type Scales and Validation of Inferential Methods Using Experiments and Simulations (2013, background): “Whether Likert-type scales should generally be regarded as ordinal or interval has been extensively debated among researchers over the years (e.g. Carifio and Perla, 2007; Jamieson, 2004; Michell, 1986), since the choice of statistical methodology depends on the result of this debate.”
Seeing eye to eye? The academic writing needs of graduate and undergraduate students from students’ and instructors’ perspectives (2010, background): “As also firmly stated by Carifio and Perla (2007), ‘The F-test was incredibly robust to violations of the interval data assumption (as well as moderate skewing)’ and ‘One does not have to lose statistical power and sensitivity by using non-parametric statistical tests in its place when analysing…”
Understanding and Promoting Thinking About Knowledge (2016, background): “This weight of competing evidence and interpretations about epistemic development led Chandler et al. (2002) to argue that epistemic development is recursive, that people pass through epistemic levels more than once within and across domains of epistemic cognition. At the moment, it is not clear wha”
What happens if you treat ordinal ratings as interval data? Human evaluations in NLP are even more under-powered than you think (2021, background): “…whether and when ordinal scales can be treated as interval has been fought for several decades (Glass et al., 1972; Knapp, 1990; Jamieson, 2004; Cariﬁo and Perla, 2007; Wu and Leung, 2017; Liddell and Kruschke, 2018), we argue that ordinal data needs to be analysed as ordinal in NLP: in this…”
The Academic Motivation Scale: Dimensionality, Reliability, and Construct Validity Among Vocational Students (2016, neutral): “Nevertheless, it is important to ensure that nonmetric data are used appropriately in statistical techniques (Carifio & Perla, 2007; Hair et al., 2010) and to be aware that that represents a limitation.”