Methods for the detection of carelessly invalid responses in survey data

By P. Curran. 2016

1,152 citations recorded by the source corpus.

Self-report data collections, particularly through online measures, are ubiquitous in both experimental and non- experimental psychology. Invalid data can be present in such data collections for a number of reasons. One reason is careless or insufficient effort (C/IE) responding. The past decade has seen a rise in research on techniques to detect and remove these data before normal analysis (Huang, Curran, Keeney, Poposki, & DeShon, 2012; Johnson, 2005; Meade & Craig, 2012). The rigorous use of these techniques is valuable tool for the removal of error that can impact survey results (Huang, Liu, & Bowling, 2015). This research has encompassed a number of sub-fields of psychology, and this paper aims to integrate different perspectives into a review and assessment of current tech- niques, an introduction of new techniques, and a generation of recommendations for practical use. Concerns about C/IE responding are a factor any time self-report data are collected, and all such researchers should be well-versed on methods to detect this pattern of response.

Subfields: Applied Psychology, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/deb576459476939eacb53937bacc9f42def17341

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Sociology and Political Science

Citations in the corpus

The quality of data collected online: An investigation of careless responding in a crowdsourced sample (2020, methodology): “The other attention check item was an Instructed Response Item (IRI) similar to Meade and Craig (2012) and Curran (2016).”
Are Attention Check Questions a Threat to Scale Validity? (2018, background): “Despite their benefits, attention checks have limitations (e.g. Curran, 2016; Curran & Hauser, 2015).”
Applying Social Psychology to Prevent Careless Responding during Online Surveys (2018, methodology): “See Curran (2016) for a review of the CR indicators used in the current paper.”
Careless responding in questionnaire measures: Detection, impact, and remedies (2020, methodology): “…because these measures can be faked (DeSimone & Harms, 2018; Edwards, 2019), and second, because the wording of these items can have an influence on the participants' responses (Edwards, 2019) (for a detailed discussion on the potential caveats of direct measures, see also Curran, 2016, pp. 13–15).”
Evaluating the Impact of Careless Responding on Aggregated-Scores: To Filter Unmotivated Examinees or Not? (2017, result): “…incompetence, and LongString responding [providing the same response consecutively across a number of items]) and/or surveys and questionnaires; however, there has been extensive previous work conducted in these areas (e.g., Curran, 2015; Huang, Liu, & Bowling, 2015; Meade & Craig, 2011).”
Love in the Time of COVID: Perceived Partner Responsiveness Buffers People From Lower Relationship Quality Associated With COVID-Related Stressors (2020, methodology): “As commonly employed in the literature (e.g., Berinsky et al., 2014; Curran, 2016), this study included attention check questions, wherein we asked participants to select a particular answer choice for that question (e.”
Paying Attention to Inattentive Survey Respondents (2019, background): “…general inattention detection, but IMCs are particularly useful for survey experiments where “the manipulation of a study is hidden in similar text somewhere else in that study” (Curran 2016, p. 14) and instructed-response items are particularly useful for grid and check-all-that-apply questions.”
No Harm in Checking: Using Factual Manipulation Checks to Assess Attentiveness in Experiments (2018, background): “, along with a variety of closed-ended response options (Oppenheimer, Meyvis, and Davidenko 2009; also see Curran 2016; Lovett et al. 2018; Maniaci and Rogge 2014).”
The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers (2020, background): “One attention check was based on the person-total correlation that captures the person internal consistency relative to the expected patterns generated by all other participants (Curran, 2016).”
Dirty Data: The Effects of Screening Respondents Who Provide Low-Quality Data in Survey Research (2017, methodology): “The suitability of screening techniques depends on survey design and methodology (Curran, 2016; DeSimone et al., 2015; Dunn et al., in press).”
Too Fast, too Straight, too Weird: Non-Reactive Indicators for Meaningless Data in Internet Surveys (2019, background): “Negative effects of such items have been discussed (Curran, 2016; Goldsmith, 1989), yet Breitsohl and Steidelmüller (2018) present empirical evidence that these bogus items have little effect on response behavior.”
Will the Questions Ever End? Person-Level Increases in Careless Responding During Questionnaire Completion (2020, methodology): “We assessed careless responding using three methods: the (a) infrequency, (b) response time, and (c) longstring indices (for detailed descriptions of these indices, see Bowling et al., 2016; Curran, 2016; DeSimone et al., 2015).”