Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons

By T. D. Cook, W. Shadish, Vivian C. Wong. 2008

654 citations recorded by the source corpus.

Subfields: Education, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e0f3adb4e640da445a105ce986f9535e6d24b04e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

In Statistics and Probability

Citations in the corpus

Effects of school resource officers on school crime and responses to school crime (2020, methodology): “Second, to address that limitation, the study used monthly data in an interrupted time series (ITS) design (Cook et al., 2008; Shadish et al., 2002) with a comparison series to examine whether study outcomes changed in the two- or three-month period coincidingwith the intervention.”
Another Way Out: The Impact of Juvenile Arrests on High School Dropout (2009, background): “…“warning stage” en route to juvenile court referrals and thus a systematic selection mechanism for formal arrest.10 Controlling for the number of station adjustments helps ensure parity between arrested and nonarrested youths in their adjusted risk of more formal processing (Cook et al. 2008).”
A Practical Guide to Regression Discontinuity (2012, result): “Tom Cook has recently argued that a variety of nonexperimental methods can provide causal estimates that are comparable to those obtained from experiments (Cook, Shadish, and Wong, 2008).”
Performance Funding in Higher Education: Do Financial Incentives Impact College Completions? (2014, methodology): “…a “focal” matching strategy via coarsened exact matching (cem) to identify two additional comparison groups. this matching technique aided in generating unbiased parameter estimates and it provided an opportunity to test the robustness of our findings (Cook et al., 2008; Shadish et al., 2002).”
Regression Discontinuity Designs With Multiple Rating-Score Variables (2012, background): “Because the mechanism for selection into the treatment/control condition is known and observable in a regression discontinuity design, RD can provide unbiased estimates of treatment effects under much weaker assumptions than required for other quasi-experimental methods (Cook et al., 2008).”
On making causal claims: A review and recommendations (2010, methodology): “After the randomized experiment the RDD is the design that most closely approximates a randomized experiment (Cook et al., 2008); however, it is underutilized in social-sciences research and not well understood (Shadish et al.”
Systematic Review. What is the evidence of the impact of microfinance on the well-being of poor people? (2011, background): “Cook et al. (2008) non-systematically summarised several recent social and educational studies that compared causal estimates from a high quality randomised experiment with those from high quality nonrandomised intervention or observational data.”
The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data (2019, background): “…of testing the generalizability of causal inferences drawn from observational data [Zhao et al., 2019, Keane and Wolpin, 2007] and comparing causal effects drawn from observational and experimental data [Cook et al., 2008, Eckles et al., 2016, Eckles and Bakshy, 2017, Gordon et al., 2019].”
What You Can and Can’t Properly Do with Regression (2010, background): “But strong quasi-experiments will often perform nearly as well (Cook et al., 2008).”
State ownership reinvented? Explaining performance differences between state‐owned and private firms (2018, neutral): “As matching covariates, we use all our previously discussed controls and, following the suggestion by Cook, Shadish and Wong (2008), also add pre-event performance as an additional covariate.”
Developing a Conceptual Framework for Life Skills Interventions Ψ (2013, methodology): “For example, regression discontinuity designs can be used to make causal inferences when random assignment is not practical or ethical (Lesik, 2006), while interrupted timeseries designs (Scriven, 2008), within-study comparisons (Cook et al., 2008), success case method designs with a time-series ele”
Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: challenging folklore in evaluation research in crime and justice (2010, result): “These studies suggest that especially when techniques such as regression discontinuity designs or propensity score matching are used the results gained are similar to those from randomized controlled trials (e.g., see Cook et al. 2008; Shadish et al. 2008).”