Evaluating digital health interventions: key questions and approaches

By Elizabeth Murray, Eric B. Hekler, Gerhard Andersson, Linda M. Collins, Aiden R. Doherty, C. Hollis, D. Rivera, Robert West, J. Wyatt. American Journal of Preventive Medicine, 2016

758 citations recorded by the source corpus.

Digital health interventions (DHI) have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety and personalisation. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of DHI. However, evaluations of DHI present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the Research Questions (RQs) needed to appraise DHIs. As DHI are at the intersection of biomedical, behavioural, computing and engineering research, methods drawn from all these disciplines are required. Relevant RQs include defining the problem and the likely benefit of the DHI, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost-effectiveness and harms. While Randomised Controlled Trials (RCTs) are important for evaluation of effectiveness and cost-effectiveness, they are best undertaken only when: a) the intervention and its delivery package are stable; b) these can be implemented with high fidelity and c) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at less cost). Broadening the portfolio of RQs and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice and research.

Subfields: Applied Psychology, General Health Professions, Public Health, Environmental and Occupational Health

View Paper: https://www.semanticscholar.org/paper/3271289e5445a65bee3a40586d77f30e73b34f21

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In General Health Professions

In Public Health, Environmental and Occupational Health

Citations in the corpus

Annual Research Review: Digital health interventions for children and young people with mental health problems – a systematic and meta‐review (2017, background): “…the type of technical platform and presentation features such as gamification; variables and decision rules used to tailor intervention strategy, content or intensity to individuals; or any other aspect of an intervention that can profitably be separated out for examination (Murray et al., 2016).”
Inequities in Health Care Services Caused by the Adoption of Digital Health Technologies: Scoping Review (2021, background): “Digital health technology interventions, that is, those delivered through digital technologies, such as smartphones and websites [15], can improve health care delivery and contribute to the triple aim of health care, that is, better care, better health outcomes, and reduction in medical spending [16”
Recommendations for policy and practice of telepsychotherapy and e-mental health in Europe and beyond. (2020, background): “…of causal models of how the intervention will work to achieve its benefits, outlining key components, and providing a multitude of evidence on efficacy, effectiveness, cost-effectiveness, clinical meaningfulness and potential harm (Kassianos, et al., 2017; Murray et al. 2016; Ollf, 2015).”
Developing and Evaluating Digital Interventions to Promote Behavior Change in Health and Health Care: Recommendations Resulting From an International Workshop (2017, background): “The workshop led to a series of publications [3-7].”
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions. (2016, background): “The novel research designs that can support these analyses are discussed in companion papers in this issue.(15,34,38) Psychophysiological measurements, ranging from skin conductance and heart rate to facial expression or fMRI, have been used to measure users’ task engagement.”
Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review (2021, background): “The term effectiveness is used in many different contexts with varying definitions [71], consisting of several components: feasibility, cost, safety, and applicability to specific contexts [72].”