Reproducibility of computational workflows is automated using continuous analysis

By Brett K. Beaulieu-Jones, Casey S. Greene. 2017

157 citations recorded by the source corpus.

Replication, validation and extension of experiments are crucial for scientific progress. Computational experiments are scriptable and should be easy to reproduce. However, computational analyses are designed and run in a specific computing environment, which may be difficult or impossible to match using written instructions. We report the development of continuous analysis, a workflow that enables reproducible computational analyses. Continuous analysis combines Docker, a container technology akin to virtual machines, with continuous integration, a software development technique, to automatically rerun a computational analysis whenever updates or improvements are made to source code or data. This enables researchers to reproduce results without contacting the study authors. Continuous analysis allows reviewers, editors or readers to verify reproducibility without manually downloading and rerunning code and can provide an audit trail for analyses of data that cannot be shared.

Subfields: Computer Networks and Communications, Information Systems, Information Systems and Management, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Zhiyong Lu
2. Zhiwei Steven Wu
3. Yanjun Qi

In Information Systems

1. Brett K. Beaulieu-Jones
2. Zhiyong Lu
3. Zhiwei Steven Wu
4. Yanjun Qi
5. Jinbo Xu
6. Sergio E. Baranzini
7. Casey S. Greene
8. Yifan Peng

In Information Systems and Management

1. Brett K. Beaulieu-Jones
2. Casey S. Greene
3. Gail L. Rosen

In Evaluation & Benchmarks

1. Zhiyong Lu
2. Anne E Carpenter
3. Zhiwei Steven Wu
4. Yanjun Qi
5. Jinbo Xu
6. Yifan Peng
7. Anshul B Kundaje
8. Srinivas C. Turaga

Citations in the corpus

• Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing (2017, methodology): “In addition, continuous analysis [2] was used to re-run all analyses, to generate docker images matching the environment of the original analysis, and to track intermediate results and logs.”
• Opportunities and obstacles for deep learning in biology and medicine (2017, methodology): “Continuous Analysis [113] can allow reproducible computing on private data.”
• Systematic integration of biomedical knowledge prioritizes drugs for repurposing (2017, methodology): “Finally, we created a public Neo4j instance [212], which leverages several modern technologies such Neo4j Browser guides, cloud hosting with HTTPS, and Docker deployment [213,214].”