Recommendations and practices for open and reproducible research

As a reviewer or researcher interested in the work of others, if you were to reproduce a study, you would

Very cool; But If you had to write a reproducible paper… How do you start?

We have heard many excuses to avoid taking reproducible practices seriously in one’s research, such as “I have never heard of reproducibility before”, “I’m not good at using tools; Did you say github and docker?”, “I have never been trained”, “I don’t know where to find tutorials, guidelines…”, “I’m superbusy! I do not have enough time”, and “I do theoretical stuff. Reproduction is not for me”

No matter your excuse; If you do not want to fool yourself, reproducible research practices are essential to enable not only reproducible but transparent, open and honest science. Indeed, conducting reproducible research is neither overly difficult nor does it require encyclopedic knowledge of multiple research tools, standards, and protocols. The good news is that most researchers already do much of the work necessary to make research reproducible. The point here is then to identify those aspects that could improve the level of reproducibility of your research and adapt them to your workflow and research habits.

You should ask yourself:

Key guidelines

There are excellent practical guides with general recommendations for promoting reproducibility, research data management and open science. As an author, who wants to integrate reproducibility research practices into her next research paper, the following resources are an excellent starting point:

  • The British Ecological Society publishes brief guidelines for conducting open science on ecology but applicable to any discipline. Among them, the guide Reproducible code (Cooper and Hsing 2019) explains organizational and managerial aspects for making software code more reproducible.

  • Passport for Open Science: A Practical Guide for PhD Students (Berti et al. 2022) explains how the principles of open science can be applied to doctoral research. The proposed practices are applicable to any discipline and, although focused on doctoral students, they are aimed at any researcher regardless of their previous experience.

  • A Beginner’s Guide to Conducting Reproducible Research (Alston and Rick 2021) is a very concise, practical guide to apply reproducible practices to a research project.

  • The Turing Way is indeed a compendium of guides on open science, reproducibility, and research ethics, among other interesting topics. Have a look at the Guide for Reproducible Research (The Turing Way Community 2022).

  • Reproducible Publications at AGILE Conferences: Guidelines for Authors, Scientific Reviewers, and Reproducibility Reviewers (Nüst et al. 2020) describes specific aspects to write conference papers in the field of GIScience.

Before, during and after

flowchart TB
  A[Before data analysis] --> B[During data analysis]
  B --> C[After data analysis]

(Alston and Rick 2021) outlined basic steps toward making research more reproducible base on three stages of a research project: (1) before data analysis, (2) during analysis, and (3) after analysis.

We borrow this simple outline to better categorise and explain the set of practical tips and recommendations that follow. Note that most of the following recommendations are already widely accepted best practices for scientific research and that striving for a reasonable level of reproducibility is more achievable than you may expect. Essentially, researchers must think carefully before, during, and after analysis to ensure the reproducibility, openness and transparency of their work.

References

Alston, Jesse M., and Jessica A. Rick. 2021. “A Beginner’s Guide to Conducting Reproducible Research.” The Bulletin of the Ecological Society of America 102 (2): e01801. https://doi.org/10.1002/bes2.1801.
Berti, Johann, Marin Dacos, Gabriel Gallezot, Madeleine Géroudet, Sabrina Granger, Joanna Janik, Claire Josserand, et al. 2022. “Passport for Open Science - a Practical Guide Ofr PhD Students.” University of Lille. https://www.ouvrirlascience.fr/passport-for-open-science-a-practical-guide-for-phd-students/.
Cooper, Natalie, and Pen-Yuah Hsing. 2019. “Reproducible Code.” British Ecological Society. https://www.britishecologicalsociety.org/publications/guides-to/.
Keshav, S. 2007. “How to Read a Paper.” ACM SIGCOMM Computer Communication Review 37 (3): 83–84. https://doi.org/10.1145/1273445.1273458.
Nüst, Daniel, Carl Boettiger, and Ben Marwick. 2018. “How to Read a Research Compendium.” arXiv. https://doi.org/10.48550/ARXIV.1806.09525.
Nüst, Daniel, Frank Ostermann, Rusne Sileryte, Barbara Hofer, Carlos Granell, Marta Teperek, Anita Graser, et al. 2020. “Reproducible Publications at AGILE Conferences: Guidelines for Authors, Scientific Reviewers, and Reproducibility Reviewers.” https://doi.org/10.17605/OSF.IO/CB7Z8.
The Turing Way Community. 2022. “The Turing Way: A Handbook for Reproducible, Ethical and Collaborative Research.” Zenodo. https://doi.org/10.5281/zenodo.3233853.