References

This page lists resources that this course draws on frequently or at least occasionally.

All materials described below are (should be) freely available online. If you cannot get to them, let me know. Note that a lot of the listed resources are dynamic and ever changing. That means occasionally links might not work, sites go down. If you notice anything that’s not quite right, please let me know.

Aden-Buie, Garrick, Barret Schloerke, JJ Allaire, and Alexander Rossell Hayes. 2023. Learnr: Interactive Tutorials for r. https://rstudio.github.io/learnr/.
Alsheikh-Ali, Alawi A., Waqas Qureshi, Mouaz H. Al-Mallah, and John P. A. Ioannidis. 2011. “Public Availability of Published Research Data in High-Impact Journals.” PLOS ONE 6 (9): 1–4. https://doi.org/10.1371/journal.pone.0024357.
Alston, Jesse M., and Jessica A. Rick. 2021. “A Beginner’s Guide to Conducting Reproducible Research.” The Bulletin of the Ecological Society of America 102 (2): e01801. https://doi.org/https://doi.org/10.1002/bes2.1801.
Baker, M. 2015. “First Results from Psychology’s Largest Reproducibility Test.” Nature News. https://doi.org/10.1038/nature.2015.17433.
———. 2016. “1,500 Scientists Lift the Lid on Reproducibility.” Nature 533 (7604): 452–54. https://doi.org/10.1038/533452a.
———. 2017. “Check Your Chemistry.” Nature 548 (7668): 485–88. https://doi.org/10.1038/548485a.
Barba, L. 2018. “Terminologies for Reproducible Research.” arXiv Preprint arXiv:1802.03311. https://arxiv.org/abs/1802.03311.
Barba, Lorena A. 2021. “Trustworthy Computational Evidence Through Transparency and Reproducibility.” Computing in Science & Engineering 23 (1): 58–64. https://doi.org/10.1109/MCSE.2020.3048406.
Barker, Michelle, Neil P. Chue Hong, Daniel S. Katz, Anna-Lena Lamprecht, Carlos Martinez-Ortiz, Fotis Psomopoulos, Jennifer Harrow, et al. 2022. “Introducing the FAIR Principles for Research Software.” Scientific Data 9 (1). https://doi.org/10.1038/s41597-022-01710-x.
Berti, Johann, Marin Dacos, Gabriel Gallezot, Madeleine Géroudet, Sabrina Granger, Joanna Janik, Claire Josserand, et al. 2022. “Passport for Open Science - a Practical Guide Ofr PhD Students.” University of Lille. https://www.ouvrirlascience.fr/passport-for-open-science-a-practical-guide-for-phd-students/.
Biggs, John, and Catherine Tang. 2011. Teaching for Quality Learning at University, 4th Edition. Open University Press.
Bion, R, R Chang, and J Goodman. 2018. “How r Helps Airbnb Make the Most of Its Data.” The American Statistician 72 (1): 46–52. https://doi.org/10.1080/00031305.2017.1392362.
Bryan, Jennifer. 2018. “Excuse Me, Do You Have a Moment to Talk about Version Control?” The American Statistician 72 (1): 20–27. https://doi.org/10.1080/00031305.2017.1399928.
Bryan, Jenny, and Jim Hester. 2020. Happy Git and GitHub for the useR. https://happygitwithr.com/.
Button, KS, JPA Ioannidis, C Mokrysz, BA Nosek, J Flint, ESJ Robinson, and MR Munafò. 2013. “Power Failure: Why Small Sample Size Undermines the Reliability of Neuroscience.” Nature Reviews Neuroscience 14 (5): 365–76. https://doi.org/10.1038/nrn3475.
Callaghan, S. 2014. “Joint Declaration of Data Citation Principles.” https://www.force11.org/datacitationprinciples.
Chiarelli, Andrea, Lucia Loffreda, and Rob Johnson. 2021. “The Art of Publishing Reproducible Research Outputs: Supporting emerging practices through cultural and technological innovation.” Zenodo. https://doi.org/10.5281/zenodo.5521077.
Claerbout, JF, and M Karrenbach. 1992. “Electronic Documents Give Reproducible Research a New Meaning.” In SEG Technical Program Expanded Abstracts 1992, 601–4. Society of Exploration Geophysicists. https://doi.org/10.1190/1.1822162.
Cooper, Natalie, and Pen-Yuah Hsing. 2019. “Reproducible Code.” British Ecological Society. https://www.britishecologicalsociety.org/publications/guides-to/.
Delgado-López-Cózar, E, I Ràfols, and E Abadal. 2021. “Letter: A Call for a Radical Change in Research Evaluation in Spain.” Profesional de La Información 30 (3): e300309. https://doi.org/10.3145/epi.2021.may.09.
Dhar, Payal. 2023. “Octopus and ResearchEquals Aim to Break the Publishing Mould.” Nature. Springer Science; Business Media LLC. https://doi.org/10.1038/d41586-023-00861-0.
Donoho, DL, A Maleki, IU Rahman, M Shahram, and V Stodden. 2009. “Reproducible Research in Computational Harmonic Analysis.” Computing in Science & Engineering 11 (1): 8–18. https://doi.org/10.1109/MCSE.2009.15.
Editorial. 2021. “Good Research Begins Long Before Papers Get Written.” Nature 593 (7857): 8–8. https://doi.org/10.1038/d41586-021-01167-9.
Fanelli, D. 2018. “Opinion: Is Science Really Facing a Reproducibility Crisis, and Do We Need It To?” Proceedings of the National Academy of Sciences 115 (11): 2628–31. https://doi.org/10.1073/pnas.1708272114.
Granell, C. 2019. “Directrices Para Articulos Reproducibles.” https://doi.org/10.17605/OSF.IO/MF9BE.
Granell, C, B Hofer, Daniel Nüst, FO Ostermann, and R Sileryte. 2020. “Reproducibilidad En AGILE: Experiencias, Logros y Recomendaciones.” Revista Cartográfica, no. 100: 155–72. https://doi.org/10.35424/rcarto.i100.668.
Granell, C, Daniel Nüst, FO Ostermann, and R Sileryte. 2018. “Reproducible Research Is Like Riding a Bike.” PeerJ Preprints 6: e27216v1. https://doi.org/10.7287/peerj.preprints.27216v1.
Harrison, Kate. 2018. “Data Management.” British Ecological Society. https://www.britishecologicalsociety.org/publications/guides-to/.
Hasselbring, Wilhelm, Leslie Carr, Simon Hettrick, Heather Packer, and Thanassis Tiropanis. 2020. “Open Source Research Software.” Computer 53 (8): 84–88. https://doi.org/10.1109/MC.2020.2998235.
Hong, NPC, T Crick, IP Gent, L Kotthoff, and K Takeda. 2015. “Top Tips to Make Your Research Irreproducible.” arXiv Preprint arXiv:1504.00062. https://arxiv.org/abs/1504.00062.
Ioannidis, JP. 2005. “Why Most Published Research Findings Are False.” PLOS Medicine 2 (8): e124. https://doi.org/10.1371/journal.pmed.0020124.
Ioannidis, JP, TD Stanley, and H Doucouliagos. 2017. “The Power of Bias in Economics Research.” The Economic Journal 127 (605): F236–65. https://doi.org/10.1111/ecoj.12461.
Jolly, M, AC Fletcher AC, and PE Bourne. 2012. “Ten Simple Rules to Protect Your Intellectual Property.” PLoS Computacional Biology 8: e1002766. https://doi.org/10.1371/journal.pcbi.1002766.
Keshav, S. 2007. “How to Read a Paper.” ACM SIGCOMM Computer Communication Review 37 (3): 83–84. https://doi.org/10.1145/1273445.1273458.
Knuth, DE. 1984. “Literate Programming.” The Computer Journal 11 (2): 97–111. https://doi.org/10.1093/comjnl/27.2.97.
Leek, JT, and LR Jager. 2017. “Is Most Published Research Really False?” Annual Review of Statistics and Its Application 4: 109–22. https://doi.org/10.1146/annurev-statistics-060116-054104.
Leipzig, Jeremy, Daniel Nüst, Charles Tapley Hoyt, Karthik Ram, and Jane Greenberg. 2021. “The Role of Metadata in Reproducible Computational Research.” Patterns 2 (9): 100322. https://doi.org/https://doi.org/10.1016/j.patter.2021.100322.
Macleod, Malcolm, Andrew M. Collings, Chris Graf, Veronique Kiermer, David Mellor, Sowmya Swaminathan, Deborah Sweet, and Valda Vinson. 2021. “The MDAR (Materials Design Analysis Reporting) Framework for Transparent Reporting in the Life Sciences.” Proceedings of the National Academy of Sciences 118 (17): e2103238118. https://doi.org/10.1073/pnas.2103238118.
Markowetz, F. 2015. “Five Selfish Reasons to Work Reproducibly.” Genome Biology 16: 274. https://doi.org/10.1186/s13059-015-0850-7.
Mesnard, Olivier, and Lorena A. Barba. 2017. “Reproducible and Replicable Computational Fluid Dynamics: It’s Harder Than You Think.” Computing in Science Engineering 19 (4): 44–55. https://doi.org/10.1109/MCSE.2017.3151254.
Miralles, Ignacio, Carlos Granell, Laura Díaz-Sanahuja, William Van Woensel, Juana Bretón-López, Adriana Mira, Diana Castilla, and Sven Casteleyn. 2020. “Smartphone Apps for the Treatment of Mental Disorders: Systematic Review.” JMIR Mhealth Uhealth 8 (4): e14897. https://doi.org/10.2196/14897.
Morin, A, J Urban, and P Sliz. 2012. “A Quick Guide to Software Licensing for the Scientist-Programmer.” PLoS Computacional Biology 8 (7): e1002598. https://doi.org/10.1371/journal.pcbi.1002598.
MR, Munafò, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie du Sert N, Simonsohn U, Wagenmakers E-J, Ware JJ, and Ioannidis JPA. 2017. “A Quick Guide to Software Licensing for the Scientist-Programmer.” Nature Human Behaviour 1 (0021). https://doi.org/10.1038/s41562-016-0021.
National Academies of Sciences, Engineering, and Medicine. 2019. Reproducibility and Replicability in Science. The National Academies Press. https://doi.org/10.17226/25303.
Nosek, Brian A., and Timothy M. Errington. 2020. “What is replication?” PLOS Biology 18 (3): e3000691. https://doi.org/10.1371/journal.pbio.3000691.
Noy, Natasha, and Carole Goble. 2023. “Are We Cobblers without Shoes?” Communications of the ACM 66 (1): 36–38. https://doi.org/10.1145/3528574.
Noy, Natasha, and Aleksandr Noy. 2019. “Let Go of Your Data.” Nature Materials 19 (1): 128–28. https://doi.org/10.1038/s41563-019-0539-5.
Nust, Daniel, Carl Boettiger, and Ben Marwick. 2018. “How to Read a Research Compendium.” arXiv. https://doi.org/10.48550/ARXIV.1806.09525.
Nust, Daniel, C Granell, B Hofer, M Konkol, FO Ostermann, R Sileryte, and V Cerutti. 2018. “Reproducible Research and GIScience: An Evaluation Using AGILE Conference Papers.” PeerJ 6: e5072. https://doi.org/10.7717/peerj.5072.
Nust, Daniel, FO Ostermann, R Sileryte, B Hofer, C Granell, M Teperek, A Graser, K Broman, and K Hettne. 2018. “AGILE Reproducible Paper Guidelines.” https://doi.org/10.17605/OSF.IO/CB7Z8.
Nust, Daniel, Frank Ostermann, Rusne Sileryte, Barbara Hofer, Carlos Granell, Marta Teperek, Anita Graser, et al. 2020. “Reproducible Publications at AGILE Conferences: Guidelines for Authors, Scientific Reviewers, and Reproducibility Reviewers.” https://doi.org/10.17605/OSF.IO/CB7Z8.
Nust, Daniel, Vanessa Sochat, Ben Marwick, Stephen J. Eglen, Tim Head, Tony Hirst, and Benjamin D. Evans. 2020. “Ten Simple Rules for Writing Dockerfiles for Reproducible Data Science.” PLOS Computational Biology 16 (11): 1–24. https://doi.org/10.1371/journal.pcbi.1008316.
Nüst, Daniel, and Stephen J Eglen. 2021. “CODECHECK: an Open Science initiative for the independent execution of computations underlying research articles during peer review to improve reproducibility.” F1000Research 10 (March): 253. https://doi.org/10.12688/f1000research.51738.1.
Ostermann, FO, and C Granell. 2017. “Advancing Science with VGI: Reproducibility and Replicability of Recent Studies Using VGI.” Transactions in GIS 21 (2): 224–37. https://doi.org/10.1111/tgis.12195.
Ostermann, Frank O., Daniel Nüst, Carlos Granell, Barbara Hofer, and Markus Konkol. 2021. “Reproducible Research and GIScience: An Evaluation Using GIScience Conference Papers.” In 11th International Conference on Geographic Information Science (GIScience 2021) - Part II, edited by Krzysztof Janowicz and Judith A. Verstegen, 208:2:1–16. Leibniz International Proceedings in Informatics (LIPIcs). Dagstuhl, Germany: Schloss Dagstuhl – Leibniz-Zentrum für Informatik. https://doi.org/10.4230/LIPIcs.GIScience.2021.II.2.
Parsons, MA, RE Duerr, and MB Jones. 2019. “The History and Future of Data Citation in Practice.” Data Science Journal 18 (1). https://doi.org/10.5334/dsj-2019-052.
Peng, RD. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–27. https://doi.org/10.1126/science.1213847.
Perez-Riverol, Y, L Gatto, R Wang, T Sachsenberg, J Uszkoreit, F da Veiga Leprevost, C Fufezan, et al. 2016. “Ten Simple Rules for Taking Advantage of Git and GitHub.” PLoS Computational Biology 12 (7). https://doi.org/10.1371/journal.pcbi.1004947.
Pimentel, João Felipe, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. “A Large-Scale Study about Quality and Reproducibility of Jupyter Notebooks.” In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), 507–17. https://doi.org/10.1109/MSR.2019.00077.
Rule, A, A Birmingham, C Zuniga, I Altintas, SC Huang, R Knight, N Moshiri, et al. 2019. “Ten Simple Rules for Reproducible Research in Jupyter Notebooks.” PLoS Computational Biology 15 (7): e1007007. https://doi.org/10.1371/journal.pcbi.1004947.
Sandve, GK, A Nekrutenko, J Taylor, and E Hovig. 2017. “Ten Simple Rules for Reproducible Computational Research.” PLoS Computacional Biology 9 (10): e1003285. https://doi.org/10.1371/journal.pcbi.1003285.
Smith, AM, DS Katz, and KE Niemeyer. 2016. “Software Citation Principles.” PeerJ Computer Science 2: e86. https://doi.org/10.7717/peerj-cs.86.
Stark, PB. 2018. “Before Reproducibility Must Come Preproducibility.” Nature 557 (7706): 613–14. https://doi.org/10.1038/d41586-018-05256-0.
Stodden, Victoria, Jennifer Seiler, and Zhaokun Ma. 2018. “An Empirical Analysis of Journal Policy Effectiveness for Computational Reproducibility.” Proceedings of the National Academy of Sciences 115 (11): 2584–89. https://doi.org/10.1073/pnas.1708290115.
Stodden, V, and SB Miguez. 2014. “Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research.” Journal of Open Research Software 2 (1): e21. https://doi.org/10.5334/jors.ay.
Tenopir, C, S Allard, K Douglass, AU Aydinoglu, L Wu, E Read, M Manoff, and M Frame. 2011. “Data Sharing by Scientists: Practices and Perceptions.” PLoS ONE 6 (6): e21101. https://doi.org/10.1371/journal.pone.0021101.
Tenopir, C, ED Dalton, S Allard, M Frame, I Pjesivac, B Birch, D Pollock, and K Dorsett. 2015. “Changes in Data Sharing and Data Reuse Practices and Perceptions Among Scientists Worldwide.” PLoS ONE 10 (8): e0134826. https://doi.org/10.1371/journal.pone.0134826.
Tenopir, C, NM Rice, S Allard, L Baird, J Borycz, L Christian, B Grant, R Olendorf, and RJ Sandusky. 2020. “Data Sharing, Management, Use, and Reuse: Practices and Perceptions of Scientists Worldwide.” PLoS ONE 15 (3): e0229003. https://doi.org/10.1371/journal.pone.0229003.
The Turing Way Community. 2019. “The Turing Way: A Handbook for Reproducible Data Science.” Zenodo. https://doi.org/10.5281/zenodo.3233986.
Tso, Chak Hau Michael, Michael Hollaway, Rebecca Killick, Peter Henrys, Don Monteith, John Watkins, and Gordon Blair. 2022. “The r Journal: Advancing Reproducible Research by Publishing r Markdown Notebooks as Interactive Sandboxes Using the Learnr Package.” The R Journal 14: 255–63. https://doi.org/10.32614/RJ-2022-021.
Vasilevsky, NA, J Minnier, MA Haendel, and RE Champieux. 2017. “Reproducible and Reusable Research: Are Journal Data Sharing Policies Meeting the Mark?” PeerJ 5: e3208. https://doi.org/10.7717/peerj.3208.
Wilkinson, Mark D., Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, et al. 2016. “The FAIR Guiding Principles for Scientific Data Management and Stewardship.” Scientific Data 3 (1). https://doi.org/10.1038/sdata.2016.18.
Wilson, C. 2021. “Impact Factor Abandoned by Dutch University in Hiring and Promotion Decisions.” Nature 595 (465). https://doi.org/10.1038/d41586-021-01759-5.
Wilson, G, J Bryan, K Cranston, J Kitzes, L Nederbragt, and TK Teal. 2017. “Good Enough Practices in Scientific Computing.” PLoS Computacional Biology 13 (6): e1005510. https://doi.org/10.1371/journal.pcbi.1005510.
Wilson, Greg. 2019. Teaching Tech Together: How to Make Lessons That Work and Build a Teaching Community Around It. Taylor & Francis. https://teachtogether.tech/.
Wood-Charlson, Elisha M., Zachary Crockett, Chris Erdmann, Adam P. Arkin, and Carly B. Robinson. 2022. “Ten Simple Rules for Getting and Giving Credit for Data.” PLOS Computational Biology 18 (9): 1–11. https://doi.org/10.1371/journal.pcbi.1010476.
Xie, Y. 2019. Bookdown: Authoring Books and Technical Documents with r Markdown. CRC Press. https://bookdown.org/yihui/bookdown/.
Xie, Y, JJ Allaire, and G Grolemund. 2018. R Markdown: The Definitive Guide. CRC Press. https://bookdown.org/yihui/rmarkdown/.

References

Corrections

If you see mistakes or want to suggest changes, please create an issue on the source repository.

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY-NC-SA 4.0. Source code is available at https://github.com/cgranell/rrp-uji, unless otherwise noted. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".

By Carlos Granell and Sergi Trilles © Copyright 2023.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

The site made with the distill R-package powered by the distill web framework