name: title class: Left, middle # After analysis · writing and sharing reproducible resources ### [Act II] Recomendations and practices for open and reproducible research .large[Reproducible Research Practices (RRP'23) · April 2023] .right[Carlos Granell · Sergi Trilles] .right[Universitat Jaume I] --- class: inverse, bottom, middle ## Nobel laureate and physics professor Wolfgang Pauli .large["I do not mind if you think slowly, but I do object when you publish more quickly than you think.”] ??? [Wolfgang Pauli](https://es.wikipedia.org/wiki/Wolfgang_Pauli), [Quotes](https://en.wikiquote.org/wiki/Wolfgang_Pauli) --- name: rec21 class: inverse, center, middle # .blue.bg-white[\#21] # Avoid *Data (or code) available upon request* --- class: left ### Data (or code) available upon request .huge[.small[<a name=cite-stodden2018></a>[[SSM18](https://www.pnas.org/doi/abs/10.1073/pnas.1708290115)]] - An empirical analysis of journal policy effectiveness for computational reproducibility] .large[...only .gray.bg-blue[44%] of requests led to receiving data and/or code from the original authors] --- name: rec22 class: inverse, center, middle # .blue.bg-white[\#22] # Report the software version you use --- class: left, top ### Software version .huge[Specify versions of relevant software components (packages, libraries)] .huge[A different version can lead to different results!] .large[.small[<a name=cite-mesnard2017></a>[[MB17](https://doi.org/10.1109/MCSE.2017.3151254)]] - Reproducible and Replicable Computational Fluid Dynamics: It's Harder Than You Think] - .large[Check "Story 4: Different Versions of Code, External Libraries, or Compilers can Challenge Reproducibility"] --- name: rec23 class: inverse, center, middle # .blue.bg-white[\#23] # Cite the software you use --- class: left ### Software citation .huge[Cite software that's important to the research outcome] - .large[[R engine](https://ropensci.org/blog/2021/11/16/how-to-cite-r-and-r-packages/)?] - type `citation()` to get reference publication in bibtex entry - add "All analyses were performed using R Statistical Software (v4.2.2; R Core Team 2022)" to **DASA section** - .large[[R packages](https://ropensci.org/blog/2021/11/16/how-to-cite-r-and-r-packages/)? It depends...] - type `citation("<packageName>")` to get reference publication in bibtex format - add "Temperature data was obtained from Environment and Climate Change Canada via the `weathercan` R package (v0.6.2; LaZerte and Albers 2018)" to **DASA section**. --- class: left ### Software citation .huge[[`report`](https://cran.r-project.org/package=report) R Package] ```r report::report_system() ``` ``` ## Analyses were conducted using the R Statistical language (version 4.2.2; R Core ## Team, 2022) on Windows 10 x64 (build 22621) ``` ```r report::report_packages() ``` ``` ## - xaringanthemer (version 0.4.2; Aden-Buie G, 2022) ## - xaringanExtra (version 0.7.0; Aden-Buie G, Warkentin M, 2022) ## - report (version 0.5.6; Makowski D et al., 2023) ## - RefManageR (version 1.3.0; McLean MW, 2017) ## - here (version 1.0.1; Müller K, 2020) ## - R (version 4.2.2; R Core Team, 2022) ``` --- class: left ### Software citation .pull-left[ .huge[Do not use URL to cite external library/code/software] - .large[Cite `SciPy` as a footnote: https://scipy.org/?] ] -- .pull-right[ .huge[Cite software as any other paper instead] - .large[Cite `SciPy` as a paper: [Virtanen et al. (2020)](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=SciPy+1.0%3A+fundamental+algorithms+for+scientific+computing+in+Python&btnG=)?] ] -- .huge[Readings:] - .large[[Software Citation Checklist for Authors](https://zenodo.org/record/3479199#.YYmfT73MKAk)] - .large[Software citation principles .small[<a name=cite-smith2016></a>[[SKN16](https://doi.org/10.7717/peerj-cs.86)]]] --- name: rec24 class: inverse, center, middle # .blue.bg-white[\#24] # Hello Quarto (and friends) --- class: left ### Hello Quarto (and friends) .huge[You can weave together narrative text and code to produce elegantly formatted output as documents, web pages, blog posts, books and more] .huge[Truly reproducible figures and tables are created directly with code and integrated into documents in a way that allows automatic updating when analyses are re-run, creating a “dynamic” document] .huge[[Quarto in 2 hours](https://jthomasmock.github.io/quarto-2hr-webinar/materials/01-intro-quarto.html)] --- class: left ### Hello Quarto (and friends) .huge[`knitr` started in 2011, RMarkdown in 2014]  --- class: left ### Hello Quarto (and friends) .huge[`Quarto` started in 2022]  --- class: left ### Hello Quarto (and friends) .huge[`Quarto` started in 2022]  --- class: left ### Hello Quarto (and friends)  ??? Quarto uses an engine like `knitr` to execute code and generate a temporary output `.md` The `.md` file is processed via Pandoc and Quarto's Lua filters + Bootstrap CSS for HTML or LaTeX for PDF and converted to a final output format Lua filters written by R/Python/Julia developers should be interchangeable between formats - typically not language specific! --- name: rec25 class: inverse, center, middle # .blue.bg-white[\#25] # Pre-print your work --- class: left ### Pre-print your works .huge[**Preprints** are versions of your paper that have not yet been sent to a journal for review] - .large[Choose the most appropriate copyright license for your work] - .large[Increase access and visibility to (pre-acceptance) versions of manuscripts] .huge[Where to publish a preprint paper?] - .large[[arXiv](https://arxiv.org/), [BioRxiv](https://www.biorxiv.org/), [ChemRxiv](https://chemrxiv.org/engage/chemrxiv/public-dashboard), [EarthArXiv](https://eartharxiv.org/), [psyArXiv](https://psyarxiv.com/), [SocArXiv](https://osf.io/preprints/socarxiv)] --- class: left ### Pre-print your works .huge[**[What UJI recommends...](https://www.uji.es/serveis/cd/bib/serveis/investigacio/oa/)**] > *Guardar siempre la última versión del artículo que se ha enviado en la revista después del proceso de peer review (esta versión recibe nombres diferentes: postprint author's final draft; accepted author manuscript; accepted version, postprint, etc.), puesto que algunas editoriales solo permiten publicar esta versión en los repositorios institucionales. Esta versión, y la versión definitiva publicada, cumplen los mandatos de la Ley de la ciencia y Horizonte Europa.* > *¿Qué se tiene que depositar en acceso abierto? Obligatoriamente, artículos de revistas científicas.* > *Se depositará la versión postprint, es decir, el manuscrito revisado y aceptado para su publicación, antes de las adaptaciones al formato final del editor, o la versión preprint.* ??? - [arXiv](https://arxiv.org/): physics, mathematics, computer science, quantitative biology, statistics, electrical engineering, and system sciences. - [BioRxiv](https://www.biorxiv.org/): biological sciences. - [ChemRxiv](https://chemrxiv.org/engage/chemrxiv/public-dashboard): chemical science. - [EarthArXiv](https://eartharxiv.org/)]: Earth Science and related domains of planetary science - [psyArXiv](https://psyarxiv.com/): psychology, powered by [OSFPreprints](https://osf.io/preprints/). - [SocArXiv](https://osf.io/preprints/socarxiv): social sciences, powered by [OSFPreprints](https://osf.io/preprints/). --- name: rec26: class: inverse, center, middle # .blue.bg-white[\#26] # Research Compendia --- class: left ### Research Compendia .huge[Data, code, software, and products of a research project are archived together] .huge[Standardized and easily recognizable way to organize digital materials of a research project] .huge[Tools to share every step of the scientific process] - .large[[Open Science Framework (OSF)](http://osf.io/) [Getting started on the OSF](https://help.osf.io/article/342-getting-started-on-the-osf)] - .large[[Octopus](https://www.octopus.ac/) .small[<a name=cite-dhar2023></a>[[Dha23](http://dx.doi.org/10.1038/d41586-023-00861-0)]]] - .large[[ResearchEquals](https://www.researchequals.com/) .small[[[Dha23](http://dx.doi.org/10.1038/d41586-023-00861-0)]]] --- name: rec27 class: inverse, center, middle # .blue.bg-white[\#26] # Interactive papers --- class: left ### Interactive papers .huge[Reproducible entire analysis...] - .large[remote execution via Binder, Google Colab, ...] - .large[local execution via Docker or `renv`] -- .huge[What if interested in a small fraction of an analysis] -- - .large[protected computing environment (sandbox) + notebooks + `learnr` R package .small[<a name=cite-aden-buie2023></a>[[Ade+23](https://rstudio.github.io/learnr/)]]] - .large[Ex: [GB rainfall chemistry paper](https://cptecn-sandboxdemo.datalabs.ceh.ac.uk/) .small[<a name=cite-tso2022></a>[[Tso+22](https://doi.org/10.32614/RJ-2022-021)]]] --- name: summary class: inverse, center, middle # Summary --- - .large[[Avoid *Data (or code) available upon request*](#rec21)] - .large[[Report the software version you use](#rec22)] - .large[[Cite the software you use](#rec23)] - .large[[Hello Quarto (and friends)](#rec24)] - .large[[Pre-print your work](#rec25)] - .large[[Research Compendia](#rec26)] - .large[[Interactive papers](#rec27)] --- # References .tiny[ <a name=bib-smith2016></a>[Smith, AM, DS Katz, et al.](#cite-smith2016) (2016). "Software citation principles". In: _PeerJ Computer Science_ 2, p. e86. URL: [https://doi.org/10.7717/peerj-cs.86](https://doi.org/10.7717/peerj-cs.86). <a name=bib-mesnard2017></a>[Mesnard, Olivier and Lorena A. Barba](#cite-mesnard2017) (2017). "Reproducible and Replicable Computational Fluid Dynamics: It's Harder Than You Think". In: _Computing in Science Engineering_ 19.4, pp. 44-55. URL: [https://doi.org/10.1109/MCSE.2017.3151254](https://doi.org/10.1109/MCSE.2017.3151254). <a name=bib-stodden2018></a>[Stodden, Victoria, Jennifer Seiler, et al.](#cite-stodden2018) (2018). "An empirical analysis of journal policy effectiveness for computational reproducibility". In: _Proceedings of the National Academy of Sciences_ 115.11, pp. 2584-2589. URL: [https://www.pnas.org/doi/abs/10.1073/pnas.1708290115](https://www.pnas.org/doi/abs/10.1073/pnas.1708290115). <a name=bib-tso2022></a>[Tso, Chak Hau Michael, Michael Hollaway, et al.](#cite-tso2022) (2022). "The R Journal: Advancing Reproducible Research by Publishing R Markdown Notebooks as Interactive Sandboxes Using the learnr Package". In: _The R Journal_ 14 (1), pp. 255-263. ISSN: 2073-4859. URL: [https://doi.org/10.32614/RJ-2022-021](https://doi.org/10.32614/RJ-2022-021). <a name=bib-aden-buie2023></a>[Aden-Buie, Garrick, Barret Schloerke, et al.](#cite-aden-buie2023) (2023). _learnr: Interactive Tutorials for R_. URL: [https://rstudio.github.io/learnr/](https://rstudio.github.io/learnr/). <a name=bib-dhar2023></a>[Dhar, Payal](#cite-dhar2023) (2023). _Octopus and ResearchEquals aim to break the publishing mould_. URL: [http://dx.doi.org/10.1038/d41586-023-00861-0](http://dx.doi.org/10.1038/d41586-023-00861-0). ]