Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2021 Oct;18(10):1161-1168.
doi: 10.1038/s41592-021-01254-9. Epub 2021 Sep 23.

Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers

Laura Wratten  1 , Andreas Wilm  2 , Jonathan Göke  3
Affiliations
Review

Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers

Laura Wratten et al. Nat Methods. 2021 Oct.

Abstract

The rapid growth of high-throughput technologies has transformed biomedical research. With the increasing amount and complexity of data, scalability and reproducibility have become essential not just for experiments, but also for computational analysis. However, transforming data into information involves running a large number of tools, optimizing parameters, and integrating dynamically changing reference data. Workflow managers were developed in response to such challenges. They simplify pipeline development, optimize resource usage, handle software installation and versions, and run on different compute platforms, enabling workflow portability and sharing. In this Perspective, we highlight key features of workflow managers, compare commonly used approaches for bioinformatics workflows, and provide a guide for computational and noncomputational users. We outline community-curated pipeline initiatives that enable novice and experienced users to perform complex, best-practice analyses without having to manually assemble workflows. In sum, we illustrate how workflow managers contribute to making computational analysis in biomedical research shareable, scalable, and reproducible.

PubMed Disclaimer

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Stephens, Z. D. et al. Big data: astronomical or genomical? PLoS Biol. 13, e1002195 (2015). - PubMed - PMC - DOI
    1. Goodwin, S., McPherson, J. D. & McCombie, W. R. Coming of age: ten years of next-generation sequencing technologies. Nat. Rev. Genet. 17, 333–351 (2016). - PubMed - DOI
    1. Dozmorov, M. G. GitHub statistics as a measure of the impact of open-source bioinformatics software. Front. Bioeng. Biotechnol. 6, 198 (2018). - PubMed - PMC - DOI
    1. Nowogrodzki, A. How to support open-source software and stay sane. Nature 571, 133–134 (2019). - PubMed - DOI
    1. Mangul, S. et al. Challenges and recommendations to improve the installability and archival stability of omics computational tools. PLoS Biol. 17, e3000333 (2019). - PubMed - PMC - DOI

Publication types

MeSH terms

LinkOut - more resources

-