Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Next-generation data filtering in the genomics era

Nature Reviews Genetics (2024)Cite this article

Subjects

Abstract

Genomic data are ubiquitous across disciplines, from agriculture to biodiversity, ecology, evolution and human health. However, these datasets often contain noise or errors and are missing information that can affect the accuracy and reliability of subsequent computational analyses and conclusions. A key step in genomic data analysis is filtering — removing sequencing bases, reads, genetic variants and/or individuals from a dataset — to improve data quality for downstream analyses. Researchers are confronted with a multitude of choices when filtering genomic data; they must choose which filters to apply and select appropriate thresholds. To help usher in the next generation of genomic data filtering, we review and suggest best practices to improve the implementation, reproducibility and reporting standards for filter types and thresholds commonly applied to genomic datasets. We focus mainly on filters for minor allele frequency, missing data per individual or per locus, linkage disequilibrium and Hardy–Weinberg deviations. Using simulated and empirical datasets, we illustrate the large effects of different filtering thresholds on common population genetics statistics, such as Tajima’s D value, population differentiation (FST), nucleotide diversity (π) and effective population size (Ne).

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Pre-variant filtering — challenges and potential solutions related to filtering before variant calling.
Fig. 2: Post-variant filtering — challenges associated with four common filters after variant discovery.
Fig. 3: Flow chart to facilitate thoughtful, systematic and reproducible filtering for representative studies and questions using genomic DNA.

Similar content being viewed by others

Data availability

Information on the empirical and simulated data used for the analyses shown in this review is available in the Supplementary Information.

Code availability

The simulation code is available on GitHub at: https://github.com/ChristieLab/filtering_simulation_paper.

References

  1. Allendorf, F. W., Hohenlohe, P. A. & Luikart, G. Genomics and the future of conservation genetics. Nat. Rev. Genet. 11, 697–709 (2010).

    Article  CAS  PubMed  Google Scholar 

  2. Athanasopoulou, K., Boti, M. A., Adamopoulos, P. G., Skourou, P. C. & Scorilas, A. Third-generation sequencing: the spearhead towards the radical transformation of modern genomics. Life 12, 30 (2022).

    Article  CAS  Google Scholar 

  3. Fiedler, P. L. et al. Seizing the moment: the opportunity and relevance of the California Conservation Genomics Project to state and federal conservation policy. J. Hered. 113, 589–596 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Hu, T., Chitnis, N., Monos, D. & Dinh, A. Next-generation sequencing technologies: an overview. Hum. Immunol. 82, 801–811 (2021).

    Article  CAS  PubMed  Google Scholar 

  5. Pompanon, F., Bonin, A., Bellemain, E. & Taberlet, P. Genotyping errors: causes, consequences and solutions. Nat. Rev. Genet. 6, 847–859 (2005). This review summarizes the sources of many common types of sequencing errors and provides some laboratory and bioinformatic ways to mitigate them.

    Article  CAS  PubMed  Google Scholar 

  6. Stoler, N. & Nekrutenko, A. Sequencing error profiles of Illumina sequencing instruments. NAR Genom. Bioinform. 3, lqab019 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Fountain, E. D., Pauli, J. N., Reid, B. N., Palsbøll, P. J. & Peery, M. Z. Finding the right coverage: the impact of coverage and sequence quality on single nucleotide polymorphism genotyping error rates. Mol. Ecol. Resour. 16, 966–978 (2016).

    Article  CAS  PubMed  Google Scholar 

  8. O’Leary, S. J., Puritz, J. B., Willis, S. C., Hollenbeck, C. M. & Portnoy, D. S. These aren’t the loci you’re looking for: principles of effective SNP filtering for molecular ecologists. Mol. Ecol. 27, 3193–3206 (2018). This helpful review discusses the effects of missing data, MAC and other filters on genotyping error rates for RADseq data.

    Article  Google Scholar 

  9. Rochette, N. C., Rivera-Colón, A. G. & Catchen, J. M. Stacks 2: analytical methods for paired-end sequencing improve RADseq-based population genomics. Mol. Ecol. 28, 4737–4754 (2019).

    Article  CAS  PubMed  Google Scholar 

  10. Ahrens, C. W. et al. Regarding the F-word: the effects of data filtering on inferred genotype–environment associations. Mol. Ecol. Resour. 21, 1460–1474 (2021).

    Article  PubMed  Google Scholar 

  11. Andrews, K. R. & Luikart, G. Recent novel approaches for population genomics data analysis. Mol. Ecol. 23, 1661–1667 (2014).

    Article  PubMed  Google Scholar 

  12. Shafer, A. B. A. et al. Bioinformatic processing of RAD-seq data dramatically impacts downstream population genetic inference. Methods Ecol. Evol. 8, 907–917 (2017). This study demonstrates the effects of different filtering and alignment choices on several downstream statistics and demographic reconstruction in RADseq data.

    Article  Google Scholar 

  13. Larson, W. A., Isermann, D. A. & Feiner, Z. S. Incomplete bioinformatic filtering and inadequate age and growth analysis lead to an incorrect inference of harvested-induced changes. Evol. Appl. 14, 278–289 (2021).

    Article  CAS  PubMed  Google Scholar 

  14. Nazareno, A. G. & Knowles, L. L. There is no ‘rule of thumb’: genomic filter settings for a small plant population to obtain unbiased gene flow estimates. Front. Plant Sci. 12, 677009 (2021). This comprehensive analysis of empirical data demonstrates how missing data and MAF thresholds affect estimates of gene flow.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Sethuraman, A. et al. Continued misuse of multiple testing correction methods in population genetics — a wake-up call? Mol. Ecol. Resour. 19, 23–26 (2019).

    Article  PubMed  Google Scholar 

  16. Allendorf, F. W. et al. Conservation and the Genomics of Populations (Oxford Univ. Press, 2022).

  17. Gervais, L. et al. RAD-sequencing for estimating genomic relatedness matrix-based heritability in the wild: a case study in roe deer. Mol. Ecol. Resour. 19, 1205–1217 (2019).

    Article  CAS  PubMed  Google Scholar 

  18. Crow, J. F. & Kimura, M. An Introduction to Population Genetics Theory (Scientific Publishers, 2017).

  19. Van Etten, J., Stephens, T. G. & Bhattacharya, D. A k-mer-based approach for phylogenetic classification of taxa in environmental genomic data. Syst. Biol. 72, 1101–1118 (2023).

    Article  CAS  PubMed  Google Scholar 

  20. Todd, E. V., Black, M. A. & Gemmell, N. J. The power and promise of RNA-seq in ecology and evolution. Mol. Ecol. 25, 1224–1241 (2016).

    Article  CAS  PubMed  Google Scholar 

  21. Conesa, A. et al. A survey of best practices for RNA-seq data analysis. Genome Biol. 17, 13 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Olofsson, D., Preußner, M., Kowar, A., Heyd, F. & Neumann, A. One pipeline to predict them all? On the prediction of alternative splicing from RNA-seq data. Biochem. Biophys. Res. Commun. 653, 31–37 (2023).

    Article  CAS  PubMed  Google Scholar 

  23. Upton, R. N. et al. Design, execution, and interpretation of plant RNA-seq analyses. Front. Plant Sci. 14, 1135455 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Rehn, J. et al. RaScALL: rapid (Ra) screening (Sc) of RNA-seq data for prognostically significant genomic alterations in acute lymphoblastic leukaemia (ALL). PLOS Genet. 18, e1010300 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Boshuizen, H. C. & te Beest, D. E. Pitfalls in the statistical analysis of microbiome amplicon sequencing data. Mol. Ecol. Resour. 23, 539–548 (2023).

    Article  PubMed  Google Scholar 

  26. Combrink, L. et al. Best practice for wildlife gut microbiome research: a comprehensive review of methodology for 16S rRNA gene investigations. Front. Microbiol. 14, 1092216 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  27. Cheng, Z. et al. Transcriptomic analysis of circulating leukocytes obtained during the recovery from clinical mastitis caused by Escherichia coli in Holstein dairy cows. Animals 12, 2146 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Yang, L. & Chen, J. Benchmarking differential abundance analysis methods for correlated microbiome sequencing data. Brief. Bioinformatics 24, bbac607 (2023).

    Article  PubMed  Google Scholar 

  29. Patin, N. V. & Goodwin, K. D. Capturing marine microbiomes and environmental DNA: a field sampling guide. Front. Microbiol. 13, 1026596 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Ruppert, K. M., Kline, R. J. & Rahman, M. S. Past, present, and future perspectives of environmental DNA (eDNA) metabarcoding: a systematic review in methods, monitoring, and applications of global eDNA. Glob. Ecol. Conserv. 17, e00547 (2019).

    Google Scholar 

  31. Deyneko, I. V. et al. Modeling and cleaning RNA-seq data significantly improve detection of differentially expressed genes. BMC Bioinformatics 23, 488 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Giusti, A., Malloggi, C., Magagna, G., Filipello, V. & Armani, A. Is the metabarcoding ripe enough to be applied to the authentication of foodstuff of animal origin? A systematic review. Compr. Rev. Food Sci. Food Saf. 23, 1–21 (2024).

    Article  Google Scholar 

  33. da Fonseca, R. R. et al. Next-generation biology: sequencing and data analysis approaches for non-model organisms. Mar. Genomics 30, 3–13 (2016).

    Article  PubMed  Google Scholar 

  34. Zhao, M. et al. Exploring conflicts in whole genome phylogenetics: a case study within manakins (Aves: Pipridae). Syst. Biol. 72, 161–178 (2023).

    Article  CAS  PubMed  Google Scholar 

  35. Koboldt, D. C. Best practices for variant calling in clinical sequencing. Genome Med 12, 91 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Giani, A. M., Gallo, G. R., Gianfranceschi, L. & Formenti, G. Long walk to genomics: history and current approaches to genome sequencing and assembly. Comput. Struct. Biotechnol. J. 18, 9–19 (2020).

    Article  CAS  PubMed  Google Scholar 

  37. Kumar, K. R., Cowley, M. J. & Davis, R. L. Next-generation sequencing and emerging technologies. Semin. Thromb. Hemost. 45, 661–673 (2019).

    Article  CAS  PubMed  Google Scholar 

  38. Shendure, J. et al. DNA sequencing at 40: past, present and future. Nature 550, 345–353 (2017).

    Article  CAS  PubMed  Google Scholar 

  39. Lou, R. N., Jacobs, A., Wilder, A. P. & Therkildsen, N. O. A beginner’s guide to low-coverage whole genome sequencing for population genomics. Mol. Ecol. 30, 5966–5993 (2021). This reviews discusses the production and analysis of low-coverage WGS data.

    Article  PubMed  Google Scholar 

  40. Olson, N. D. et al. Variant calling and benchmarking in an era of complete human genome sequences. Nat. Rev. Genet. 24, 464–483 (2023).

    Article  CAS  PubMed  Google Scholar 

  41. Rochette, N. C. & Catchen, J. M. Deriving genotypes from RAD-seq short-read data using Stacks. Nat. Protoc. 12, 2640–2659 (2017).

    Article  CAS  PubMed  Google Scholar 

  42. Paris, J. R., Stevens, J. R. & Catchen, J. M. Lost in parameter space: a road map for stacks. Methods Ecol. Evol. 8, 1360–1373 (2017).

    Article  Google Scholar 

  43. Ceballos, F. C., Joshi, P. K., Clark, D. W., Ramsay, M. & Wilson, J. F. Runs of homozygosity: windows into population history and trait architecture. Nat. Rev. Genet. 19, 220–234 (2018).

    Article  CAS  PubMed  Google Scholar 

  44. Heller, R. et al. A reference-free approach to analyse RADseq data using standard next generation sequencing toolkits. Mol. Ecol. Resour. 21, 1085–1097 (2021).

    Article  CAS  PubMed  Google Scholar 

  45. Bohling, J. Evaluating the effect of reference genome divergence on the analysis of empirical RADseq datasets. Ecol. Evol. 10, 7585–7601 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Valiente-Mullor, C. et al. One is not enough: on the effects of reference genome for the mapping and subsequent analyses of short-reads. PLOS Comput. Biol. 17, e1008678 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Hendricks, S. et al. Recent advances in conservation and population genomics data analysis. Evol. Appl. 11, 1197–1211 (2018).

    Article  PubMed Central  Google Scholar 

  48. Vaux, F., Dutoit, L., Fraser, C. I. & Waters, J. M. Genotyping-by-sequencing for biogeography. J. Biogeogr. 50, 262–281 (2023).

    Article  Google Scholar 

  49. Jackson, B. C., Campos, J. L. & Zeng, K. The effects of purifying selection on patterns of genetic differentiation between Drosophila melanogaster populations. Heredity 114, 163–174 (2015).

    Article  CAS  PubMed  Google Scholar 

  50. Luikart, G., England, P. R., Tallmon, D., Jordan, S. & Taberlet, P. The power and promise of population genomics: from genotyping to genome typing. Nat. Rev. Genet. 4, 981–994 (2003).

    Article  CAS  PubMed  Google Scholar 

  51. Benestan, L. et al. Sex matters in massive parallel sequencing: evidence for biases in genetic parameter estimation and investigation of sex determination systems. Mol. Ecol. 26, 6767–6783 (2017).

    Article  CAS  PubMed  Google Scholar 

  52. Yang, Z. et al. Multi-omics provides new insights into the domestication and improvement of dark jute (Corchorus olitorius). Plant J. 112, 812–829 (2022).

    Article  CAS  PubMed  Google Scholar 

  53. Zeng, L. et al. Whole genomes and transcriptomes reveal adaptation and domestication of pistachio. Genome Biol. 20, 79 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  54. Zhernakova, D. V. et al. Genome-wide sequence analyses of ethnic populations across Russia. Genomics 112, 442–458 (2020).

    Article  CAS  PubMed  Google Scholar 

  55. Langmead, B. & Salzberg, S. L. Fast gapped-read alignment with Bowtie 2. Nat. Methods 9, 357–359 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Li, H. & Durbin, R. Fast and accurate short read alignment with Burrows–Wheeler transform. Bioinformatics 25, 1754–1760 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Pfeifer, S. P. From next-generation resequencing reads to a high-quality variant data set. Heredity 118, 111–124 (2017).

    Article  CAS  PubMed  Google Scholar 

  58. Lefouili, M. & Nam, K. The evaluation of BCFtools mpileup and GATK HaplotypeCaller for variant calling in non-human species. Sci. Rep. 12, 11331 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Chen, N.-C., Solomon, B., Mun, T., Iyer, S. & Langmead, B. Reference flow: reducing reference bias using multiple population genomes. Genome Biol. 22, 8 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  60. Günther, T. & Nettelblad, C. The presence and impact of reference bias on population genomic studies of prehistoric human populations. PLOS Genet. 15, e1008302 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Rhie, A. et al. Towards complete and error-free genome assemblies of all vertebrate species. Nature 592, 737–746 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Ho, S. S., Urban, A. E. & Mills, R. E. Structural variation in the sequencing era. Nat. Rev. Genet. 21, 171–189 (2020).

    Article  CAS  PubMed  Google Scholar 

  63. Singh, A. K. et al. Detecting copy number variation in next generation sequencing data from diagnostic gene panels. BMC Med. Genomics 14, 214 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Willis, S. C., Hollenbeck, C. M., Puritz, J. B., Gold, J. R. & Portnoy, D. S. Haplotyping RAD loci: an efficient method to filter paralogs and account for physical linkage. Mol. Ecol. Resour. 17, 955–965 (2017).

    Article  CAS  PubMed  Google Scholar 

  65. Ou, S. et al. Benchmarking transposable element annotation methods for creation of a streamlined, comprehensive pipeline. Genome Biol. 20, 275 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Rochette, N. C. et al. On the causes, consequences, and avoidance of PCR duplicates: towards a theory of library complexity. Mol. Ecol. Resour. 23, 1299–1318 (2023).

    Article  CAS  PubMed  Google Scholar 

  67. Van der Auwera, G. A. & O’Connor, B. D. Genomics in the Cloud: Using Docker, GATK, and WDL in Terra (O’Reilly Media, 2020).

  68. Korneliussen, T. S., Albrechtsen, A. & Nielsen, R. ANGSD: analysis of next generation sequencing data. BMC Bioinformatics 15, 356 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  69. Eaton, D. A. R. & Overcast, I. ipyrad: interactive assembly and analysis of RADseq datasets. Bioinformatics 36, 2592–2594 (2020).

    Article  CAS  PubMed  Google Scholar 

  70. Layer, R. M., Chiang, C., Quinlan, A. R. & Hall, I. M. LUMPY: a probabilistic framework for structural variant discovery. Genome Biol. 15, R84 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  71. Danecek, P. et al. Twelve years of SAMtools and BCFtools. Gigascience 10, giab008 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  72. Mona, S., Benazzo, A., Delrieu-Trottin, E. & Lesturgie, P. Population genetics using low coverage RADseq data in non-model organisms: biases and solutions. Preprint at Authorea https://doi.org/10.22541/au.168252801.19878064/v1 (2023).

  73. Nielsen, R., Korneliussen, T., Albrechtsen, A., Li, Y. & Wang, J. SNP calling, genotype calling, and sample allele frequency estimation from new-generation sequencing data. PLoS ONE 7, e37558 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  74. Warmuth, V. M. & Ellegren, H. Genotype-free estimation of allele frequencies reduces bias and improves demographic inference from RADseq data. Mol. Ecol. Resour. 19, 586–596 (2019).

    Article  CAS  PubMed  Google Scholar 

  75. Wright, B. et al. From reference genomes to population genomics: comparing three reference-aligned reduced-representation sequencing pipelines in two wildlife species. BMC Genomics 20, 453 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Huang, H. & Knowles, L. L. Unforeseen consequences of excluding missing data from next-generation sequences: simulation study of RAD sequences. Syst. Biol. 65, 357–365 (2016).

    Article  CAS  PubMed  Google Scholar 

  77. Duntsch, L., Whibley, A., Brekke, P., Ewen, J. G. & Santure, A. W. Genomic data of different resolutions reveal consistent inbreeding estimates but contrasting homozygosity landscapes for the threatened Aotearoa New Zealand hihi. Mol. Ecol. 30, 6006–6020 (2021).

    Article  CAS  PubMed  Google Scholar 

  78. Kardos, M. & Waples, R. S. Low-coverage sequencing and Wahlund effect severely bias estimates of inbreeding, heterozygosity, and effective population size in North American wolves. Mol. Ecol. https://doi.org/10.1111/mec.17415 (2024). This study reports biases that could affect management decisions caused by next-generation sequencing filtering choices, low-coverage data and the sampling strategy.

  79. Schmidt, T. L., Jasper, M.-E., Weeks, A. R. & Hoffmann, A. A. Unbiased population heterozygosity estimates from genome-wide sequence data. Methods Ecol. Evol. 12, 1888–1898 (2021).

    Article  Google Scholar 

  80. Sopniewski, J. & Catullo, R. A. Estimates of heterozygosity from single nucleotide polymorphism markers are context-dependent and often wrong. Mol. Ecol. Resour. 24, e13947 (2024).

    Article  CAS  PubMed  Google Scholar 

  81. Pritchard, J. K., Stephens, M. & Donnelly, P. Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  82. Waples, R. S. Testing for Hardy–Weinberg proportions: have we lost the plot? J. Hered. 106, 1–19 (2015).

    Article  PubMed  Google Scholar 

  83. Gautier, M. et al. The effect of RAD allele dropout on the estimation of genetic variation within and between populations. Mol. Ecol. 22, 3165–3178 (2013).

    Article  CAS  PubMed  Google Scholar 

  84. McKinney, G. J., Waples, R. K., Seeb, L. W. & Seeb, J. E. Paralogs are revealed by proportion of heterozygotes and deviations in read ratios in genotyping-by-sequencing data from natural populations. Mol. Ecol. Resour. 17, 656–669 (2017).

    Article  CAS  PubMed  Google Scholar 

  85. Bitarello, B. D., Brandt, D. Y. C., Meyer, D. & Andrés, A. M. Inferring balancing selection from genome-scale data. Genome Biol. Evol. 15, evad032 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  86. Pearman, W. S., Urban, L. & Alexander, A. Commonly used Hardy–Weinberg equilibrium filtering schemes impact population structure inferences using RADseq data. Mol. Ecol. Resour. 22, 2599–2613 (2022). This study demonstrates the impact of pooling or splitting sample-groups when applying HWP filters to FST and other population structure inferences.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  87. Linderoth, T. P. Identifying population histories, adaptive genes, and genetic duplication from population-scale next generation sequencing. Genome Res. 20, 291–300 (2018).

    Google Scholar 

  88. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B Methodol. 57, 289–300 (1995).

    Article  Google Scholar 

  89. Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979).

    Google Scholar 

  90. Graffelman, J., Jain, D. & Weir, B. A genome-wide study of Hardy–Weinberg equilibrium with next generation sequence data. Hum. Genet. 136, 727–741 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Larson, W. A. et al. Genotyping by sequencing resolves shallow population structure to inform conservation of Chinook salmon (Oncorhynchus tshawytscha). Evol. Appl. 7, 355–369 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  92. Waples, R. K., Larson, W. A. & Waples, R. S. Estimating contemporary effective population size in non-model species using linkage disequilibrium across thousands of loci. Heredity 117, 233–240 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  93. Gattepaille, L. M., Jakobsson, M. & Blum, M. G. Inferring population size changes with sequence and SNP data: lessons from human bottlenecks. Heredity 110, 409–419 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  94. Tajima, F. Statistical method for testing the neutral mutation hypothesis by DNA polymorphism. Genetics 123, 585 LP–585595 (1989).

    Article  Google Scholar 

  95. Arantes, L. S. et al. Scaling-up RADseq methods for large datasets of non-invasive samples: lessons for library construction and data preprocessing. Mol. Ecol. Resour. https://doi.org/10.1111/1755-0998.13859 (2023).

  96. Cubry, P., Vigouroux, Y. & François, O. The empirical distribution of singletons for geographic samples of DNA sequences. Front. Genet. 8, 139 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Linck, E. & Battey, C. J. Minor allele frequency thresholds strongly affect population structure inference with genomic data sets. Mol. Ecol. Resour. 19, 639–647 (2019). This study demonstrates how MAF thresholds affect population structure inferences using both simulated and empirical data.

    Article  CAS  PubMed  Google Scholar 

  98. Andersson, B. A., Zhao, W., Haller, B. C., Brännström, Å. & Wang, X.-R. Inference of the distribution of fitness effects of mutations is affected by single nucleotide polymorphism filtering methods, sample size and population structure. Mol. Ecol. Resour. 23, 1589–1603 (2023).

    Article  CAS  PubMed  Google Scholar 

  99. Díaz-Arce, N. & Rodríguez-Ezpeleta, N. Selecting RAD-seq data analysis parameters for population genetics: the more the better? Front. Genet. 10, 533 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  100. Holsinger, K. E. & Weir, B. S. Genetics in geographically structured populations: defining, estimating and interpreting FST. Nat. Rev. Genet. 10, 639–650 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  101. Roesti, M., Salzburger, W. & Berner, D. Uninformative polymorphisms bias genome scans for signatures of selection. BMC Evol. Biol. 12, 94 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  102. Yin, X. et al. Rapid, simultaneous increases in the effective sizes of adaptively divergent yellow perch (Perca flavescens) populations. Preprint at bioRxiv https://doi.org/10.1101/2024.04.21.590447 (2024).

  103. Visscher, P. M. et al. 10 years of GWAS discovery: biology, function, and translation. Am. J. Hum. Genet. 101, 5–22 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  104. Tennessen, J. A. et al. Evolution and functional impact of rare coding variation from deep sequencing of human exomes. Science 337, 64–69 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  105. Dementieva, N. V. et al. Assessing the effects of rare alleles and linkage disequilibrium on estimates of genetic diversity in the chicken populations. Animal 15, 100171 (2021).

    Article  CAS  PubMed  Google Scholar 

  106. De Meeûs, T. Revisiting FIS, FST, Wahlund effects, and null alleles. J. Hered. 109, 446–456 (2018).

    Article  PubMed  Google Scholar 

  107. Levy-Sakin, M. et al. Genome maps across 26 human populations reveal population-specific patterns of structural variation. Nat. Commun. 10, 1025 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  108. Zhang, H., Yin, L., Wang, M., Yuan, X. & Liu, X. Factors affecting the accuracy of genomic selection for agricultural economic traits in maize, cattle, and pig populations. Front. Genet. 10, 189 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  109. Anderson, E. C. & Garza, J. C. The power of single-nucleotide polymorphisms for large-scale parentage inference. Genetics 172, 2567–2582 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  110. Dussault, F. M. & Boulding, E. G. Effect of minor allele frequency on the number of single nucleotide polymorphisms needed for accurate parentage assignment: a methodology illustrated using Atlantic salmon. Aquac. Res. 49, 1368–1372 (2018).

    Article  Google Scholar 

  111. Thompson, E. The estimation of pairwise relationships. Ann. Hum. Genet. 39, 173–188 (1975).

    Article  CAS  PubMed  Google Scholar 

  112. Goubert, C. et al. A beginner’s guide to manual curation of transposable elements. Mob. DNA 13, 7 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  113. Storer, J. M., Hubley, R., Rosen, J. & Smit, A. F. A. Curation guidelines for de novo generated transposable element families. Curr. Protoc. 1, e154 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  114. Hemstrom, W. B., Freedman, M. G., Zalucki, M. P., Ramírez, S. R. & Miller, M. R. Population genetics of a recent range expansion and subsequent loss of migration in monarch butterflies. Mol. Ecol. 31, 4544–4557 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  115. Escoda, L., González-Esteban, J., Gómez, A. & Castresana, J. Using relatedness networks to infer contemporary dispersal: application to the endangered mammal Galemys pyrenaicus. Mol. Ecol. 26, 3343–3357 (2017).

    Article  PubMed  Google Scholar 

  116. Brown, A. V. et al. Ten quick tips for sharing open genomic data. PLOS Comput. Biol. 14, e1006472 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Zhang, D. et al. PhyloSuite: an integrated and scalable desktop platform for streamlined molecular sequence data management and evolutionary phylogenetics studies. Mol. Ecol. Resour. 20, 348–355 (2020).

    Article  PubMed  Google Scholar 

  118. Tanjo, T., Kawai, Y., Tokunaga, K., Ogasawara, O. & Nagasaki, M. Practical guide for managing large-scale human genome data in research. J. Hum. Genet. 66, 39–52 (2021).

    Article  PubMed  Google Scholar 

  119. Del Fabbro, C., Scalabrin, S., Morgante, M. & Giorgi, F. M. An extensive evaluation of read trimming effects on illumina NGS data analysis. PLoS ONE 8, e85024 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  120. Yang, S.-F., Lu, C.-W., Yao, C.-T. & Hung, C.-M. To trim or not to trim: effects of read trimming on the de novo genome assembly of a widespread East Asian passerine, the rufous-capped babbler (Cyanoderma ruficeps Blyth). Genes 10, 737 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  121. Hotaling, S. et al. Demographic modelling reveals a history of divergence with gene flow for a glacially tied stonefly in a changing post-Pleistocene landscape. J. Biogeogr. 45, 304–317 (2018).

    Article  Google Scholar 

  122. Cumer, T. et al. Double-digest RAD-sequencing: do pre- and post-sequencing protocol parameters impact biological results? Mol. Genet. Genomics 296, 457–471 (2021).

    Article  CAS  PubMed  Google Scholar 

  123. Mastretta-Yanes, A. et al. Restriction site-associated DNA sequencing, genotyping error estimation and de novo assembly optimization for population genetic inference. Mol. Ecol. Resour. 15, 28–41 (2015).

    Article  CAS  PubMed  Google Scholar 

  124. Ebbert, M. T. W. et al. Evaluating the necessity of PCR duplicate removal from next-generation sequencing data and a comparison of approaches. BMC Bioinformatics 17, 239 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  125. Euclide, P. T. et al. Attack of the PCR clones: rates of clonality have little effect on RAD-seq genotype calls. Mol. Ecol. Resour. 20, 66–78 (2020).

    Article  CAS  PubMed  Google Scholar 

  126. Flanagan, S. P. & Jones, A. G. Substantial differences in bias between single-digest and double-digest RAD-seq libraries: a case study. Mol. Ecol. Resour. 18, 264–280 (2018).

    Article  CAS  PubMed  Google Scholar 

  127. Martins, F. B. et al. A semi-automated SNP-based approach for contaminant identification in biparental polyploid populations of tropical forage grasses. Front. Plant Sci. 12, 737919 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  128. Deo, T. G. et al. High-resolution linkage map with allele dosage allows the identification of regions governing complex traits and apospory in guinea grass (Megathyrsus maximus). Front. Plant Sci. 11, 15 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  129. Zhang, F. et al. Ancestry-agnostic estimation of DNA sample contamination from sequence reads. Genome Res. 30, 185–194 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  130. Christie, M. R., Marine, M. L., Fox, S. E., French, R. A. & Blouin, M. S. A single generation of domestication heritably alters the expression of hundreds of genes. Nat. Commun. 7, 10676 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  131. Lou, R. N. & Therkildsen, N. O. Batch effects in population genomic studies with low-coverage whole genome sequencing data: causes, detection and mitigation. Mol. Ecol. Resour. 22, 1678–1692 (2022).

    Article  CAS  PubMed  Google Scholar 

  132. Danecek, P. et al. The variant call format and VCFtools. Bioinformatics 27, 2156–2158 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  133. Mirchandani, C. D. et al. A fast, reproducible, high-throughput variant calling workflow for population genomics. Mol. Biol. Evol. 41, msad270 (2024).

    Article  PubMed  Google Scholar 

  134. Peñalba, J. V., Peters, J. L. & Joseph, L. Sustained plumage divergence despite weak genomic differentiation and broad sympatry in sister species of Australian woodswallows (Artamus spp.). Mol. Ecol. 31, 5060–5073 (2022).

    Article  PubMed  Google Scholar 

  135. Thompson, N. F. et al. A complex phenotype in salmon controlled by a simple change in migratory timing. Science 370, 609–613 (2020).

    Article  CAS  PubMed  Google Scholar 

  136. Howe, K. et al. Significantly improving the quality of genome assemblies through curation. Gigascience 10, giaa153 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  137. Nurk, S. et al. The complete sequence of a human genome. Science 376, 44–53 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  138. Michael, T. P. & VanBuren, R. Building near-complete plant genomes. Genome Stud. Mol. Genet. 54, 26–33 (2020).

    CAS  Google Scholar 

  139. Tettelin, H. & Medini, D. The Pangenome: Diversity, Dynamics and Evolution of Genomes (Springer, 2020).

  140. Wang, T. et al. The Human Pangenome Project: a global resource to map genomic diversity. Nature 604, 437–446 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  141. Hemstrom, W. Thirty-Four Kilometers and Fifteen Years: Rapid Adaptation at a Novel Chromosomal Inversion in Recently Introduced Deschutes River Three-Spined Stickleback. Thesis, Oregon State Univ. (2016).

  142. Halvorsen, S., Korslund, L., Mattingsdal, M. & Slettan, A. Estimating number of European eel (Anguilla anguilla) individuals using environmental DNA and haplotype count in small rivers. Ecol. Evol. 13, e9785 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  143. Whitlock, M. C. & Lotterhos, K. E. Reliable detection of loci responsible for local adaptation: inference of a null model through trimming the distribution of FST. Am. Nat. 186, S24–S36 (2015).

    Article  PubMed  Google Scholar 

  144. vonHoldt, B. M. et al. Demographic history shapes North American gray wolf genomic diversity and informs species’ conservation. Mol. Ecol. 33, e17231 (2024).

    Article  CAS  PubMed  Google Scholar 

  145. Alonso-Blanco, C. et al. 1,135 genomes reveal the global pattern of polymorphism in Arabidopsis thaliana. Cell 166, 481–491 (2016).

    Article  Google Scholar 

  146. Maruki, T., Ye, Z. & Lynch, M. Evolutionary genomics of a subdivided species. Mol. Biol. Evol. 39, msac152 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  147. Kessler, C., Wootton, E. & Shafer, A. B. A. Speciation without gene-flow in hybridizing deer. Mol. Ecol. 32, 1117–1132 (2023).

    Article  CAS  PubMed  Google Scholar 

  148. Martchenko, D. & Shafer, A. B. A. Contrasting whole-genome and reduced representation sequencing for population demographic and adaptive inference: an alpine mammal case study. Heredity 131, 273–281 (2023).

    Article  CAS  PubMed  Google Scholar 

  149. Lowy-Gallego, E. et al. Variant calling on the GRCh38 assembly with the data from phase three of the 1000 Genomes Project. Wellcome Open Res. 4, 50 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  150. Schweizer, R. M. et al. Broad concordance in the spatial distribution of adaptive and neutral genetic variation across an elevational gradient in deer mice. Mol. Biol. Evol. 38, 4286–4300 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  151. Kardos, M. et al. Inbreeding depression explains killer whale population dynamics. Nat. Ecol. Evol. 7, 675–686 (2023).

    Article  PubMed  Google Scholar 

  152. Malison, R. L. et al. Landscape connectivity and genetic structure in a mainstem and a tributary stonefly (Plecoptera) species using a novel reference genome. J. Hered. 113, 453–471 (2022).

    Article  CAS  PubMed  Google Scholar 

  153. Robinson, J. M. et al. Traditional ecological knowledge in restoration ecology: a call to listen deeply, to engage with, and respect Indigenous voices. Restor. Ecol. 29, e13381 (2021).

    Article  Google Scholar 

  154. Lynch, M. The Origins of Genome Architecture (Sinauer Associates, 2007).

  155. Lynch, M. & O’Hely, M. Captive breeding and the genetic fitness of natural populations. Conserv. Genet. 2, 363–378 (2001).

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank E. Anderson, A. Leaché, M. Kardos and the reviewers for their helpful comments that greatly improved this manuscript. The authors also thank M. Exposito-Alonso and the 1001 Genomes Consortium, the 1000 Genomes Project, B. Hand, M. Freedman, M. Kardos, C. Kessler, M. Lynch, R. Malison, D. Martchenko, M. Miller, R. Schweizer, A.B.A. Shafer and X. Yin for allowing their datasets to be reviewed and re-filtered. M.R.C. was funded, in part, by NSF DEB-1856710 and OCE-1924505. G.L. was funded, in part, by NSF-DOB-M66230.

Author information

Author notes

  1. These authors contributed equally: William Hemstrom, Jared A. Grummer.

Authors and Affiliations

  1. Department of Biological Sciences, Purdue University, West Lafayette, IN, USA

    William Hemstrom & Mark R. Christie

  2. Flathead Lake Biological Station, Wildlife Biology Program and Division of Biological Sciences, University of Montana, Missoula, MT, USA

    Jared A. Grummer & Gordon Luikart

  3. Department of Forestry and Natural Resources, Purdue University, West Lafayette, IN, USA

    Mark R. Christie

Authors
  1. William Hemstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jared A. Grummer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gordon Luikart
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark R. Christie
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors conceptualized, wrote and edited the manuscript. W.H. and J.A.G. conducted the simulations and analyses in Box 2.

Corresponding authors

Correspondence to William Hemstrom or Mark R. Christie.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Genetics thanks Mark Ravinet and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

DNA DataBank of Japan Sequence Read Archive: https://www.ddbj.nig.ac.jp/dra/index-e.html

European Variation Archive: https://www.ebi.ac.uk/eva/

GATK: https://gatk.broadinstitute.org/hc/en-us

NCBI Short-Read Archive: https://www.ncbi.nlm.nih.gov/sra

Supplementary information

Glossary

Alignment

The mapping of sequencing reads and/or contigs to either each other (pairwise/multiple alignment) or to a reference. Alignments can vary in the strength of the evidence that supports them. Most alignment tools will return map quality (mapQ) scores, the derivation and meaning of which varies by program. Filtering thresholds based on this score must consider the specific aligner used.

Base quality score

The value in a logarithmic, Phred scale given to each base on a sequencing read that indicates a quantitative degree of confidence in the nucleotide called from the sequencing instrument.

Contigs

Contiguous sequences of DNA assembled from many overlapping sequence reads, representing a fragment of a chromosome.

De novo assembly

The reference-free alignment of sequencing reads into overlapping stacks or contigs for subsequent use in variant discovery and genotyping.

F IS

A measure of inbreeding; the degree of subpopulation divergence from Hardy–Weinberg proportions — the correlation between alleles at specific loci within individuals relative to the subpopulation.

F ST

A measure of population differentiation; the proportion of the total genetic variance due to differences in allele frequencies between subpopulations.

Genetic variants

Differences in DNA sequence compared with a reference sequence or other individuals within a population. The term includes short variants (single-nucleotide polymorphisms (SNPs) or insertions and deletions) and structural variants (chromosomal inversions and copy number variations (CNVs)). In the context of this Review, used interchangeably with ‘locus’.

Genome-wide association studies

(GWAS). Tests for statistical relationships between a phenotype (including disease) and the allelic/genotypic state of an (ideally) large cohort of individuals across the entire set of sequenced loci.

Genotyping

Also referred to as genotype or variant calling. Calling allelic states at a locus (for example, A/A, A/C or C/C at a biallelic single-nucleotide polymorphism (SNP) in a diploid organism) or loci from sequence data. Genotyping algorithms often consist of multiple steps during which filtering can occur.

Haplotype phase

The complete sequence of variants that occur in a region along a single chromatid.

Hardy–Weinberg proportions

(HWP). The expected frequencies of the genotypes at a given locus under Hardy–Weinberg equilibrium. Filtering on HWP is often executed via an exact test, with loci that deviate significantly from HWP removed from subsequent analyses.

Imputation

The filling in of missing data for specific genotypes and/or loci by leveraging linkage disequilibrium (LD) between missing genotypes and genotypes called at other loci or samples. Imputation can use reference panels of well-described haplotypes to improve performance when available, usually in well-studied model organisms.

Linkage disequilibrium

(LD). The non-random association of alleles at different loci within a population or sample-group. This association can either be caused by physical linkage, when alleles are co-inherited due to non-independent assortment caused by close physical proximity, or occur across chromosomes when inbreeding, paralogy, genetic drift or other factors make certain alleles at different loci more likely to co-occur.

Low-coverage whole-genome sequencing

Whole-genome sequencing (WGS) with small numbers of reads covering most genomic loci (low coverage); the number of reads constituting low coverage varies widely depending on the discipline, methodology and research question. Low-coverage WGS often requires genotype likelihood-based methods.

Mapping quality

The score given to a read or other DNA sequence indicating the uniqueness of the alignment to a reference sequence; mapping quality score interpretations vary across alignment programs.

Minor allele count

(MAC). The number of gene copies or individuals carrying the minor (that is, least frequent) allele at a locus.

Minor allele frequency

(MAF). The proportion (frequency) of the least common allele at a locus across a study or sample-group; in this Review, we refer to filtering out loci with MAFs below a given threshold as MAF filtering.

Missing data

Missing genotype calls at a specific locus or individual. Missing data can be caused by many factors, such as the absence of a sufficient number of reads covering a locus to call a genotype in an individual with any degree of confidence.

N50 or L50 scores

In a genome assembly after sorting contigs or scaffolds by length, either the length of the contig/scaffold that reaches 50% of the cumulative genome length (N50) or the number of contigs needed to reach 50% of the cumulative genome length (L50); used to evaluate the assembly quality.

Paralogues

Duplicated genomic regions that have arisen via either the duplication of that specific region or the duplication of the entire genome. A type of homologue (loci identical by descent) distinct from orthologues, which arise due to speciation events.

PCR duplicates

Technical duplicates resulting in spurious, usually identical read copies caused by repeatedly sequencing the same piece of template DNA multiple times.

Population structure

Also known as population subdivision. Non-independence among individuals in a study area/region caused by spatial, temporal, behavioural or other forms of reproductive isolation. Population structure is characterized by divergent allele frequencies across loci.

Read depth

The number of reads that cover a given or fixed genomic position. Also referred to as ‘coverage’.

Reference bias

The propensity for reads containing the non-reference allele (the allele not in the reference genome) to have lower mapping quality scores or map to the wrong location compared with those containing the allele present in the reference genome.

Runs of homozygosity

Contiguous homozygous regions of the genome caused by the inheritance of identical haplotypes from both parents (for example, identical by descent). Useful for estimating inbreeding and population demographics.

Sample-group

A group of samples that are not independent due to natural causes (such as geographic or temporal separation) and/or experimental treatments.

Single-nucleotide polymorphisms

(SNPs). Genetic variants where the allelic state of the population varies at a single base pair.

Singletons

Alleles that appear only once in a sample of individuals. Sometimes alternatively defined as an allele sequenced in only one individual (which may be homozygous for that allele).

Site-frequency spectra

(SFS). The distributions of allele frequencies across loci within a study or sample-group. Can be either an ‘unfolded’ or ‘polarized’ derived allele frequency spectrum which describes the frequency distribution of derived alleles or a ‘folded’ or ‘unpolarized’ minor allele frequency (MAF) spectrum which describes the frequency distribution of the minor alleles. Also known as the allele frequency distribution.

Structural variation

Genetic variation in the order, number and/or arrangement of loci.

Study-wide filtering

Applying a filtering threshold ‘globally’ (simultaneously across all samples in the entire dataset) rather than separately within each sample-group.

VCF file

A file in the variant call format, which contains genotype calls (or likelihoods, posteriors) alongside a flexible suite of metadata such as filtering and processing history and quality information.

Wahlund effect

A reduction in observed heterozygosity (HO) relative to the expected heterozygosity (He) under Hardy–Weinberg proportions (HWP) (that is, HO < He) at many/most loci caused by the underlying population structure. When multiple (sub)populations are included in a sample, any differences in allele frequency between (sub)populations will cause there to be considerably more homozygous individuals at those loci than would be expected under HWP (causing an elevated FIS, the fixation index in individuals relative to a subpopulation).

Within-group filtering

Applying a filtering threshold within each sample-group separately rather than across all individuals simultaneously (for example, study wide or globally).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hemstrom, W., Grummer, J.A., Luikart, G. et al. Next-generation data filtering in the genomics era. Nat Rev Genet (2024). https://doi.org/10.1038/s41576-024-00738-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41576-024-00738-6

Search

Advanced search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research
-