External validation of a measurement tool to assess systematic reviews (AMSTAR)
- PMID: 18159233
- PMCID: PMC2131785
- DOI: 10.1371/journal.pone.0001350
External validation of a measurement tool to assess systematic reviews (AMSTAR)
Abstract
Background: Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.
Methodology: AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.
Conclusions: The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
Conflict of interest statement
Figures
Similar articles
-
Assessment of systematic reviews and meta-analyses available for bovine and equine veterinarians and quality of abstract reporting: A scoping review.Prev Vet Med. 2018 Dec 1;161:50-59. doi: 10.1016/j.prevetmed.2018.10.011. Epub 2018 Oct 23. Prev Vet Med. 2018. PMID: 30466658 Review.
-
Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR.BMC Med Res Methodol. 2018 May 8;18(1):37. doi: 10.1186/s12874-018-0493-y. BMC Med Res Methodol. 2018. PMID: 29739339 Free PMC article.
-
[Assessment of reliability and validity of assessment of multiple systematic reviews in Chinese systematic reviews on stomatology].Hua Xi Kou Qiang Yi Xue Za Zhi. 2013 Feb;31(1):49-52. Hua Xi Kou Qiang Yi Xue Za Zhi. 2013. PMID: 23484302 Chinese.
-
Quality assessment of systematic reviews on periodontal regeneration in humans.J Periodontol. 2013 Feb;84(2):176-85. doi: 10.1902/jop.2012.120021. Epub 2012 Apr 17. J Periodontol. 2013. PMID: 22509753 Review.
-
AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.J Clin Epidemiol. 2009 Oct;62(10):1013-20. doi: 10.1016/j.jclinepi.2008.10.009. Epub 2009 Feb 20. J Clin Epidemiol. 2009. PMID: 19230606
Cited by
-
Analysis of clinical evidence on traditional Chinese medicine for the treatment of diabetic nephropathy: a comprehensive review with evidence mapping.Front Endocrinol (Lausanne). 2024 Mar 27;15:1324782. doi: 10.3389/fendo.2024.1324782. eCollection 2024. Front Endocrinol (Lausanne). 2024. PMID: 38601203 Free PMC article. Review.
-
The relationship between nonsteroidal anti-inflammatory drugs and cancer incidence: An umbrella review.Heliyon. 2024 Jan 12;10(2):e23203. doi: 10.1016/j.heliyon.2023.e23203. eCollection 2024 Jan 30. Heliyon. 2024. PMID: 38312641 Free PMC article. Review.
-
Effectiveness and safety of awake prone positioning in COVID-19-related acute hypoxaemic respiratory failure: an overview of systematic reviews.BMC Pulm Med. 2024 Jan 2;24(1):5. doi: 10.1186/s12890-023-02829-2. BMC Pulm Med. 2024. PMID: 38166818 Free PMC article. Review.
-
Methodological quality and reporting quality of COVID-19 living systematic review: a cross-sectional study.BMC Med Res Methodol. 2023 Jul 31;23(1):175. doi: 10.1186/s12874-023-01980-y. BMC Med Res Methodol. 2023. PMID: 37525117 Free PMC article.
-
Effect of sugar-sweetened beverage taxation on sugars intake and dental caries: an umbrella review of a global perspective.BMC Public Health. 2023 May 27;23(1):986. doi: 10.1186/s12889-023-15884-5. BMC Public Health. 2023. PMID: 37237341 Free PMC article. Review.
References
-
- Young D. Policymakers, experts review evidence-based medicine. Am. J. Health Syst Pharm. 2005;62(4):342–343. - PubMed
-
- Dolan-Mullen P, Ramírez G. The Promise and Pitfalls of Systematic Reviews. Annual Review of Public Health. 2006;27:81–102. - PubMed
-
- Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44(11):1271–78. - PubMed
-
- Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16(1):62–73. - PubMed
-
- Shea B, Dube C, Moher D. Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. Systematic review in health care meta-analysis in context. London: BMJ Books. 2001;(7):122–39.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources