Evaluation of meta-analyses in the otolaryngological literature

Objective: To evaluate the quality of meta-analyses written on otolaryngological topics and define areas that can be improved upon in future studies. Data Sources: MEDLINE (PubMed) and EMBASE databases were searched. The Cochrane database of systematic reviews was excluded, because these meta-analyses have already been critically evaluated and found to be of high quality. Review Methods: A systematic review of otolaryngological meta-analyses published between 1997 and 2006 (10 years) was performed in duplicate and independently by two authors. The search included 16 common otolaryngological terms. Inclusion criteria were meta-analytic methodology, otolaryngological topic, and at least one author from a department of otolaryngology. Fifty-one articles fulfilled eligibility criteria. In duplicate and independently, two reviewers assessed the quality of eligible meta-analyses using a validated 10-item index called the Overview Quality Assessment Questionnaire. Using the methods of Spearman, correlation coefficients are reported for associations examined between covariates and the Overall Score Quality. Results: The majority of studies had methodologic flaws (mean score 3.9, scale of 1-7). Variables predicting higher-quality meta-analyses were publication in journals with higher impact factors (P = 0.0007) and authors who previously published meta-analyses (P = 0.0001). Using and reporting about a validity assessment tool needs to be improved upon in future studies. Conclusion: The quality of meta-analyses on otolaryngological topics is moderate. Future meta-analyses can be improved upon by following evidence-based guidelines for the reporting of meta-analyses, which include the use of a validity assessment tool, and consulting with an author familiar with meta-analysis methodology.