Use of quality assessment tools within systematic reviews in orthodontics during the last decade: looking for a threshold?

Abstract

Objective: To record the prevalence and extent of use of quality assessment/ risk of bias tools in orthodontic systematic reviews and to identify whether systematic reviews authors stipulated a threshold during the evaluation process of the primary studies included in systematic reviews, published across the previous decade and until now. Associations with publication characteristics including the journal of publication, year, the inclusion of a meta-analysis, design of primary studies and others, were sought.

Materials and methods: Electronic search within 6 orthodontic journals and the Cochrane Database of Systematic Reviews was conducted to identify relevant systematic reviews from 1 January 2010 and 31 December 2020. The outcomes of interest pertained to the use, type and extent of quality appraisal/ risk of bias tools utilized as a standard process within the systematic reviews, and also whether a threshold had been stipulated by the systematic reviews authors. Predictor variables included journal, year of publication, geographic region, number of authors, involvement of a methodologist, type of systematic reviews, inclusion of meta-analysis, type/design of primary studies.

Results: A total of 262 systematic reviews were eligible for inclusion, with 41 quality appraisal/ risk of bias sets of tools being described either jointly or in isolation. One-third of the systematic reviews of the present sample (88/262; 33.6%) included a threshold, while this was mostly represented by the stipulation of sensitivity analyses in this respect (64/88; 72.8%). Journal of publication (non-Cochrane systematic reviews versus Cochrane systematic reviews: adjusted odds ratio, OR: 0.04, 95%CI: 0.01, 0.16; P < 0.001) and inclusion of a meta-analysis (adjusted OR: 8.76; 95%CI: 4.18, 18.37; P < 0.001), were identified as significant predictors for preplanning of thresholds.

Conclusions and implications: Quality assessment tools for primary studies are largely used and varied in orthodontic systematic reviews, while a threshold-level has been stipulated in only one third. Additional efforts should be endorsed by the scientific community, to embrace more straightforward adoption of the most rigorous reporting guidelines in this respect.