Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist

Objective: To assess the conditions under which employing an overview of systematic reviews is likely to lead to a high risk of bias. Study Design: To synthesise existing guidance concerning overview practice, a scoping review was conducted. Four electronic databases were searched with a pre-specified strategy (PROSPERO 2015:CRD42015027592) ending October 2015. Included studies needed to describe or develop overview methodology. Data were narratively synthesised to delineate areas highlighted as outstanding challenges or where methodological recommendations conflict. Results: Twenty-four papers met the inclusion criteria. There is emerging debate regarding overlapping systematic reviews; systematic review scope; quality of included research; updating; and synthesizing and reporting results. While three functions for overviews have been proposed—identify gaps, explore heterogeneity, summarize evidence—overviews cannot perform the first; are unlikely to achieve the second and third simultaneously; and can only perform the third under specific circumstances. Namely, when identified systematic reviews meet the following four conditions: (1) include primary trials that do not substantially overlap, (2) match overview scope, (3) are of high methodological quality, and (4) are up-to-date. Conclusion: Considering the intended function of proposed overviews with the corresponding methodological conditions may improve the quality of this burgeoning publication type.