Quantifying and Adjusting the Effects of Publication Bias in Continuous Meta-Analysis

Authors: N.R.N. Idris

Abstract:

This study uses simulated meta-analysis to assess the effects of publication bias on meta-analysis estimates and to evaluate the efficacy of the trim and fill method in adjusting for these biases. The estimated effect sizes and the standard error were evaluated in terms of the statistical bias and the coverage probability. The results demonstrate that if publication bias is not adjusted it could lead to up to 40% bias in the treatment effect estimates. Utilization of the trim and fill method could reduce the bias in the overall estimate by more than half. The method is optimum in presence of moderate underlying bias but has minimal effects in presence of low and severe publication bias. Additionally, the trim and fill method improves the coverage probability by more than half when subjected to the same level of publication bias as those of the unadjusted data. The method however tends to produce false positive results and will incorrectly adjust the data for publication bias up to 45 % of the time. Nonetheless, the bias introduced into the estimates due to this adjustment is minimal

Keywords: Publication bias, Trim and Fill method, percentage relative bias, coverage probability

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1081721

References:

[1] C.B.Begg and J.A. Berlin, "Publication bias: a problem in interpreting medical data" J. R. Stat. Soc. Series A, vol 151, pp. 419-463, 1988.
[2] L.J. Peters, A.J. Sutton , D.R. Jones, K.R. Abrams, L. Rushton, "Comparison of two methods to detect publication bias in metaanalysis", JAMA, vol 295, pp. 676-680, 2006.
[3] M. Egger., S.G. Davey, M. Schneider, C. Mider, "Bias in meta-analyses detected by a simple graphical test", BMJ, vol 315, pp. 629-634, 1997.
[4] S.J. Duval and R.L. Tweedie. "A non-parametric "trim and fill" method of accounting for publication bias in meta-analysis", J.Amer Statist Assoc, vol 95 , pp.89-98, 2000a.
[5] S.J. Duval and R.L. Tweedie, "Trim and Fill : A simple funnel- plotbased method of testing and adjusting for publication bias in metaanalysis", Biometric, vol 56, pp. 455-463, 2000b.
[6] K.Dickersin, K., 2005. "Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm". In H. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in Meta Analysis: Prevention, Assessment and Adjustments, pp. 11-34,. Chichester, UK: Wiley, 2005.
[7] A.J. Sutton, S.J. Duvall, R.L. Tweedie, K.R Abrams., D.R. Jones, "Empirical assessment of effect of publication bias on meta analyses" BMJ, vol 320, pp. 1574-1576, 2000.
[8] M.D. Jennions and A.P. Moller, "Publication bias in ecology and evolution: an empirical assessment using the trim and fill method", Biological Reviews, vol 77, pp. 211-222, 2002.
[9] M.Borenstein, L.V.Hedges, P.T.Higgins, H.R.Rothstein,"Introduction to meta-analysi", John Wiley&sons, London ,2009.
[10] N.R. Idris, "Estimating meta analysis parameters in non standard data". PhD Thesis. Glasgow: University of Strathclyde, 1996.
[11] P. Macaskill, S.D. Walter and L. Irwig, "A comparison of methods to detect publication bias in meta analysis", Statist. Med., vol 20, pp. 641- 654, 2001.
[12] C.B.Begg and Mazumdar, M., 1994. "Operating characteristics of a rank correlation test for publication bias", Biometrics, vol 50, pp.1088-1101, 1994.
[13] J.A.C. Sterne, D. Gavaghan, M. Egger, "Publication and related bias in meta analysis", J.ournal of Clinical Epidemiology, vol 53, pp. 1119- 1129, 2000.
[14] N. Terrin, C. H.Schmid, J. Lau, "An empirical evaluation of the funnel plot, researchers could not visually identify publication bias", J. Clin. Epidemiol, vol 58, pp. 894-901, 2003.