A Sequential Approach to Random-Effects Meta-Analysis

Authors: Samson Henry Dogo, Allan Clark, Elena Kulinskaya

Abstract:

The objective of meta-analysis is to combine results from several independent studies in order to create generalization and provide evidence base for decision making. But recent studies show that the magnitude of effect size estimates reported in many areas of research significantly changed over time and this can impair the results and conclusions of meta-analysis. A number of sequential methods have been proposed for monitoring the effect size estimates in meta-analysis. However they are based on statistical theory applicable only to fixed effect model (FEM) of meta-analysis. For random-effects model (REM), the analysis incorporates the heterogeneity variance, τ 2 and its estimation create complications. In this paper we study the use of a truncated CUSUM-type test with asymptotically valid critical values for sequential monitoring in REM. Simulation results show that the test does not control the Type I error well, and is not recommended. Further work required to derive an appropriate test in this important area of applications.

Keywords: Meta-analysis, random-effects model, sequential testing, temporal changes in effect sizes.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1337807

References:

[1] Casper W Bollen, Cuno SPM Uiterwaal, Adrianus J van Vught, and Ingeborg van der Tweel. Sequential meta-analysis of past clinical trials to determine the use of a new trial. Epidemiology, 17(6):644–649, 2006.
[2] S. Brugger, J.M. Davis, S. Leucht, and J.M. Stone. Proton magnetic resonance spectroscopy and illness stage in schizophreniaa systematic review and meta-analysis. Biological psychiatry, 69(5):495–503, 2011.
[3] Rebecca DerSimonian and Raghu Kacker. Random-effects model for meta-analysis of clinical trials: an update. Contemporary clinical trials, 28(2):105–114, 2007.
[4] Rebecca. DerSimonian and Nan. Laird. Meta-analysis in clinical trials. Controlled clinical trials, 7(3):177–188, 1986.
[5] Lynn Friedman. Estimators of random effects variance components in meta-analysis. Journal of Educational and Behavioral Statistics, 25(1):1–12, 2000.
[6] B. Gehr, C. Weiss, and F. Porzsolt. The fading of reported effectiveness. a meta-analysis of randomised controlled trials. BMC medical research methodology, 6(1):25, 2006.
[7] E Gombay. Sequential change-point detection and estimation. Sequential Analysis, 22(3):203–222, 2003.
[8] E. Gombay and D. Serbian. An adaptation of Pages CUSUM test for change detection. Periodica Mathematica Hungarica, 50(1):135–147, 2005.
[9] Shelly Grabe, L Monique Ward, and Janet Shibley Hyde. The role of the media in body image concerns among women: a meta-analysis of experimental and correlational studies. Psychological bulletin, 134(3):460, 2008
[10] Larry V Hedges and Jack L Vevea. Fixed-and random-effects models in meta-analysis. Psychological methods, 3(4):486, 1998.
[11] Julian Higgins, Anne Whitehead, and Mark Simmonds. Sequential methods for random-effects meta-analysis. Statistics in medicine, 30(9):903–921, 2011.
[12] M. J. Hodgson, D. K. Parkinson, and M. Karpf. Chest x-ray in hypersensitivity pneumonities: A meta analysis of secular trends. American journal of industrial medicine, 16(1):45–53, 1989.
[13] Mingxiu Hu, Joseph C Cappelleri, and KK Gordon Lan. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clinical Trials, 4(4):329–340, 2007.
[14] J.S. Hyde, E. Fennema, and S.J. Lamon. Gender differences in mathematics performance: a meta-analysis. Psychological bulletin, 107(2):139, 1990.
[15] John Ioannidis and Thomas A Trikalinos. Early extreme contradictory estimates may appear in published research: The proteus phenomenon in molecular genetics research and randomized trials. Journal of clinical epidemiology, 58(6):543–549, 2005.
[16] E. Kulinskaya and J. Koricheva. Use of quality control charts for detection of outliers and temporal trends in cumulative meta-analysis. Research Synthesis Methods, 2010.
[17] Elena Kulinskaya and John Wood. Trial sequential methods for meta-analysis. Research Synthesis Methods, 2013.
[18] Sofie Kuppens and Patrick Onghena. Sequential meta-analysis to determine the sufficiency of cumulative knowledge: The case of early intensive behavioral intervention for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 6(1):168–176, 2012.
[19] KK Gordon Lan, Mingxiu Hu, and Joseph C Cappelleri. Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Statistica Sinica, 13(4):1135–1146, 2003.
[20] J. Lau, E.M. Antman, J. Jimenez-Silva, B. Kupelnick, F. Mosteller, and T.C. Chalmers. Cumulative meta-analysis of therapeutic trials for myocardial infarction. New England Journal of Medicine, 327(4):248–254, 1992.
[21] R. Leimu and J. Koricheva. Cumulative meta–analysis: a new tool for detection of temporal trends and publication bias in ecology. Proceedings of the Royal Society of London. Series B: Biological Sciences, 271(1551):1961–1966, 2004.
[22] Dennis J Nieuwkamp, Larissa E Setz, Ale Algra, Francisca HH Linn, Nicolien K de Rooij, and Gabri¨el JE Rinkel. Changes in case fatality of aneurysmal subarachnoid haemorrhage over time, according to age, sex, and region: a meta-analysis. The Lancet Neurology, 8(7):635–642, 2009.
[23] Robert C. Paule and John Mandel. Consensus values and weighting factors. Journal of Research of the National Bureau of Standards, 87(5):377–385, 1982.
[24] J.M. Pogue and S. Yusuf. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled clinical trials, 18(6):580–593, 1997.
[25] Jean M. Twenge., Sara. Konrath, Joshua D. Foster., W Keith Campbell., and Brad J. Bushman. Egos inflating over time: A cross-temporal meta-analysis of the narcissistic personality inventory. Journal of personality, 76(4):875–902, 2008.
[26] Wolfgang Viechtbauer. Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in medicine, 26(1):37–52, 2007.
[27] J. Wetterslev, K. Thorlund, J. Brok, and C. Gluud. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology, 61(1):64–75, 2008.
[28] Anne Whitehead. A prospectively planned cumulative meta-analysis applied to a series of concurrent clinical trials. Statistics in medicine, 16(24):2901–2913, 1997.
[29] John Whitehead. The design and analysis of sequential clinical trials. John Wiley & Sons, 1997.