Comparison of Methods of Estimation for Use in Goodness of Fit Tests for Binary Multilevel Models

Authors: I. V. Pinto, M. R. Sooriyarachchi

Abstract:

It can be frequently observed that the data arising in our environment have a hierarchical or a nested structure attached with the data. Multilevel modelling is a modern approach to handle this kind of data. When multilevel modelling is combined with a binary response, the estimation methods get complex in nature and the usual techniques are derived from quasi-likelihood method. The estimation methods which are compared in this study are, marginal quasi-likelihood (order 1 & order 2) (MQL1, MQL2) and penalized quasi-likelihood (order 1 & order 2) (PQL1, PQL2). A statistical model is of no use if it does not reflect the given dataset. Therefore, checking the adequacy of the fitted model through a goodness-of-fit (GOF) test is an essential stage in any modelling procedure. However, prior to usage, it is also equally important to confirm that the GOF test performs well and is suitable for the given model. This study assesses the suitability of the GOF test developed for binary response multilevel models with respect to the method used in model estimation. An extensive set of simulations was conducted using MLwiN (v 2.19) with varying number of clusters, cluster sizes and intra cluster correlations. The test maintained the desirable Type-I error for models estimated using PQL2 and it failed for almost all the combinations of MQL. Power of the test was adequate for most of the combinations in all estimation methods except MQL1. Moreover, models were fitted using the four methods to a real-life dataset and performance of the test was compared for each model.

Keywords: Goodness-of-fit test, marginal quasi-likelihood, multilevel modelling, type-I error, penalized quasi-likelihood, power, quasi-likelihood.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.2643816

References:

[1] G. Rodriguez and N. Goldman, "An assessment of estimation procedures for multilevel models with binary responses," Journal of the Royal Statistical Society, vol. 158, no. 1, pp. 73-89, 1995.
[2] J. J. Hox, Multilevel analysis- Techiniques and applications, New York: Routledge, 2010.
[3] F. L. Huang, "Alternatives to multilevel modeling for the analysis of clustered data," The Journal of Experimental Education, vol. 84, no. 1, pp. 175-196, 2016.
[4] J. Rasbash, F. Steele, W. Browne and H. Goldstein, A user’s guide to MLwiN, Version 3.00, Centre for Multilevel Modelling, University of Bristol, 2017.
[5] H. Goldstein and J. Rasbash, "Improved approximations for mulitlevel models with binary respones," Journal of the royal statistical society, vol. 159, no. 3, pp. 505-513, 1996.
[6] D. Hosmer and S. Lemeshow, "Goodness of fit tests for the multiple logistic regression model," Communications in Statistics - Theory and Methods, 1980.
[7] S. Lipsitz, G. Fitzmaurice and G. Molenberghs, "Goodness-of-fit tests for ordinal response regression models," Journal of the Royal Statistical Society. Series C (Applied Statistics), vol. 45, no. 2, pp. 175-190, 1996.
[8] A. Perera, M. Sooriyarachchi and Wickramasuriya, "A goodness of fit test for the multilevel logistic model," Communications in statistics: Simulation and computation, vol. 45, no. 2, pp. 643-659, 2016.
[9] K. Archer, S. Lemeshow and D. Hosmer, "Goodness-of-fit tests for logistic regression models when data are collected using a complex sampling design," Computational statistics and data analysis, 2007.
[10] S. A. Knox and P. Chondros, "Observed intra-cluster correlation coefficients in a cluster survey sample of patient encounters in general practice in Australia," BMC Medical Research Methodology, 2004.
[11] C. Maas and J. Hox, "Sufficient sample sizes for multilevel modeling," 2005.
[12] I. G. Kreft and J. de Leeuw, Introducing multilevel modeling, Newbury Park, CA:: Sage, 1998.
[13] N. M. Huq and J. Cleland, "Bangladesh fertility survery, 1989," National Institute of Population Research and Training (NIPORT)., Dhaka, 1990.
[14] W. M. Abeysekera and R. Sooriyarachchi, "A novel method for testing goodness of fit of a proportional odds model: An application to AIDS study," Journal of National Science Foundation Sri Lanka, vol. 36, no. 2, pp. 125-135, 2008.
[15] J. Cohen, Statistical power analysis for the behavioral sciences, 2 ed., New York: Lawrence Erlbaum Associates, Publishers, 1988.
[16] J. A. Schoeneberger, "The impact of sample size and other factors when estimating multilevel logistic models," The Journal of Experimental Education, vol. 84, no. 2, pp. 373-397, 2016.