Kernel Matching versus Inverse Probability Weighting: A Comparative Study

Authors: Andy Handouyahia, Tony Haddad, Frank Eaton

Abstract:

Recent quasi-experimental evaluation of the Canadian Active Labour Market Policies (ALMP) by Human Resources and Skills Development Canada (HRSDC) has provided an opportunity to examine alternative methods to estimating the incremental effects of Employment Benefits and Support Measures (EBSMs) on program participants. The focus of this paper is to assess the efficiency and robustness of inverse probability weighting (IPW) relative to kernel matching (KM) in the estimation of program effects. To accomplish this objective, the authors compare pairs of 1,080 estimates, along with their associated standard errors, to assess which type of estimate is generally more efficient and robust. In the interest of practicality, the authorsalso document the computationaltime it took to produce the IPW and KM estimates, respectively.

Keywords: Treatment effect, causal inference, observational studies, Propensity score based matching, Kernel Matching, Inverse Probability Weighting, Estimation methods for incremental effect.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1086597

References:

[1] Huber, Martin, Michael Lechner, and ConnyWunsch, October 2010, “How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score”, Institute for the Study of Labor, Discussion Paper IZA DP No. 5268, Bonn, Germany.
[2] Ibid.
[3] Robins, J. M., S. D. Mark, and W. K. Newey (1992): "Estimating Exposure Effects by Modelling the Expectation of Exposure Conditional on Confounders", Biometrics, 48, 479-495.
[4] Horvitz, D., and D. Thompson (1952): "A Generalization of Sampling Without Replacement from a Finite Population", Journal of the American Statistical Association, 47, 663-685.
[5] Rubin, D. B. (1974): "Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies", Journal of Educational Psychology, 66, 688-701.
[6] Rosenbaum, P. R., and D. B. Rubin (1983): "The Central Role of the Propensity Score in Observational Studies for Causal Effects", Biometrika, 70, 41-55.
[7] Heckman, J. J., H. Ichimura, and P. Todd (1998): "Matching as an Econometric Evaluation Estimator", Review of Economic Studies, 65, 261-294.
[8] Busso, Matias, John DiNardo, and Justin McCrary (2008), “Finite Sample Properties of Semiparametric Estimators of Average Treatment Effects”, University of Michigan.
[9] Heckman, James, Hidehiko Ichimura, Jeffrey Smith, and Petra Todd. (1996). “Sources of Selection Bias in Evaluating Social Programs: An Interpretation of Conventional Measures and Evidence on the Effectiveness of Matching as a Program Evaluation Method”, Proceedings of the National Academy of Sciences. 93(23): 13416- 13420.
[10] Heckman, James, Robert LaLonde, and Jeffrey Smith. (1999). “The Economics and Econometrics of Active Labor Market Programs” in OrleyAshenfelter and David Card (eds.), Handbook of Labor Economics, Volume 3A. Amsterdam: North-Holland, 1865-2097.
[11] Smith, Jeffrey. (2000). “Evaluating Active Labor Market Policies: Lessons from North America,” in MittAB-Schwerpunktheft 2000: Evaluation aktiver Arbeitsmarktpolitik, Nuremberg: IAB, 345-356.
[12] Smith, Jeffrey and Petra Todd. (2005). "Does Matching Overcome LaLonde's Critique of Nonexperimental Estimators?" Journal of Econometrics 125(1-2):305-353.
[13] Galdo, Jose, Jeffrey Smith and Dan Black. (2008). "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data." Annales d'Economie et Statistique, forthcoming.
[14] Fitzenberger, Bernd, Michael Lechner and Jeffrey Smith. (2013). "Evaluation of Treatment Effects: Recent Developments and Applications." Empirical Economics, forthcoming.
[15] Keisuke Hirano and Guido W. Imbens, titled “Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization” and published in 2002 in Health Services & Outcomes Research Methodology, by Kluwer Academic Publishers, the Netherlands.
[16] Martin Huber & Michael Lechner &ConnyWunsch, 2010. "How to control for many covariates? Reliable estimators based on the propensity score," University of St. Gallen Department of Economics working paper series 2010 2010-30, Department of Economics, University of St. Gallen.
[17] Rosenbaum, P. R. and D. B. Rubin. (1985) “Constructing a control group using multivariate matched sampling methods that incorporate the propensity score.” The American Statistician 39, 33–38.
[18] Sianesi, B. (2004): “An Evaluation of the Active Labour Market Programmes in Sweden,” The Review of Economics and Statistics, 86(1), 133–155.
[19] Smith, J. A., and P. Todd (2005a): “Does Matching Overcome LaLonde’s Critique of Nonexperimental Estimators?,” Journal of Econometrics, 125(1/2), 305–354.
[20] Smith, J. A. and P. Todd (2005b): “Does Matching Overcome LaLonde’s Critique of Nonexperimental Estimators? Rejoinder,” Journal of Econometrics, 125(1/2), 365–375.