A Comparison of the Sum of Squares in Linear and Partial Linear Regression Models

Authors: Dursun Aydın

Abstract:

In this paper, estimation of the linear regression model is made by ordinary least squares method and the partially linear regression model is estimated by penalized least squares method using smoothing spline. Then, it is investigated that differences and similarity in the sum of squares related for linear regression and partial linear regression models (semi-parametric regression models). It is denoted that the sum of squares in linear regression is reduced to sum of squares in partial linear regression models. Furthermore, we indicated that various sums of squares in the linear regression are similar to different deviance statements in partial linear regression. In addition to, coefficient of the determination derived in linear regression model is easily generalized to coefficient of the determination of the partial linear regression model. For this aim, it is made two different applications. A simulated and a real data set are considered to prove the claim mentioned here. In this way, this study is supported with a simulation and a real data example.

Keywords: Partial Linear Regression Model, Linear RegressionModel, Residuals, Deviance, Smoothing Spline.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1073471

References:

[1] Mayers, Raymond. H., Classical and Modern Regression with Applications, Duxbury Classical Series, United States, 1990.
[2] Montgomarey, C. Douglas., Peck, A. Elizabeth., Vining, G. Geoffrey., Introduction to Linear Regression Analysis, John Wiley&Sons,Inc., Toronto, 2001.
[3] Hardle, Wolfang., M├╝ller, Marlene., Sperlich, Stefan., Weratz, Axel., Nonparametric and Semiparametric Models, Springer, Berlin, 2004.
[4] Eubank, R. L., Nonparametric Regression and Smoothing Spline, Marcel Dekker Inc., 1999.
[5] Wahba, G., Spline Model for Observational Data, Siam, Philadelphia Pa., 1990.
[6] Green, P.J. and Silverman, B.W., Nonparametric Regression and Generalized Linear Models, Chapman & Hall, 1994.
[7] Schimek, G. Michael, Estimation and Inference in Partially Linear Models with Smoothing Splines, Journal of Statistical Planning and Inference, 91, 525-540, 2000.
[8] Hastie, T.J. and Tibshirani, R.J., Generalized Additive Models, Chapman & Hall /CRC, 1999.
[9] Wood, N. Simon., Generalized Additive Models An Introduction With R, Chapman & Hall/CRC, New York, 2006.
[10] Hastie, T., The gam Package, Generalized Additive Models, R topic documented, http://cran.r.project.org/packages/gam.pdf, February 16, 2008.