Evaluation of Model Evaluation Criterion for Software Development Effort Estimation

Authors: S. K. Pillai, M. K. Jeyakumar

Abstract:

Estimation of model parameters is necessary to predict the behavior of a system. Model parameters are estimated using optimization criteria. Most algorithms use historical data to estimate model parameters. The known target values (actual) and the output produced by the model are compared. The differences between the two form the basis to estimate the parameters. In order to compare different models developed using the same data different criteria are used. The data obtained for short scale projects are used here. We consider software effort estimation problem using radial basis function network. The accuracy comparison is made using various existing criteria for one and two predictors. Then, we propose a new criterion based on linear least squares for evaluation and compared the results of one and two predictors. We have considered another data set and evaluated prediction accuracy using the new criterion. The new criterion is easy to comprehend compared to single statistic. Although software effort estimation is considered, this method is applicable for any modeling and prediction.

Keywords: Software effort estimation, accuracy, Radial Basis Function, linear least squares.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1099563

References:

[1] M. Jørgensen, “A Review of Studies on Expert Estimation of Software Development Effort,”The Journal of Systems and Software, vol. 70, pp.37–60, 2004.
[2] M.J. Shepperd, C. Schofield, “ Estimating Software Project Effort Using Analogies,” IEEE Trans. Software Eng., vol. 23, pp. 736-743,1997.
[3] B. Boehm, E. Horowitz, R. Madachy, D. Reifer, B.K. Clark, B.Steece, A.W. Brown, S. Chulani,C. Abts, Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
[4] J. Wen, S. Li, Z. Lin, Y. Hu, C. Huang, “Systematic literature review of machine learning based software development effort estimation models,”Information & Software Technology, vol. 54,pp. 41-59, 2012.
[5] V. S Dave, K. Dutta,” Neural Network based Models for Software Effort Estimation: A Review,” Artificial Intelligence Review, Springer, online 06 May 2012.
[6] Simon Haykin, Neural Networks and Learning Machines, PHI learning Private Limited, New Delhi, 3rd edition, 2010.
[7] A. Adri, A. Zakrani, “Design of Radial Basis Function Neural Networks for Software Effort Estimation,” International Journal of Computer Science Issues, vol. 4,pp. 11-17, 2010.
[8] P.V.G.D. Prasad Reddy, K.R. Sudha, P. Rama Sree, S.N.S.V.C. Ramesh, “Software Effort Estimation using Radial Basis and Generalized Regression Neural Networks,” Journal of Computing, vol. 2,pp. 87-92, 2010.
[9] V.S.Dave, K. Dutta, “Comparison of Regression model, Feed-forwardNeural Network and Radial Basis Neural Network for Software Development Effort estimation,” ACM SIGSOFT Software Engineering Notes, vol. 36,pp. 1-5, 2011.
[10] B.A.Kitchenham, L.M.Pickard, S.G.MacDonell and M.J.Shepperd, “What accuracy statistics really measure,” IEE Proc. Software, vol. 148,pp. 81-85, 2001.
[11] C. Lopez-Martin, “A Fuzzy Logic Model for predicting the Development effort of Short Scale Programs based upon Two Independent Variables,” Applied Soft Computing,vol.11, pp.724-732, 2011.
[12] N. Garcia-Diaz, C. Lopez-Martin, A. Chavoya, “A Comparative study of Two Fuzzy Logic Models for Software Development Effort Estimation,”Procedia Technology,vol. 7,pp. 305-314, 2013.
[13] A. Arcuri, L. Briand, “A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering,” ICSE’11,Honolulu, USA,May 21-28, 2011, pp. 1-10.
[14] E. Praynlin, P. Latha, “Software Effort Estimation Models Using Radial Basis Function,” International Journal of Computer, Information, Systems and Control Engineering, vol. 8, no. 1, World Academy of Science, Engineering and Technology, pp. 248-253, 2014.
[15] C. Lopez-Martin, J. LeboeufPasquier, Cornelio Yanez.M, Augustin Gutierrez, T, “Software Development Effort Estimation Using Fuzzy Logic: A case study,”Proceedings of the sixth International Conference on Computer Science, (ENC’05), IEEE Computer Society, 26-30 Sep. 2005, pp.113-120.