Performance Analysis of Software Reliability Models using Matrix Method

Authors: RajPal Garg, Kapil Sharma, Rajive Kumar, R. K. Garg

Abstract:

This paper presents a computational methodology based on matrix operations for a computer based solution to the problem of performance analysis of software reliability models (SRMs). A set of seven comparison criteria have been formulated to rank various non-homogenous Poisson process software reliability models proposed during the past 30 years to estimate software reliability measures such as the number of remaining faults, software failure rate, and software reliability. Selection of optimal SRM for use in a particular case has been an area of interest for researchers in the field of software reliability. Tools and techniques for software reliability model selection found in the literature cannot be used with high level of confidence as they use a limited number of model selection criteria. A real data set of middle size software project from published papers has been used for demonstration of matrix method. The result of this study will be a ranking of SRMs based on the Permanent value of the criteria matrix formed for each model based on the comparison criteria. The software reliability model with highest value of the Permanent is ranked at number – 1 and so on.

Keywords: Matrix method, Model ranking, Model selection, Model selection criteria, Software reliability models.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1330575

References:

[1] H. Pham, and X. Zhang, "A software cost model with warranty and risk costs" IEEE Trans. on Computers, 48 (1), 1999, pp. 71-75.
[2] M. Xie, Software Reliability Modelling, World Scientific Publishing Co. Ltd., 1991.
[3] A. Goel, and K. Okumoto, "Time dependent error-detection rate model for software reliability and other performance measures," IEEE Trans. on Reliability, R-28(3), 1979, pp. 206-211.
[4] M. R. Lyu, Handbook of Software Reliability Eng., McGraw-Hill, 1996.
[5] J. D. Musa, and K. Okumoto, "A logarithmic Poisson execution time model for software reliability measurement," Conf. Proc. 7th International Conf. on Softw. Engineering, 1983, pp. 230-237.
[6] C. Wohlin, "Software testing and reliability for telecommunication systems," Softw. Engineering-86", ed. D. Barnes and P. Brown, Peter Peregrinus Ltd., Stevenage, United Kingdom, 1986, pp. 27-42.
[7] J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability Measurement, Predication, Aplication, McGraw Hill, 1987.
[8] C. Y. Huang, M. R. Lyu, and S. Y. Kuo, "A unified scheme of some non-homogenous Poisson process models for software reliability estimation," IEEE Trans. on Softw. Engineering, vol. 29, no. 3, March 2003, pp. 261-269.
[9] C. G. Bai,"Bayesian network based software reliability prediction with an operational profile," J. Syst. Softw., vol. 77, no. 2, 2005, pp.103-112.
[10] L. Tian, and A. Noore, "On-line prediction of software reliability using an evolutionary connectionist model," J. Syst. Softw., vol. 77, no. 2, 2005, pp. 173-180.
[11] W. L. Wang, D. Pan, and M. H. Chen, "Architecture-based software reliability modeling," J. Syst. Softw., vol. 79, no. 1, 2006, pp. 132-146.
[12] X. Zhang, and H. Pham, "Software field failure rate prediction before software deployment," J. Syst. Softw., Vol.79, no. 3, 2006, pp. 291-300.
[13] C. Y. Huang, "Performance analysis of software reliability growth models with testing-effort and change-point," J. Syst. Softw., vol. 76, no. 2, 2005, pp. 181-194.
[14] C. Y. Huang, and C. T. Lin, "Software reliability analysis by considering fault dependency and debugging time lag," IEEE Trans. Reliability, vol. 55, no. 3, 2006, pp. 436-450.
[15] Q. P. Hu, M. Xie, S. H. Ng, and G. Levitin, "Robust recurrent neural network modeling for software fault detection and correction prediction," Reliability Engineering and System Safety, vol. 92 no. 3, 2007, pp. 332-340.
[16] D. R. Jeske, and X. Zhang, "Some successful approaches to software reliability modeling in industry," J. Syst. Softw., vol. 74, no. 1, 2005, pp. 85-99.
[17] G.H. Schick, and R.W. Wolverton, "An analysis of competing software reliability models," IEEE Trans. on Softw. Engineering, March 1978, pp. 104-120.
[18] A.N. Sukert, "Empirical validation of three software errors predictions models," IEEE Trans. on Reliability, August 1979, pp. 199-205.
[19] S. Brocklehurst, P.Y. Chan, B. Littlewood, and J. Snell, "Recalibrating software reliability models," IEEE Trans. on Softw. Engineering, vol. SE-16 no. 4, April 1990, pp. 458-470.
[20] A.L Goel, "Software reliability models: assumption, limitations, and applicability," IEEE Trans. on Softw. Engineering, December 1985, pp. 1411-1423.
[21] Abdel-Ghaly, P.Y. Chan, and B. Littlewood, "Evaluation of competing software reliability predictions," IEEE Trans. on Softw. Engineering, vol. SE-12 no. 12, September 1986, pp. 950-967.
[22] T.M Khoshgoftaar, and T. Woodcock, "Software reliability model selection: A case study," Proc. of the 2nd International Symposium on Softw. Reliability Engineering. IEEE Computer Society Press, Austin, TX: 1991, pp. 183-191.
[23] C. Y. Huang, and S. Y. Kuo, "Analysis of incorporating logistic testing effort function into software reliability modeling," IEEE Trans. on Reliability, vol. 51, no. 3, 2002, pp. 261-270.
[24] K. Pillai, and V. S. S. Nair, "A model for software development effort and cost estimation," IEEE Trans. on Softw. Engineering, vol. 23, no. 8, 1997, pp. 485-497.
[25] S. Yamada, K. Tokuno, and S. Osaki, "Imperfect debugging models with fault introduction rate for software reliability assessment," International J. Syst. Science, vol. 23, no. 12, 1992.
[26] H. Pham, "Software reliability and cost models: perspectives, comparison and practice", European J. of Operational Research, vol. 149, 2003, p. 475- 489.
[27] H. Pham, L. Nordmann, and X. Zhang, "A general imperfect software debugging model with s-shaped fault detection rate," IEEE Trans. Reliability, vol. 48, June 1999, pp. 169-175.
[28] H. Pham and X. Zhang, "An NHPP software reliability models and its comparison," International J. of Reliability, Quality and Safety Engineering, vol. 14, no. 3, 1997, pp. 269-282.
[29] H. Pham, System software reliability, Springer London, 2006.
[30] X. Zhang, X. Teng and H. Pham, "Considering Fault Removal Efficiency in Software Reliability Assessment", IEEE Trans. on Systems, Man, & Cybernetics- Part A, vol. 33, no.1, 2003, pp. 114-120.
[31] S. Hwang and H. Pham, "Quasi-renewal time-delay fault-removal consideration in software reliability modelling," IEEE Trans. on systems, man and cybernetics-Part A:Systems and humans, vol. 39, no. 1, January 2009.
[32] K. C. Chiu, Y. S. Huang, and T. Z. Lee, "A study of software reliability growth from the perspective of learning effects,"Reliability Engineering and System Safety, 2008, pp. 1410-1421.
[33] M. Zhao, and M. Xie, "On the log-power NHPP software reliability model,". Proceedings of the Third IEEE International Symposium on Softw. Reliability Engineering, Research Triangle Park, North Carolina, 1992, pp. 14-22.
[34] M. R. Lyu, and A. Nikora, "Applying software reliability models more effectively," IEEE Softw., 1992, 43-52.
[35] H. Pham and C. Deng, "Predictive-ratio risk criterion for selecting software reliability models," Proc. Ninth International Conf. On Reliability and Quality in Design, August 2003.
[36] P. L. Li, J. Herbsleb, and M. Shaw, "Forecasting field defect rates using a combined time-based and metrics-based approach: a case study of OpenBSD," Proceedings of the 16th IEEE International Symposium on Softw. Reliability Engineering, Chicago, IL, 2005, pp. 193-202.
[37] R. Peng, Q.P. Hu,and S.H. Ng, ,"Incorporating Fault Dependency and Debugging Delay in Software Reliability Analysis," Proceedings of the IEEE ICMIT, 2008, pp.641-645.
[38] S. Dahiya et al.,"Power quality evaluation in deregulated power system using matrix method," International J. Global Energy Issues, vol. 28, no. 1, 2007.
[39] R. K. Garg, V. K. Gupta, and V. P. Agrawal,"Quality evaluation of thermal power plants by graph theoretical methodology," International J. of Power and Energy Systems, 27 (1), 2007, pp. 42-48.
[40] M. Marcus, and H. Minc," Permanents," American Mathematics, 72, 1965, pp. 571-91.