Modelling the Occurrence of Defects and Change Requests during User Acceptance Testing

Authors: Kevin McDaid, Simon P. Wilson

Abstract:

Software developed for a specific customer under contract typically undergoes a period of testing by the customer before acceptance. This is known as user acceptance testing and the process can reveal both defects in the system and requests for changes to the product. This paper uses nonhomogeneous Poisson processes to model a real user acceptance data set from a recently developed system. In particular a split Poisson process is shown to provide an excellent fit to the data. The paper explains how this model can be used to aid the allocation of resources through the accurate prediction of occurrences both during the acceptance testing phase and before this activity begins.

Keywords: User acceptance testing. Software reliability growth modelling. Split Poisson process. Bayesian methods.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1329120

References:

[1] B. Hetzel. The Complete Guide to Software Testing (second edition), Wiley, 1988.
[2] G.J. Myers. Software Reliability, Principles and Practices, Wiley, 1976.
[3] J.D. Musa. Software Reliability Engineering, McGraw-Hill, New York, 1998.
[4] T.A. Chavez. Decision-Analytic Stopping Rule for Validation of Commercial Software Systems. IEEE Transactions on Software Engineering, 26: 907-918, 2000.
[5] S.R. Dalal and C.L. Mallows. When should one stop testing? Journal of the American Statistical Association, 83: 872-879, 1988.
[6] S.A. Hossain and R.C. Dahiya. Estimating the parameters of a nonhomogeneous Poisson process for software reliability. IEEE Transactions on Reliability, 42(4):604-612, 1993.
[7] L. Yin and K.S. Trivedi. Confidence Interval Estimation of NHPP-Based Software Reliability Models. Proceedings of the International Symposium on Software Reliability Engineering, 1999.
[8] M. Xie, G.Y. Hong and C. Wohlin. A Practical Method for the Estimation of Software Reliability Growth in the Early Stage of Testing. Proceedings of the International Symposium on Software Reliability Engineering, 1997.
[9] H. Hoffmann and K. McDaid An Analysis of Failure Occurrence Patterns during Testing and Operation Supplemental Proceedings of the International Symposium on Software Reliability Engineering, 2004.
[10] H. Hoffmann and K. McDaid Using Software Reliability Growth Models to Predict the Occurrence of Defects during Testing Proceedings of the International Conference on Software Development, Iceland, May 2005.
[11] S. Gokhale and K. S. Trivedi. A Time/Structure Based Software Reliability Model. Annals of Software Engineering , vol. 8, pp. 85-121, 1999.
[12] N.D. Singpurwalla and S.P. Wilson. Statistical Methods in Software Engineering. Springer, 1999.
[13] N.D. Singpurwalla and S.P. Wilson. Software Reliability Modeling. International Statistical Review, 62 3:289-317, 1994.
[14] A.L. Goel and K. Okumoto. Time dependent error-detection rate models for Software Reliability and Other performance Measures. IEEE Transactions on Software Engineering, R-29, pp206-211, 1979.
[15] S. Yamada, M. Ohba and S. Osaki. S-shaped Reliability Growth Modeling for Software Error Detection. IEEE Trans. on Reliability, R-32, pp. 475-485, 1983.
[16] S. Gokhale. Analysis of Software Reliability and Performance, Ph.D. thesis, Duke University, 1998.
[17] G. Knafl and J. Morgan. Solving ML equations for 2-parameter poissonprocess models for ungrouped software failure data. IEEE Transactions on Reliability, 45(1):42-53, 1996.
[18] K. McDaid and S. Wilson. Deciding how long to test software. Journal of the Royal Statistical Society, Series D, R-50. part 2: 117-134, 2001.