Suitability of Black Box Approaches for the Reliability Assessment of Component-Based Software

Authors: Anjushi Verma, Tirthankar Gayen

Abstract:

Although, reliability is an important attribute of quality, especially for mission critical systems, yet, there does not exist any versatile model even today for the reliability assessment of component-based software. The existing Black Box models are found to make various assumptions which may not always be realistic and may be quite contrary to the actual behaviour of software. They focus on observing the manner in which the system behaves without considering the structure of the system, the components composing the system, their interconnections, dependencies, usage frequencies, etc.As a result, the entropy (uncertainty) in assessment using these models is much high.Though, there are some models based on operation profile yet sometimes it becomes extremely difficult to obtain the exact operation profile concerned with a given operation. This paper discusses the drawbacks, deficiencies and limitations of Black Box approaches from the perspective of various authors and finally proposes a conceptual model for the reliability assessment of software.

Keywords: Black Box, faults, failure, software reliability.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1124009

References:

[1] R. C. Cheung. “A user-oriented software reliability model”, IEEE Transactions on Software Engineering,vol.6, pp.118-125,1980.
[2] A. L. Goel. “An analysis of competing software reliability models”, IEEE Transactions on Software Engineering, vol.6, pp.501-502,1980.
[3] S N. Weiss and E.J Weyuker “An Extended Domain-Based Model of Software Reliability”, IEEE Transactions on Software Engineering, vol. 14, pp.1512-1524,1988.
[4] S. Yacoub, B. Cukic, H. H Ammar, “A scenario-based reliability analysis approach for component-based software”, IEEE Transactions on Reliability, vol.53, pp. 22-31,2004.
[5] T. Gayen. “Analysis and proposition of error based model to predict the minimum reliability of software”, International Conference on Education Technology and Computer, Singapore, pp.40-44, 2009.
[6] C. J. Hsu and C. Y. Huang. “An adaptive reliability analysis using path testing for complex component-based software systems”, IEEE Transactions on Reliability, vol. 60, pp.158-170, 2011.
[7] G. Popstojanova, K. S. Trivedi, “Architecture-based approach to reliability assessment of software systems”, Performance Evaluation, vol. 45, pp.179-204, 2001.
[8] S. Garnaik, T. Gayen., S. Mishra S, "Reliability Enhancement of Software by Minimizing the Overflow Errors", International Journal of Systems Assurance Engineering and Management, Springer, Vol. 5, No. 4, pp. 724-730, 2014.
[9] Michael R. Lyu Handbook of Software Reliability Engineering. McGraw-Hill publishing, 1995, ISBN 0-07-039400-8
[10] ANSI/IEEE, "Standard Glossary of Software Engineering Terminology", STD-729-1991, ANSI/IEEE, 1991
[11] R. Mirandola, P. Potena, E. Riccobene, P. Scandurra, “A reliability model for Service Component Architectures”, Vol. 89, pp. 109–127, 2014.
[12] Fouad ben Nasr Omri, Ralf Reussner, “Towards Reliability Estimation of Large Systems-of-Systems with the Palladio Component Model”, Trusted Cloud Computing, Springer, 2014.
[13] Whittaker, J. A., Poore, J.H., “Markov analysis of software specifications. ACM Transactions on Software Engineering Methodology”, Vol. 2, Issue 1, 93–106, 1993.
[14] Poore, J., Mills, H., Mutchler, D, “Planning and certifying software system reliability”, IEEE Software,Vol. 10, Issue 1, pp.88–99, 1993.
[15] B. W. Boehm, J. R. Brown, and M. Lipow, “Quantitative evaluation of software quality,” in Proceedings of Second International Conference on Software Engineering, San Francisco, CA, Oct. 1976, pp. 592-605.
[16] A. L. Goel, “A guidebook for software reliability assessment,’’ Syracuse Univ., Syracuse, NY, Tech. Rep. No. 83-11, Apr. 1983.
[17] H. Hecht, “Mini-tutorial on software reliability,” in Proc. IEEE COMPSAC80, Chicago, IL, pp. 383-385, 1980
[18] IEEE Standard Dictionary of Electrical and Electronic Terms.2nd ed., New York: IEEE Press, 1977.
[19] W. H. MacWilliams, “Reliability of large real-time control software systems,” in Rec. 1973 IEEE Symp. Computer Software Reliability, New York, pp. 1-6, 1973.
[20] M. L. Shooman, Sofrware Engineering. New York: McGraw-Hill, 1983.
[21] T. A. Thayer, M. Lipow, and E. C. Nelson, Software Reliability (TRW Ser. Software Technol. 2). New York: North-Holland, 1978.
[22] J. R. Brown and M. Lipow, “Testing for software reliability,” in Proceedings of International Conference on Reliable Software, Los Angeles, CA, pp.5 18-527, 1975.
[23] A. L Goel, “A summary of the discussion on ‘An analysis of competing software reliability models” IEEE Transactions Software Engineering, Vol. SE-6, pp. 501-502, Sept. 1980.
[24] G. J. Schick and R. W. Wolverton, “An analysis of competing software reliability models,” IEEE Transactions on Software Engineering, vol. SE-4, pp.104-120, Mar. 1978.
[25] A. Iannino, I. D. Musa, K. Okumoto, and B. Littlewood. “Criteriafor software reliability model comparisons,” IEEE Tranactions Software Engineering, vol. SE-IO, pp. 687-691, Nov. 1984.
[26] Bev Littlewood, “How to Measure Software Reliability and How Not To”, IEEE Transactions on Reliability, Vol.R-28, Issue 2, pp. 103 – 110, 1979.
[27] Bev Littlewood, “Theories of Software Reliability: How Good Are They and How Can They Be Improved”, IEEE Transactions on Software Engineering, Vol. SE-6, no. 5, 1980.
[28] G.Popstojanova, K S Trivedi, “Architecture-based approaches to software reliability prediction”, Computers & Mathematics with Applications, Elsevier, Vol. 46, Issue 7, pp. 1023–1036, 2003.
[29] R. W. Butler, G. B. Finelli, The infeasibility of quantifying the reliability of life-critical real-time software, IEEE Transactions on Software Engineering Vol.19 Issue1, pp. 3–12, 1993.
[30] J. C. Laprie, Dependability of computer systems: concepts, limits, improvements, in: Proceedings of the Sixth International Symposium on Software Reliability Engineering (ISSRE’95), pp. 2–11, 1995.
[31] S. Krishnamurthy, A.P. Mathur, On the estimation of reliability of a software system using reliabilities of its components, in: Proceedings of the Eighth International Symposium on Software Reliability Engineering (ISSRE’97), pp. 146–155, 1997
[32] J. M. Voas, Certifying off-the-shelf software components, IEEE Computer, Vol. 31, Issue 6, pp. 53–59, 1998.
[33] J. M. Voas, C. C. Michael, K. W. Miller, confidently assessing a zero probability of software failure, High Integrity Systems, Vol. 1, Issue 3, pp. 269–275, 1995
[34] Swapna S. Gokhale, W. Eric Wong, J. R. Horgan, Kishor S. Trivedi, “An analytical approach to architecture-based software performance and reliability prediction”, Performance Evaluation Journal, Elsevier, Vol. 58, Issue 4, pp. 391 – 412, 2004.
[35] D. Hamlet. “Are we testing for true reliability?”. IEEE Software, 13(4):21–27, 1992.
[36] J. R. Horgan and A. P. Mathur, "Handbook of Software Reliability Engineering", M. R. Lyu, Editor, chapter “Software Testing and Reliability”, McGraw-Hill, NewYork, NY, pp. 531–566, 1996.
[37] D. L. Parnas, P. C. Clements, and D. M. Weiss. “The modular structure of complex systems”. IEEE Transactions on Software Engineering, SE-11(3):259–266,1985.
[38] W.W. Everett. “Software component reliability analysis”. proc. of Application Specific Software Engineering and Technology, Dallas, TX, March 1999.
[39] S. Gokhale, M. R. Lyu, and K. S. Trivedi. “Reliability simulation of component-based software systems”. In Proc. of Ninth Intl. Symposium on Software Reliability Engineering (ISSRE 98), pages 192–201, Paderborn, Germany, 1998.
[40] S. Gokhale and K. S. Trivedi. “Structure–based software reliability prediction”. In Proc.of Fifth Intl. Conference on Advanced Computing (ADCOMP 97), pages 447–452, India, 1997
[41] D. Hamlet, D. Woit, and D. Mason. “Theory of software reliability based on components”, In Proc. of Intl. Conference on Software Engineering, pages 361–370, Toronto, Canada, 2001.
[42] J. Ledoux and G. Rubino. “A counting model for software reliability analysis”. IASTED Journal on Simulation, 1997.
[43] J. Ledoux and G. Rubino. “Simple formulas for counting processes in reliability models”, Theory of Applied Probability, 1997.
[44] W. Wang, Y. Wu, and M. H. Chen. “An architecture-based software reliability model”, In Proc. of Pacific Rim Dependability Symposium, Hong Kong, December 1999.
[45] Hoang Pham, “System Software Reliability”, Springer, 2005.
[46] Hoang Pham, “Software Reliability”, Springer, 2000.