Comparative Analysis and Evaluation of Software Vulnerabilities Testing Techniques

Authors: Khalid Alnafjan, Tazar Hussain, Hanif Ullah, Zia ul haq Paracha

Abstract:

Software and applications are subjected to serious and damaging security threats, these threats are increasing as a result of increased number of potential vulnerabilities. Security testing is an indispensable process to validate software security requirements and to identify security related vulnerabilities. In this paper we analyze and compare different available vulnerabilities testing techniques based on a pre defined criteria using analytical hierarchy process (AHP). We have selected five testing techniques which includes Source code analysis, Fault code injection, Robustness, Stress and Penetration testing techniques. These testing techniques have been evaluated against five criteria which include cost, thoroughness, Ease of use, effectiveness and efficiency. The outcome of the study is helpful for researchers, testers and developers to understand effectiveness of each technique in its respective domain. Also the study helps to compare the inner working of testing techniques against a selected criterion to achieve optimum testing results.

Keywords: Software Security, Security Testing, Testing techniques, vulnerability, AHP.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1075352

References:

[1] (CERT/CC), C. C. C. "Cataloged vulnerabilities." from http://www.cert.org/stats/
[2] Antunes, N. and M. Vieira (2009). Comparing the Effectiveness of Penetration Testing and Static Code Analysis on the Detection of SQL Injection Vulnerabilities in Web Services. Dependable Computing, 2009. PRDC '09. 15th IEEE Pacific Rim International Symposium on.
[3] Aznar Bellver Jeronimo, C. R., Roberto, Romero Civera, Agustin (2011). "New Spanish Banking Conglomerates.Application of the Analytic Hierarchy Process (AHP) to their Market Value "International Research Journal of Finance and Economics (78).
[4] Bieman, J. M., D. Dreilinger, et al. (1996). Using fault injection to increase software test coverage. Seventh International Symposium on Software Reliability Engineering, 1996. Proceedings.
[5] Database, N. V. (2012). "National Vulnerability Database Version 2.2." Retrieved 10/08/2012, 2012, from http://nvd.nist.gov/.
[6] IEEE (1986). ANSI/IEEE Standard 1008-1987, IEEE Standard for Software Unit Testing.
[7] IEEE (1990). IEEE Standards Collection: Glossary of Software Engineering Terminology, IEEE Standard 610.12-1990.
[8] Khan, M. A. and M. Sadiq (2011). Analysis of black box software testing techniques: A case study. Current Trends in Information Technology (CTIT), 2011 International Conference and Workshop on.
[9] Lavenhar, C. M. a. S. R. "Code Analysis Tools - Overview." Retrieved 02/06/2012, from at https://buildsecurityin.us- cert.gov/bsi/articles/tools/code/263-BSI.html.
[10] Lavenhar, S. (2006). "code Analysis." Retrieved 01/06/2012, from https://buildsecurityin.us-cert.gov/bsi/articles/best-practices/code/214- BSI.html.
[11] Mitre. (2012). "Common vulnerability managment database." Retrieved 10/08/2012.
[12] Romero-Mariona, J., H. Ziv, et al. (2010). Increasing Trustworthiness through Security Testing Support. Social Computing (SocialCom), 2010 IEEE Second International Conference on.
[13] Rothermel, G. and M. J. Harrold (1996). "Analyzing regression test selection techniques." Software Engineering, IEEE Transactions on 22(8): 529-551.
[14] Saaty, T. (1980). The Analytic Hierarchy Process. New York, McGraw Hill.
[15] Savola, R. and K. Karppinen (2007). Practical Security Testing of Telecommunications Software--A Case Study. Telecommunications, 2007. AICT 2007. The Third Advanced International Conference on.
[16] Shahriar, H. and M. Zulkemine (2009). Automatic Testing of Program Security Vulnerabilities. Computer Software and Applications Conference, 2009. COMPSAC '09. 33rd Annual IEEE International.
[17] Thomas, L., X. Weifeng, et al. (2011). Mutation Analysis of Magento for Evaluating Threat Model-Based Security Testing. Computer Software and Applications Conference Workshops (COMPSACW), 2011 IEEE 35th Annual.
[18] Weyuker, E. J. (1993). Can we measure software testing effectiveness? Software Metrics Symposium, 1993. Proceedings., First International.
[19] Will Radosevich , C. C. M. (2009). "Black Box Security Testing Tools." Retrieved 31/05/2012, 2012, from https://buildsecurityin.us- cert.gov/bsi/articles/tools/black-box/261-BSI.html.
[20] Wyk, G. J. a. K. v. (2009). "White Box Testing." Retrieved 31/05/2012, 2012, from https://buildsecurityin.us-cert.gov/bsi/articles/best- practices/white-box/259-BSI.html.
[21] Wyk, K. R. v. (2007). "Adapting Penetration Testing for Software Development Purposes." Retrieved 03/06/2012, 2012, from https://buildsecurityin.us-cert.gov/bsi/articles/best¬practices/penetration/655-BSI.html.
[22] Wysopal, C. (2009). White Box Better Than Black Box Retrieved 31/05/2012, 2012, from http://www.veracode.com/blog/2009/10/white¬box-better-than-black-box/.