A Proposal for Systematic Mapping Study of Software Security Testing, Verification and Validation

Authors: Adriano Bessa Albuquerque, Francisco Jose Barreto Nunes

Abstract:

Software vulnerabilities are increasing and not only impact services and processes availability as well as information confidentiality, integrity and privacy, but also cause changes that interfere in the development process. Security test could be a solution to reduce vulnerabilities. However, the variety of test techniques with the lack of real case studies of applying tests focusing on software development life cycle compromise its effective use. This paper offers an overview of how a Systematic Mapping Study (MS) about security verification, validation and test (VVT) was performed, besides presenting general results about this study.

Keywords: Software test, software security verification validation and test, security test institutionalization, systematic mapping study.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1124545

References:

[1] Gary McGraw, “Software security”, IEEE Security and Privacy, March/April 2004, pages 32-35.
[2] CERT.BR. Available: http://www.cert.br/stats/incidentes/.
[3] NIST (2010) Special Publication 800-53A, Revision 1, 2010 - Guide for Assessing the Security Controls in Federal Information Systems and Organizations - Building Effective Security Assessment Plans.
[4] Sommerville, I. (2010), Software Engineering, Addison Wesley, 9th edition.
[5] Kitchenham, B., Brereton P., Budgen, D., Turner M., Bailey J., Linkman, S. (2009) “Systematic literature reviews in software engineering - A systematic literature review”. Information and Software Technology Journal. Vol. 51. Issue 1. Pages 7 - 15. Elsevier. January 2009.
[6] Kitchenham, B. and Charters, S. (2007) “Guidelines for performing Systematic Literature Reviews in Software Engineering”. Technical Report EBSE 2007-001, Keele University and Durham University Joint Report.
[7] Mafra, S., Barcelos, R., Travassos, G. (2006) “Aplicando uma Metodologia Baseada em Evidência na Definição de Novas Tecnologias de Software”, v. 1, pages 239 – 254.
[8] Kitchenham et al. (2010) “Systematic literature reviews in software engineering – A tertiary study”. Information and Software Technology 52 (2010) 792–805. Elsevier.
[9] R. Wieringa, N.A.M. Maiden, N.R. Mead, C. Rolland. Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requirements Engineering, 11 (1) (2006), pp. 102–107.
[10] K. Petersen, R. Feldt, S. Mujtaba, M. Mattsson, Systematic mapping studies in software engineering, in: 12th International Conference on Evaluation and Assessment in Software Engineering (EASE), 2008, pp. 71–80.
[11] Budgen, D., Turner, M., Brereton, P., Kitchenham, B. (2008) “Using Mapping Studies in Software Engineering”. Available: https://community.dur.ac.uk/ebse/biblio.php?id=86.
[12] Petticrew, Mark and Roberts, Helen. Systematic Reviews in the Social Sciences: A Practical Guide, Blackwell Publishing, 2005, ISBN 1405121106.
[13] RUP. (2013) IBM - Rational Unified Process ®. IBM Corporation. Copyright © 1987 – 2013.
[14] UBC. (2014) Snowballing technique. Available: http://hlwiki.slais.ubc.ca/index.php/Snowballing.
[15] Marback, Aaron, Do, Hyunsook, He, Ke, Kondamarri, Samuel and Xu, Dianxiang (2013) "A threat model-based approach to security testing". Software: Practice and Experience, v. 43, n. 2, p. 241-258, 2013.
[16] Gilliam, David P. et al. (2006) “Security verification techniques applied to patchlink COTS software”. In: Enabling Technologies: Infrastructure for Collaborative Enterprises, 2006. WETICE'06. 15th IEEE International Workshops on. IEEE, 2006. p. 319-325.
[17] Shahmehri, Nahid et al. (2012) “An advanced approach for modeling and detecting software vulnerabilities”. Information and Software Technology, v. 54, n. 9, p. 997-1013, 2012.
[18] Austin, Andrew; Holmgreen, Casper; Williams, Laurie. (2013) “A comparison of the efficiency and effectiveness of vulnerability discovery techniques”. Information and Software Technology, v. 55, n. 7, p. 1279-1288, 2013.
[19] Mouratidis, Haralambos; Giorgini, Paolo. (2007) “Security Attack Testing (SAT)—testing the security of information systems at design time”. Information systems, v. 32, n. 8, p. 1166-1183, 2007.
[20] Jürjens, Jan. (2208) “Model-based security testing using umlsec: A case study”. Electronic Notes in Theoretical Computer Science, v. 220, n. 1, p. 93-104, 2008.
[21] Xu, Dianxiang et al. (2012) “A model-based approach to automated testing of access control policies”. In: Proceedings of the 17th ACM symposium on Access Control Models and Technologies. ACM, 2012. p. 209-218.
[22] Wei, Tian et al. (2012) “Attack model based penetration test for SQL injection vulnerability”. In: Computer Software and Applications Conference Workshops (COMPSACW), 2012 IEEE 36th Annual. IEEE, 2012. p. 589-594.
[23] Antunes, Nuno; Vieira, Marco. (2209) “Detecting SQL injection vulnerabilities in web services”. In: Dependable Computing, 2009. LADC'09. Fourth Latin-American Symposium on. IEEE, 2009. p. 17-24.
[24] Wassermann, Gary; Su, Zhendong. (2007) “Sound and precise analysis of web applications for injection vulnerabilities”. In: ACM Sigplan Notices. ACM, 2007. p. 32-41.
[25] Ciampa, Angelo; Visaggio, Corrado Aaron; Di Penta, Massimiliano. (2010) “A heuristic-based approach for detecting SQL-injection vulnerabilities in Web applications”. In: Proceedings of the 2010 ICSE Workshop on Soft. Eng. for Secure Systems. ACM, 2010. p. 43-49.
[26] Shaffer, Alan B. et al. (2008) “A security domain model to assess software for exploitable covert channels”. In: Proceedings of the third ACM SIGPLAN workshop on Programming languages and analysis for security. ACM, 2008. p. 45-56.
[27] Morais, Anderson; Cavalli, Ana; Martins, Eliane. (2011) “A model-based attack injection approach for security validation”. In: Proceedings of the 4th international conference on Security of information and networks. ACM, 2011. p. 103-110.
[28] Wang, Linzhang; Wong, Eric; Xu, Dianxiang. (2007) “A threat model driven approach for security testing”. In: Proceedings of the Third International Workshop on Software Engineering for Secure Systems. IEEE Computer Society, 2007. p. 10.
[29] Al-Azzani, Sarah; Bahsoon, Rami. (2010) “Using implied scenarios in security testing”. In: Proceedings of the 2010 ICSE Workshop on Software Engineering for Secure Systems. ACM, 2010. p. 15-21.
[30] Xu, Dianxiang. (2013) “Software security testing of an online banking system: a unique research experience for undergraduates and computer teachers”. In: Proceeding of the 44th ACM technical symposium on Computer science education. ACM, 2013. p. 705-710.
[31] Avancini, Andrea. (2012) “Security testing of web applications: A research plan”. In:Proceedings of the 2012 International Conference on Software Engineering. IEEE Press, 2012. p. 1491-1494.
[32] Huang, Song et al. (2010) “A Case Study of Software Security Test Based On Defects Threat Tree Modeling”. In: Multimedia Information Networking and Security (MINES), 2010 International Conference on. IEEE, 2010. p. 362-365.
[33] Smith, Ben; Williams, Laurie. (2012) “On the Effective Use of Security Test Patterns”. In: Software Security and Reliability (SERE), 2012 IEEE Sixth International Conference on. IEEE, 2012. p. 108-117.
[34] Gilliam, David P. et al. (2001) “Reducing software security risk through an integrated approach”. In: Software Engineering Workshop, 2001. Proceedings. 26th Annual NASA Goddard. IEEE, 2001. p. 36-42.
[35] Xu, Dianxiang et al. (2012) “Automated security test generation with formal threat models”. Dependable and Secure Computing, IEEE Transactions on, v. 9, n. 4, p. 526-540, 2012.
[36] Du, Wenliang; Mathur, Aditya P. (2002) “Testing for software vulnerability using environment perturbation”. Quality and Reliability Engineering International, v. 18, n. 3, p. 261-272, 2002.
[37] Murthy, K. Krishna; Thakkar, Kalpesh R.; Laxminarayan, Shirsh. (2009) “Leveraging Risk Based Testing in Enterprise Systems Security Validation”. In:Emerging Network Intelligence, 2009 First International Conference on. IEEE, 2009. p. 111-116.
[38] Smith, Ben. (2011) “Systematizing security test case planning using functional requirements phrases”. In: Proceedings of the 33rd International Conference on Software Engineering. ACM, 2011. p. 1136-1137.
[39] Xiong, Pulei; Peyton, Liam. (2010) “A model-driven penetration test framework for Web applications”. In: Privacy Security and Trust (PST), 2010 Eighth Annual International Conference on. IEEE, 2010. p. 173-180.
[40] Ouchani, Samir; Jarraya, Yosr; Mohamed, Otmane Aït. (2011) “Model-based systems security quantification”. In: Privacy, Security and Trust (PST), 2011 Ninth Annual International Conference on. IEEE, 2011. p. 142-149.
[41] Fonseca, José; Vieira, Marco; Madeira, Henrique. (2013) “Evaluation of Web Security Mechanisms using Vulnerability and Attack Injection”. Dependable and Secure Computing, IEEE Transactions on, v. PP, Issue 99, p. 1, 2013.
[42] Carlsson, Bengt; Baca, Dejan. (2005) “Software security analysis-execution phase audit”. In: Software Engineering and Advanced Applications, 2005. 31st EUROMICRO Conference on. IEEE, 2005. p. 240-247.
[43] Ghindici, Dorina et al. (2006) “Integrated security verification and validation: Case study”. In: Local Computer Networks, Proceedings 2006 31st IEEE Conference on. IEEE, 2006. p. 1000-1007.
[44] He, Ke; Feng, Zhiyong; Li, Xiaohong. (2008) “An attack scenario based approach for software security testing at design stage”. In: Computer Science and Computational Technology, 2008. ISCSCT'08. International Symposium on. IEEE, 2008. p. 782-787.
[45] Savola, R. M. (2009) “Software security assurance of telecommunication systems”. In: Multimedia Computing and Systems, 2009. ICMCS '09. International Conference on Multimedia Computing and Systems.
[46] Mallouli, Wissam et al. (2008) “Modeling and Testing Secure Web-Based Systems: Application to an Industrial Case Study”. In: Signal Image Technology and Internet Based Systems, 2008. SITIS'08. IEEE International Conference on. IEEE, 2008. p. 128-136.
[47] Turpe, S. et al. (2008) “Supporting Security Testers in Discovering Injection Flaws”. In: Practice and Research Techniques, 2008. TAIC PART'08. Testing: Academic & Industrial Conference. IEEE, 2008. p. 64-68.
[48] Tappenden, Andrew et al. (2005) “Agile security testing of web-based systems via httpunit”. In: Agile Conference, 2005. Proceedings. IEEE, 2005. p. 29-38.
[49] Bessayah, Fayçal; Cavalli, Ana; Martins, Eliane. (2009) “A formal approach for specification and verification of fault injection process”. In: Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human. ACM, 2009. p. 883-890.
[50] Berbar, Ahmed; Ahmednacer, Mohamed. (2009) “Testing and fault tolerance approach for distributed software systems using nematode worms”. In:Proceedings of the 4th International Conference on Queueing Theory and Network Applications. ACM, 2009. p. 7.
[51] Zech, Philipp et al. (2013) “A Concept for Language-Oriented Security Testing”. In:Software Security and Reliability-Companion (SERE-C), 2013 IEEE 7th International Conference on. IEEE, 2013. p. 53-62.
[52] Katkalov, Kuzman et al. (2012) “Model-Driven Testing of Security Protocols with SecureMDD”. In: New Technologies, Mobility and Security (NTMS), 2012 5th International Conference on. IEEE, 2012. p. 1-5.
[53] Hui, Zhanwei et al. (2010) “Software security testing based on typical SSD: A case study”. In: Advanced Computer Theory and Engineering (ICACTE), 2010 3rd International Conference on. IEEE, 2010. p. V2-312-V2-316.
[54] Jinhua, Li; Jing, Li. (2010) “Model Checking Security Vulnerabilities in Software Design”. In: Wireless Communications Networking and Mobile Computing (WiCOM), 2010 6th International Conference on. IEEE, 2010. p. 1-4.
[55] Bodeau, D. J.; Brusil, N. R.; Chang, I. N.; Reece, M. J. (1992) “Security test and evaluation for multilevel-mode accreditation: Lessons learned”. In: Proceedings of eighth Annual Computer Security Applications Conference, 1992. p. 37-45.
[56] Wang, Wenhua et al. (2011) “A combinatorial approach to detecting buffer overflow vulnerabilities”. In: Dependable Systems & Networks (DSN), 2011 IEEE/IFIP 41st International Conference on. IEEE, 2011. p. 269-278.
[57] Wang, Weiguang; Zeng, Qingkai; Mathur, Aditya P. (2012) “A Security Assurance Framework Combining Formal Verification and Security Functional Testing”. In: Quality Software (QSIC), 2012 12th International Conference on. IEEE, 2012. p. 136-139.
[58] Schanes, Christian et al. (2013) “Generic Approach for Security Error Detection Based on Learned System Behavior Models for Automated Security Tests”. In:Software Testing, Verification and Validation Workshops (ICSTW), 2013 IEEE Sixth International Conference on. IEEE, 2013. p. 453-460.
[59] Belblidia, Nadia et al. (2006) “AOP extension for security testing of programs”. In:Electrical and Computer Engineering, 2006. CCECE'06. Canadian Conference on. IEEE, 2006. p. 647-650.
[60] Mouelhi, Tejeddine; Le Traon, Yves; Baudry, Benoit. (2007) “Mutation analysis for security tests qualification”. In: Testing: Academic and Industrial Conference Practice and Research Techniques-MUTATION, 2007. TAICPART-MUTATION 2007. IEEE, 2007. p. 233-242.
[61] Hwang, JeeHyun et al. (2012) “Selection of regression system tests for security policy evolution”. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ACM, 2012. p. 266-269.
[62] Lebeau, Franck et al. (2013) “Model-Based Vulnerability Testing for Web Applications”. In: Software Testing, Verification and Validation Workshops (ICSTW), 2013 IEEE Sixth International Conference on. IEEE, 2013. p. 445-452.
[63] Huang, Yao-Wen et al. (2004) “Securing web application code by static analysis and runtime protection”. In: Proceedings of the 13th international conference on World Wide Web. ACM, 2004. p. 40-52.
[64] Li, Li et al. (2013) “The Application of Fuzzing in Web Software Security Vulnerabilities Test”. In: Information Technology and Applications (ITA), 2013 International Conference on. IEEE, 2013. p. 130-133.
[65] Fourneret, Elizabeta et al. (2011) “Model-based security verification and testing for smart-cards”. In: Availability, Reliability and Security (ARES), 2011 Sixth International Conference on. IEEE, 2011. p. 272-279.
[66] Jing-Nong, Du; Yan-Sheng, Lu. (2010) “An Effect Evaluation Model for Vulnerability Testing of Web Application”. In: Networks Security Wireless Communications and Trusted Computing (NSWCTC), 2010 Second International Conference on. IEEE, 2010. p. 382-385.
[67] Ma, Jianli et al. (2010) “Information system security function validating using model checking”. In: Computer Engineering and Technology (ICCET), 2010 2nd International Conference on. IEEE, 2010. p. V1-517-V1-521.
[68] Salas, Percy Antonio Pari; Krishnan, Padmanabhan; Ross, Kelvin J. (2007) “Model-based security vulnerability testing”. In: Software Engineering Conference, 2007. ASWEC 2007. 18th Australian. IEEE, 2007. p. 284-296.
[69] Zhang, Xiao-Song; Shao, Lin; Zheng, Jiong. (2008) “A novel method of software vulnerability detection based on fuzzing technique”. In: Apperceiving Computing and Intelligence Analysis, 2008. ICACIA 2008. Intl. Conf. on. IEEE, 2008. p. 270-273.
[70] Blackburn, Mark et al. (2001) “Model-based approach to security test automation”. In: Proceedings of Quality Week 2001.
[71] Gupta, Daya; Chatterjee, Kakali; Jaiswal, Shruti. (2013) “A Framework for Security Testing”. In: Computational Science and Its Applications–ICCSA, 2013. Springer Berlin Heidelberg, 2013. p. 187-198.
[72] Ouedraogo, Moussa et al. (2012) “Appraisal and reporting of security assurance at operational systems level”. In: Journal of Systems and Software, v. 85, n. 1, 2012, p. 193-208.