Benchmarking of Pentesting Tools

Authors: Esteban Alejandro Armas Vega, Ana Lucila Sandoval Orozco, Luis Javier García Villalba

Abstract:

The benchmarking of tools for dynamic analysis of vulnerabilities in web applications is something that is done periodically, because these tools from time to time update their knowledge base and search algorithms, in order to improve their accuracy. Unfortunately, the vast majority of these evaluations are made by software enthusiasts who publish their results on blogs or on non-academic websites and always with the same evaluation methodology. Similarly, academics who have carried out this type of analysis from a scientific approach, the majority, make their analysis within the same methodology as well the empirical authors. This paper is based on the interest of finding answers to questions that many users of this type of tools have been asking over the years, such as, to know if the tool truly test and evaluate every vulnerability that it ensures do, or if the tool, really, deliver a real report of all the vulnerabilities tested and exploited. This kind of questions have also motivated previous work but without real answers. The aim of this paper is to show results that truly answer, at least on the tested tools, all those unanswered questions. All the results have been obtained by changing the common model of benchmarking used for all those previous works.

Keywords: Cybersecurity, IDS, security, web scanners, web vulnerabilities.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1130587

References:

[1] Verizon Enterprise. 2016 Data Breach Investigations Report. Report, Verizon Enterprise, July 2016.
[2] A. Sagala and E. Manurung. Testing and Comparing Result Scanning Using Web Vulnerability Scanner. Advanced Science Letters, 21(11):3458–3462, November 2015.
[3] P. Baral. Web Application Scanners: A Review of Related Articles. IEEE Potentials, 30(2):10–14, March 2011.
[4] Y. Makino and V. Klyuev. Evaluation of Web Vulnerability Scanners. In Proceedings of the IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), volume 1, pages 399–402, Warsaw, PL, September 2015.
[5] The Open Web Application Security Project OWASP. OWASP Zed Attack Proxy Project. https://www.owasp.org/index.php/OWASP Zed Attack Proxy Project, April 2016.
[6] Google. Google Code - Skipfish. https://code.google.com/archive/p/ skipfish/, March 2016.
[7] RandomStorm. Damn Vulnerable Web Application (DVWA). http:// www.dvwa.co.uk, March 2016.
[8] Google. Google Code - WAVSEP. https://code.google.com/archive/p/ wavsep/, March 2016.
[9] F. A. Saeed. Using WASSEC to Analysis and Evaluate Open Source Web Application Security Scanners. International Journal of Computer Science and Network, 3(2):43–49, April 2014.
[10] Web Application Security Consortium. Web Application Security Scanner Evaluation Criteria WASSEC. http://goo.gl/aePtyC, April 2016.
[11] W3af. W3af - Open Source Web Application Security Scanner. http: //w3af.org, Abril 2016.
[12] N. I. Daud, K. A. A. Bakar, and M. S. Md. Hasan. A Case Study on Web Application Vulnerability Scanning Tools. In Proceedings of the Conference of Science and Information (SAI), pages 595–600, 2014.
[13] Snort - Network Intrusion Detection and Prevention System. https:// www.snort.org/, Abril 2016.
[14] H. Alnabulsi, Md. R. Islam, and Q. Mamun. Detecting SQL Injection attacks using SNORT IDS. In Proceedings of the 2014 Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), pages 1–7. IEEE, Nov 2014.
[15] M. Dabbour, I. Alsmadi, and E. Alsukhni. Efficient Assessment and Evaluation for Websites Vulnerabilities using SNORT. International Journal of Security and its Applications, 7(1), 2013.
[16] HP. HP WebInsPect. Product Manual, HP, March 2015.
[17] Arachni. ARACHNI Web Application Security Scanner Framework. http://www.arachni-scanner.com, March 2016.
[18] F. A. Saeed. Using WASSEC to Evaluate Commercial Web Application Security Scanners. International Journal of Soft Computing and Engineering (IJSCE), 4(1):177–181, March 2014.
[19] A. Doup´e, M. Cova, and G. Vigna. Detection of Intrusions and Malware, and Vulnerability Assessment. In Christian Kreibich and Marko Jahnke, editors, Proceedings of the 7th International Conference (DIMVA 2010), pages 111–131, Bonn, Germany, July 2010.
[20] A. Doup´e. WackoPicko Vulnerable Website. https://github.com/ adamdoupe/WackoPicko, March 2016.
[21] The Open Web Application Security Project OWASP. OWASP Top 10 - 2013 The Ten Most Critical Web Application Security Risks. Release, The Open Web Application Security Project OWASP, June 2013.