Feature Based Unsupervised Intrusion Detection

Authors: Deeman Yousif Mahmood, Mohammed Abdullah Hussein

Abstract:

The goal of a network-based intrusion detection system is to classify activities of network traffics into two major categories: normal and attack (intrusive) activities. Nowadays, data mining and machine learning plays an important role in many sciences; including intrusion detection system (IDS) using both supervised and unsupervised techniques. However, one of the essential steps of data mining is feature selection that helps in improving the efficiency, performance and prediction rate of proposed approach. This paper applies unsupervised K-means clustering algorithm with information gain (IG) for feature selection and reduction to build a network intrusion detection system. For our experimental analysis, we have used the new NSL-KDD dataset, which is a modified dataset for KDDCup 1999 intrusion detection benchmark dataset. With a split of 60.0% for the training set and the remainder for the testing set, a 2 class classifications have been implemented (Normal, Attack). Weka framework which is a java based open source software consists of a collection of machine learning algorithms for data mining tasks has been used in the testing process. The experimental results show that the proposed approach is very accurate with low false positive rate and high true positive rate and it takes less learning time in comparison with using the full features of the dataset with the same algorithm.

Keywords: Information Gain (IG), Intrusion Detection System (IDS), K-means Clustering, Weka.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1097122

References:

[1] Bhavin Shah and Bhushan H Trivedi, “Artificial Neural Network based Intrusion Detection System: A Survey” International Journal of Computer Applications (0975 – 8887) Volume 39– No.6, February 2012.
[2] Gaikwad, Sonali Jagtap, Kunal Thakare, and Vaishali Budhawant “Anomaly Based Intrusion Detection System Using Artificial Neural Network and Fuzzy Clustering” International Journal of Engineering Research & Technology (IJERT) Vol. 1 Issue 9, November- 2012, ISSN: 2278-0181.
[3] Sandip Sonawane , Shailendra Pardeshi, and Ganesh Prasad “A survey on intrusion detection techniques” World Journal of Science and Technology 2012, 2(3):127-133, ISSN: 2231 – 2587.
[4] Deeman Y. Mahmood, Mohammed A. Hussein “Intrusion Detection System Based on K-Star Classifier and Feature Set Reduction” International Organization of Scientific Research Journal of Computer Engineering (IOSR-JCE) Vol.15, Issue 5, PP. 107-112, Dec. 2013.
[5] Chunhua Gu and Xueqin Zhang,” A Rough Set and SVM Based Intrusion Detection Classifier”, Second International Workshop on Computer Science and Engineering, 2009.
[6] Gary Stein, Bing Chen, “Decision Tree Classifier for network intrusion detection with GA based feature selection”, University of Central Florida. ACM-SE 43, proceedings of 43rd annual Southeast regional Conference. Volume 2, 2005, ACM, New York, USA.
[7] Heba F. Eid, Ashraf Darwish, Aboul Ella Hassanien, and Ajith Abraham” Principle Components Analysis and Support Vector Machine” based Intrusion Detection System”, IEEE 2010.
[8] Horeis, T, "Intrusion detection with neural network - Combination of self-organizing maps and redial basis function networks for human expert integration", a Research report 2003. Available in hap://ieeecis. org/Jiles/ EA C-Research-2003-Report-Horeis.pdf
[9] Zargar, G. R. “Category Based Intrusion Detection Using PCA”, International Journal of Information Security (October 2012), 3, 259- 271.
[10] Yogendra Kumar Jain, Upendra “Intrusion Detection using Supervised Learning with Feature Set Reduction”, International Journal of Computer Applications (0975 – 8887) Volume 33– No.6, November 2011.
[11] A. M. Riad, Ibrahim Elhenawy ,Ahmed Hassan and Nancy Awadallah, “Visualize Network Anomaly Detection by Using K-Means Clustering Algorithm”, International Journal of Computer Networks & Communications (IJCNC) Vol.5, No.5, September 2013.
[12] The Knowledge Discovery in Databases, NSL-KDD dataset, http://nsl.cs.unb.ca/NSL-KDD/
[13] University of Waikato, WEKA: Waikato environment for knowledge analysis. Data Mining Software in Java. http://www.cs.waikato.ac.nz/ml/weka/.