Real-time Network Anomaly Detection Systems Based on Machine-Learning Algorithms

Authors: Zahra Ramezanpanah, Joachim Carvallo, Aurelien Rodriguez

Abstract:

This paper aims to detect anomalies in streaming data using machine learning algorithms. In this regard, we designed two separate pipelines and evaluated the effectiveness of each separately. The first pipeline, based on supervised machine learning methods, consists of two phases. In the first phase, we trained several supervised models using the UNSW-NB15 data set. We measured the efficiency of each using different performance metrics and selected the best model for the second phase. At the beginning of the second phase, we first, using Argus Server, sniffed a local area network. Several types of attacks were simulated and then sent the sniffed data to a running algorithm at short intervals. This algorithm can display the results of each packet of received data in real-time using the trained model. The second pipeline presented in this paper is based on unsupervised algorithms, in which a Temporal Graph Network (TGN) is used to monitor a local network. The TGN is trained to predict the probability of future states of the network based on its past behavior. Our contribution in this section is introducing an indicator to identify anomalies from these predicted probabilities.

Keywords: Cyber-security, Intrusion Detection Systems, Temporal Graph Network, Anomaly Detection.

References:

[1] Godala, Sravanthi, and Rama Prasad V. Vaddella. ”A study on intrusion detection system in wireless sensor networks.” International Journal of Communication Networks and Information Security 12.1 (2020): 127-141.
[2] Lyon GF. Nmap network scanning: The official Nmap project guide to network discovery and security scanning. Insecure. Com LLC (US); 2008.
[3] Sarker, Iqbal H., et al. ”Cybersecurity data science: an overview from machine learning perspective.” Journal of Big data 7.1 (2020): 1-29.
[4] Moustafa N, Slay J. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In2015 military communications and information systems conference (MilCIS) 2015 Nov 10 (pp. 1-6). IEEE.
[5] Seufert, Stefan, and Darragh O’Brien. ”Machine learning for automatic defence against distributed denial of service attacks.” 2007 IEEE International Conference on Communications. IEEE, 2007.
[6] Alazab, Ammar, et al. ”Using feature selection for intrusion detection system.” 2012 international symposium on communications and information technologies (ISCIT). IEEE, 2012.
[7] Buczak, Anna L., and Erhan Guven. ”A survey of data mining and machine learning methods for cyber security intrusion detection.” IEEE Communications surveys & tutorials 18.2 (2015): 1153-1176.
[8] Sarker, Iqbal H., A. S. M. Kayes, and Paul Watters. ”Effectiveness analysis of machine learning classification models for predicting personalized context-aware smartphone usage.” Journal of Big Data 6.1 (2019): 1-28.
[9] Li, Yinhui, et al. ”An efficient intrusion detection system based on support vector machines and gradually feature removal method.” Expert systems with applications 39.1 (2012): 424-430.
[10] Brugger, T. ”KDD cup’99 dataset (network intrusion) considered harmful, 15 September 2007. Retrieved January 26, 2008.” (2007).
[11] Hosseinzadeh, Mehdi, et al. ”Improving security using SVM-based anomaly detection: issues and challenges.” Soft Computing 25.4 (2021): 3195-3223.
[12] Yang, Kun, Samory Kpotufe, and Nick Feamster. ”An Efficient One-Class SVM for Anomaly Detection in the Internet of Things.” arXiv preprint arXiv:2104.11146 (2021).
[13] Shapoorifard, Hossein, and Pirooz Shamsinejad. ”Intrusion detection using a novel hybrid method incorporating an improved KNN.” Int. J. Comput. Appl 173.1 (2017): 5-9.
[14] Serpen, Gursel, and Ehsan Aghaei. ”Host-based misuse intrusion detection using PCA feature extraction and kNN classification algorithms.” Intelligent Data Analysis 22.5 (2018): 1101-1114.
[15] Salama, Mostafa A., et al. ”Hybrid intelligent intrusion detection scheme.” Soft computing in industrial applications. Springer, Berlin, Heidelberg, 2011. 293-303.
[16] Al-Yaseen, Wathiq Laftah, Zulaiha Ali Othman, and Mohd Zakree Ahmad Nazri. ”Real-time intrusion detection system using multi-agent system.” IAENG International Journal of Computer Science 43.1 (2016): 80-90.
[17] Yan, Weizhong, and Lijie Yu. ”On accurate and reliable anomaly detection for gas turbine combustors: A deep learning approach.” arXiv preprint arXiv:1908.09238 (2019).
[18] Protogerou, Aikaterini, et al. ”A graph neural network method for distributed anomaly detection in IoT.” Evolving Systems 12.1 (2021): 19-36.
[19] Kolias, Constantinos, et al. ”DDoS in the IoT: Mirai and other botnets.” Computer 50.7 (2017): 80-84.
[20] Rossi, Emanuele, et al. ”Temporal graph networks for deep learning on dynamic graphs.” arXiv preprint arXiv:2006.10637 (2020).
[21] B´eres, Ferenc, et al. ”Node embeddings in dynamic graphs.” Applied Network Science 4.1 (2019): 1-25.
[22] Kazemi, Seyed Mehran, et al. ”Representation Learning for Dynamic Graphs: A Survey.” J. Mach. Learn. Res. 21.70 (2020): 1-73.
[23] Sokolova, Marina, and Guy Lapalme. ”A systematic analysis of performance measures for classification tasks.” Information processing & management 45.4 (2009): 427-437.
[24] DataReportal (2021), “Digital 2021 Global Digital Overview,” retrieved from https://datareportal.com/reports/digital-2021-global-digital-overview
[25] Liu, Hongyu, and Bo Lang. ”Machine learning and deep learning methods for intrusion detection systems: A survey.” applied sciences 9.20 (2019): 4396.