Detecting Abnormal Traffic in Large-Scale Networks (original) (raw)
Related papers
Deep Learning Approaches for Network Intrusion Detection
MS Thesis, 2019
As the scale of cyber attacks and volume of network data increases exponentially, organizations must develop new ways of keeping their networks and data secure from the dynamic nature of evolving threat actors. With more security tools and sensors being deployed within the modern day enterprise network, the amount of security event and alert data being generated continues to increase, making it more difficult to find the needle in the haystack. Organizations must rely on new techniques to assist and augment human analysts when dealing with the monitoring, prevention, detection, and response to cybersecurity events and potential attacks on their networks. The focus for this Thesis is on classifying network traffic flows as benign or malicious. The contribution of this work is two-fold. First, a feedforward fully connected Deep Neural Network (DNN) is used to train a Network Intrusion Detection System (NIDS) via supervised learning. Second, an autoencoder is used to detect and classify attack traffic via unsupervised learning in the absence of labeled malicious traffic. Deep neural network models are trained using two more recent intrusion detection datasets that overcome limitations of other intrusion detection datasets which have been commonly used in the past. Using these more recent datasets, deep neural networks are shown to be highly effective in performing supervised learning to detect and classify modern-day cyber attacks with a high degree of accuracy, high detection rate, and low false positive rate. In addition, an autoencoder is shown to be effective for anomaly detection.
Detecting Unbalanced Network Traffic Intrusions with Deep Learning
IEEE access, 2024
The growth of cyber threats demands a robust and adaptive intrusion detection system (IDS) capable of effectively recognizing malicious activities from network traffic. However, the existing imbalance of class in network data possess a significant challenge to traditional IDS. To overcome these challenges, this project proposes a novel hybrid Intrusion Detection System using machine learning algorithms, which includes XGBoost, Long Short-Term Memory (LSTM), Mini-VGGNet, and AlexNet, which is used to handle the unbalanced network traffic data. Furthermore, the Random Forest Regressor is used to ascertain the importance of features for enhancing detection accuracy and interpretability. Addressing the inherent class imbalance in network data is crucial for ensuring the IDS's effectiveness. The proposed system employs a combination of oversampling techniques for minority classes and under sampling techniques for majority classes during data preprocessing. This balanced representation of network traffic data helps prevent the IDS from being biased towards the majority class and improves its ability to detect rare or novel intrusions. The utilization of Random Forest Regressor for feature extraction serves a dual purpose. It helps identify the most relevant features within the network traffic data that contribute significantly to detecting intrusions. It enables the system to prioritize and focus on these important features during model training, thereby enhancing detection accuracy while reducing computational complexity. This research contributes to the ongoing efforts to mitigate cyber threats and safeguard critical network infrastructures.
Network Anomaly Detection Using LSTM Based Autoencoder
Proceedings of the 16th ACM Symposium on QoS and Security for Wireless and Mobile Networks
Anomaly detection aims to discover patterns in data that do not conform to the expected normal behaviour. One of the significant issues for anomaly detection techniques is the availability of labeled data for training/validation of models. In this paper, we proposed a hyper approach based on Long Short Term Memory (LSTM) autoencoder and One-class Support Vector Machine (OC-SVM) to detect anomalies based attacks in an unbalanced dataset, by training the models using only examples of normal classes. The LSTM-autoencoder is trained to learn the normal traffic pattern and to learn the compressed representation of the input data (i.e. latent features) and then feed it to an OC-SVM approach. The hybrid model overcomes the shortcomings of the separate OC-SVM, in which its low capability to operate with massive and high-dimensional datasets. Additionally, we perform our experiments using the most recent dataset (InSDN) of Intrusion Detection Systems (IDSs) for SDN environments. The experimental results show that the proposed model provides higher detection rate and reduces the processing time significantly. Hence, our method provides great confidence in securing SDN networks from malicious traffic.
Network Traffic Anomaly Detection via Deep Learning
Information
Network intrusion detection is a key pillar towards the sustainability and normal operation of information systems. Complex threat patterns and malicious actors are able to cause severe damages to cyber-systems. In this work, we propose novel Deep Learning formulations for detecting threats and alerts on network logs that were acquired by pfSense, an open-source software that acts as firewall on FreeBSD operating system. pfSense integrates several powerful security services such as firewall, URL filtering, and virtual private networking among others. The main goal of this study is to analyse the logs that were acquired by a local installation of pfSense software, in order to provide a powerful and efficient solution that controls traffic flow based on patterns that are automatically learnt via the proposed, challenging DL architectures. For this purpose, we exploit the Convolutional Neural Networks (CNNs), and the Long Short Term Memory Networks (LSTMs) in order to construct robust ...
IOSR Journal of Engineering (IOSR JEN), 2019
Basically anomaly detection is the part of intrusion detection system which is categorized in network intrusion detection system as well as host based intrusion detection system. Various existing systems has been developed on synthetic as well as some real time data. It system illustrates some challenges like false positive ratio of system, elevate, low accuracy these are the major challenges of anomaly based intrusion detection system. This work we proposed a deep learning based anomaly intrusion detection system which can eliminate label as well as a label attacks IDS focus on identifying possible incidents or threats, logging information, attempting to stop intrusion or malicious activities, and report it to the management station. In addition, it record information related to observed actions, notify security administrators of significantly observed actions and generate reports. Many Intrusion detection systems can also react to a detected hazard by attempting to prevent it from following. For stopping attack itself, they use numerous response techniques, altering the security surroundings for example reconfiguring a firewall or altering the attack's content. Thus IDS helps in statistical analysis for malicious behavior. In this work we proposed a Deep Learning base intrusion detection system for synthetic as well as real time network environment. Various dataset have been used to evaluate the proposed experimental analysis. The partial implementation of system shows the better results than existing systems. CIDDS-001, KKDCUP99, NSLKDD, ISCX network dataset used for evaluate the system with different algorithms..
Towards Detecting and Classifying Network Intrusion Traffic Using Deep Learning Frameworks
Recent breakthroughs in deep learning algorithms have enabled researchers and practitioners to make significant progress in various hard computer science problems and applications from computer vision and perception, natural language processing and interpretation to complex reasoning tasks such as playing board games (e.g., Go, Chess, etc.) and even overthrowing human champions. Considering the expected acceleration and increase in computer threats, in this article, we explore the utility and capability of deep learning algorithms in the important area of network intrusion detection. We apply and compare various state-of-the-art deep learning frameworks (e.g., Keras, TensorFlow, Theano, fast.ai, and PyTorch) in detecting network intrusion traffic and also in classifying common network attack types using the recent CSE-CIC-IDS2018 dataset. Experimental results show that fast.ai, a highly opinionated wrapper for PyTorch, provided the highest accuracy of about 99% with low false positive and negative rates in both detecting and classifying various intrusion types. Our results provide evidence of the utility of various deep learning frameworks detecting network intrusion traffic.
Efficient Early Anomaly Detection of Network Security Attacks Using Deep Learning
2023
We present a deep-learning (DL) anomaly-based Intrusion Detection System (IDS) for networked systems, which is able to detect in real-time anomalous network traffic corresponding to security attacks while they are ongoing. Compared to similar approaches, our IDS does not require a fixed number of network packets to analyze in order to make a decision on the type of traffic and it utilizes a more compact neural network which improves its real-time performance. As shown in the experiments using the CICIDS2017 and USTC-TFC-2016 datasets, the approach is able to detect anomalous traffic with high precision and recall. In addition, the approach is able to classify the network traffic by using only a very small portion of the network flows.
Early Detection of Network Attacks Using Deep Learning
2022
The Internet has become a prime subject to security attacks and intrusions by attackers. These attacks can lead to system malfunction, network breakdown, data corruption or theft. A network intrusion detection system (IDS) is a tool used for identifying unauthorized and malicious behavior by observing the network traffic. State-of-the-art intrusion detection systems are designed to detect an attack by inspecting the complete information about the attack. This means that an IDS would only be able to detect an attack after it has been executed on the system under attack and might have caused damage to the system. In this paper, we propose an end-to-end early intrusion detection system to prevent network attacks before they could cause any more damage to the system under attack while preventing unforeseen downtime and interruption. We employ a deep neural networkbased classifier for attack identification. The network is trained in a supervised manner to extract relevant features from raw network traffic data instead of relying on a manual feature selection process used in most related approaches. Further, we introduce a new metric, called earliness, to evaluate how early our proposed approach detects attacks. We have empirically evaluated our approach on the CICIDS2017 dataset. The results show that our approach performed well and attained an overall 0.803 balanced accuracy.
SecureNet: Network Intrusion Detection using Machine Learning and Deep Learning Techniques
International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2024
In the ever-evolving landscape of cybersecurity, the need for robust intrusion detection systems has become paramount. This paper introduces a cutting-edge intrusion detection algorithm designed to enhance network security through the integration of advanced machine learning and deep learning methodologies. The proposed algorithm capitalizes on the strengths of both paradigms to achieve a comprehensive and adaptive approach to identifying malicious activities within a network. This research focuses on enhancing network security through the development and evaluation of a novel intrusion detection system leveraging both deep learning and traditional machine learning approaches. Utilizing the NSL-KDD dataset, we employ the Long Short-Term Memory (LSTM) model, a superior version of Recurrent Neural Networks (RNNs), and the K-Nearest Neighbors (KNN) algorithm for binary and multi-class classification of network intrusion anomalies. The LSTM model excels in capturing temporal dependencies, enabling the detection of nuanced sequential patterns, while the KNN algorithm contributes to a comprehensive classification framework. Experimental results demonstrate the effectiveness of the hybrid methodology, showcasing improved accuracy, precision, and recall compared to traditional methods. This research underscores the potential of integrating deep learning and classical machine learning techniques to bolster the capabilities of intrusion detection systems in safeguarding against evolving cyber threats.
Deep Learning for Cyber Security Intrusion Detection: Approaches, Datasets, and Comparative Study
Journal of Information Security and Applications , 2020
In this paper, we present a survey of deep learning approaches for cyber security intrusion detection, the datasets used, and a comparative study. Specifically, we provide a review of intrusion detection systems based on deep learning approaches. The dataset plays an important role in intrusion detection, therefore we describe 35 well-known cyber datasets and provide a classification of these datasets into seven categories; namely, network traffic-based dataset, electrical network-based dataset, internet traffic-based dataset, virtual private network-based dataset, android apps-based dataset, IoT traffic-based dataset, and internet-connected devices-based dataset. We analyze seven deep learning models including recurrent neural networks, deep neural networks, restricted Boltzmann machines, deep belief networks, convolutional neural networks, deep Boltzmann machines , and deep autoencoders. For each model, we study the performance in two categories of classification (binary and multiclass) under two new real traffic datasets, namely, the CSE-CIC-IDS2018 dataset and the Bot-IoT dataset. In addition, we use the most important performance indicators, namely, accuracy, false alarm rate, and detection rate for evaluating the efficiency of several methods.