Hunter in the Dark: Deep Ensemble Networks for Discovering Anomalous Activity from Smart Networks (original) (raw)
Related papers
Hunter in the Dark: Discover Anomalous Network Activity Using Deep Ensemble Network
2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), 2021
Machine learning (ML)-based intrusion detection systems (IDSs) play a critical role in discovering unknown threats in a large-scale cyberspace. They have been adopted as a mainstream hunting method in many organizations, such as financial institutes, manufacturing companies and government agencies. However, existing designs achieve a high threat detection performance at the cost of a large number of false alarms, leading to alert fatigue. To tackle this issue, in this paper, we propose a neural-network-based defense mechanism named DarkHunter. DarkHunter incorporates both supervised learning and unsupervised learning in the design. It uses a deep ensemble network (trained through supervised learning) to detect anomalous network activities and exploits an unsupervised learning-based scheme to trim off mis-detection results. For each detected threat, DarkHunter can trace to its source and present the threat in its original traffic format. Our evaluations, based on the UNSW-NB15 dataset, show that DarkHunter outperforms the existing ML-based IDSs and is able to achieve a high detection accuracy while keeping a low false positive rate. Index Terms-Network intrusion detection, ensemble learning, neural networks, deep learning, machine learning. • We develop a deep ensemble neural network, Ensem-bleNet, for efficient threat detection. Unlike the traditional ensemble designs, which are mainly based on simple and weak ML models, our ensemble design is constructed with the DNN models so that the high learning potential of DNN can be utilized for good detection performance.
A Deep Learning Ensemble for Network Anomaly and Cyber-Attack Detection
Sensors
Currently, expert systems and applied machine learning algorithms are widely used to automate network intrusion detection. In critical infrastructure applications of communication technologies, the interaction among various industrial control systems and the Internet environment intrinsic to the IoT technology makes them susceptible to cyber-attacks. Given the existence of the enormous network traffic in critical Cyber-Physical Systems (CPSs), traditional methods of machine learning implemented in network anomaly detection are inefficient. Therefore, recently developed machine learning techniques, with the emphasis on deep learning, are finding their successful implementations in the detection and classification of anomalies at both the network and host levels. This paper presents an ensemble method that leverages deep models such as the Deep Neural Network (DNN) and Long Short-Term Memory (LSTM) and a meta-classifier (i.e., logistic regression) following the principle of stacked ge...
Smart Intrusion Detection System Comprised of Machine Learning and Deep Learning
European Journal of Engineering Research and Science, 2020
In the present world, digital intruders can exploit the vulnerabilities of a network and are capable to collapse even a country. Attack in Estonia by digital intruders, attack in Iran's nuclear plant and intrusion of spyware in smart phone depicts the efficiency of attackers. Furthermore, centralized firewall system is not enough for ensuring a secured network. Hence, in the age of big data, where availability of data is huge and computation capability of PC is also high, there machine learning and network security have become two inseparable issues. In this thesis, KDD Cup’99 intrusion detection dataset is used. Total 3, 11,030 numbers of records with 41 features are available in the dataset. For finding the anomalies of the network four machine learning methods are used like Classification and Regression Tree (CART), Random Forest, Naive Bayes and Multi-Layer Perception. Initially all 41 features are used to find out the accuracy. Among all the methods, Random Forest provides ...
Deep Learning Approaches for Network Intrusion Detection
MS Thesis, 2019
As the scale of cyber attacks and volume of network data increases exponentially, organizations must develop new ways of keeping their networks and data secure from the dynamic nature of evolving threat actors. With more security tools and sensors being deployed within the modern day enterprise network, the amount of security event and alert data being generated continues to increase, making it more difficult to find the needle in the haystack. Organizations must rely on new techniques to assist and augment human analysts when dealing with the monitoring, prevention, detection, and response to cybersecurity events and potential attacks on their networks. The focus for this Thesis is on classifying network traffic flows as benign or malicious. The contribution of this work is two-fold. First, a feedforward fully connected Deep Neural Network (DNN) is used to train a Network Intrusion Detection System (NIDS) via supervised learning. Second, an autoencoder is used to detect and classify attack traffic via unsupervised learning in the absence of labeled malicious traffic. Deep neural network models are trained using two more recent intrusion detection datasets that overcome limitations of other intrusion detection datasets which have been commonly used in the past. Using these more recent datasets, deep neural networks are shown to be highly effective in performing supervised learning to detect and classify modern-day cyber attacks with a high degree of accuracy, high detection rate, and low false positive rate. In addition, an autoencoder is shown to be effective for anomaly detection.
A Method For Network Intrusion Detection Using Deep Learning
Journal of Student Research
In an increasingly digitally reliant world, organizations are facing the ever more challenging problem of how to best defend their digital information and infrastructure. Current non-machine learning methods for detecting network intrusion, like signature-based and anomaly-based algorithms, are slow and unreliable. Signature based detection holds signatures, or known information and warning signs, about a known attack and compares them to the current flow of data. If a signature matches with the network activity, users and network administrators are notified. Anomaly based detection is where the system monitors current network traffic and compares it to a set baseline traffic. Again, if any unusual traffic occurs, members of the network are notified. In this research, new advancements in deep learning algorithms are used to bolster the defenses of digital networks. Neural networks are used to create a multi-class classifier, which will determine whether the network activity is a cer...
Network Intrusion Detection System using Deep Learning
Procedia Computer Science, 2021
The widespread use of interconnectivity and interoperability of computing systems have become an indispensable necessity to enhance our daily activities. Simultaneously, it opens a path to exploitable vulnerabilities that go well beyond human control capability. The vulnerabilities deem cyber-security mechanisms essential to assume communication exchange. Secure communication requires security measures to combat the threats and needs advancements to security measures that counter evolving security threats. This paper proposes the use of deep learning architectures to develop an adaptive and resilient network intrusion detection system (IDS) to detect and classify network attacks. The emphasis is how deep learning or deep neural networks (DNNs) can facilitate flexible IDS with learning capability to detect recognized and new or zero-day network behavioral features, consequently ejecting the systems intruder and reducing the risk of compromise. To demonstrate the model's effectiveness, we used the UNSW-NB15 dataset, reflecting real modern network communication behavior with synthetically generated attack activities.
Transactions on Emerging Telecommunications Technologies, 2020
The rapid advances in the internet and communication fields have resulted in a huge increase in the network size and the corresponding data. As a result, many novel attacks are being generated and have posed challenges for network security to accurately detect intrusions. Furthermore, the presence of the intruders with the aim to launch various attacks within the network cannot be ignored. An intrusion detection system (IDS) is one such tool that prevents the network from possible intrusions by inspecting the network traffic, to ensure its confidentiality, integrity, and availability. Despite enormous efforts by the researchers, IDS still faces challenges in improving detection accuracy while reducing false alarm rates and in detecting novel intrusions. Recently, machine learning (ML) and deep learning (DL)-based IDS systems are being deployed as potential solutions to detect intrusions across the network in an efficient manner. This article first clarifies the concept of IDS and then provides the taxonomy based on the notable ML and DL techniques adopted in designing network-based IDS (NIDS) systems. A comprehensive review of the recent NIDS-based articles is provided by discussing the strengths and limitations of the proposed solutions. Then, recent trends and advancements of ML and DL-based NIDS are provided in terms of the proposed methodology, evaluation metrics, and dataset selection. Using the shortcomings of the proposed methods, we highlighted various research challenges and provided the future scope for the research in improving ML and DL-based NIDS.
Efficient Early Anomaly Detection of Network Security Attacks Using Deep Learning
2023
We present a deep-learning (DL) anomaly-based Intrusion Detection System (IDS) for networked systems, which is able to detect in real-time anomalous network traffic corresponding to security attacks while they are ongoing. Compared to similar approaches, our IDS does not require a fixed number of network packets to analyze in order to make a decision on the type of traffic and it utilizes a more compact neural network which improves its real-time performance. As shown in the experiments using the CICIDS2017 and USTC-TFC-2016 datasets, the approach is able to detect anomalous traffic with high precision and recall. In addition, the approach is able to classify the network traffic by using only a very small portion of the network flows.
A Review on Conceptual Model of Cyber Attack Detection and Mitigation Using Deep Ensemble Model
When communication networks and the internet of things are integrated into business control systems, they become more vulnerable to cyber-attacks, which can have disastrous consequences. An Intrusion Detection System is critical for identifying and blocking attacks in IoT networks. As a result, utilizing a unique Classification and Encryption approach, this article offered a novel architecture for attack node mitigation. Design/Methodology/Approach: This study reviews the current status of various cyberattack detection models and their mitigation techniques. The proposed model works so that the system is first trained on the dataset, including the DDoS attack and ransomware components. The model examines if it contains malware from DDoS or Ransomware. When tested, we use trained information or a data set to provide the results on attack existence and what sort of attack we offer the extracted characteristics of the input. When the model identifies the attacker node, it is removed via the BAIT technique from the network. Findings/Result: Recognizing the importance of information security is critical to combating cybercrime and encouraging cyber security. There are numerous tactics, strategies, and equipment currently in use to detect intrusion in a computer network, and continuing research is being conducted to improve their ability to detect intrusion. The basic version of a cyberassault detection and mitigation system using the BRELU-RESNET method was evaluated in this study. Originality/Value: This review-based research article examines the present state of cyberattack detection and mitigation, as well as the research gaps and research goals.
Nanotechnology Perceptions, 2024
The rapidly evolving landscape of cyber threats poses significant challenges to traditional security measures, necessitating more advanced and adaptive approaches to anomaly detection and threat mitigation. This review paper explores innovative hybrid deep learning techniques that aim to address the limitations of existing cybersecurity solutions. Current approaches often struggle with the increasing sophistication of attacks, the expanding attack surface due to Internet of Things (IoT) and cloud adoption, and the overwhelming volume and velocity of network data. Moreover, traditional machine learning models frequently fall short in detecting novel threats, adapting to evolving attack patterns, and providing explainable results—critical factors in effective cybersecurity management. The review covers a spectrum of innovations, including: (1) ensemble methods that improve generalization and robustness against adversarial attacks; (2) hybrid deep learning models that excel in analyzing both spatial and temporal aspects of network behaviour; (3) autoencoder-based anomaly detection integrated with supervised classifiers for improved threat categorization; and (4) reinforcement learning-enhanced systems for dynamic, adaptive defence strategies. We also explore the application of explainable AI techniques to hybrid models, addressing the critical need for interpretability in security decisions