Big data analysis and distributed deep learning for next-generation intrusion detection system optimization (original) (raw)
Related papers
A novel scalable intrusion detection system based on deep learning
International Journal of Information Security, 2020
This paper successfully tackles the problem of processing a vast amount of security related data for the task of network intrusion detection. It employs Apache Spark, as a big data processing tool, for processing a large size of network traffic data. Also, we propose a hybrid scheme that combines the advantages of deep network and machine learning methods. Initially, stacked autoencoder network is used for latent feature extraction, which is followed by several classification-based intrusion detection methods, such as support vector machine, random forest, decision trees, and naive Bayes which are used for fast and efficient detection of intrusion in massive network traffic data. A real time UNB ISCX 2012 dataset is used to validate our proposed method and the performance is evaluated in terms of accuracy, f-measure, sensitivity, precision and time.
Computational Intelligence and Neuroscience, 2022
Keeping computers secure is becoming challenging as networks grow and new network-based technologies emerge. Cybercriminals' attack surface expands with the release of new internet-enabled products. As many cyberattacks a ect businesses' con dentiality, availability, and integrity, network intrusion detection systems (NIDS) show an essential role. Network-based intrusion detection uses datasets like CSE-CIC-IDS2018 to train prediction models. With fourteen types of attacks included, the latest big data set for intrusion detection is available to the public. is work proposes three models, two deep learning convolutional neural networks (CNN), long short-term memory (LSTM), and Apache Spark, to improve the detection of all types of attacks. To reduce the dimensionality, random forests (RF) was employed to select the important features; it gave 19 from 84 features. e dataset is imbalanced; thus, oversampling and undersampling techniques reduce the imbalance ratio. e Apache Spark model produced the best results across all 15 classes, with accuracy as high as 100% for all classes, as seen by the experiments' ndings. For the F1-score, Apache Spark showed the highest results with 1.00 for most classes. e ndings of the three models showed outstanding results for multiclassi cation network intrusion detection.
A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network
Symmetry, 2019
With the rapid advancements of ubiquitous information and communication technologies, a large number of trustworthy online systems and services have been deployed. However, cybersecurity threats are still mounting. An intrusion detection (ID) system can play a significant role in detecting such security threats. Thus, developing an intelligent and accurate ID system is a non-trivial research problem. Existing ID systems that are typically used in traditional network intrusion detection system often fail and cannot detect many known and new security threats, largely because those approaches are based on classical machine learning methods that provide less focus on accurate feature selection and classification. Consequently, many known signatures from the attack traffic remain unidentifiable and become latent. Furthermore, since a massive network infrastructure can produce large-scale data, these approaches often fail to handle them flexibly, hence are not scalable. To address these i...
SecureNet: Network Intrusion Detection using Machine Learning and Deep Learning Techniques
International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2024
In the ever-evolving landscape of cybersecurity, the need for robust intrusion detection systems has become paramount. This paper introduces a cutting-edge intrusion detection algorithm designed to enhance network security through the integration of advanced machine learning and deep learning methodologies. The proposed algorithm capitalizes on the strengths of both paradigms to achieve a comprehensive and adaptive approach to identifying malicious activities within a network. This research focuses on enhancing network security through the development and evaluation of a novel intrusion detection system leveraging both deep learning and traditional machine learning approaches. Utilizing the NSL-KDD dataset, we employ the Long Short-Term Memory (LSTM) model, a superior version of Recurrent Neural Networks (RNNs), and the K-Nearest Neighbors (KNN) algorithm for binary and multi-class classification of network intrusion anomalies. The LSTM model excels in capturing temporal dependencies, enabling the detection of nuanced sequential patterns, while the KNN algorithm contributes to a comprehensive classification framework. Experimental results demonstrate the effectiveness of the hybrid methodology, showcasing improved accuracy, precision, and recall compared to traditional methods. This research underscores the potential of integrating deep learning and classical machine learning techniques to bolster the capabilities of intrusion detection systems in safeguarding against evolving cyber threats.
Network Intrusion Detection System using Deep Learning
Procedia Computer Science, 2021
The widespread use of interconnectivity and interoperability of computing systems have become an indispensable necessity to enhance our daily activities. Simultaneously, it opens a path to exploitable vulnerabilities that go well beyond human control capability. The vulnerabilities deem cyber-security mechanisms essential to assume communication exchange. Secure communication requires security measures to combat the threats and needs advancements to security measures that counter evolving security threats. This paper proposes the use of deep learning architectures to develop an adaptive and resilient network intrusion detection system (IDS) to detect and classify network attacks. The emphasis is how deep learning or deep neural networks (DNNs) can facilitate flexible IDS with learning capability to detect recognized and new or zero-day network behavioral features, consequently ejecting the systems intruder and reducing the risk of compromise. To demonstrate the model's effectiveness, we used the UNSW-NB15 dataset, reflecting real modern network communication behavior with synthetically generated attack activities.
AI-Driven Intrusion Detection Systems:Leveraging Deep Learning for Network Security
Nanotechnology Perceptions, 2024
In order to improve network security, this study investigates the integration of deep learning and artificial intelligence (AI) in the development of advanced intrusion detection systems (IDS). The inadequacy of traditional security methods has been demonstrated by the exponential rise in cyber threats that target complex network systems. Deep learning techniques are used by AI-driven IDS to evaluate large datasets, allowing for the real-time identification and categorisation of normal and deviant behaviour. This paper examines many deep learning approaches, including Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), and Recurrent Neural Networks (RNNs), emphasising how well these methods detect sophisticated attacks, such as advanced persistent threats and zero-day exploits. Furthermore, these systems' performance is assessed using important metrics including recall, accuracy, and precision. The results highlight how deep learning has the ability to transform intrusion detection and hence greatly increase the overall resilience of network security frameworks against changing cyber threats.
Deep Learning for Cyber Security Intrusion Detection: Approaches, Datasets, and Comparative Study
Journal of Information Security and Applications , 2020
In this paper, we present a survey of deep learning approaches for cyber security intrusion detection, the datasets used, and a comparative study. Specifically, we provide a review of intrusion detection systems based on deep learning approaches. The dataset plays an important role in intrusion detection, therefore we describe 35 well-known cyber datasets and provide a classification of these datasets into seven categories; namely, network traffic-based dataset, electrical network-based dataset, internet traffic-based dataset, virtual private network-based dataset, android apps-based dataset, IoT traffic-based dataset, and internet-connected devices-based dataset. We analyze seven deep learning models including recurrent neural networks, deep neural networks, restricted Boltzmann machines, deep belief networks, convolutional neural networks, deep Boltzmann machines , and deep autoencoders. For each model, we study the performance in two categories of classification (binary and multiclass) under two new real traffic datasets, namely, the CSE-CIC-IDS2018 dataset and the Bot-IoT dataset. In addition, we use the most important performance indicators, namely, accuracy, false alarm rate, and detection rate for evaluating the efficiency of several methods.
Implementing a Deep Learning Model for Intrusion Detection on Apache Spark Platform
IEEE Access
Internet evolution produced a connected world with a massive amount of data. This connectivity advantage came with the price of more complex and advanced attacks. Intrusion Detection System (IDS) is an essential component for security in modern networks. The IDS methodology is either signature-based detection or anomaly behavior detection. Recently, researchers adopted Deep Learning (DL) because it has a better performance than traditional machine learning algorithms. The use of DL to produce a model for the IDS may take a long time because of computation complexity and a large number of hyperparameters. Different DL models for IDS on Apache Spark have been implemented in this article. This article uses the famous Network Security Lab-Knowledge Discovery and Data Mining (NSL-KDD) dataset and presents a computation delay comparison between Apache Spark and regular implementation. Moreover, an enhanced model is used to improve attack detection accuracy.
A content-based deep intrusion detection system
International Journal of Information Security
The growing number of Internet users and the prevalence of web applications make it necessary to deal with very complex software and applications in the network. This results in an increasing number of new vulnerabilities in the systems, and leading to an increase in cyber threats and, in particular, zeroday attacks. The cost of generating appropriate signatures for these attacks is a potential motive for using machine learning-based methodologies. Although there are many studies on using learning-based methods for attack detection, they generally use extracted features and overlook raw contents. This approach can lessen the performance of detection systems against contentbased attacks like SQL injection, Cross-site Scripting (XSS), and various viruses. In this work, we propose a framework, called deep intrusion detection (DID) system, that uses the pure content of traffic flows in addition to traffic metadata in the learning and detection phases of a passive DNN IDS. To this end, we deploy and evaluate an offline IDS following the framework using LSTM as a deep learning technique. Due to the inherent nature of deep learning, it can process high dimensional data content and, accordingly, discover the sophisticated relations between the auto extracted features of the traffic. To evaluate the proposed DID system, we use the CIC-IDS2017 and CSE-CIC-IDS2018 datasets. The evaluation met
LSTM deep learning method for network intrusion detection system
International Journal of Electrical and Computer Engineering (IJECE), 2020
The security of the network has become a primary concern for organizations. Attackers use different means to disrupt services, these various attacks push to think of a new way to block them all in one manner. In addition, these intrusions can change and penetrate the devices of security. To solve these issues, we suggest, in this paper, a new idea for Network Intrusion Detection System (NIDS) based on Long Short-Term Memory (LSTM) to recognize menaces and to obtain a long-term memory on them, in order to stop the new attacks that are like the existing ones, and at the same time, to have a single mean to block intrusions. According to the results of the experiments of detections that we have realized, the Accuracy reaches up to 99.98 % and 99.93 % for respectively the classification of two classes and several classes, also the False Positive Rate (FPR) reaches up to only 0,068 % and 0,023 % for respectively the classification of two classes and several classes, which proves that the proposed model is effective, it has a great ability to memorize and differentiate between normal traffic and attacks, and its identification is more accurate than other Machine Learning classifiers. 1. INTRODUCTION Nowadays, the world is experiencing a great revolution in the field of information technology, everybody is exchanging continuously information across the network. This implies the establishment of new tools and mechanisms of prevention and detection, and the strengthening of those that exist, like Network Intrusion Detection System (NIDS), in order to enhance security and protect the network from intrusions. The function of a NIDS is to observe, evaluate and classify traffic transiting through the network, it is based, in advance, on established methods and techniques in order to differentiate between normal and suspicious traffic. Furthermore, attackers are attracted by information and knowledge passing through the network, and to exploit and profit from them, they are forced to overcome obstacles and barriers of security by creating new attacks, and evolving the existing ones. While the current NIDS are not evolutionary, their identification algorithms do not progress to identify automatically new menaces, which pushes us to think about advanced and intelligent methods of detection that can identify new attacks and accompany the progression of the existing ones. Moreover, attacks can be of different types, like DoS (Denial-of-Service) and U2R (User to Root) etc…, this problem of diversity leads us to find a resolution to detect and stop them all in a unique way. Currently, Deep Learning is experiencing huge success in several domains, it is a set of techniques used to recognize objects, extract information hidden in the data, and make predictive analytics [1], one of these methods characterized by its long-term memory is the Long Short-Term Memory (LSTM) [2]. And, to solve the issues cited above, we propose in this paper a new approach for NIDS based on the Deep Learning