High-throughput linked-pattern matching for intrusion detection systems (original) (raw)
Related papers
Fast and Scalable Pattern Matching for Network Intrusion Detection Systems
IEEE Journal on Selected Areas in Communications, 2000
High-speed packet content inspection and filtering devices rely on a fast multi-pattern matching algorithm which is used to detect predefined keywords or signatures in the packets. Multi-pattern matching is known to require intensive memory accesses and is often a performance bottleneck. Hence specialized hardware-accelerated algorithms are required for line-speed packet processing. We present hardware-implementable pattern matching algorithm for content filtering applications, which is scalable in terms of speed, the number of patterns and the pattern length. Our algorithm is based on a memory efficient multi-hashing data structure called Bloom filter. We use embedded on-chip memory blocks in FPGA/VLSI chips to construct Bloom filters which can suppress a large fraction of memory accesses and speed up string matching. Based on this concept, we first present a simple algorithm which can scan for several thousand short (up to 16 bytes) patterns at multi-gigabit per second speeds with a moderately small amount of embedded memory and a few mega bytes of external memory. Furthermore, we modify this algorithm to be able to handle arbitrarily large strings at the cost of a little more on-chip memory. We demonstrate the merit of our algorithm through theoretical analysis and simulations performed on Snort's string set.
Hardware Efficient Pattern Matching Algorithms and Architectures for Fast Intrusion Detection
NCSU PHD Dissertation, 2006
Intrusion detection processors are becoming a predominant feature in the field of network hardware. As demand on more network speed increases and new network protocols emerge, network intrusion detection systems are increasing in importance and are being integrated in network processors. Currently, most intrusion detection systems are software running on a general purpose processor. Unfortunately, it is becoming increasingly difficult for software based intrusion detection systems to keep up with increasing network speeds (OC192 and 10Gbps at backbone networks). Signature-based intrusion detection systems monitor network traffic for security threats by scanning packet payloads for attack signatures. Intrusion detection systems have to run at wire speed and need to be configurable to protect against emerging attacks. This dissertation describes the concept, structure and algorithms for a special purpose hardware accelerator designed to meet those demands. We consider the problem of string matching which is the most computationally intensive task in intrusion detection. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software intrusion detection systems. A hardware algorithm for efficient data storage and fast retrieval is used to compress, store and retrieve attack signatures. Our algorithms reduce the size of the rules to fit on chip and enables intrusion detection to run at line rates and faster.
Configurable string matching hardware for speedup up intrusion detection
2004
Systems (IDSs) monitor network traffic for security threats by scanning packet payloads for attack signatures. IDSs have to run at wire speed and need to be configurable to protect against emerging attacks. In this paper we consider the problem of string matching which is the most computationally intensive task in IDS. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software IDSs. Our preliminary results suggest that the hardware accelerator offers an overall system performance of up to 14Gbps. Index Terms—Intrusion detection, Snort accelerator, string matchin
Efficient hardware support for pattern matching in network intrusion detection
2010
Deep packet inspection forms the backbone of any Network Intrusion Detection (NID) system. It involves matching known malicious patterns against the incoming traffic payload. Pattern matching in software is prohibitively slow in comparison to current network speeds. Due to the high complexity of matching, only FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit) platforms can provide efficient solutions. FPGAs facilitate target architecture specialization due to their field programmability. Costly ASIC designs, on the other hand, are normally resilient to pattern updates. Our FPGA-based solution performs high-speed pattern matching while permitting pattern updates without resource reconfiguration. To its advantage, our solution can be adopted by software and ASIC realizations, however at the expense of much lower performance and higher price, respectively. Our solution permits the NID system to function while pattern updates occur. An off-line optimization method first finds common subpatterns across all the patterns in the SNORT database of signatures . A novel technique then compresses each pattern into a bit vector, where each bit represents such a sub-pattern. This approach reduces drastically the required on-chip storage as well as the complexity of pattern matching. The bit vectors for newly discovered patterns can be generated easily using a simple high-level language program before storing them into the on-chip RAM. Compared to earlier approaches, not only is our strategy very efficient while supporting runtime updates but it also results in impressive area savings; it utilizes just 0.052 logic cells for processing and 17.77 bits for storage per character in the current SNORT database of 6455 patterns. Also, the total number of logic cells for processing the traffic payload does not change with pattern updates.
Configurable string matching hardware for speeding up intrusion detection
ACM SIGARCH Computer Architecture News, 2005
Systems (IDSs) monitor network traffic for security threats by scanning packet payloads for attack signatures. IDSs have to run at wire speed and need to be configurable to protect against emerging attacks. In this paper we consider the problem of string matching which is the most computationally intensive task in IDS. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software IDSs. Our preliminary results suggest that the hardware accelerator offers an overall system performance of up to 14Gbps.
A Methodology for Synthesis of Efficient Intrusion Detection Systems on FPGAs
12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines
Intrusion detection for network security is a computation intensive application demanding high system performance. System level design, a relatively unexplored field in this area, allows more efficient communication and extensive reuse of hardware components for dramatic increases in area-time performance. By applying optimization strategies to the entire database, we reduce hardware requirements compared to architectures designed with single pattern matchers in mind. We present a methodology for system-wide integration of graph-based partitioning of large intrusion detection pattern databases. Integrating ruleset-based graph creation and min-cut partitioning, our methodology allows efficient multi-byte comparisons and partial matches for high performance FPGA-based network security. Through pre-processing, this methodology yields designs with competitive clock frequencies that are a minimum of 8x more area efficient than any other shift-andcompare architectures, and 2x that of other predecoded architectures.
A Memory-Efficient and Modular Approach for String Matching on FPGAs
2010
In Network Intrusion Detection Systems (NIDSs), string matching demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency 1 . Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-ofthe-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching (MASM), based on pipelined binary search tree. With memory efficiency close to 1 byte/char, MASM can support a dictionary 2 of over 4 MBytes, using a single FPGA device. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over 8 MBytes. Our implementation results show a sustained throughput of 3.5 Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.
Proceedings of the 11th International Conference on Advances in Information Technology, 2020
An intrusion detection system monitors and analyzes all the incoming packets, on a given network, to detect any corresponding vulnerabilities and intrusions. It consists of four major modules: packet capturing, packet decoding, packet preprocessing and string/pattern matching. Among these, the string matching is computationally the most intensive part and a number of hardware architectures/designs have already been proposed to accelerate its performance. Consequently, an exploration of existing hardware architectures for string matching algorithms is critical. This paper identifies the most frequently used string matching algorithms and techniques, utilized for the hardware implementation. Subsequently, an exploration of various hardware architectures is provided for the identified algorithms and techniques. Finally, the implementation details of explored architectures are discussed in terms of the used device, consumed hardware resources, operational clock frequency and throughput.
Novel FPGA-Based Signature Matching for Deep Packet Inspection
Lecture Notes in Computer Science, 2010
Deep packet inspection forms the backbone of any Network Intrusion Detection (NID) system. It involves matching known malicious patterns against the incoming traffic payload. Pattern matching in software is prohibitively slow in comparison to current network speeds. Thus, only FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit) solutions could be efficient for this problem. Our FPGA-based solution performs high-speed matching while permitting pattern updates without resource reconfiguration. An off-line optimization method first finds subpattern similarities across all the patterns in the SNORT database of signatures [17]. A novel technique then compresses each pattern into a bit vector where each bit represents such a sub-pattern. Our approach reduces drastically the required on-chip storage as well as the complexity of matching, utilizing just 0.05 logic cells for processing and 17.74 bits for storage per character in the current SNORT database of 6456 patterns.