PAMELA: Pattern Matching Engine with Limited-Time Update for NIDS/NIPS (original) (raw)

Fast and Scalable Pattern Matching for Network Intrusion Detection Systems

IEEE Journal on Selected Areas in Communications, 2000

High-speed packet content inspection and filtering devices rely on a fast multi-pattern matching algorithm which is used to detect predefined keywords or signatures in the packets. Multi-pattern matching is known to require intensive memory accesses and is often a performance bottleneck. Hence specialized hardware-accelerated algorithms are required for line-speed packet processing. We present hardware-implementable pattern matching algorithm for content filtering applications, which is scalable in terms of speed, the number of patterns and the pattern length. Our algorithm is based on a memory efficient multi-hashing data structure called Bloom filter. We use embedded on-chip memory blocks in FPGA/VLSI chips to construct Bloom filters which can suppress a large fraction of memory accesses and speed up string matching. Based on this concept, we first present a simple algorithm which can scan for several thousand short (up to 16 bytes) patterns at multi-gigabit per second speeds with a moderately small amount of embedded memory and a few mega bytes of external memory. Furthermore, we modify this algorithm to be able to handle arbitrarily large strings at the cost of a little more on-chip memory. We demonstrate the merit of our algorithm through theoretical analysis and simulations performed on Snort's string set.

Massively Parallel Cuckoo Pattern Matching Applied for NIDS/NIPS

2010

This paper describes a Cuckoo-based Pattern Matching (CPM) engine based on a recently developed hashing algorithm called Cuckoo Hashing. We implement the improved parallel Cuckoo Hashing suitable for hardware-based multipattern matching with arbitrary length. CPM can rapidly update the static pattern set without reconfiguration while consuming the lowest amount of hardware. With the power of massively parallel processing, the speedup of CPM is up to 128X as compared with serial Cuckoo implementation. Compared to other hardware systems, CPM is far better in performance and saves 30% of the area.

High-throughput linked-pattern matching for intrusion detection systems

Proceedings of the 2005 symposium on Architecture for networking and communications systems - ANCS '05, 2005

This paper presents a hardware architecture for highly efficient intrusion detection systems. In addition, a software tool for automatically generating the hardware is presented. Intrusion detection for network security is a compute-intensive application demanding high system performance. By moving both the string matching and the linking of multi-part rules to hardware, our architecture leaves the host system free for higher-level analysis. The tool automates the creation of efficient Field Programmable Gate Array architectures (FPGA). The generated hardware allows an FPGAbased system to perform deep-packet inspection of streams at up to 10 Gb/s line rates at a high level of area efficiency. Going beyond previous basic string-matching implementations that offer only single-string matching, the architecture provides support for rules requiring complex, linked (correlated-content) constructions. This allows most Snort content-linking extensions including 'distance' and 'within' bounding restrictions.

Hardware Efficient Pattern Matching Algorithms and Architectures for Fast Intrusion Detection

NCSU PHD Dissertation, 2006

Intrusion detection processors are becoming a predominant feature in the field of network hardware. As demand on more network speed increases and new network protocols emerge, network intrusion detection systems are increasing in importance and are being integrated in network processors. Currently, most intrusion detection systems are software running on a general purpose processor. Unfortunately, it is becoming increasingly difficult for software based intrusion detection systems to keep up with increasing network speeds (OC192 and 10Gbps at backbone networks). Signature-based intrusion detection systems monitor network traffic for security threats by scanning packet payloads for attack signatures. Intrusion detection systems have to run at wire speed and need to be configurable to protect against emerging attacks. This dissertation describes the concept, structure and algorithms for a special purpose hardware accelerator designed to meet those demands. We consider the problem of string matching which is the most computationally intensive task in intrusion detection. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software intrusion detection systems. A hardware algorithm for efficient data storage and fast retrieval is used to compress, store and retrieve attack signatures. Our algorithms reduce the size of the rules to fit on chip and enables intrusion detection to run at line rates and faster.

Efficient hardware support for pattern matching in network intrusion detection

2010

Deep packet inspection forms the backbone of any Network Intrusion Detection (NID) system. It involves matching known malicious patterns against the incoming traffic payload. Pattern matching in software is prohibitively slow in comparison to current network speeds. Due to the high complexity of matching, only FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit) platforms can provide efficient solutions. FPGAs facilitate target architecture specialization due to their field programmability. Costly ASIC designs, on the other hand, are normally resilient to pattern updates. Our FPGA-based solution performs high-speed pattern matching while permitting pattern updates without resource reconfiguration. To its advantage, our solution can be adopted by software and ASIC realizations, however at the expense of much lower performance and higher price, respectively. Our solution permits the NID system to function while pattern updates occur. An off-line optimization method first finds common subpatterns across all the patterns in the SNORT database of signatures . A novel technique then compresses each pattern into a bit vector, where each bit represents such a sub-pattern. This approach reduces drastically the required on-chip storage as well as the complexity of pattern matching. The bit vectors for newly discovered patterns can be generated easily using a simple high-level language program before storing them into the on-chip RAM. Compared to earlier approaches, not only is our strategy very efficient while supporting runtime updates but it also results in impressive area savings; it utilizes just 0.052 logic cells for processing and 17.77 bits for storage per character in the current SNORT database of 6455 patterns. Also, the total number of logic cells for processing the traffic payload does not change with pattern updates.

High performance string matching algorithm for a network intrusion prevention system (NIPS

2006

Intrusion Detection systems (IDS) were developed to identify and report attacks in the late 1990s, as hacker attacks and network worms began to affect the internet. Traditional IDS technologies detect hostile traffic and send alerts but do nothing to stop the attacks. Network Intrusion Prevention Systems (NIPS) are deployed in-line with the network segment being protected. As the traffic passes through the NIPS, it is inspected for the presence of an attack. Like viruses, most intruder activities have some sort of signatures. Therefore, a pattern-matching algorithm resides at the heart of the NIPS. When an attack is identified, the NIPS blocks the offending data. There is an alleged trade-off between the accuracy of detection and algorithmic efficiency. Both are paramount in ensuring that legitimate traffic is not delayed or disrupted as it flows through the device. For this reason, the pattern-matching algorithm must be able to operate at wire speed, while simultaneously detecting the main bulk of intrusions. With networking speeds doubling every year, it is becoming increasingly difficult for software based solutions to keep up with the line rates. This paper presents a novel pattern-matching algorithm. The algorithm uses a Ternary Content Addressable Memory (TCAM) and is capable of matching multiple patterns in a single operation. The algorithm achieves line-rate speed of several orders of magnitude faster than current works, while attaining similar accuracy of detection. Furthermore, our system is fully compatible with Snort's rules syntax, which is the de facto standard for intrusion prevention systems.

Piranha: Fast and Memory-Efficient Pattern Matching for Intrusion Detection

IFIP Advances in Information and Communication Technology, 2005

Network Intrusion Detection Systems (NIDS) provide an important security function to help defend against network attacks. As network speeds and detection workloads increase, it is important for NIDSes to be highly efficient. Most NIDSes need to check for thousands of known attack patterns in every packet, making pattern matching the most expensive part of signature-based NIDSes in terms of processing and memory resources. This paper describes Piranha, a new algorithm for pattern matching tailored specifically for intrusion detection. Piranha is based on the observation that if the rarest substring of a pattern does not appear, then the whole pattern will definitely not match. Our experimental results, based on traces that represent typical NIDS workloads, indicate that Piranha can enhance the performance of a NIDS by 11% to 28% in terms of processing time and by 18% to 73% in terms of memory usage compared to existing NIDS pattern matching algorithms.

A Systemfor High Throughput Performanceand Reduce Low Memoryusing Pattern Matching with Hash Key

2014

Pattern matching is one of the most critical elements because it allows for the system to make decisions based not just on the headers, but the actual content flowing through the network. Network Intrusion detection and Prevention systems have emerged as one of the most effective ways of providing security to those connected to the network, and at the heart of almost every modern intrusion detection system is a pattern matching algorithm. I have developed an approach that relies on a special purpose architecture that executes novel pattern matching algorithms specially optimized for implementation in our design.

Configurable string matching hardware for speedup up intrusion detection

2004

Systems (IDSs) monitor network traffic for security threats by scanning packet payloads for attack signatures. IDSs have to run at wire speed and need to be configurable to protect against emerging attacks. In this paper we consider the problem of string matching which is the most computationally intensive task in IDS. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software IDSs. Our preliminary results suggest that the hardware accelerator offers an overall system performance of up to 14Gbps. Index Terms—Intrusion detection, Snort accelerator, string matchin