Massively Parallel Cuckoo Pattern Matching Applied for NIDS/NIPS (original) (raw)
Related papers
PAMELA: Pattern Matching Engine with Limited-Time Update for NIDS/NIPS
Ieice Transactions, 2009
Several hardware-based pattern matching engines for network intrusion/prevention detection systems (NIDS/NIPSs) can achieve high throughput with less hardware resources. However, their flexibility to update new patterns is limited and still challenging. This paper describes a PAttern Matching Engine with Limited-time updAte (PAMELA) engine using a recently proposed hashing algorithm called Cuckoo Hashing. PAMELA features on-the-fly pattern updates without reconfiguration, more efficient hardware utilization, and higher performance compared with other works. First, we implement the improved parallel exact pattern matching with arbitrary length based on Cuckoo Hashing and linkedlist technique. Second, while PAMELA is being updated with new attack patterns, both stack and FIFO are utilized to bound insertion time due to the drawback of Cuckoo Hashing and to avoid interruption of input data stream. Third, we extend the system for multi-character processing to achieve higher throughput. Our engine can accommodate the latest Snort rule-set, an open source NIDS/NIPS, and achieve the throughput up to 8.8 Gigabit per second while consuming the lowest amount of hardware. Compared to other approaches, ours is far more efficient than any other implemented on Xilinx FPGA architectures.
High-speed data deduplication using parallelized cuckoo hashing
TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES
Data deduplication is a capacity optimization technology used in backup systems for identifying and storing the nonredundant data blocks. The CPU intensive tasks involved in a hash-based deduplication system remain as challenges in improving the performance of the system. In this paper, we propose a parallel variant of the standard cuckoo hashing that enables the hashing technique to be performed in parallel. The CPU intensive tasks of fingerprint insertion and lookup operations are performed in parallel and distributed among the nodes of the deduplication cluster. Furthermore, the uniform handling of the blocks by the cluster nodes involved in the process of duplicate identification provides good load balance. Experimental evaluations using real-world backup and Linux kernel data sets reveal that the proposed deduplication system achieves up to 100% higher backup speed, up to 28% reduced lookup latency, and up to 24% reduced backup time than the other deduplication systems.
One Algorithm to Match Them All: On a Generic NIPS Pattern Matching Algorithm
2007
Today's Network Intrusion Prevention Systems (NIPS) provide an important defense mechanism against security threats. The detection of network attacks utilizes a highspeed pattern matching algorithm that can be implemented in either hardware or software. Adapting a software-based pattern matching algorithm to an hardware-based device is a complicated task. This paper presents a cost effective multi-pattern matching algorithm based on Field Programmable Gate Arrays (FPGAs) and standard RAM. The algorithm achieves line-rate speed of several orders of magnitude faster than the current state of the art, while attaining similar accuracy of detection. The algorithm can be easily adapted to operate in hardware-based NIPS and attain even higher line-speed by utilizing a TCAM memory. Several common software based algorithms can be found in [4], [5], [6], [7], [8], [9], [10], [11] and [12].
A Systemfor High Throughput Performanceand Reduce Low Memoryusing Pattern Matching with Hash Key
2014
Pattern matching is one of the most critical elements because it allows for the system to make decisions based not just on the headers, but the actual content flowing through the network. Network Intrusion detection and Prevention systems have emerged as one of the most effective ways of providing security to those connected to the network, and at the heart of almost every modern intrusion detection system is a pattern matching algorithm. I have developed an approach that relies on a special purpose architecture that executes novel pattern matching algorithms specially optimized for implementation in our design.
Hashing+ memory= low cost, exact pattern matching
2005
Abstract In this paper we propose the combination of hashing and use of memory to achieve low cost, exact matching of SNORT-like intrusion signatures. The basic idea is to use hashing to generate a distinct address for each candidate pattern, which is stored in memory. Our implementation, hash-mem, uses simple CRC-style polynomials implemented with XOR gates, to achieve low cost hashing of the input patterns.
A Multi-dimensional Progressive Perfect Hashing for High-Speed String Matching
2011 ACM/IEEE Seventh Symposium on Architectures for Networking and Communications Systems, 2011
Aho-Corasick (AC) automaton is widely used for multi-string matching in today's Network Intrusion Detection System (NIDS). With fast-growing rule sets, implementing AC automaton with a small memory without sacrificing its performance has remained challenging in NIDS design. In this paper, we propose a multidimensional progressive perfect hashing algorithm named P 2 -Hashing, which allows transitions of an AC automaton to be placed in a compact hash table without any collision. P 2 -Hashing is based on the observation that a hash key of each transition consists of two dimensions, namely a source state ID and an input character. When placing a transition in a hash table and causing a collision, we can change the value of a dimension of the hash key to rehash the transition to a new location of the hash table. For a given AC automaton, P 2 -Hashing first divides all the transitions into many small sets based on the two-dimensional values of the hash keys, and then places the sets of transitions progressively into the hash table until all are placed. Hash collisions that occurred during the insertion of a transition will only affect the transitions in the same set. The proposed P 2 -Hashing has many unique properties, including fast hash index generation and zero memory overhead, which are very suitable for the AC automaton operation. The feasibility and performance of P 2 -Hashing are investigated through simulations on the full Snort (6.4k rules) and ClamAV (54k rules) rule sets, each of which is first converted to a single AC automaton. Simulation results show that P 2 -Hashing can successfully construct the perfect hash table even when the load factor of the hash table is as high as 0.91.
2007
Network Intrusion Detection Systems (NIDS) are more and more important for identifying and preventing the malicious attacks over the network. This paper proposes a novel cost-effective high speed pattern matching algorithm (named MSH) for NIDS. By applying the characteristics of magic states, a new observation from the deterministic finite state automata (DFA), the proposed MSH constructs a tiny data structure which can be stored into the on-chip memory of modern cost effective FPGA. Prototype and experimental results show the overall efficiency of the proposed MSH is at least 7 times faster than that of the baseline model. The MSH enables the design of cost effective FPGA-based accelerator to furnish over 1Gbps throughput. It can also be scaled to multi-gigabit and realized on various silicon implementations.
Journal of Computer Science and Engineering (JCSE)
The strategy of packing several data values in a single computer word and refreshing them all in a solitary operation is referred to bit parallelism. It assumes a significant part in pattern matching because it can handle in parallel the length of pattern sizes. In this paper, an Improved Pattern Matching model (IPM) proposed, which makes searching process quicker and decreases how much memory used in processing input data. C# was used for the development of the model. With a computer word size of 64bits and pattern length ranging from 8 characters to 72 characters, the system decides how much memory is used. The developed model was evaluated and contrasted with the existing model using 64bits computer word size (cws) and the pattern length of 72 characters. The assessment showed that the IPM had minimal worth of MU contrasted with the existing model (BNDM, SBNDM, and FSBNDM). This IPM model can be embraced for improvement of the size of string data stored in computer word because o...
High Speed Low Area Pattern Matching Algorithm for Memory Architecture.
International Journal of Engineering Sciences & Research Technology, 2013
Pattern matching is one of the most important components for the content inspection based applications of network security, and it requires well designed algorithms and architectures to keep up with the increasing network speed. Due to the advantages of easy re-configurability and scalability, the memory-based string matching architecture is widely adopted by network intrusion detection systems (NIDS). In order to accommodate the increasing number of attack patterns and meet the throughput requirement of networks, a successful NIDS system must have a memory-efficient pattern-matching algorithm and hardware design. In this paper, we propose a memory-efficient pattern-matching algorithm which can significantly reduce the memory requirement. For Snort rule sets, the new algorithm achieves 21% of memory reduction compared with the traditional Aho-Corasick algorithm. In addition, we can gain 24% of memory reduction by integrating our approach to the bit-split algorithm which is the state of the art memory-based approach.