mDFA: A Memory Efficient DFA-Based Pattern Matching Engine on FPGA (original) (raw)
Related papers
A memory-based NFA regular expression match engine for signature-based intrusion detection
2013
Signature-based intrusion detection Regular expression matching Non-deterministic finite automaton Memory-based architecture a b s t r a c t Signature-based intrusion detection is required to inspect network traffic at wire-speed. Matching packet payloads against patterns specified with regular expression is a computation intensive task. Hence, the design of hardware accelerator to speed up regular expression matching has been an active research area. A systematic approach to detect regular expression is based on finite automaton. The space-time tradeoff between deterministic finite automaton (DFA) and non-deterministic finite automaton (NFA) is wellknown. DFA can offer constant throughput but it may suffer from the state explosion problem. Hence, implementation of DFA for large pattern sets on embedded device with limited on-chip memory may not be viable. NFA requires linear space but the throughput can be very low. Implementations of NFA with hardwired circuits can overcome the speed deficiency by exploiting the massive parallelism offered by dedicated hardware circuitries, but this approach does not support efficient dynamic updates. In this paper, we shall present a memory-based architecture for the implementation of NFA to speed up regular expression matching for signature-based intrusion detection. The proposed method supports dynamic updates and offers constant throughput so that it can be used to supplement the existing DFA-based methods in handling large pattern sets.
2007
Network Intrusion Detection Systems (NIDS) are more and more important for identifying and preventing the malicious attacks over the network. This paper proposes a novel cost-effective high speed pattern matching algorithm (named MSH) for NIDS. By applying the characteristics of magic states, a new observation from the deterministic finite state automata (DFA), the proposed MSH constructs a tiny data structure which can be stored into the on-chip memory of modern cost effective FPGA. Prototype and experimental results show the overall efficiency of the proposed MSH is at least 7 times faster than that of the baseline model. The MSH enables the design of cost effective FPGA-based accelerator to furnish over 1Gbps throughput. It can also be scaled to multi-gigabit and realized on various silicon implementations.
A Memory-Efficient and Modular Approach for String Matching on FPGAs
2010
In Network Intrusion Detection Systems (NIDSs), string matching demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency 1 . Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-ofthe-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching (MASM), based on pipelined binary search tree. With memory efficiency close to 1 byte/char, MASM can support a dictionary 2 of over 4 MBytes, using a single FPGA device. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over 8 MBytes. Our implementation results show a sustained throughput of 3.5 Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.
ENREM: An efficient NFA-based regular expression matching engine on reconfigurable hardware for NIDS
Journal of Systems Architecture, 2013
Regular expression is a critical mechanism in modern network security and widely used in network intrusion detection system to describe malicious patterns. In order to speed up the pattern matching process, a number of studies have been investigated to implement regular expression matching on reconfigurable hardware. Several optimizations have been proposed, however the problem of sharing sub-patterns between multiple regular expressions is not solved completely. In this paper we present ENREM, an Efficient NFA-based Regular Expression Matching Engine on reconfigurable hardware. We introduce a new infix and suffix sharing architecture and employ it along with several techniques to optimize the required area of pattern matching circuits. In addition we developed tools for automatically generating the Verilog HDL source code of ENREM circuit from any given set of Perl compatible regular expression patterns. In order to evaluate proposed architecture, we exploit Snort rules and implement ENREM on Xilinx Virtex-II Pro XC2VP-50 FPGA. The system is tested on NetFPGA platform with DARPA intrusion detection as input data to verify the accuracy of circuit. The experimental results show that ENREM can reduce 42% LUTs and 32% FlipFlops compared with previous approaches while maintains high-speed matching throughput from 1.45 to 2.35 Gbps.
Time and area efficient pattern matching on FPGAs
Proceeding of the 2004 ACM/SIGDA 12th international symposium on Field programmable gate arrays - FPGA '04, 2004
Pattern matching for network security and intrusion detection demands exceptionally high performance. Much work has been done in this field, and yet there is still significant room for improvement in efficiency, flexibility, and throughput. We develop a novel linear-array string matching architecture using a buffered, two-comparator variation on the Knuth-Morris-Pratt(KMP) algorithm. For small (16 or fewer characters) patterns, it compares favorably with the state-of-the-art while providing better scalability and reconfiguration, and more efficient hardware utilization. KMP is a well-known, efficient string matching technique using a single comparator and a precomputed transition table. We add a second comparator and an input buffer, allowing the system to accept at least one character in each cycle and terminate after a number of clock cycles at maximum equal to the length of the input string plus the size of the buffer. The system also provides a clean, modular route to reconfiguring the patterns on-the-fly and scaling the system to support more units, using several rows of linear array elements. In this paper, we prove the bound on the buffer size and running time, and provide performance comparisons against other approaches.
An Evaluation of FPGA-based IDS Pattern Matching Techniques
2005
Abstract Pattern matching is one of the most computationally intensive tasks in network security systems. Numerous pattern matching approaches have been proposed in the past. The most common ones use: regular expressions, discrete comparators or CAM, Pre-decoding, and Hashing to match patterns. The researchers' first concern was to achieve high operating throughput in order to process incoming packets in wire rates.
Enhanced FPGA-based architecture for regular expression matching in NIDS
Perl Compatible Regular Expression (PCRE) is increasingly used in Network Intrusion Detection System due to its efficiency. However, there are many issues that have not been completely solved for PCRE matching on hardware platform. In this paper, we propose an FPGA-based PCRE matching architecture that effectively improves the constraint repetition, an importance feature of PCRE. We enhance our architecture to handle mflag which is a powerful PCRE modifier. Besides, a toolchain for auto-generating PCRE matching engine is also implemented. Our experimental results on low-cost Altera Cyclone II chip shows that our architecture can achieve throughput up to 1Gbps and save up to 92.74% hardware resource as compared with the conventional architecture.
BBFex: a bloom-bloomier filter extension for long patterns in FPGA-based pattern matching system
FPGA '11 Proceedings of the 19th ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2011
There are many pattern matching engines in Network Intrusion Detection Systems (NIDS) have been developed on FPGA-based platforms to accelerates the performance of pattern matching process in order to keep up with the gradually increasing in speed of current networks. However, those systems only support small number of short patterns which are not appropriate to large database such as Clam Antivirus patterns. In this paper, we propose Bloom-Bloomier Filter Extension (BBFex) as a practical pattern matching engine that handles large various-length pattern database. The basic idea in designing BBFex is the combination of Bloom Filter and Bloomier Filter to index patterns and an efficient pattern fragmenting method to split and to merge long patterns. Therefore, BBFex can recognize nearly 84,000 Clam Antivirus static patterns of which lengths vary from 4 to 255 characters with rather low on chip memory density, approximately 0.4 bits per character while keeping the off-chip memory access rate 5X lower compared to previous similar system and achieving throughput of 1.36 Gbps. In addition, BBFex is not only limited to Clam Antivirus database because its architecture is designed in respect to general character-based database. Moreover, as a hash-based system, BBFex does not require entire system reconfiguration when updating database. (The attached file is a draft of the work)
ECEB: Enhanced Constraint Repetition Block for Regular Expression Matching on FPGA
Recent Network Intrusion Detection Systems (NIDSs) utilize Perl Compatible Regular Expression to describe malicious patterns existing in the content payload of packets more and more efficiently. Several techniques are introduced to optimize the performance or complete the system for full support all of PCRE features in hardware platform, but some issues have just been solved partially. Constraint Repetition is among important characteristics of PCRE but its effective hardware-based implementation solutions are still limited. This paper describes an Enhanced Constraint rEpetition Block (ECEB) for regular expression matching engine in FPGA. To support more PCRE features, we improve our implementation to handle flag ’m’ modifier. We also implement the block memory (BRAM) based character matching which saves a lot of LUTs and effectively improves overall system’s throughput compared to related techniques. A software tool-chain for auto-generating PCRE matching system is also introduced. We do experiments on low-cost XC2VP50 Xilinx Virtex II Pro chip with the rule set of SNORT, an open source NIDS, to evaluate our implementation. The results verify that our architecture can achieve throughput up to 1Gbps and save up to 90% hardware resources compared with the conventional architecture