An Evaluation of FPGA-based IDS Pattern Matching Techniques (original) (raw)
Related papers
Time and area efficient pattern matching on FPGAs
Proceeding of the 2004 ACM/SIGDA 12th international symposium on Field programmable gate arrays - FPGA '04, 2004
Pattern matching for network security and intrusion detection demands exceptionally high performance. Much work has been done in this field, and yet there is still significant room for improvement in efficiency, flexibility, and throughput. We develop a novel linear-array string matching architecture using a buffered, two-comparator variation on the Knuth-Morris-Pratt(KMP) algorithm. For small (16 or fewer characters) patterns, it compares favorably with the state-of-the-art while providing better scalability and reconfiguration, and more efficient hardware utilization. KMP is a well-known, efficient string matching technique using a single comparator and a precomputed transition table. We add a second comparator and an input buffer, allowing the system to accept at least one character in each cycle and terminate after a number of clock cycles at maximum equal to the length of the input string plus the size of the buffer. The system also provides a clean, modular route to reconfiguring the patterns on-the-fly and scaling the system to support more units, using several rows of linear array elements. In this paper, we prove the bound on the buffer size and running time, and provide performance comparisons against other approaches.
mDFA: A Memory Efficient DFA-Based Pattern Matching Engine on FPGA
Wireless Personal Communications, 2014
Security applications such as network intrusion detection system (NIDS) and virus scanning engine utilize pattern matching as an essential mechanism for detecting harmful activities or malicious codes. The increase of pattern set in size and complexity as well as the high demand of scanning data volume make pattern matching task on general purpose processor more challenging. One solution for this issue is employing reconfigurable device, field programmable gate array (FPGA), to offload this time-consuming task. In this paper, we introduce a memory efficient FPGA-based pattern matching architecture. We utilized Deterministic Finite Automata (DFA) as main pattern matching algorithm and propose modifications (mDFA) to reduce redundant logic. The proposed design, with better memory utilization, is capable of dynamic update and compatible to stateful NIDSs and virus scanners. The analysis of memory efficiency and the hardware implementation of proposed architecture are also presented in this paper. We experiment our approach on contemporary NIDS pattern sets and virus signature database and build a prototype using NetFPGA 1G platform to test on real network environment. The results show that our design could save up to 90 % hardware resources as compared to traditional DFA approach and gain a throughput of 1.9 Gbps. The prototype could achieve 2.7-4.5× speed up to software-based matching engine.
Hardware Efficient Pattern Matching Algorithms and Architectures for Fast Intrusion Detection
NCSU PHD Dissertation, 2006
Intrusion detection processors are becoming a predominant feature in the field of network hardware. As demand on more network speed increases and new network protocols emerge, network intrusion detection systems are increasing in importance and are being integrated in network processors. Currently, most intrusion detection systems are software running on a general purpose processor. Unfortunately, it is becoming increasingly difficult for software based intrusion detection systems to keep up with increasing network speeds (OC192 and 10Gbps at backbone networks). Signature-based intrusion detection systems monitor network traffic for security threats by scanning packet payloads for attack signatures. Intrusion detection systems have to run at wire speed and need to be configurable to protect against emerging attacks. This dissertation describes the concept, structure and algorithms for a special purpose hardware accelerator designed to meet those demands. We consider the problem of string matching which is the most computationally intensive task in intrusion detection. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software intrusion detection systems. A hardware algorithm for efficient data storage and fast retrieval is used to compress, store and retrieve attack signatures. Our algorithms reduce the size of the rules to fit on chip and enables intrusion detection to run at line rates and faster.
2011
String matching has become essential for modern computers. It is used in many applications ranging from data mining to network security. A problem is that current general purpose computers are no longer fast enough to deal with the ever increasing amounts of data that are passed though them due to the massive increases in network traffic and data storage capacities offered. This paper aims to demonstrate the significant performance gains that can be achieved by employing string matching algorithms directly on hardware using an FPGA, as opposed to the traditional software-only solution. A possible future FPGA-based string matching board that could be installed in current computers is discussed.
A signature match processor architecture for network intrusion detection
Proceedings - 13th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, FCCM 2005, 2005
In this paper, we introduce a novel architecture for a hardware based network intrusion detection system (NIDS). NIDSs are becoming critical components of the network infrastructure as they serve as a key line of defense in network protection. However, current methods are much too compute intensive and can not begin to meet the bandwidth requirements of a moderate sized corporate network. Thus, hardware techniques are desired to speed up network processing. This paper introduces a FPGA based signature match processor that can serve as the core of a hardware based NIDS. The signature match processor's key feature is a CAM-based cellular processor architecture that can match strings in an area efficient manner. Using a unique binary tree structure, we are also able to generate priority encoded addresses corresponding to multiple signature matches.
Graphics Processor-based High Performance Pattern Matching Mechanism for Network Intrusion Detection
Intrusion Detection Systems, 2011
As high-speed networking technology has progressed, the current network environment comprises many applications. However, many users still feel uncertain about these network applications due to security issues. Intrusion detection and prevention systems (IDS/IPS) are designed to detect and identify diverse threats over the network, such as worms, virus, spyware, and malicious codes, by performing deep packet inspection on packet payloads. Deep packet inspection is used to perform various processing operations in the entire packet, including the header and payload. Therefore, searching keywords in each traffic stream forms a bottleneck. That is, string matching is always an important issue as well as significant challenge in high speed network processing. For instance, Snort (Roesch, 1999), the most famous and popular open source IDS, takes over 2,500 patterns as signatures and takes more than 80% of CPU time for pattern matching. Thus, IDS need an efficient pattern matching algorithm or other mechanisms to speed up this key operation. Otherwise, an under-performing system not only becomes the network bottleneck but also misses some critical attacks. Pattern matching algorithms have been studied for a long time, such algorithms include the Boyer Moore algorithm which solves single-pattern matching problem (Boyer & Moore, 1977) and the Aho-Corasick (AC) (Aho & Corasick, 1975) and Wu-Manber (Wu & Manber, 1994) algorithms, which solve multi-pattern string-matching problems. Research in this field has recently become popular again owing to the requirements for processing packets, especially for deep packet inspection applications. Various new concepts and algorithms have been proposed and implemented, such as Bitmap AC (Tuck et al., 2004), parallel bloom-filter (Dharmapurikar et al., 2004), reconfigure silicon hardware (Moscola et al., 2003) and TCAM-based mechanism (Yu et al., 2004). Implementations of IDS can be categorized into hardware-based approaches and softwarebased approaches. The design concept for data structures and algorithms are usually different for these two implementations. The hardware approach is often used for networkbased IDS, which is usually placed in the entrance of a local area network (LAN) and is responsible for scanning suspicious packets through it. Most of them store the famous Snort signatures, which are the collection of the characteristic of many network attacks, in the database to perform pattern matching. In order to process packets quickly and flexibly, parallel processing is the main architecture employed for network processing. The network www.intechopen.com Intrusion Detection Systems 288 processor (NP) is the most representative of these implementations. However, traditional network processors still suffer from poor performance and high cost when perform deep packet inspection, even though applying network processors for pattern matching has been proposed (Liu et al., 2004). Hence, many network security chip vendors have released special-purpose silicon products for accelerating the work of pattern matching. Nevertheless, such solutions are always expensive because of insufficient sales volume in the market. On the other hand, software-based solutions, such as anti-virus software, personal firewalls, are very popular, especially in personal computers and servers. According to the reports, the security software market in Asia/Pacific (excluding Japan) is expect to grow up to over US1100millionsin2007(IDC,2006,Asia)andthemarketinJapanwillalsoreachUS1100 millions in 2007 (IDC, 2006, Asia) and the market in Japan will also reach US1100millionsin2007(IDC,2006,Asia)andthemarketinJapanwillalsoreachUS1927 million in 2010 (IDC, 2006, Japan), respectively. In terms of software, pattern matching is still necessary to detect network intrusion or to scan suspicious files. Form example, some famous network security software, such as Norton anti-virus, Trend-Micro pc-cillin, and Kaspersky anti-virus, have implemented the intrusion detection component in it. That is, host-based IDS becomes more and more common nowadays. However, the task of pattern matching slows down the system performance significantly because there is no additional hardware for accelerating. The problem is more crucial for servers, which often have to handle hundreds to thousands of connections simultaneously. This study has found that graphics processors could constitute a solution for end hosts to perform pattern matching efficiently. With the parallel nature of graphics processors, the performance of pattern matching is greatly improved, even outperforms some previous hardware solutions. The personal computer has now become a standard consumer electronic device, particularly because of its ability to play PC/TV games, which increasingly require 3D processing. Players now demand for real-time, smooth and vivid frame transition, leading to the rapid development of graphics related technologies. Graphics processors are capable of increasingly powerful computation, even surpassing that of general processors in floating point computation. Developers of games or multimedia can design their own features by programming the graphics processor. This feature also catches the eye of developers of software other than games or graphics. Non-graphics applications using the programming power of graphics processors are called General-Purpose Computations on Graphics Processor Units (GPGPU). This study proposes a novel approach and architecture to speed up pattern matching by using the GPUs. GPU is also capable of processing network traffic of multiple sessions in parallel. The contributions of this study can be summarized as follows: • Generic: The proposed architecture is generic, and can be integrated with other systems accelerating pattern matching, such as network security system or content-intuitive systems. • Economics: The GPUs are commodity and cost-effective. For example, the solution using NVIDIA GeForce 6800GT (NVIDIA GeForce 6800, 2005) costs 1/10 of other silicon solutions with the same performance. • Effective Utilization: In general, the graphics processing subsystem is often idle in a PC. The computation power of GPU is not always fully utilized even when running games and other GPU-consuming applications. Hence, using a GPU to reduce the system load when performing pattern matching computations, such as virus scans or intrusion prevention, or using a GPU as a co-processor, could improve the performance of systems or applications.
2007
Network Intrusion Detection Systems (NIDS) are more and more important for identifying and preventing the malicious attacks over the network. This paper proposes a novel cost-effective high speed pattern matching algorithm (named MSH) for NIDS. By applying the characteristics of magic states, a new observation from the deterministic finite state automata (DFA), the proposed MSH constructs a tiny data structure which can be stored into the on-chip memory of modern cost effective FPGA. Prototype and experimental results show the overall efficiency of the proposed MSH is at least 7 times faster than that of the baseline model. The MSH enables the design of cost effective FPGA-based accelerator to furnish over 1Gbps throughput. It can also be scaled to multi-gigabit and realized on various silicon implementations.
A Memory-Efficient and Modular Approach for String Matching on FPGAs
2010
In Network Intrusion Detection Systems (NIDSs), string matching demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency 1 . Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-ofthe-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching (MASM), based on pipelined binary search tree. With memory efficiency close to 1 byte/char, MASM can support a dictionary 2 of over 4 MBytes, using a single FPGA device. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over 8 MBytes. Our implementation results show a sustained throughput of 3.5 Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.
Enhanced FPGA-based architecture for regular expression matching in NIDS
Perl Compatible Regular Expression (PCRE) is increasingly used in Network Intrusion Detection System due to its efficiency. However, there are many issues that have not been completely solved for PCRE matching on hardware platform. In this paper, we propose an FPGA-based PCRE matching architecture that effectively improves the constraint repetition, an importance feature of PCRE. We enhance our architecture to handle mflag which is a powerful PCRE modifier. Besides, a toolchain for auto-generating PCRE matching engine is also implemented. Our experimental results on low-cost Altera Cyclone II chip shows that our architecture can achieve throughput up to 1Gbps and save up to 92.74% hardware resource as compared with the conventional architecture.
A Methodology for Synthesis of Efficient Intrusion Detection Systems on FPGAs
12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines
Intrusion detection for network security is a computation intensive application demanding high system performance. System level design, a relatively unexplored field in this area, allows more efficient communication and extensive reuse of hardware components for dramatic increases in area-time performance. By applying optimization strategies to the entire database, we reduce hardware requirements compared to architectures designed with single pattern matchers in mind. We present a methodology for system-wide integration of graph-based partitioning of large intrusion detection pattern databases. Integrating ruleset-based graph creation and min-cut partitioning, our methodology allows efficient multi-byte comparisons and partial matches for high performance FPGA-based network security. Through pre-processing, this methodology yields designs with competitive clock frequencies that are a minimum of 8x more area efficient than any other shift-andcompare architectures, and 2x that of other predecoded architectures.