Scalable multigigabit pattern matching for packet inspection (original) (raw)
Related papers
Fast and Scalable Pattern Matching for Network Intrusion Detection Systems
IEEE Journal on Selected Areas in Communications, 2000
High-speed packet content inspection and filtering devices rely on a fast multi-pattern matching algorithm which is used to detect predefined keywords or signatures in the packets. Multi-pattern matching is known to require intensive memory accesses and is often a performance bottleneck. Hence specialized hardware-accelerated algorithms are required for line-speed packet processing. We present hardware-implementable pattern matching algorithm for content filtering applications, which is scalable in terms of speed, the number of patterns and the pattern length. Our algorithm is based on a memory efficient multi-hashing data structure called Bloom filter. We use embedded on-chip memory blocks in FPGA/VLSI chips to construct Bloom filters which can suppress a large fraction of memory accesses and speed up string matching. Based on this concept, we first present a simple algorithm which can scan for several thousand short (up to 16 bytes) patterns at multi-gigabit per second speeds with a moderately small amount of embedded memory and a few mega bytes of external memory. Furthermore, we modify this algorithm to be able to handle arbitrarily large strings at the cost of a little more on-chip memory. We demonstrate the merit of our algorithm through theoretical analysis and simulations performed on Snort's string set.
Communications in Computer and Information Science, 2012
The network equipment has capable of inspecting packets in order to discover the worms and virus over the network. Many network users are hacked by attackers through malicious functions are mapped on network applications. Such unauthorized activities are required to delete by deep packet inspection in application layer. The high level network equipment provides in-depth packet inspection through pattern matching in network detection system. By presenting centralized parallel pattern matching algorithm for efficient packet inspection with network processor and coprocessor in order to retrieve the pattern with less time.
Graphics Processor-based High Performance Pattern Matching Mechanism for Network Intrusion Detection
Intrusion Detection Systems, 2011
As high-speed networking technology has progressed, the current network environment comprises many applications. However, many users still feel uncertain about these network applications due to security issues. Intrusion detection and prevention systems (IDS/IPS) are designed to detect and identify diverse threats over the network, such as worms, virus, spyware, and malicious codes, by performing deep packet inspection on packet payloads. Deep packet inspection is used to perform various processing operations in the entire packet, including the header and payload. Therefore, searching keywords in each traffic stream forms a bottleneck. That is, string matching is always an important issue as well as significant challenge in high speed network processing. For instance, Snort (Roesch, 1999), the most famous and popular open source IDS, takes over 2,500 patterns as signatures and takes more than 80% of CPU time for pattern matching. Thus, IDS need an efficient pattern matching algorithm or other mechanisms to speed up this key operation. Otherwise, an under-performing system not only becomes the network bottleneck but also misses some critical attacks. Pattern matching algorithms have been studied for a long time, such algorithms include the Boyer Moore algorithm which solves single-pattern matching problem (Boyer & Moore, 1977) and the Aho-Corasick (AC) (Aho & Corasick, 1975) and Wu-Manber (Wu & Manber, 1994) algorithms, which solve multi-pattern string-matching problems. Research in this field has recently become popular again owing to the requirements for processing packets, especially for deep packet inspection applications. Various new concepts and algorithms have been proposed and implemented, such as Bitmap AC (Tuck et al., 2004), parallel bloom-filter (Dharmapurikar et al., 2004), reconfigure silicon hardware (Moscola et al., 2003) and TCAM-based mechanism (Yu et al., 2004). Implementations of IDS can be categorized into hardware-based approaches and softwarebased approaches. The design concept for data structures and algorithms are usually different for these two implementations. The hardware approach is often used for networkbased IDS, which is usually placed in the entrance of a local area network (LAN) and is responsible for scanning suspicious packets through it. Most of them store the famous Snort signatures, which are the collection of the characteristic of many network attacks, in the database to perform pattern matching. In order to process packets quickly and flexibly, parallel processing is the main architecture employed for network processing. The network www.intechopen.com Intrusion Detection Systems 288 processor (NP) is the most representative of these implementations. However, traditional network processors still suffer from poor performance and high cost when perform deep packet inspection, even though applying network processors for pattern matching has been proposed (Liu et al., 2004). Hence, many network security chip vendors have released special-purpose silicon products for accelerating the work of pattern matching. Nevertheless, such solutions are always expensive because of insufficient sales volume in the market. On the other hand, software-based solutions, such as anti-virus software, personal firewalls, are very popular, especially in personal computers and servers. According to the reports, the security software market in Asia/Pacific (excluding Japan) is expect to grow up to over US1100millionsin2007(IDC,2006,Asia)andthemarketinJapanwillalsoreachUS1100 millions in 2007 (IDC, 2006, Asia) and the market in Japan will also reach US1100millionsin2007(IDC,2006,Asia)andthemarketinJapanwillalsoreachUS1927 million in 2010 (IDC, 2006, Japan), respectively. In terms of software, pattern matching is still necessary to detect network intrusion or to scan suspicious files. Form example, some famous network security software, such as Norton anti-virus, Trend-Micro pc-cillin, and Kaspersky anti-virus, have implemented the intrusion detection component in it. That is, host-based IDS becomes more and more common nowadays. However, the task of pattern matching slows down the system performance significantly because there is no additional hardware for accelerating. The problem is more crucial for servers, which often have to handle hundreds to thousands of connections simultaneously. This study has found that graphics processors could constitute a solution for end hosts to perform pattern matching efficiently. With the parallel nature of graphics processors, the performance of pattern matching is greatly improved, even outperforms some previous hardware solutions. The personal computer has now become a standard consumer electronic device, particularly because of its ability to play PC/TV games, which increasingly require 3D processing. Players now demand for real-time, smooth and vivid frame transition, leading to the rapid development of graphics related technologies. Graphics processors are capable of increasingly powerful computation, even surpassing that of general processors in floating point computation. Developers of games or multimedia can design their own features by programming the graphics processor. This feature also catches the eye of developers of software other than games or graphics. Non-graphics applications using the programming power of graphics processors are called General-Purpose Computations on Graphics Processor Units (GPGPU). This study proposes a novel approach and architecture to speed up pattern matching by using the GPUs. GPU is also capable of processing network traffic of multiple sessions in parallel. The contributions of this study can be summarized as follows: • Generic: The proposed architecture is generic, and can be integrated with other systems accelerating pattern matching, such as network security system or content-intuitive systems. • Economics: The GPUs are commodity and cost-effective. For example, the solution using NVIDIA GeForce 6800GT (NVIDIA GeForce 6800, 2005) costs 1/10 of other silicon solutions with the same performance. • Effective Utilization: In general, the graphics processing subsystem is often idle in a PC. The computation power of GPU is not always fully utilized even when running games and other GPU-consuming applications. Hence, using a GPU to reduce the system load when performing pattern matching computations, such as virus scans or intrusion prevention, or using a GPU as a co-processor, could improve the performance of systems or applications.
Hardware Efficient Pattern Matching Algorithms and Architectures for Fast Intrusion Detection
NCSU PHD Dissertation, 2006
Intrusion detection processors are becoming a predominant feature in the field of network hardware. As demand on more network speed increases and new network protocols emerge, network intrusion detection systems are increasing in importance and are being integrated in network processors. Currently, most intrusion detection systems are software running on a general purpose processor. Unfortunately, it is becoming increasingly difficult for software based intrusion detection systems to keep up with increasing network speeds (OC192 and 10Gbps at backbone networks). Signature-based intrusion detection systems monitor network traffic for security threats by scanning packet payloads for attack signatures. Intrusion detection systems have to run at wire speed and need to be configurable to protect against emerging attacks. This dissertation describes the concept, structure and algorithms for a special purpose hardware accelerator designed to meet those demands. We consider the problem of string matching which is the most computationally intensive task in intrusion detection. A configurable string matching accelerator is developed with the focus on increasing throughput while maintaining the configurability provided by the software intrusion detection systems. A hardware algorithm for efficient data storage and fast retrieval is used to compress, store and retrieve attack signatures. Our algorithms reduce the size of the rules to fit on chip and enables intrusion detection to run at line rates and faster.
High-throughput linked-pattern matching for intrusion detection systems
Proceedings of the 2005 symposium on Architecture for networking and communications systems - ANCS '05, 2005
This paper presents a hardware architecture for highly efficient intrusion detection systems. In addition, a software tool for automatically generating the hardware is presented. Intrusion detection for network security is a compute-intensive application demanding high system performance. By moving both the string matching and the linking of multi-part rules to hardware, our architecture leaves the host system free for higher-level analysis. The tool automates the creation of efficient Field Programmable Gate Array architectures (FPGA). The generated hardware allows an FPGAbased system to perform deep-packet inspection of streams at up to 10 Gb/s line rates at a high level of area efficiency. Going beyond previous basic string-matching implementations that offer only single-string matching, the architecture provides support for rules requiring complex, linked (correlated-content) constructions. This allows most Snort content-linking extensions including 'distance' and 'within' bounding restrictions.
Proceedings of the 11th International Conference on Advances in Information Technology, 2020
An intrusion detection system monitors and analyzes all the incoming packets, on a given network, to detect any corresponding vulnerabilities and intrusions. It consists of four major modules: packet capturing, packet decoding, packet preprocessing and string/pattern matching. Among these, the string matching is computationally the most intensive part and a number of hardware architectures/designs have already been proposed to accelerate its performance. Consequently, an exploration of existing hardware architectures for string matching algorithms is critical. This paper identifies the most frequently used string matching algorithms and techniques, utilized for the hardware implementation. Subsequently, an exploration of various hardware architectures is provided for the identified algorithms and techniques. Finally, the implementation details of explored architectures are discussed in terms of the used device, consumed hardware resources, operational clock frequency and throughput.
Multi-pattern signature matching for hardware network intrusion detection systems
GLOBECOM '05. IEEE Global Telecommunications Conference, 2005., 2005
Network Intrusion Detection System (NIDS) performs deep inspections on the packet payload to identify, deter and contain the malicious attacks over the Internet. It needs to perform exact matching on multi-pattern signatures in real time. In this paper we introduce an efficient data structure called Extended Bloom Filter (EBF) and the corresponding algorithm to perform the multi-pattern signature matching. We also present a technique to support long signature matching so that we need only to maintain a limited number of supported signature lengths for the EBFs. We show that at reasonable hardware cost we can achieve very fast and almost time-deterministic exact matching for thousands of signatures. The architecture takes the advantages of embedded multi-port memories in FPGAs and can be used to build a full-featured hardware-based NIDS.
Efficient hardware support for pattern matching in network intrusion detection
2010
Deep packet inspection forms the backbone of any Network Intrusion Detection (NID) system. It involves matching known malicious patterns against the incoming traffic payload. Pattern matching in software is prohibitively slow in comparison to current network speeds. Due to the high complexity of matching, only FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit) platforms can provide efficient solutions. FPGAs facilitate target architecture specialization due to their field programmability. Costly ASIC designs, on the other hand, are normally resilient to pattern updates. Our FPGA-based solution performs high-speed pattern matching while permitting pattern updates without resource reconfiguration. To its advantage, our solution can be adopted by software and ASIC realizations, however at the expense of much lower performance and higher price, respectively. Our solution permits the NID system to function while pattern updates occur. An off-line optimization method first finds common subpatterns across all the patterns in the SNORT database of signatures . A novel technique then compresses each pattern into a bit vector, where each bit represents such a sub-pattern. This approach reduces drastically the required on-chip storage as well as the complexity of pattern matching. The bit vectors for newly discovered patterns can be generated easily using a simple high-level language program before storing them into the on-chip RAM. Compared to earlier approaches, not only is our strategy very efficient while supporting runtime updates but it also results in impressive area savings; it utilizes just 0.052 logic cells for processing and 17.77 bits for storage per character in the current SNORT database of 6455 patterns. Also, the total number of logic cells for processing the traffic payload does not change with pattern updates.
A Fast Multi-pattern Matching Algorithm for Deep Packet Inspection on a Network Processor
Deep Packet Inspection (DPI) is a critical function in network security applications such as Firewalls and Intrusion Detection Systems (IDS). Signature based scanners used in DPI apply multi-pattern matching algorithms to check whether the packet payload or flow content contains a specified signature in a signature set. Existing multi-pattern matching algorithms sacrifice memory space to achieve better performance. In this paper a novel fast multi-pattern matching algorithm, the Hash Boyer-Moore (HBM) Algorithm, is presented, which reduces the memory footprint of the heuristic table using a hash function and adds another heuristic table to reduce the false-positive ratio. Analyses and simulations show HBM offers higher speed and lower memory cost than some existing algorithms. The HBM algorithm was implemented on the Intel IXP 2400 Network Processor (NP) platform and experiments show suitable performance results in a Gigabit Ethernet LAN environment.