IJERT-A Survey On Architectural Design Of Bloom Filter For Signature Detection (original) (raw)

High Speed Signature Matching in Network Interface Device using Bloom Filters

Abstract—Network intrusion detection systems play a critical role in protecting the information infrastructure of an organization. Due to the sophistication and complexity of techniques used for the analysis they are commonly based on general-purpose workstations. Although cost-efficient, these general-purpose systems are found to be inadequate as they fail to perform efficiently at high packet rates. The resulting packet loss degrades the system’s overall effectiveness, as the analyzing capability of the system is reduced. It has been found that the performance of these systems can be improved significantly by filtering out unwanted packets. This paper presents the design of a Programmable Ethernet Interface Card that is used to offload signature matching from software and thereby improve the detection ratio and performance of the system.

Aggregated Bloom Filters for Intrusion Detection and Prevention Hardware

IEEE GLOBECOM 2007-2007 IEEE Global Telecommunications Conference, 2007

Bloom Filters (BFs) are fundamental building blocks in various network security applications, where packets from high-speed links are processed using state-of-the-art hardwarebased systems. In this paper, we propose Aggregated Bloom Filters (ABFs) to increase the throughput and scalability of BFs. The proposed ABF has two methods to improve average speed and scalability. The first method leverages the query mechanism for hardware BFs. We ptimize queries by removing redundant hash calculations and memory accesses. First, to remove redundancy, the hash functions for each query are calculated sequentially. As soon as we have a no match in any of the hash results, the query is immediately abandoned. We then aggregate multiple queries and query a BF with all of these queries in parallel, which maximizes the throughput of the BF. The second method addresses scalability issues regarding the on-chip memory resources. In most applications multiple BFs are required to store many sets with different numbers of elements. These sets may also be too small for the unit memory on-chip. So, most of the memory is left unused, causing low memory utilization. The second method aggregates small distributed BFs to a single BF allowing better on-chip memory utilization. For the application of Network Intrusion Detection and Prevention Systems (NIDPSs), our proposed ABF shows seven-fold improvement in the average query throughput and four times less memory usage.

Implementation of bloom filters in reconfigurable hardware for tracing network attacks

IFAC Proceedings Volumes, 2006

A Bloom filter is a data structure for representing a set of strings in order to support. membership querie.s. It was first. int.roduced in 1970 for the database query matchmg. Recently this structure ha.s been rediscovered and widely lIsro in the area of network processing. The main prohlems t.hat can he solvro lIsing t.he nloom filt.ers an~: dat.agram t.racehack, mult.i pat.tern mat.ching, packet classification and malicious code fingerprint.ing. In the art.icle we will descrihe our experiences with the implementation of Bloom filters in field-Programmable Gate Arrays for tracing network attacks. The prepared module can operate with the throughput over 1 Gbps and can store up to ::; seconds of traffic using less than 262,144 kB of memory.

Deep packet inspection using parallel bloom filters

IEEE Micro, 2004

Recent advances in network packet processing focus on payload inspection for applications that include contentbased billing, layer-7 switching and Internet security. Most of the applications in this family need to search for predefined signatures in the packet payload. Hence an important building block of these processors is string matching infrastructure. Since conventional software-based algorithms for string matching have not kept pace with high network speeds, specialized high-speed, hardware-based solutions are needed. We describe a technique based on Bloom filters for detecting predefined signatures (a string of bytes) in the packet payload. A Bloom filter is a data structure for representing a set of strings in order to support membership queries. We use hardware Bloom filters to isolate all packets that potentially contain predefined signatures. Another independent process eliminates false positives produced by Bloom filters.

A Cache Architecture for Counting Bloom Filters: Theory and Application

2011

Within packet processing systems, lengthy memory accesses greatly reduce performance. To overcome this limitation, network processors utilize many different techniques, for example, utilizing multilevel memory hierarchies, special hardware architectures, and hardware threading. In this paper, we introduce a multilevel memory architecture for counting Bloom filters. Based on the probabilities of incrementing of the counters in the counting Bloom filter, a multi-level cache architecture called the cached counting Bloom filter (CCBF) is presented, where each cache level stores the items with the same counters. To test the CCBF architecture, we implement a software packet classifier that utilizes basic tuple space search using a 3-level CCBF. The results of mathematical analysis and implementation of the CCBF for packet classification show that the proposed cache architecture decreases the number of memory accesses when compared to a standard Bloom filter. Based on the mathematical analysis of CCBF, the number of accesses is decreased by at least 53%. The implementation results of the software packet classifier are at most 7.8% (3.5% in average) less than corresponding mathematical analysis results. This difference is due to some parameters in the packet classification application such as number of tuples, distribution of rules through the tuples, and utilized hashing functions.

C2BF: Cache-based Counting Bloom Filter for Precise Matching in Network Packet Processing

Procedia Engineering, 2012

Bloom filter is widely used in network packet processing due to its fast lookup speed and small memory cost. However, the non-negligible false positive rate and the difficulty of online update still prevent it from extensive utilization. In this paper, we propose a cache-based counting Bloom filter architecture, C 2 BF, which is not only easy to update online but also benefical for fast verification for precise matching. We also present a high speed hardware C 2 BF architecture with off-chip memory and fast cache replacement method. This paper includes three contributions: 1) compressed CBF implementation and its updating algorithm; 2) pattern grouping for higher cache hit rate; 3) onchip cache organization and replacement policy. Experiments show that our prototype of C 2 BF reduces more than 70% of the verification processing time with cache design compared with traditional schemes without cache.

L-CBF: a low-power, fast counting Bloom filter architecture

2008

Abstract An increasing number of architectural techniques have relied on hardware counting bloom filters (CBFs) to improve upon the energy, delay, and complexity of various processor structures. CBFs improve the energy and speed of membership tests by maintaining an imprecise and compact representation of a large set to be searched. This paper studies the energy, delay, and area characteristics of two implementations for CBFs using full custom layouts in a commercial 0.13-mum fabrication technology.

BBFex: a bloom-bloomier filter extension for long patterns in FPGA-based pattern matching system

FPGA '11 Proceedings of the 19th ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2011

There are many pattern matching engines in Network Intrusion Detection Systems (NIDS) have been developed on FPGA-based platforms to accelerates the performance of pattern matching process in order to keep up with the gradually increasing in speed of current networks. However, those systems only support small number of short patterns which are not appropriate to large database such as Clam Antivirus patterns. In this paper, we propose Bloom-Bloomier Filter Extension (BBFex) as a practical pattern matching engine that handles large various-length pattern database. The basic idea in designing BBFex is the combination of Bloom Filter and Bloomier Filter to index patterns and an efficient pattern fragmenting method to split and to merge long patterns. Therefore, BBFex can recognize nearly 84,000 Clam Antivirus static patterns of which lengths vary from 4 to 255 characters with rather low on chip memory density, approximately 0.4 bits per character while keeping the off-chip memory access rate 5X lower compared to previous similar system and achieving throughput of 1.36 Gbps. In addition, BBFex is not only limited to Clam Antivirus database because its architecture is designed in respect to general character-based database. Moreover, as a hash-based system, BBFex does not require entire system reconfiguration when updating database. (The attached file is a draft of the work)

Hunting the Pertinency of Bloom Filter in Computer Networking and Beyond: A Survey

Bloom filter is a probabilistic data structure to filter a membership of a set. Bloom filter returns "true" or "false" with an error tolerance depending on the presence of the element in the set. Bloom filter is used to boost up the performance of a system using small space overhead. It is extensively used since its inception. e Bloom filter has met wide area of applications. Bloom filter is used in entire computing field irrespective of application and research domain. Bloom filter poses (i) high adaptability, (ii) low memory space overhead as compared to hashing algorithms, (iii) high scalability, and (iv) high performance. In this article, we uncover the application area of Bloom filter in computer networking and its related domain.