A Real-Time Gabor Primal Sketch for Visual Attention (original) (raw)
Related papers
Steerable properties dominate the design of traditional filters, e.g., Gabor filters, and endow features the capability of dealing with spatial transformations. However, such excellent properties have not been well explored in the popular deep convolutional neural networks (DCNNs). In this paper, we propose a new deep model, termed Gabor Convo-lutional Networks (GCNs or Gabor CNNs), which incorporates Gabor filters into DCNNs to enhance the resistance of deep learned features to the orientation and scale changes. By only manipulating the basic element of DCNNs based on Gabor filters, i.e., the convolution operator, GCNs can be easily implemented and are compatible with any popular deep learning architecture. Experimental results demonstrate the super capability of our algorithm in recognizing objects, where the scale and rotation changes occur frequently. The proposed GCNs have much fewer learnable network parameters, and thus is easier to train with an end-to-end pipeline. The source code will be here 1 .
1998
Gabor schemes of multiscale image representation are useful in many computer vision applications. However, the classic Gabor expansion is computationally expensive due to the lack of orthogonality of Gabor functions. Some alternative schemes, based on the application of a bank of Gabor filters, have important advantages such as computational efficiency and robustness, at the cost of redundancy and lack of completeness. In a previous work we proposed a quasicomplete Gabor transform, suitable for fast implementations in either space or frequency domains. Reconstruction was achieved by simply adding together the even Gabor channels. We develop an optimized spatial-domain implementation, using one-dimensional 11-tap filter masks, that is faster and more flexible than Fourier implementations. The reconstruction method is improved by applying fixed and independent weights to the Gabor channels before adding them together. Finally, we analyze and implement, in the spatial domain, two ways to incorporate a highpass residual, which permits a visually complete representation of the image. © 1998 SPIE and IS&T. [S1017-9909(98)01601-8]
2019
This work takes a step towards investigating the benefits of merging classical vision techniques with deep learning models. Formally, we explore the effect of replacing the first layers of neural network architectures with convolutional layers that are based on Gabor filters with learnable parameters. As a first result, we observe that architectures utilizing Gabor filters as low-level kernels are capable of preserving test set accuracy of deep convolutional networks. Therefore, this architectural change exalts their capabilities in extracting useful low-level features. Furthermore, we observe that the architectures enhanced with Gabor layers gain advantages in terms of robustness when compared to the regular models. Additionally, the existence of a closed mathematical expression for the Gabor kernels allows us to develop an analytical expression for an upper bound to the Lipschitz constant of the Gabor layer. This expression allows us to propose a simple regularizer to enhance the ...
Multi-Scale Gridded Gabor Attention for Cirrus Segmentation
2022 IEEE International Conference on Image Processing (ICIP), 2022
In this paper, we address the challenge of segmenting global contaminants in large images. The precise delineation of such structures requires ample global context alongside understanding of textural patterns. CNNs specialise in the latter, though their ability to generate global features is limited. Attention measures long range dependencies in images, capturing global context, though at a large computational cost. We propose a gridded attention mechanism to address this limitation, greatly increasing efficiency by processing multiscale features into smaller tiles. We also enhance the attention mechanism for increased sensitivity to texture orientation, by measuring correlations across features dependent on different orientations, in addition to channel and positional attention. We present results on a new dataset of astronomical images, where the task is segmenting large contaminating dust clouds.
Accurate and Efficient Computation of Gabor Features in Real-Time Applications
2009
Gabor features are widely used in many computer vision applications such as image segmentation and pattern recognition. To extract Gabor features, a set of Gabor filters tuned to several different frequencies and orientations is utilized. The computational complexity of these features, due to their non-orthogonality, prevents their use in many real-time or near real-time tasks. Many research efforts have been made to address the computational complexity of Gabor filters. Most of these techniques utilize the separability of Gabor filters by decomposing them into 1-D Gaussian filter. The main issue in these techniques is the efficient pixel interpolation along the desired direction. Sophisticated interpolation mechanisms minimize the interpolation error with the increased computational complicity. This paper presents a novel framework in computation of Gabor features by utilizing a sophisticated interpolation scheme – quadratic spline – without increasing the overall computational complexity of the process. The main contribution of this work is the process of performing the interpolation and the convolution in a single operation. The proposed approach has been used successfully in real-time extraction of Gabor features from video sequence. The experimental results show that the proposed framework improves the accuracy of the Gabor features while reduces the computational complexity.
Programmable logic devices, such as Field Programmable Gate Arrays, are well-suited for implementing biologicallyinspired visual processing algorithms and among those algorithms is HMAX model. This model mimics the feedforward path of object recognition in the visual cortex. Methods: HMAX includes several layers and its most computation intensive stage could be the S1 layer which applies 64 2D Gabor filters with various scales and orientations on the input image. A Gabor filter is the product of a Gaussian window and a sinusoid function. Using the separability property in the Gabor filter in the 0° and 90° directions and assuming the isotropic filter in the 45° and 135° directions, a 2D Gabor filter converts to two more efficient 1D filters. Results: The current paper presents a novel hardware architecture for the S1 layer of the HMAX model, in which a 1D Gabor filter is utilized twice to create a 2D filter. Using the even or odd symmetry properties in the Gabor filter coefficients reduce the required number of multipliers by about 50%. The normalization value in every input image location is also calculated simultaneously. The implementation of this architecture on the Xilinx Virtex-6 family shows a 2.83ms delay for a 128×128 pixel input image that is a 1.86X-speedup relative to the last best implementation. Conclusion: In this study, a hardware architecture is proposed to realize the S1 layer of the HMAX model. Using the property of separability and symmetry in filter coefficients saves significant resources, especially in DSP48 blocks.
Improving object recognition by transforming Gabor filter responses
Network: Computation in Neural Systems, 1996
Previous work described a biologically motivated object recognition system with Gabor wavelets as basic feature type. These features are robust against slight distortion, rotation and variation in illumination. We here describe extensions of the system that address image variance due to arbitrary in-plane rotation, substantial scale changes and moderate depth rotation of objects, and to background variation, using simple linear transformation of the Gabor lter responses. The performance of the system is enhanced signi cantly.
Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression
IEEE Transactions on Signal Processing, 1988
A three-layered neural network is described for transforming two-dimensional discrete signals into generalized nonorthogonal 2-D Gabor representations for image analysis, segmentation, and compression. These transforms are conjoint spatial/spectral representations, which provide a complete image description in terms of locally windowed 2-D spectral coordinates embedded within global 2-D spatial coordinates. In the present neural network approach, based on interlaminar interactions involving two layers with fixed weights and one layer with adjustable weights, the network finds coefficients for complete conjoint 2-D Gabor transforms without restrictive conditions. In wavelet expansions based on a biologically inspired log-polar ensemble of dilations, rotations, and translations of a single underlying 2-D Gabor wavelet template, image compression is illustrated with ratios up to 20:1. Also demonstrated is image segmentation based on the clustering of coefficients in the complete 2-D Gabor transform
A fast learning algorithm for Gabor transformation
IEEE Transactions on Image Processing, 1996
An adaptive learning approach for the computation of the coefficients of the generalized nonorthogonal 2•D Gabor transform representation is introduced in this correspondence. The algorithm uses a recursive least squares (RLS) type algorithm. The aim is to achieve minimum mean squared error for the reconstructed image from the set of the Gabor coefficients. The proposed RLS learning offers better accuracy and faster convergence behavior when compared with the least mean squares (LMS)•based algorithms. Applications of this scheme in image data reduction are also demonstrated.
Gabor filter-based edge detection
Pattern Recognition, 1992
It is common practice to utilize evidence from biological and psychological vision experiments to develop computational models for low-level feature extraction. The receptive profiles of simple cells in mammalian visual systems have been found to closely resemble Gabor filters. ...