Multi-modal Registration Using a Combined Similarity Measure (original) (raw)
Related papers
IEEE Transactions on Geoscience and Remote Sensing, 2020
Many similarity measures (SMs) were proposed to measure the similarity between multimodal remote sensing (RS) images. Each SM is efficient to a different degree in different registration cases (we consider visible-to-infrared, visible-to-radar, visible-to-digital elevation model (DEM), and radar-to-DEM ones), but no SM was shown to outperform all other SMs in all cases. In this article, we investigate the possibility of deriving a more powerful SM by combining two or more existing SMs. This combined SM relies on a binary linear support vector machine (SVM) classifier trained using real RS images. In the general registration case, we order SMs according to their impact on the combined SM performance. The three most important SMs include two structural SMs based on modality independent neighborhood descriptor (MIND) and scale-invariant feature transform-octave (SIFT-OCT) descriptors and one area-based logarithmic likelihood ratio (logLR) SM: the former ones are more robust to structural changes of image intensity between registered modes, the latter one is to image noise. Importantly, we demonstrate that a single combined SM can be applied in the general case as well as in each particular considered registration case. As compared to existing multimodal SMs, the proposed combined SM [based on five existing SMs, namely, MIND, logLR, SIFT-OCT, phase correlation (PC), histogram of orientated phase congruency (HOPC)] increases the area under the curve (AUC) by from 1% to 21%. From a practical point of view, we demonstrate that complex multimodal image pairs can be successfully registered with the proposed combined SM, while existing single SMs fail to detect enough correspondences for registration. Our results demonstrate that MIND, SIFT, and logLR SMs capture essential aspects of the similarity between RS modes, and their properties are complementary for designing a new more efficient multimodal SM.
International Journal of Advanced Robotic Systems, 2017
In order to assess the performance of multisensor image registration algorithms that are used in the multirobot information fusion, we propose a model based on structural similarity whose name is vision registration assessment model. First of all, this article introduces a new image concept named superimposed image for testing subjective and objective assessment methods. Therefore, we assess the superimposed image but not the registered image, which is different from previous image registration assessment methods that usually use reference and sensed images. Then, we calculate eight assessment indicators from different aspects for superimposed images. After that, vision registration assessment model fuses the eight indicators using canonical correlation analysis, which is used for evaluating the quality of an image registration results in different aspects. Finally, three kinds of images which include optical images, infrared images, and SAR images are used to test vision registration assessment model. After evaluating three state-of-the-art image registration methods, experiments indict that the proposed structural similarity-motivated model achieved almost same evaluation results with that of the human object with the consistency rate of 98.3%, which shows that vision registration assessment model is efficient and robust for evaluating multisensor image registration algorithms. Moreover, vision registration assessment model is independent of the emotional factors and outside environment, which is different from the human.
Article Distance-Dependent Multimodal Image Registration for Agriculture Tasks
2015
Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria.
Enhanced Multimodality Image Registration Based On Mutual Information
Different modalities can be achieved by the maximization of suitable statistical similarity measures within a given class of geometric transformations. The registration functions are less sensitive to low sampling resolution, do not contain incorrect global maxima which are sometimes found in the mutual information. This paper proposes a novel and straightforward multimodal image registration method based on mutual information, in which two matching criteria are used. It has been extensively shown that metrics based on the evaluation of mutual information are well suited for overcomin g the difficulties of multi-modality registration.
Distance-Dependent Multimodal Image Registration for Agriculture Tasks
Sensors, 2015
Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria.
Image and Signal Processing for Remote Sensing XXI, 2015
Registration of multi-modal remote sensing images is an essential and challenging task in different remote sensing applications such as image fusion and multi-temporal change detection. Mutual Information (MI) has shown to be successful similarity measure for multi-modal image registration applications, however it has some drawbacks. 1. MI surface is highly non-convex with many local maxima. 2. Spatial information is completely lost in the calculation of the joint intensity probability distribution. In this paper, we present an improved MI similarity measure based on a new concept in integrating other image features as well as spatial information in the estimation of the joint intensity histogram which is used as an estimate of the joint probability distribution. The proposed method is based on the idea that each pixel in the reference image is assigned a weight, then each bin in the joint histogram is calculated as the summations of the weights of the pixels corresponding to that bin. The weight given to each pixel in the reference image is an exponential function of the corresponding pixel values in a distance image and a normalized gradient image such that higher weights are given to points close to one or more selected key points as well as points with high normalized gradient values. The proposed method is in essence a kind of calculating similarity measure using irregular sampling where sampling frequency is higher in areas close to key-points or areas with higher gradients. We have compared the proposed method with the conventional MI and Normalized MI methods for registration of pairs of multi-temporal multi-modal remote sensing images. We observed that the proposed method produces considerably better registration function containing fewer erroneous maxima and leading to higher success rate.
A novel Approach of Image Registration Based on Normalized Dissimilarity Index
2016
Image Registration is a fundamental task in image processing used to match two or more pictures taken at different times and from different sensors. A various registration measurement have been developed for different types of problems. In this paper we present a novel registration approach based on normalized dissimilarity index which is resulting from local dissimilarity map that is a useful means to compare images. We compared its quality to classical registration measurements (Correlation and Mutual information) carried out on both binarized and grey-level images. We obtained good results for the two types of images. We showed afterwards the robustness of the proposed measure against the pepper and salt and Gaussian noise. We applied finally our registration approach on a medical images database on which we confirmed that the accuracy of alignment error estimation is improve compared to classical registration methods. Index Term— Correlation, images registration, local dissimila...
Feature Neighbourhood Mutual Information for Multi-modal Image Registration: An
2015
Multi-modal image registration is becoming an increasingly powerful tool for medical diagnosis and treatment. The combination of di↵erent image modalities facilitates much greater understanding of the underlying condi-tion, resulting in improved patient care. Mutual Information is a popu-lar image similarity measure for performing multi-modal image registration. However, it is recognised that there are limitations with the technique that can compromise the accuracy of the registration, such as the lack of spatial information that is accounted for by the similarity measure. In this paper, we present a two-stage non-rigid registration process using a novel similarity measure, Feature Neighbourhood Mutual Information. The similarity measure eciently incorporates both spatial and structural image properties that are not traditionally considered by MI. By incorporating such features, we find that this method is capable of achieving much greater registration accuracy when compared to exis...
Image registration is a fundamental task in remote sensing image processing that is used to match two or more images taken, for example, at different times, from different sensors or from different viewpoints. A lot of automation has been achieved in this field but ever sprouting data quality and characteristics compel innovators to design new and/or improve existing registration techniques. In this paper, image registration methodologies are broadly classified into intensity and feature based approaches and analyzed critically.