Aligning images in the wild (original) (raw)

A framework for efficient correspondence using feature interrelations

2008 19th International Conference on Pattern Recognition, 2008

We propose a formulation for solving the point pattern correspondence problem, relying on transformation invariants. Our approach can accommodate any degree of descriptors thus modeling any kind of potential deformation according to the needs of each specific problem. Other potential descriptors such as color or local appearance can also be incorporated. A brief study on the complexity of the methodology is made which proves to be inherently polynomial while allowing for further adjustments via thresholding. Initial experiments on both synthetic and real data demonstrate its potentials in terms of accuracy and robustness to noise and outliers.

Alignment Using Distributions of Local Geometric Properties

IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999

AbstractÐWe describe a framework for aligning images without needing to establish explicit feature correspondences. We assume that the geometry between the two images can be adequately described by an affine transformation and develop a framework that uses the statistical distribution of geometric properties of image contours to estimate the relevant transformation parameters. The estimates obtained using the proposed method are robust to illumination conditions, sensor characteristics, etc., since image contours are relatively invariant to these changes. Moreover, the distributional nature of our method alleviates some of the common problems due to contour fragmentation, occlusion, clutter, etc. We provide empirical evidence of the accuracy and robustness of our algorithm. Finally, we demonstrate our method on both real and synthetic images, including multisensor image pairs.

Multi-Image Matching Using Multi-Scale Oriented Patches

2005

This paper describes a novel multi-view matching framework based on a new type of invariant feature. Our features are located at Harris corners in discrete scale-space and oriented using a blurred local gradient. This defines a rotationally invariant frame in which we sample a feature descriptor, which consists of an 8 × 8 patch of bias/gain normalised intensity values. The density of features in the image is controlled using a novel adaptive non-maximal suppression algorithm, which gives a better spatial distribution of features than previous approaches. Matching is achieved using a fast nearest neighbour algorithm that indexes features based on their low frequency Haar wavelet coefficients. We also introduce a novel outlier rejection procedure that verifies a pairwise feature match based on a background distribution of incorrect feature matches. Feature matches are refined using RANSAC and used in an automatic 2D panorama stitcher that has been extensively tested on hundreds of sample inputs.

Feature-based image alignment via coupled Hough transforms

2009

We investigate the problem of feature based image alignment under vertical and horizontal translation and scaling. The problem of finding the two dimensional affine alignment without rotation can be separated into two coupled simpler problems. The resulting two model fitting problems are coupled through a consensuses set of points that fit both models. The proposed solution can be viewed as a coordinate-wise optimization that separates the two dimensional shift and stretch problem into two coupled one-dimensional problems. Each one dimensional alignment problem is solved through a Hough transform that allows fast and robust estimation of the two parameters of a one dimensional affine deformation model. Experimental results demonstrate the efficiency and accuracy of the proposed method in finding the spatial transform between two given video sequences.

Towards a Robust Scale Invariant Feature Correspondence

Lecture Notes in Computer Science, 2015

In this paper, we introduce an improved scale invariant feature correspondence algorithm which depends on the Similarity-Topology Matching algorithm. It pays attention not only to the similarity between features but also to the spatial layout of every matched feature and its neighbours. The features are represented as an undirected graph where every node represents a local feature and every edge represents adjacency between them. The topology of the resulting graph can be considered as a robust global feature of the represented object. The matching process is modeled as a graph matching problem; which in turn is formulated as a variation of the quadratic assignment problem. The Similarity-Topology Matching algorithm achieves superior performance in almost all the experiments except when the image has been exposed to scaling deformations. An amendment has been done to the algorithm in order to cope with this limitation. In this work, we depend not only on the distance between the two interest points but also on the scale at which the interest points are detected to decide the neighbourhood relations between every pair of features. A set of challenging experiments conducted using 50 images (contain repeated structure) representing 5 objects from COIL-100 data-set with extra synthetic deformations reveal that the modified version of the Similarity-Topology Matching algorithm has better performance. It is considered more robust especially under the scale deformations.

GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence

2017

Incorporating smoothness constraints into feature matching is known to enable ultra-robust matching. However, such formulations are both complex and slow, making them unsuitable for video applications. This paper proposes GMS (Grid-based Motion Statistics), a simple means of encapsulating motion smoothness as the statistical likelihood of a certain number of matches in a region. GMS enables translation of high match numbers into high match quality. This provides a real-time, ultra-robust correspondence system. Evaluation on videos, with low textures, blurs and wide-baselines show GMS consistently out-performs other real-time matchers and can achieve parity with more sophisticated, much slower techniques.

Unsupervised Joint Alignment of Complex Images

2007 IEEE 11th International Conference on Computer Vision, 2007

Many recognition algorithms depend on careful positioning of an object into a canonical pose, so the position of features relative to a fixed coordinate system can be examined. Currently, this positioning is done either manually or by training a class-specialized learning algorithm with samples of the class that have been hand-labeled with parts or poses. In this paper, we describe a novel method to achieve this positioning using poorly aligned examples of a class with no additional labeling. Given a set of unaligned examplars of a class, such as faces, we automatically build an alignment mechanism, without any additional labeling of parts or poses in the data set. Using this alignment mechanism, new members of the class, such as faces resulting from a face detector, can be precisely aligned for the recognition process. Our alignment method improves performance on a face recognition task, both over unaligned images and over images aligned with a face alignment algorithm specifically developed for and trained on hand-labeled face images. We also demonstrate its use on an entirely different class of objects (cars), again without providing any information about parts or pose to the learning algorithm.

DAISY Filter Flow: A Generalized Discrete Approach to Dense Correspondences

2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014

Establishing dense correspondences reliably between a pair of images is an important vision task with many applications. Though significant advance has been made towards estimating dense stereo and optical flow fields for two images adjacent in viewpoint or in time, building reliable dense correspondence fields for two general images still remains largely unsolved. For instance, two given images sharing some content exhibit dramatic photometric and geometric variations, or they depict different 3D scenes of similar scene characteristics. Fundamental challenges to such an image or scene alignment task are often multifold, which render many existing techniques fall short of producing dense correspondences robustly and efficiently. This paper presents a novel approach called DAISY filter flow (DFF) to address this challenging task. Inspired by the recent PatchMatch Filter technique, we leverage and extend a few established methods: 1) DAISY descriptors, 2) filter-based efficient flow inference, and 3) the Patch-Match fast search. Coupling and optimizing these modules seamlessly with image segments as the bridge, the proposed DFF approach enables efficiently performing dense descriptor-based correspondence field estimation in a generalized high-dimensional label space, which is augmented by scales and rotations. Experiments on a variety of challenging scenes show that our DFF approach estimates spatially coherent yet discontinuity-preserving image alignment results both robustly and efficiently.