A Coarse-to-Fine Algorithm for Registration in 3D Street-View Cross-Source Point Clouds (original) (raw)

Fine scale image registration in large-scale urban LIDAR point sets

Computer Vision and Image Understanding, 2017

Urban scenes acquisition is very often performed using laser scanners onboard a vehicle. In parallel, color information is also acquired through a set of coarsely aligned camera pictures. The question of combining both measures naturally arises for adding color to the 3D points or enhancing the geometry, but it faces important challenges. Indeed, 3D geometry acquisition is highly accurate while the images suffer from distortion and are only coarsely registered to the geometry. In this paper, we introduce a two-step method to register images to large-scale complex point clouds. Our method performs the image-to-geometry registration by iteratively registering the real image to a synthetic image obtained from the estimated camera pose and the point cloud, using either reflectance or normal information. First a coarse registration is performed by generating a wide-angle synthetic image and considering that small pitch and yaw rotations can be estimated as translations in the image plane. Then a fine registration is performed using a new image metric which is adapted to the difference of modality between the real and synthetic images. This new image metric is more resilient to missing data and large transformations than standard Mutual Information. In the process, we also introduce a method to generate synthetic images from a 3D point cloud that is adapted to large-scale urban scenes with occlusions and sparse areas. The efficiency of our algorithm is demonstrated both qualitatively and quantitatively on datasets of urban scans and associated images.

A Coarse-to-Fine Algorithm for Matching and Registration in 3D Cross-Source Point Clouds

IEEE Transactions on Circuits and Systems for Video Technology, 2017

We propose an efficient method to deal with the matching and registration problem found in cross-source point clouds captured by different types of sensors. This task is especially challenging due to the presence of density variation, scale difference, a large proportion of noise and outliers, missing data and viewpoint variation. The proposed method has two stages: in the coarse matching stage, we use the ESF descriptor to select potential K regions from the candidate point clouds for the target. In the fine stage, we propose a scale embedded generative GMM registration method to refine the results from the coarse matching stage. Following the fine stage, both the best region and accurate camera pose relationships between the candidates and target are found. We conduct experiments in which we apply the method to two applications: one is 3D object detection and localization in street-view ourdoor (LiDAR/VSFM) cross-source point clouds, and the other is 3D scene matching and registration in indoor (KinectFusion/VSFM) cross-source point clouds. The experiment results show that the proposed method performs well when compared with the existing methods. It also shows that the proposed method is robust under various sensing techniques such as LiDAR, Kinect and RGB camera.

Improving 3D Lidar Point Cloud Registration Using Optimal Neighborhood Knowledge

ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012

Automatic 3D point cloud registration is a main issue in computer vision and photogrammetry. The most commonly adopted solution is the well-known ICP (Iterative Closest Point) algorithm. This standard approach performs a fine registration of two overlapping point clouds by iteratively estimating the transformation parameters, and assuming that good a priori alignment is provided. A large body of literature has proposed many variations of this algorithm in order to improve each step of the process. The aim of this paper is to demonstrate how the knowledge of the optimal neighborhood of each 3D point can improve the speed and the accuracy of each of these steps. We will first present the geometrical features that are the basis of this work. These low-level attributes describe the shape of the neighborhood of each 3D point, computed by combining the eigenvalues of the local structure tensor. Furthermore, they allow to retrieve the optimal size for analyzing the neighborhood as well as the privileged local dimension (linear, planar, or volumetric). Besides, several variations of each step of the ICP process are proposed and analyzed by introducing these features. These variations are then compared on real datasets, as well with the original algorithm in order to retrieve the most efficient algorithm for the whole process. Finally, the method is successfully applied to various 3D lidar point clouds both from airborne, terrestrial and mobile mapping systems.

Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge

ISPRS Journal of Photogrammetry and Remote Sensing, 2013

Automatic 3D point cloud registration is a main issue in computer vision and remote sensing. One of the most commonly adopted solution is the well-known Iterative Closest Point (ICP) algorithm. This standard approach performs a fine registration of two overlapping point clouds by iteratively estimating the transformation parameters, assuming good a priori alignment is provided. A large body of literature has proposed many variations in order to improve each step of the process (namely selecting, matching, rejecting, weighting and minimizing). The aim of this paper is to demonstrate how the knowledge of the shape that best fits the local geometry of each 3D point neighborhood can improve the speed and the accuracy of each of these steps. We first present the geometrical features that are the basis of this work. These low-level attributes indeed describe the neighborhood shape around each 3D point. They allow to retrieve the optimal size for analyzing the neighborhoods at various scales as well as the privileged local dimension (linear, planar, or volumetric). Several variations of each step of the ICP process are then proposed and analyzed by introducing these features. Such variants are compared on real datasets, as well with the original algorithm in order to retrieve the most efficient algorithm for the whole process. The method is therefore successfully applied to various 3D lidar point clouds from airborne, terrestrial, and mobile mapping systems. Improvements are noticed for two of the five ICP steps, while concluding our features may not be relevant for very dissimilar object samplings.

A comprehensive survey on point cloud registration

arXiv: Computer Vision and Pattern Recognition, 2021

Registration is a transformation estimation problem between two point clouds, which has a unique and critical role in numerous computer vision applications. The developments of optimization-based methods and deep learning methods have improved registration robustness and efficiency. Recently, the combinations of optimization-based and deep learning methods have further improved performance. However, the connections between optimization-based and deep learning methods are still unclear. Moreover, with the recent development of 3D sensors and 3D reconstruction techniques, a new research direction emerges to align cross-source point clouds. This survey conducts a comprehensive survey, including both same-source and cross-source registration methods, and summarize the connections between optimization-based and deep learning methods, to provide further research insight. This survey also builds a new benchmark to evaluate the state-of-the-art registration algorithms in solving cross-source challenges. Besides, this survey summarizes the benchmark data sets and discusses point cloud registration applications across various domains. Finally, this survey proposes potential research directions in this rapidly growing field.

Point Cloud Registration Refinement in an Urban Environment using 2D Edge-Maps

2018

As 3D point cloud acquisition sensors become increasingly prevalent in urban environments (e.g., LiDAR sensors for autonomous vehicles), the need arises to find efficient ways to align large amounts of such 3D data, often in real-time. In this work, we propose a novel method for 3D point cloud registration refinement in an urban environment (e.g., between Terrestrial LiDAR Scans - TLS - and Airborne LiDAR Scans - ALS), assuming an initial coarse registration is available. The proposed method is based on estimation of the direction of gravity, wall detection, projection of the point clouds on a perpendicular horizontal plane, and conversion into 2D edge-maps. Then, two methods are considered for alignment between the 2D edge-maps: a 2D variant of the well-known ICP (Iterative Closest Point) algorithm, and Edge-Map Phase-Correlation (EMPC). We demonstrate the usefulness of the proposed methods for registration in this challenging task, where the 2D variant of ICP achieves a meaningful...

A New Variant of the ICP Algorithm for Pairwise 3D Point Cloud Registration

2022

Pairwise 3D point cloud registration derived from Terrestrial Laser Scanner (TLS) in static mode is an essential task to produce locally consistent 3D point clouds. In this work, the contributions are twofold. First, a non-iterative scheme by merging the SIFT (Scale Invariant Feature Transform) 3D algorithm and the PFH (Point Feature Histograms) algorithm to find initial approximation of the transformation parameters is proposed. Then, a correspondence model based on a new variant of the ICP (Iterative Closest Point) algorithm to refine the transformation parameters is also proposed. To evaluate the local consistency of the pairwise 3D point cloud registration is used a point-to-distance approach. Experiments were performed using seven pairs of 3D point clouds into an urban area. The results obtained showed that the method achieves point-to-plane RMSE (Root of the Mean Square Error) mean values in the order of 2 centimeters

Challenging data sets for point cloud registration algorithms

International Journal of Robotics Research, 2012

Many registration solutions have bloomed lately in the literature. The iterative closest point, for example, could be considered as the backbone of many laser-based localization and mapping systems. Although they are widely used, it is a common challenge to compare registration solutions on a fair base. The main limitation is to overcome the lack of accurate ground truth in current data sets, which usually cover environments only over a small range of organization levels. In computer vision, the Stanford 3D Scanning Repository pushed forward point cloud registration algorithms and object modeling fields by providing high-quality scanned objects with precise localization. We aim at providing similar high-caliber working material to the robotic and computer vision communities but with sceneries instead of objects. We propose 8 point cloud sequences acquired in locations covering the environment diversity that modern robots are susceptible to encounter, ranging from inside an apartment to a woodland area. The core of the data sets consists of 3D laser point clouds for which supporting data (Gravity, Magnetic North and GPS) are given at each pose. A special effort has been made to ensure a global positioning of the scanner within millimeter range precision, independently of environmental conditions. This will allow for the development of improved registration algorithms when mapping challenging environments, such as found in real world situations.

Point cloud registration using a viewpoint dictionary

2016 IEEE International Conference on the Science of Electrical Engineering (ICSEE), 2016

The use of 3D point clouds is currently of much interest. One of the cornerstones of 3D point cloud research and applications is point cloud registration. Given two point clouds, the goal of registration is aligning them in a common coordinate system. In particular, we seek in this work to align a sparse and noisy local point cloud, created from a single stereo pair of images, to a dense and large-scale global point cloud, representing an urban outdoors environment. The common approach of keypointbased registration, tends to fail due to the sparsity and low quality of the stereo local cloud. We propose here a new approach. It consists of the creation of a dictionary of much smaller clouds using a grid of synthetic viewpoints over the dense global cloud. We then perform registration via an efficient dictionary search. Our approach shows promising results on data acquired in an urban environment.

Efficient Pairwise 3-D Registration of Urban Scenes via Hybrid Structural Descriptors

IEEE Transactions on Geoscience and Remote Sensing

Automatic registration of point clouds captured by terrestrial laser scanning (TLS) plays an important role in many fields including remote sensing (e.g., transportation management, 3D reconstruction in large-scale urban areas and environment monitoring), computer vision, virtual reality and robotics, among others. However, noise, outliers, non-uniform point density and small overlaps are inevitable when collecting multiple views of data, which poses great challenges to 3D registration of point clouds. Since conventional registration methods aim to find point correspondences and estimate transformation parameters directly in the original point space, the traditional way to address these difficulties is to introduce many restrictions during the scanning process (e.g., more scanning and careful selection of scanning positions), thus making the data acquisition more difficult. In this paper, we present a novel 3D registration framework that performs in a "middle-level structural space" and is capable of robustly and efficiently reconstructing urban, semi-urban and indoor scenes, despite disturbances introduced in the scanning process. The new structural space is constructed by extracting multiple types of middle-level geometric primitives (planes, spheres, cylinders, and cones) from the 3D point cloud. We design a robust method to find effective primitive combinations corresponding to the 6D poses of the raw point clouds and then construct hybrid-structure-based descriptors. By matching descriptors and computing rotation and translation parameters, successful registration is achieved. Note that the whole process of our method is performed in the structural space, which has the advantages of capturing geometric structures (the relationship between primitives) and semantic features (primitive types and parameters) in larger fields. Experiments show that our method achieves state-of-the-art performance in several point cloud registration benchmark datasets at different scales and even obtains good registration results for data without overlapping areas.