Camera and Lidar Cooperation for 3D Feature Extraction (original) (raw)

Fast planar surface 3D SLAM using LIDAR

Robotics and Autonomous Systems, 2017

In this paper we propose a fast 3D pose based SLAM system that estimates a vehicle's trajectory by registering sets of planar surface segments, extracted from 360 • field of view (FOV) point clouds provided by a 3D LIDAR. Full FOV and planar representation of the map gives the proposed SLAM system the capability to map large-scale environments while maintaining fast execution time. For efficient point cloud processing we apply image-based techniques to project it to three two-dimensional images. The SLAM backend is based on Exactly Sparse Delayed State Filter as a non-iterative way of updating the pose graph and exploiting sparsity of the SLAM information matrix. Finally, our SLAM system enables reconstruction of the global map by merging the local planar surface segments in a highly efficient way. The proposed point cloud segmentation and registration method was tested and compared with the several state-of-the-art methods on two publicly available datasets. Complete SLAM system was also tested in one indoor and one outdoor experiment. The indoor experiment was conducted using a research mobile robot Husky A200 to map our university building and the outdoor experiment was performed on the publicly available dataset provided by the Ford Motor Company, in which a car equipped with a 3D LIDAR was driven in the downtown Dearborn Michigan.

Liborg: a lidar-based robot for efficient 3D mapping

Applications of Digital Image Processing XL, 2017

In this work we present Liborg, a spatial mapping and localization system that is able to acquire 3D models on the fly using data originated from lidar sensors. The novelty of this work is in the highly efficient way we deal with the tremendous amount of data to guarantee fast execution times while preserving sufficiently high accuracy. The proposed solution is based on a multi-resolution technique based on octrees. The paper discusses and evaluates the main benefits of our approach including its efficiency regarding building and updating the map and its compactness regarding compressing the map. In addition, the paper presents a working prototype consisting of a robot equipped with a Velodyne Lidar Puck (VLP-16) and controlled by a Raspberry Pi serving as an independent acquisition platform.

LIV-LAM: LiDAR and Visual Localization and Mapping

2020 American Control Conference (ACC), 2020

This paper presents a framework for Simultaneous Localization and Mapping (SLAM) by combining a novel method for object discovery and localization from a monocular camera with depth information provided by Light Detection and Ranging (LiDAR). One major challenge in vision is discovering unknown objects without prior training/supervision, in the wild, and on-the-fly. In our framework, no training samples are available prior to the deployment. We develop an efficient proposal-matching method to discover object temporal saliency, and then finetune these frequently matched object proposals according to tracking information. Detected features of the objects are used as landmark features, and are merged with the LiDAR data in the proposed LIV-LAM (LiDAR and Visual Localization and Mapping). Compared to most visual SLAM or LiDAR-based SLAM, the novelty of this method is the computationally-efficient object detection and localization for feature set-and-match, in order to increase the accur...

Efficient 3D Lidar Odometry Based on Planar Patches

Sensors

In this paper we present a new way to compute the odometry of a 3D lidar in real-time. Due to the significant relation between these sensors and the rapidly increasing sector of autonomous vehicles, 3D lidars have improved in recent years, with modern models producing data in the form of range images. We take advantage of this ordered format to efficiently estimate the trajectory of the sensor as it moves in 3D space. The proposed method creates and leverages a flatness image in order to exploit the information found in flat surfaces of the scene. This allows for an efficient selection of planar patches from a first range image. Then, from a second image, keypoints related to said patches are extracted. This way, our proposal computes the ego-motion by imposing a coplanarity constraint between pairs whose correspondences are iteratively updated. The proposed algorithm is tested and compared with state-of-the-art ICP algorithms. Experiments show that our proposal, running on a single...

System for 3D mapping using affordable LIDAR

Telfor Journal

In this paper a new system for 3D (three-dimensional) mapping using affordable LIDAR (light detection and ranging) is presented. The implementation of LIDAR technology-based approach enables obtaining a point cloud as a representation of indoor surrounding. In recent years with the help of LIDAR this kind of sensing has found numerous applications across various industries. Here, a cloud of points is generated during room scanning using Arduino platform based rotating system. The obtained results are promising, and the proposed solution can find its practical application in different fields. Moreover, it can provide many possibilities for future experiments with surrounding mappings, image matching, autonomous driving, obstacle observation, collision avoidance, material type detection such as transparent ones.

Lidar-Monocular Surface Reconstruction Using Line Segments

2021 IEEE International Conference on Robotics and Automation (ICRA)

Structure from Motion (SfM) often fails to estimate accurate poses in environments that lack suitable visual features. In such cases, the quality of the final 3D mesh, which is contingent on the accuracy of those estimates, is reduced. One way to overcome this problem is to combine data from a monocular camera with that of a LIDAR. This allows fine details and texture to be captured while still accurately representing featureless subjects. However, fusing these two sensor modalities is challenging due to their fundamentally different characteristics. Rather than directly fusing image features and LIDAR points, we propose to leverage common geometric features that are detected in both the LIDAR scans and image data, allowing data from the two sensors to be processed in a higher-level space. In particular, we propose to find correspondences between 3D lines extracted from LIDAR scans and 2D lines detected in images before performing a bundle adjustment to refine poses. We also exploit the detected and optimized line segments to improve the quality of the final mesh. We test our approach on the recently published dataset, Newer College Dataset. We compare the accuracy and the completeness of the 3D mesh to a ground truth obtained with a survey-grade 3D scanner. We show that our method delivers results that are comparable to a state-of-the-art LIDAR survey while not requiring highly accurate ground truth pose estimates.

PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments

IEEE Transactions on Robotics, 2019

Traditional approaches to stereo visual SLAM rely on point features to estimate the camera trajectory and build a map of the environment. In lowtextured environments, though, it is often difficult to find a sufficient number of reliable point features and, as a consequence, the performance of such algorithms degrades. This paper proposes PL-SLAM, a stereo visual SLAM system that combines both points and line segments to work robustly in a wider variety of scenarios, particularly in those where point features are scarce or not well-distributed in the image. PL-SLAM leverages both points and segments at all the instances of the process: visual odometry, keyframe selection, bundle adjustment, etc. We contribute also with a loop closure procedure through a novel bag-ofwords approach that exploits the combined descriptive power of the two kinds of features. Additionally, the resulting map is richer and more diverse in 3D elements, which can be exploited to infer valuable, highlevel scene structures like planes, empty spaces, ground plane, etc. (not addressed in this work). Our proposal has been tested with several popular datasets (such as KITTI and EuRoC), and is compared to state of the art methods like ORB-SLAM, revealing a more robust performance in most of the experiments, while still running in real-time. An open source version of the PL-SLAM C++ code will be released for the benefit of the community.

Monocular-vision based SLAM using Line Segments

This paper presents a method to incorporate 3D line segments in vision based SLAM. A landmark initialization method that relies on the Plücker coordinates to represent a 3D line is introduced: a Gaussian sum approximates the feature initial state and is updated as new observations are gathered by the camera. Once initialized, the landmarks state is estimated along an EKF-based SLAM approach: constraints associated with the Plücker representation are considered during the update step of the Kalman filter. The whole SLAM algorithm is validated in simulation runs and results obtained with real data are presented.

FAST RANGE IMAGE SEGMENTATION FOR INDOOR 3D-SLAM

6th IFAC Symposium on Intelligent …, 2007

Real-time 3D localization and mapping is eventually needed in many service robotic applications. Toward a light and practical SLAM algorithm, we focus on feature extraction via segmentation of range images. Using horizontal and vertical traces of the range matrix, 2D observed polygons are considered for calculation of a one-dimensional measure of direction, called Bearing Angle (BA). BA is the incident angle between the laser beam and edges of the observed polygon by the scanner in the selected direction. Based on this measure, two different approaches to range image segmentation, region-and edge-based, are proposed and evaluated through a set of standard analysis. It is experimentally shown that for navigation applications, edge based approaches are more efficient. Extensive tests on real robots prove BA-based segmentation is successful for SLAM.

LIDAR-based 3 D Object Perception

2008

This paper describes a LIDAR-based perception system for ground robot mobility, consisting of 3D object detection, classification and tracking. The presented system was demonstrated on-board our autonomous ground vehicle MuCAR-3, enabling it to safely navigate in urban traffic-like scenarios as well as in off-road convoy scenarios. The efficiency of our approach stems from the unique combination of 2D and 3D data processing techniques. Whereas fast segmentation of point clouds into objects is done in a 2 1 2 D occupancy grid, classifying the objects is done on raw 3D point clouds. For fast switching of domains, the occupancy grid is enhanced to act like a hash table for retrieval of 3D points. In contrast to most existing work on 3D point cloud classification, where realtime operation is often impossible, this combination allows our system to perform in real-time at 0.1s frame-rate.