Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity (original) (raw)

Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor

Optics Express, 2011

Range acquisition systems such as light detection and ranging (LIDAR) and time-of-flight (TOF) cameras operate by measuring the time difference of arrival between a transmitted pulse and the scene reflection. We introduce the design of a range acquisition system for acquiring depth maps of piecewise-planar scenes with high spatial resolution using a single, omnidirectional, time-resolved photodetector and no scanning components. In our experiment, we reconstructed 64 × 64-pixel depth maps of scenes comprising two to four planar shapes using only 205 spatially-patterned, femtosecond illuminations of the scene. The reconstruction uses parametric signal modeling to recover a set of depths present in the scene. Then, a convex optimization that exploits sparsity of the Laplacian of the depth map of a typical scene determines correspondences between spatial positions and depths. In contrast with 2D laser scanning used in LIDAR systems and low-resolution 2D sensor arrays used in TOF cameras, our experiment demonstrates that it is possible to build a non-scanning range acquisition system with high spatial resolution using only a standard, low-cost photodetector and a spatial light modulator.

A single photon detector array with 64-64 resolution and millimetric depth accuracy for 3D imaging

ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005., 2005

An avalanche photodiode array uses single-photon counting to perform time-of-flight range-finding on a scene uniformly hit by 100ps 250mW uncollimated laser pulses. The 32×32 pixel sensor, fabricated in a 0.8 μm CMOS process uses a microscanner package to enhance the effective resolution in the application to 64×64 pixels. The application achieves a measurement depth resolution of 1.3mm to a depth

Single-pixel imaging via compressive sampling

IEEE Signal Processing …, 2008

Humans are visual animals, and imaging sensors that extend our reach—cameras—have improved dramatically in recent times thanks to the introduction of CCD and CMOS digital technology. Consumer digital cameras in the megapixel range are now ubiquitous thanks to the happy coincidence that the semiconductor material of choice for large-scale electronics integration (silicon) also happens to readily convert photons at visual wavelengths into electrons. On the contrary, imaging at wavelengths where silicon is blind is considerably more complicated, bulky, and expensive. Thus, for comparable resolution, a US$500 digital camera for the visible becomes a US$50,000 camera for the infrared. In this article, we present a new approach to building simpler, smaller, and cheaper digital cameras that can operate efficiently across a much broader spectral range than conventional silicon-based cameras. Our approach fuses a new camera architecture based on a digital micromirror device with the new mathematical theory and algorithms of compressive sampling. CS combines sampling and compression into a single nonadaptive linear measurement process [1]–[4]. Rather than measuring pixel samples of the scene under view, we measure inner products between the scene and a set of test functions. Interestingly, random test functions play a key role, making each measurement a random sum of pixel values taken across the entire image. When the scene under view is compressible by an algorithm like JPEG or JPEG2000, the CS theory enables us to stably reconstruct an image of the scene from fewer measurements than the number of reconstructed pixels. In this manner we achieve sub-Nyquist image acquisition. Our “single-pixel” CS camera architecture is basically an optical computer (comprising a DMD, two lenses, a single photon detector, and an analog-to-digital (A/D) converter) that computes random linear measurements of the scene under view. The image is then recovered or processed from the measurements by a digital computer. The camera design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of exotic detectors that would be impossible in a conventional digital camera. The random CS measurements also enable a tradeoff between space and time during image acquisition. Finally, since the camera compresses as it images, it has the capability to efficiently and scalably handle high-dimensional data sets from applications like video and hyperspectral imaging.

Multireturn compressed gated range imaging

Optical Engineering, 2015

Active range imaging (RI) systems utilize actively controlled light sources emitting laser pulses that are subsequently recorded by an imaging system and used for depth profile estimation. Classical RI systems are limited by their need for a large number of frames required to obtain high resolution depth information. In this work, we propose an RI approach motivated by the recently proposed compressed sensing framework to dramatically reduce the number of necessary frames. Compressed gated range sensing employs a random gating mechanism along with state-of-the-art reconstruction algorithms for the estimation of the timing of the reflected pulses and the inference of distances. In addition to efficiency, the proposed scheme is also able to identify multiple reflected pulses that can be introduced by semi-transparent elements in the scene such as clouds, smoke, and foliage. Simulations under highly realistic conditions demonstrate that the proposed architecture is capable of accurately recovering the depth profile of a scene from as few as 10 frames at 100 depth bins resolution, even under very challenging conditions. The results further indicate that the proposed architecture is able to extract multiple reflected pulses with a minimal increase in the number of frames, in situations where state-of-the-art methods fail to accurately estimate the correct depth signals. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Low Intensity LiDAR using Compressed Sensing and a Photon Number Resolving Detector

Proc. SPIE 10546, Emerging Digital Micromirror Device Based Systems and Applications X, 105460J, 2018

LiDAR (laser based radar) systems are a major part of many new real-world interactive systems, one of the most notable being autonomous cars. The current market LiDAR systems are limited by detector sensitivity: when output power is at eye-safe levels, the range is limited. Long range operation also slows image acquisition as flight-time increases. We present an approach that combines a high sensitivity photon number resolving diode with machine learning and a micro-mechanical digital mirror device to achieve safe and fast long range 3D scanning.

Depth sensing using active coherent illumination

2012

Abstract We examine the use of active coherent sensing—an increasingly available technology—for sensing the depth of scenes. A scene is a sparse signal but also exhibits significant structure which cannot be exploited using standard sparse recovery algorithms. Instead, inspired by the model-based compressive sensing literature we develop a scene model that incorporates occlusion constraints in recovering the depth map.

3D imaging and ranging by time-correlated single photon counting

Computing & Control Engineering Journal

3D imaging is an important tool for metrology and reverse engineering of components and structures in a wide variety of contexts, from automobile and aeroplane manufacture to the creative media and architectural surveying. In this article, we review briefly the principal methods in current use for 3D imaging, then present a new method for time of flight depth measurement, which is more accurate and sensitive than the current techniques. To illustrate its potential, we show a number of examples of 3D data acquired from both small and large objects, taking examples from cars, planes and archaeological artefacts. here is an increasing need for 3D imaging systems to acquire accurate range and image data for a variety of industrial applications.

A Real-time Compact Structured-light based Range Sensing System

JSTS:Journal of Semiconductor Technology and Science, 2012

In this paper, we propose a new approach for compact range sensor system for real-time robot applications. Instead of using off-the-shelf camera and projector, we devise a compact system with a CMOS image-sensor and a DMD (Digital Micromirror Device) that yields smaller dimension (168x50x60mm) and lighter weight (500g). We also realize one chip hard-wired processing of projection of structured-light and computing the range by exploiting correspondences between CMOS imagesensor and DMD. This application-specific chip processing is implemented on an FPGA in real-time. Our range acquisition system performs 30 times faster than the same implementation in software. We also devise an efficient methodology to identify a proper light intensity to enhance the quality of range sensor and minimize the decoding error. Our experimental results show that the total-error is reduced by 16% compared to the average case.

Processing time-correlated single photon counting data to acquire range images

IEE Proceedings - Vision, Image, and Signal Processing, 1998

The processing and analysis are described of range data in a time-of-flight imaging system based on time-correlated single photon counting. The system is capable of acquiring range data accurate to 1Opm at a standoff distance in the order of lm, although this can be varied substantially. It is shown how fitting of the pulsed histogram data by a combination of a symmetric key and polynomial functions can improve the accuracy and robustness of the depth data, in comparison with methods based on upsampling and centroid estimation. The imaging capability of the system is also demonstrated.

An Architecture for Compressive Imaging

2006

Compressive Sensing is an emerging field based on the revelation that a small group of non-adaptive linear projections of a compressible signal contains enough information for reconstruction and processing. In this paper, we propose algorithms and hardware to support a new theory of Compressive Imaging. Our approach is based on a new digital image/video camera that directly acquires random projections of the signal without first collecting the pixels/voxels. Our camera architecture employs a digital micromirror array to perform optical calculations of linear projections of an image onto pseudorandom binary patterns. Its hallmarks include the ability to obtain an image with a single detection element while measuring the image/video fewer times than the number of pixels -this can significantly reduce the computation required for video acquisition/encoding. Because our system relies on a single photon detector, it can also be adapted to image at wavelengths that are currently impossible with conventional CCD and CMOS imagers. We are currently testing a prototype design for the camera and include experimental results.