IMAGE BASED RENDERING WITH DEPTH CAMERAS: HOW MANY ARE NEEDED (original) (raw)
Related papers
Error Analysis for Image-Based Rendering With Depth Information
IEEE Transactions on Image Processing, 2000
We quantitatively analyze the rendering quality of image-based rendering (IBR) algorithms using per-pixel depth. Assuming the ideal pinhole camera model, we show that IBR errors can be quantified using IBR configurations such as the depth and intensity estimate errors, the scene geometry and texture, the number of actual cameras, their positions and resolution. Experiments with synthetic and real scenes show that the theoretical bounds accurately characterize the rendering errors. In particular, the proposed analysis suggests that, in smooth regions, the decay rates of the virtual images' mean absolute errors are O(λ −1) and O(λ −2), where λ is the local density of actual samples, for 2D and 3D scenes, respectively. We discuss the implications of the proposed analysis on camera placement, budget allocation, and bit allocation.
Depth Images: Representations and Real-Time Rendering
Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), 2006
Depth Images are viable representations that can be computed from the real world using cameras and/or other scanning devices. The depth map provides 2-1 2 D structure of the scene. A set of Depth Images can provide hole-free rendering of the scene. Multiple views need to blended to provide smooth hole-free rendering, however. Such a representation of the scene is bulky and needs good algorithms for real-time rendering and efficient representation. In this paper, we present a discussion on the Depth Image representation and provide a GPU-based algorithm that can render large models represented using DIs in real time. We then present a proxy-based compression scheme for Depth Images and provide results for the same. Results are shown on synthetic scenes under different conditions and on some scenes generated from images. Lastly, we initiate discussion on varying quality levels in IBR and show a way to create representations using DIs with different trade-offs between model size and rendering quality. This enables the use of this representation for a variety of rendering situations.
Rendering Layered Depth Images
1997
In this paper we present an efficient image based rendering system that renders multipleframes per second on a PC. Our method performs warping from an intermediate representationcalled a layered depth image (LDI). An LDI is a view of the scene from a single inputcamera view, but with multiple pixels along each line of sight. When n input images arepreprocessed to
Plenoptic Layer-Based Modeling for Image Based Rendering
IEEE Transactions on Image Processing, 2000
Image based rendering is an attractive alternative to model based rendering for generating novel views due to its lower complexity and potential for photo-realistic results. In order to reduce the number of images necessary for aliasfree rendering, some geometric information for the 3D scene is normally necessary. In this paper, we present a fast automatic layer-based method for synthesising an arbitrary new view of a scene from a set of existing views. Our algorithm takes advantage of the knowledge of the typical structure of multiview data in order to perform occlusion-aware layer extraction. Moreover, the number of depth layers used to approximate the geometry of the scene is chosen based on Plenoptic sampling theory with the layers placed non-uniformly to account for the scene distribution. The rendering is achieved by using a probabilistic interpolation approach and by extracting the depth layer information on a small number of key images. Numerical results demonstrate that the algorithm is fast and yet is only 0.25 dB away from the ideal performance achieved with the ground-truth knowledge of the 3D geometry of the scene of interest. This indicates that there are measurable benefits from following the predictions of Plenoptic theory and that they remain true when translated into a practical system for real world data.
International Conference in Central Europe on Computer Graphics and Visualization, 2017
Layered Depth Images (LDI) compactly represent multiview images and videos and have widespread usage in image-based rendering applications. In its typical use case scenario of representing a scanned environment, it has proven to be a less costly alternative than separate viewpoint encoding. However, higher quality laser scanner hardware and different user interaction paradigms have emerged, creating scenarios where traditional LDIs have considerably lower efficacy. Wide-baseline setups create surfaces aligned to the viewing rays producing a greater amount of sparsely populated layers. Free viewpoint visualization suffers from the variant quantization of depths on the LDI algorithm, reducing resolution of the dataset in uneven directions. This paper presents an alternative representation to the LDI, in which each layer of data is positioned in different viewpoints that coincide with the original scanning viewpoints. A redundancy removal algorithm based on world-space distances as opposed to to image-space is discussed, ensuring points are evenly distributed and are not viewpoint dependent. We compared our proposed representation with traditional LDIs and viewpoint dependent encoding. Results showed the multiview LDI (MVLDI) creates a smaller number of layers and removes higher amounts of redundancy than traditional LDIs, ensuring no relevant portion of data is discarded in wider baseline setups.
A framework for depth image-based modeling and rendering
2003
We present an image-based system for automatic modeling and interactive rendering of 3D objects. We describe our contribution to image-based modeling and interactive multi-resolution rendering algorithms. Our representation is based on images with depths, which allow it to be compact and flexible, suitable for static and animated scenes, simplify streaming network transmission. The representation has been proposed and accepted into MPEG-4 AFX (Animated Framework eXtension).
1998
In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan's warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alphacompositing can be done efficiently without depth sorting. This makes splatting an efficient solution to the resampling problem.
Rendering multi-view plus depth data on light-field displays
2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2011
This paper presents an approach for rendering heavily extrapolated novel views to be used as input for light-field displays. This view generation method builds on a combination and enhancement of existing methods. The interpolation quality is assured by detecting and keeping the most reliable gap area information from the content using depth layers. Concerning the extrapolation process, which is the most important part of this paper, we implemented an algorithm that prefers isophotes lines in order to reconstruct objects and patterns using gradient filling and Poisson reconstruction. Using the algorithms described, it is possible to generate wide baseline light field data from Multi-View plus Depth (MVD) data of moderate baseline. The approach is demonstrated by generating interpolated and extrapolated views for feeding a HoloVizio large-scale display with captured video data.
Accurate non-iterative depth layer extraction algorithm for image based rendering
2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011
Image based rendering is an attractive alternative for generating novel views compared to model based rendering due to its lower complexity and potential for photo-realistic results. We present a fast unsupervised method for synthesising arbitrary viewpoints of a scene from a set of existing views. Our novel improvements include optimising the placement of depth layers to take advantage of the composition of real world scenes and hierarchically building our simple geometric model to maximise its accuracy.
Enhancing Depth Accuracy on the Region of Interest in a Scene for Depth Image Based Rendering
This research proposed the domain division depth map quantization for multiview intermediate image generation using Depth Image-Based Rendering (DIBR). This technique used per-pixel depth quantization according to the percentage of depth bits assigned in domains of depth range. A comparative experiment was conducted to investigate the potential benefits of the proposed method against the linear depth quantization on DIBR multiview intermediate image generation. The experiment evaluated three quantization methods with computer-generated 3D scenes, which consisted of various scene complexities and backgrounds, under varying the depth resolution. The results showed that the proposed domain division depth quantization method outperformed the linear method on the 7-bit or lower depth map, especially in the scene with the large object.