View Synthesis Research Papers - Academia.edu (original) (raw)

5 Followers

Recent papers in View Synthesis

In this paper a novel intermediate view synthesis algorithm for 3D video rendering is presented. The proposal is based on the representation of the pixels to be warped as intervals of finite length. The warping, hole-filling and merging... more

In this paper a novel intermediate view synthesis algorithm for 3D video rendering is presented. The proposal is based on the representation of the pixels to be warped as intervals of finite length. The warping, hole-filling and merging stages of the view synthesis procedure can be recast according to this notation with significant advantages from the point of view of algorithm complexity, memory requirements and image synthesis quality. A novel interpolation rule based on the definition of a foreground and a background weight associated to each warped pixel is proposed and compared with linear interpolation based on pixels distance. The experimental results show that the designed technique yields a noticeable gain in terms of rendered image quality, while at the same time halving the execution time with respect to the MPEG view synthesis reference software.

In this paper we describe an image-based rendering pipeline for interactive real-time Free Viewpoint Television (FTV) on embedded systems. Description of the processing steps and optimizations implemented targeting the hardware... more

In this paper we describe an image-based rendering pipeline for interactive real-time Free Viewpoint Television (FTV) on embedded systems. Description of the processing steps and optimizations implemented targeting the hardware acceleration of a commercial programmable Graphics Processing Unit (GPU) is given. As a result, real-time view synthesis at 70 fps in XGA resolution has bee achieved. Restrictions and modifications introduced to support the application on OpenGL ES 2.0 based GPUs for embedded systems have also been discussed.

Multi-view stereo, novel view synthesis and high dynamic range (HDR) imaging are three pertinent areas of concern for high quality 3D view generation. This paper presents a novel parameterized variety based model that integrates these... more

Multi-view stereo, novel view synthesis and high dynamic range (HDR) imaging are three pertinent areas of concern for high quality 3D view generation. This paper presents a novel parameterized variety based model that integrates these different domains into one common framework with an envisioned goal, to accommodate multi-view stereo for multiple exposure input views and to render photo-realistic HDR images from arbitrary virtual viewpoints for high quality 3D reconstruction. We extend the parameterized variety approach for rendering presented earlier by Genc and Ponce [1] to handle full perspective cameras. An efficient algebraic framework is proposed to construct an explicit parameterization of the space of all multi-view multi-exposed images. This characterization of differently exposed views allow to simultaneously recover artifacts free HDR images, and reliable depth maps from arbitrary camera viewpoints. High quality, HDR textured 3D model of the scene is obtained using these images and recovered geometry information.

This paper presents a novel rendering algorithm based on depth image warping to support virtual pan-tilt-zoom (PTZ) functionalities during 3D view generation. A method based on "3D-ness" knob is proposed for automatically specifying... more

This paper presents a novel rendering algorithm based on
depth image warping to support virtual pan-tilt-zoom (PTZ)
functionalities during 3D view generation. A method based
on "3D-ness" knob is proposed for automatically specifying
the virtual camera positions along a path, to model the PTZ
mechanism in projective framework. Two novel quality enhancing techniques based on segmentation cues are proposed to add pan, tilt and zoom capabilities during arbitrary view synthesis. In addition to reduce the computational load that results in providing such functionalities, a novel selective warping scheme is presented to reduce the computational cost by as much as 40% while maintaining an acceptable quality of rendering results. Experiments are performed using standard "Breakdancers" and “Ballet” video sequences to demonstrate the effectiveness of the proposed methods as compared to currently published results.

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the... more

Coliseum is a multiuser immersive remote teleconferencing system designed to provide collaborative workers the experience of face-to-face meetings from their desktops. Five cameras are attached to each PC display and directed at the participant. From these video streams, view synthesis methods produce arbitrary-perspective renderings of the participant and transmit them to others at interactive rates, currently about 15 frames per second. Combining these renderings in a shared synthetic environment gives the appearance of having all participants interacting in a common space. In this way, Coliseum enables users to share a virtual world, with acquired-image renderings of their appearance replacing the synthetic representations provided by more conventional avatar-populated virtual worlds. The system supports virtual mobility---participants may move around the shared space---and reciprocal gaze, and has been demonstrated in collaborative sessions of up to ten Coliseum workstations, an...

This paper presents a new depth-image-based rendering algorithm for free-viewpoint 3DTV applications. The cracks, holes, ghost countors caused by visibility, disocclusion, resampling problems associated with 3D warping lead to serious... more

This paper presents a new depth-image-based rendering algorithm
for free-viewpoint 3DTV applications. The cracks,
holes, ghost countors caused by visibility, disocclusion, resampling
problems associated with 3D warping lead to serious
rendering artifacts in synthesized virtual views. This challenging
problem of hole filling is formulated as an algebraic
matrix completion problem on a higher dimensional space of
monomial features described by a novel variety model. The
high-level idea of this work is to exploit the linear or nonlinear
structures of the data and interpolate missing values by solving
algebraic varieties associated with Hankel matrices as a
member of Krylov subspace. The proposed model effectively
handles artifacts appear in wide-baseline spatial view interpolation
and arbitrary camera movements. Our model has a
low runtime and results excel with state-of-the-art methods in
quantitative and qualitative evaluation.

Mug shot photography has been used to identify criminals by the police for more than a century. However, the common scenario of face recognition using frontal and side-view mug shots as gallery remains largely uninvestigated in... more

Mug shot photography has been used to identify criminals by the police for more than a century. However, the common scenario of face recognition using frontal and side-view mug shots as gallery remains largely uninvestigated in computerized face recognition across pose. This paper presents a novel appearance-based approach using frontal and sideface images to handle pose variations in face recognition,

ABSTRACT D Video (3DV) delivery standardization is currently ongoing in MPEG. Now time is to choose 3DV data representation format. What is at stake is the final quality for end-users, i.e. synthesized views' visual quality. We... more

ABSTRACT D Video (3DV) delivery standardization is currently ongoing in MPEG. Now time is to choose 3DV data representation format. What is at stake is the final quality for end-users, i.e. synthesized views' visual quality. We focus on two major rival depth-based formats, namely Multiview Video plus Depth (MVD) and Layered Depth Video (LDV). MVD can be considered as the basic depth-based 3DV format, generated by disparity estimation from multiview sequences. LDV is more sophisticated, with the compaction of multiview data into color- and depth-occlusions layers. We compare final views quality using MVD2 and LDV (both containing two color channels plus two depth components) coded with MVC at various compression ratios. Depending on the format, the appropriate synthesis process is performed to generate final stereoscopic pairs. Comparisons are provided in terms of SSIM and PSNR with respect to original views and to synthesized references (obtained without compression). Eventually, LDV outperforms significantly MVD when using state-of-the-art reference synthesis algorithms. Occlusions management before encoding is advantageous in comparison with handling redundant signals at decoder side. Besides, we observe that depth quantization does not induce much loss on the final view quality until a significant degradation level. Improvements in disparity estimation and view synthesis algorithms are therefore still expected during the remaining standardization steps.

3D-TV applications require the generation of novel viewpoints through Depth-Image-Based-Rendering methods. These synthesized views need to be assessed by a reliable quality metric. Most of the proposed metrics are inspired from 2D... more

3D-TV applications require the generation of novel viewpoints through Depth-Image-Based-Rendering methods. These synthesized views need to be assessed by a reliable quality metric. Most of the proposed metrics are inspired from 2D commonly used quality metrics. Yet, the latter were originally designed to address 2D compression distortions which are different from the distortions related to DIBR processes. We propose an

Log In