Single-image reflectance estimation for relighting by iterative soft grouping (original) (raw)

Smooth Reconstruction and Compact Representation of Reflectance Functions for Image-based Relighting

Abstract In this paper we present a new method,to reconstruct reectance,functions for image-based relighting. A reectance,function describes how,a pixel in a photograph is observed,depending,on the incident illumination on the depicted object. Additionally we present a compact representation of the reconstructed reectance,functions. The reectance,functions are sampled,from real objects by illuminating the object from a set of directions while recording photographs. Each pixel in a photograph is a sample of the reectance,function. Next, a smooth continuous function is reconstructed, using different reconstruction techniques, from the sampled reectance,function. The presented method,maintains important high frequency,features such as highlights and self-shadowing and ensures visually pleasing relit images, computed with incident illumination containing high and low frequency features. The reconstructed reectance,functions and incident illumination can be expressed by a common,set of b...

Relighting objects from image collections

2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009

We present an approach for recovering the reflectance of a static scene with known geometry from a collection of images taken under distant, unknown illumination. In contrast to previous work, we allow the illumination to vary between the images, which greatly increases the applicability of the approach. Using an all-frequency relighting framework based on wavelets, we are able to simultaneously estimate the per-image incident illumination and the persurface point reflectance. The wavelet framework allows for incorporating various reflection models. We demonstrate the quality of our results for synthetic test cases as well as for several datasets captured under laboratory conditions. Combined with multi-view stereo reconstruction, we are even able to recover the geometry and reflectance of a scene solely using images collected from the Internet.

Example-based reflectance estimation for capturing relightable models of people

IET 5th European Conference on Visual Media Production (CVMP 2008), 2008

We present a new approach to reflectance estimation for dynamic scenes. Non-parametric image statistics are used to transfer reflectance properties from a static example set to a dynamic image sequence. The approach allows reflectance estimation for surface materials with inhomogeneous appearance, such as those which commonly occur with patterned or textured clothing. Material reflectance properties are initially estimated from static images of the subject under multiple directional illuminations using photometric stereo. The estimated reflectance together with the corresponding image under uniform ambient illumination form a prior set of reference material observations. Material reflectance properties are then estimated for video sequences of a moving person captured under uniform ambient illumination by matching the observed local image statistics to the reference observations. Results demonstrate that the transfer of reflectance properties enables estimation of the dynamic surface normals and subsequent relighting. This approach overcomes limitations of previous work on material transfer and relighting of dynamic scenes which was limited to surfaces with regions of homogeneous reflectance. We evaluate for relighting 3D model sequences reconstructed from multiple view video. Comparison to previous model relighting demonstrates improved reproduction of detailed texture and shape dynamics.

Interactive relighting with dynamic BRDFs

ACM Transactions on Graphics, 2007

We present a technique for interactive relighting in which source radiance, viewing direction, and BRDFs can all be changed on the fly. In handling dynamic BRDFs, our method efficiently accounts for the effects of BRDF modification on the reflectance and incident radiance at a surface point. For reflectance, we develop a BRDF tensor representation that can be factorized into adjustable terms for lighting, viewing, and BRDF parameters. For incident radiance, there exists a non-linear relationship between indirect lighting and BRDFs in a scene, which makes linear light transport frameworks such as PRT unsuitable. To overcome this problem, we introduce precomputed transfer tensors (PTTs) which decompose indirect lighting into precomputable components that are each a function of BRDFs in the scene, and can be rapidly combined at run time to correctly determine incident radiance. We additionally describe a method for efficient handling of high-frequency specular reflections by separating...

Low-cost photometric calibration for interactive relighting

In Proceedings of the First French-British …, 2000

Computer augmented reality is a rapidly emerging field allowing users to mix virtual and real worlds. Our interest is to allow relighting and remodelling of real scenes, using a reflectance estimation method. Most previous work focused on the quality of the results without considering the expense in computation and the price of acquisition equipment. In this paper, we present a low–cost photometric calibration method which improves the reflectance estimate of real scenes. This is achieved by adapting high-dynamic range ...

Recovering surface reflectance and multiple light locations and intensities from image data

Pattern Recognition Letters

We present a method to recover the reflectance of objects and the parameters of multiple lights using a 3D image acquired by a depth sensor and a stereo intensity pair. Experimental evaluation shows the ability to recover varying diffuse and constant specular reflectance parameters from object images, and simultaneously the locations and intensities of up to three distinct light sources.

Image relighting using shading proxies

2014

We present a practical solution to the problem of single-image relighting of objects with arbitrary shapes. It is based on a shading-ratio image obtained from some userprovided guess of the original and target illuminations. Our approach is flexible and robust, being applicable to objects with non-uniform albedos and arbitrary shapes, as well as to nonphotorrealistic depictions. We demonstrate its effectiveness by relighting many photographs, paintings, and drawings containing a variety of objects of different materials. Additionally, our technique can transfer smooth normal and depth maps from 3-D models to pictures. Preliminary evaluation has shown that our approach is intuitive, allowing novice users to relight images in just a couple of minutes. Keywords-image relighting; painting relighting; normal-map estimation; shading transferring;

Reflectance Transfer for Material Editing and Relighting

2010

We present a new approach to diffuse reflectance estimation for dynamic scenes. Non-parametric image statistics are used to transfer reflectance properties from a static example set to a dynamic image sequence. The approach allows diffuse reflectance estimation for surface materials with inhomogeneous appearance, such as those which commonly occur with patterned or textured clothing. Material editing is also possible by transferring edited reflectance properties. Material reflectance properties are initially estimated from static images of the subject under multiple directional illuminations using photometric stereo. The estimated reflectance together with the corresponding image under uniform ambient illumination form a prior set of reference material observations. Material reflectance properties are then estimated for video sequences of a moving person captured under uniform ambient illumination by matching the observed local image statistics to the reference observations. Results...

Reflectance Sharing: Image-based Rendering from a Sparse Set of Images

When the shape of an object is known, its appearance is determined by the spatially-varying reflectance function defined on its surface. Image-based rendering methods that use geometry seek to estimate this function from image data. Most existing methods recover a unique angular reflectance function (e.g., BRDF) at each surface point and provide reflectance estimates with high spatial resolution. Their angular accuracy is limited by the number of available images, and as a result, most of these methods focus on capturing parametric or low-frequency angular reflectance effects, or allowing only one of lighting or viewpoint variation. We present an alternative approach that enables an increase in the angular accuracy of a spatially-varying reflectance function in exchange for a decrease in spatial resolution. By framing the problem as scattered-data interpolation in a mixed spatial and angular domain, reflectance information is shared across the surface, exploiting the high spatial resolution that images provide to fill the holes between sparsely observed view and lighting directions. Since the BRDF typically varies slowly from point to point over much of an object's surface, this method enables image-based rendering from a sparse set of images without assuming a parametric reflectance model. In fact, the method can even be applied in the limiting case of a single input image.

Near Light Correction for Image Relighting and 3D Shape Recovery

In this paper, we propose a near-light illumination model for image relighting and 3D shape recovery. Classic methods such as used by the popular RTI software from Cultural Heritage Imaging assume that lighting is infinitely far away from the scene. However, this constraint is impossible to achieve in practice: light sources cannot be too far away from the scene due to space and power constraints. This causes non-uniform illumination artifacts due to variations in the distance between the light source and points in the scene. We correct this effect to provide much more uniformly lit images that yield more appealing image for relighting applications. Furthermore, we use our near-light model for more accurate photometric stereo calculations of surface normals, eliminating the “potato-chip” shaped surface reconstruction error that results from violating the far-light assumption. We verify our model with both free-form capture using hand-held flash as the illumination source, and capture using LED lights mounted on a dome shaped surface.