Optimisation of photometric stereo methods by non-convex variational minimisation (original) (raw)
Related papers
A Non-convex Variational Approach to Photometric Stereo under Inaccurate Lighting
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
This paper tackles the photometric stereo problem in the presence of inaccurate lighting, obtained either by calibration or by an uncalibrated photometric stereo method. Based on a precise modeling of noise and outliers, a robust variational approach is introduced. It explicitly accounts for self-shadows, and enforces robustness to castshadows and specularities by resorting to redescending Mestimators. The resulting non-convex model is solved by means of a computationally efficient alternating reweighted least-squares algorithm. Since it implicitly enforces integrability, the new variational approach can refine both the intensities and the directions of the lighting.
Procedings of the British Machine Vision Conference 2016, 2016
Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. In this paper we propose a joint variational method that combines the advantages of both strategies. By integrating recent stereo and SfS models into a single minimisation framework, we obtain an approach that exploits shading information to improve upon the reconstruction quality of robust stereo methods. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the novel model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes it applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images show the advantages of our combined approach: While the stereo part overcomes the albedo-depth ambiguity inherent to all SfS methods, the SfS part improves the degree of details of the reconstruction compared to pure stereo methods.
Direct Shape Recovery from Photometric Stereo with Shadows
2013 International Conference on 3D Vision, 2013
Reconstruction of 3D objects based on images is useful in many applications. One of the methods based on multi-image data is the Photometric Stereo technique relying on several photographs of the observed object from the same point of view, each one taken under a different illumination condition. The common approach is to estimate the gradient field of the surface by minimizing a functional, integrating the distance from the camera and thereby obtaining the geometry of the observed object. We propose an alternative method that consists of a novel differential approach for multi-image Photometric Stereo and permits a direct solution of a novel PDE based model without going through the gradient field while naturally dealing with shadowed regions. The mathematical well-posedness of the problem in terms of numerical stability yields a fast algorithm that efficiently converges, even for pictures of sizes in the order of several megapixels affected by noise.
Solving the Uncalibrated Photometric Stereo Problem Using Total Variation
Lecture Notes in Computer Science, 2013
In this paper we propose a new method to solve the problem of uncalibrated photometric stereo, making very weak assumptions on the properties of the scene to be reconstructed. Our goal is to solve the generalized bas-relief ambiguity (GBR) by performing a total variation regularization of both the estimated normal field and albedo. Unlike most of the previous attempts to solve this ambiguity, our approach does not rely on any prior information about the shape or the albedo, apart from its piecewise smoothness. We test our method on real images and obtain results comparable to the state-of-the-art algorithms.
A L^1$$ -TV Algorithm for Robust Perspective Photometric Stereo with Spatially-Varying Lightings
Lecture Notes in Computer Science, 2015
We tackle the problem of perspective 3D-reconstruction of Lambertian surfaces through photometric stereo, in the presence of outliers to Lambert's law, depth discontinuities, and unknown spatiallyvarying lightings. To this purpose, we introduce a robust L 1 -TV variational formulation of the recovery problem where the shape itself is the main unknown, which naturally enforces integrability and permits to avoid integrating the normal field.
Solving Uncalibrated Photometric Stereo Using Total Variation
Journal of Mathematical Imaging and Vision, 2014
Estimating the shape and appearance of an object, given one or several images, is still an open and challenging research problem called 3D-reconstruction. Among the different techniques available, photometric stereo produces highly accurate results when the lighting conditions have been identified. When these conditions are unknown, the problem becomes the so-called uncalibrated photometric stereo problem, which is illposed. In this paper, we will show how total variation (TV) can be used to reduce the ambiguities of uncalibrated photometric stereo, and we will study two methods for estimating the parameters of the generalized bas-relief ambiguity. These methods will be evaluated through the 3D-reconstruction of real-world objects.
Variational Shape and Reflectance Estimation Under Changing Light and Viewpoints
Lecture Notes in Computer Science, 2006
Fitting parameterized 3D shape and general reflectance models to 2D image data is challenging due to the high dimensionality of the problem. The proposed method combines the capabilities of classical and photometric stereo, allowing for accurate reconstruction of both textured and non-textured surfaces. In particular, we present a variational method implemented as a PDE-driven surface evolution interleaved with reflectance estimation. The surface is represented on an adaptive mesh allowing topological change. To provide the input data, we have designed a capture setup that simultaneously acquires both viewpoint and light variation while minimizing self-shadowing. Our capture method is feasible for real-world application as it requires a moderate amount of input data and processing time. In experiments, models of people and everyday objects were captured from a few dozen images taken with a consumer digital camera. The capture process recovers a photo-consistent model of spatially varying Lambertian and specular reflectance and a highly accurate geometry.
Perspective Photometric Stereo with Shadows
Lecture Notes in Computer Science, 2013
High resolution reconstruction of 3D surfaces from images remains an active area of research since most of the methods in use are based on practical assumptions that limit their applicability. Furthermore, an additional complication in all active illumination 3D reconstruction methods is the presence of shadows, whose presence cause loss of information in the image data. We present an approach for the reconstruction of surfaces via Photometric Stereo, based on the perspective formulation of the Shape from Shading problem, solved via partial differential equations. Unlike many photometric stereo solvers that use computationally costly variational methods or a two-step approach, we use a novel, well-posed, differential formulation of the problem that enables us to solve a first order partial differential equation directly via an alternating directions raster scanning scheme. The resulting formulation enables surface computation for very large images and allows reconstruction in the presence of shadows.
Photometric Stereo with General, Unknown Lighting
International Journal of Computer Vision, 2006
Work on photometric stereo has shown how to recover the shape and reflectance properties of an object using multiple images taken with a fixed viewpoint and variable lighting conditions. This work has primarily relied on the presence of a single point source of light in each image. In this paper we show how to perform photometric stereo assuming that all lights in a scene are isotropic and distant from the object but otherwise unconstrained. Lighting in each image may be an unknown and arbitrary combination of diffuse, point and extended sources. Our work is based on recent results showing that for Lambertian objects, general lighting conditions can be represented using low order spherical harmonics. Using this representation we can recover shape by performing a simple optimization in a low-dimensional space. We also analyze the shape ambiguities that arise in such a representation.
Photometric Stereo Using Constrained Bivariate Regression for General Isotropic Surfaces
2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014
This paper presents a photometric stereo method that is purely pixelwise and handles general isotropic surfaces in a stable manner. Following the recently proposed sumof-lobes representation of the isotropic reflectance function, we constructed a constrained bivariate regression problem where the regression function is approximated by smooth, bivariate Bernstein polynomials. The unknown normal vector was separated from the unknown reflectance function by considering the inverse representation of the image formation process, and then we could accurately compute the unknown surface normals by solving a simple and efficient quadratic programming problem. Extensive evaluations that showed the state-of-the-art performance using both synthetic and real-world images were performed.