Variable albedo surface reconstruction from stereo and shape from shading (original) (raw)

We p r e s e n t a m ultiview method for the computation of object shape and re ectance characteristics based on the integration of shape from shading (SFS) and stereo, for nonconstan talbedo and non-uniformly Lambertian surfaces. First we perform stereo tting on the input stereo pairs or image sequences. When the images are uncalibrated, w e recover the camera parameters using bundle adjustment. Based on the stereo result, we can automatically segment the albedo map (which i s t a k en to be piece-wise constant) using a Minimum Description Length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas) and to deriv e illumination information. The shape and the illumination parameter estimates are re ned using a deformable model SFS algorithm, which i terates bet w een computing shape and illumination parameters. Our method takes into accoun tthe viewing angle dependent foreshortening and specularity e ects, and compensates as much as possible by utilizing information from more than one images. We demonstrate that we can extend the applicability of SFS algorithms to real world situations when some of its traditional assumptions are violated. We demonstrate our method by applying it to face shape reconstruction. Experimental results indicate a signi cant improvement over SFS-only or stereo-only based reconstruction. Model accuracy and detail are improved, especially in areas of low texture detail. Albedo information is retrieved and can be used to accurately re-render the model under di erent illumination conditions.