Automated Scene-Specific Selection of Feature Detectors for 3D Face Reconstruction (original) (raw)
Related papers
3D Face Reconstruction From Single 2D Image Using Distinctive Features
3D face reconstruction is considered to be a useful computer vision tool, though it is difficult to build. This paper proposes a 3D face reconstruction method, which is easy to implement and computationally efficient. It takes a single 2D image as input, and gives 3D reconstructed images as output. Our method primarily consists of three main steps: feature extraction, depth calculation, and creation of a 3D image from the processed image using a Basel face model (BFM). First, the features of a single 2D image are extracted using a two-step process. Before distinctive-features extraction, a face must be detected to confirm whether one is present in the input image or not. For this purpose, facial features like eyes, nose, and mouth are extracted. Then, distinctive features are mined by using scale-invariant feature transform (SIFT), which will be used for 3D face reconstruction at a later stage. Second step comprises of depth calculation, to assign the image a third dimension. Multivariate Gaussian distribution helps to find the third dimension, which is further tuned using shading cues that are obtained by the shape from shading (SFS) technique. Thirdly, the data obtained from the above two steps will be used to create a 3D image using BFM. The proposed method does not rely on multiple images, lightening the computation burden. Experiments were carried out on different 2D images to validate the proposed method and compared its performance to those of the latest approaches. Experiment results demonstrate that the proposed method is time efficient and robust in nature, and it outperformed all of the tested methods in terms of detail recovery and accuracy. INDEX TERMS 3D face reconstruction, feature extraction, facial modeling, gaussian distribution.
IMAGE BASED 3D FACE RECONSTRUCTION: A SURVEY
International Journal of Image and Graphics, 2009
The use of 3D data in face image processing applications has received considerable attention during the last few years. A major issue for the implementation of 3D face processing systems is the accurate and real time acquisition of 3D faces using low cost equipment. In this paper we provide a survey of 3D reconstruction methods used for generating the 3D appearance of a face using either a single or multiple 2D images captured with ordinary equipment such as digital cameras and camcorders. In this context we discuss various issues pertaining to the general problem of 3D face reconstruction such as the existence of suitable 3D face databases, correspondence of 3D faces, feature detection, deformable 3D models and typical assumptions used during the reconstruction process. Different approaches to the problem of 3D reconstruction are presented and for each category the most important advantages and disadvantages are outlined. In particular we describe example-based methods, stereo methods, video-based methods and silhouette-based methods. The issue of performance evaluation of 3D face reconstruction algorithms, the state of the art and future trends are also discussed.
Automatic 3D Reconstruction for Face Recognition
2004
An analysis-by-synthesis framework for face recognition with variant pose, illumination and expression (PIE) is proposed in this paper. First, an efficient 2D-to-3D integrated face reconstruction approach is introduced to reconstruct a personalized 3D face model from a single frontal face image with neutral expression and normal illumination; Then, realistic virtual faces with different PIE are synthesized based on the personalized 3D face to characterize the face subspace; Finally, face recognition is conducted based on these representative virtual faces. Compared with other related works, this framework has the following advantages: 1) only one single frontal face is required for face recognition, which avoids the burdensome enrollment work; 2) the synthesized face samples provide the capability to conduct recognition under difficult conditions like complex PIE; and 3) the proposed 2D-to-3D integrated face reconstruction approach is fully automatic and more efficient. The extensive experimental results show that the synthesized virtual faces significantly improve the accuracy of face recognition with variant PIE.
Pattern Analysis for an Automatic and Low-Cost 3D Face Acquisition Technique
2009
This paper proposes an automatic 3D face modeling and localizing technique, based on active stereovision. In the offline stage, the optical and geometrical parameters of the stereosensor are estimated. In the online acquisition stage, alternate complementary patterns are successively projected. The captured right and left images are separately analyzed in order to localize left and right primitives with sub-pixel precision. This analysis also provides us with an efficient segmentation of the informative facial region. Epipolar geometry transforms a stereo matching problem into a one-dimensional search problem. Indeed, we employ an adapted, optimized dynamic programming algorithm to pairs of primitives which are already located in each epiline. 3D geometry is retrieved by computing the intersection of optical rays coming from the pair of matched features. A pipeline of geometric modeling techniques is applied to densify the obtained 3D point cloud, and to mesh and texturize the 3D final face model. An appropriate evaluation strategy is proposed and experimental results are provided.
Towards Robust 3D Face Recognition from Noisy Range Images with Low Resolution
For a number of dierent security and industrial applications, there is the need for reliable person identification methods. Among these meth- ods, face recognition has a number of advantages such as being non-invasive and potentially covert. Since the device for data acquisition is a conventional camera, other advantages of a 2D face recognition system are its low data cap- ture duration and its low cost. However, the recent introduction of fast and comparatively inexpensive time-of-flight (TOF) cameras for the recording of 2.5D range data calls for a closer look at D face recognition in this context. One major disadvantage, however, is the low quality of the data aquired with such cameras. In this paper, we introduce a robust D face recognition system based on such noisy range images with low resolution.
Robust representation of 3D faces for recognition
2006
Automatic face recognition is becoming increasingly important due to the security applications derived from it. Although the face recognition problem has focused on 2D images, recently, due to the proliferation of 3D scanning hardware, 3D face recognition has become a feasible application. This 3D approach do not use the colour of the faces, so, when it is compared to the more traditional 2D approach, has the main advantages of being robust under lighting variations and also of providing more relevant information. In this paper we present a new 3D face model based on curvature properties of the face surface. As will be presented, our system is able to detect the subset of characteristics of the face with higher discrimination power among a large set of them. The robustness of the model is tested by comparing recognition rates using both controlled and non-controlled environments regarding facial expressions and face rotations. The difference between the recognition rate of the two environments of only 5% proves that the model has a high degree of robustness against pose and facial expressions and that is robust enough to implement facial recognition applications, which can achieve up to 91% correct recognition rate.
2002
We describe an algorithm for the automatic features detection in 2D color images of either frontal or rotated human faces. Such features allow to initialize robustly a bundle-adjustment in order to fit a generic 3D face model to the images. The algorithm first identifies the sub-images containing each feature (eyes, nose and lips), afterwards, it processes them separately to extract fiducial points. The features are looked for in down-sampled images, the fiducial points are identified in the high-resolution ones. The method uses both color and shape information and does not require any manual setting or operator intervention. It has been tested on a database of 130 color images.
Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild
2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018
This paper investigates the evaluation of dense 3D face reconstruction from a single 2D image in the wild. To this end, we organise a competition that provides a new benchmark dataset that contains 2000 2D facial images of 135 subjects as well as their 3D ground truth face scans. In contrast to previous competitions or challenges, the aim of this new benchmark dataset is to evaluate the accuracy of a 3D dense face reconstruction algorithm using real, accurate and high-resolution 3D ground truth face scans. In addition to the dataset, we provide a standard protocol as well as a Python script for the evaluation. Last, we report the results obtained by five state-of-the-art 3D face reconstruction systems on the new benchmark dataset. The competition is organised along with the 2018 13th IEEE Conference on Automatic Face & Gesture Recognition. This is an extended version of the original conference submission with two additional 3D face reconstruction systems evaluated on the benchmark.
Selection and Extraction of Patch Descriptors for 3D Face Recognition
2005
In 3D face recognition systems, 3D facial shape information plays an important role. 3D face recognizers usually depend on point cloud representation of faces where faces are represented as a set of 3D point coordinates. In many of the previous studies, faces are represented holistically and the discriminative contribution of local regions are assumed to be equivalent. In this work, we aim to design a local region-based 3D face representation scheme where the discriminative contribution of local facial regions are taken into account by using a subset selection mechanism. In addition to the subset selection methodology, we have extracted patch descriptors and coded them using Linear Discriminant Analysis (LDA). Our experiments on the 3D_RMA database show that both the proposed floating backward subset selection scheme and the LDA-based coding of region descriptors improve the classification accuracy, and reduce the representation complexity significantly.