Three-dimensional facial model reconstruction and plastic surgery simulation (original) (raw)

Simulating facial surgery using finite element models

Proceedings of the 23rd annual conference on Computer graphics and interactive techniques - SIGGRAPH '96, 1996

This paper describes a prototype system for surgical planning and prediction of human facial shape after craniofacial and maxillofacial surgery for patients with facial deformities. For this purpose it combines, unifies, and extends various methods from geometric modeling, finite element analysis, and image processing to render highly realistic 3D images of the post surgical situation. The basic concept of the system is to join advanced geometric modeling and animation systems such as Alias with a special purpose finite element model of the human face developed under AVS. In contrast to existing facial models we acquire facial surface and soft tissue data both from photogrammetric and CT scans of the individual. After initial data preprocessing, reconstruction, and registration, a finite element model of the facial surface and soft tissue is provided which is based on triangular finite elements. Stiffness parameters of the soft tissue are computed using segmentations of the underlying CT data. All interactive procedures such as bone and soft tissue repositioning are performed under the guidance of the modeling system which feeds the processed geometry into the FEM solver. The resulting shape is generated from minimizing the global energy of the surface under the presence of external forces. Photorealistic pictures are obtained from rendering the facial surface with the advanced animation system on which this prototype is built. Although we do not claim any of the presented algorithms themselves to be new, the synthesis of several methods offers a new facial model quality. Our concept is a significant extension to existing ones and, due to its versatility, can be employed in different applications such as facial animation, facial reconstruction, or the simulation of aging. We illustrate features of our system with some examples from the Visible Human Data Set.

Facial modeling for plastic surgery using magnetic resonance imagery and 3D surface data

SPIE Electronic …, 2004

We describe a novel facial modeling and real-time pre-surgery planning + visualization tool for surgical and esthetic plastic surgery. The modeling pipeline accepts geometry both in the form of Magnetic Resonance Imagery (MRI) and/or 3D facial surface scans. The MRI data is first used to segment the skin surface as well as the underlying tissues of interest. Textures taken with a digital camera are mapped on the 3D model for display purposes. The facial scanner, when available, provides an alternate means for doctors to obtain high quality 3D skin information and texture at the same time. Multiple facial expressions can be recorded to study tissue deformation. Our solution allows for using the two types of sensors separately or in conjunction with one another. The resulting models and additional information, such as the animated MRI slides, provide doctors with better means of surgical planning. Currently, we are further developing the system to integrate finite element calculations (FEM) that help better planning and understanding of the possible outcomes of a surgery and thereby reduce the risk for any given patient. The presentation reviews the methodologies, algorithms and tools in the context of a real-life application.

Head Modeling from Pictures and Morphing in 3D with Image Metamorphosis based on triangulation

1998

This paper describes a combined method of facial reconstruction and morphing between two heads, showing the extensive usage of feature points detected from pictures. We first present an efficient method to generate a 3D head for animation from picture data and then a simple method to do 3Dshape interpolation and 2D morphing based on triangulation. The basic idea is to generate an individualized head modified from a generic model using orthogonal picture input, then process automatic texture mapping with texture image generation by combining orthogonal pictures and coordinate generation by projection from a resulted head in front, right and left views, which results a nice triangulation on texture image. Then an intermediate shape can be obtained from interpolation between two different persons. The morphing between 2D images is processed by generating an intermediate image and new texture coordinate. Texture coordinates are interpolated linearly, and the texture image is created using Barycentric coordinates for each pixel in each triangle given from a 3D head. Various experiments, with different ratio between shape, images and various expressions, are illustrated.

A New 3-D Tool for Planning Plastic Surgery

IEEE Transactions on Biomedical Engineering, 2000

Face plastic surgery (PS) plays a major role in today medicine. Both for reconstructive and cosmetic surgery, achieving harmony of facial features is an important, if not the major goal. Several systems have been proposed for presenting to patient and surgeon possible outcomes of the surgical procedure. In this paper, we present a new 3D system able to automatically suggest, for selected facial features as nose, chin, etc, shapes that aesthetically match the patient's face. The basic idea is suggesting shape changes aimed to approach similar but more harmonious faces. To this goal, our system compares the 3D scan of the patient with a database of scans of harmonious faces, excluding the feature to be corrected. Then, the corresponding features of the k most similar harmonious faces, as well as their average, are suitably pasted onto the patient's face, producing k+1 aesthetically effective surgery simulations. The system has been fully implemented and tested. To demonstrate the system, a 3D database of harmonious faces has been collected and a number of PS treatments have been simulated. The ratings of the outcomes of the simulations, provided by panels of human judges, show that the system and the underlying idea are effective.

Morphing Technique for Creating Composites of Face Images

International Journal of Engineering Sciences & Research Technology, 2014

Face morphing is a technique that makes a transition from one face image to another face image. It has been extensively used in multiple fields of work, such as animation, movie production, games, and mobile applications. There are two types of methods used to conduct face morphing. First is semiautomatic mapping methods, in which users map corresponding pixels between two face images, and produce a transition of result images. Mapping the corresponding pixel between two facial images is usually not trivial. And second morphing method is fully automatic method. It is used for morphing between two images having similar face properties, where the results depend on the similarity of the input face images. This paper describes a method for creating composites with only parts of the face being morphed of facial images. Applications of this technique show how to change only local aspects of a facial image. The experimental results show that position of each face component plays a more important role than the edge and color of the face.

Bone model morphing for enhanced surgical visualization

Biomedical Imaging: Nano …, 2004

We propose a novel method for reconstructing a complete 3D model of a given anatomy from minimal information. This reconstruction provides an appropriate intra-operative 3D visualization without the need for a pre or intra-operative imaging. Our method £ts a statistical deformable model to sparse 3D data consisting of digitized landmarks and bone surface points. The method also allows the incorporation of non-spatial data such as patient height and weight. The statistical model is constructed using Principal Component Analysis (PCA) from a set of training objects. Our morphing method then computes a Mahalanobis distance weighted least square £t of the model by solving a linear equation system. First experimental promising results with model generated from 14 femoral head are presented.

Part-Based 3D Face Morphable Model with Anthropometric Local Control

2020

We propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute editing workflow. Current face modeling methods using 3DMM suffer from a lack of local control. We thus create a 3DMM by combining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions. Our local PCA-based approach uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive, while allowing accurate reconstruction. The editing controls we provide to the user are intuitive as they are extracted from anthropometric measurements found in the literature. Out of a large set of possible anthropometric measurements, we filter those that have meaningful generative power given the face data set. We bind the measurements to the part-based 3DMM through mapping matrices derived from our data set of facial scans. Our part-based 3DMM is compact, yet accurate, and compared to other 3DMM methods, ...

Designing a Virtual Reality Model for Aesthetic Surgery

Plastic and Reconstructive Surgery, 2005

Background: Aesthetic surgery deals in large part with the manipulation of softtissue structures that are not amenable to visualization by standard technologies. As a result, accurate three-dimensional depictions of relevant surgical anatomy have yet to be developed. This study presents a method for the creation of detailed virtual reality models of anatomy relevant to aesthetic surgery. Methods: Two-dimensional histologic sections of a cadaver from the National Library of Medicine's Visible Human Project were imported into Alias's Maya, a computer modeling and animation software package. These two-dimensional data were then "stacked" as a series of vertical planes. Relevant anatomy was outlined in cross-section on each two-dimensional section, and the resulting outlines were used to generate three-dimensional representations of the structures in Maya. Results: A detailed and accurate threedimensional model of the soft tissues germane to aesthetic surgery was created. This model is optimized for use in surgical animation and can be modified for use in surgical simulators currently being developed. Conclusions: A model of facial anatomy viewable from any angle in three-dimensional space was developed. The model has applications in medical education and, with future work, could play a role in surgi

A physically based model to simulate maxillo-facial surgery from 3D CT images

Future Generation Computer Systems, 1999

Computer-based surgery simulation represents a rapidly emerging and increasingly important area of research that combines a number of disciplines for the common purpose of improving health care. Generally, the goal of computer-based surgery simulation is to enable a surgeon to experiment with different surgical procedures in an artificial environment. This paper describes an approach for elastic modelling of human tissue based on the use of embedded boundary condition techniques. Embedded boundary condition models allow to simulate the cranio-facial surgery directly on the grid of the 3D CT image of the patient. Previously simulated operations have been performed using surface models or by using a low detailed model of the tissue volume. The approach proposed here involves complete 3D modelling of the solid highly detailed structure of the object, starting from the information present in the 3D diagnostic images. Due to the huge amount of data and the computational complexity of the problem, a parallel version of the software has been implemented on the supercomputer CRAY T3E. The application of this approach for modelling the elastic deformation of human tissue in response to movement of bones is demonstrated both on the visible human data set of the National Library of Medicine and on the CT data set of real patients. 0167-739X/99/$ -see front matter c 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 -7 3 9 X ( 9 8 ) 0 0 0 6 5 -X