Site News  |   Monitor Keywords  |   Monitor Archive  |   Organizer  |   Account Info  |

3d face reconstruction from 2d images

This application claims priority to U.S. patent application Ser. No. 11/669,099, filed on Jan. 30, 2007, which claims priority to U.S. Provisional Application 60/764,007, filed Jan. 31, 2006. The disclosure of the prior application is considered part of (and is incorporated by reference in) the disclosure of this application.

The U.S. Government may have certain rights in this invention pursuant to Grant No. HMI-582-04-1-2002.

Conventional face reconstruction techniques often use a two dimensional image or images (e.g. digital photographs) of a face to create a three dimensional representation of the face. The representation that is created may be a file, such as an electronic file, indicative of individual characteristics of different faces. The file can then be used, e.g., for facial recognition, animation, or rendering.

The images, once obtained, are often processed based on prior knowledge or assumptions of what faces usually look like. This knowledge is often called “domain knowledge”, a “prior model”, or more specifically a “generic face”. For example, the prior face knowledge may indicate the presence or likely locations of different kinds of facial features, such as eyes, nose, etc. The prior face knowledge may assume that the face is formed of a linear combination of basis face shapes and appearances, camera parameters, lighting parameters, and other known elements, or elements that are susceptible of estimation. These elements can be combined to estimate the likely appearance of a face. More specifically, the domain knowledge may come in the form of a generic face shape defined by an artist or an average face shape computed from a plurality of known face shapes.

One common technique for face reconstruction uses prior face knowledge of a generic face, and possibly a set of face metrics or deformation parameters, throughout the reconstruction process. Another common technique attempts to eschew the use of prior face knowledge and instead uses a purely data-driven approach to reconstruct the face. This can be done, for example, using triangulation of two-dimensional points in multiple images from multiple calibrated cameras. Unfortunately, the former approach may provide unrealistic data, due to the use of the generic face throughout the process. The latter approach requires additional hardware infrastructure which is difficult to practically implement at a reasonable cost. A single-camera purely data-driven approach alleviates some of the hardware constraints of multi-view stereo methods, but may itself be unstable due to the lack of constraints at stages of the process.

The present application describes techniques for obtaining three-dimensional face information using an assisted technique. According to aspects, prior knowledge of face structure is used at some points during the processing operation, but other parts during the processing operation are purely data driven.

Another operation uses a single camera for determination of 3D information from a set of 2D images.