CS180 Project 3: Face Morphing

Project Overview

In this project, I worked on face morphing, computing average faces, and creating caricatures. I implemented different image processing techniques to morph one face into another, calculate the mean face from a group of faces, and make exaggerated versions of faces.

Part 1: Defining Correspondences

For the correspondences, I used the tool from a previous student's project (available here). I selected images of Conor McGregor and Lionel Messi from Martin Schoeller's close-up series. I then hand-picked the correspondences, making sure I had enough points for a good warp and a good number of triangles.

Part 2: Computing the "Mid-way Face"

To create the mid-way face between McGregor and Messi, I followed these steps:

1. Computed the average shape by averaging the corresponding points from both images.
2. For each triangle in the original images and the average shape, I calculated an affine transformation matrix.
3. Using these matrices, I performed inverse warping to map pixels from the original images to the average shape.
4. I used polygon masking to ensure only relevant pixels were warped.
5. Finally, I cross-dissolved the two warped images by averaging their pixel values to create the mid-way face.

Part 3: The Morph Sequence

I created a morph sequence by generating 45 frames. For each frame, I linearly interpolated the warp_frac and dissolve_frac from 0 to 1. I then created the morphed image for each frame using these interpolated values. Finally, I saved the frames together as an animated GIF.

Replay

Part 4: The "Mean Face" of a Population

For this part, I used the FEI Face Database. I clipped some points that were going outside of the image range. I then chose one image to warp to the average face (and vice versa), as well as doing it on my own face after choosing corresponding points.

Part 5: Caricatures - Extrapolating from the Mean

I used different alpha values to create caricatures by extrapolating from the mean. The caricature points were calculated using this formula:

caricature_points = α * mean_neutral_points + (1 - α) * adnan_points

Bells & Whistles

I computed the eigenfaces by performing PCA on the dataset. Here are the top 16 principal components:

I then used these eigenfaces along with some of the neutral normal faces to produce caricatures by adding a random variable alpha. Here are the results:

Conclusion

In this project, I successfully implemented face morphing between two images, created a morph sequence, computed mean faces from a dataset, and generated caricatures. I also explored using eigenfaces for creating caricatures. These techniques helped me understand how to manipulate and analyze facial images using computer vision methods.