CS 674 Computational Photography

In this assignment you will produce a "morph" animation of your face into another student's face. Each of you will generate two seconds of animation, which we will record onto video for you. The final video will thus be a metamorphosis through each of the faces in the class.

A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.

We will take pictures of everyone in the class. Each student will then get a starting image ("picture A") and an ending image ("picture B") for the animation sequence. Picture A will be your face. Picture B will be another student or other primate. Your pictures will be available here. Use your photo as Picture A and the next person's photo (in alphabetical order) as Picture B.

You'll
morph still picture A into still picture B and produce 61 frames of
animation numbered 0-60, where frame 0 must be identical to picture A and
frame 60 must be identical to picture B. In the video, each frame will be
displayed for 1/30 of a second. Name your frames `morph??.jpg` where
?? is 00 to 60.

First, you will need to define pairs of corresponding points on the two
images by hand (the more points, the better the morph, generally). The
simplest way is probably to use the `cpselect` tool
or write your own little tool using `ginput` and `plot` commands
(with `hold on` and `hold
off` ). Now, you need
to provide a triangulation of these points that will be used for
morphing. You can compute a triangulation any way you like, or even
define it by hand. A Delaunay triangulation (see `dalaunay` and
related functions) is a good choice since it does not produce overly
skinny triangles that might lead to folding problems and therefore
unwanted morphing artifacts. You can compute the Delaunay triangulation
on either of the point sets (but not both -- the triangulation has to be
the same throughout the morph!). But __the best approach would be to
compute the triangulation at midway shape__ (i.e. mean of the two
point sets) to lessen the potential triangle deformations, and avoid
running into folding problems.

You need to write a function:

morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac);

that
produces a warp between `im1` and `im2` using
point correspondences defined in `im1_pts` and `im2_pts` (which
are both n-by-2 matrices of (x,y) locations) and the triangulation
structure `tri` .
The parameters `warp_frac` and `dissolve_frac` control
shape warping and cross-dissolve, respectively. In particular, images `im1` and `im2` are
first warped into an intermediate shape configuration controlled by `warp_frac`,
and then cross-dissolved according to `dissolve_frac`.
For interpolation, both parameters lie in the range [0,1]. They are the
only parameters that will vary from frame to frame in the animation. For
your starting frame, they will both be equal to 0, and for your ending
frame, they will both be equal to 1.

Given a new intermediate shape, the main task is implementing an affine warp for each triangle in the triangulation from the original images into this new shape. This will involve computing an affine transformation matrix A between two triangles:

A = computeAffine(tri1_pts,tri2_pts)

A
set of these transformation matrices will then need to be used to
implement an inverse warp (as discussed in class) of all pixels.
Functions `tsearch` and `interp2` can
come very handy here. But note that you are not allowed to use Matlab's
build-in offerings for computing transformations, (e.g. `imtransform`, `cp2tform`, `maketform`,
etc). Note, however, that `tsearch` assumes
that your triangulation is always Delaunay. In our case, this might not
always be true -- you may start with a Delaunay triangulation, but
through the course of the morph it might produce skinny triangles and
stop being Delaunay. This
version of `tsearch`
instead works on any triangulation: mytsearch.m and
mytsearch.c (compile
by typing "`mex mytsearch.c`" in Matlab; if you get compiler
errors try replacing mytsearch.c with this
one. If you get lots of type errors, try this
one).

Several
fun things are possible with our new morpher. For example, we can
compute the mean face of the students in the class. This would involve:
1) computing the average shape, 2) warping all faces into that shape,
and 3) averaging the colors together. However, this would also require a
__consistent__ labeling
of all the faces. So, what we will do is ask everyone to label their own
face (and maybe a few others of their own choosing), but do it in a
consistent manner as shown in the following two images:
points, point_labels.
For each face image, you produce a text file with coordinates of x,y
positions, one per line (43 lines total). Please name the file with the
same name as your picture, but with a ".txt" extension. You can
read/write such files with the `save
-ascii` and `load
-ascii` commands. If
everyone does this in a timely manner, then everyone can use all of the
data to compute their mean image.

Show
the mean image that you got, as well as: 1) your face warped onto the
average geometry, and 2) the average face warped onto your face
geometry.

To get you started, these are two test faces and points used during the last year.

**NOTE:**
The .txt files with the labels **must** be collected **by
November 7** as noted above in order to give everybody the
chance to do their project. **This is a hard deadline with no
extensions / delays!!**! Please send the .txt files to me via
email. As soon as I collect them, I will copy them

- (up
to
**20 points**) Use the mean face to produce caricatures (extrapolations), masculitizing and feminizing transforms and any other fun things you can think of. - (up
to
**20 points**) Use the class data to compute a PCA basis for the face space. Try performing caricatures and other transformations in the new basis. Compare your results to doing this in normal basis. Are the results better? - (up
to
**20 points**) Implement a non-triangle based morphing algorithm, or use an automatic form of face alignment where face feature correspondences are found automatically. There are a lot out there, meaning this Bell/Whistle is somewhat undefined. Points will be based both on how good your new scheme looks and how hard it was to actually code up. (But mostly the former.)

**You should deliver the .txt
file via email as noted above by November 7.**

You will need to implement a warping
algorithm and turn in a video morph (sequence of images). You will also
need to compute the mean face and show results of warping your face into
the mean face ... (as noted above). You are **expected** to
do **at least 20 points of bells and whistles for full credit.**

The project will be graded out of 130
points, with points allocated as noted above and below.

To turn in your assignment, you will need
to deliver a zipped file (**Use these
guidelines to deliver your files) **containing the
following:

**Reaction
Report:** Reaction report of this
paper (project
page) (worth **20
points**).
The report has to be written according to the following guidelines.

**Code:** Please
supply all the code you used to generate your results, along with a `README` file
to allow me to test it out. (Basic code is worth
**40 points**.)

**Web:** Create
a project page to showcase your results. By now you are an expert and do
not need suggestions! (This is worth **50 points**.)