In this assignment you will produce a "morph" animation of your face into another student's face. Each of you will generate two seconds of animation, which we will record onto video for you. The final video will thus be a metamorphosis through each of the faces in the class.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.
We will take pictures of everyone in the class. Each student will then get a starting image ("picture A") and an ending image ("picture B") for the animation sequence. Picture A will be your face. Picture B will be another student or other primate. Your pictures will be available here. Use your photo as Picture A and the next person's photo (in alphabetical order) as Picture B.
You'll morph still picture A into still picture B and produce 61 frames of animation numbered 0-60, where frame 0 must be identical to picture A and frame 60 must be identical to picture B. In the video, each frame will be displayed for 1/30 of a second. Name your frames morph??.jpg where ?? is 00 to 60.
First, you will need to define pairs of corresponding points on the two images by hand (the more points, the better the morph, generally). The simplest way is probably to use the cpselect tool or write your own little tool using ginput and plot commands (with hold on and hold off ). Now, you need to provide a triangulation of these points that will be used for morphing. You can compute a triangulation any way you like, or even define it by hand. A Delaunay triangulation (see dalaunay and related functions) is a good choice since it does not produce overly skinny triangles that might lead to folding problems and therefore unwanted morphing artifacts. You can compute the Delaunay triangulation on either of the point sets (but not both -- the triangulation has to be the same throughout the morph!). But the best approach would be to compute the triangulation at midway shape (i.e. mean of the two point sets) to lessen the potential triangle deformations, and avoid running into folding problems.
You need to write a function:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac);
that produces a warp between im1 and im2 using point correspondences defined in im1_pts and im2_pts (which are both n-by-2 matrices of (x,y) locations) and the triangulation structure tri . The parameters warp_frac and dissolve_frac control shape warping and cross-dissolve, respectively. In particular, images im1 and im2 are first warped into an intermediate shape configuration controlled by warp_frac, and then cross-dissolved according to dissolve_frac. For interpolation, both parameters lie in the range [0,1]. They are the only parameters that will vary from frame to frame in the animation. For your starting frame, they will both be equal to 0, and for your ending frame, they will both be equal to 1.
Given a new intermediate shape, the main task is implementing an affine warp for each triangle in the triangulation from the original images into this new shape. This will involve computing an affine transformation matrix A between two triangles:
A = computeAffine(tri1_pts,tri2_pts)
A set of these transformation matrices will then need to be used to implement an inverse warp (as discussed in class) of all pixels. Functions tsearch and interp2 can come very handy here. But note that you are not allowed to use Matlab's build-in offerings for computing transformations, (e.g. imtransform, cp2tform, maketform, etc). Note, however, that tsearch assumes that your triangulation is always Delaunay. In our case, this might not always be true -- you may start with a Delaunay triangulation, but through the course of the morph it might produce skinny triangles and stop being Delaunay. This version of tsearch instead works on any triangulation: mytsearch.m and mytsearch.c (compile by typing "mex mytsearch.c" in Matlab; if you get compiler errors try replacing mytsearch.c with this one. If you get lots of type errors, try this one).
Several fun things are possible with our new morpher. For example, we can compute the mean face of the students in the class. This would involve: 1) computing the average shape, 2) warping all faces into that shape, and 3) averaging the colors together. However, this would also require a consistent labeling of all the faces. So, what we will do is ask everyone to label their own face (and maybe a few others of their own choosing), but do it in a consistent manner as shown in the following two images: points, point_labels. For each face image, you produce a text file with coordinates of x,y positions, one per line (43 lines total). Please name the file with the same name as your picture, but with a ".txt" extension. You can read/write such files with the save -ascii and load -ascii commands. If everyone does this in a timely manner, then everyone can use all of the data to compute their mean image.
Show
the mean image that you got, as well as: 1) your face warped onto the
average geometry, and 2) the average face warped onto your face
geometry.
To get you started, these are two test faces and points used during the last year.
NOTE:
The .txt files with the labels must be collected by
November 7 as noted above in order to give everybody the
chance to do their project. This is a hard deadline with no
extensions / delays!!! Please send the .txt files to me via
email. As soon as I collect them, I will copy them
You should deliver the .txt
file via email as noted above by November 7.
You will need to implement a warping
algorithm and turn in a video morph (sequence of images). You will also
need to compute the mean face and show results of warping your face into
the mean face ... (as noted above). You are expected to
do at least 20 points of bells and whistles for full credit.
The project will be graded out of 130
points, with points allocated as noted above and below.
To turn in your assignment, you will need
to deliver a zipped file (Use these
guidelines to deliver your files) containing the
following:
Reaction Report: Reaction report of this paper (project page) (worth 20 points). The report has to be written according to the following guidelines.
Code: Please supply all the code you used to generate your results, along with a README file to allow me to test it out. (Basic code is worth 40 points.)
Web: Create a project page to showcase your results. By now you are an expert and do not need suggestions! (This is worth 50 points.)
