Assignment #1: Hybrid Images
CS 674 Computational Photography

(Look at image on right from very close, then from very far away.)

Due Date: 2:00pm on Tuesday, Sept. 19, 2017

Overview

The goal of this assignment is to create hybrid images using the approach described in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns. Hybrid images are static images that change in interpretation as a function of the viewing distance. The basic idea is that high frequency tends to dominate perception when it is available, but, at a distance, only the low frequency (smooth) part of the signal can be seen. By blending the high frequency portion of one image with the low-frequency portion of another, you get a hybrid image that leads to different interpretations at different distances.

Details

This project is intended to familiarize you with image filtering and frequency representations. Here, you will find two sample images and some starter code that can be used to load two images and align them. The alignment is important because it affects the perceptual grouping (read the paper for details).

First, you'll need to get a few pairs of images that you want to make into hybrid images. You can use the sample images for debugging, but you should use your own images in your results. Then, you will need to write code to low-pass filter one image, high-pass filter the second image, and add (or average) the two images. For a low-pass filter, Oliva et al. suggest using a standard 2D Gaussian filter. For a high-pass filter, they suggest using the impulse filter minus the Gaussian filter (which can be computed by subtracting the Gaussian-filtered image from the original). The cutoff-frequency of each filter should be chosen with some experimentation.

For your favorite result, you should also illustrate the process through frequency analysis. Show the log magnitude of the Fourier transform of the two input images, the filtered images, and the hybrid image. In Matlab, you can compute and display the 2D Fourier transform with: imagesc(log(abs(fftshift(fft2(gray_image)))))

Try creating a variety of types of hybrid images (change of expression, morph between different objects, change over time, etc.). This site has several examples that may inspire you.

Bells & Whistles (Extra Points)

Try using color to enhance the effect. Does it work better to use color for the high-frequency component, the low-frequency component, or both? (5 pts)

Illustrate the process by implementing Gaussian and Laplacian pyramids and displaying them for your favorite result. This should look similar to Figure 7 in the Oliva et al. paper. (15 pts)

Deliverables

Use these guidelines to deliver your files.

To turn in your assignment, you will need to provide me with a zipped file containing the following:

Source code with a README file explaining how to run the code, and what parameters to use.
Reaction report of the original SIGGRAPH 2006 paper. The report has to be written according to the following guidelines.

The zipped archive should also contain a webpage and any supporting media describing your project. This does not need to be a comprehensive report. Just include your results and any parameters or special things that you tried. The page should be sufficiently clear so that the professor and other students can easily understand what your results mean and how you got them. You don't need to include a lengthy introduction or detailed description. As an example, you can take a look at the webpage describing how this assignment was executed last year by one of your fellow students. In particular, please:

Show us your favorite result. Include: 1) the original and filtered input images; 2) the hybrid image; and 3) the FFT images. Briefly (a few sentences) explain how it works, using the included images as illustrations. Explain any clever ideas that you've incorporated and any parameters. This should be with your own images (not the included samples).
Next, show us at least two more results, including one that doesn't work so well (failure example). Briefly explain how you got the good results (e.g., chosen cut-off frequencies, alignment tricks, other techniques), as well as any difficulties and the possible reasons for the bad results. If you are so fortunate that everything that you try works well, try to figure out what shouldn't work.
Describe any bells and whistles under a separate heading.

Scoring

The core assignment is worth 130 points, as follows:

50 points for implementation to create hybrid images from two aligned input images.
50 points for the project html page description: 20 points for FFT images; 20 points for including at least two examples beyond the first (including at least one failure); 10 points for clarity.
30 points for the reading assignments.

You can also earn up to 20 extra points for the bells & whistles mentioned above (5 for experimenting with color; 15 for Gaussian/Laplacian pyramids) or suggest your own extensions (check with prof first).

Acknowledgements

This assignment is derived from Derek Hoiem, and used here with permission.

Assignment #1: Hybrid Images CS 674 Computational Photography