Assignment #2: Seam Carving
CS 674 Computational Photography

Overview

Details

For a brief overview of the algorithm and some inspiring results, check out the video. Your algorithm will shrink images as follows:

1. Determine the 'importance' each pixel has using an energy function.
2. Until image has shrunk to the desired dimension:
   1. Find the lowest-importance seam in the image.
   2. Remove it.

Feel free to use any energy function in the paper (or cook up something on your own). Defining a good energy function seems to be the key to achieving good results.

Seam examples. Black: horizontal seam. Gray:vertical seam. Green: not a seam -- contains more than one pixel from some columns.

A horizontal seam is a connected path from one side of the image to the other that chooses exactly one pixel from each column. (A vertical seam is the same, but from top to bottom and with rows.) Finding the lowest-importance seam is a simple dynamic programming exercise.

Your final code should be able to, given an input image, produce an output image shrunk by a given number of pixels either in width or in height (but not both at once!). A simple way to achieve this goal is to write your code to shrink only in one dimension and then create a wrapper function that transposes the image to shrink in the other dimension.

Example: Horizontal Carving

Example: Vertical Carving

Notice how the content in the image is preserved, and the large, boring, area of asphalt is removed. (This does ruin the composition, but such is life.)

Example: Carving Artifacts

As you push the algorithm to its limits, artifacts -- such as shape distortions and odd cubist angles -- will begin to crop up. A lot of time on this project will be spent tweaking your code to mitigate these effects, as well as searching for pictures which don't exhibit them. Generally, highly-structured scenes and human faces will suffer, while scenes with textured objects on simple backgrounds will work better.

Bells & Whistles (Extra Points)

Try some special moves to up your multiplier, chain a longer combo, and increase your score:

• (5 points) Make it fast. Divide your function into a precompute section and an on-line section. The on-line section should be able to produce a resized version of an image very quickly (like in the Seam Carving video).
• (10 points) It seems that some energy functions may be better than others for certain scenes. Implement two energy functions which produce good results on different kinds of images. Propose and test a heuristic to select the correct energy function for an input image.
• (10 points) Add "optimal" diagonal resizing as described in the paper -- that is, find the optimal ordering of horizontal and vertical seams to simultaneously achieve a given width and height.
• (10 points) Stretch images using seam insertion (expansion).
• (10 points) Remove objects using seam carving and seam insertion.
• (5-15 points) Compose and photograph an engaging scene. Seam carve your masterpiece to a smaller size. Use The GIMP or similar along with your enormous (compared to matlab, at least) human brain to do better. Don't limit yourself to simple cropping or scaling -- use cloning, reposition objects, and generally go all-out.

Deliverables

The project will be graded out of 130 points, with points allocated as noted below and above.

To turn in your assignment, you will need to deliver a zipped file (Use these guidelines to deliver your files) containing the following:

Reaction Report: Reaction report of the original SIGGRAPH 2007 paper (worth 20 points). The report has to be written according to the following guidelines.

Code: Please supply all the code you used to generate your results, along with a README file to allow us to test it out. (Basic dynamic programming seam horizontal and vertical carving is worth 50 points. On top of this base score, you will earn points for your report [see Web below].)

Web: Create a project page to showcase your results, and put it in a sub-folder. By now you know the drill: simple html, nothing too complicated. Please:

• (30 points) Include lots (six or more) great examples of resized images (either from yourself or the internet, but remember to attribute!). Show us your implementation works.
• (N points) Dazzle us with any bells and/or whistles you've added in.
• (20 points) Once we're riding high on your success, calm us down with some examples of where your code doesn't work.
• Make sure to tell your story with weighty words as well as potent pictures.

NOTE 1: Your example images don't need to be larger than the above examples (i.e. around 500x500 pixels).

NOTE 2: Students have to add some bells and/or whistles in order to achieve or surpass 130 points.