Harvey Mudd College
Computer Science 153
Assignment 1
Due Friday, September 12, by 4:30pm
"If only Minsky had been at Mudd..."
Goals
This assignment simply seeks to familiarize you with the two
environments in which you may choose to work: OpenCV (a C++ and C based
library that is the most commonly used for real-time vision
applications) and Matlab (everyone's -- well, at least the computer
vision community's -- favorite mathematical and prototyping tool).
Just to keep it interesting, however, the context for getting
into these systems will be to solve a crucial piece of the vision problem,
, i.e.,
to take in arbitrary images and indicate where the spam
is located in that image. Don't worry -- your program will not have to work perfectly
to receive full credit -- but it will have to work at least sometimes on images
that it has not "seen before."
The assignment
is to load and manipulate images using both Matlab and the OpenCV library, i.e.,
C / C++ code. You may choose either environment in which to implement your spam-finding algorithm.
Also, there is a short write-up required on a web or wiki page, as well.
These next paragraphs contain the details on what to write-up.
The systems
Part of this project will be done using Matlab,
which is the language of choice for the vast majority
of computer vision researchers.
Head to this Matlab link for a quick
review of matlab and introducton to its image-processing support,
which is quite substantial.
You can get started with OpenCV by going to the guide at
this
HMC OpenCV wiki page.
That page describes a particular problem, Set, which you don't need to solve.
On the way, however, it provides links to software, references, and a description as to what's going on.
The challenge
- First, go through one of the links above to the point
at which you can load, display, and modify images.
For OpenCV, that is the "Segmentation" heading. For Matlab,
that is entirety of the small tutorial linked above.
- Next, choose any of the spam images available on the CS
machines (knuth) at /cs/cs153/Images/spamImages2008.
Write small programs that use that image in the following ways:
-
Invert all of the pixels in (R,G,B) space, i.e., change the value of
each of those three channels for each pixel in the image from x
to 255-x. Save the resuling images for inclusion in your write-up.
You should do this in both Matlab and OpenCV.
-
In Matlab, then, create three images: a hue image, a saturation image, and a value image
from your original. Each of the images should be greyscale and should
represent the hue, saturation, and value quantity of each pixel as an
intensity from 0 (min) to 255 (max). There are different choices of
what the minimum hue is -- any choice is OK, as long as you explain it
in your web/wiki page. Save these three images for your write-up.
In OpenCV, create and display a single "hsv" image of the original. That is, convert
the original to HSV representation. Then, display the image in the customary
way, i.e., interpreting the pixels as R, G, and B. You will get a "false-color"
image because of the (intentional) misinterpretation of the data. Save this
image for your write-up, as well.
-
Finding spam Last, but not least, choose
either Matlab OR OpenCV and write a script/program
that identifies the spam in an image. To show this, your program should
draw a plus at the center of the word "SPAM" --
in addition, your program should print out the location
of that point to the console.
You only need to implement spam-finding in one of the two systems:
OpenCV or Matlab.
For this problem, you may
assume the image contains a can of spam, in a roughly frontal view.
Gather at least two images in which your algorithm worked, and two
in which it did not work. If there aren't two on which your
algorithm did not work, grab a couple more from the web!
Also, show the results on two images (of your choice) where there
was no spam to be found... .
One note: your program should not simply guess where
the spam is! It should run a deterministic algorithm that uses the
values of the input image's pixels to locate the spam.
Deliverables
- Demo You should demo your system to me at some point
before Friday at 4:30 pm. Office hours are a good time; Fridays I'm largely
free and hold office hours in Beckman B102 from 2-4:30, as well.
- Write-up
By Sunday night (at 11:59 pm), you should create a web or wiki page with at least the nine images
mentioned in the description above:
- Your chosen spam image
- That image, inverted
- That image's hue , depicted in greyscale (from Matlab)
- That image's saturation , depicted in greyscale (from Matlab)
- That image's value, depicted in greyscale (from Matlab)
- The false-color HSV image (from OpenCV)
- Two or more images in which spam was found successfully
- Two or more images in which spam was not found successfully, but it was there.
- Two or more images in which there was no spam (and their results).
Although you'll create and demo these images via both Matlab and OpenCV,
simply choose your favorites to display on your write-up site.
You're welcome to include additional images, as well -- I encourage sharing any ones
that show interesting or unusual outputs... .
In addition, include a write-up of 2-3 paragraphs explaining your
spam-finding algorithm. You should explain the conditions under which
your algorithm will work (in general), as well as why it failied on the images it did.
Also, include a short paragraph that suggests how your spam-finder could be
improved.
Link your write-up to the
CS 153 wiki page - let me know if you run into permission or other problems.
You don't need to link your source code on your web/wiki page -- rather,
I'll ask you to copy it to a class archive when you demo your system. I will try to make
sure everyone has a Wiki account on Wednesday during lab.
If you extended your spam-finding in any way -- see the
suggestions below for possibilities -- describe those extensions, as well. Include
additional images or screenshots, as appropriate to convey your ideas.
Possible extentions
This week additional extensions are not
required, but can still be completed for additional credit on this assignment.
Almost any idea is likely to be OK, including those that show you've delved into the systems, e.g.,
- Optionally showing a visualization of some of the intermediate processing that
led to your algorithm's final result.
- Creating and explaing a method that detects spam, alongside
the location mechanism described above. That is, your system will
first determine if there is a can of spam in the image, and, only
if it decides there is one will it mark its center.
- Determining the bounding box for the can of spam: drawing a box, rather than
a plus sign at the center. You may decide how much of the spam you want to
enclose in the box -- but you need to decide that before declaring success!
- Other ideas welcome :-)