In many applications, an object of interest can be characterized by a particular range of intensities or by a particular set of colors. A first step in interpreting images of such objects is determining exactly which pixels belong to them. This part of the assignment asks you to explore the setting and testing of thresholds in order to segment objects of interest from images.
As an introduction to thresholding, write a function threshold.m that takes as input a (grayscale) image and a threshold, t, and returns a new image that is a thresholded version of the input image. That is, the output image should consist only of pixels with intensity 0 or 255: 0 where the original image was less than the threshold, 255 where it was greater. (Actually any equivalent format for your output is all right -- if its other than black on white or vice versa, please include a note to explain ... .) Grayscale images for testing are available in /cs/cs153/Images/a2/grayscale.
A histogram of intensity values is helpful for summarizing the pixels that make up a grayscale image. A histogram is simply a plot of the frequency of different pixel intensities (from an image) plotted against those intensities. Matlab has several built-in functions for creating and drawing histograms:
You may want to write a function along the lines of L = flatten(A) that converts an image to a vector of pixels, in order to help get at lists of intensities to histogram. Also, the "SUB" button in the imframe display allows you to choose a rectangular subimage and sets two global variables: imf_subimage and imf_pixellist. The first is the rectangular image (note that it will have three colorbands, even if the original image was grayscale -- you can ignore the latter two); the second is a list of pixels (again RGB) in that subimage. You can use the globals in your workspace if you type
>> global imf_subimage imf_pixellistat some point.
Ideally, no human intervention would be needed to choose a threshold for an image. This section asks you to implement an automatic thresholder for grayscale images. You may choose to create one of your own design, or to implement the one described from the handout ("an iterative (optimal) threshold selection" from the book by Sonka, Hlavac, and Boyle). Create your threshold selector as a matlab function t = tChoose(A), where A is the grayscale image to be thresholded and t is the threshold output by the function.
If you decide to write your own algorithm for automatic thresholding, you may want to consider smoothing the histogram of A. One way to do this is by replacing each bin's value (# of pixels) with an average of the values of a neighborhood of bins.
Try out your automatic thresholder by making it choose the threshold in the script written for the previous section. Test it against the images in /cs/cs153/Images/a2/grayscale. For your results, include links to your code from the previous sections, as well as an example in which your automatic thresholder succeeded and one in which it failed. What about an image (or its histogram) makes your automatic thresholder likely to do well or poorly in segmenting an image?
A natural cue to use in segmenting objects from their surroundings in images is color. This section contrasts segmenting color regions using red, green, and blue thresholds (aligned with the RGB axes of color space) with segmentation using hue, saturation, and intensity bounds (aligned with the coordinate system of HSI or HSV space).
First, try segmenting the image /cs/cs153/Images/a2/color/stopcup.tif into foreground objects and the dark background using a threshold on the intensity of the pixels. You can get a grayscale image from an RGB image simply by averaging the three color components of each pixel. Demonstrate your segmentation by replacing the black pixels with visually distinct blue ones. (In fact, just the reverse -- replacing blue pixels with those of some preset image -- is the technique used in TV or movies to superimpose objects against some preset background. Since thresholding is used, this (and not fashion) is why so few weathercasters wear saturated blue items). blue Second, use thresholds in each colorband (a particular one or two or all three) to isolate the red portion of the stop sign, the green cup, and the white lettering/signpost from the image. Again, display the results by "bluing out" the intended region. (You might try your automatic thresholder or simply choose thresholds by hand.)
Include the results after the following section (which involves the same task in HSI space).
Repeat the segmentation of the four regions (background, red sign, white lettering, and green cup) using thresholds in HSI space. Matlab has a function for converting from RGB to HSI (matlab calls it HSV):
B = rgb2hsv(A);Note that when described in matlab's HSV space, the hue (first component) of a pixel ranges from 0.0 (red) to 1.0 (red again), passing through orange, yellow, green, cyan, blue, purple, and magenta along the way. The second component, saturation, varies from 0.0 (grayscale) to 1.0 (completely saturated -- no white at all). The final component, intensity (or "value"), also ranges from 0.0 (no intensity) to 1.0 (max intensity).
There is also an inverse function
A = hsv2rgb(B);It returns an rgb image with pixel components between 0.0 and 1.0 -- You need to scale this up to the usual 0-255 range (and convert to uint8) to display the image.
Note that a "threshold" of the hue component of pixels must be an interval, because the hue actually wraps around and is best envisioned as a circle. Thus, to segment a blue region, you need to accept only hues around 2/3 (0 = red, 1/3 = green, 2/3 = blue).
Lastly, test how well your thresholds generalize by trying the same ones (RGB and HSI) that you used to segment the red sign in the stopcup image against the images stop1.tif, stop2.tif, stop3.tif, and stop4.tif. They are in the same directory as stopcup.
For the results of this section, show your four segmentations of stopcup for each colorspace, RGB and HSI. Are there any qualitative differences between them? You need not include all four stop#.tif images, but choose an example or two that indicate how well RGB and HSI-aligned segmentations generalize to different lighting conditions, environments, etc.
Remember that you need to pursue only one extension in one part of the assignment, and it can be anything you'd like to investigate. These are only suggestions.