This part of the assignment gives you a chance to experiment with face recognition using appearance-based models and optimal data reduction.
In the directory
/cs/cs153/Images/a5/faces/orl_faces/there are a large number of faces (of subjects s1 to s40). For the purposes of this problem, consider the images 1.pgm to 8.pgm in each subject's subdirectory as models to be used for building up a representation of a face. Images in pgm format are not by default readable by matlab, but you can use the function pgmread in /cs/cs153/matlab/visioncode/pgmread.m . That function is called in matlab as
A = pgmread('/cs/cs153/Images/a5/faces/orl_faces/s19/1.pgm');
The images 9.pgm and 10.pgm should
be saved (they will be the unmodeled images for testing the
recognition algorithm).
There are thus forty sets of eight faces available for modeling, but (at least to start) I would advise choosing some subset of those forty (say, fifteen) to try out this recognition technique. Once all of the matlab (or C) code is in place, updating your system to handle all forty subjects will not be a problem.
Consider each of these images simply as vectors of pixels. First, determine the "average" face image, subtract it from each model, and then create a covariance matrix with the resulting vectors. From this matrix find the most important K eigenvectors (these will be "eigenfaces") -- the matlab built-in function svd will do this for you.
Create these "eigenfaces" by normalizing the values of their pixels to be bewteen 0 and 255, and show your results below. (The number of eigenfaces you use is up to you -- you may want to create a relatively large number (25+) and then use subsets of this initial set.)
Choose one face in the database of modeled faces and decompose it into a set of coefficients indicating how much of each eigenface it contains. (Don't forget to remove the average face image first.) Using these coefficients and the eigenfaces themselves, show the reduced-data model of the face you chose vs. the original image:
For each image in the original database, create the vector of coefficients that indicates that image's location in "eigenspace." Then, create the same vector of coefficients for the unmodeled images (numbers 9 and 10). Using a nearest-neighbor criterion, determine whether or not the unmodeled images match the same subject (i.e., the face has been recognized correctly). In your results section, below, show each unmodeled face and the person from the modeled database that looked most similar.
There are two natural extensions for this part of the assignment. One is to use all forty subjects and see if that additional information helps, hurts, or doesn't affect recognition performance. The second is to use a neural net to do the classification instead of a nearest-neighbor technique -- that is, after the images are transformed into vectors of eigenface coefficients.