Face Recognition Using Face Unit Radial Basis Function Networks

Introduction and Original Proposal

This page is home to my final project for my CS152 - Neural Networks course, taught at Harvey Mudd College by Prof. Robert Keller. For the final project in the class, we were given the chance to do a self-directed project in the field of neural networks and genetic algorithms. I began by researching the area of radial basis function (RBF) networks. My eventual goal was to duplicate some of the results Dr. A. J. Howell got by adapting this class of neural net to the problem of face-recognition.

Examples

The generalized RBF network structure.

MPEG Video (667Kb)

GIF still image (30Kb)

Dr. Howell's RBF network demonstrating classification from an unconstrained image sequence.

Images and video above taken from Dr. Howell's web site.

Problem Statement

Using the face unit RBF network architecture (Howell 1995), train a collection of face unit networks to recognize individuals from a set a sample face images.
A face unit network is structured to recognize only one person, using a hybrid RBF architecture.
Each face unit network has two linear outputs, one indicating a positive identification of the person, the other a negative identification.
A p+a face unit network has p radial neurons linked to the + output, and a neurons linked to the - output.
Positive or negative classification based on which of the two output values are greater.
Trained various sizes of face unit network to recognize each individual. Networks will be sized from 1+1 up to 6+12.

Approach

Sample data
- 10 people x 10 poses of each person ranging from 0° (head-on) to 90° (side profile) = 100 sample images
- All images were 384x287 pixel grayscale Sun rasterfiles, courtesy of Univ. of Sussex face database.
- 5 men and 5 women in sample set, mostly Caucasian.
- Below is an example of the range of sample poses for one individual in the set. Note that the images are actually subsampled to 25x25 pixels from the 100x100 pixel windows used.
Gabor Wavelet Analysis
- Problem: Bitmap faces are big dimensionally even if we only use a 100x100 pixel window. This is expensive in terms of the time and space complexity of the network. How can we reduce the input space dimensionality, while extracting only the most relevant information?
- A Good Solution: Use 2D Gabor wavelets, class of orientation and position selective functions.
- Biologically plausible: Cells in visual cortex respond selectively to stimulation that is both local in retinal position and local in angle of orientation.
- In this case, we reduced the dimensionality of the input space from 10,000 (a 100x100 pixel window) down to 126.
Preprocessing
- Used a 100x100 pixel window centered around the pixel at the tip of the nose.
  - Wrote NosePicker Java application to display images and save manually clicked nose coordinates.
- Used Gabor orientations (0°, 60°, 120°) with sine and cosine masks = 6 functions.
- Calculated the 6 Gabor masks on 99x99, 4 51x51, and 16 25x25 pixel subsamples = 126 Gabor coefficients = input dimensionality of 126.
- Below are the sampling regions and orientations. Note that for each orientation, both cosine and sine (i.e. real and imaginary component) masks were calculated.
Network Setup/Training
- All input vectors were unit normalized, and the unit normalized gaussian function was used.
- For each p+a face unit network, a fixed set of p poses were used to center the + neurons.
- For each + neuron, the nearest p/a unique negative input vectors were used to center p/a of the - neurons.
- Setting appropriate widths for + and - neurons remains a problem. I've implemented two algorithms to compute the widths, neither of which yields acceptable results. In the first algorithm, each neuron's widths is set to the average euclidean distance between it and all other neurons. In the second, the global average distance is found between each pair of neurons. All neuron widths are then set to this global average.
- Linear output weights were computed by finding the pseudoinverse of the matrix of hidden neuron outputs for each input, A.
  - Since we want Aw = d => w = A^-1 d
  - Used singular value decomposition method to approximate A^-1 since A is non-square and hence is singular.
- Advantage of using the pseudoinverse is nearly instantaneous "training", since training is no longer an iterative process, unlike gradient descent.
- We only need to calculate the pseudoinverse and then perform one matrix vector multiplication to calculate linear output weight vector.
Network Testing
- Inputs vectors that were used to set neuron centers are excluded from the testing set. Network is seperately tested with neuron centers as testing vectors.
- Wrote perl script to automatically generate input files for all people and all desired network sizes.
- Hope to soon have a perl script that automatically runs the input files and compiles network performance results.

Results

Because of the manual width tuning neccessary, I only have complete data for 6+12 networks. I've tested all sizes of networks up to 6+12 but did not systematically compile performance data, as I hadn't tuned the neuron widths. Without good width values, performance was extremely variable across networks trained to identify different individuals.
Selection of neuron widths remains a problem, with manual tweaking necessary for good results.
- Problem of + neurons "crowding out" - neurons, yielding high rate of false positives.
- Related problem of - neurons "crowding out" + neurons, yeilding a low rate of positive IDs and a high rate a false negatives.
- Both these problems could potentially be solved by a more intellegent widths determining algorithm.
- Currently all + neurons widths are set to a default and all - neuron widths are set to a different default.

With some limited width adjustments, the following results we found for 6+12 networks.

Avg. correct	Avg. pro	Avg. anti
72.6%	55.0%	73.5%

Max correct	Max pro	Max anti
95.1%	100%	98.7%

Min correct	Min pro	Min anti
37.8%	0%	37.2%

Note the average correct classification rate was 22.6% better than what would be expected from a random classifier.
Also note that neuron widths were carefully tuned to yield the best network, giving nearly perfect classification.
The worst performing networks had little effort put in tuning the neuron widths, so these numbers understate classification performance with well selected neuron widths.

The following results were published in (Howell 1995a) by Dr. A. J. Howell for his 6+12 face unit networks. Note that same sample image file were used by both Dr. Howell and I.

Avg. correct	Max pro	Max anti	Min pro	Min anti
89%	100%	100%	50%	83%

Dr. Howell's network performed better overall, though my best network was nearly as good as his best.
- Possible explaination: Dr. Howell used a more elaborate preprocessing scheme yielding 510 Gabor coefficients for each image.

References

Chen, S. et al. (1991). "Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks." IEEE Trans. on Neural Networks 2(2), 302-309.
Duahman, J. G. (1988). "Complete Discrete 2-D Gabor Transforms by Neural Networks for Image Analysis and Compression." IEEE Trans. on Acoustics, Speech, and Signal Processing 36(7), 1169-1179.
Duvdevani-Bar, S., Edelman, S., Howell, A. J. & Buxton, H. (1998). "A similarity based method for the generalization of face recognition over pose and expression." (available online).
Howell, A. J. & Buxton, H. (1995). "Invariance in Radial Basis Function Neural Networks in Human Face Classification." (available online).
Howell, A. J. & Buxton, H. (1995a). "A Scaleable Approach to Face Identification." (available online).
Howell, A. J. & Buxton, H. (1995). "Receptive Field Functions for Face Recognition." (available online).
Howell, A. J. & Buxton, H. (1996). "Improving Generalization in Radial Basis Function Networks for Face Recognition." (available online).
Howell, A. J. & Buxton, H. (1998). "Automatic face recognition using radial basis function networks." (available online).
Howell, A. J. & Buxton, H. (1998). "Towards Visually Mediated Interaction using Appearance-Based Models." (available online).
Moody, J. & Darken, C. J. (1989). "Fast Learning in Networks of Locally-Tuned Processing Units." Neural Computation 1, 281-294.
Musavi, M. T. et al. (1989). "On the Training of Radial Basis Function Classifiers." Neural Networks 5, 595-603.

Acknowledgments

Dr. A. J. Howell, School of Cognitive and Computing Sciences, Univ. of Sussex, UK.
- Provided Gabor data and sample face images.
Dr. Robert Oostenveld, Dept. of Medical Physics and Clinical Neurophysiology, University Nijmegen, The Netherlands.
- Provided C routine for singular value decomposition pseudoinverse calculation.
Numerical Recipies Software, Numerical Recipies in C: The Art of Scientific Computing.
- Used their published singular value decomposition routine in C.
And last, but not least... Prof. Robert Keller.
- Valuable guidance and advice regarding this project.

Code Directory

My code directory can be found here.
The following are the significant programs I've written in the process of completing this project:
- RBFnet class and rbf application in C++, implementing a radial basis function network with n dimensional input and 1 linear output neuron.
  - Uses k-means clustering, global first nearest neighbor heuristic, and gradient descent.
- FaceUnit class and face_net application in C++, implementing a scalable face unit network.
- gabor2D program in C++, performing image sampling and Gabor wavelet preprocessing.
- ras2mtx program in C++, converting binary rasterfiles into arrays of gray-values.
- NosePicker Java application, displaying images and saving manually clicked nose coordinates.
- Wrote indexer perl script to generate index.html files that serve as directory listings.
- Wrote make_ifiles perl script to generate input files to face_net program.
- Eventually hope to develop a perl script to automatically run input files and compile performance results.

Presentations Available Online

A PowerPoint presentation I made on John Moody and Christian J. Darken's seminal paper on RBFs from 1989.
A PowerPoint presentation I made for my final presentation of the project.

Created on Thursday, November 4th 1999 by Ben Feinstein.

Last updated on Friday, December 17th 1999.