The goal of this project is to experiment with different algorithms for isolated gesture or shape recognition.   What you need to do is simple to explain: choose two shape recognition algorithms from the papers on the reading page, implement both, and compare their performance on data you collect.  Once again, however, this assignment is not as easy as it sounds (which is why you have 3 weeks to complete it), so please start early!  

As usual, you can implement additional functionality (suggested below) for extra credit.

Software Resources for this assignment

You will find a number of resources that you may find useful in implementing this assignment on the Sakai resources site.  You do not have to use these resources.  For some recognizers, they may not make sense.  Similarly, if you come across any resources that you think would really help you, let me know and I'll see if I can track them down.  Please look at the Sakai site for more detail.

Basic Functionality [60 points]

Implement a WPF application with the following functionality:

• Recognizes a set of at least 15 shapes or gestures (you may include more) with at least two different shape recognition algorithms of your choice.  Which gestures/shapes you choose is up to you, but you should choose a good variety (i.e., not all characters).  No matter which algorithms you choose, you should include some single-stroke and some multi-stroke gestures/shapes in your set.
• Allows the user to draw a gesture and then see the results of both classifiers.  You can report your results however you like.  Simple text boxes are fine. 
• Allows the user to trigger recognition either using a button, a gesture, or just pausing for a second.  You can choose to clear the screen with each new gesture (although you should keep the user's ink around when you display the results so that you can look at the results along side the original input), or you can provide a way for the user to clear all their ink at once.   Note that you will need some way of grouping users' strokes into symbols.  It's fine to use explicit grouping (e.g., the user taps a button or lassos the strokes, or pauses... etc).
• Most of the recognizers you will need to train.  You do not need to allow the user to re-train the recognizer (although you can for extra credit), but you should save the data you used to train your recognizer.  That is, you should implement your training by reading data from a file or multiple files.  The easiest way to do this is simply to have one file for every training example, or one file for every set of training examples or the same type, with the different examples sufficiently spaced out so that you easily determine where one symbol ends and the next begins.  Once you've trained your recognizer, it's find to hard-code trained data into your program (although for maximum flexibility it would be nice to read this data from a file so that the same code can be used for different training sets).

Advanced Functionality [up to 15 points extra credit]

For extra credit, implement any or all of the following extensions:

• Allow the user to interactively train either or both of your recognizers by entering a "training mode".  Perform experiments to measure the impact of this additional (user-specific) training.  Comment in your writeup about what impact this additional (user-specific) training has in practice.
• Perform an experiment that compares the impact of training each recognizer on user-specific data vs. mixed data.  Collect training data from multiple people, and train and test your recognizers on each individual separately vs. training and testing the recognizer on mixed set of data.  Report detailed results in your writeup.
• Perform an experiment to compare the amount of training data to each recognizers' performance.  Graph the recognition accuracy vs. amount of training data and comment on your results.

Testing and Writeup [40 points]

Once again, a major component of this project will be your writeup.  Submit the same three files as in the previous assignment: a README file, a Design file and a Testing file.  Details on what should be in each are given below.

Readme file

In a file called README.txt describe how to run your program, how to use it, any missing features or bugs that you are aware of, and any extra credit that you implemented.  Also describe your gesture/shape set, including how to draw them, and pictures where appropriate.

Design/approach Justification

In the first part of your writeup, you will analyze your design.  In a file called "Design.txt", write up answers/explanations for the following:

1. Which recognizers did you choose to implement and why?    What were the most challenging parts of implementing each and how did you solve these challenges?
2. Give an overview and critique of your design from a software engineering perspective.  How did you organize your data?  What worked well and what would you change if you had to do it over again.
3. Comment on any other aspect of your implementation that you feel needs explaining (e.g. running time, data storage, user interface issues, etc).

Testing and Results

In the second part of your writeup, you will report on how well the two algorithms perform.  Unlike the previous assignment, you should focus mainly on quantitative data here.

Once you have your recognizers trained, test them on 5-20 examples of each symbol and keep track of each recognizer's performance.  Report your data in two formats: First, give a side-by-side comparison of each recognizer's performance on each symbol by reporting the percentage of examples of each symbol that was recognized correctly (i.e., was the best match) by each recognizer. Second, give a confusion matrix for each recognizer to show which symbols were confused with which others.  
Next, write one or two paragraphs about the recognizers results and why you believe they behave the way they do.

Finally, comment on which recognizer you would choose in the future and why.

Criteria

Your writeup will be graded on the following criteria:

• Overall completeness: Did you provide all the information requested above?
• Soundness/completeness of your Justification: Is your design justification compelling (both for your algorithms and your code)?  Are there solid reasons for your decisions?  Even if your design is not ideal, do you clearly identify what could be improved?
• Depth and Breadth of results: Do you discuss your results at a non-trivial level in order to illustrate the high-level strengths and weaknesses of your algorithms?  Have you tested your code on a wide variety of real-world examples?
• Clarity of expression: Is your writeup easy to read and grammatically sound?

What to Submit

Through Sakai, submit the following:

• Your code, including everything I need to run it
• The three writeup files described above
• Your training data for each recognizer.   If you can, also submit your testing data (but if you did not save your testing data, that's OK).