Computer Science 121 - Project #4

Introduction

Once we have a design we can develop test plans to exercise that design. A good test plan will enable us to gain high confidence in our code as soon as it is written.

In this project, you will, as individuals, develop test plans for the modules that you designed in project 3. The primary purposes of this exercise are:

• to give you practical experience with the identification and definition of necessary test cases.
• to enable you to assess (and benefit from or pay for) the testability that you did or did not design into your module in project 3.

Component To Be Tested

I strongly encourage you to continue working with the module you designed for project 3. Any module design or mechanism prototype that was sufficiently interesting as a design problem, will almost surely give you the opportunity to develop all of the required test cases.

If you did a user interface prototype, and the mechanisms are so trivial that there is very little to test, you have two alternatives:

1. Obtain a module design from a friend on another team, and take great care not to talk to them again about that module until after the end of project 5.
2. I will find you a module that I assure you will contain ample opportunities for testing.

The Assignment

1. Starting with the component requirements that you developed (as a group for project 3), specify a complete set of acceptance test cases for your module.
2. Starting with the component specifications that you developed (as a group for project 3), specify additional test cases to validate specified functionality that was not directly implied by the basic requirements.
3. Based on the detailed designs (that you prepared individually) specify additional (white box) test cases to thoroughly exercise all interesting functions and mechanisms that are not reasonably exercised by the requirements and specifications based test cases.
4. Describe any components that will have to be stubbed or simulated in order to test your component, and where simulation is in order, describe how you will do it.
5. find or describe a practical framework for automating all of these test cases.
6. describe the set-up and execution for each test case, and how the results will be processed to generate a pass/fail result.
7. write a post-mortem for
   • the process of developing the test cases.
   • the process of elaborating the test case specifications
   • the testability of the design you inherrited from project 3.
   • this project as an educational exercise.

The deliverable work products should include:

1. an index of work products
2. references to the requirements, specifications, and design for the component you are testing.
3. a description of the overall testing strategy and proposed automation framework, including a discussion of how you will simulate any missing functionality needed to exercise your component.
4. a complete set of requirements based acceptance test cases.
5. a complete set of additional test cases, derived from the specifications for your component.
6. a complete set of additional test cases, derived from the design of your component.
7. a post-mortem

I am not specifying formats for any of these work products, and will accept them in any combination of ASCII text, html, word, powerpoint, and standard graphics formats. Part of the exercise is for you to review what we've said about the purposes of these work products, and to figure out what information each should contain, and a way to effectively convey that information. I believe that forcing you to work this out for yourselves will have greater educational value than giving you forms to fill in.

Rules, Submissions and Grading

This is an individual project, but it is based on Projects 2 and 3 (which were team projects). It is entirely reasonable for you to discuss general matters like stubbing and testing frameworks with your team members. You must, however, define and describe all of your own test cases, and write your own approach to automated testing.

1. You may not include (uncited) material written by anyone else in a work-product for which you have primary responsibility.
2. You may not share your work products with anyone else.

You may do research on testing frameworks and approaches to problems. Cite all of your sources for each work product, and explain how each source contributed to your work products.

All of your primary work-products will be developed, maintained, and submitted in subdirectories of the subversion repositories that have already been created for your team. I expect the primary work products to evolve as you gain experience, and to see that evolution reflected in the svn commits and comments.

When you are done, package together:

1. the URL for your subversion repository
2. an index of your work products, briefly enumerating and describing each.

And send all of these things to me in an e-mail with the subject "cs121 - proj4". If you will be submitting a project late, please notify me of this on or before the project due date. When you finally do submit it, please tell me how many of your (3 each) slip-days you want to use.

You will be graded on:

• producing reasonable test cases of each specified type.
• the clarity and quality of your test cases, judged by the criteria discussed in the reading and lectures.
• the completeness and reasonableness of your proposed testing regimen as a means of gaining confidence about the correctnes of your component. (Note that you can lose points for missing test cases or clearly redundant or unnecessary test cases)
• The completeness and reasonableness of your test automation proposal.
• The completeness and quality of your post-mortem analysis.

So that you will know exactly what I expect to see, here is a pointer to the grading guidelines that will be used to score your submissions.