Project 3 - Fall 2012
Planning, Specifications, Design, Review
Dates prefixed with a tilde (~) are suggestions.
You will decide when you actually want to complete these
milestones and include those dates in your prelinary management plan.
1 Submissions are due by mid-night of the due-date, and must be e-mailed to
cs181-e@cs.pomona.edu .
2 Unless otherwise specified, all written materials should be
submitted as URLs to ASCII text files on github.
3 less 10% for each unexcused late day.
Table of Contents
- Introduction
- Major Activities and Deliverables
- Rules
1. Introduction
The first two projects were largely team activities. In the next two projects
you will do much of your work as individuals, tho with considerable assistance
from other team members. Each member of the team will (over the next two projects)
be responsible for the design and implementation of one piece from your architecture.
Depending on your architecture, these could each be a complete architectural component,
or small pieces (e.g. an applet or a few classes) from a single architectural component.
Choose your pieces carefully:
- each of the chosen pieces should be somewhere in the range of 100-400 lines of code.
- each person must be capable of independently building and testing his/her own piece.
- each of these pieces must be testable in isolation.
- when you are done, it should be possible to combine all
of these individual pieces together to create a working
(all the pieces working together) and demo-able component.
Your pieces can be implemented in any language or combination of languages,
and use any tool-kits or middle-ware you find convenient ... but it must be
code (not HTML or images).
In this project you will create specifications, designs and testing plans
for the chosen pieces.
In the next (and final) project you will execute these plans, building,
testing, integrating, and demonstrating working software.
2. Major Activities and Deliverables
There are several major parts to this project, each of which has its
own goals, processes, and deliverables:
- Initial Plan
You will, as a team, select the pieces to be designed
and implemented, and develop a plan for this project,
dividing the work over your team members and the available time.
- Component Specifications
You will, as a team, elaborate the architecture above
the chosen pieces sufficiently to generate detailed
specifications for the pieces. You will then write
up detailed component specifications for each of the
pieces to be designed in this project.
- Detailed Component Design
You will, as individuals, prepare a detailed design for
one of the selected pieces.
- Unit Testing Plan
You will, as individuals, prepare a detailed unit testing
plan for the piece that you have designed.
- Submit for Review
You will, as individuals, package your specifications,
design, and test plan, and submit it for a design review
by the other members of your team.
- Design Review
You will, for each submitted package:
- as individuals, study the package and prepare notes.
- as a team, conduct a formal design review.
- one team member will write up the results of that reivew.
- Revision and Final Designs
You will, as individuals, revise your designs and plans
to address all important issues raised in the design review.
- Post Mortem
You will, as a team, review the processes you followed in
this project to see what lessons you can learn for how
to do things more effectively in the future.
- Project Management
All of the above will be planned, documented, and managed.
2.1 Plan
The amount of work required to refine your architecture to the point that
it is possible to identify and specify your chosen components will vary
greatly from one project to the next, and I would encourage you to get
this behind you as quickly as possible. Once you have a sense of what
the chosen components are, you should have a pretty good idea of how much
work it will be to do the designs and test plans. You should, however,
leave yourself ample time for discovering issues in the review process
and making the required changes.
2.1.1 Deliverables
Each team will prepare a task-breakdown, identify the dependency relationships between
tasks, and owners for each sub-task, assign due-dates, and schedule regular reviews
of both work-products and progress (to enable adequate time to deal with the
problems that will happen).
A good management plan will include regular status
checks, the results of which should be recorded in the plan document.
As your understanding of the problem
evolves and you respond to unanticipated events,
you will have to revise your plan (not merely estimates, but
the work to be done). Make sure that you document each of these
problems and the manner in which you decide to respond to it.
If deadlines are missed, or deliverables fail to pass review, the fact, as well
as the causes and the plan to remedy them must be documented.
2.1.2 Grading
Your initial Management Plan will be graded on the basis of:
- 40% good use of time and resources (work apportioned reasonably over the available time)
- 20% specificity of plan (clear responsibilities: who, what, when)
- 20% provisions for early detection of problems, and time to deal with them
- 20% completeness
2.1.3 Submission
Maintain your plan on github.
You will surely be updating it, and we will be reviewing your history
(changes and comments) to see how you dealt with problems.
When you are ready to submit it for grading, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3A - Preliminary Plan
Plan: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.2 Component Specifications
Each team member will take ownership of one or more components.
To prepare a component specifications, you will:
- expand the architecture (above those components) to describe
those components and the components with which they interact.
Assuming that you have decided to implement pieces of your system that
are smaller than complete architectural components (from project 2),
you will have to expand and refine that architecture down to the
level of the pieces you want to use for projects 3 and 4. Note that
you do not have to expand everything in your architecture to this
level of detail. You only have to do a top-down refinement along
the path to the components you will be using for this project.
- generally describe the functionality of each of the components
to be designed, and its role in the overall architecture.
- determine the requirements to be imposed on the components to
be designed, based on the product requirements they implement,
and their role in the overall architecture.
- characterize the external interfaces between the components to be
designed and the rest of the system.
- describe those interfaces and the required functionality with
sufficient clarity and specificity as to be a complete set
of acceptance criteria.
2.2.1 Deliverables
A specification will be prepared for each component to be designed and
implemented in projects 3 and 4. Each component specification should
include:
-
An addendum to the (Project 2) architectural description
filling in any additional context required for the new
component descriptions.
It is quite possible that a single addendum (created by the
entire team) can be used for all of the component specifications.
-
A general description of the component and its role in the system.
-
A complete specification of all external interfaces to this component
(both form and functionality).
-
A complete set of requirements that must be satisfied by this component.
2.2.2 Grading
This submission be graded on the basis of:
- 10% reasonableness and understandability of the architectural
addendum (if any).
- 10% reasonableness and understandability of the components
role in the overall system.
- 10% reasonablness and understanability of the component functionality
- 20% completeness of the external interface specifications
- 20% reasonableness of the external interface specifications
- 10% abstraction and modularity of the component definition
- 20% completeness, reasonableness, and measurability of the requirements
2.2.3 Submission
When you are ready to submit your preliminary component specifications,
for review fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3B - Component X Specifications
Revised Architecture: https://github.com/your_repo/...
primary author: person1
Component X Specifications: https://github.com/your_repo/...
primary author: person2
...
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.3 Component Design
Each team member will prepare a detailed design for one or
more modules, ideally comprising 100-400 lines of code when complete.
This design need not be at the level of complete pseudo-code, but
(in combination with the specifications) should be sufficient to
enable a skilled programmer to easily implement the specified module(s).
Each module design should include:
- a description of the purpose of this module.
- complete declarations for all public methods and instance variables.
- complete declarations for all private methods and instance variables.
- declarations for all private routines.
- descriptions of purposes for all methods/routines.
- descriptions and meanings for all parameters.
- descriptions and uses of all data structures.
- descriptions and uses of all public and non-trivial private variables
(where trivial variables would be things like loop indexes,
saved values, and intermediate results)
- general descriptions of all non-obvious algorithms.
These need not be pseudo code, but could simply be
descriptions of an approach (e.g. "allocate from the heap",
"bubble-sort" or "compute SHA1 hash"). If a named
algorithm is used, provide a reference to a description
of its implementation (in case the programmer who will implement
it isn't familiar with it).
- where the code is to be based on standard libraries,
call those out (e.g. "synchronization will be with
boost thread shared locks") and include a reference to
the documentation for the functions to be used.
If any of these design elements are non-obvious, the rationale
for those decisions should be described so that the implementer
can better understand what they are to do. People are more likely
to make mistakes when working on things they do not understand.
2.3.1 Deliverables
You can prepare your designs in any form you find convenient,
but you will probably find it easiest to create them as
code modules with compilable declarations, extensive comments,
and very little actual code. Overview and rationale can be
presented as comments in front of the described elements.
2.3.2 Grading
This design be graded on the basis of:
- 20% all external methods and instance variables are
described well enough to enable a skilled programmer
figure out how to use them correctly.
- 20% all implementations and internal methods are
described well enough to enable a skilled programmer
to implement them (with no more research than having
to read the referenced documentation).
- 15% all variables and data structures are described
well enough to enable a skilled programmer to use
and update them correctly.
- 10% the reasonableness (after having read the rationale)
of the specified implementations.
- 10% the completeness of the descriptions (no major questions,
and all required documentation references are provided).
- 10% the correctness of the described implementations.
- 5% the simplicity (relative to the problem) of the proposed implementations.
- 5% the likely efficiency of the described implementations.
- 5% the readability of the design document (and referenced documentation).
2.3.3 Submission
When your design is ready for submission, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3C - Component X design
Design: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.4 Test Plan
Once you have specifications and a design to satisfy them, you have
to figure out how you will test your implementation, to ascertain
that it actually works.
-
Review your specifications, and develop a set of automated comprehensive
(black box) test cases to determine whether or not your component meets
its requirements.
-
Review your design, and consider any possible/plausible component
misbehavior that would not be detected by the the above-described
acceptance tests. If you find such behavior, define additional
(white box) test cases to exercise and validate those situations.
If there is no possible mis-behavior not covered by the acceptance
criteria, you can earn the white-box points by offering a convincing
proof of this assertion.
-
Consider your component's interfaces and all of the above test
cases, and design a means for performing all of those test cases
in a fully automated fashion ... ideally using an existing
unit testing framework (to structure your test case implementations,
invoke them, and collect the results).
-
Review the overall list for completeness (is everything covered),
efficiency (are there redundant tests) and value (are there
tests for things that don't matter or will never happen), and
try to optimize out the low-value tests. If the optimizations
are not obvious, you should document these decisions in your test plan.
It is likely that, in the process of developing your test plan, you
assertions that are hard to measure, or functionality that is difficult
to verify (or verify automatically). If this happens, you may need to
revisit your requirements, specifications and design.
2.4.1 Deliverables
Your test plan should include:
-
A general overview of the functionality to be verified,
and the general approach(es) that will be taken to verification.
-
An index of test cases.
-
A description of the framework under which this testing will be performed.
-
A complete list of test cases ... for each:
- a brief summary of the assertion to be tested
- the justification for including this test case
- any special set-up required to run this test
- how the situation to be tested will be generated
- how correctness of behavior will be ascertained
- any special clean-up required after this test
2.4.2 Grading
Your test plan will be graded on the basis of:
- 30% black box tests
- traceability to specifications
- reasonableness of test cases
- completeness of suite
- form and quality of descriptions
- 30% white box tests
- traceability to design
- reasonableness of test cases
- completeness of suite
- form and quality of descriptions
- 20% testing framework
- automation
- reasonablenss practicality
- use of off-the-shelf technology
- 10% overall completeness (likely confidence)
- 10% overall efficiency (development and execution)
2.4.3 Submission
When your test plan is ready for review,
fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3D - Component X test plan
Test Plan: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.5 Review Notes
Each member of the team will submit his preliminary specifications,
design and test plans for review by the other team members. Each
team member will participate in the reviews of the designs and plans
submitted by the other members of his/her team.
2.5.1 Deliverables
Prior to each review meeting, each of you (individually) will read
the submitted specifications, designs, and test plan and prepare detailed notes
on all questions and concerns. These notes must be submitted at
least 24 hours prior to the actual review session. They should be
neat notes, describing legitimate issues clearly enough to be sent
as email, and organized for discussion (e.g. in the recommended
review order).
2.5.2 Grading
Each set of review notes will be graded on the basis of:
- 40% submitted 24 hours before scheduled review
- 30% the thoroughness of study to which they attest
- 10% how well articulated the issues are
- 10% all comments appropriate and within scope
- 10% issues reasonably organized for discussion
2.5.3 Submission
When you have prepared your study notes, check them in to
github, fill out the following form, and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3E - Component X - study notes
notes: https://github.com/your_repo/...
author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.6 Review and Report
Your team will conduct a formal design review for each submitted
Specification/Design/Test Plan package. The process will be the
same as for the architectural review ... but because you have already
followed this process (for your architectural reviews) these
reviews will not be observed and graded. But you are still
responsible for producing a report for each review, and these reports
will be graded. Each team member will lead one design review, and
act as scribe for one design review.
As with the architectural review,
this is not "meeting minutes". Rather it is a distillation of key issues and
decisions. It should be carefully written and reviewed. It must contain:
- the time, date, and attendees
- the component to be reviewed
- a clear summary of each important issue rased
- a characterization as a defect, question, or issue
- a characterization as must-fix, should-fix, or comment
- a disposition for the entire proposal of
- approved
- approved with required changes
- requires another meeting
- rejected (this cannot be made to work)
2.6.1 Deliverables
Each team member should act as scribe and prepare the report for
one of the design reviews.
2.6.2 Grading
Each review report will be graded on the basis of:
- 10% form: time, place, project, attendees
- 20% clear disposition for each issue
- 30% clarity with which issues are presented
- 20% scope of the report (just issues, no opinions)
- 10% completeness and accuracy
- 10% disposition of the entire project
2.6.3 Submission
When you have completed your Review Report, the scribe should fill out
the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3F - Component X - review report
report: https://github.com/your_repo/...
scribe: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.7 Final Component Specifications, Design and Test Plan
It is likely that your design review will turn up issues that require
changes to your specifications, design and test plan. Address those
issues (by revising your specifications, design and test plan), document
the changes that were made to address each, and get agreement from the
reviewers that the issues have been satisfactorally addressed.
2.7.1 Deliverables
- summary of changes made and reasons for each.
- updated architectural context.
- updated component specification.
- updated updated component design.
- updated test plan.
2.7.2 Grading
This submission be graded on the basis of:
- 25% responses to raised issues and resolution w/reviewers
(as verified by e-mail or plan updates)
- 25% quality of final specifications (using 2.2 grading metrics)
- 25% quality of final design (using 2.3 grading metrics)
- 25% quality of final test plan (using 2.4 grading metrics)
2.7.3 Submission
When you have addressed all of the issues raised in your review and are
ready to submit your final component specifications, design and plan,
fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 3G - Component X - final design/plan
Summary of changes: https://github.com/your_repo/...
Specifications: https://github.com/your_repo/...
Design: https://github.com/your_repo/...
Test Plan: https://github.com/your_repo/...
author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.8 Post-Mortem Report
This project is a learning exercise, and one of the major ways
we learn is by analyzing past mistakes. You will, as a team,
review all aspects of this project. One of you will then
summarize that process into a post-mortem analysis report.
2.8.1 Deliverables
A report, summarizing the key issues raised in your post-mortem,
and the conclusions you came to. Your post-mortem discussion
should include:
- architectural refinement and specification development.
- development of the component designs.
- development of the test plans.
- the review process and the resulting design/plan changes.
- the planning and ongoing management of these activities.
- the overall project as an educational exercise.
Note that if you make no mistakes, you will not be able to earn any points
on the post-mortem. Fortunately, no teams have ever found it necessary
to deliberately make mistakes in order to have something to analyze.
2.8.2 Grading
This report be graded on the basis of:
- 50% whether or not you meaningfully discuss each of the required activities.
- 20% whether or not you identify all of the important incidents.
- 30% the extent to which you are able to derive and articulate useful lessons
(and good future advice) from those experiences.
2.8.3 Submission
Make sure that you have kept your management plan up-to-date,
upload the notes from your post-mortem discussion to github,
fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: #3H - Post-Mortem
Plan: https://github.com/your_repo/...
primary maintainer: person1
Report: https://github.com/your_repo/...
primary editor: person2
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.9 Management
The team will be graded on how effectively they managed themselves
and the work:
- Perform the work according to the plan.
We will determine when work was done by looking
at the dates of the final check-ins.
- Monitor progress and detect problems.
Monitoring plans should be included in your
project plan. Status should be reviewed regularly,
and logged as updates to the project plan.
Problems that come up should be recorded as updates
to the project plan and discussed in your post-mortem.
- Reasonably adjust the plan in response to problems.
Detected problems and responses should be
reflected in revisions to the plan, and discussed
in your post-mortem.
3. Rules
3.1 Due Dates and Slip Days
Any assignment that is not turned in by its due date,
will have its grade reduced by 10% (of its nominal value) for each
late day. I understand that problems happen, and so each student is
granted three slip days. A team of four students would have
twelve slip days between them.
One slip-day will excuse one deliverable
being late by one day.
If a design is one day late, and that delays the
design review by a day, then you may find that you need two slip-days.
Don't make the mistake of using up your slip-days too soon, because
the pressures generally get tighter later in the course.
When a team needs to use slip-days, they need to decide who's
days they are using. However you decide is OK with me, but
we will be tracking the slip days per student.
If will be using any slip days on a deliverable, tell me in
advance that you will be doing so (so that I don't nag you
about whether or not I lost your submission message).
3.2 Collaboration and Citation
This is a team project, but different individuals will have primary
responsibility for different processes or work products (or different
parts of a single work product or process). Each team will be working
on a different type of product. You are free to talk your team members
(and for that matter other teams) about the processes you are following.
You may review your work products with your own team members, and revise
them based on their feedback ... but ...
- If you are the primary author of a work-product, you must
cite the source of text you did not write (if it is more
than a sentence), or any information that did not originate
within your team or from your interviews.
You will be doing research to develop your product definition and
requirements. I expect that many of the ideas for your product
will come from this research. Cite all of your sources for each
work product, and explain how each source contributed to your work
products.
- You may not share any of your work products (other than as required
for reviews) with members of other teams.