Project 4 - Fall 2012
Implementation and Testing Sprint
1 Submissions are due by mid-night of the due-date, and must be e-mailed to
cs181-f@cs.pomona.edu .
2 Unless otherwise specified, all written materials should be
submitted as URLs to ASCII text files on github.
3 less 10% for each unexcused late day.
Table of Contents
- Introduction
- Major Activities and Deliverables
- Rules
1. Introduction
The first three projects took us through all of the activities that precede
implementation. In this final project you will actually implement the components
you designed in project 3 ... and by now it should come as little surprise to you
that only part of your time will be spent coding, and (if you do it right) very
little of your time will be spent debugging.
2. Major Activities and Deliverables
The major activities in this project are:
- Implement your components
Each of you will implement the component you designed in project 3.
- Pair Programming exercise
Two of you will spend (at least) one session doing pair-programming
on one of these components, after which you will write a brief
(joint) report on the experience.
- Test Driven Development
(at least) one of you will develop your component in a Test Driven
fashion (implementing and running tests as you complete each routine
or feature), and then write a brief report on the experience.
- Code Review
(at least) one of you will implement your component and then submit
it for code review by the other members of your team. The reviewers
will submit their notes, and the reviewee will write a brief report
on the experience.
- Test Suite
Each of you will implement the component test plan you designed in
project 3, and use it to validate the correctness of your component
implementation.
- Sprint Review/Demo
You will, as a team, prepare a brief sprint review:
- describing the components you implemented and their functionality
- demonstrating a successful build and test execution for each
component
- demonstrating the functionality of the combined set of components.
- summarizing your key accomplishments and what progress this
represents for the (greater) project.
- Post Mortem
You will, as a tream, review the process you followed in this project
to see what lessons you can learn for how to do things more effectively
in the future.
Each member of the team must incorporate one of the three development exercises
(Pair Programming, Test Driven Development, or Code Review) into his/her
implemenation, and all team members will participate (as a submitter or reviewer)
in at least one code review. While most of these activities will involve selected
individuals, you are encouraged to discuss each of these activities as a team,
and the reports from those activities will be graded as team deliverables.
You may note that there is no management plan or grade associated with this
project. You should, by now be able to plan and coordinate activities for
yourselves, and you are already being graded on your ability to deliver the
required work on schedule ... which is the point.
2.1 Final Code
The primary activity in this project is for each person to implement
and test the component they designed in project 3. There will be
many processes and exercises surrounding this implementation, but
the primary deliverable is working code that implements the requirements
and specifications set out in project 3.
2.1.1 Deliverables
Source files and instructions/scripts (e.g. ant/makefile) required to build them.
Make sure that you document the build environment that is required to build your
components, because part of your grade will depend on the TA being able to
independently build your product from the checked-in sources and instructions.
If this is not practical (e.g. because your component cannot be built on a Linux
system) make special arrangements with the TA to have him observe a check-out
build-from-scratch.
You should also re-submit the
specifications and design for this component (from project 3)
with any changes you have made since then.
2.1.2 Grading
This submission be graded on the basis of:
- 10% appropriateness of chosen languages, middleware, and tools.
- 20% completeness relative to project 3 design and specifications
- 20% how well it corresponds to the (revised) design
- 20% code quality (use of language features, simplicity, efficiency)
- 20% code understandability (structure, layout, comments)
- 10% grader is able to build it from source
2.1.3 Submission
After you have completed all of your implementation and testing, and
you believe your code is entirely complete, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4A - Final Code - Component X
design: https://github.com/your_repo/...
source: https://github.com/your_repo/...
build: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.2 Pair Programming Exercise
At least one member of the team will ask another member to join
them for at least one pair-programming session. How you divide
up your effort (think/code, code/review, code/test) is entirely
up to you, and you are welcome to try various approaches.
For all work done in a pair-programming mode, the commit comments
made during those sessions should reflect the division of
responsibilities under which that work was done.
NOTE: what ever component this is done for should not also be used
for code review or Test Driven Development.
2.2.1 Deliverables
After the end of each pair-programming session, each of the
people involved should jot down notes on what happened. After
the component has been completed, the two people should get
together (ideally discussing it with the entre team) and
write up a report on the experience. This report should cover:
- what role divisions you tried and how you decided how to organize those activities.
- how effectively each persons time was used (with each division of roles).
- the speed of code development, vs working alone.
- the quality of the code, vs working alone.
- pleasant or unpleasant aspects of the experience.
- how you would do this differently next time.
2.2.2 Grading
This activity and report be graded on the basis of:
- 20% documentation of collaboration in commit comments
- 20% reasonable role divisions, reasonably carried out
- 20% amount of work accomplished in this mode
- 20% quality of work accomplished in this mode
- 20% reasonable insights into the process
2.2.3 Submission
When you are ready to submit this report for grading, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4B - Pair Programming Report - Component X
report: https://github.com/your_repo/...
authors: person1, person2
commits: ...
...
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
It will make the grader's job much easier if you can refer us
to specific commits done during the pair programming sessions.
2.3 Code Review
At least one member of the team will write all of his/her code,
and before running test suites against it, submit that code for
review by the other members of their team.
The other team members
will study the code, prepare, nodes and conduct a formal code review,
producing a report with must-fix/should-fix/advice items.
The author will make the appropriate revisions, and then move on
to testing. After the code is working, the author will discuss
the process with the rest of the team and then write
up a report on the process.
NOTE: What ever component this is done for should not also be used for
TDD or pair programming.
2.3.1 Deliverables
The author will prepare a review package containing the component
requirements, specifications, and design (perhaps modified) from project 3,
and the implementing code.
Each of the other team members will review that package and prepare
written notes, which will be checked in to github
well prior to the review.
After the review, a formal report will be prepared (listing all important
conclusions) and checked in to github.
The author will address all of the raised issues, and then complete the
testing.
The report on this activity (different from the report from the review)
should cover:
- how valuable the input received from the code review process was,
and what kinds of problems it turned up.
- what kinds of problems remainined in the code after the code
review process, and why they weren't found.
- the relative merits of doing code review before or after testing.
- would knowing that you were going to have a code review have
caused you to make any changes to your test plan?
- how would you do this differently next time?
2.3.2 Grading
Team submissions
- 10% notes: submitted well before scheduled review
- 40% notes: thoroughness of study to which they attest
- 5% notes: articulation and organization of issues
- 5% notes: comments appropriate and within scope
- 15% report: completness with respect to issues raised
- 10% report: clear disposition of every issue
- 10% report: issue clarity
- 5% report: all comments appropriate and within scope
Author's submissions:
- 10% review package: adequate background
- 20% review package: clarity of requirements, specifications, design
- 20% review package: code quality
- 10% review package: code understandability
- 20% report: code improvement value extracted from the review process
- 20% report: insights gained from the process
2.3.3 Submission
When you have your code package ready for review, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4C - Code Review package - Component X
requirements: https://github.com/your_repo/...
design: https://github.com/your_repo/...
code: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
Each team member should check their review notes into github as soon
as they are ready. After the review has been completed and the final
report written, it too should be checked into github.
Fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4C - Code Review - Component X
notes: https://github.com/your_repo/...
author: person1
notes: https://github.com/your_repo/...
author: person2
notes: https://github.com/your_repo/...
author: person3
review report: https://github.com/your_repo/...
author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
After the code is working the author should prepare a report on the
review process and the value it added, and then
fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4C - Review Report - Component X
report: https://github.com/your_repo/...
author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.4 Test Driven Development
At least one member of the team will use Test Driven Development to
implement his/her component, building and running the test cases for
each increment of code as the new code is added. The rewards for
this approach should be:
- the there should be little debugging to do, and what little
there is should be quite simple.
- by the time the coding is done, most of the testing will also be done.
But it will require the test framework to be working first, and more up-front
planning about the order in which things should be implemented. Check in this
plan before you start coding. There are a few tricks to this planning:
- there is a natural order to implementation and testing, because
some features fundamentally depend on others. These dependencies
must be recognized.
- some test cases may be only applicable to incomplete code
and become obsolete after the code has been completed. These
represent a waste of work and should be avoided as much as
possible.
As evidence that you did infact follow a TDD process, and for keeping a
record of the problems found, please:
- commit the new code and associated test cases together,
before running them.
- after you have passed the tests, commit the updated code
and test cases (again together), with comments describing
the problems found and fixed during the testing. If they
worked first time (this will happen) follow this up with
a trivial commit and a comment to that effect.
- do not move on to implementing new functionality until
you have passed all the tests for the existing functionality.
NOTE: What ever component this is done for should not also be used for
code review or pair programming.
2.4.1 Deliverables
After completing development, the/each person who uses this methodology
will discuss the experience with the team and write a brief report, covering
what they did, and specifically addressing the following questions:
- how the implementation order was decided, and how that order worked.
- the efficiency of the process was (e.g. how much time went into building
test cases that were only useful during the construction process).
- to what extent do you believe that knowing how you were going to
test the code caused you to write code that was more correct?
- were there bugs that showed up later that were not found by the
TDD process? If so, why do you think they were not turned up
earlier?
- how would you do this differently next time?
2.4.2 Grading
This activity and report be graded on the basis of:
- 10% a reasonable plan for implementation order (checked in before starting)
- 30% tests were written and passed incrementally (from the commit history)
- 20% each test meaningfully validated the associated code (from the commit history)
- 20% automation framework enabled testing from the start
- 20% reasonable insights gained from the process
2.4.3 Submission
After you have completed your report, check it in to github, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4D - Test Driven Development - Component X
report: https://github.com/your_repo/...
primary author: person1
commit: ...
commit: ...
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
It will make things much easier on the graders if you list the commits
that represent each increment of functionality and the associated tests.
2.5 Final Test Suite
Each team member will, for his/her component, implment the
test plan proposed in project 3, and run those tests against
their component implementation.
The execution of your test cases should be automated (e.g. so that
all tests can be run with a single command), and all of the test
cases and scripts should be checked in to git-hub.
Make sure that you document the environment and procedure for running
these tests, because part of your grade will depend on the TA being able to
independently test your product from the checked-in sources and instructions.
If this is not practical (e.g. because your component cannot be tested on a Linux
system) make special arrangements with the TA to have him observe a check-out
build-and-test-from-scratch.
2.5.1 Deliverables
Each team member should submit:
- a reference to your (perhaps updated) test plan from project 3.
- the code that implements your test suite (and tools you needed to build).
- the output of having run that test suite against your component.
It is quite likely that you will, as a result of lessons learned during
the implementation, decide you want to change your test plan. If this
happens (a) update your project 3 test plan and (b) submit a summary of
and explanations for the changes.
2.5.2 Grading
This activity and report be graded on the basis of:
- 20% completeness relative to project 3 test plan
- 20% does the program pass the test suite
- 10% tests: implementation code quality
- 10% tests: readability
- 10% choice and exploitation of automation framework
- 10% automated test execution and pass determination
- 10% clarity of tests run and pass/fail indications
- 10% grader is able to run and pass test suite
2.5.3 Submission
When you are ready to submit this package for grading, fill
out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4E - Final Test Suite - component X
test plan: https://github.com/your_repo/...
test suite: https://github.com/your_repo/...
test results: https://github.com/your_repo/...
primary author: person1
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
2.6 Sprint Review/Demo
You have built your components, and your test suite, and you have
run your tests. Hopefully you have been able to combine your
components and demonstrate functionality for the integrated whole.
Now it is time for you to review what you have produced with your
product owner.
At the end of each SCRUM sprint, the team presents the work that was
completed during that sprint to the product owner. This is, in part,
ceremonial (the team can claim success and receive feedback on the
work they have completed) but it also a very practical sign-off:
- the team briefly reviews the requirements to have been
met and demonstrates that the product now meets them.
- the team overviews the testing that has been done.
- the product owner decides whether or not this work
has actually been completed (i.e. deliverable
functionality).
- velocity points are earned for accepted work.
2.6.1 Deliverables
Your review presentation should include:
- a brief summary of the functionality of each of the components
built during this sprint and the (component level) requirements
it was to meet.
- a brief overview of the scope of the automated testing plan for
each component.
- a brief overview of the expected functionality of the integrated
pieces.
- a demonstration of a check-out from git-hub and (error-free)
build and automated test of each component (according to that test plan).
- a demonstration of the functionality of the combined pieces,
showing them working together, and that all (applicable) key
requirements have been met.
- a summary of what progress this represents towards the construction
of your larger project (what this means and what comes next).
This is not a long presentation (4-5 minutes will be fine).
It might be useful to have slides to cover the components,
their requirements, and their test plans, but this is not
necessary.
2.6.2 Grading
This presentation be graded on the basis of:
- 10% overview of components and their requirements
- 10% overview of component test plans
- 10% overview of integration and resulting functionality
of the combined sub-components.
- 20% error free check-out and build from scratch
- 20% error free check-out and execution of all test cases
- 20% demo shows component meets all of its key functional requirements
- 10% demo shows clear evidence of successful integration
(that all sub-components are clearly interoperating).
2.6.3 Submission
When you have an idea when you will be ready for your review,
contact either the professor or the TA to schedule it.
2.7 Post-Mortem Report
This project is a learning exercise, and one of the major ways
we learn is by analyzing past mistakes. You will, as a team,
review all aspects of this project. One of you will then
summarize that process into a post-mortem analysis report.
2.7.1 Deliverables
A report, summarizing the key issues raised in your post-mortem,
and the conclusions you came to. Your post-mortem discussion
should include:
- the implementation of your component design and the resulting code quality
- the implementation of your test plan, the problems found, and the confidence gained
- the relative efficacy of the competing development processes
- the integration of your components together
- the preparation of the release review/demo
- the overall project as an educational exercise
2.7.2 Grading
This report be graded on the basis of:
- 50% whether or not you meaningfully discuss each of the required activities.
- 20% whether or not you identify all of the important incidents.
- 30% the extent to which you are able to derive and articulate useful lessons
(and good future advice) from those experiences.
2.7.3 Submission
Upload the notes from your post-mortem discussion to github,
fill out the following form and e-mail it to the submission alias.
Team: name-of-your-team
Members: person1, person2, person3, person4
Project: 4F - Post-Mortem
Report: https://github.com/your_repo/...
primary editor: person2
if you have other files to submit, please include a brief
description of each.
Late points:
if this submission is late, who is using up how many late points
3. Rules
3.1 Due Dates and Slip Days
Any assignment that is not turned in by its due date,
will have its grade reduced by 10% (of its nominal value) for each
late day. I understand that problems happen, and so each student is
granted three slip days. A team of four students would have
twelve slip days between them.
One slip-day will excuse one deliverable
being late by one day.
If a design is one day late, and that delays the
design review by a day, then you may find that you need two slip-days.
Don't make the mistake of using up your slip-days too soon, because
the pressures generally get tighter later in the course.
When a team needs to use slip-days, they need to decide who's
days they are using. However you decide is OK with me, but
we will be tracking the slip days per student.
If will be using any slip days on a deliverable, tell me in
advance that you will be doing so (so that I don't nag you
about whether or not I lost your submission message).
3.1 Primary Authorship
While many activities (e.g. Post Mortem review) are fundamentally team
activities, each major work product will typically have a primary author:
one person who works up the basic organization, pulls together the contribution
from the other team members and writes most of the prose.
The ability to organize large collections of information into a coherent
narrative is an important skill, and each team members should take primary
authorship for multiple work products. The quality portion of the individual
contribution grade is based on the work products for which that person was
the primary author (presided over a process, or wrote at least 2/3 of
the text in a document).
Each work product submission must include an indication of the
primary author (or authors if no individual produced 2/3 of the text).
Note, however, that since the whole team will be graded
on the quality of each work product, it is only prudent
for the entire team to review any high-value work product
before it is submitted.
I will keep these as confidential, and average them together to get a sense
of the individual contributions to each team's efforts.
3.2 Collaboration and Citation
This is a team project, but different individuals will have primary
responsibility for different processes or work products (or different
parts of a single work product or process). Each team will be working
on a different type of product. You are free to talk your team members
(and for that matter other teams) about the processes you are following.
You may review your work products with your own team members, and revise
them based on their feedback ... but ...
- If you are the primary author of a work-product, you must
cite the source of text you did not write (if it is more
than a sentence), or any information that did not originate
within your team or from your interviews.
You will be doing research to develop your product definition and
requirements. I expect that many of the ideas for your product
will come from this research. Cite all of your sources for each
work product, and explain how each source contributed to your work
products.
- You may not share any of your work products (other than as required
for reviews) with members of other teams.