S/W Estimation Principles

version 1.0 of 09 Jan 2007
Mark Kampe

1. Introduction

One of the greatest books ever written on the subject of strategy begins:

The same thing can be said of sofware project estimation. Estimates are a pre-condition for any non-trivial software project, in that time and cost are key considerations in deciding whether or not to pursue a project. Good estimates lay the foundation for successful projects, by identifying the required resources and enabling us to predict how long it will take to develop the required functionality. Bad estimates can ensure the failure of a project ... if it cannot be built with the assigned resources, or cannot be delivered within the required time frame. If we are to be successful at software construction, we must develop skill at software project estimation. Unfortunately, as another great commentator on the human condition (Yogi Berra) once observed ...

Predictions are tough, particularly when they involve the future.

Project estimation involves many imponderables:

We aren't exactly sure what we are going to do, or how we are going to do it.
We don't know what problems will arrise.
We don't know what distractions we will have.

These simple facts alone would seem to preclude the possibility of accurate estimates. Because the preparation of estimates is so difficult, the results so disappointing, and the consequences so dire, estimation is an activity that most developers and managers avoid like the plague. While accurate predictions of the future are not generally possible, there are techniques that can help us to make resonable predictions about the work that will be required to build a piece of software. Because of the importance and difficulty of estimation, the study of these techniques is a subject of inquiry, which can on no account be neglected.

2. Principles of Estimation

There are numerous tools and techniques for estimating the size of a software task, and numerous books written about each. More important, however, are a few basic principles that should guide all estimation efforts.

2.1 Estimates are not Guesses

They are the result of an analytical process. The quality of the estimate is determined by the sophistication and dilligence with which the analysis is performed. Better (deeper and more methodical) analysis inevitably yields better estimates. Your skill in analyzing projects and preparing estimates will improve with experience and study.

Estimates are based on knowledge and data: an understanding of the project to be undertaken, and about previous experience with the same team, tools, and methology. The more complete your understanding of a project, the better will be your estimates. If you have access to data on the productivity of a team, on various problems, using various tools, this will enable you to make better predictions about their productivity on the next project. Data you collect on this project will help you with future projects.

2.2 Estimates are neither Precise or Deterministic

Because we do not have complete knowledge, it is not possible to accurately predict how long it will take to complete a project. Because of this uncertainty, an estimate is not a number, but rather a probability density function. The confidence estimates are every bit as important as the time estimates ... and so they must be included along with the estimate.

By the time resources are being committed, it may be important for estimates to be correct to within 5-10%.

If, in preparing an estimate, you find areas where uncertainty is unacceptably high, these are the areas on which you need to focus more effort and reduce your ignorance:

seek opinions from people with more experience in this area
do research on the problems and approaches
elaborate the design to gain a better understanding of the proposed solution
construct models and prototypes to explore potential problems and approaches.

Estimates should be regularly revisited and refined to monitor the progress of the project, and update our predictions of when we expect it to complete. As investigations, plans, and implementations proceed, unknowns will be resolved, probability functions will collapse, and the estimates should converge. If the estimates do not converge, but rather continue to move outwards, this is a warning sign that something is very wrong.

2.3 Estimate at the lowest possible level of detail

A single estimate for a 20 staff-year project could easily be high or low by a power of ten. An estimate for a 50 line routine will probably be accurate to within an hour:

smaller tasks are better defined and leave room for less doubt
smaller tasks are simpler and more easily envisioned
estimates for smaller tasks can be easily sub-contracted out to groups or individuals with expertise in that particular area.
the work to decompose a large project into smaller sub-tasks tends to expose tasks and problems that might not have been obvious when the projecct was being viewed at a higher level of abstraction.
the work to decompose a large project into smaller sub-tasks will tend to resolve many issues that were unclear when the project was being viewed at a higher level of abstraction

The result is that bottom-up estimates (summing the estimates for all of the sub-tasks) are inevitably more accurate than whole-project estimates. Not only are the individual sub-task estimates more accurate, but the enumeration of tasks to be performed is more complete and the errors in these smaller estimates tend to cancel out.

2.4 Get Multiple Independent Estimates

There are many approaches to estimating the size of a project:

Highly interactive projects can often be estimated based on the number of specified use cases.
Given an architecture, we can attempt to estimate the size of each component.
If we have worked on similar projects, we can attempt to assess the relative difficulty of this one, and to estimate the size delta relative to other benchmark projects.
Object oriented implementations can often be estimated based on the the number and richness of classes.
Given a component design, we can attempt to estimate the number of required lines of code or Function Points.

Different techniques are appropriate to different projects at different stages of specification ... but each of these techniques yields a completely independent assessment of how much work will be involved.

Different people have had different experiences. When they look at a project description, they will see different problems, and different approaches. Assessments from different sources can points, use cases, comparisons to past projects).

If 3-5 independent estimates cluster well, you can feel good about the estimate. If estimates are widely scattered there are two possible explainations:

the problem is not yet well enough understood or specified
some of the estimators or estimation techniques are ill suited to this problem.

Which ever the cause, a group discussion about the disputed estimates usually resolves the question quickly. If the diversity of opinion results from an inadequate understanding of the problem, the answer is additional research, design, or prototyping. If the diversity results from the application of an inappropriate estimation technique, find better ones.

2.5 Be Honest and Clear

Do the best analysis you can to develop your estimates. Projects that begin with estimates that are slanted to bolster some agena, or to tell the executives what they want to hear usually end badly. The best decisions are the ones that are based on the best possible data. A project that begins with a bad estimate is on the road to ruin.

Executives will want to know what the bottom line is. When can I have it and how much is it going to cost? Clearly state your assumptions, and make sure that you always give your times and costs within a confidence band. If they tell you that a confidence band is too wide, you need to be prepared to suggest investigations to reduce that uncertainty.

Management will almost certainly "push back" on your estimates, telling you that they are too high. Don't get defensive! Be prepared to explain the process you used to develop your estimates, the data sources on which you drew, the confirmation that you obtained, and how much confidence you have. Project size estimates can often be reduced by changing the project requirements, scope, design, methodology, or understanding of the problem. Making a number smaller simply because an executive thinks it is too large is self deception (and unlikely to end well).

3 Summary

Estimation is difficult, messy, and does not admit of closed form (or even correct) solutions. This does not, however, mean that estimation methodology is futile or that all estimates are equally bad. People who take project estimation seriously will find their efforts rewarded. During the age of exploration (when the proceeds of expeditions had to be divided among investors) anyone who could do long division found that his career was made. Today, when so much value is created by sucessful software projects, the same can be said about someone who can do project estimation.