Process, and Process Models are considered, by many people, to be among the most theoretical areas of software engineering. This view reminds me of the response an Irish musician once gave an American interviewer who asked if there was any Irish tradition of fight songs. His answer was:
In the readings on formal process models, and agile process we will see that there are a great many ways to structure the software development process. All models, however, seem to agree that the work can be divided into a few basic phases.
This includes activities like creating the initial concept, developing product descriptions, gathering requirements, and negotiating specifications.
This includes both technical planning (architecture, designs, modeling, prototypes) and managerial planning (people, budgets, schedules) and the validation of those plans.
This includes the coding of both the product and testing tools, unit testing, the development of content and documentation, integration, whole system testing, and acceptance testing.
This includes activities like release packaging and product manufacturing, early access programs, distribution, support, bug-fixing, patches, and on-going maintenance and enhancement.
Exactly what each one of these activities implies may be different for every project, but most of these activities are (in some form) applicable to most software projects. Where people differ is on how to structure these phases and activities into a software project.
There are a few basic models, into which the above phases are traditionally organized.
The first, and most primitive development model is the waterfall model so named because its diagram resembles a succession of cascading waterfalls, where each waterfall in the cascade feeds the pond that supplies the next. It is a simple and intuitive model, where each phase is completed (and verified) before the next phase is begun, and the whole sequence culminates in a finished product.
Experience has shown this model to be a simpler solution than most interesting software projects will admit of. We seldom know enough (about the problem to be solved or the obstacles we will encounter) to enable us to completely specify the solution up front. As a result, very few problems can actually be solved in a simple one-pass waterfall project.
If we cannot solve our problem in one-pass, we will have to solve it with a series of successive approximations. All of the other models represent different ways of structuring that succession.
Often, we have a grand vision, but we clearly understand how to realize only a small subset of it. If that initial subset would be a useful product in its own right, there is no reason for us to even try to create the whole grand vision in a single project.
Define, plan, build, and deploy the initial subset that we understand. In this process we will learn valuable lessons about both the problems to be solved, and how users use the software. Based on this new information, we can define and plan the next release, with additional features.
This model of successive releases is employed in most major software projects. Each successive release is much simpler than the grand vision, and we only build the parts for which we have a good understanding of the requirements and design. This approach has the potential to get us to our desired end point, in a series of smaller, simpler, safer steps.
It is important to recognize the difference between research and development:
The difference between Incremental and Iterative models is the difference between a cascading succession of development projects, and a cascading succession of research projects.
If our goal is to figure out what to build (or how to build it), we cannot be certain that, at the end of the next cycle of activities, we will have a product, or that we will even have a clear definition and plan for a product. We may have go through many iterations of definition, planning, construction, and evaluation before we are prepared to define and plan a product cycle. Because these models go through an unpredictable succession of development cycles, they are often referred to as spiral models).
The decision of whether to use a waterfall, incremental, or iterative model is not a matter of personal choice. It is a characterization of our goals and how confident we are of them:
The high level models describe a general sequencing of activities, and set expectations for what kind of a project we are undertaking. They do not actually tell us what we have to do. In section 2, I gave a sample list of activities for each phase, but for any given project, this list might have inappropriate or be missing important activities. Any one of those activities (e.g. gathering requirements) could be broken down into a more detailed list of tasks:
Many people consider these specifications to be very helpful, as they guide the project team from one step to the next, and clearly spell out what is expected of them at each point. Other people consider such specifications to be meddlesome micro-management, focusing on forms and process rather than working software. There is validity to both views. More enlightened processes ensure that the important things are covered while giving projects wide lattitude in the interpretation of requirements and means of demonstrating satisfaction.
It may be tempting to thing of check-lists as mnemonic aids for people who have not yet mastered the concepts. This would be a mistake. Jet Fighter pilots are some of the smartest, proudest, and highly trained people in the world, and they depend heavily on check-lists. They do not use check-lists because they haven't been adequately trained, or haven't yet memorized the procedures. They use check-lists because they perform complex tasks, where any mistake is likely to result in disaster. In times of emergency, when response time is most critical, they still adhere rigorously to their check-lists. Check-lists are for people who can't afford to make mistakes.
Most software process definitions, and most software engineering texts, contain numerous check-lists: things that you should make sure have been covered. There is considerably more variation among software projects than there is in flight operations, which means that these check-lists are not as comprehensive, and may contain steps that are not applicable to all projects:
The transition from one phase to another is often accompanied by a review and approval.
Some people use the term task to refer to something that has a clear beginning and end (e.g. reading this article), whereas an activity (e.g. learning software engineering principles) may continue indefinitely.
Any work-product definition must specify the required content and qualities, and many organizations specify standard forms for each standard work product. Many people believe that all specified work products should be inputs to subsequent process steps.
An understanding of the basic process models and taxonomy enables us to more intelligently plan and discuss projects. An understanding of the relationship between abstract software process models and real software processes keeps those discussions real (useful).