Software process models

A software process is the way a team turns ideas into working software: which activities happen, in what order, who does them and how the team knows when something is done. Every team has a process, even if nobody wrote it down; the question is whether it suits the project. A team building flight-control software for an aircraft, where every requirement must be traced to a test and certified by a regulator, needs a different process from a start-up trying to discover whether anyone wants its app. This lesson surveys the main families of process models, from plan-driven waterfall and the V-model through iterative and spiral models to agile methods such as Scrum, Kanban and Extreme Programming, and finally DevOps, which extends the process through operations. It closes with how to choose and tailor a process. CMPUT-401 later puts these ideas into practice with real clients; ROB-499 used a hybrid process for a robotics capstone.

Plan-driven development: waterfall and the V-model

The waterfall model arranges the life-cycle activities in sequence: requirements, design, implementation, testing, deployment and maintenance, each phase finishing and producing documents before the next begins. It is usually attributed to a 1970 paper by Winston Royce, which ironically presented the strict sequential version as risky and recommended iteration and early prototyping. Waterfall became standard in government and defence contracting because it fits fixed-price contracts, milestone payments and formal approvals: each phase ends with a document that can be reviewed and signed.

Waterfall works best when requirements are well understood and stable, the technology is familiar, and changes are expensive or regulated. Its weakness is that working software appears only near the end, so misunderstandings about requirements surface late, when they are most expensive to fix. Integration of separately built parts also happens late, which is exactly what went wrong in several of the failures described in lesson 1.

The V-model is a variant that pairs each development phase with a testing phase. Requirements analysis is paired with acceptance testing, system design with system testing, architectural design with integration testing, and detailed design with unit testing. Drawn on paper, the phases go down the left side of a V to coding at the bottom and back up the right side. Its contribution is to plan each level of testing when the corresponding specification is written, which forces testable specifications. The V-model remains common in safety-critical industries such as automotive, medical devices and rail, where standards require traceability between requirements and tests.

Iterative, incremental and spiral models

Iterative development repeats the life-cycle activities in short cycles, refining the product each time. Incremental development delivers the product in pieces, each adding usable functionality. Most modern processes are both: each iteration of a few weeks produces a working, tested increment that users can see. The benefits are early feedback, early integration and the ability to change direction when the team learns something. The cost is that the architecture must accommodate growth, and management must accept that the full scope is not fixed at the start.

Barry Boehm's spiral model of 1986 organized iterations around risk. Each turn of the spiral has four parts: determine objectives and constraints, identify and resolve the biggest risks (often by building a prototype), develop and test the next level of the product, and plan the next turn. A project with an uncertain technical risk might spend its first turn building a throwaway prototype of the risky part. The spiral model's lasting idea is that the riskiest things should be tackled first, an idea every process can borrow. The Rational Unified Process, popular in the late 1990s and 2000s, packaged iterative development with phases called inception, elaboration, construction and transition.

The agile movement

In February 2001 seventeen practitioners met at a ski resort in Utah and wrote the Manifesto for Agile Software Development. It states four value preferences: individuals and interactions over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; and responding to change over following a plan. The manifesto adds that while there is value in the items on the right, the authors value the items on the left more. Twelve principles accompany the values, including delivering working software frequently, welcoming changing requirements even late in development, having business people and developers work together daily, measuring progress primarily by working software, maintaining a sustainable pace, and reflecting regularly on how to become more effective.

Agile is a set of values, not a single method. It is often misread as no documentation, no planning or no architecture. The manifesto says none of these things: agile teams plan constantly, at several horizons, and write the documentation that is worth its cost. What agile rejects is planning everything up front and then treating deviation from the plan as failure.

Scrum

Scrum is the most widely used agile framework. Its rules are defined in the Scrum Guide by Ken Schwaber and Jeff Sutherland, last revised in 2020. Scrum organizes work into sprints, fixed time boxes of one month or less, commonly two weeks, each producing a usable increment. It defines three accountabilities: the product owner, who maximizes the value of the product and owns the product backlog; the developers, who do the work of creating each increment; and the Scrum master, who helps the team and organization understand and apply Scrum. A Scrum team is small, typically ten or fewer people.

Scrum has five events. Sprint planning decides what can be delivered in the sprint and how. The daily scrum is a fifteen-minute meeting where developers inspect progress toward the sprint goal and adapt their plan. The sprint review shows the increment to stakeholders and adapts the backlog. The sprint retrospective looks at how the team worked and chooses improvements. The sprint itself contains all the other events. Scrum also has three artifacts, each with a commitment: the product backlog with its product goal, the sprint backlog with its sprint goal, and the increment, which must meet the team's definition of done.

The definition of done is easy to overlook and crucial. It is a shared checklist that every increment must meet, for example: code reviewed, unit tests written and passing, integrated on the main branch, acceptance criteria met, documentation updated, deployed to the test environment. Without it, 'done' means different things to different people, and unfinished work piles up hidden behind items marked complete.

Teams track progress within a sprint with a burndown chart, which plots the remaining work against time. A healthy burndown slopes down to zero by the end of the sprint; a flat line in the middle signals that work is blocked or that items are too large to finish piece by piece.

# Sprint burndown: remaining story points at the end of each day of a 10-day sprint
committed = 34
completed_each_day = [0, 3, 5, 0, 0, 8, 3, 5, 2, 5]   # points finished per day

remaining = committed
ideal_step = committed / len(completed_each_day)
print("day  remaining  ideal")
for day, done in enumerate(completed_each_day, start=1):
    remaining -= done
    ideal = committed - ideal_step * day
    flag = "  <- behind" if remaining > ideal + 5 else ""
    print(f"{day:3d}  {remaining:9d}  {ideal:5.1f}{flag}")
print("finished" if remaining == 0 else f"{remaining} points carried over")

day  remaining  ideal
  1         34   30.6
  2         31   27.2
  3         26   23.8
  4         26   20.4  <- behind
  5         26   17.0  <- behind
  6         18   13.6
  7         15   10.2
  8         10    6.8
  9          8    3.4
 10          3    0.0
3 points carried over

Kanban and flow

Kanban, adapted for software from lean manufacturing by David J. Anderson and others in the mid-2000s, does not use time-boxed iterations. Work items flow across a board with columns for each state, such as to do, in progress, review and done. Its key practice is to limit work in progress (WIP): each column has a maximum number of items, and when a column is full, people help finish items downstream rather than starting new ones. Limiting WIP reduces multitasking and exposes bottlenecks, because work visibly piles up in front of the slowest step.

Kanban rests on a simple but powerful result from queueing theory, Little's law. For a stable system, the average number of items in progress equals the average arrival rate times the average time each item spends in the system:

If a team finishes 5 items a week and has 20 items in progress, each item takes 4 weeks on average from start to finish. Cutting work in progress to 10 halves the lead time without anyone working faster. BUS-230 used the same law for manufacturing and service operations; here it explains why teams that start fewer things finish each one sooner.

# Little's law: lead time = WIP / throughput
throughput = 5.0                     # items finished per week
for wip in (20, 15, 10, 5):
    print(f"WIP {wip:2d} -> average lead time {wip / throughput:.1f} weeks")

# Measuring it from a board history: (item, started_week, finished_week)
history = [("A", 0, 2), ("B", 0, 3), ("C", 1, 3), ("D", 1, 5), ("E", 2, 4),
           ("F", 2, 6), ("G", 3, 5), ("H", 4, 6)]
lead = [f - s for _, s, f in history]
weeks = max(f for _, _, f in history)
print("average lead time:", sum(lead) / len(lead), "weeks")
print("throughput:", round(len(history) / weeks, 2), "items per week")
avg_wip = sum(sum(1 for _, s, f in history if s <= w < f) for w in range(weeks)) / weeks
print("average WIP:", round(avg_wip, 2), "= throughput x lead time =",
      round(len(history) / weeks * sum(lead) / len(lead), 2))

WIP 20 -> average lead time 4.0 weeks
WIP 15 -> average lead time 3.0 weeks
WIP 10 -> average lead time 2.0 weeks
WIP  5 -> average lead time 1.0 weeks
average lead time: 2.625 weeks
throughput: 1.33 items per week
average WIP: 3.5 = throughput x lead time = 3.5

Extreme Programming and lean ideas

Extreme Programming (XP), developed by Kent Beck in the late 1990s, contributed many of the engineering practices that agile teams now use regardless of framework: test-driven development, pair programming, continuous integration, small releases, refactoring, collective code ownership, coding standards and a sustainable pace. Lessons 7, 9 and 11 cover several of these in depth. Lean software development, adapted by Mary and Tom Poppendieck from the Toyota Production System, contributed the idea of eliminating waste: partially done work, extra features, relearning, handoffs, task switching, delays and defects. A useful lean exercise is value stream mapping, which follows one feature from request to release and measures how much of the elapsed time anyone was actually working on it. The answer is often a small fraction; the rest is waiting.

DevOps

Traditional organizations separated development teams, who built software, from operations teams, who ran it in production. The split created conflicting incentives: developers were rewarded for change, operations for stability, and releases became rare, large and risky. DevOps, a movement that took shape around 2009, joins the two through shared responsibility, automation and fast feedback. Its practices include continuous integration and delivery, infrastructure as code, automated monitoring, and teams that build and run their own services ('you build it, you run it'). The DORA research program, now part of Google Cloud, has studied thousands of teams and found that frequent, small, automated releases are associated with both higher speed and higher stability, contradicting the old assumption that speed and stability trade off. Lesson 11 covers DevOps and its metrics in depth.

Choosing and tailoring a process

Barry Boehm and Richard Turner suggested judging a project along five dimensions: size (number of people), criticality (what a failure costs, from comfort to lives), dynamism (how fast requirements change), personnel (the skill of the team), and culture (whether people thrive on order or on freedom). Small, low-criticality, fast-changing projects with skilled people suit agile methods; large, safety-critical, stable projects suit plan-driven methods. Most real projects fall in between and combine elements: a safety-critical project may still build in increments, and an agile team may still produce a formal architecture document for a regulator.

Tailoring means keeping the parts of a process that address your project's risks and dropping ceremony that does not. Two warnings apply. First, cargo-cult agile, where a team holds daily stand-ups and calls its phases sprints but still plans everything up front and releases once a year, gets the meetings without the benefits. Second, scaling frameworks such as SAFe (Scaled Agile Framework), LeSS and Disciplined Agile exist for coordinating many teams, but they add coordination overhead of their own; it is usually better to reduce dependencies between teams through architecture than to manage them through process.

Process improvement

A process is never finished. Scrum's retrospective builds improvement into every sprint; Kanban teams review their flow metrics; DevOps teams hold blameless post-incident reviews. At the organizational level, models such as CMMI (Capability Maturity Model Integration), originally developed at Carnegie Mellon's Software Engineering Institute for the US defence sector, describe levels of process maturity, and ISO/IEC/IEEE 12207 defines software life-cycle processes for contracts and audits. These frameworks matter mainly in government and regulated industries; for a single team, the habit of regularly asking what slowed us down and what we will try next matters more than any certificate.