Capstone: rescuing an untested project · PYTH-230 Lesson 12

Every business has one: a script that someone wrote in a hurry years ago, that everyone depends on, and that nobody dares to touch. At Ridgeline it is ship_report.py. Each afternoon Theo runs it on the day's online orders, and it tells him which orders ship on which day and how many boxes to pack. It has no tests, no comments and one-letter variable names. In December, with Christmas orders tripling, it started giving answers Theo did not trust: an order placed on Friday afternoon appeared under Saturday, when the courier does not collect, and a 13-bag order was given two boxes, though a box holds only six bags.

Rosa asked Jordan and Sam to make the script reliable before the Christmas rush, without stopping Theo from using it. This capstone follows their work from start to finish and uses every technique from the course: characterization tests to pin down current behaviour, refactoring in small steps, unit tests at the boundaries, test doubles and injected dependencies, debugging, logging, type hints, linting and coverage. As you read, notice the order of the steps: understanding and protection first, changes second.

Step 1: read it and run it

The first rule of rescuing legacy code is to understand before changing. Jordan and Theo sat together and Theo explained the business rules the script was meant to follow. Orders placed before 2 pm on a weekday ship the same day. Orders at or after 2 pm ship the next business day. The courier does not collect on Saturday or Sunday, so anything that would ship on a weekend ships on Monday. A box holds up to six bags, so a 7-bag order needs two boxes and a 13-bag order needs three. Rows that cannot be read should be reported, not silently dropped.

Then they ran the script on a sample of real orders, with the customer details removed. Here is the script as they found it, run on that sample. The ORDERS text stands in for the CSV file Theo exports each day.

ship_report.py as found, run on six sample orders

ORDERS = """order,placed,bags
3121,2026-12-03 09:15,2
3122,2026-12-03 14:00,7
3123,2026-12-04 16:40,6
3124,2026-12-05 10:05,1
3125,2026-12-04 11:30,two
3126,2026-12-04 13:59,13
"""
from datetime import datetime, timedelta

def ship_report(text):
    out = {}
    for line in text.strip().splitlines()[1:]:
        o, p, b = line.split(",")
        try:
            b = int(b)
        except:
            continue
        d = datetime.strptime(p, "%Y-%m-%d %H:%M")
        if d.hour > 14:
            d = d + timedelta(days=1)
        if d.weekday() == 6:
            d = d + timedelta(days=1)
        day = d.strftime("%a %d %b")
        boxes = round(b / 6)
        out.setdefault(day, []).append((o, b, boxes))
    res = ""
    for day in sorted(out):
        res += day + "\n"
        for o, b, x in out[day]:
            res += "  " + o + " " + str(b) + " bags " + str(x) + " box(es)\n"
    return res

print(ship_report(ORDERS))

Output

Fri 04 Dec
  3126 13 bags 2 box(es)
Sat 05 Dec
  3123 6 bags 1 box(es)
  3124 1 bags 0 box(es)
Thu 03 Dec
  3121 2 bags 0 box(es)
  3122 7 bags 1 box(es)

Comparing the output with Theo's rules shows several problems at once. Order 3122, placed at exactly 14:00, is listed for Thursday, but the rule says 2 pm or later ships the next day: the script tests hour > 14, so anything from 14:00 to 14:59 slips through. Order 3123, placed Friday at 16:40, moves to Saturday, but the courier does not collect then; the script only handles Sunday. Order 3124, placed on Saturday morning, stays on Saturday. Order 3126 is 13 bags in two boxes, because round(13 / 6) is 2. And order 3125, with two written as a word, has silently vanished. The days are also sorted alphabetically, not by date.

It is tempting to start fixing immediately. Jordan resisted. The script is used every day, and a fix that breaks something else in the week before Christmas would be worse than the current bugs. The bugs were written down as issues #47 to #51, and the next step was protection.

Step 2: characterization tests

Before changing anything, Jordan pinned down what the script does now, bugs included, with characterization tests. The output for the sample, and for a few extra cases, went into tests exactly as the script produced it. These tests are not a statement that the output is right; they are an alarm that rings if any change alters the output unintentionally.

Characterization tests pin the current output before any change

import unittest

def report(*cases):
    """Run TestCase classes and print one line per test (a small test runner)."""
    loader = unittest.TestLoader()
    tests = [t for case in cases for t in loader.loadTestsFromTestCase(case)]
    result = unittest.TestResult()
    unittest.TestSuite(tests).run(result)
    problems = {}
    for kind, items in (("FAIL", result.failures), ("ERROR", result.errors)):
        for test, tb in items:
            problems.setdefault(test.id().split(" ")[0], []).append((kind, tb.strip().splitlines()[-1]))
    for t in tests:
        found = problems.get(t.id(), [])
        print(f"{t.id().rsplit('.', 1)[-1]:<34} {found[0][0] if found else 'ok'}")
        for kind, line in found:
            print("    " + line)
    print(f"{len(tests)} tests: {len(tests) - len(problems)} passed, {len(problems)} failed")
ORDERS = """order,placed,bags
3121,2026-12-03 09:15,2
3122,2026-12-03 14:00,7
3123,2026-12-04 16:40,6
3124,2026-12-05 10:05,1
3125,2026-12-04 11:30,two
3126,2026-12-04 13:59,13
"""
from datetime import datetime, timedelta

def ship_report(text):
    out = {}
    for line in text.strip().splitlines()[1:]:
        o, p, b = line.split(",")
        try:
            b = int(b)
        except:
            continue
        d = datetime.strptime(p, "%Y-%m-%d %H:%M")
        if d.hour > 14:
            d = d + timedelta(days=1)
        if d.weekday() == 6:
            d = d + timedelta(days=1)
        day = d.strftime("%a %d %b")
        boxes = round(b / 6)
        out.setdefault(day, []).append((o, b, boxes))
    res = ""
    for day in sorted(out):
        res += day + "\n"
        for o, b, x in out[day]:
            res += "  " + o + " " + str(b) + " bags " + str(x) + " box(es)\n"
    return res

class TestShipReportAsFound(unittest.TestCase):
    """Characterization: what ship_report does today, bugs and all (see issues #47-#51)."""
    def test_sample_orders(self):
        self.assertEqual(ship_report(ORDERS),
            "Fri 04 Dec\n  3126 13 bags 2 box(es)\n"
            "Sat 05 Dec\n  3123 6 bags 1 box(es)\n  3124 1 bags 0 box(es)\n"
            "Thu 03 Dec\n  3121 2 bags 0 box(es)\n  3122 7 bags 1 box(es)\n")

    def test_header_only(self):
        self.assertEqual(ship_report("order,placed,bags\n"), "")

report(TestShipReportAsFound)

Output

test_header_only                   ok
test_sample_orders                 ok
2 tests: 2 passed, 0 failed

Writing them out exposed two more bugs that had been hiding in plain sight: a 1-bag order and a 2-bag order are each given zero boxes, because round rounds 1/6 and 2/6 down to 0. Theo had been packing them correctly by eye and never noticed the report. That went in as issue #52.

Step 3: refactor for testability

The script was one function that parsed text, applied the shipping rules, counted boxes and formatted a report, all mixed together. Nothing could be tested on its own. With the characterization tests as a safety net, Jordan split it in small steps, running the tests after each one: extract parse_rows, which turns the text into records; extract ship_day, which applies the shipping rule to one timestamp; extract boxes_needed; extract format_report. Each step kept the behaviour identical, bugs included, and the characterization tests stayed green throughout. Only the structure changed.

With the rules in their own small functions, they could finally be tested directly, one rule at a time, at their boundaries. That is where the bugs became impossible to ignore, and the next step dealt with them.

Before and after. Before: one box labelled ship_report(text) containing parsing, the shipping rule, box counting and formatting tangled together, with no tests. After: four boxes in a pipeline, parse_rows, ship_day, boxes_needed and format_report, each with its own small set of unit tests beside it, and a characterization test wrapped around the whole pipeline. A logger feeds from parse_rows to a log of skipped rows. — From one tangled function to four small, separately tested pieces

Step 4: fix the bugs, one at a time, test first

Each bug was now a behaviour change, so each got the full treatment from lesson 2: write a failing unit test from Theo's rule, watch it fail, fix the code, watch it pass, update the characterization test in the same commit with a note, and commit. Five small commits, each easy to review and easy to undo. The cut-off rule became a test that 13:59 ships the same day and 14:00 the next; the weekend rule became tests for Friday afternoon, Saturday and Sunday; the box rule became a table from 0 to 13 bags.

Two design changes made the new code easier to test. First, the functions take plain values rather than reading files or the clock themselves: parse_rows takes text, and a main function at the bottom does the file reading. Second, unreadable rows are logged through a logger passed in, rather than silently skipped, so a test can check that the warning appears and Theo can see it in the afternoon log.

The rescued script

Here is ship_report after the rescue: four small typed functions, a constant for every business number, logging for skipped rows, the business rules written as docstrings, and a test suite. Run it to see the corrected report and the tests.

The rescued ship_report: small typed functions, constants, logging and a test suite

import unittest

def report(*cases):
    """Run TestCase classes and print one line per test (a small test runner)."""
    loader = unittest.TestLoader()
    tests = [t for case in cases for t in loader.loadTestsFromTestCase(case)]
    result = unittest.TestResult()
    unittest.TestSuite(tests).run(result)
    problems = {}
    for kind, items in (("FAIL", result.failures), ("ERROR", result.errors)):
        for test, tb in items:
            problems.setdefault(test.id().split(" ")[0], []).append((kind, tb.strip().splitlines()[-1]))
    for t in tests:
        found = problems.get(t.id(), [])
        print(f"{t.id().rsplit('.', 1)[-1]:<34} {found[0][0] if found else 'ok'}")
        for kind, line in found:
            print("    " + line)
    print(f"{len(tests)} tests: {len(tests) - len(problems)} passed, {len(problems)} failed")
ORDERS = """order,placed,bags
3121,2026-12-03 09:15,2
3122,2026-12-03 14:00,7
3123,2026-12-04 16:40,6
3124,2026-12-05 10:05,1
3125,2026-12-04 11:30,two
3126,2026-12-04 13:59,13
"""
import logging, math, sys
from datetime import date, datetime, timedelta
from typing import NamedTuple

CUTOFF_HOUR = 14          # orders at or after 2 pm ship the next business day
BAGS_PER_BOX = 6
log = logging.getLogger("ridgeline.shipping")

class Order(NamedTuple):
    order: str
    placed: datetime
    bags: int

def parse_rows(text: str, log: logging.Logger = log) -> list[Order]:
    """Read 'order,placed,bags' rows; log and skip rows that cannot be read."""
    orders = []
    for number, line in enumerate(text.strip().splitlines()[1:], start=2):
        try:
            order, placed, bags = line.split(",")
            orders.append(Order(order, datetime.strptime(placed, "%Y-%m-%d %H:%M"), int(bags)))
        except ValueError:
            log.warning("row %d skipped: %r", number, line)
    return orders

def ship_day(placed: datetime) -> date:
    """Before the cut-off on a weekday: same day. Otherwise the next business day."""
    day = placed.date()
    if placed.hour >= CUTOFF_HOUR or day.weekday() >= 5:
        day += timedelta(days=1)
    while day.weekday() >= 5:            # Saturday or Sunday: move to Monday
        day += timedelta(days=1)
    return day

def boxes_needed(bags: int) -> int:
    """Up to six bags per box; a part-filled box still counts."""
    return math.ceil(bags / BAGS_PER_BOX)

def format_report(orders: list[Order]) -> str:
    by_day: dict[date, list[Order]] = {}
    for o in orders:
        by_day.setdefault(ship_day(o.placed), []).append(o)
    lines = []
    for day in sorted(by_day):
        lines.append(day.strftime("%a %d %b"))
        for o in by_day[day]:
            lines.append(f"  {o.order} {o.bags} bags {boxes_needed(o.bags)} box(es)")
    return "\n".join(lines) + ("\n" if lines else "")

class TestShipDay(unittest.TestCase):
    def test_before_cutoff_ships_same_day(self):
        self.assertEqual(ship_day(datetime(2026, 12, 3, 13, 59)), date(2026, 12, 3))

    def test_at_cutoff_ships_next_day(self):
        self.assertEqual(ship_day(datetime(2026, 12, 3, 14, 0)), date(2026, 12, 4))

    def test_friday_afternoon_ships_monday(self):
        self.assertEqual(ship_day(datetime(2026, 12, 4, 16, 40)), date(2026, 12, 7))

    def test_weekend_orders_ship_monday(self):
        for placed in (datetime(2026, 12, 5, 10, 5), datetime(2026, 12, 6, 9, 0)):
            with self.subTest(placed=placed):
                self.assertEqual(ship_day(placed), date(2026, 12, 7))

class TestBoxes(unittest.TestCase):
    def test_box_table(self):
        for bags, boxes in [(0, 0), (1, 1), (6, 1), (7, 2), (12, 2), (13, 3)]:
            with self.subTest(bags=bags):
                self.assertEqual(boxes_needed(bags), boxes)

class TestParseRows(unittest.TestCase):
    def test_unreadable_row_is_logged_and_skipped(self):
        with self.assertLogs("ridgeline.shipping", level="WARNING") as cm:
            orders = parse_rows(ORDERS)
        self.assertEqual([o.order for o in orders], ["3121", "3122", "3123", "3124", "3126"])
        self.assertEqual(len(cm.output), 1)
        self.assertIn("row 6 skipped", cm.output[0])

class TestReport(unittest.TestCase):
    def test_sample_orders(self):
        self.assertEqual(format_report(parse_rows(ORDERS, logging.getLogger("quiet"))),
            "Thu 03 Dec\n  3121 2 bags 1 box(es)\n"
            "Fri 04 Dec\n  3122 7 bags 2 box(es)\n  3126 13 bags 3 box(es)\n"
            "Mon 07 Dec\n  3123 6 bags 1 box(es)\n  3124 1 bags 1 box(es)\n")

logging.getLogger("quiet").addHandler(logging.NullHandler())
logging.getLogger("quiet").propagate = False
print(format_report(parse_rows(ORDERS, logging.getLogger("quiet"))))
report(TestShipDay, TestBoxes, TestParseRows, TestReport)

Output

Thu 03 Dec
  3121 2 bags 1 box(es)
Fri 04 Dec
  3122 7 bags 2 box(es)
  3126 13 bags 3 box(es)
Mon 07 Dec
  3123 6 bags 1 box(es)
  3124 1 bags 1 box(es)

test_at_cutoff_ships_next_day      ok
test_before_cutoff_ships_same_day  ok
test_friday_afternoon_ships_monday ok
test_weekend_orders_ship_monday    ok
test_box_table                     ok
test_unreadable_row_is_logged_and_skipped ok
test_sample_orders                 ok
7 tests: 7 passed, 0 failed

Compare the new report with the old one. Order 3122 now ships Friday, because 14:00 is at the cut-off. Orders 3123 and 3124 ship Monday. Every order has the right number of boxes. The days are in date order. And the unreadable row is reported in the log as row 6, so Theo can fix it in the export rather than lose an order.

The characterization test from step 2 is gone, replaced by TestReport, whose expected output was worked out by hand from Theo's rules, not copied from the program. That is the end of the journey for a characterization test: once the code is understood and tested properly, the pinned old behaviour is replaced by tests of the correct behaviour.

Step 5: the finishing checks

With the behaviour right, Jordan ran the rest of the toolkit. ruff check and ruff format found two long lines and an unused import left over from the refactoring. mypy, run with strict settings for the new module, found one problem: format_report originally had no annotation for by_day, and mypy could not infer the dictionary's value type from the empty literal; adding dict[date, list[Order]] fixed it. coverage run --branch showed every line and branch of the module covered except main, which only reads a file and calls the other functions, and is exercised by one end-to-end test in the integration suite.

Lint, format, type check and coverage on the rescued module

> ruff check src/ridgeline/shipping.py && ruff format src/ridgeline/shipping.py
All checks passed!
1 file left unchanged
> mypy --strict src/ridgeline/shipping.py
Success: no issues found in 1 source file
> coverage run --branch -m pytest tests/test_shipping.py -q
........                                                   [100%]
8 passed in 0.06s
> coverage report -m --include=src/ridgeline/shipping.py
Name                         Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------
src/ridgeline/shipping.py       44      3     14      0    95%   71-73
------------------------------------------------------------------------

The pull request contained eleven commits: two adding characterization tests, four refactoring steps, five bug fixes each with its test, and a final one adding logging, hints and the CI configuration for the new module. Sam reviewed it in an hour, which would have been impossible for one giant change. Theo used the new version the next afternoon, and the morning after that the log showed two rows skipped from the export, each with its row number, both fixed in minutes.

Looking back

The rescue followed a pattern that works for almost any untested code. Understand the rules with the people who own them. Protect the current behaviour with characterization tests. Refactor in small steps until the rules live in small functions with plain inputs. Test those functions at their boundaries, and let the tests reveal the bugs. Fix each bug as its own test-first change. Then finish with logging, hints, linting and coverage, and wire everything into continuous integration so the module never slides back.

None of the techniques is complicated on its own. The skill is in the order, and in the discipline of small, checked steps. That discipline is what lets a team change code that the business depends on, confidently and quickly, which is the real purpose of everything in this course.

Worked problem

Planning the next rescue

The loyalty export script that sends points to the marketing email service is next: 200 lines, no tests, and Mei suspects double-counted points after refunds. Write the plan.

Show solution

Meet Mei to write down the rules: points per dollar, refunds, tiers, export format.
Characterize: run the script on anonymized sample data with a fake email client, and pin the exported file.
Refactor: extract parsing, point calculation and the export call, with the email client injected; tests green after each step.
Unit-test the point rules at their boundaries, including refunds; fix the double counting test-first.
Add logging, hints, lint and coverage; add the module to CI; replace the characterization test with hand-checked expectations.

Answer: Rules, characterization, small refactorings, boundary tests, test-first fixes, then the finishing checks and CI: the same pattern as ship_report.

Try it yourself

The shipping day

Write ship_day(placed) for a datetime placed, returning a date. Orders before 14:00 on a weekday ship that day. Orders at or after 14:00 ship the next business day. Orders placed on Saturday or Sunday ship on Monday. The courier never collects on a weekend. (In December 2026, the 4th is a Friday.)

from datetime import date, datetime, timedelta

def ship_day(placed):
    day = placed.date()
    if placed.hour > 14:
        day += timedelta(days=1)
    if day.weekday() == 6:
        day += timedelta(days=1)
    return day

Try it yourself

Count the boxes

Write boxes_needed(bags) that returns how many boxes an order needs when a box holds up to BAGS_PER_BOX = 6 bags. A part-filled box still counts, and 0 bags need 0 boxes. Define the constant.

def boxes_needed(bags):
    return round(bags / 6)

Try it yourself

Read the rows, report the bad ones

Write parse_rows(text, log) for CSV text with a header line order,placed,bags. Return a list of (order, bags) tuples, with bags as an int, for every readable row. For a row whose bags are not a whole number or that does not have three fields, call log.warning with a message that includes the row's line number in the text (the header is line 1) and skip it.

def parse_rows(text, log):
    rows = []
    for line in text.strip().splitlines()[1:]:
        order, placed, bags = line.split(',')
        rows.append((order, int(bags)))
    return rows

Common mistakes

MistakeStarting to fix bugs in untested legacy code straight away.
InsteadUnderstand the rules, then protect current behaviour with characterization tests.
MistakeMixing refactoring and bug fixes in one big change.
InsteadRefactor with behaviour unchanged, then fix each bug in its own test-first commit.
MistakeKeeping characterization tests forever.
InsteadReplace them with hand-checked tests of the correct behaviour once the code is understood.
MistakeLetting functions read files and the clock themselves.
InsteadPass in text, dates and loggers; keep file reading in main.
MistakeSilently skipping rows that cannot be read.
InsteadLog each skipped row with its line number so someone can fix the data.
MistakeStopping once the tests pass.
InsteadFinish with linting, type checking, coverage and CI so the code stays healthy.

Summary and key terms

Rescuing untested code starts with understanding the rules with the people who own them.
Characterization tests protect current behaviour, bugs included, before any change.
Refactor in small checked steps until rules live in small functions with plain inputs.
Boundary tests on those functions reveal bugs; fix each as its own test-first change.
Inject dependencies such as text, dates and loggers; log skipped data instead of dropping it.
Finish with linting, type checks, branch coverage and CI.
Small, reviewable commits make even risky changes to important code safe and fast.

legacy code: Existing code, often without tests, that a business depends on.
characterization test: A test that pins down what existing code currently does.
cut-off time: The time after which an order is handled on the next business day.
business day: A day on which work is done, here Monday to Friday.
test-first fix: Writing a failing test that reproduces a bug before fixing it.
pull request: A proposed set of commits submitted for review before merging.
continuous integration: Automatically running checks and tests on every change to the shared repository.

Practice problems

Work each problem on paper first, then reveal the answer to check.

Why did Jordan write characterization tests before fixing the cut-off bug?
Show answer
To detect any unintended change in behaviour while refactoring and fixing.
Which line of the legacy script caused orders at exactly 14:00 to ship the same day?
Show answer
if d.hour > 14, which should be >= 14.
Why does round(13 / 6) give the wrong number of boxes?
Show answer
It rounds 2.17 down to 2; part-filled boxes need math.ceil, giving 3.
What was the benefit of passing a logger into parse_rows?
Show answer
Tests can check the warning, and Theo sees skipped rows in the log.
Why was the old characterization test removed at the end?
Show answer
It pinned the buggy behaviour; it was replaced by a test of the correct, hand-checked output.
Name the finishing checks run after the behaviour was right.
Show answer
Linting and formatting, strict type checking, branch coverage and CI.