Writing tests with Copilot

The first session of the second week, on Monday, August 2, was about the use of Copilot that Priya valued most: writing tests. TrailCheck had grown faster than its tests, and the team knew several modules had little coverage. "Tests are where Copilot pays for itself," Priya said, "as long as you remember who decides what correct means. It can write a hundred assertions in a minute. It cannot know which of them describe what we actually want." This lesson covers asking for unit tests, finding edge cases, writing good assertions, integration tests, generating sample data, and the checks that keep generated tests honest.

Why tests suit an assistant

Tests have a regular structure: set up some input, call the code, check the result. That structure is easy for a model to continue once it has seen one example, which is why the GH-300 study guide lists generating unit and integration tests, identifying edge cases and writing assertions as core productivity skills. Tests are also low-risk to generate, because a wrong test usually fails loudly or tests too little, rather than breaking production. And tests are a form of documentation: a clear test file shows a newcomer how a function is meant to behave.

The risk is subtle. A model asked to "write tests for this function" reads the function and writes tests that the current code passes. If the code has a bug, the tests may enshrine the bug as expected behavior. Priya called this "testing the code instead of testing the requirement", and the habits in this lesson are designed to avoid it.

Asking for unit tests

There are several ways to ask. In VS Code and Visual Studio, the /tests slash command drafts tests for selected code. You can also ask in plain language, run the team's write-tests prompt file from lesson 3, or write the first test yourself and let completions suggest the rest. Whatever the route, the prompt should say which function, which testing framework, where the tests go, and what kinds of cases to include. Tidewater's instructions file already says pytest with plain asserts, so Dmitri's prompt could be short.

#file:trails/conditions.py #file:docs/report-format.md
Write pytest tests for parse_condition in trails/conditions.py.
Base the expected results on docs/report-format.md, not on the current code.
Include: two normal lines, a line with extra spaces, an upper-case condition,
a missing field, an empty string and a date in the future.
Put them in tests/test_conditions.py. Do not change parse_condition.

The line "base the expected results on the format note, not on the current code" is the important one. It tells the model where correctness comes from. Without it, the model infers the expected values from the implementation. With it, a test may fail, and a failing test written from the specification is exactly how you find a bug.

Edge cases

Copilot is good at suggesting edge cases, the unusual inputs where bugs hide. Ask for them explicitly: "List the edge cases for this function before writing tests" produces a list you can review and extend. Typical edge cases for a parser include empty input, extra or missing separators, surrounding spaces, unexpected letter case, very long values, characters from other languages, and values at the boundaries of what is allowed. Domain knowledge adds cases the model cannot know: TrailCheck reports sometimes arrive with a trail name in French, because some trails in the guidebooks have official names in both languages.

The empty-string case is a good example of why expected values must be checked. Splitting an empty string on the vertical bar gives one empty field, not zero, so the error message says "got 1". A model might guess "got 0". Running the case settles it.

The tests Dmitri kept

Copilot produced eight tests. Dmitri read each one against the format note, corrected one expected value, added the accented-name case and ran them. One test failed: the parser accepted a date in the future. That was a real bug, found because the tests were written from the requirement. The example below shows a cut-down version of the function after the fix, with the tests written as plain asserts so they run here without pytest.

from datetime import date

def parse_condition(line: str, today: date) -> dict:
    """Parse a report line date|trail|condition into a dict, or return an error dict."""
    parts = [p.strip() for p in line.split("|")]
    if len(parts) != 3:
        return {"error": f"expected 3 fields, got {len(parts)}"}
    when, trail, condition = parts
    try:
        day = date.fromisoformat(when)
    except ValueError:
        return {"error": "bad date"}
    if day > today:
        return {"error": "date is in the future"}
    return {"date": when, "trail": trail, "condition": condition.lower()}

today = date(2027, 8, 2)
cases = [
    ("2027-07-20|Headland Loop Trail|muddy", {"date": "2027-07-20", "trail": "Headland Loop Trail", "condition": "muddy"}),
    (" 2027-07-20 | Headland Loop Trail | MUDDY ", {"date": "2027-07-20", "trail": "Headland Loop Trail", "condition": "muddy"}),
    ("2027-07-20|Headland Loop Trail", {"error": "expected 3 fields, got 2"}),
    ("", {"error": "expected 3 fields, got 1"}),
    ("2099-01-01|Headland Loop Trail|dry", {"error": "date is in the future"}),
    ("2027-07-20|Sentier de la Pointe|sec", {"date": "2027-07-20", "trail": "Sentier de la Pointe", "condition": "sec"}),
]
passed = 0
for line, expected in cases:
    result = parse_condition(line, today)
    assert result == expected, (line, result)
    passed += 1
print(passed, "of", len(cases), "tests passed")

6 of 6 tests passed

The function takes today's date as a parameter rather than reading the clock. Copilot suggested that change when Dmitri asked how to test the future-date rule, and it is a good habit: code that reads the clock directly is hard to test, because the right answer changes every day.

Writing good assertions

An assertion is the line that checks a result. Weak assertions are a common problem with generated tests: a test that only checks that the result "is not None" or "is a dict" passes even when the content is wrong. Good assertions check the exact value that matters. Priya's rule was that each test should fail if the behavior it describes broke. A quick way to check is mutation by hand: change one thing in the code, such as removing the lower() call, and see whether any test fails. If none does, the assertions are too weak.

Writing the test first

A different way to work turns the usual order around. In test-first prompting you write one or two tests yourself, describing the behavior you want, and then ask Copilot to write code that makes them pass. The tests become a precise specification: they say exactly what the function receives and returns, including the error format, so the model has very little room to guess. Dmitri tried it for a new function, trail_status, which had to turn a list of recent reports into a single word for the trail page. He wrote three tests in about five minutes: mostly dry reports give "open", any report of a washout gives "closed", and no reports in the last week gives "unknown". Copilot's first implementation passed two of the three. The failing test showed it treated a week-old report as recent, an off-by-one error in the date comparison that he would probably have missed in review. Test-first prompting works best for small functions with clear rules; for exploratory work, where you are not yet sure what the behavior should be, asking for a draft and then writing tests from the agreed requirement is often quicker.

Integration tests

Unit tests check one function in isolation. Integration tests check that pieces work together, for example that uploading a report through TrailCheck's web form stores it and shows it on the trail page. Copilot can draft these too, but they need more context: how the app starts, which test client to use, how test data is set up and cleaned up. Lena asked chat to "write an integration test that posts a report to /reports and checks it appears on /trails/headland-loop", referencing the app module and an existing integration test. The draft used the right test client because the existing test was in context, which is few-shot prompting again.

Integration tests are also where generated code most often invents things: a fixture that does not exist, a route with the wrong name, a helper the project never had. Running the test immediately shows these problems. The habit is the same as for any suggestion: read it, run it, fix it.

Generating sample data

Tests need data, and real data is often off limits. The volunteer list, for example, contains names, phone numbers and addresses that must never go into prompts, code or tests. Copilot is good at generating synthetic data that has the same shape: invented names, made-up phone numbers in the right format, trail names that look plausible. The GH-300 guide lists generating sample data as a productivity skill. The prompt should describe the format and say explicitly that the data must be invented.

Generate 10 invented rows for tests/data/volunteers_sample.csv with columns
name,phone,trail,start_date. Use clearly fictional names, phone numbers in the
format 555-0100 to 555-0199, trails from data/trails.csv and dates in 2027.
Do not use any real person's details.

The 555-01xx range is reserved for fictional use in the United States and Canada, so test data in that range cannot reach a real person by accident.

Keeping generated tests honest

Priya ended with the checks she wanted on every pull request that added generated tests. They take a few minutes and prevent the most common failures.