JAVA-220 · Lesson 3 of 12 · 22 min

Assertions and parameterized tests

▶ Video coming soon A video version of this lesson is on the way.

By Thursday, September 23, Hamza had eleven tests for Money and one complaint. "Half of them are the same test with different numbers," he said, scrolling through formatsCents, formatsWholeDollars, formatsZero and their cousins. Ingrid agreed that was a smell, and also pointed at a test he had not written: what should happen when someone builds a booking with an end date before its start date? ShopBook throws an exception for that, and a test should prove it. Today's session covers JUnit's assertions beyond assertEquals, and parameterized tests, which run one test method over a whole table of inputs.

The common assertions

All of JUnit's assertions are static methods in org.junit.jupiter.api.Assertions. Every one of them has an optional last argument, a message, that is added to the failure report; use it when the assertion alone would not make the problem obvious. The message can also be a lambda that builds the text only if the assertion fails, which matters when building it is expensive.

JUnit Jupiter assertions
AssertionPasses whenTypical use
assertEquals(expected, actual)the two are equal by equals()Return values, formatted text, records
assertEquals(expected, actual, delta)two doubles differ by no more than deltaMeasurements and averages
assertTrue(condition) / assertFalse(condition)the condition is true / falseisLow(), isEmpty(), flags
assertNull(x) / assertNotNull(x)x is null / not nullLookups that may find nothing
assertSame(a, b)a and b are the same objectCaches and singletons
assertIterableEquals(expected, actual)two collections hold equal elements in orderLists and sorted results
assertThrows(Type.class, () -> ...)the code throws that exception typeRejecting bad input
assertAll(() -> ..., () -> ...)every grouped assertion passes; reports all failuresChecking several fields of one result
assertTimeout(Duration, () -> ...)the code finishes in timeGuarding against very slow code
fail("message")never; it always failsMarking a branch that should not be reached

Comparing doubles

Numbers stored as double are binary approximations, so arithmetic that is exact on paper often is not exact in the computer. That is one reason Second Gear's Money class keeps amounts in whole cents as a long. For measurements that really are doubles, such as the average days a rental bike is out or a wheel's circumference, compare with a tolerance: assertEquals with a third argument, delta, passes when the two values are within delta of each other.

Java Example · Main.java
public class Main {
    public static void main(String[] args) {
        double total = 0.1 + 0.2;
        System.out.println("0.1 + 0.2 = " + total);
        System.out.println("Exactly 0.3? " + (total == 0.3));
        System.out.println("Within 1e-9 of 0.3? " + (Math.abs(total - 0.3) <= 1e-9));

        double[] daysOut = {2.5, 3.0, 4.5};
        double average = (daysOut[0] + daysOut[1] + daysOut[2]) / daysOut.length;
        System.out.println("Average days out: " + average);
        System.out.println("Within 0.001 of 3.3333? " + (Math.abs(average - 3.3333) <= 0.001));
    }
}
Output
0.1 + 0.2 = 0.30000000000000004
Exactly 0.3? false
Within 1e-9 of 0.3? true
Average days out: 3.3333333333333335
Within 0.001 of 3.3333? true

What this code does

Adding 0.1 and 0.2 gives a value a tiny bit above 0.3, so an exact comparison with == says false even though the arithmetic is right. Comparing the difference with a small tolerance gives the answer a person expects, and that is exactly what assertEquals(expected, actual, delta) does. The average of three rental periods shows the usual case, where the expected value itself is rounded and only a tolerance makes sense.

Choose the tolerance from the meaning of the number, not from habit. For an average of days, a thousandth of a day is plenty; for a wheel circumference in millimetres, a tenth of a millimetre might be the right limit. Never use a tolerance for money: keep money in whole cents or BigDecimal, and compare exactly.

Testing that code throws

Good code rejects bad input loudly, usually by throwing an exception, and that behaviour deserves a test as much as the happy path. assertThrows takes the exception class you expect and a lambda containing the code that should throw. It passes only if the lambda throws that exception, or a subclass of it; if the code completes normally, or throws something else, the test fails. It also returns the exception, so the test can check its message.

Java Example · A JUnit test for a rejected rental (listing)
@Test
void rejectsEndBeforeStart() {
    LocalDate start = LocalDate.of(2027, 5, 10);
    LocalDate end = LocalDate.of(2027, 5, 8);
    IllegalArgumentException e = assertThrows(IllegalArgumentException.class,
            () -> new Rental("CITY-2", start, end));
    assertEquals("end date 2027-05-08 is before start date 2027-05-10", e.getMessage());
}

What this code does

The lambda builds a Rental whose end date is two days before its start, which ShopBook's Rental constructor must refuse. assertThrows runs the lambda, catches the IllegalArgumentException and hands it back, and the second assertion checks that the message names both dates. If a later change made the constructor accept the bad dates, this test would fail with a message saying no exception was thrown.

Java Example · Main.java: assertThrows by hand
import java.util.function.*;

public class Main {
    static <T extends Throwable> T assertThrows(Class<T> expected, Runnable code) {
        try {
            code.run();
        } catch (Throwable actual) {
            if (expected.isInstance(actual)) return expected.cast(actual);
            throw new AssertionError("expected " + expected.getSimpleName() + " but " + actual.getClass().getSimpleName() + " was thrown");
        }
        throw new AssertionError("expected " + expected.getSimpleName() + " to be thrown, but nothing was thrown");
    }

    static int parseCount(String text) {
        int n = Integer.parseInt(text.trim());
        if (n < 0) throw new IllegalArgumentException("count cannot be negative: " + n);
        return n;
    }

    static void report(String name, Runnable test) {
        try { test.run(); System.out.println("PASSED " + name); }
        catch (AssertionError e) { System.out.println("FAILED " + name + ": " + e.getMessage()); }
    }

    public static void main(String[] args) {
        report("negative count is rejected", () -> {
            IllegalArgumentException e = assertThrows(IllegalArgumentException.class, () -> parseCount("-3"));
            System.out.println("  message: " + e.getMessage());
        });
        report("text is rejected", () -> assertThrows(NumberFormatException.class, () -> parseCount("ten")));
        report("valid count is rejected (wrong expectation)", () -> assertThrows(IllegalArgumentException.class, () -> parseCount("12")));
    }
}
Output
  message: count cannot be negative: -3
PASSED negative count is rejected
PASSED text is rejected
FAILED valid count is rejected (wrong expectation): expected IllegalArgumentException to be thrown, but nothing was thrown

What this code does

assertThrows runs the code and returns the exception if it is the expected type, so the caller can inspect its message. If nothing is thrown, or a different exception appears, it throws an AssertionError that explains which. The third test expects an exception from a valid count, so it fails with the nothing-was-thrown message, which is how JUnit reports the same mistake.

One subtle point: NumberFormatException is a subclass of IllegalArgumentException, so assertThrows(IllegalArgumentException.class, ...) would also pass for parseCount("ten"). When the difference matters, JUnit has assertThrowsExactly, which accepts only the exact class.

Grouped assertions

When one result has several parts, such as a record with five fields, a run of separate assertEquals calls stops at the first wrong field, and you fix one field per test run. assertAll takes several lambdas, runs every one of them and then reports all the failures together. It is the right tool for checking one behaviour that has several observable parts.

Java Example · Main.java: grouped assertions
import java.util.*;

public class Main {
    record Line(String code, int quantity, long priceCents, long totalCents) { }

    static Line line(String code, int quantity, long priceCents) {
        return new Line(code, quantity, priceCents, priceCents * quantity + 1); // a deliberate bug
    }

    static void assertAll(String heading, Runnable... checks) {
        List<String> failures = new ArrayList<>();
        for (Runnable c : checks) {
            try { c.run(); } catch (AssertionError e) { failures.add(e.getMessage()); }
        }
        if (!failures.isEmpty()) {
            throw new AssertionError(heading + " (" + failures.size() + " failure(s))\n    " + String.join("\n    ", failures));
        }
    }

    static void eq(String what, Object expected, Object actual) {
        if (!Objects.equals(expected, actual)) throw new AssertionError(what + ": expected <" + expected + "> but was <" + actual + ">");
    }

    public static void main(String[] args) {
        Line l = line("BELL", 3, 650);
        try {
            assertAll("order line",
                () -> eq("code", "BELL", l.code()),
                () -> eq("quantity", 3, l.quantity()),
                () -> eq("price", 650L, l.priceCents()),
                () -> eq("total", 1950L, l.totalCents()));
            System.out.println("PASSED");
        } catch (AssertionError e) {
            System.out.println("FAILED " + e.getMessage());
        }
    }
}
Output
FAILED order line (1 failure(s))
    total: expected <1950> but was <1951>

What this code does

Each lambda checks one field of the order line, and assertAll runs all four even after one fails, collecting the messages. Only the total is wrong, so the report lists one failure, with its expected and actual values, under the heading. JUnit's assertAll produces a similar grouped report, with the heading as its first line.

Parameterized tests

Hamza's Money formatting tests differ only in their numbers. A parameterized test is one test method that JUnit runs once for each set of arguments, reporting each run separately. It is marked @ParameterizedTest instead of @Test and has a source annotation that supplies the arguments. Parameterized tests live in the junit-jupiter-params artifact, which junit-jupiter already includes.

Java Example · Parameterized tests (listing)
@ParameterizedTest(name = "{0} cents prints as {1}")
@CsvSource({
    "899,    8.99",
    "1200,   12.00",
    "0,      0.00",
    "-50,    -0.50",
    "-1250,  -12.50"
})
void formatsAmounts(long cents, String expected) {
    assertEquals(expected, new Money(cents).toString());
}

@ParameterizedTest
@ValueSource(ints = {-1, -40})
void rejectsNegativeCounts(int count) {
    assertThrows(IllegalArgumentException.class, () -> new Part("A-02", "700x25c tube", count, 40));
}

@ParameterizedTest
@EnumSource(Urgency.class)
void everyUrgencyHasALabel(Urgency u) {
    assertFalse(u.label().isBlank());
}

What this code does

@CsvSource gives one row of comma-separated values per run, and JUnit converts each value to the method's parameter type, so the five rows become five reported tests. @ValueSource supplies single values, and @EnumSource runs once for every constant of an enum, which keeps the test complete when someone adds a new urgency. The name attribute builds each run's display name from its arguments, so a failure reads -50 cents prints as -0.50.

Argument sources for @ParameterizedTest
SourceSuppliesExample
@ValueSourceone value per run, of a simple type@ValueSource(strings = {"", " "})
@CsvSourcerows of values written in the annotation"899, 8.99"
@CsvFileSourcerows from a CSV file in test resourcesresources = "/money-cases.csv"
@EnumSourceevery constant of an enum, or a chosen subset@EnumSource(Urgency.class)
@MethodSourcearguments returned by a static factory methoda Stream<Arguments> of records
@NullSource, @EmptySourcea null or empty value, often combined@NullAndEmptySource

Under the hood a parameterized test is a loop over rows that runs the same check and reports each row by name. Writing that loop by hand shows why the report is more useful than one test with five assertions: a failing row is named, and every other row still runs.

Java Example · Main.java: a parameterized test by hand
import java.util.*;

public class Main {
    static String format(long cents) {
        String sign = cents < 0 ? "-" : "";
        long a = Math.abs(cents);
        return String.format("%s%d.%02d", sign, a / 100, a % 100);
    }

    public static void main(String[] args) {
        String rows = """
            899,    8.99
            1200,   12.00
            0,      0.00
            -50,    -0.50
            -1250,  -12.50
            5,      0.5
            """;
        int run = 0, failed = 0;
        for (String row : rows.strip().split("\n")) {
            String[] cells = row.split(",");
            long cents = Long.parseLong(cells[0].trim());
            String expected = cells[1].trim();
            String actual = format(cents);
            run++;
            String name = "[" + run + "] " + cents + " cents prints as " + expected;
            if (expected.equals(actual)) System.out.println("PASSED " + name);
            else { failed++; System.out.println("FAILED " + name + ": was " + actual); }
        }
        System.out.println(run + " runs, " + failed + " failed");
    }
}
Output
PASSED [1] 899 cents prints as 8.99
PASSED [2] 1200 cents prints as 12.00
PASSED [3] 0 cents prints as 0.00
PASSED [4] -50 cents prints as -0.50
PASSED [5] -1250 cents prints as -12.50
FAILED [6] 5 cents prints as 0.5: was 0.05
6 runs, 1 failed

What this code does

The text block holds the same rows a @CsvSource would, and each row is split into its cents and expected text, then checked. Every row gets its own numbered, readable name, as JUnit's name pattern produces. The last row has a wrong expectation, 0.5 instead of 0.05, to show that a bad row fails on its own while the other five still pass.

A failing row can mean the code is wrong or the row is wrong; here it is the row. Tables of cases are code too, and they deserve the same care. When a table grows large or comes from Kofi's real data, move it into a CSV file under src/test/resources and use @CsvFileSource.

A diagram of one parameterized test method, formatsAmounts, with a CsvSource of five rows beside it. Arrows lead from the method to five separate result rows in a test report: 899 cents prints as 8.99, passed; 1200 cents prints as 12.00, passed; 0 cents prints as 0.00, passed; -50 cents prints as -0.50, passed; -1250 cents prints as -12.50, passed.
One parameterized test method becomes one reported run per row

Readable assertions with AssertJ

Many teams add AssertJ, a library of fluent assertions, alongside JUnit. Instead of assertEquals(expected, actual), you write assertThat(actual).isEqualTo(expected), and chains such as assertThat(list).hasSize(3).contains("BELL") read like sentences and give detailed failure messages for collections and strings. Hamcrest is an older library with a similar aim. Both are optional; JUnit's own assertions are enough for everything in this course.

Try it yourself

Write assertThrows

Complete assertThrows so it runs code, returns the exception if it is an instance of expected, throws an AssertionError saying wrong exception: X (where X is the simple class name of what was thrown) if another exception appears, and throws an AssertionError saying nothing was thrown if the code finishes normally.

The program should print caught: no stock, nothing was thrown and wrong exception: ArithmeticException.

public class Main {
    static <T extends Throwable> T assertThrows(Class<T> expected, Runnable code) {
        // Run the code and check what it throws.
        return null;
    }

    public static void main(String[] args) {
        try {
            IllegalStateException e = assertThrows(IllegalStateException.class, () -> { throw new IllegalStateException("no stock"); });
            System.out.println("caught: " + e.getMessage());
        } catch (Throwable t) { System.out.println(t.getMessage()); }
        try {
            assertThrows(IllegalStateException.class, () -> { int x = 1 + 1; });
            System.out.println("passed by mistake");
        } catch (AssertionError e) { System.out.println(e.getMessage()); }
        try {
            assertThrows(IllegalStateException.class, () -> { int z = 0; int y = 10 / z; });
            System.out.println("passed by mistake");
        } catch (AssertionError e) { System.out.println(e.getMessage()); }
    }
}

Try it yourself

A table of tax cases

withTax adds tax at a rate given in hundredths and rounds to the nearest cent. Fill in the missing expected values in the rows text block so that every row passes, working each one out by hand.

The program should finish by printing 4 runs, 0 failed.

public class Main {
    static long withTax(long cents, int ratePer100) {
        return Math.round(cents * (100 + ratePer100) / 100.0);
    }

    public static void main(String[] args) {
        String rows = """
            1000, 13, 1130
            899,  13, 0
            899,  8,  0
            0,    13, 0
            """;
        int run = 0, failed = 0;
        for (String row : rows.strip().split("\\n")) {
            String[] c = row.split(",");
            long cents = Long.parseLong(c[0].trim());
            int rate = Integer.parseInt(c[1].trim());
            long expected = Long.parseLong(c[2].trim());
            long actual = withTax(cents, rate);
            run++;
            if (actual != expected) {
                failed++;
                System.out.println("FAILED " + cents + " at " + rate + ": expected " + expected + " but was " + actual);
            }
        }
        System.out.println(run + " runs, " + failed + " failed");
    }
}

Common mistakes

  • MistakeComparing doubles with assertEquals and no tolerance.

    InsteadPass a delta chosen from the meaning of the number, or keep money in whole cents.

  • MistakeTesting only that valid input works.

    InsteadUse assertThrows to prove that bad input is rejected with the right exception and message.

  • MistakeCopying the same test with different numbers.

    InsteadUse a parameterized test with a source such as @CsvSource.

  • MistakeA string of assertEquals calls for one record's fields.

    InsteadUse assertAll so every wrong field is reported in one run.

  • MistakeTrusting a table of cases without checking it.

    InsteadWork out expected values independently; a wrong row is a bug in the test.

Summary and key terms

  • JUnit's Assertions class offers assertEquals, assertTrue and assertFalse, assertNull, assertSame, assertIterableEquals, assertThrows, assertAll, assertTimeout and fail, each with an optional message.
  • Doubles are compared with a tolerance, assertEquals(expected, actual, delta), while money stays exact in whole cents. assertThrows takes an exception class and a lambda, passes only if that exception is thrown and returns it so the message can be checked; assertThrowsExactly rejects subclasses. assertAll runs a group of checks and reports every failure together.
  • A @ParameterizedTest runs once per set of arguments from a source such as @ValueSource, @CsvSource, @CsvFileSource, @EnumSource or @MethodSource, naming each run.
  • AssertJ adds fluent assertions if a team wants them.
Assertion message
Extra text added to a failure report, given as the last argument of an assertion.
Tolerance (delta)
The largest difference allowed when comparing two doubles.
assertThrows
The assertion that passes only if the code throws the expected exception type, and returns it.
assertAll
The assertion that runs a group of checks and reports all their failures together.
Parameterized test
A test method run once for each set of arguments from a source.
@CsvSource
An argument source of comma-separated rows written in the annotation.
@MethodSource
An argument source that calls a static factory method for the arguments.
Display name
The readable name JUnit shows for a test or a parameterized run.
AssertJ
A popular library of fluent assertions such as assertThat(x).isEqualTo(y).

Practice problems

Work each problem on paper first, then reveal the answer to check.

  1. Why does 0.1 + 0.2 == 0.3 give false?

    Show answer

    Doubles are binary approximations, so the sum is a tiny bit above 0.3.

  2. What does assertThrows return?

    Show answer

    The exception that was thrown, so the test can check its message.

  3. When is assertAll better than several assertEquals calls?

    Show answer

    When checking several parts of one result, so all failures are reported at once.

  4. Which annotation replaces @Test for a test run over many rows?

    Show answer

    @ParameterizedTest, with a source such as @CsvSource.

  5. A parameterized row fails. Is the code necessarily wrong?

    Show answer

    No; the expected value in the row may be wrong, so check it independently.

Lesson quiz · 5 questions

Score 60% or more to complete this lesson.

Take the quiz