Full mock exams

Practice sets build skills; mock exams test whether the skills hold up under exam conditions for the full length of the exam. From week 5 Kai sat a mock every Saturday morning. Two of them, in weeks 6 and 10, were full length and timed exactly like the real exam; the others were half length. This lesson covers how to set up a mock so that it tells the truth, how to read the results, and how to decide whether the trend is good enough to book the exam with confidence.

Setting up a true mock

A mock exam is only useful if it is close to the real thing. Aggie's rules were strict. The questions must be new to the candidate, drawn across all areas in the blueprint's proportions. The time limit must be the real one. There are no notes, no phone, no breaks other than those the real exam allows, and only the calculator that will be used on the day. The mock is written at the same time of day as the exam, in a quiet room that is not the candidate's usual study spot, because familiar surroundings make recall easier and give a falsely high score.

For the full mocks, Aggie built 120-question papers in the Red Seal blueprint proportions: 24 questions from Activity A, 38 from B, 41 from C, 10 from D and 7 from E. She used questions from three sources, so that Kai could not have seen most of them, and ran the papers at Keelson's training room on Saturday mornings.

# Build a mock paper in blueprint proportions from a tagged question bank
import random
blueprint = {"A": 24, "B": 38, "C": 41, "D": 10, "E": 7}
bank = {area: [f"{area}-{i:03d}" for i in range(1, 200)] for area in blueprint}
rng = random.Random(6)                      # week 6 paper
paper = []
for area, n in blueprint.items():
    paper += rng.sample(bank[area], n)
rng.shuffle(paper)
print(f"{len(paper)} questions; first five: {paper[:5]}")
for area, n in blueprint.items():
    print(f"  {area}: {n}")

120 questions; first five: ['A-139', 'C-135', 'B-170', 'A-006', 'C-080']
  A: 24
  B: 38
  C: 41
  D: 10
  E: 7

Reading the trend

One mock score is a weak signal. A single result can be pulled up or down by a few unusually easy or hard questions, by a bad night's sleep, or by a topic that happens to come up more than usual. The trend over several mocks is much more reliable. Kai's scores rose from 62 in week 5 to 82 in week 12, with two small dips. Aggie pointed out that the dips were not failures: week 7 was the week he changed to the five-step method, which slowed him down at first, and week 10 was a full-length mock, where fatigue in the last hour cost him a few marks.

The dashed line on the chart is the pass mark. Kai crossed it in week 8 and stayed above it from then on. Aggie's own guide is that a candidate who has scored at least 75 percent on the last two or three mocks, written under real conditions with new questions, is well placed. A candidate whose scores are hovering around the pass mark is not ready yet, because the real exam has its own nerves and surprises.

From mock score to pass probability

A useful way to think about readiness is to treat each exam question as a separate chance of being right, with a probability equal to the candidate's recent mock average. The number of correct answers then follows a binomial distribution, and the chance of reaching the pass mark can be calculated. The model is simplified, because questions are not all equally hard and they are not truly independent, but it shows something that surprises most people: a mock average just above the pass mark gives only about an even chance of passing, while a few points more gives a large margin.

# Chance of reaching the pass mark if each question is right with probability p
from math import comb
n = 120                     # questions on the exam
k = 84                     # correct answers needed for 70 percent
for p in (0.70, 0.74, 0.78, 0.82):
    prob = sum(comb(n, i) * p ** i * (1 - p) ** (n - i) for i in range(k, n + 1))
    print(f"mock average {100 * p:.0f} %  ->  chance of passing about {100 * prob:.0f} %")

mock average 70 %  ->  chance of passing about 54 %
mock average 74 %  ->  chance of passing about 86 %
mock average 78 %  ->  chance of passing about 98 %
mock average 82 %  ->  chance of passing about 100 %

With a mock average of 70 percent, the model gives only about a 55 percent chance of passing. At 74 percent the chance is about 85 percent, and at 78 percent it is above 97 percent. Kai's average over his last three mocks was about 78 percent. That did not guarantee a pass, but it told him and Aggie that he was in a strong position and that the final weeks should keep him there rather than push for more.

Analysing a full mock

After each full mock, Kai's review had three parts. First, the score by area, compared with the blueprint weight, to see whether any area was dragging the total down. Second, the error log, with causes, as in Lesson 8. Third, the timing: the time he reached each pacing checkpoint, and the score in each quarter of the paper. The timing review of the week 10 mock showed that his accuracy fell in the last quarter, from about 80 percent in the first quarter to about 64 percent in the last. That pointed to fatigue, not knowledge, and the fix was practical: a proper breakfast, water and a small snack if allowed, and a stretch and slow breath at each checkpoint.

# Score by quarter of the paper: is accuracy falling late in the exam?
answers = "011111111111001011101011111111111111111100001101111111101101111110101101100110111111010111111110110110001111000110011101"   # 1 = correct, 0 = wrong, from the week 10 full mock
q = len(answers) // 4
for i in range(4):
    part = answers[i * q:(i + 1) * q]
    print(f"quarter {i + 1}: {part.count('1')}/{len(part)} = {100 * part.count('1') / len(part):.0f} %")
print(f"total: {answers.count('1')}/{len(answers)} = {100 * answers.count('1') / len(answers):.0f} %")

quarter 1: 24/30 = 80 %
quarter 2: 23/30 = 77 %
quarter 3: 22/30 = 73 %
quarter 4: 19/30 = 63 %
total: 88/120 = 73 %

Adjusting the final plan

Each full mock is followed by a decision about the remaining weeks. After the week 6 mock, Kai's plan shifted hours toward layout and roofing. After the week 10 mock, with his average near 78 percent, the decision was the opposite: no new topics, short daily reviews of the error log's repeat list, two more half-length mocks, and a gradual drop in study hours so that he would arrive at the exam rested. Aggie's view was that the last fortnight is for keeping, not for building.

Kai booked his exam with the provincial apprenticeship agency in week 8, once his mocks had crossed the pass mark, choosing a sitting in week 12 so the date was fixed and the plan had an end point.

Simulating exam day

The two full mocks were also rehearsals of the whole day, not just the paper. Kai got up at the same time he would on exam day, ate the same breakfast, drove to Keelson's training room, and began at the same clock time. He had the same items with him that he would take to the exam: identification, pencils, the approved calculator, water and a snack. After the first full mock he noticed that he had been hungry by the third hour, so for the second he ate a larger breakfast; after the second he noticed that his calculator battery indicator was low, and he replaced it. Small problems like these are much better found in a mock than on the day.

Rehearsal also helps with the length of the exam. Few people write for three or four hours at a stretch in normal life, and the last hour of a long paper is a different experience from the first. Kai's first full mock showed him how tired he was after two and a half hours. By the second, his checkpoints, food, water and stretches had kept his accuracy steadier to the end, and his final quarter score was only a few points below his first.

Scoring by area

A full mock in blueprint proportions also gives a score for each area, which can be compared with the earlier topic sets. Kai's week 10 mock, his 73 percent paper, showed common occupational skills at 75 percent, fabrication at 74 percent, installation at 78 percent, roofing at 60 percent and maintenance at 57 percent. The weighted plan in Lesson 2 still applied, but with much less room to move: roofing and maintenance were weak but small, so a few more hours there would gain perhaps one or two questions each, while lifting fabrication and installation protected about two thirds of the paper.

# Mock score by area: where would one more week of study earn the most?
areas = {"A common": (24, 18), "B fabrication": (38, 28), "C installation": (41, 32), "D roofing": (10, 6), "E maintenance": (7, 4)}         # area: (questions on the paper, correct answers)
for a, (n, c) in areas.items():
    print(f"{a:14s} {c:2d}/{n:2d} = {100 * c / n:3.0f} %, marks lost {n - c}")
total = sum(c for _, c in areas.values()) / sum(n for n, _ in areas.values())
print(f"weighted total {100 * total:.0f} %")

A common       18/24 =  75 %, marks lost 6
B fabrication  28/38 =  74 %, marks lost 10
C installation 32/41 =  78 %, marks lost 9
D roofing       6/10 =  60 %, marks lost 4
E maintenance   4/ 7 =  57 %, marks lost 3
weighted total 73 %

The marks lost show where effort pays. Kai's largest losses were in fabrication and installation, simply because those areas have the most questions, even though his percentages there were higher than in the small areas. That is why the final plan kept steady work in both, and used only short sessions for the smaller areas.

When the mock is bad

Not every mock goes well, and a bad one late in the plan can shake a candidate's confidence. Aggie's response was always to look for the cause before reacting. Was the paper harder than usual, or drawn from an unfamiliar source? Was the candidate tired, ill or distracted? Did the error log show a new cause or an old one returning? Only after answering those questions did she decide whether anything needed to change. Most bad mocks, she said, are noise; a few are a signal, and the error log tells them apart.