Building practice exams and keeping an error log

Content knowledge gets a candidate into the range where passing is possible. Practice turns it into a pass. This lesson explains how to use practice questions well: where to find good ones, how to simulate the adaptive exam, how to read practice scores without being misled by them, and how to keep an error log that turns every wrong answer into a specific fix. It also covers spaced review and retrieval practice, two study methods with strong support from research on learning.

Hannah and Tomás reach this stage in week 5 of their ten-week plan. They have worked through lessons 1 to 10, taken their diagnostic and started tagging errors. From here on, practice is the center of their study.

Choosing practice material

Practice questions vary widely in quality. Good items look like the real exam: a short scenario, a clear stem, four plausible options and an explanation that says why the right answer is right and why each wrong answer is wrong. Poor items test trivia, contain errors or reward memorizing a particular book's wording. A large bank of poor items does more harm than a small bank of good ones, because it trains the wrong habits.

Sources worth considering include the official practice materials sold by ISC2, the question banks that come with well-reviewed study guides and courses, and the practice items in this track. Avoid anything that claims to contain real exam questions. Such material breaches the candidate agreement and the Code of Ethics, as lesson 9 explained, and using it can lead to a result being canceled or a certification revoked.

Simulating the real exam

Most practice banks are fixed sets, not adaptive engines, so they cannot reproduce the CAT exactly. They can still simulate the conditions that matter: forward-only answering, a time limit, and a mix of domains in proportion to the outline. Hannah and Tomás build each practice exam the same way: 125 items drawn across domains by weight, answered in one sitting of 2 hours 30 minutes, with no going back, no notes and no phone. They take one short break after item 75, as they plan to on the day.

Doing practice exams under exam conditions matters more than candidates expect. Concentration over two to three hours is a skill, and so is the discipline of not returning to earlier items. A candidate who has only ever answered questions in sets of 20, with the option to look things up, will find the real exam far more tiring than one who has sat several full simulations.

Reading practice scores

Practice scores are useful but easy to misread. Different banks are harder or easier than the real exam, and a score on a fixed set is a percentage correct, not a scaled score like the exam's 700. Training providers commonly suggest that consistent scores of about 75 % to 80 % on good-quality, unfamiliar items indicate readiness, but this is a rule of thumb, not an official threshold.

Three habits make practice scores more trustworthy. Use fresh items for simulations, because a repeated item measures memory, not understanding. Track the trend over several weeks rather than reacting to one result. And look at scores by domain, because a strong overall score can hide one weak domain that the adaptive engine will find.

The error log

An error log is a simple record of every practice item answered wrongly, or answered rightly by guessing. For each item it records the domain, the topic, the cause of the error and the fix. Over time it shows patterns that a score alone hides. Lesson 2 introduced three causes: missing content, misreading and choosing a technical answer when a manager's answer was expected. Hannah and Tomás add two more: falling for a distractor type, and running short of time.

The fix column is where the value lies. 'Study Domain 7 more' is not a fix. 'Make a card listing the five disaster recovery test types in order, with what each one does' is. For misreading errors, the fix is usually a reading habit, such as naming the task word. For mindset errors, it is a sentence naming the priority the correct answer served. For distractor errors, it is the distractor family from lesson 4 and a note on how to spot it next time.

Retrieval practice and spaced review

Research on learning over several decades points to two methods that work better than rereading. Retrieval practice means testing yourself, trying to recall information without looking at it, rather than reading it again. Spacing means reviewing material at increasing intervals rather than all at once. Combined, they produce stronger and longer-lasting memory than the same time spent rereading notes, even though rereading often feels more productive.

In practice, Hannah and Tomás turn each error log fix into a question on a flash card or in a simple spreadsheet, and review each card on a schedule that stretches out as they get it right: the next day, then three days later, then a week, then two weeks. Cards they get wrong go back to the start. Many free and paid tools automate this kind of schedule, but a spreadsheet works too.

# A simple spaced-review schedule: intervals grow when a card is recalled, reset when missed
from datetime import date, timedelta
intervals = [1, 3, 7, 14, 28]
start = date(2026, 11, 2)
history = {
    "DR test types in order": [True, True, False, True, True],
    "SOC 2 Type 1 vs Type 2": [True, True, True, True],
    "Canon order":            [True, False, True, True],
}
for card, results in history.items():
    day, step = start, 0
    schedule = []
    for ok in results:
        day = day + timedelta(days=intervals[step])
        schedule.append(f"{day:%b %d}{'' if ok else ' (missed)'}")
        step = min(step + 1, len(intervals) - 1) if ok else 0
    print(f"{card:<24} " + ", ".join(schedule))

DR test types in order   Nov 03, Nov 06, Nov 13 (missed), Nov 14, Nov 17
SOC 2 Type 1 vs Type 2   Nov 03, Nov 06, Nov 13, Nov 27
Canon order              Nov 03, Nov 06 (missed), Nov 07, Nov 10

The output shows how a missed card returns to a one-day interval and then climbs again, while a card recalled every time moves quickly to two- and four-week gaps. By the exam date, the material Hannah and Tomás find hardest has been retrieved many more times than material they already knew, which is exactly where the extra effort should go.

Analyzing an error log

Every two weeks, Hannah and Tomás sort their error logs to see what has changed. The program below summarizes a small sample from Tomás's log by domain and cause. It is short, but it answers the questions that matter: which domains still produce the most errors, and which causes are shrinking.

# Summarize an error log by domain and by cause
log = [
    (5, "D8", "content"), (5, "D1", "mindset"), (5, "D6", "content"), (5, "D1", "mindset"),
    (6, "D8", "content"), (6, "D7", "content"), (6, "D2", "misread"), (6, "D8", "distractor"),
    (7, "D8", "content"), (7, "D6", "time"), (7, "D3", "content"), (7, "D1", "mindset"),
    (8, "D8", "content"), (8, "D5", "distractor"), (8, "D6", "content"),
]
from collections import Counter
by_domain = Counter(d for _, d, _ in log)
by_cause = Counter(c for _, _, c in log)
mindset_by_week = Counter(w for w, _, c in log if c == "mindset")
print("By domain:", ", ".join(f"{d} {n}" for d, n in by_domain.most_common()))
print("By cause: ", ", ".join(f"{c} {n}" for c, n in by_cause.most_common()))
print("Mindset errors by week:", dict(sorted(mindset_by_week.items())))

By domain: D8 5, D1 3, D6 3, D7 1, D2 1, D3 1, D5 1
By cause:  content 8, mindset 3, distractor 2, misread 1, time 1
Mindset errors by week: {5: 2, 7: 1}

Domain 8 dominates Tomás's recent errors, and nearly all of them are content gaps, which confirms the trend in his practice scores. His mindset errors have fallen from two in week 5 to one or none a week since. That tells him where the last weeks of study should go: Domain 8 content, especially the stages of the development life cycle and the supply chain topics from CISSP-117.

Mixing practice with review

The best weekly rhythm alternates three kinds of session. Short, mixed practice sets of 25 to 50 items keep every domain in play and expose misreading habits. Targeted sessions on one weak topic, guided by the error log, close specific gaps. Review sessions use the spaced cards and summary notes. Hannah and Tomás aim for roughly equal time on each, with full simulations in weeks 6, 8 and 9. When a practice set goes badly, they resist the urge to take another straight away; they review it first, because the review is where the improvement happens.

Studying with a partner helps too. Hannah and Tomás meet for an hour each week to explain the answers to items one of them got wrong. Explaining a concept aloud is another form of retrieval practice, and their opposite strengths mean each can teach the other: Tomás explains IPsec modes, and Hannah explains why the risk owner, not the engineer, signs off a risk acceptance.

Common traps in the final weeks

A few habits undermine otherwise good preparation. Some candidates take practice exam after practice exam without reviewing their errors, which measures progress but does not create it. Others switch between many different question banks, never staying with one long enough to learn from its explanations. Some memorize answers to items they have seen before, which produces rising scores that mean little. And many neglect sleep, which research on memory consistently shows is when learning is consolidated.