Practice set analysis

A practice set is only as useful as what you do afterward. In the first week of December 2026, Mateo plans to take his first full timed practice set: 65 original questions in 90 minutes, under exam conditions, with his phone in another room. Gwen will not let him look at the score first. "The number tells you very little," she says. "The pattern tells you everything. We will spend longer on the review than you spent on the set." This lesson shows how to analyze a practice set so that each one moves you closer to the pass, using a worked example built from the kind of results Mateo and Kwame see in their autumn practice.

Taking a practice set well

Take full practice sets under conditions that match exam day: one sitting, 90 minutes, no notes, no searching, the same pacing plan from lesson 10. Use sets of original questions from AWS, such as the official practice question set on AWS Skill Builder, or from reputable providers. As you go, flag questions you are unsure of, exactly as you will on the day, and note your confidence for each answer if the tool allows. Do not stop to look things up. The aim is to measure your readiness honestly, including your timing.

Space practice sets out. Taking several in a row teaches you the questions in that set rather than the material, and a high score on a set you have seen before means little. Gwen's rule is no more than one full set every few days, with study between them, and never the same set twice in a week. If you reuse a set, treat its score as a check on memory, not on readiness.

The four-step review

After each set, work through four steps. First, record the overall result and the result by domain. Second, review every question you got wrong and every question you got right but flagged or guessed, not just the wrong ones. Third, classify each of those questions using the four mistake types from lesson 9: knowledge gap, confusion, reading error or timing error. Fourth, decide one or two specific actions for the next few days and write them in your study log. The classification is the heart of it, because each type has a different fix, and a score alone hides which one you need.

A mistakes heat map

A heat map turns the classification into a picture. Draw a grid with the four domains as rows and the four mistake types as columns, and put the count of mistakes in each cell. Dark cells show where errors cluster. A dark cell in one domain under knowledge gap means you need to study that domain. A dark column under reading error, spread across all domains, means the problem is not knowledge at all but how you read questions. A dark timing column means your pacing plan is not working. The picture makes the next action obvious in a way a single score never does.

# Build a simple mistakes heat map from a practice-set review
from collections import Counter
mistakes = [
    ("Security", "confusion"), ("Security", "confusion"), ("Security", "gap"),
    ("Billing", "gap"), ("Billing", "gap"), ("Billing", "reading"),
    ("Technology", "confusion"), ("Technology", "reading"),
    ("Concepts", "reading"), ("Technology", "timing"),
]
counts = Counter(mistakes)
types = ["gap", "confusion", "reading", "timing"]
print("Domain".ljust(12), *[t.ljust(10) for t in types])
for d in ["Concepts", "Security", "Technology", "Billing"]:
    print(d.ljust(12), *[str(counts[(d, t)]).ljust(10) for t in types])

Domain       gap        confusion  reading    timing    
Concepts     0          0          1          0         
Security     1          2          0          0         
Technology   0          1          1          1         
Billing      2          0          1          0

The example is a review of ten mistakes from one practice set. Security confusions and billing knowledge gaps stand out, and there are reading errors in three domains. That tells the candidate to do three things next: add the confused security pairs to the lesson 8 list, study pricing and support for billing, and slow down with the reading routine. One timing mistake is not yet a pattern.

A worked review of Mateo's autumn practice

Mateo's last practice set before the plan was written, in early November, had 65 questions. He answered every question, flagged 12 along the way, and finished with 8 minutes to spare. His review found 14 questions to analyze: 9 wrong and 5 right but guessed. Of the 14, 5 were confusions, most of them in security: GuardDuty versus Inspector twice, Shield versus WAF, security groups versus network ACLs, and Artifact versus Audit Manager. Four were knowledge gaps: two on the Cloud Adoption Framework perspectives, one on Storage Gateway, one on support plan features. Four were reading errors: two "not" questions read the wrong way round, and two where he missed "least operational effort" and chose a more hands-on option. One was a timing error near the end, where he rushed.

Look at what the review does not do. It does not say "study security more" in general; it names the exact pairs he confuses. It does not treat the reading errors as a knowledge problem; it names the habit that will fix them. And it does not overreact to one timing error. The next practice set, planned for mid-December after his study weeks on security, will show whether the actions worked: the confusion count should fall, and if reading errors persist, he will spend more time on the reading routine.

Reading the score with care

Practice set scores are useful as a trend, not as a prediction. Different providers write questions of different difficulty, and none of them can reproduce AWS's scaled scoring exactly. A rising trend across several sets from good sources, with fewer mistakes of each type, is a much better sign of readiness than any single high score. Gwen suggests a simple readiness test: across your last two or three unseen sets, you consistently answer most questions confidently and correctly in every domain, your mistakes log shows no repeating confusions, and you finish with time to review.

Beware of two traps. The first is memorizing a set: if you retake a set you have reviewed, your score will rise because you remember answers, not because you understand more. The second is chasing an arbitrary number: some candidates keep taking practice sets in search of a target score and stop studying the gaps the sets reveal. The point of practice is to find gaps and close them.

Writing good review notes

The quality of a review depends on the notes it produces. A weak note says "got the CloudTrail one wrong". A strong note says what the question asked, what you chose, why that was tempting, what the right answer was, and the rule that will get you there next time. Gwen shows Mateo one of hers from years ago: "Asked who changed a security group. I chose Config because I saw 'configuration'. Right answer CloudTrail: 'who' means API activity. Rule: 'who' beats 'configuration' as a signpost." That note took a minute to write and fixed a confusion for good.

Write the rule in your own words, and make it short enough to remember. Rules like "who means CloudTrail", "interruptible means Spot" and "least effort means managed" are the raw material for the final review sheet in lesson 12. When the same rule appears in your notes three times, it belongs on the sheet. When a rule turns out to be wrong in a new question, rewrite it rather than abandoning it; the exam often tests the exceptions to a rule of thumb.

Also note what went right. When you answer a hard question correctly for a clear reason, record the reasoning. Confidence on exam day comes partly from evidence that your methods work, and a log that only lists failures can make a well-prepared candidate feel unready. Mateo adds a short "good calls" section at the bottom of each review page, listing two or three questions where the reading routine or a deciding difference saved him.

Using domain results without over-reading them

Most practice tools report results by domain, and these are useful, but the numbers are small. With only a dozen or so questions from a domain in a practice set, one or two answers can swing the domain result a long way. Look for patterns that repeat across two or three sets before deciding a domain is weak or strong. A single poor showing in Domain 4, for example, may simply mean that set had unusual billing questions. Two or three poor showings, together with knowledge gaps in the log, are a real signal.

The same caution applies to AWS's own score report after the real exam. It describes each domain's performance in words, such as whether you met the competencies, rather than exact counts. Treat it as a direction for study rather than a precise measurement, and combine it with your own mistakes log, which you trust more because you wrote it.

Gwen ends the session by asking both candidates to set a simple rule for the remaining weeks: every practice set must produce at least one specific action, written in the log before the next study session begins. A practice set that produces no action, she says, was either too easy or not reviewed properly. Mateo writes the rule on the first page of his log, under the photo of the whiteboard from the first session.

The review also feeds the study plan from lesson 9. If a domain keeps appearing in the heat map, its share of the remaining hours rises; if it has been clear for two sets, its share can fall. The plan is not fixed at the start: each practice set adjusts it, and by the final weeks the hours should be going almost entirely to the few gaps that remain.