Penetration testing methodology and reporting

A vulnerability scan asks which known weaknesses are present. A penetration test asks a harder question: could a skilled, motivated adversary use the weaknesses in this environment, alone or chained together, to reach something that matters? Penetration testers combine tools with human judgement to explore paths that automated scanners cannot, such as abusing business logic, chaining a minor misconfiguration with a weak password, or moving from a compromised workstation to a database server. The value of a test lies not in the drama of getting in, but in the evidence it produces about which controls failed and which held.

This lesson explains penetration testing at the level a security manager needs: the phases of a test, the recognized methodologies, how to manage testers and risk during the engagement, and how to read and act on the report. It deliberately does not teach attack techniques; those belong in specialist training, under supervision, against systems you are authorized to test.

Vulnerability assessment versus penetration test

The two are often confused, and the exam expects you to distinguish them. A vulnerability assessment identifies, quantifies and ranks weaknesses, mostly through scanning and configuration review, and aims for breadth. A penetration test attempts to exploit weaknesses in a controlled way to demonstrate impact, and aims for depth along realistic attack paths. Assessments are cheaper and can run frequently; penetration tests are more expensive, more intrusive and usually periodic. Penetration tests also validate scanning: if testers find a path through weaknesses the scanner missed, the scanning program needs attention.

The phases of a penetration test

Methodologies differ in names and detail, but nearly all follow the same arc. NIST SP 800-115, Technical Guide to Information Security Testing and Assessment, describes four phases: planning, discovery, attack and reporting. The Penetration Testing Execution Standard (PTES) uses seven: pre-engagement interactions, intelligence gathering, threat modelling, vulnerability analysis, exploitation, post-exploitation and reporting. The Open Source Security Testing Methodology Manual (OSSTMM) emphasizes measurable operational security across channels, including human, physical, wireless and data networks. For web applications, the OWASP Web Security Testing Guide is the usual reference.

Planning, covered in lesson 2, sets objectives, scope, rules of engagement and authorization. Discovery has two parts: gathering information about the target, from public sources (open-source intelligence such as job postings, domain records and leaked credential lists) and from active probing within scope, then analysing that information for potential vulnerabilities. Attack is where testers attempt to verify vulnerabilities by exploiting them in a controlled way: gaining access, escalating privileges, moving to other systems and demonstrating access to agreed objectives such as a test record in a customer database. Each success typically reveals new information, which feeds further discovery; NIST draws this as a loop. Reporting documents what was found, how, what it means and how to fix it.

Professional testers stop short of real harm. They prove that data could be read rather than exfiltrating it, demonstrate that a change could be made by creating a harmless marker file rather than altering records, and avoid techniques that could disrupt availability unless explicitly authorized. At the end of the attack phase they clean up: remove accounts, files and tools they introduced and report anything they could not remove. A clean-up log is part of a professional deliverable.

Managing the engagement

The security manager's role during a test is oversight. Daily or regular check-ins keep both sides informed; testers report critical findings immediately under the rules of engagement; the operations team watches for unintended impact; and the manager decides quickly on scope questions that arise. If testers discover evidence that a real attacker is already present, testing pauses and the incident response plan takes over. Activity logs from the testers, with timestamps and source addresses, let defenders separate test activity from genuine attacks during and after the engagement.

Rating findings

Findings in a penetration test report are rated by risk, combining how likely exploitation is (given what the testers needed: skill, access, time) with the business impact if it occurred. Many reports use a likelihood-impact matrix, and also include CVSS scores for individual vulnerabilities. The most valuable findings are often attack paths rather than single flaws: a medium-rated weakness in password policy plus a medium-rated lack of segmentation may together produce a critical path to sensitive data. Good reports show these chains explicitly, often as diagrams.

# Rating findings with a 5x5 likelihood x impact matrix (illustrative scale)
LEVELS = {1: 'very low', 2: 'low', 3: 'medium', 4: 'high', 5: 'very high'}
def rating(likelihood, impact):
    s = likelihood * impact
    return 'Critical' if s >= 20 else 'High' if s >= 12 else 'Medium' if s >= 6 else 'Low'
findings = [
    ('Default password on building management console', 4, 4),
    ('Verbose error messages reveal software versions', 4, 1),
    ('Flat network: workstations reach database servers', 3, 5),
    ('Weak password policy allowed guessing of a service account', 4, 5),
    ('Missing security headers on marketing site', 2, 1),
]
for name, l, i in sorted(findings, key=lambda f: -f[1] * f[2]):
    print(f'{rating(l, i):8s} L={l} I={i} score={l*i:2d}  {name}')

Critical L=4 I=5 score=20  Weak password policy allowed guessing of a service account
High     L=4 I=4 score=16  Default password on building management console
High     L=3 I=5 score=15  Flat network: workstations reach database servers
Low      L=4 I=1 score= 4  Verbose error messages reveal software versions
Low      L=2 I=1 score= 2  Missing security headers on marketing site

Look at how the two highest-rated findings in the output are not exotic. Weak passwords and a flat network are among the most common contributors to successful internal tests year after year. That is a key lesson for managers: penetration tests often confirm that basic controls, consistently applied, matter more than advanced ones.

Reading and acting on the report

A good penetration test report has two audiences. The executive summary, usually two pages or less, tells leaders in plain language what the testers set out to do, what they achieved, the overall risk, the most important themes and the strategic recommendations, without jargon. The technical section gives each finding a title, risk rating, affected assets, a description of the weakness, evidence (screenshots, request and response extracts with sensitive data redacted), the steps that demonstrate it at a level sufficient for remediation and retesting, and specific remediation advice with references. It also records scope, methodology, limitations, dates, tester names and the clean-up log.

After delivery, the organization owns the work. Each finding becomes a tracked remediation item with an owner and due date, prioritized as in lesson 4. Systemic themes, such as weak credential management, become improvement projects rather than one-off fixes. A retest, ideally by the same testers, verifies that fixes work and that no new path has opened. The report itself is sensitive: it is a map of how to break into the organization, so it should be classified, distributed on a need-to-know basis and stored securely. Sharing reports with customers or auditors usually happens through a summary letter of attestation rather than the full document.