Security principles and threat modeling

Computer security is the discipline of keeping systems doing what their owners intend while someone else is trying to make them do something else. That second clause is what separates security from ordinary reliability engineering. A bridge engineer designs for wind, load and corrosion, which follow physical laws and do not adapt. A security engineer designs for an adversary who reads the design, probes for the weakest point and changes tactics when one door closes. This course takes the computer scientist's view: how the mechanisms that protect systems actually work, why they fail, and how to reason about them precisely. Every lesson includes small programs that run for real, so you can see a hash collide, a cipher leak a pattern or an injection succeed and then be fixed.

The course builds on earlier ones. CMPUT-313 explained how networks carry data; CMPUT-201 showed how C manages memory and where buffer overflows come from; CMPUT-291 covered SQL injection and database permissions; CMPUT-272 introduced modular arithmetic and a toy RSA; and CMPUT-301 placed security inside the software development life cycle. The CISSP track (CISSP-110 to CISSP-117) covers the same ground from a manager's point of view: governance, risk programs, audits and certification. Here we stay close to the code and the mathematics. Throughout the course every vulnerability is shown in a tiny, self-contained toy program running on this machine, next to its fix. Nothing here is a tool for attacking real systems, and the final lesson discusses the law and ethics that apply to security work.

Confidentiality, integrity and availability

The classic goals of security are confidentiality, integrity and availability, known as the CIA triad. Confidentiality means information is disclosed only to those authorized to see it: your marks, your medical file, a company's source code. Integrity means information and systems are changed only in authorized ways, and that unauthorized changes are detected: a bank balance, a firmware image, a grade. Availability means authorized users can use the system when they need it: a hospital records system during an emergency, a course registration site in the first week of term. The three pull against each other. Encrypting a disk improves confidentiality but threatens availability if the key is lost; adding replicas improves availability but creates more copies to protect.

Several further properties are often added. Authenticity means we can verify who or what originated a message. Accountability means actions can be traced to the party responsible, which needs logging and reliable identification; its legal cousin is non-repudiation, where a party cannot credibly deny having performed an action. Privacy, which Lesson 11 treats in depth, concerns individuals' control over information about themselves and is broader than confidentiality: a system can keep data perfectly confidential from outsiders and still violate privacy by using it for purposes the person never agreed to.

Assets, threats, vulnerabilities and risk

Security reasoning starts with a vocabulary. An asset is anything of value: data, computing capacity, reputation, the safety of people. A threat is a potential cause of harm, often a threat actor with a motive and capability: a curious student, a criminal group selling stolen data, an insider with a grudge, a state agency. A vulnerability is a weakness that a threat could use: a missing access check, an unpatched library, a password stored in plain text. An exploit is a specific technique that uses a vulnerability. A control, or countermeasure, reduces risk by preventing, detecting or limiting harm. Risk combines how likely a harmful event is with how bad it would be.

Threat actors differ enormously in skill, resources and persistence, and a design should state which ones it defends against. Protecting a student club's website from automated scanning bots is a different problem from protecting a power grid from a well-funded foreign intelligence service. The threat model is the explicit statement of what we are defending, against whom, and under what assumptions. A good threat model also says what is out of scope: for example, that the cloud provider's hypervisor is trusted, or that physical theft of servers is handled by the data center rather than by the application.

Saltzer and Schroeder's design principles

In 1975 Jerome Saltzer and Michael Schroeder published a set of design principles for protecting information in computer systems. Half a century later they remain the best short guide to building secure systems, and later lessons keep returning to them. Economy of mechanism says keep the design small and simple, because complex mechanisms hide flaws and are hard to review. Fail-safe defaults says base access decisions on permission rather than exclusion: if the code is unsure, deny. Complete mediation says check every access to every object, every time, rather than trusting an earlier check. Open design says the security of a mechanism should not depend on its design being secret; this is the computing form of Kerckhoffs' principle, which Lesson 2 discusses.

Separation of privilege says require more than one condition to grant access when the stakes are high, such as two keys or two people. Least privilege says every program and user should operate with the smallest set of privileges needed for the task, so that an accident or a compromise does limited damage. Least common mechanism says minimize mechanisms shared between users, because every shared mechanism is a potential channel between them. Psychological acceptability says the protection must be easy enough to use that people use it correctly; a mechanism that users routinely bypass provides no protection. Two later additions are widely used: defense in depth, which layers independent controls so that one failure is not fatal, and secure by default, which ships the safe configuration rather than leaving it as an option.

Threat modeling with STRIDE

Threat modeling is a structured way of asking, before and during development, what could go wrong. Adam Shostack summarizes it as four questions: what are we working on, what can go wrong, what are we going to do about it, and did we do a good job? The first question is usually answered with a data flow diagram that shows external entities (users, other systems), processes (the code), data stores (databases, files) and data flows between them, with trust boundaries drawn wherever data crosses from a less trusted to a more trusted zone. The internet-facing edge of a web application is a trust boundary; so is the line between the application server and the database, and the line between the kernel and a user process.

For the second question, Microsoft's STRIDE mnemonic gives six categories, each the negation of a security property: Spoofing violates authenticity, Tampering violates integrity, Repudiation violates accountability, Information disclosure violates confidentiality, Denial of service violates availability, and Elevation of privilege violates authorization. Walking each element of the data flow diagram through the six categories produces a list of candidate threats far more complete than unstructured brainstorming. Each threat is then ranked and assigned a response: mitigate it with a control, eliminate the feature, transfer the risk (insurance, a contract) or accept it with a documented reason.

# Ranking threats for a small course-registration web app
# risk = likelihood (1-5) x impact (1-5); ties broken by impact
threats = [
    ("Spoofing",               "attacker logs in with a stolen password",          4, 4),
    ("Tampering",              "student edits a grade field in a hidden form input", 3, 5),
    ("Repudiation",            "instructor denies changing a mark; no audit log",     2, 3),
    ("Information disclosure", "error page prints the database connection string",   3, 4),
    ("Denial of service",      "login endpoint flooded during registration week",     3, 3),
    ("Elevation of privilege", "student reaches /admin because only the link is hidden", 3, 5),
]
ranked = sorted(threats, key=lambda t: (t[2] * t[3], t[3]), reverse=True)
print(f"{'rank':<5}{'STRIDE category':<24}{'L':>2}{'I':>3}{'risk':>6}  example")
for i, (cat, ex, l, imp) in enumerate(ranked, 1):
    print(f"{i:<5}{cat:<24}{l:>2}{imp:>3}{l*imp:>6}  {ex}")

rank STRIDE category          L  I  risk  example
1    Spoofing                 4  4    16  attacker logs in with a stolen password
2    Tampering                3  5    15  student edits a grade field in a hidden form input
3    Elevation of privilege   3  5    15  student reaches /admin because only the link is hidden
4    Information disclosure   3  4    12  error page prints the database connection string
5    Denial of service        3  3     9  login endpoint flooded during registration week
6    Repudiation              2  3     6  instructor denies changing a mark; no audit log

The ranking above is deliberately simple. Real teams use richer schemes such as DREAD (now largely abandoned because its scores were too subjective), the Common Vulnerability Scoring System for known vulnerabilities, or quantitative estimates of expected annual loss. What matters is that the reasoning is written down and reviewed. Note how two of the top threats, tampering with a hidden form field and reaching an admin page whose link is merely hidden, both violate complete mediation: the server trusted the client to behave. Lesson 9 shows both in code.

Attack trees

Bruce Schneier popularized attack trees in 1999. The root is the attacker's goal. Each child is a way of achieving its parent: an OR node is satisfied if any child is, an AND node only if all children are. Leaves are concrete actions and can be annotated with cost, required skill, probability of detection or time. Evaluating the tree from the leaves up tells you which path an economically rational attacker would choose, and therefore where a defender's money buys the most. The tree in the figure is evaluated by the next program.

# An attack tree: OR nodes need any child, AND nodes need all children.
# Each leaf has an estimated attacker cost in dollars; the tree gives the cheapest attack.
tree = ("OR", "Read another student's grades", [
    ("AND", "Steal a session", [
        ("LEAF", "Find an XSS flaw", 2000),
        ("LEAF", "Lure the victim to the page", 200),
    ]),
    ("LEAF", "Guess a weak password (no rate limit)", 300),
    ("AND", "Abuse the API", [
        ("LEAF", "Discover the /api/grades endpoint", 50),
        ("LEAF", "Change the student id in the URL (no access check)", 0),
    ]),
    ("LEAF", "Bribe a teaching assistant", 5000),
])

def cost(node, depth=0):
    kind, name = node[0], node[1]
    if kind == "LEAF":
        c = node[2]
    else:
        costs = [cost(ch, depth + 1) for ch in node[2]]
        c = min(costs) if kind == "OR" else sum(costs)
    print("  " * depth + f"{kind:<4} {name}: ${c:,}")
    return c

cheapest = cost(tree)
print("cheapest attack costs about", f"${cheapest:,}")

    LEAF Find an XSS flaw: $2,000
    LEAF Lure the victim to the page: $200
  AND  Steal a session: $2,200
  LEAF Guess a weak password (no rate limit): $300
    LEAF Discover the /api/grades endpoint: $50
    LEAF Change the student id in the URL (no access check): $0
  AND  Abuse the API: $50
  LEAF Bribe a teaching assistant: $5,000
OR   Read another student's grades: $50
cheapest attack costs about $50

The cheapest path costs about fifty dollars because the grades API checks only that the caller is logged in, not that the requested record belongs to them. This class of flaw, called an insecure direct object reference or broken object-level authorization, tops the OWASP lists discussed in Lesson 9. Fixing it raises the cheapest attack to three hundred dollars (password guessing), and adding rate limiting and multifactor authentication raises it again. Security is rarely about making attacks impossible; it is about making them more expensive than the asset is worth to the attacker.

The attacker's advantage and the economics of security

Defenders must protect every path; attackers need only one. This asymmetry explains why simple, small systems are easier to secure than large ones, and why reducing the attack surface (the set of entry points an attacker can reach) is so effective: every disabled service, closed port and removed feature is one less thing to get right. Security also has an economic dimension. A vulnerability in a widely used library is worth finding for criminals because it opens many systems at once; the vendor may underinvest in fixing it because the cost of breaches falls on customers. Ross Anderson's work on security economics argues that many failures are failures of incentives rather than of technology.

Security as a process

No design is secure once and for all. Software changes, new attacks are discovered, configurations drift and people make mistakes. Mature organizations treat security as a continuous cycle: identify assets and risks, protect them, detect incidents, respond, and recover, which is the structure of the NIST Cybersecurity Framework (version 2.0, released in 2024, which added a sixth function, govern). Engineers contribute at every stage, from threat modeling in design to writing logs that make detection possible. The rest of this course works through the mechanisms: cryptography in Lessons 2 to 6, authentication and access control in Lesson 7, software vulnerabilities in Lessons 8 and 9, operating systems and networks in Lesson 10, privacy in Lesson 11, and an end-to-end capstone in Lesson 12.