Secure design principles and threat modelling for applications

Most serious software vulnerabilities are not typing mistakes. They are design decisions: an API that trusts an identifier supplied by the caller, a service that runs with far more privilege than it needs, an administrative function reachable from the internet, a workflow that assumes steps will always happen in order. Code review and testing can find such problems, but by then they are expensive to change. This lesson looks at how secure design principles and application threat modelling prevent design flaws from being built in the first place.

CISSP-112 Lesson 1 introduced secure design principles and threat modelling at the level of whole systems and architectures, and CISSP-110 Lesson 12 placed threat modelling in the risk management process. Here we apply those ideas to individual applications and features, the level at which product teams actually work, and we look at how a manager can make threat modelling a routine part of development rather than an occasional exercise.

Classic design principles

In 1975 Jerome Saltzer and Michael Schroeder published a set of design principles for protecting information in computer systems. Half a century later they remain the foundation of secure design, and they appear, sometimes under new names, in every modern framework.

Later practice added principles such as defence in depth (layers of independent controls), secure defaults (the product is secure as shipped, without the customer needing to harden it), minimizing the attack surface (expose as few interfaces, features and privileges as possible), separation of duties, zero trust (never grant implicit trust based on network location) and privacy by design. On the exam, recognize the principle that a scenario illustrates and the one it violates. A service account with database administrator rights used for reading reports violates least privilege; an authorization check performed only on the first page of a multi-step workflow violates complete mediation.

Secure by design and secure by default

Since 2023, government cyber security agencies have pushed software makers to take ownership of their customers' security outcomes. The joint guidance known as Secure by Design, first published in April 2023 by the US Cybersecurity and Infrastructure Security Agency (CISA) with partners including Canada's national cyber security agency and agencies from several allied countries, asks manufacturers to build products that are secure by design (security considered from the earliest stages) and secure by default (the safest settings enabled out of the box, with no extra cost or effort for the customer). It names concrete practices: eliminate default passwords, adopt memory-safe languages, make multifactor authentication standard, provide logs at no extra charge, and publish vulnerability disclosure policies.

In May 2024 CISA launched a voluntary Secure by Design pledge in which software companies committed to measurable progress on goals such as these within a year. The pledge carries no legal force, and its current status should be checked on CISA's website, but it remains a useful benchmark. A manager assessing a vendor, or setting goals for an internal platform, can ask the same questions the pledge asks: do our products ship with default credentials, is multifactor authentication on by default, can customers get security logs without paying more, and do we publish how to report vulnerabilities to us?

What application threat modelling is

Threat modelling is a structured way of answering four questions, as popularized by Adam Shostack: What are we working on? What can go wrong? What are we going to do about it? Did we do a good enough job? For an application, the first question is usually answered with a diagram of the system: its components, the data flows between them, the data stores, the external entities that interact with it, and the trust boundaries where data crosses from a less trusted to a more trusted zone. The second question uses a method such as STRIDE to generate threats systematically. The third turns threats into mitigations, which become requirements, design changes or accepted risks. The fourth checks the model against reality, for example by testing the mitigations.

Data flow diagrams use a small vocabulary: external entities (people or systems outside the team's control), processes (code that transforms data), data stores (databases, files, queues) and data flows (arrows). Trust boundaries are drawn as dashed lines wherever privilege or control changes, such as between the internet and the web tier, between services owned by different teams, or between the company and a third party. Most threats cluster at trust boundaries, because that is where untrusted input enters and where assumptions about identity and authorization are made.

STRIDE per element

STRIDE, developed at Microsoft, is a mnemonic for six categories of threat, each the opposite of a security property. Spoofing violates authenticity; tampering violates integrity; repudiation violates non-repudiation; information disclosure violates confidentiality; denial of service violates availability; elevation of privilege violates authorization. Applying STRIDE per element means asking which categories apply to each kind of element in the diagram. External entities can be spoofed and can repudiate actions; data flows can be tampered with, disclosed or flooded; data stores can be tampered with, disclosed or exhausted, and if they hold logs, repudiation applies; processes are subject to all six.

# STRIDE-per-element threat enumeration with a simple risk score (illustrative)
stride = {
    'external entity': 'SR',
    'process': 'STRIDE',
    'data store': 'TID',
    'data flow': 'TID',
}
names = {'S': 'Spoofing', 'T': 'Tampering', 'R': 'Repudiation',
         'I': 'Information disclosure', 'D': 'Denial of service', 'E': 'Elevation of privilege'}
elements = [
    ('Merchant user', 'external entity'),
    ('Refunds API', 'process'),
    ('Payments database', 'data store'),
    ('Dashboard -> API request', 'data flow'),
]
# team's likelihood and impact estimates (1-3) for selected threats
estimates = {
    ('Merchant user', 'S'): (3, 3), ('Refunds API', 'E'): (2, 3),
    ('Refunds API', 'R'): (2, 2), ('Payments database', 'I'): (1, 3),
    ('Dashboard -> API request', 'T'): (1, 2), ('Refunds API', 'D'): (2, 2),
}
threats = []
for name, kind in elements:
    for c in stride[kind]:
        lik, imp = estimates.get((name, c), (1, 1))
        threats.append((lik * imp, name, names[c]))
print(f'{len(threats)} candidate threats generated')
print('Top five by likelihood x impact:')
for score, name, cat in sorted(threats, reverse=True)[:5]:
    print(f'  {score}  {name:26s} {cat}')

14 candidate threats generated
Top five by likelihood x impact:
  9  Merchant user              Spoofing
  6  Refunds API                Elevation of privilege
  4  Refunds API                Repudiation
  4  Refunds API                Denial of service
  3  Payments database          Information disclosure

The enumeration produces many candidate threats, most of which are already handled by standard controls. The value is in the conversation it forces. For the refund feature, the highest-ranked items are spoofing of the merchant user (stolen credentials or session theft) and elevation of privilege in the refunds API (a user issuing refunds for another merchant, or a support agent exceeding their role). Each leads to concrete mitigations: phishing-resistant multifactor authentication for merchant administrators, object-level authorization on every refund request, and an audit trail that defeats repudiation.

Other methods: PASTA, attack trees and LINDDUN

STRIDE is not the only approach. PASTA (Process for Attack Simulation and Threat Analysis) is a seven-stage, risk-centric method that starts from business objectives and ends with residual risk analysis; it suits organizations that want threat modelling tightly connected to business impact, at the cost of more effort. Attack trees represent an attacker's goal as the root of a tree, with branches showing alternative or combined ways of achieving it; they help analyse which single control would cut off the most paths. LINDDUN focuses on privacy threats such as linkability, identifiability and non-compliance, and pairs well with privacy impact assessments. Teams also use threat libraries and frameworks such as MITRE ATT&CK to check that they have considered relevant adversary behaviour, at a level of description appropriate to defenders.

Rating schemes such as DREAD (damage, reproducibility, exploitability, affected users, discoverability) were once common for scoring threats, but they produce inconsistent results between people and have largely been abandoned, including at Microsoft where DREAD originated. Many teams now use a simple likelihood and impact matrix aligned with the organization's risk management method, or they rate threats with the same severity scale used for vulnerabilities. Consistency with enterprise risk reporting matters more than the particular formula.

Making threat modelling routine

The biggest obstacle to threat modelling is not method but habit. Teams that see it as a long workshop led by outside experts do it rarely, usually too late. Teams that treat it as a short, regular design conversation do it often enough to matter. A practical program sets clear triggers, keeps sessions short and focused, stores models alongside the code so they evolve with it, and tracks mitigations in the same backlog as other work.

Threat models are living documents. A model drawn once at project start and never updated becomes misleading as the system changes. Lesson 12 returns to how a program measures threat modelling coverage, such as the share of services with a model reviewed in the past year.

Design review

A secure design review is an expert evaluation of a proposed design against security principles, standards and the threat model before significant implementation begins. Reviewers look for missing controls (no rate limiting on a public endpoint), misplaced trust (authorization decisions made in the mobile app rather than on the server), unnecessary exposure (an administrative interface on the public internet), weak patterns (custom cryptography or home-grown session management) and gaps between the design and regulatory requirements. Findings are recorded and tracked like any other defect. For a manager, the important point is that design reviews should be scheduled while change is still cheap, which in agile terms means before the work is broken into stories, not after the first release.