Security capabilities of information systems: the TCB, memory protection, TPMs and HSMs

Sub-domain 3.4 asks you to understand the security capabilities of information systems. Underneath every operating system, database and application lies hardware and low-level software that either enforces separation or does not. If a program can read another program's memory, or the operating system can be replaced before it starts, no amount of careful application code will keep data safe. This lesson covers the foundations: the trusted computing base and reference monitor, processor modes and memory protection, firmware and secure boot, the trusted platform module, hardware security modules, virtualization and newer confidential computing features.

The trusted computing base and the reference monitor

The trusted computing base, or TCB, is the combination of hardware, firmware and software that enforces a system's security policy. Everything inside the TCB must work correctly for the system to be secure; everything outside it is treated as untrusted. Designers try to keep the TCB as small as possible, because a smaller TCB is easier to review and test. The boundary between the TCB and the rest of the system is called the security perimeter, and communication across it happens through carefully controlled interfaces.

The reference monitor is an abstract concept: a component that mediates every access by a subject to an object and decides whether to allow it according to the security policy. To be trustworthy, a reference monitor must meet three requirements, sometimes remembered by the acronym NEAT minus one letter or simply as the three Ts: it must be always invoked, so no access can bypass it; tamperproof, so it cannot be modified by untrusted code; and small enough to be verified, so it can be analysed and tested completely. The security kernel is the actual hardware, firmware and software that implements the reference monitor concept. When an exam question asks which component enforces access decisions for every request, the answer is the reference monitor, implemented by the security kernel.

Processor modes, rings and process isolation

Modern processors support at least two privilege levels. Kernel mode, also called supervisor or privileged mode, can execute every instruction and access all memory; user mode cannot execute privileged instructions and can access only memory it has been given. The x86 architecture defines four protection rings, numbered 0 to 3, with ring 0 for the kernel and ring 3 for applications; most operating systems use only rings 0 and 3. When an application needs a privileged service, such as reading a file, it makes a system call, which switches the processor into kernel mode in a controlled way and back again afterwards. Hypervisors add a further level, sometimes called ring minus 1, below the guest operating systems.

Process isolation ensures that one process cannot interfere with another. Each process runs in its own virtual address space, mapped by the memory management unit to physical memory, so a pointer in one process cannot reach memory belonging to another. Further memory protections make attacks harder. Address space layout randomization, ASLR, places code and data at unpredictable addresses so that an attacker cannot easily jump to known locations. Data execution prevention, also known as the no-execute or NX bit, marks data pages as non-executable, blocking many buffer overflow attacks that inject code into the stack or heap. Stack canaries detect overwritten return addresses, and control-flow integrity features in recent processors check that indirect jumps and returns go only to valid targets.

Hardware isolation is not perfect. The Spectre and Meltdown vulnerabilities, disclosed in January 2018, showed that speculative execution in processors could leak data across these boundaries through cache timing side channels. Fixes required changes to operating systems, browsers, compilers and processor microcode, and some cost performance. Since then many further transient-execution flaws have been found. The lesson for architects is that even the TCB depends on assumptions that can fail, so defence in depth and prompt patching of firmware and microcode matter.

Firmware, secure boot and measured boot

Before the operating system loads, firmware initializes the hardware. Most current systems use the Unified Extensible Firmware Interface, UEFI, which replaced the older BIOS. Firmware is a prime target because malware that infects it, often called a bootkit or rootkit, runs before any security software and can survive reinstalling the operating system. Secure boot defends against this by having each stage verify the digital signature of the next before running it: the firmware checks the boot loader, the boot loader checks the kernel, and so on. Code that is not signed by a trusted key is refused. Measured boot complements secure boot by recording a cryptographic hash of each component into a tamper-resistant log, so that a remote system can later check, through attestation, exactly what was loaded.

The trusted platform module

A trusted platform module, or TPM, is a small secure cryptoprocessor, specified by the Trusted Computing Group and standardized as ISO/IEC 11889, built into most business computers either as a separate chip or as firmware within the processor. TPM 2.0 is the current version and is required by Windows 11. A TPM can generate and store keys that never leave the chip, perform cryptographic operations, provide a hardware random number generator and hold platform configuration registers, which record measurements during measured boot. Keys can be sealed to a particular platform state, so they are released only if the measured boot chain matches expected values. Full-disk encryption products such as BitLocker use this: the disk unlocks automatically only if the boot chain has not been tampered with, often combined with a PIN for stronger protection. The TPM's endorsement key, burned in at manufacture, gives each chip a unique identity that supports device attestation.

Hardware security modules

A hardware security module, or HSM, is a dedicated, tamper-resistant device that generates, stores and uses cryptographic keys for servers and applications. Where a TPM protects a single computer, an HSM serves many systems: certificate authorities protect their signing keys in HSMs, banks use them for card PINs and payment processing, and cloud providers offer HSM-backed key management services. HSMs resist physical tampering, often wiping keys if their casing is opened, and enforce role-based access with dual control for sensitive operations such as exporting a key backup. Their strength is usually certified under FIPS 140-3, whose four security levels range from Level 1, basic requirements with no physical security, to Level 4, which requires detection and response to environmental and physical attacks. Payment HSMs are also certified under the PCI PTS HSM standard.

Virtualization and confidential computing

Virtualization lets one physical machine run many virtual machines, each believing it has its own hardware. The hypervisor, which enforces isolation between them, is part of the TCB for every virtual machine on the host. A type 1 or bare-metal hypervisor runs directly on the hardware and is used in data centres and clouds; a type 2 or hosted hypervisor runs on top of a normal operating system and is common on desktops. The main risks are virtual machine escape, where code in a guest breaks out to the hypervisor or other guests, virtual machine sprawl, where unmanaged machines accumulate, and weak separation of management networks. Hardware features such as Intel VT-x and AMD-V, and input-output memory management units that stop devices accessing memory they should not, make isolation stronger.

Confidential computing extends protection to data in use. Trusted execution environments, such as Intel Software Guard Extensions enclaves, Intel Trust Domain Extensions, AMD Secure Encrypted Virtualization and Arm TrustZone, keep code and data encrypted in memory and isolated even from the operating system and hypervisor. Remote attestation lets a customer verify that the expected code is running inside a genuine enclave before sending it secrets. This allows sensitive workloads to run in a public cloud with less trust in the provider, although side-channel research shows that these protections also have limits.

Thinking like a manager

Questions on system capabilities often test whether you can name the component that provides a protection. Access decisions on every request point to the reference monitor; storing a single device's disk key bound to its boot state points to the TPM; protecting a certificate authority's private key points to an HSM; blocking injected code from running on the stack points to data execution prevention. When asked how to respond to a hardware flaw such as Spectre, the managerial answer combines vendor microcode and operating system patches with defence in depth, not replacing every computer overnight.