Concurrency I: threads, processes and shared state

A concurrent program has several activities in progress at the same time. Concurrency lets a server handle many clients at once, keeps a user interface responsive while work happens in the background, and uses the several cores of a modern processor. It also introduces a new class of bugs, because the order in which the activities' steps interleave is no longer fixed by the program text. This lesson introduces the vocabulary of concurrency and parallelism, the two basic units of concurrent execution, threads and processes, and the central difficulty of shared mutable state: the race condition. Lesson 10 covers the tools for controlling races. How an operating system schedules threads and implements these mechanisms is the subject of CMPUT-379; here we take the programmer's view.

Concurrency and parallelism

Concurrency and parallelism are related but different. Concurrency is about structure: a program is concurrent if it is organized as several independent activities whose lifetimes overlap. Parallelism is about execution: steps literally run at the same instant on different processors. A concurrent program on a single core is not parallel, because the core switches between activities, interleaving their steps. A parallel program is always concurrent. Rob Pike's summary is useful: concurrency is dealing with many things at once; parallelism is doing many things at once.

Work is also either CPU-bound or I/O-bound. A CPU-bound task, such as compressing a video or multiplying matrices, keeps a processor busy computing; it benefits from parallelism across cores. An I/O-bound task, such as downloading web pages or querying a database, spends most of its time waiting for a device or network; it benefits from concurrency, because while one task waits, another can run, even on one core. Matching the tool to the kind of work is the first design decision in any concurrent program.

Threads and processes

A process is a running program with its own address space: its own memory, open files and resources, isolated from other processes by the operating system. A thread is a flow of control inside a process. All threads in a process share the same memory, so they can communicate cheaply by reading and writing the same objects, but they can also interfere with each other. Creating a process costs more than creating a thread, and processes communicate only through explicit channels such as pipes, sockets or shared memory segments, but a crash or memory corruption in one process cannot directly damage another.

import threading

results = {}

def worker(name, n):
    total = sum(i * i for i in range(n))      # some work
    results[name] = total                      # each thread writes its own key

threads = [threading.Thread(target=worker, args=(f"t{k}", 10_000 * (k + 1))) for k in range(4)]
for t in threads:
    t.start()                                  # begin running concurrently
for t in threads:
    t.join()                                   # wait for each to finish
for name in sorted(results):
    print(name, results[name])
print("threads alive after join:", sum(t.is_alive() for t in threads))

t0 333283335000
t1 2666466670000
t2 8999550005000
t3 21332533340000
threads alive after join: 0

`start` begins a thread and returns immediately; `join` waits for it to finish. The threads here avoid interference because each writes a different key, and the dictionary operations they use are atomic in CPython. Most real problems are not so tidy: threads need to update shared counters, append to shared lists or modify shared objects, and that is where races appear.

Race conditions

A race condition occurs when the result of a program depends on the timing or interleaving of operations in different threads. The classic example is incrementing a shared counter. The statement `count += 1` looks like one step but is really three: read the current value, add one, and write the result back. If two threads both read the value 5 before either writes, both write 6, and one increment is lost. Whether this happens depends on exactly when the scheduler switches threads, so the bug may appear once in a million runs, only under load, or only on a particular machine.

import itertools

def run(schedule):
    """Simulate two threads each doing count += 1 as read / add / write steps.
    schedule lists which thread takes the next step."""
    count = 0
    regs = {"A": None, "B": None}
    steps = {"A": iter(["read", "add", "write"]), "B": iter(["read", "add", "write"])}
    for who in schedule:
        op = next(steps[who])
        if op == "read":  regs[who] = count
        elif op == "add": regs[who] += 1
        else:             count = regs[who]
    return count

orders = sorted(set(itertools.permutations("AAABBB")))
outcomes = {}
for o in orders:
    outcomes.setdefault(run(o), []).append("".join(o))
for result, scheds in sorted(outcomes.items()):
    print(f"final count {result}: {len(scheds):2d} of {len(orders)} interleavings, e.g. {scheds[0]}")

final count 1: 18 of 20 interleavings, e.g. AABABB
final count 2:  2 of 20 interleavings, e.g. AAABBB

The simulation enumerates all 20 ways to interleave the two threads' three steps. Only the interleavings in which one thread finishes its write before the other reads give the correct answer, 2; in all the others one update is lost. Real threads are not this tidy, because the scheduler decides the interleaving, but the possible outcomes are exactly these. A test that runs the real program a few times may see only the correct result and give false confidence. This is why reasoning about interleavings, rather than testing alone, is essential for concurrent code.

The global interpreter lock

Standard CPython has a global interpreter lock, the GIL, which allows only one thread at a time to execute Python bytecode. The GIL protects the interpreter's internal data, such as reference counts, and makes some operations, like appending to a list, atomic. It does not make your code thread-safe: a switch can still happen between the read and the write of `count += 1`. And it prevents threads from running Python code in parallel, so CPU-bound Python code gets no speed-up from threads on a multicore machine. Threads still help I/O-bound code, because the GIL is released while a thread waits for I/O, and many libraries written in C, such as NumPy, release it during long computations.

This is changing. PEP 703 introduced an optional free-threaded build of CPython without the GIL, available experimentally in Python 3.13 (October 2024), and PEP 779 made it an officially supported, though still optional, build from Python 3.14 (October 2025). In the free-threaded build, CPU-bound threads can run in parallel, at some cost in single-threaded speed and with some extension libraries not yet compatible. The default build still has the GIL as of 2026. Either way, races in your own code remain your responsibility: removing the GIL makes correct synchronization more important, not less.

Processes for CPU-bound parallelism

To use several cores for CPU-bound Python code in the standard build, use processes. The `multiprocessing` module and `concurrent.futures.ProcessPoolExecutor` start worker processes, each with its own interpreter and its own GIL, and send them work. Arguments and results travel between processes by pickling, Python's serialization format, so they must be picklable and should be reasonably small; sending large data structures back and forth can cost more than the computation saves.

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

def count_primes(bounds):
    lo, hi = bounds
    def is_prime(n):
        if n < 2: return False
        i = 2
        while i * i <= n:
            if n % i == 0: return False
            i += 1
        return True
    return sum(1 for n in range(lo, hi) if is_prime(n))

chunks = [(k * 50_000, (k + 1) * 50_000) for k in range(4)]   # split 0..200,000 into 4 parts

with ProcessPoolExecutor(max_workers=4) as pool:               # one interpreter per worker
    parts = list(pool.map(count_primes, chunks))
print("per chunk:", parts, "total:", sum(parts))

with ThreadPoolExecutor(max_workers=4) as pool:                # same answer, but no CPU speed-up under the GIL
    print("threads agree:", sum(pool.map(count_primes, chunks)) == sum(parts))

per chunk: [5133, 4459, 4256, 4136] total: 17984
threads agree: True

`pool.map` returns results in the same order as the inputs, regardless of which worker finishes first, so the output is deterministic even though the work runs in parallel. Dividing work into independent chunks, processing them separately, and combining the results is the map–reduce pattern from lesson 2, applied to parallel execution. It works best when the chunks share no mutable state, which is exactly the property pure functions provide: a pure function can run anywhere, in any order, in any process.

Shared state is the root of the problem

Every race condition needs two ingredients: state that is shared between threads, and at least one thread that modifies it. Remove either ingredient and races disappear. Immutable data, from lesson 3, can be shared freely, because nobody can change it. Unshared mutable data, owned by one thread, is safe, because only its owner touches it. The remaining case, shared mutable state, needs synchronization, which lesson 10 covers. This is why the functional paradigm's emphasis on immutability and the actor model's emphasis on message passing are so valuable for concurrent programs: they design away the dangerous quadrant.

Rust enforces this table at compile time. Its ownership rules allow either one mutable reference to a value or any number of read-only references, never both. A value can be sent to another thread only if its type is marked `Send`, and shared between threads only if it is `Sync`. The program below shares a counter between threads safely by wrapping it in `Arc`, an atomically reference-counted pointer, and `Mutex`, a lock; trying to mutate a plain shared integer from several threads would not compile.

use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
    let counter = Arc::new(Mutex::new(0u64));
    let mut handles = Vec::new();
    for _ in 0..4 {
        let c = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..100_000 {
                *c.lock().unwrap() += 1;      // lock, increment, unlock automatically
            }
        }));
    }
    for h in handles {
        h.join().unwrap();
    }
    println!("count = {}", *counter.lock().unwrap());
}

count = 400000