Asynchronous, event-driven and other paradigms

The paradigms so far organize programs around steps, functions, objects or concurrent workers. This lesson covers four more ways of thinking that are widely used today. Event-driven programming structures a program as handlers that run when something happens. Asynchronous programming, with Python's `async` and `await`, lets one thread juggle thousands of waiting tasks. Logic programming describes what is true and lets a search engine find answers. Dataflow and reactive programming describe values that update automatically when their inputs change. Each is a different answer to the question of what drives a computation forward: the program counter, an event, a query or a change in data.

Event-driven programming

In an event-driven program, the main loop belongs to a framework, not to you. The framework waits for events, such as a mouse click, a key press, an arriving network packet or a timer expiring, and calls the handler you registered for each one. Your code is a set of callbacks. Graphical user interfaces, web servers, games and embedded controllers all work this way. The observer pattern from lesson 6 is the building block: event sources are subjects, and handlers are observers.

import heapq

class EventLoop:
    """A tiny event loop: handlers are registered per event type; events are processed in time order."""
    def __init__(self):
        self.handlers, self.queue, self.now, self.seq = {}, [], 0.0, 0
    def on(self, kind, handler):
        self.handlers.setdefault(kind, []).append(handler)
    def emit(self, kind, at=None, **data):
        self.seq += 1
        heapq.heappush(self.queue, (self.now if at is None else at, self.seq, kind, data))
    def run(self):
        while self.queue:
            self.now, _, kind, data = heapq.heappop(self.queue)
            for h in self.handlers.get(kind, []):
                h(self, **data)

loop = EventLoop()
loop.on("click", lambda lp, x, y: print(f"t={lp.now:.1f} click at ({x}, {y})"))
loop.on("click", lambda lp, x, y: x > 100 and lp.emit("open_menu", at=lp.now + 0.2))
loop.on("open_menu", lambda lp: print(f"t={lp.now:.1f} menu opened"))
loop.on("tick", lambda lp, n: print(f"t={lp.now:.1f} tick {n}"))

loop.emit("tick", at=1.0, n=1)
loop.emit("click", at=0.5, x=40, y=10)
loop.emit("click", at=1.5, x=150, y=20)
loop.emit("tick", at=2.0, n=2)
loop.run()

t=0.5 click at (40, 10)
t=1.0 tick 1
t=1.5 click at (150, 20)
t=1.7 menu opened
t=2.0 tick 2

Two features of event-driven code deserve attention. Handlers must return quickly, because while one runs, no other event is processed; a slow handler freezes a user interface. And control flow is inverted: you do not call the framework, it calls you, sometimes called the Hollywood principle, don't call us, we'll call you. The second click handler shows a consequence: a handler can schedule further events, so the program's behavior emerges from chains of events rather than from a single readable sequence. Long chains of callbacks nested inside callbacks, known as callback hell, motivated the asynchronous syntax in the next section.

Asynchronous programming with async and await

Asynchronous programming lets one thread run many tasks that spend most of their time waiting, typically for network or disk I/O. In Python, a function defined with `async def` is a coroutine. Inside it, `await` marks a point where the coroutine may pause while it waits for something, letting the event loop run other coroutines in the meantime. When the awaited operation completes, the coroutine resumes where it left off. The code reads like ordinary sequential code, but the waits overlap.

import asyncio, time

async def fetch(name, delay):
    print(f"start {name}")
    await asyncio.sleep(delay)          # stands in for a network request; the loop runs others meanwhile
    print(f"done  {name}")
    return f"{name}:{int(delay * 1000)}ms"

async def main():
    t0 = time.perf_counter()
    results = await asyncio.gather(     # run three coroutines concurrently
        fetch("weather", 0.30), fetch("news", 0.10), fetch("stocks", 0.20))
    elapsed = time.perf_counter() - t0
    print(results)
    print("total waiting 0.60 s; elapsed under 0.45 s:", elapsed < 0.45)

asyncio.run(main())

start weather
start news
start stocks
done  news
done  stocks
done  weather
['weather:300ms', 'news:100ms', 'stocks:200ms']
total waiting 0.60 s; elapsed under 0.45 s: True

All three coroutines start immediately, and each finishes in order of its delay, so the whole run takes about as long as the slowest request rather than the sum of all three. `asyncio.gather` returns the results in the order the coroutines were passed, regardless of finishing order. Everything ran on one thread, so there was no parallelism and no possibility of the read–add–write races from lesson 9 in between `await` points: a coroutine is only ever interrupted where it says `await`.

That last property is both the strength and the weakness of asynchronous code. It is a form of cooperative multitasking: tasks yield control voluntarily at `await`. Shared state is safe between awaits, which simplifies reasoning. But a coroutine that computes for a long time without awaiting, or that calls a blocking function such as `time.sleep` or a synchronous database driver, blocks the entire event loop and every other task. CPU-heavy work must be moved to a thread or process pool, for example with `asyncio.to_thread` or `loop.run_in_executor`.

Logic programming

Logic programming describes a problem as facts and rules and asks questions about them. You do not say how to find an answer; the system searches for values that make the query true. Prolog, created in 1972 by Alain Colmerauer and Philippe Roussel, is the classic language. A Prolog program might state the facts parent(ana, ben) and parent(ben, cy), and the rule that X is a grandparent of Z if X is a parent of some Y and Y is a parent of Z. The query grandparent(ana, Who) then returns Who = cy. The same rule answers grandparent(Who, cy) as well: relations can be queried in any direction, which functions cannot do.

Two mechanisms do the work. Unification matches a query against facts and rule heads, binding variables so that both sides become equal. Backtracking explores alternatives: when one choice leads to a dead end, the system undoes its bindings and tries the next possibility, a depth-first search through the space of proofs. The program below implements both in a few dozen lines of Python, enough to answer queries about a small family tree.

def is_var(t): return isinstance(t, str) and t[0].isupper()   # variables start with a capital

def walk(t, s):
    while is_var(t) and t in s:
        t = s[t]
    return t

def unify(a, b, s):
    a, b = walk(a, s), walk(b, s)
    if a == b: return s
    if is_var(a): return {**s, a: b}
    if is_var(b): return {**s, b: a}
    if isinstance(a, tuple) and isinstance(b, tuple) and len(a) == len(b):
        for x, y in zip(a, b):
            s = unify(x, y, s)
            if s is None: return None
        return s
    return None

FACTS = [("parent", "ana", "ben"), ("parent", "ana", "dee"),
         ("parent", "ben", "cy"), ("parent", "dee", "eli"), ("parent", "dee", "fay")]
RULES = [(("grandparent", "X", "Z"), [("parent", "X", "Y"), ("parent", "Y", "Z")]),
         (("sibling", "A", "B"), [("parent", "P", "A"), ("parent", "P", "B"), ("neq", "A", "B")])]

counter = [0]
def rename(term):                                   # fresh variables for each rule use
    if is_var(term): return f"{term}_{counter[0]}"
    if isinstance(term, tuple): return tuple(rename(t) for t in term)
    return term

def solve(goals, s):
    if not goals:
        yield s; return
    goal, rest = goals[0], goals[1:]
    if goal[0] == "neq":
        if walk(goal[1], s) != walk(goal[2], s):
            yield from solve(rest, s)
        return
    for fact in FACTS:                              # try every fact: backtracking
        s2 = unify(goal, fact, s)
        if s2 is not None:
            yield from solve(rest, s2)
    for head, body in RULES:                        # try every rule
        counter[0] += 1
        head, body = rename(head), [rename(g) for g in body]
        s2 = unify(goal, head, s)
        if s2 is not None:
            yield from solve(body + rest, s2)

def ask(query, var):
    return sorted({walk(var, s) for s in solve([query], {})})

print("grandchildren of ana:", ask(("grandparent", "ana", "Who"), "Who"))
print("grandparent of fay: ", ask(("grandparent", "Who", "fay"), "Who"))
print("siblings of eli:    ", ask(("sibling", "eli", "S"), "S"))

grandchildren of ana: ['cy', 'eli', 'fay']
grandparent of fay:  ['ana']
siblings of eli:     ['fay']

The same two rules answered three different kinds of question: who are ana's grandchildren, who is fay's grandparent, and who are eli's siblings. The engine is a generator, so it produces answers lazily, one per successful proof, exactly as lesson 3's generators produced values on demand. Logic programming lives on in database query languages such as Datalog, in type checkers, which solve constraints about types, in build systems and package managers, which resolve dependency constraints, and in constraint solvers used for scheduling and planning.

Dataflow and reactive programming

In the dataflow paradigm, a program is a graph of values connected by dependencies, and a value is recomputed whenever one of its inputs changes. Spreadsheets, from lesson 1, are the most familiar example. Reactive programming applies the same idea to streams of events over time: a stream of mouse positions, of stock prices or of search keystrokes can be filtered, mapped, combined and throttled with operators much like map and filter, and subscribers receive the results as they happen. Libraries such as RxJS and Reactor, and user-interface frameworks such as React and SwiftUI, apply these ideas, and the signals used in many recent JavaScript frameworks are fine-grained reactive values.

class Stream:
    """A tiny push-based reactive stream."""
    def __init__(self):
        self.subs = []
    def subscribe(self, fn):
        self.subs.append(fn); return self
    def push(self, value):
        for fn in self.subs: fn(value)
    def map(self, f):
        out = Stream(); self.subscribe(lambda v: out.push(f(v))); return out
    def filter(self, p):
        out = Stream(); self.subscribe(lambda v: p(v) and out.push(v)); return out
    def distinct(self):
        out = Stream(); last = [object()]
        def on(v):
            if v != last[0]:
                last[0] = v; out.push(v)
        self.subscribe(on); return out

keys = Stream()
queries = (keys.map(str.strip).map(str.lower)
               .filter(lambda q: len(q) >= 3)
               .distinct())
queries.subscribe(lambda q: print("search for:", repr(q)))

for typed in ["p", "pa", "par", "para", "Para ", "parad", "parad"]:
    keys.push(typed)                         # each keystroke is an event

search for: 'par'
search for: 'para'
search for: 'parad'

The pipeline reads as a declarative description of when to search: normalize the text, ignore queries under three characters, and skip repeats. Seven keystrokes produce three searches. A full reactive library would also offer time-based operators, such as debounce, which waits until the user pauses typing, and operators that combine several streams. Reactive code unites three earlier ideas: the observer pattern supplies the push mechanism, functional operators supply the transformations, and the event loop supplies the timing.