Deploying web applications

The wholesale ordering site is a Python web application: Flask code from PYTH-220 that reads and writes the ridgeline_db store. During development Jordan runs it with flask run, which starts a convenient single-process server with automatic reloading and a debugger in the browser. That server is explicitly not for production. It handles one request at a time, its debugger would let anyone who triggered an error run code on the server, and it has none of the safeguards a public website needs. Deploying a web application means putting it behind the right kind of server, in the right arrangement, so it is fast, secure and can be updated without taking it offline.

This lesson covers the interface between Python web frameworks and servers (WSGI and ASGI), production application servers such as Gunicorn and Uvicorn, reverse proxies and load balancers, health checks, static files, database migrations during a release, graceful shutdown and zero-downtime deployments.

WSGI and ASGI: how servers talk to Python

Python web frameworks do not listen on network sockets themselves in production. Instead, a separate server accepts connections, parses HTTP, and calls the application through a standard interface. The long-standing interface is WSGI, the Web Server Gateway Interface (PEP 3333): the application is a callable that receives a dictionary describing the request (the environ) and a start_response function, and returns the response body as an iterable of bytes. Flask and Django applications are WSGI applications, which is why any WSGI server can run them.

import json

def app(environ, start_response):
    """A complete WSGI application: one health endpoint and one stock endpoint."""
    path = environ.get("PATH_INFO", "/")
    if path == "/healthz":
        status, body = "200 OK", {"status": "ok"}
    elif path == "/stock/RC-104":
        status, body = "200 OK", {"sku": "RC-104", "name": "Sumatra Dark", "bags": 9}
    else:
        status, body = "404 Not Found", {"error": "not found"}
    data = json.dumps(body).encode()
    start_response(status, [("Content-Type", "application/json"), ("Content-Length", str(len(data)))])
    return [data]

def call(path):
    """What a WSGI server does for each request, in miniature."""
    captured = {}
    def start_response(status, headers):
        captured["status"], captured["headers"] = status, headers
    body = b"".join(app({"REQUEST_METHOD": "GET", "PATH_INFO": path}, start_response))
    return captured["status"], body.decode()

for path in ["/healthz", "/stock/RC-104", "/admin"]:
    print(path, "->", *call(path))

/healthz -> 200 OK {"status": "ok"}
/stock/RC-104 -> 200 OK {"sku": "RC-104", "name": "Sumatra Dark", "bags": 9}
/admin -> 404 Not Found {"error": "not found"}

ASGI, the Asynchronous Server Gateway Interface, is the newer equivalent for asynchronous frameworks such as FastAPI and Starlette, and for Django's async views. An ASGI application is an async callable that can handle long-lived connections such as WebSockets and many slow requests concurrently. The deployment ideas in this lesson apply to both; only the server program differs.

Application servers

An application server runs the Python application and handles concurrency. For WSGI apps the most common choice on Linux is Gunicorn, which starts a master process and several worker processes, each running a copy of the application; the master restarts workers that crash or hang. Waitress is a pure-Python WSGI server that also runs on Windows. For ASGI apps, Uvicorn and Hypercorn are common, and Gunicorn can manage Uvicorn workers.

How many workers? Each synchronous worker handles one request at a time, so the number of workers sets how many requests are served at once. A widely quoted starting point from Gunicorn's documentation is two workers per CPU core plus one, then adjusting by measurement: more if requests spend most of their time waiting on the database, fewer if memory is tight. In containers, where the platform scales by running more copies, two to four workers per container is typical, and the wholesale site uses two.

def suggested_workers(cpu_cores, memory_mb, mb_per_worker=120):
    """Gunicorn's rule of thumb (2 x cores + 1), capped by available memory."""
    by_cpu = 2 * cpu_cores + 1
    by_memory = max(1, (memory_mb - 100) // mb_per_worker)   # leave room for the master process
    return min(by_cpu, by_memory)

for cores, memory in [(1, 512), (2, 1024), (4, 512), (4, 4096)]:
    print(f"{cores} core(s), {memory:>4} MB -> {suggested_workers(cores, memory)} workers")

1 core(s),  512 MB -> 3 workers
2 core(s), 1024 MB -> 5 workers
4 core(s),  512 MB -> 3 workers
4 core(s), 4096 MB -> 9 workers

> gunicorn --bind 0.0.0.0:8000 --workers 2 --timeout 30 --access-logfile - ridgeline.web:app
[INFO] Starting gunicorn 23.0.0
[INFO] Listening at: http://0.0.0.0:8000
[INFO] Using worker: sync
[INFO] Booting worker with pid: 8
[INFO] Booting worker with pid: 9

The --timeout option matters: a worker that does not answer within 30 seconds is killed and replaced, so one stuck request cannot freeze the site forever. --access-logfile - writes one line per request to standard output, where the platform collects it.

Reverse proxies and load balancers

Application servers are good at running Python and poor at facing the internet directly. In front of them sits a reverse proxy, such as nginx or Caddy on a single server, or a managed load balancer in the cloud. The proxy accepts connections from the internet and forwards requests to one of several application servers. It handles the jobs that do not need Python:

• TLS termination: it holds the certificate, decrypts HTTPS and forwards plain HTTP over the private network.
• Load balancing: it spreads requests across several application containers, and stops sending traffic to any that fail their health checks.
• Buffering and protection: it absorbs slow clients so workers are not tied up, limits request sizes, and can rate-limit abusive traffic.
• Static files: it serves images, stylesheets and scripts directly, or a content delivery network does, without touching Python at all.

Because the proxy forwards requests, the application sees the proxy's address rather than the customer's, and sees plain HTTP rather than HTTPS. Proxies add headers such as X-Forwarded-For and X-Forwarded-Proto with the original values, and frameworks provide middleware (in Flask and Werkzeug, ProxyFix) to trust them. Trust them only from your own proxy, because anyone can send those headers.

Health checks

A load balancer needs to know which containers can take traffic, and a platform needs to know which containers to restart. Both ask through health checks: small HTTP endpoints that answer quickly. Two kinds are common. A liveness check answers whether the process is alive at all; if it fails, the platform restarts the container. A readiness check answers whether the container can serve requests right now, for example whether it has finished starting and can reach the database; if it fails, the load balancer sends traffic elsewhere but leaves the container running.

Health checks should be cheap and should not cascade. If the readiness check of every container fails because the database is briefly slow, the load balancer may take the whole site offline at once, which is worse than serving slowly. Many teams make liveness depend only on the process itself, and readiness on the critical dependencies with short timeouts.

import json, time

def health(checks, timeout_ms=200):
    """Run named dependency checks and summarize them as a readiness response."""
    results, ready = {}, True
    for name, check in checks.items():
        start = time.perf_counter()
        try:
            ok = bool(check())
        except Exception as exc:          # a failing dependency must not crash the endpoint
            ok = False
        ms = (time.perf_counter() - start) * 1000
        results[name] = "ok" if ok and ms <= timeout_ms else "failing"
        ready = ready and results[name] == "ok"
    return (200 if ready else 503), {"status": "ready" if ready else "not ready", "checks": results}

def database():
    return True
def shop_api():
    raise ConnectionError("shop API unreachable")

print(*health({"database": database}), sep="  ")
print(*health({"database": database, "shop_api": shop_api}), sep="  ")

200  {'status': 'ready', 'checks': {'database': 'ok'}}
503  {'status': 'not ready', 'checks': {'database': 'ok', 'shop_api': 'failing'}}

Database migrations during a release

Most releases of the wholesale site change only Python code. Some also change the database schema: a new column for delivery notes, a new table for standing orders. Schema changes are the riskiest part of a deployment, because during a rolling update old and new versions of the code run at the same time against the same database, and the database cannot be in two shapes at once.

The safe technique is called expand and contract. First, expand: add the new column or table in a way the old code ignores, such as a nullable column with no constraint the old code would violate. Deploy the new code, which writes and reads the new column. Later, once no old code is running and existing rows have been filled in, contract: add constraints, and remove columns or tables that are no longer used. Every step is compatible with the version before and after it, so the site never breaks mid-deployment. Migration tools such as Alembic and Django's migrations record which steps have run, and the pipeline runs pending migrations as a separate job before the new containers start.

steps = [
    ("release 0.6", "ALTER TABLE orders ADD COLUMN delivery_note TEXT NULL", "old code ignores the new column"),
    ("release 0.6", "deploy code that writes delivery_note when given", "old and new code both work"),
    ("release 0.7", "UPDATE orders SET delivery_note = '' WHERE delivery_note IS NULL", "backfill existing rows"),
    ("release 0.7", "deploy code that requires delivery_note", "no old code is running now"),
    ("release 0.8", "ALTER TABLE orders ALTER COLUMN delivery_note SET NOT NULL", "contract: tighten the schema"),
]
for release, action, why in steps:
    print(f"{release}: {action}\n             ({why})")

release 0.6: ALTER TABLE orders ADD COLUMN delivery_note TEXT NULL
             (old code ignores the new column)
release 0.6: deploy code that writes delivery_note when given
             (old and new code both work)
release 0.7: UPDATE orders SET delivery_note = '' WHERE delivery_note IS NULL
             (backfill existing rows)
release 0.7: deploy code that requires delivery_note
             (no old code is running now)
release 0.8: ALTER TABLE orders ALTER COLUMN delivery_note SET NOT NULL
             (contract: tighten the schema)

Graceful shutdown and zero-downtime deployment

Containers are stopped all the time: during deployments, when the platform scales down, when a machine is replaced. The platform first sends the process the SIGTERM signal and waits a grace period, often 30 seconds, before forcing it to stop. A well-behaved server stops accepting new connections, finishes the requests already in progress, closes database connections, and exits. Gunicorn does this on SIGTERM by default; custom background workers must handle the signal themselves.

With graceful shutdown and readiness checks in place, a rolling deployment can replace every container without any customer seeing an error. The platform starts a new container, waits until its readiness check passes, adds it to the load balancer, then drains and stops one old container, and repeats. At every moment enough healthy containers are serving traffic. If a new container never becomes ready, the rollout stops and the old version keeps running.

def rolling_update(old, new_ready_after, surge=1):
    """Replace `old` containers one batch at a time; capacity never drops below `old`."""
    serving_old, serving_new, minute, log = old, 0, 0, []
    while serving_old > 0:
        minute += new_ready_after          # start `surge` new containers and wait until ready
        serving_new += surge
        log.append((minute, serving_old + serving_new, f"+{surge} new ready"))
        serving_old -= surge               # then drain and stop old ones
        log.append((minute, serving_old + serving_new, f"-{surge} old drained"))
    return log

for minute, capacity, event in rolling_update(old=3, new_ready_after=2):
    print(f"minute {minute}: {event:<16} serving containers: {capacity}")

minute 2: +1 new ready     serving containers: 4
minute 2: -1 old drained   serving containers: 3
minute 4: +1 new ready     serving containers: 4
minute 4: -1 old drained   serving containers: 3
minute 6: +1 new ready     serving containers: 4
minute 6: -1 old drained   serving containers: 3

Try it yourself

def app(environ, start_response):
    start_response("200 OK", [("Content-Type", "text/plain")])
    return [b"ok"]

def readiness(results):
    return 200, []

def min_capacity(replicas, max_unavailable):
    return replicas

def safe(replicas, max_unavailable, needed):
    return True