Virtual machines and compute services

Compute is the work of running programs: processor time and memory. In the cloud, compute comes in several forms, from virtual machines that behave like complete computers, through containers, to small functions that run for a fraction of a second. This lesson covers virtual machines, the oldest and most flexible form and the foundation that the others are built on. Even if Ridgeline's wholesale application ends up in containers, Sam will run one small virtual machine for the roast logger, and every cloud technician needs to know how to choose, launch, connect to, scale and look after a virtual machine.

Virtualization underneath

A cloud virtual machine, often called an instance, runs on a physical server under the control of a hypervisor. Lesson 1 introduced hypervisors as software that lets one physical server run many virtual machines. There are two kinds. A type 1 hypervisor, also called bare metal, runs directly on the hardware with no operating system underneath; this is what cloud providers use, with examples such as the AWS Nitro system, Microsoft's Hyper-V and the KVM hypervisor used by Google and many others. A type 2 hypervisor runs as an application on an ordinary operating system, such as VirtualBox or VMware Workstation on a laptop, and is used mostly for testing and learning.

The hypervisor divides the physical server's processor cores, memory, storage and network bandwidth among the virtual machines and keeps them isolated from each other. Each virtual machine sees what looks like its own processors, called virtual CPUs or vCPUs, its own memory and its own network card, and runs its own operating system. Modern processors include features that make virtualization efficient, and providers add custom hardware to offload networking and storage work, so a virtual machine today runs nearly as fast as the bare hardware.

Choosing an instance type

Providers sell virtual machines in fixed sizes called instance types at AWS and Google Cloud and VM sizes at Azure. Each type has a set number of vCPUs, an amount of memory, and limits on network and storage performance. Types are grouped into families for different kinds of work. The names are cryptic at first, but they follow patterns: a letter for the family, a number for the generation, and a size word or number.

Burstable instances deserve a special mention because they suit so many small workloads. A burstable instance, such as AWS's t-family, Azure's B-series or Google's shared-core e2 types, has a low baseline level of processor performance and earns credits while it is idle. When work arrives, it spends credits to run faster than the baseline. For a server that is quiet most of the time and busy in short bursts, like the roast logger receiver, this is much cheaper than paying for full-time processor capacity. If the credits run out, performance drops back to the baseline, which is the most common reason a burstable server suddenly becomes slow.

The skill in sizing is to choose the smallest type that does the job, and then measure. Beginners often pick a large instance to be safe; the bill then arrives. A better approach is to start small, watch processor, memory and disk use for a week or two, and resize if needed. In the cloud, resizing a virtual machine usually means a short restart, not a new purchase, so a wrong guess is cheap to correct.

# Pick the cheapest size that meets the requirements (illustrative sizes and prices)
sizes = [
    {"name": "small",   "vcpu": 2, "ram_gb": 2,  "hourly": 0.021},
    {"name": "medium",  "vcpu": 2, "ram_gb": 4,  "hourly": 0.042},
    {"name": "large",   "vcpu": 2, "ram_gb": 8,  "hourly": 0.083},
    {"name": "xlarge",  "vcpu": 4, "ram_gb": 16, "hourly": 0.166},
    {"name": "c-large", "vcpu": 4, "ram_gb": 8,  "hourly": 0.150},
]

def cheapest(sizes, vcpu, ram_gb):
    fits = [s for s in sizes if s["vcpu"] >= vcpu and s["ram_gb"] >= ram_gb]
    return min(fits, key=lambda s: s["hourly"]) if fits else None

for need in [(1, 2), (2, 6), (4, 8), (8, 32)]:
    s = cheapest(sizes, *need)
    if s:
        print(f"{need[0]} vCPU, {need[1]} GB -> {s['name']}, about {s['hourly'] * 730:.2f} dollars a month")
    else:
        print(f"{need[0]} vCPU, {need[1]} GB -> no size fits")

1 vCPU, 2 GB -> small, about 15.33 dollars a month
2 vCPU, 6 GB -> large, about 60.59 dollars a month
4 vCPU, 8 GB -> c-large, about 109.50 dollars a month
8 vCPU, 32 GB -> no size fits

Cloud providers usually quote compute prices per hour or per second and assume 730 hours in an average month, which is 365 days times 24 hours divided by 12. Remember that number; it appears in almost every cloud cost estimate.

sizes = [
    {"name": "small", "vcpu": 2, "ram_gb": 2, "hourly": 0.021},
    {"name": "medium", "vcpu": 2, "ram_gb": 4, "hourly": 0.042},
    {"name": "large", "vcpu": 2, "ram_gb": 8, "hourly": 0.083},
    {"name": "c-large", "vcpu": 4, "ram_gb": 8, "hourly": 0.150},
    {"name": "xlarge", "vcpu": 4, "ram_gb": 16, "hourly": 0.166},
]

def cheapest(sizes, vcpu, ram_gb):
    return None

def monthly(size_name, sizes):
    return 0

print(cheapest(sizes, 2, 3), monthly("medium", sizes))

Images and launching an instance

Every virtual machine starts from an image: a template containing an operating system and sometimes preinstalled software. AWS calls these Amazon Machine Images, or AMIs; Azure and Google Cloud call them images. Providers publish official images for popular operating systems such as Ubuntu, Debian, Red Hat Enterprise Linux, Amazon Linux and Windows Server, and organizations often build their own images with their software and security settings baked in, so that new servers start ready to work.

Launching an instance means making a handful of decisions: the image, the instance type, the region and zone, the virtual network and subnet, the security rules that control which traffic can reach it, the size and type of its disk, and how administrators will sign in. Many providers also let you pass a startup script, called user data at AWS, custom data at Azure and a startup script at Google Cloud, which runs on first boot to install software or apply settings. Startup scripts are an early step toward the automation covered in lesson 11.

# Launching a small Ubuntu server for the roast logger (listing only; needs a cloud account)

# AWS
aws ec2 run-instances --image-id ami-0abc1234example --instance-type t3.micro \
    --subnet-id subnet-0abc1234 --key-name ridgeline-admin --tag-specifications \
    "ResourceType=instance,Tags=[{Key=Name,Value=roast-logger}]"

# Azure
az vm create --resource-group ridgeline --name roast-logger --image Ubuntu2404 \
    --size Standard_B1s --admin-username sam --generate-ssh-keys

# Google Cloud
gcloud compute instances create roast-logger --machine-type=e2-micro \
    --image-family=ubuntu-2404-lts-amd64 --image-project=ubuntu-os-cloud

Connecting and administering

Linux virtual machines are normally administered over SSH, using a key pair rather than a password: the public key is placed on the server at launch, and the administrator keeps the private key. Windows servers use Remote Desktop. The safest practice, which Sam insists on, is not to expose SSH or Remote Desktop to the whole internet. Instead, administrators connect through a provider's browser-based or agent-based session service, such as AWS Systems Manager Session Manager, Azure Bastion or Google Cloud's Identity-Aware Proxy, or through a VPN, so the administrative ports stay closed to everyone else. Scanning bots try to log in to every open SSH port on the internet within minutes of it appearing.

Once connected, administration is the same as on any server, which is why NET-103 on Linux administration matters for cloud work. Somebody must apply operating system updates, ideally on a schedule using the provider's patch management service, watch disk space and logs, and keep the software current. Under the IaaS model, that somebody is the customer.

Pricing models

The default way to pay for a virtual machine is on demand: pay per second or per hour while it runs, with no commitment. On-demand pricing is the most flexible and the most expensive per hour. The providers offer three main ways to pay less.

The art is to mix them: commitments for the base load that never goes away, on demand for normal variation, and spot capacity for work that can be interrupted and restarted. Spot capacity is unsuitable for anything that must stay up, such as a web server holding customer sessions, because the provider can take the machine back with as little as two minutes' notice at AWS and thirty seconds at Azure and Google Cloud.

Stopping a virtual machine stops the compute charge, but not every charge: its disk, and any reserved public IP address, keep costing money until they are deleted. A stopped server that nobody needs should be deleted, after checking that its data is backed up.

Scaling and load balancing

There are two ways to give an application more capacity. Vertical scaling, or scaling up, means moving to a bigger instance type. It is simple, but it has a ceiling, usually needs a restart, and still leaves a single point of failure. Horizontal scaling, or scaling out, means adding more instances of the same size and spreading the work among them. It has almost no ceiling, survives the loss of an instance, and is the way cloud applications are normally designed.

Horizontal scaling needs two services. A load balancer receives all incoming requests and passes each one to a healthy instance, checking health regularly and stopping traffic to any instance that fails. The providers' load balancers include AWS Elastic Load Balancing, Azure Load Balancer and Application Gateway, and Google Cloud Load Balancing. An autoscaling group then adds or removes instances automatically according to rules, such as keeping average processor use near half. AWS calls these Auto Scaling groups, Azure calls them Virtual Machine Scale Sets, and Google Cloud calls them managed instance groups. Each group has a minimum, a maximum and a desired number of instances, and launches new ones from a template.

The rules that drive autoscaling need care. Scale out too eagerly and costs jump with every small spike; scale in too eagerly and instances are removed just before the next spike. Most rules therefore use a threshold to add instances, a lower threshold to remove them, and a cooldown period during which no further change is made. The listing below simulates a simple rule over a busy morning.

def simulate(cpu_readings, start=2, minimum=2, maximum=6, high=70, low=30):
    count, history = start, []
    for cpu in cpu_readings:
        if cpu > high and count < maximum:
            count += 1
        elif cpu < low and count > minimum:
            count -= 1
        history.append(count)
    return history

cpu = [25, 40, 75, 82, 78, 65, 50, 35, 28, 22, 20]
print("CPU readings:", cpu)
print("Instances:   ", simulate(cpu))

CPU readings: [25, 40, 75, 82, 78, 65, 50, 35, 28, 22, 20]
Instances:    [2, 2, 3, 4, 5, 5, 5, 5, 4, 3, 2]

def simulate(readings, start, minimum, maximum, high, low, cooldown):
    count, history = start, []
    for r in readings:
        if r > high and count < maximum:
            count += 1
        elif r < low and count > minimum:
            count -= 1
        history.append(count)
    return history

print(simulate([80, 80, 80, 80, 20, 20, 20], 2, 2, 6, 70, 30, 1))