Supervised, unsupervised and reinforcement learning

At the second AI hour, Noor brought a printout of Jonah's own monthly booking report and circled a column he had added years earlier: Missed? Yes or No. "This column," she said, "is the most valuable thing in the building for machine learning. It is a label. Every appointment in the last three years has an answer attached to it. That is what makes the no-show idea possible."

Machine learning methods are grouped by what kind of answer, if any, the training data contains. The three groups the exam expects you to know are supervised learning, unsupervised learning and reinforcement learning. Knowing which group a problem belongs to is often the first and most useful decision in a project, because it tells you what data you need.

Supervised learning

In supervised learning, every training example comes with the correct answer, called the label or target. The model learns the relationship between the inputs, called features, and the label, so that it can predict the label for new examples where the answer is not yet known. The word supervised refers to those answers: the data supervises the learning by telling the algorithm when it is right.

Supervised problems come in two main kinds, and the exam often asks you to tell them apart.

• Classification predicts a category. Will this appointment be missed: yes or no? Is this review positive, negative or neutral? Is this image of a cat or a dog? When there are two categories it is binary classification; with more, it is multiclass classification.
• Regression predicts a number on a continuous scale. How many appointments will the Lethbridge clinic have next Tuesday? How long will this surgery take, in minutes? What will next month's drug order cost?

A quick test helps: if the answer is a name from a fixed list, it is classification; if it is a quantity that could take many values, it is regression. Forecasting, predicting future values in a sequence over time, is a special kind of regression that uses the order of the data.

The no-show idea is binary classification. The features are things known when the booking is made: how many days ahead it was booked, the day of the week and time, the clinic, the type of visit, whether the client has missed before, whether they confirmed the reminder. The label is whether the appointment was missed. Lesson 3 covers which of those features are safe and fair to use.

Unsupervised learning

In unsupervised learning, the data has no labels. The algorithm looks for structure on its own. The most common unsupervised task is clustering: grouping similar items together. Others include dimensionality reduction, which compresses many features into a few that capture most of the variation, and anomaly detection, which flags items unlike the rest.

Noor gave an example from Lorraine Beaudry's world. The finance team wanted to understand its client households without imposing categories in advance. A clustering algorithm run on visit frequency, spending, number of pets and services used might reveal groups such as "one older dog, frequent visits", "several cats, wellness only" and "emergency-only clients". Nobody labeled those groups; the algorithm found them, and people then had to decide whether they were meaningful. That last step matters: clustering always produces groups, even when the data has no real structure, so the output needs human judgment.

Anomaly detection is often unsupervised too. An algorithm that learns what normal drug-dispensing patterns look like can flag a clinic whose controlled-substance counts suddenly drift, without anyone having labeled examples of diversion. The flag is a prompt for a person to look, not a conclusion.

Semi-supervised and self-supervised learning

Between the two sit methods that use a small amount of labeled data with a large amount of unlabeled data, called semi-supervised learning. It helps when labels are expensive, such as when a vet must review each radiograph to mark it as acceptable or not. Self-supervised learning creates its own labels from the data: hiding a word in a sentence and asking the model to predict it, for example. That is how large language models are pre-trained on huge amounts of text, which AWS-112 explains.

Reinforcement learning

In reinforcement learning (RL), an agent learns by trial and error in an environment. At each step it observes the current state, takes an action, and receives a reward or penalty. Over many attempts it learns a policy, a strategy for choosing actions that maximize the total reward over time. There are no labeled answers; the only feedback is the reward signal.

Reinforcement learning powers game-playing systems, robot control and some recommendation and pricing systems. It needs an environment where the agent can try many actions safely, often a simulation. That makes it a poor fit for most clinic problems: you cannot let a system experiment with real appointment schedules for thousands of trials and learn from the failures. A related technique, reinforcement learning from human feedback (RLHF), uses human ratings of model answers as the reward, and it is one of the steps used to make large language models more helpful and safer. You will meet it again in AWS-112.

Matching techniques to problems

The exam guide asks you to select the right technique for a use case. Noor's checklist is short. First, is there a known correct answer for past examples? If yes, it is supervised; then ask whether the answer is a category (classification) or a number (regression). If there is no answer and the goal is to find structure, it is unsupervised; grouping is clustering. If the system must learn a sequence of decisions from rewards, it is reinforcement learning. And if the problem has a known rule, it may not need machine learning at all, which lesson 11 explores.

# Classification versus regression on the same bookings.
bookings = [
    # (days booked ahead, past misses, minutes of visit, missed?)
    (21, 2, 30, 1), (2, 0, 20, 0), (14, 1, 45, 1), (1, 0, 30, 0),
    (30, 0, 60, 0), (10, 3, 30, 1), (3, 0, 15, 0), (25, 1, 30, 1),
]

# A classification question: will it be missed? (category)
def classify(days, past):
    return 1 if days >= 10 and past >= 1 else 0

correct = sum(classify(d, p) == m for d, p, _, m in bookings)
print("classification: correct", correct, "of", len(bookings))

# A regression question: how many minutes will the visit take? (number)
avg = sum(mins for _, _, mins, _ in bookings) / len(bookings)
print("regression baseline: predict", round(avg, 1), "minutes for every visit")

classification: correct 8 of 8
regression baseline: predict 32.5 minutes for every visit

The same rows support both kinds of question. The classification rule predicts a yes or no; the regression baseline predicts a number. Note that the regression baseline, the average, is a real and useful starting point. Every regression model should be compared with "just predict the average" and every classifier with "just predict the most common class", because a model that cannot beat those baselines adds nothing.

Choosing labels carefully

A supervised model can only learn what its label means, so defining the label is a design decision, not a formality. Jonah's Missed? column looked simple until Noor asked what counted. Was an appointment canceled 10 minutes before the start a miss? What about a client who arrived 40 minutes late and was seen anyway, or a booking moved by the clinic because a vet was sick? Andre Leclerc, the operations manager, settled it: a no-show is an appointment where the client neither arrived nor canceled before the start time. Late cancellations within two hours became a separate label for a possible second model. Writing the definition down took 20 minutes and avoided months of confusion, because a model trained on a muddled label learns a muddled pattern.

The same care applies to regression targets. "Visit length" can mean the booked slot, the time from check-in to check-out, or the time the vet actually spent in the room. Each gives a different model, useful for a different decision. Jonah wrote a one-line definition for every label and target in the project and stored it with the data.

Which algorithms belong to which type

The exam rarely asks you to tune an algorithm, but it expects you to recognize common names and the learning type they serve. For supervised learning, linear regression fits a straight-line relationship to predict a number; logistic regression, despite its name, is a classification method that outputs a probability for each class; decision trees split the data with a series of yes-or-no questions; random forests and gradient-boosted trees such as XGBoost combine many trees and are often the strongest choice for tabular data; and neural networks handle images, audio and text. For unsupervised learning, k-means is the classic clustering algorithm, grouping points around a chosen number of centers, and principal component analysis (PCA) is the classic dimensionality-reduction method. For anomaly detection, AWS offers a built-in algorithm in SageMaker AI called Random Cut Forest.

Jonah asked whether he needed to memorize how each works. Noor's answer was no: for a practitioner, the useful knowledge is which family suits which problem, what kind of data it needs, and how easy its results are to explain. A gradient-boosted tree on appointment records and a k-means grouping of households are both reasonable first tries; a neural network on a 20-column table usually is not.

Ranking, recommending and generating

A few tasks do not fit neatly into classification or regression. Ranking orders items by how relevant they are, such as search results or which clients to call first; it is often built from a classifier's scores, as the no-show list is ordered by risk. Recommendation predicts which items a person is likely to want from patterns in many people's behavior, and AWS offers Amazon Personalize for it, covered in lesson 10. Generation, producing new text, images or code, is the territory of generative AI and AWS-112. When an exam question describes one of these tasks, look first at what the output is: a category, a number, an ordering, a suggestion or new content. That single observation usually narrows the answer to one learning type and a handful of services.