Learning: conditioning and observation

Week ten begins with Navarro's phone. During the lecture she lets it buzz twice with the sound she uses for messages, and several students glance at their own pockets, though their phones are silent. 'You have been trained,' she says. 'A sound that meant nothing to you a few years ago now makes you reach for a device. Your phone did not teach you that on purpose, but the people who design notification sounds understand exactly what happened.' Owen admits that the same principle is why he chose a distinctive sound for his channel's intro. This week is about learning, the relatively lasting change in behaviour or knowledge that comes from experience.

Learning takes several forms. The simplest is habituation, a decrease in response to a repeated stimulus that turns out to be unimportant, such as ceasing to notice the hum of a refrigerator. In associative learning, an organism learns that certain events occur together. In classical conditioning, it learns to associate two stimuli. In operant conditioning, it learns to associate a behaviour with its consequences. In observational learning, it learns by watching others. Each form is adaptive: it lets animals, including people, predict and control the important events in their lives.

Pavlov and classical conditioning

Ivan Pavlov, a Russian physiologist, won a Nobel Prize in 1904 for research on digestion. While measuring how much dogs salivated when food was placed in their mouths, his team noticed that the dogs began to salivate before the food arrived: at the sight of the dish, the person who usually brought it or the sound of footsteps. Pavlov spent the next three decades studying this 'psychic secretion' systematically.

In his experiments, a neutral stimulus, one that produced no salivation on its own, such as a tone, was presented just before food. The food was an unconditioned stimulus, because it triggered salivation naturally, and salivation to food was an unconditioned response. After several pairings, the tone alone triggered salivation. The tone had become a conditioned stimulus, and salivation to the tone was a conditioned response. The terms are easy to mix up, so Navarro offers a rule: 'unconditioned' means unlearned, and 'conditioned' means learned.

Acquisition, extinction and recovery

Acquisition is the initial stage, when the association is formed. It works best when the neutral stimulus comes just before the unconditioned stimulus, about half a second in many experiments, so that it predicts what is coming. If the order is reversed, so that the food comes first and the tone after, little conditioning occurs. This finding helped show that conditioning is not just about two events occurring together; it is about one event signalling another.

If the conditioned stimulus is presented repeatedly without the unconditioned stimulus, the conditioned response weakens, a process called extinction. But extinction is not forgetting. After a rest, the response often reappears when the conditioned stimulus is presented again, a phenomenon called spontaneous recovery. Pavlov concluded that extinction suppresses the response rather than erasing it, which matters in therapy: fears that have been extinguished can return under stress or in new settings.

Generalization is the tendency to respond to stimuli similar to the conditioned stimulus; a dog conditioned to a particular tone will salivate somewhat to tones of nearby pitch. Discrimination is the learned ability to tell the conditioned stimulus apart from stimuli that do not signal the unconditioned stimulus. Both are useful. A child bitten by one dog may become afraid of all dogs, which is generalization; with good experiences, the child can learn to discriminate between friendly and threatening ones.

Conditioning in people

In 1920, John B. Watson and Rosalie Rayner conditioned fear in a nine-month-old infant they called Little Albert. Albert at first showed no fear of a white rat. Then, each time he reached for the rat, the researchers struck a steel bar behind his head with a hammer, producing a loud noise that made him cry. After several pairings, Albert cried at the sight of the rat alone and showed fear of other furry objects, such as a rabbit and a fur coat. The study is now seen as unethical, because it deliberately frightened a child and made no attempt to remove the fear, and historians have questioned some of its details. But it showed that emotional responses can be conditioned.

Classical conditioning helps explain some phobias and the cravings that can trigger relapse in addiction: places and objects associated with drug use can become conditioned stimuli that produce craving. It also underlies some treatments. In exposure therapy, people gradually face the feared object or situation in a safe setting until the fear response weakens, applying the principle of extinction. Ruth recalls customers at the pharmacy who felt nauseated on entering the cancer clinic before treatment began, an example of conditioned nausea that clinicians now try to prevent.

Biological and cognitive limits

Early behaviourists believed that any stimulus could be paired with any response with equal ease. In the 1960s, John Garcia showed otherwise. Rats that became ill from radiation hours after drinking flavoured water learned to avoid the flavour after a single experience, even though the illness came long after the drink, which breaks the usual rule about timing. But rats did not easily learn to associate the illness with a light or sound. Garcia concluded that animals are biologically prepared to learn associations that help them survive, such as linking taste with illness. Many people have experienced this taste aversion after one bout of food poisoning.

In the late 1960s, Robert Rescorla showed that conditioning also depends on information. A tone that is always followed by food produces strong conditioning; a tone that is followed by food only as often as food appears without it produces little, even with the same number of pairings, because it does not help predict the food. The model he published with Allan Wagner in 1972 holds that learning on each trial depends on surprise: the difference between what is expected and what happens. That is the same prediction error that dopamine neurons signal, as described in lesson 4. Marcus implements the model.

# Rescorla-Wagner: change in associative strength = rate * (maximum - current strength)
rate, maximum, v = 0.3, 1.0, 0.0
for trial in range(1, 9):
    v += rate * (maximum - v)
    print(f"trial {trial}: strength {v:.2f}")

trial 1: strength 0.30
trial 2: strength 0.51
trial 3: strength 0.66
trial 4: strength 0.76
trial 5: strength 0.83
trial 6: strength 0.88
trial 7: strength 0.92
trial 8: strength 0.94

The strength rises quickly at first, when each pairing is surprising, and then levels off as the tone comes to predict the food fully and there is little left to learn. The curve matches the shape of acquisition in many experiments.

Operant conditioning

Classical conditioning involves responses that are automatic, such as salivating or feeling afraid. Much behaviour, however, operates on the environment to produce consequences. Edward Thorndike placed hungry cats in puzzle boxes from which they could escape by pressing a lever or pulling a loop to reach food. At first the cats escaped by accident, but they gradually got faster. Thorndike summarized the pattern in 1898 as the law of effect: behaviours followed by favourable consequences become more likely, and those followed by unfavourable consequences become less likely.

B. F. Skinner developed these ideas into operant conditioning. He designed an operant chamber, now often called a Skinner box, in which an animal could press a bar or peck a key to receive food or water, with every response recorded automatically. Skinner showed that complex behaviours can be built through shaping, reinforcing successive approximations of a desired behaviour: first any movement toward the bar, then touching it, then pressing it. Animal trainers, teachers and parents use shaping, and it is how Owen taught his dog to ring a bell at the door.

Reinforcement and punishment

A reinforcer is any consequence that strengthens the behaviour it follows. Positive reinforcement strengthens a behaviour by adding something desirable, such as praise, money or a 'like'. Negative reinforcement strengthens a behaviour by removing something unpleasant: taking a painkiller removes a headache, and fastening a seatbelt stops the car's beeping. Negative reinforcement is not punishment; it increases behaviour.

Punishment is a consequence that decreases the behaviour it follows. Positive punishment adds something unpleasant, such as a scolding or a fine. Negative punishment removes something desirable, such as losing phone privileges or a time-out from play.

Primary reinforcers, such as food and relief from pain, are naturally satisfying. Conditioned or secondary reinforcers, such as money, grades and points in a game, gain their power through association with primary reinforcers. Immediate reinforcers are much more effective than delayed ones, which is why the immediate pleasure of a video or a snack so often beats the delayed reward of a good grade or better health.

Punishment can suppress behaviour quickly, but research shows important drawbacks. Punished behaviour is often suppressed only while the punisher is present. Punishment teaches what not to do, not what to do instead. It can create fear and anxiety, and physical punishment in particular is associated with increased aggression in children, who learn that hitting is a way to solve problems. Most psychologists therefore recommend reinforcing desired behaviour and, where punishment is needed, using mild, immediate, consistent consequences paired with an explanation.

Schedules of reinforcement

With continuous reinforcement, every response is reinforced; learning is fast, but the behaviour extinguishes quickly when reinforcement stops. With partial or intermittent reinforcement, only some responses are reinforced; learning is slower, but the behaviour is far more resistant to extinction. Skinner identified four basic schedules.

On a fixed-ratio schedule, reinforcement comes after a set number of responses, such as a free coffee after every ten purchases. On a variable-ratio schedule, it comes after an unpredictable number of responses, as with slot machines; this schedule produces high, steady rates of responding that are very hard to extinguish. On a fixed-interval schedule, the first response after a set time is reinforced, producing a burst of responding as the time approaches, like checking the mailbox more often as the usual delivery time nears. On a variable-interval schedule, the first response after an unpredictable time is reinforced, producing slow, steady responding, like checking for a reply to a message.

Owen recognizes his audience's behaviour, and his own. Social media feeds deliver interesting posts, likes and messages on unpredictable schedules, a mix of variable-ratio and variable-interval reinforcement, which is one reason checking and scrolling are so persistent. Understanding the schedule does not make it lose its power, Navarro says, but it can help people design their own environments, for example by turning off notifications.

Observational and cognitive learning

People also learn without being directly reinforced. In observational learning, people learn by watching others, called models. In Albert Bandura's Bobo doll studies in the early 1960s, preschool children who watched an adult punch and kick an inflatable clown were much more likely to imitate those actions, often with the same words, when later left alone with the doll than children who had watched a calm adult. Later studies found that children were less likely to imitate a model who was punished, but that they had still learned the behaviour and would perform it when offered a reward. Bandura emphasized the difference between learning and performance.

Observational learning has positive effects too: people learn skills, kindness and helpful habits from models. Prosocial models in television programs for children can increase helping and sharing. The effects of violent media remain debated: many studies find that violent video games and media are associated with small increases in aggressive thoughts and behaviour, while others argue the effects are smaller than claimed and question how aggression is measured in the laboratory.

Cognition matters in learning more broadly. In the 1930s and 1940s, Edward Tolman found that rats allowed to explore a maze without reward later ran it as quickly as rats that had been rewarded all along, once food was introduced. They had formed a cognitive map of the maze, showing latent learning that becomes visible only when there is a reason to use it. Wolfgang Köhler described chimpanzees that, after a period of apparent thought, suddenly stacked boxes or joined sticks to reach bananas, an example of insight.

Finally, rewards can sometimes backfire. When people are rewarded for doing something they already enjoy, such as drawing or solving puzzles, they sometimes do it less once the rewards stop, an effect called overjustification. Rewards seem to undermine intrinsic motivation when they feel controlling, but not when they provide information about competence. Navarro encourages the class to notice which of their study habits are driven by grades alone and which by interest.