Research methods: how psychologists find things out

Navarro opens the second week with a question from the course forum. Marcus has read that students who sit in the front rows of lectures earn higher grades, and he wants to know whether he should move forward. 'Maybe,' Navarro says. 'But before you move, ask what kind of study produced that finding. Sitting in front might raise grades. Or the students who already work hardest might choose the front. Or something else, like arriving early, might cause both.' The point of research methods, she explains, is to tell those possibilities apart.

Every study in psychology starts with a question and a plan for answering it with evidence. The plan is called a research design. Different designs answer different kinds of questions, and each has strengths and weaknesses. Psychologists often say that descriptive methods tell us what happens, correlational methods tell us what goes together, and experiments tell us what causes what.

From question to hypothesis

Research usually follows a cycle. A researcher starts with a theory or an observation, states a hypothesis, chooses operational definitions, collects data, analyzes it, and reports the results so others can evaluate and repeat the work. The results then support, refine or challenge the theory, which leads to new questions. Navarro stresses that the cycle never really ends. A single study rarely settles anything; confidence grows when many studies, using different methods and samples, point the same way.

Researchers also choose who to study. The population is the whole group a researcher wants to draw conclusions about, such as all first-year college students in Canada. A sample is the smaller group actually studied. In a random sample, every member of the population has an equal chance of being chosen, which makes the sample likely to represent the population. Many psychology studies use convenience samples, often university students, because they are easy to recruit. Critics have pointed out that much of psychology's evidence comes from people who are Western, educated, industrialized, rich and democratic, and that findings do not always generalize to the rest of humanity.

Descriptive methods

A case study examines one person or a small group in depth. Lesson 11 describes the patient known for decades as H.M., whose memory loss after brain surgery transformed what scientists knew about memory. Case studies can reveal phenomena no one suspected and suggest hypotheses, but they cannot show that a finding applies to anyone else, and vivid cases can mislead if they are unusual.

Naturalistic observation records behaviour in its natural setting without interfering. Researchers have watched children on playgrounds, chimpanzees in the wild and shoppers in supermarkets. The behaviour is genuine, but observers cannot control the conditions, and people who know they are being watched may act differently, a problem researchers reduce by letting people get used to them or by observing unobtrusively in public places.

A survey asks many people about their attitudes, experiences or behaviour. Surveys can reach thousands of people quickly, but answers depend heavily on how questions are worded. People are more likely to support 'aid to the needy' than 'welfare', and more likely to agree with a statement than with its opposite when both are phrased as questions to agree with. People also tend to give socially desirable answers about topics such as drinking, exercise and prejudice. Good surveys test their wording, keep answers anonymous and use representative samples.

Correlational research

Correlational research measures two or more variables as they naturally occur and asks whether they are related. A correlation can be positive, when the variables rise and fall together, such as hours of sleep and exam scores; negative, when one rises as the other falls, such as hours of late-night screen use and hours of sleep; or close to zero, when they are unrelated. Lesson 3 shows how the strength of a correlation is expressed as a number between minus one and plus one.

Correlations are valuable because many important variables cannot ethically be manipulated. No one can assign children to experience poverty or adults to start smoking. Correlations also allow prediction: if scores on an entrance test correlate with first-year grades, the test helps predict who will need extra support. But correlations have a famous limitation.

Correlation does not prove causation

If X and Y are correlated, X might cause Y, Y might cause X, or a third variable might cause both. Navarro returns to the front-row finding. Sitting in front might help students pay attention, which raises grades. Or doing well in a course might make students feel comfortable sitting near the instructor, a case of reverse causation. Or motivation might lead students both to sit in front and to study more, a third-variable problem. A correlation alone cannot tell these apart.

Real examples are everywhere. Children who eat breakfast with their families tend to do better at school, but families who share meals also differ in income, schedules and many other ways. Countries that consume more chocolate have won more Nobel prizes per person, a correlation that almost certainly reflects national wealth rather than the power of chocolate. Ruth recalls customers at the pharmacy who were sure a supplement had cured their cold, when colds get better on their own in about a week.

Longitudinal and cross-sectional designs

Some questions are about change over time. A cross-sectional study compares different groups at one moment, for example measuring sleep in 18-year-olds, 40-year-olds and 70-year-olds this year. It is quick, but the groups differ in more than age: they grew up with different schools, technologies and health care, so a difference between them might reflect their generation rather than ageing. A longitudinal study follows the same people over months or decades, measuring them repeatedly. It shows how individuals change and which earlier factors predict later outcomes, but it is slow and expensive, and some participants drop out, often in ways that are not random.

Longitudinal studies also help with the causal puzzles of correlational research. If loneliness at age 14 predicts heavier social media use at 16, but heavy use at 14 does not predict loneliness at 16, the pattern suggests which way the influence may run, although a third variable could still be involved. Navarro tells the class that the strongest conclusions usually come from combining designs: experiments for cause, longitudinal studies for change, and large representative surveys for how common something is.

Reliability and validity of measures

Every study depends on its measures. A measure is reliable if it gives consistent results, for example if the same person gets similar scores on a questionnaire taken twice a week apart, or if two observers coding the same video agree. A measure is valid if it measures what it claims to measure. A bathroom scale that always reads five kilograms too heavy is reliable but not valid. Validity is checked by asking whether scores relate to other things they should relate to: a valid measure of test anxiety should predict sweating, heart rate and worse performance under exam conditions, and it should not simply be a measure of general intelligence. A study cannot be better than the measures it uses.

Experiments

An experiment is the only method that can establish cause and effect. The researcher manipulates one variable, the independent variable, and measures its effect on another, the dependent variable, while holding everything else constant. The participants who receive the manipulation form the experimental group; those who do not form the control group.

The key ingredient is random assignment: each participant has an equal chance of being placed in any condition, for example by a coin flip or a random number generator. Random assignment makes the groups similar on average in every characteristic, including ones the researcher never thought to measure, such as motivation, sleep or mood. If the groups then differ on the dependent variable, the manipulation is the most likely cause. Random assignment is different from random sampling: sampling decides who is in the study, and assignment decides which condition each person gets.

Marcus writes a short program to see random assignment at work. He gives 200 simulated students a hidden motivation score, assigns them at random to two groups and compares the averages.

import random
random.seed(7)
students = [random.gauss(50, 10) for _ in range(200)]   # hidden motivation scores
for trial in range(1, 4):
    random.shuffle(students)
    a, b = students[:100], students[100:]
    print(f"trial {trial}: group A mean {sum(a)/100:5.1f}  group B mean {sum(b)/100:5.1f}")

trial 1: group A mean  50.2  group B mean  50.1
trial 2: group A mean  49.8  group B mean  50.4
trial 3: group A mean  49.8  group B mean  50.4

In each trial the two groups end up with nearly the same average motivation, even though no one looked at the scores. With 100 people per group, chance differences are small; with 10 per group they would be much larger, which is one reason sample size matters. If Marcus then gave one group a new study technique and that group scored higher on the exam, motivation could not easily explain the difference.

Controlling for expectations

People's expectations can change their behaviour, so good experiments control for them. A placebo is an inactive treatment, such as a sugar pill or a sham procedure, given to the control group so that both groups believe they might be receiving the real thing. The placebo effect, in which people improve because they expect to, can be surprisingly strong, especially for pain, fatigue and mood.

In a single-blind study, participants do not know which condition they are in. In a double-blind study, neither the participants nor the researchers who interact with them know. Double-blind procedures protect against experimenter bias, in which researchers unintentionally treat groups differently or interpret ambiguous results in the direction they hope for. Drug trials are normally double-blind, and so are many psychology experiments where the person running a session could influence results by tone of voice or encouragement.

Experiments also face trade-offs. Laboratory control can make a situation artificial. Researchers therefore talk about internal validity, how confident we can be that the independent variable caused the change, and external validity, how well the results generalize to real life. Field experiments, run in schools, workplaces or online platforms, try to keep random assignment while improving realism.

Research ethics

Psychology's history includes studies that would never be approved today, such as experiments that deceived participants into believing they were harming others or caused lasting distress. Modern research ethics rest on a few core principles: respect for people, which requires informed consent and the right to withdraw; concern for welfare, which requires minimizing harm and protecting privacy; and justice, which requires that the burdens and benefits of research be shared fairly.

In Canada, research involving people at institutions funded by the federal research councils must follow the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans, known as TCPS 2, and be approved by a research ethics board. The policy is built on respect for persons, concern for welfare and justice, and it includes a chapter on research involving First Nations, Inuit and Métis peoples that requires community engagement. Psychologists also follow the Canadian Code of Ethics for Psychologists published by the Canadian Psychological Association.

Informed consent means participants are told what the study involves, including risks, before agreeing. Some studies use mild deception, such as not revealing the exact hypothesis, because knowing it would change behaviour. Deception is allowed only when it is necessary, the risks are low and participants are debriefed afterward, which means the study's true purpose and any deception are explained. Confidentiality means that data are stored securely and reported without identifying individuals.

Research with animals is also regulated. Animals are used when studies cannot be done with people, for example in research on brain mechanisms of learning. The Canadian Council on Animal Care sets standards for their housing and treatment, and committees must approve each study, weighing its expected benefits against the costs to the animals.