Product discovery and user research

Product discovery is the work of deciding what to build: understanding users and their problems, generating and testing possible solutions, and gathering enough evidence to commit. Delivery is the work of building, shipping and operating the chosen solution. In older processes discovery happened once, as a requirements phase at the start of a project. Modern product teams practise continuous discovery: they keep talking to users every week, alongside delivery, so their understanding keeps pace with a changing world. This lesson covers the research methods that feed discovery; lesson 3 turns what we learn into problem statements and opportunities.

CMPUT-301 introduced requirements elicitation: interviews, workshops, observation and prototypes used to capture what a system must do. Product discovery uses many of the same techniques but asks a different question. Requirements elicitation assumes the problem and asks what the system should do; discovery questions the problem itself. Is it real? How often does it happen? How painful is it? Who has it? What do people do about it today? Only when those answers are clear does it make sense to specify a solution.

Who to learn from

The first discovery decision is whom to talk to. Users are the people who directly use the product; customers are the people who pay or choose it; and stakeholders are everyone else with an interest, such as staff, regulators and partners. In a client project these may be different people. For Driftwood Ferries, users include commuters, tourists, commercial truck drivers and island residents; staff users include deckhands, ticket agents and office staff; the customer is Driftwood itself, which pays Team Lumen; and stakeholders include the provincial or state transportation regulator, the island community association and the payment processor.

Segments matter because different groups have different problems. A weekday commuter knows the schedule by heart and cares about predictable space; a summer tourist does not know the routine and needs basic orientation; a truck driver needs to know whether the commercial lane is open and whether the vessel can carry the load. Sampling only the easiest people to reach, such as the client's friends or other students, produces confident but wrong conclusions. Aim for several people from each important segment, and deliberately include people who are poorly served today, such as travellers with disabilities or limited English.

Interviewing well

Interviews are the workhorse of discovery and the method most often done badly. The central rule, made memorable by Rob Fitzpatrick's book The Mom Test, is to ask about people's lives and past behaviour, not about your idea. People are kind and optimistic: ask whether they would use a ferry app with reservations and almost everyone says yes. Ask them to tell you about the last time they took the ferry, what went wrong, what they did about it and what it cost them, and you learn what actually matters. Specific past events are evidence; opinions about hypothetical futures are noise.

A good discovery interview has a short script but stays flexible. Open with context (who the person is and how they travel), ask them to walk through a recent concrete experience, probe for problems and current workarounds, and ask about costs: time, money, stress, missed appointments. Follow interesting threads with open questions such as tell me more about that, or what happened next. Avoid leading questions (wouldn't it be great if…), yes-or-no questions, and pitching. Close by asking who else you should talk to and whether you may follow up. Two people interviewing, one asking and one taking notes, capture far more than one.

Interviews also raise ethical and legal duties. Tell participants who you are, why you are asking, how notes will be used and that they can stop at any time. Ask before recording. Store notes securely and strip names when sharing insights. In Canada, a university team doing research for academic purposes may need research ethics board approval under the Tri-Council Policy Statement (TCPS 2), and personal information collected for a business client falls under privacy law; in the United States, university research with people may need institutional review board (IRB) review under the Common Rule, and client data handling is governed by the contract and applicable state privacy laws. Ask your instructor which applies: most product discovery for quality improvement is exempt from research ethics review, but publishing results may not be.

Observation and contextual inquiry

People cannot always tell you what they do, because much of their behaviour is habitual or invisible to them. Contextual inquiry means watching people work in their real environment and asking questions as they go. Team Lumen spent two mornings at the Driftwood terminal and on the car deck. They saw what no interview had mentioned: deckhands use hand signals and a clipboard to count cars, the lineup lanes are painted but unclear after snowfall, travellers keep phoning the office from the lineup, and many tourists arrive with no idea that the ferry fills on summer Fridays. Observation turns vague complaints into precise, designable problems.

Making sense of qualitative data

After a dozen interviews and observations, the team has pages of notes. Synthesis turns them into insight. The most common technique is affinity mapping: write each observation on its own sticky note (physical or digital), then group notes that seem related, name each group, and look for patterns across groups. Groups that recur across many participants and segments are strong candidates for important problems. The code below imitates the idea with keyword matching; real affinity mapping relies on human judgement about meaning, not shared words, but the output shows the shape of a synthesis.

# Affinity mapping, the quick-and-dirty way: group interview notes by shared keywords.
# Notes are invented, paraphrased observations from Driftwood Ferries interviews.
notes = [
    "Drove 40 minutes and missed the 7:10 sailing because the lot was full",
    "Never knows if the truck lane is open before leaving home",
    "Checks the Facebook page for delays but posts are late",
    "Deckhand: people argue about who arrived first in the lineup",
    "Tourist did not know cash is not accepted on the island side",
    "Commuter would pay extra to guarantee a spot on Monday mornings",
    "Wants a text when the ferry is running late",
    "Office staff spend mornings answering phone calls about waits",
    "Elderly passenger cannot read the small schedule board at the terminal",
    "Missed the last sailing home; no warning that it was full",
]
themes = {
    "uncertainty about space":  ["full", "open", "spot", "guarantee", "missed"],
    "late or scattered information": ["late", "delays", "posts", "text", "phone"],
    "fairness and lineup":      ["argue", "first", "lineup"],
    "payment and rules":        ["cash", "accepted", "pay"],
    "accessibility":            ["read", "small", "board"],
}
groups = {t: [] for t in themes}
for n in notes:
    words = n.lower().replace(";", " ").replace(",", " ").split()
    best = max(themes, key=lambda t: sum(w in words for w in themes[t]))
    if sum(w in words for w in themes[best]) == 0:
        best = "unsorted"
        groups.setdefault(best, [])
    groups[best].append(n)
for t, ns in sorted(groups.items(), key=lambda kv: -len(kv[1])):
    print(f"{t} ({len(ns)})")
    for n in ns:
        print("   -", n)

uncertainty about space (4)
   - Drove 40 minutes and missed the 7:10 sailing because the lot was full
   - Never knows if the truck lane is open before leaving home
   - Commuter would pay extra to guarantee a spot on Monday mornings
   - Missed the last sailing home; no warning that it was full
late or scattered information (3)
   - Checks the Facebook page for delays but posts are late
   - Wants a text when the ferry is running late
   - Office staff spend mornings answering phone calls about waits
fairness and lineup (1)
   - Deckhand: people argue about who arrived first in the lineup
payment and rules (1)
   - Tourist did not know cash is not accepted on the island side
accessibility (1)
   - Elderly passenger cannot read the small schedule board at the terminal

Synthesis also produces personas and journey maps. A persona is a short, evidence-based description of a type of user, such as Maya the weekday commuter or the Reyes family on a summer trip, capturing goals, contexts and frustrations. Personas are useful when grounded in research and harmful when invented from stereotypes. A customer journey map lays out the steps a user goes through, such as planning the trip, driving to the terminal, waiting, boarding and arriving, with their actions, thoughts, feelings and pain points at each step. Journey maps reveal where the biggest problems sit and which moments matter most.

How much research is enough?

There is no fixed number, but research on usability testing and interviewing suggests that a small number of well-chosen participants per segment surfaces most common problems, while rare problems need more. The simulation below assumes twelve problems with different frequencies and counts how many a team hears after a given number of interviews. Look at the diminishing returns: the first few interviews reveal the common problems, and later ones mostly repeat them while occasionally adding a rarer one. When new interviews stop producing new themes, the team has reached saturation for that segment.

# How many interviews are enough? Simulate "theme saturation" (illustrative).
# Suppose the population has 12 distinct problems, some common and some rare.
import random
random.seed(401)
weights = [0.30, 0.25, 0.20, 0.15, 0.12, 0.10, 0.08, 0.06, 0.05, 0.04, 0.03, 0.02]
def interview():
    # each person mentions each problem with its own probability
    return {i for i, w in enumerate(weights) if random.random() < w * 2}
runs = 2000
for n in (3, 5, 8, 12, 20):
    found = 0
    for _ in range(runs):
        seen = set()
        for _ in range(n):
            seen |= interview()
        found += len(seen)
    print(f"{n:2d} interviews -> on average {found / runs:4.1f} of 12 problems heard")

 3 interviews -> on average  5.8 of 12 problems heard
 5 interviews -> on average  7.4 of 12 problems heard
 8 interviews -> on average  8.7 of 12 problems heard
12 interviews -> on average  9.8 of 12 problems heard
20 interviews -> on average 10.8 of 12 problems heard

Quantitative methods complement interviews. Once interviews suggest that uncertainty about vehicle space is a major problem, a short survey of 300 travellers can estimate how common it is in each segment, and Driftwood's phone logs can show how many calls ask about waits. Qualitative research tells you what and why; quantitative research tells you how many and how much. A team that relies only on surveys measures the wrong things precisely; a team that relies only on interviews may mistake vivid stories for common ones.

Continuous discovery habits

Teresa Torres's book Continuous Discovery Habits popularized a simple rule: the product trio, meaning the PM, a designer and an engineer, should talk with customers at least weekly, in small research activities connected to the decisions they are making. Recruiting is automated (for example, a standing invitation in the app or a panel of willing travellers), interviews are short, and learning is captured in a shared place. For a student team, a weekly rhythm might mean one interview or observation per week per pair, a fifteen-minute synthesis at the weekly team meeting, and a running document of insights linked to backlog items.

Worked examples