Customer discovery and problem interviews

Grace and Marta arrive at the third session with a spreadsheet of forecasts for Second Ascent. Dr. Marchetti asks how many of the numbers came from customers. The answer is none: they came from Cedarline data, industry reports and the founders' judgment. She asks them to put the spreadsheet away for three weeks and talk to at least forty people instead. The rest of the session explains how. Customer discovery, she says, is the cheapest research a founder will ever do, and the most often skipped, because founders are eager to build and because talking to strangers about a half-formed idea is uncomfortable.

Customer development

Steve Blank's customer development model, set out in The Four Steps to the Epiphany and later in The Startup Owner's Manual with Bob Dorf, divides the search for a business model into four stages. In customer discovery, founders turn their vision into hypotheses about customers, problems and solutions and test them by getting out of the building to talk to potential customers. In customer validation, they test whether the business model is repeatable and scalable by actually selling, for example by winning a set of paying early customers through a process that can be repeated. Only then do they move to customer creation, spending to drive demand at scale, and company building, adding the departments and processes of a larger firm. Blank's central warning is that many startups spend heavily on the last two stages before completing the first two.

The model treats every element of the business plan as a hypothesis. Who is the customer? What problem do they have, and how do they solve it today? How important is it to them? What would they pay for a better solution? How would they find and buy it? Who else influences the purchase? Each hypothesis can be stated in a form that could prove it wrong. For Second Ascent, one hypothesis is that owners of premium outdoor gear who have had an item fail in the past two years did not get it repaired because they did not know where to go, not because they did not want to pay. Another is that small outdoor brands without a workshop would pay a per-repair fee to outsource their warranty repairs.

Who to talk to

Founders should talk to the people whose behavior the venture depends on. For most ventures there are several groups: end users, buyers who control the budget, and influencers or partners who affect adoption. Second Ascent has three. Gear owners are the end users and buyers for consumer repairs. Specialty outdoor shops influence where owners go, and some might refer repairs or host drop-off points. Small and mid-sized outdoor brands are potential business customers for outsourced warranty repairs. Each group has different problems, and an interview guide written for one does not fit the others.

Within each group, the most informative interviews are with likely early adopters. Everett Rogers' work on the diffusion of innovations, which the cohort met in MBA-512, describes innovators and early adopters as people who have the problem acutely, know they have it, have tried to solve it and are willing to try something new. Blank adds that the best early customers have often built their own workaround or have budget set aside for a solution. For Second Ascent, those are people who have tried to fix gear themselves with patches and tape, have sent items back to a brand and waited weeks, or have paid a local tailor who did not know how to work with waterproof fabrics.

How many interviews are enough? There is no fixed number, but research on qualitative interviewing, including a widely cited study by Greg Guest and colleagues in 2006, found that most new themes emerge in the first dozen interviews with a fairly similar group, and that additional interviews add little after that. Founders should therefore aim for roughly a dozen or more interviews per distinct group and stop when new interviews stop producing surprises. Grace and Marta plan thirty with gear owners, eight with shops and ten with brands.

How to ask

The hardest part of customer discovery is getting honest information. People are polite, and they rarely want to tell a hopeful founder that an idea is unnecessary. Rob Fitzpatrick's 2013 book The Mom Test, named for the idea that even your mother should not be able to lie to you if you ask the right questions, offers three rules. Talk about their life, not your idea. Ask about specifics in the past, not generalities or opinions about the future. Talk less and listen more. A question such as would you use a service that repairs your gear invites a polite yes. A question such as tell me about the last time something you own broke, and what you did, produces facts.

Good interviews also avoid pitching. Once the founder describes the solution, the conversation tends to become a reaction to it, and the interviewee starts offering opinions and feature requests. Dr. Marchetti recommends spending most of each interview on the customer's problem and current behavior, and introducing the solution, if at all, only at the end. She also recommends taking notes in the customer's own words and recording facts separately from interpretations. A note saying the customer said the brand took six weeks to return a jacket is a fact; a note saying the customer is frustrated with brands is an interpretation.

Jobs to be done

Clayton Christensen and his colleagues offered another way to frame the customer's problem. Customers do not buy products; they hire them to make progress in a particular circumstance, a job to be done. The job has functional, emotional and social dimensions. Christensen's best-known example involved a fast-food chain that wanted to sell more milkshakes and discovered that many were bought early in the morning by commuters who wanted something that would last a long, dull drive and fit in a cup holder. The competitors were not other milkshakes but bananas, bagels and boredom. Understanding the job pointed to changes that the usual demographic research had missed.

Interviews revealed several jobs for which owners might hire Second Ascent. The most common was functional: keep using a jacket or tent that I like and that still mostly works, before a trip that is coming soon. The time dimension proved important; many owners had a broken item and an upcoming trip, and wanted the repair done in days rather than weeks. A second job was emotional: avoid the guilt of throwing away something expensive that could be fixed. A third, mentioned mainly by younger owners, was social: be seen to own good gear that lasts, and buy used without feeling that it is second-rate. Each job suggested different features and messages.

From interviews to insights

Raw interview notes are not evidence until they are organized. A practical method is to code each interview, tagging statements with themes, and then count how many interviewees mentioned each theme and how strongly. The counts do not have the precision of a survey, because the sample is small and not random, but they show which problems are common and which are rare, and they point to the hypotheses worth testing with larger samples and experiments.

# Coding 30 gear-owner interviews and putting honest error bars on small-sample shares
from math import sqrt

def wilson(k, n, z=1.96):
    # 95% Wilson score interval for a proportion; better than the normal interval for small n
    p = k / n
    mid = (p + z * z / (2 * n)) / (1 + z * z / n)
    half = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
    return mid - half, mid + half

n = 30
themes = {
    "item failed in past two years": 21,
    "did not repair the failed item": 13,
    "did not know where to get it repaired": 9,
    "sent to a brand, waited over three weeks": 6,
    "has bought outdoor gear used": 12,
    "would pay at least 50 for a zipper repair (stated)": 17,
}
for theme, k in themes.items():
    lo, hi = wilson(k, n)
    print(f"{theme:<52} {k:>2}/{n}  {k / n:5.0%}  (95% range {lo:.0%} to {hi:.0%})")

# Commitments are stronger evidence than statements
booked, waitlist = 7, 4
print(f"Booked a repair on the spot: {booked}; joined the waiting list: {waitlist}")

item failed in past two years                        21/30    70%  (95% range 52% to 83%)
did not repair the failed item                       13/30    43%  (95% range 27% to 61%)
did not know where to get it repaired                 9/30    30%  (95% range 17% to 48%)
sent to a brand, waited over three weeks              6/30    20%  (95% range 10% to 37%)
has bought outdoor gear used                         12/30    40%  (95% range 25% to 58%)
would pay at least 50 for a zipper repair (stated)   17/30    57%  (95% range 39% to 73%)
Booked a repair on the spot: 7; joined the waiting list: 4

The wide intervals are the most important output. With thirty interviews, the share of owners who had an item fail could be anywhere from about half to more than four-fifths, and the share who did not know where to go could be anywhere from about a sixth to about a half. Those ranges are good enough to tell Grace and Marta that the problem is common, but not to set a price or forecast volume. They also show the gap between statements and behavior. Seventeen of thirty said they would pay at least 50 dollars for a zipper repair, but only seven booked a repair when offered one at that price during the interview. Dr. Marchetti says the second number is the one to believe.

Commitment as evidence

The strongest evidence in customer discovery is a commitment that costs the customer something: time, reputation or money. Fitzpatrick lists examples. A time commitment is agreeing to a follow-up meeting or a trial. A reputation commitment is introducing the founder to a boss or colleague. A money commitment is a pre-order, a deposit or a letter of intent. Compliments, such as that sounds great, cost nothing and should be treated as noise. The founder's goal in each conversation is to move to a concrete next step, and the share of conversations that end in a commitment is a useful measure of how real the problem is.

The business interviews produced the most useful commitments. Of the ten small brands interviewed, six described their warranty process as a significant cost and a source of complaints. Three currently ship repairs to a contractor across the country and wait four to eight weeks; two simply replace items, which costs them about three times as much as a repair would. Two brand managers agreed to send a test batch of twenty warranty items to Second Ascent at an agreed fee, and one asked for a proposal for a full year. Grace says this changed her view of the business. She had thought of the brand contracts as a side line; they might be the fastest route to steady volume.

Willingness to pay

Price questions are among the hardest to ask well. People asked directly what they would pay tend to understate in some situations and overstate in others, depending on whether they think the answer will affect the price they are charged. Several methods help. Asking what they paid last time, and for what, anchors on real spending. The Van Westendorp price sensitivity meter, developed by a Dutch economist in the 1970s, asks four questions: at what price would the service be so cheap that you would doubt its quality, a bargain, getting expensive, and too expensive to consider. Plotting the cumulative answers gives a range of acceptable prices. Conjoint analysis, which the cohort used in MBA-512, is stronger but needs a larger sample. The strongest evidence comes later, from experiments in which customers actually pay different prices.

For Second Ascent the interviews suggested an acceptable range for a standard zipper replacement on a shell jacket of roughly 45 to 75 dollars, with most people citing what a new jacket costs as their reference point. Owners of expensive gear were less price-sensitive than owners of mid-range items; for them, speed mattered more. That suggested two tiers: a standard repair in about ten days and an express repair in three days at a higher price. Marta noted that express repairs disrupt workshop scheduling, which the cohort's operations course had shown can raise costs sharply when utilization is high.

Pivots and persistence

Customer discovery often changes the venture. A pivot, in Ries's terms, is a structured change in one element of the business model, such as the customer segment, the problem, the channel or the revenue model, based on what the founders have learned. Pivots are common in successful companies: many well-known firms began with a different product or customer. The opposite risk is pivoting too easily, abandoning a good idea after a few discouraging conversations. Dr. Marchetti's rule is to pivot when the evidence contradicts a core hypothesis, not when it is merely disappointing, and to write down in advance what result would count as a contradiction.

Second Ascent's interviews led to a partial pivot. The founders had planned to start with consumer repairs and add brand contracts later. The evidence suggested reversing the order, or at least running both from the start: brand contracts offered predictable volume that would keep technicians busy, while consumer repairs were seasonal, peaking before summer trips and the ski season. Refurbished resale, which Grace had thought might be the biggest opportunity, drew the weakest interest from owners in the first city, though it remained promising as a way to use items that brands would otherwise write off.