Audience research: surveys, interviews and comments

Analytics show what viewers do; they rarely show why. A retention cliff tells you where people leave, not what bothered them. A drop in returning viewers tells you something changed, not whether the cause was your content, a competitor or viewers' changing lives. Audience research fills that gap by asking people directly and listening carefully to what they already say. Combined with analytics and experiments, it is one of the most powerful tools a creator has for generating good hypotheses and understanding results. This lesson covers surveys, interviews, comment analysis and community polls, how to avoid the common biases in each, and the privacy rules that apply when you collect information from your audience.

Qualitative and quantitative research

Audience research comes in two broad types. Quantitative research collects numbers from many people, such as survey ratings or poll results, and tells you how common something is. Qualitative research collects detailed words from fewer people, such as interviews or open-ended comments, and tells you why and how. The two work best together: interviews reveal ideas and language you would not think to ask about, and surveys then measure how widely those ideas are shared.

Surveys

A survey asks a set of standard questions to many people. Creators often run surveys through a link in a video description, community post, newsletter or membership platform. Good surveys are short, usually 5 to 12 questions, and focused on decisions you actually need to make.

Several principles improve survey quality.

• Ask about behaviour and specifics rather than hypotheticals. "How many of our videos did you watch last month?" is more reliable than "Would you watch more if we posted daily?"
• Avoid leading questions. "How much did you love the new series?" pushes toward a positive answer; "How would you rate the new series?" with a balanced scale does not.
• Ask one thing at a time. "Were the videos useful and entertaining?" combines two questions.
• Use balanced scales, such as 1 to 5 from very poor to very good, with a clear middle point.
• Include at least one open-ended question, such as "What is one thing we could do better?", to capture ideas you did not anticipate.
• Test the survey with two or three people first to catch confusing wording.

Who answers: sampling and bias

The biggest weakness of creator surveys is that respondents are not a random sample of the audience. People who answer are usually the most engaged fans, who watch more, like more and have stronger opinions than casual viewers. This is called self-selection bias. Survey results therefore describe your core audience well and your casual audience poorly. To reduce the problem, place the survey link where casual viewers will see it, such as in a popular video rather than only in a members' post, keep it short, and compare respondents' viewing habits with your analytics to see how representative they are.

Even with a biased sample, sample size matters for precision. For a yes-or-no question, an approximate 95 % margin of error is 1.96 times the square root of p(1 − p) divided by the number of respondents.

Interviews

A short interview, 20 to 30 minutes on a video call, with five to ten audience members can reveal more about motivations than hundreds of survey answers. Interviews are especially useful early in strategy work, when you do not yet know which questions matter. Recruit a mix of people: long-time fans, newer viewers, and ideally a few people who used to watch and stopped. Ask open questions about their experiences, such as "Tell me about the last time you watched one of our videos. What were you doing? Why that video?", and listen more than you talk. Avoid pitching ideas and asking whether they like them; people tend to be polite to creators they admire.

After interviews, look for patterns: phrases that several people used, problems that came up repeatedly, and surprises. These become hypotheses for surveys and experiments.

Comment and message analysis

Comments, replies, messages and community discussions are free, continuous research. The challenge is that the people who comment are a small and unusual slice of viewers, and memorable comments, positive or negative, tend to dominate a creator's impression. Systematic analysis reduces that distortion.

• Sample comments in a consistent way, for example 200 comments from each of the last five videos, rather than reading only the top or the most emotional.
• Code each comment into themes, such as praise for pacing, request for a topic, confusion, criticism of audio, or off-topic.
• Count how often each theme appears and track it over time.

AI tools can help summarize and code large volumes of comments, as discussed in CRE-410, but spot-check their coding against your own reading, because summaries can miss sarcasm, context and minority views.

Community polls

Platform polls, such as YouTube community posts, Instagram stories and polls in livestream chats, are quick and engaging. They are useful for simple choices, such as which of two topics to cover next. Their limits are self-selection, a very small number of answer options, and the fact that people vote for what sounds appealing rather than what they will actually watch. Treat poll results as one input, and where possible check them against behaviour, for example by publishing both topics over time and comparing performance.

Privacy and consent

Collecting information from your audience, even through a simple survey, brings legal and ethical responsibilities. Collect only what you need, explain why you are collecting it and how it will be used, store it securely, and delete it when no longer needed. Avoid collecting sensitive information, such as health details, unless essential. Be especially careful with children.

In Canada, the Personal Information Protection and Electronic Documents Act (PIPEDA) generally applies to personal information collected in the course of commercial activity, which can include a creator business running surveys or mailing lists. It requires meaningful consent, limiting collection to what is needed for identified purposes, safeguarding data and giving people access to their information. Alberta and British Columbia have their own private-sector privacy acts that apply instead of PIPEDA for organizations within those provinces. Quebec's private-sector law, as amended by Law 25 with most changes in force by September 2023, adds requirements such as designating a person responsible for personal information (by default the person with the highest authority in the business), assessing privacy risks before transferring personal information outside Quebec, and obtaining express consent for sensitive information. Survey invitations sent by email or text can also be commercial electronic messages under Canada's Anti-Spam Legislation (CASL), which generally requires consent, identification and an unsubscribe option. Children's information calls for extra caution and, generally, parental consent.

If you plan to collect detailed personal information, sell products to people who answer, or run research with children, get advice from a privacy professional.

Turning research into action

Research is valuable only if it changes decisions. After each research activity, write down three to five findings, the hypotheses they suggest, and how you will test them. For example: "Interviews suggest newer viewers find our jargon confusing. Hypothesis: adding a one-line definition the first time each technical term appears will raise retention among new viewers. Test: randomized holdout over eight videos, comparing retention for new viewers." This closes the loop between listening and experimenting.

Combining research with experiments

Research and experiments complement each other. Research suggests what to test and helps explain surprising results; experiments show whether the ideas from research actually change behaviour. When an experiment produces an unexpected result, a few follow-up interviews or a targeted look at comments often reveals why, and that understanding usually leads to a better next test.