Marketing research and data

Good marketing decisions rest on good information about customers, competitors and markets. Should a café add a drive-through? Which of three package designs will stand out on the shelf? Is a new subscription price too high? How many people might buy an electric cargo bike in a given city? Marketing research is the systematic design, collection, analysis and reporting of data relevant to a specific marketing situation. This lesson covers the research process, the main sources and methods, sampling and survey design, digital analytics and experiments, market sizing, and the ethical and privacy rules that govern the use of customer data. Statistics in depth belong in STAT courses; here the aim is to make you an intelligent user and commissioner of research.

The research process

Research projects follow a broadly similar sequence. The first and most important step is defining the problem and research objectives. A vague brief such as 'find out why sales are down' leads to vague research. A good problem definition separates the management decision (should we reformulate the product, cut the price or change the advertising?) from the research questions that would inform it (has the product's perceived quality fallen relative to competitors? are lapsed customers buying a cheaper substitute?).

Research objectives fall into three broad types. Exploratory research gathers preliminary information to help define problems and suggest hypotheses; it often uses interviews, focus groups and reviews of existing data. Descriptive research describes things, such as the market potential for a product or the demographics and attitudes of customers, usually through surveys or analysis of transaction data. Causal research tests hypotheses about cause-and-effect relationships, such as whether a 10 % price cut will raise sales enough to increase profit, usually through experiments.

Next comes the research plan: what information is needed, where it will come from, which methods will be used, how participants will be sampled, how the data will be analyzed and what it will cost. The plan is followed by data collection, analysis and interpretation, and finally reporting the findings in a way that helps managers decide. A report that presents a mass of tables without a clear recommendation has failed its purpose.

Secondary data

Secondary data is information that already exists, having been collected for another purpose. It is usually quicker and cheaper to obtain than new data, and it should always be checked first. Internal secondary data includes sales records, customer databases, website and app analytics, call-center logs and loyalty-card data. External secondary data includes government statistics, industry association reports, published studies, news and syndicated research sold by specialist firms.

The US Census Bureau is the most important public source for American marketers. The decennial census counts the population, while the American Community Survey provides annual estimates of income, education, housing, language and commuting down to small areas. Monthly and quarterly reports cover retail sales and e-commerce, and the Bureau of Labor Statistics publishes consumer prices, consumer expenditure surveys and employment data. Industry groups such as the National Retail Federation add further detail.

Secondary data must be evaluated before use. Is it relevant to the question? Is it current, or has the market changed since it was collected? Is it accurate and impartial, or was it published by a group with an interest in a particular conclusion? Are definitions consistent? For example, Canadian and US statistics on e-commerce's share of retail sales are not directly comparable, because the two agencies define and measure online retail differently; a naive comparison would suggest a much larger gap between the countries than really exists.

Primary data and research approaches

When secondary data is not enough, the firm collects primary data for the specific purpose at hand. The main approaches are observation, surveys, experiments and qualitative methods such as interviews and focus groups.

Observational research gathers data by watching people, actions and situations. A retailer might track how shoppers move through a store, or a researcher might watch how people use a product in their kitchen. Ethnographic research sends trained observers to watch and interact with consumers in their natural environment, revealing needs people would never mention in a survey because they take them for granted. Digital observation now includes analyzing website click paths, app usage and public social media conversations, which is often called social listening.

Survey research is the most widely used method for collecting descriptive information about people's knowledge, attitudes, preferences and buying behavior. Surveys can be run online, by phone, by mail or in person. Online panels make surveys fast and inexpensive, but they raise questions about whether panel members represent the wider population and whether some respondents are inattentive or even automated. Good practice includes attention checks, screening for fraudulent responses and comparing results with known benchmarks.

Experimental research tests cause and effect by selecting matched groups, giving them different treatments, controlling unrelated factors and comparing group responses. A test market launches a product in a few cities before a national roll-out. An online A/B test randomly shows half of website visitors one version of a page and half another. Because assignment is random, a large enough difference in outcomes can be attributed to the treatment rather than to differences between the groups.

Qualitative research explores motivations, feelings and language in depth. In-depth interviews talk with one person at a time for perhaps an hour. Focus groups bring together six to ten people with a trained moderator to discuss a product, service or idea. Qualitative findings are rich but cannot be generalized statistically to a population; they are best used to generate hypotheses, develop survey questions and understand the reasons behind quantitative results.

Sampling

A sample is a segment of the population selected to represent the population as a whole. Three decisions are involved: who is to be surveyed (the sampling unit), how many people (the sample size) and how they are to be chosen (the sampling procedure). In probability samples, each population member has a known chance of being included, which allows researchers to calculate sampling error. Simple random samples, stratified samples (random samples within groups such as regions) and cluster samples are common types. In non-probability samples, such as convenience samples and quota samples, the chance of selection is unknown. Non-probability samples are cheaper and often adequate for exploratory work, but results cannot be projected with known precision.

For a probability sample, the margin of error for a proportion depends mainly on the sample size, not the population size. That surprises many people: a well-drawn random sample of about 1,000 gives roughly the same precision whether the population is a city of 500,000 or a country of 40 million. Sample size for a desired margin of error can be estimated with a standard formula, where z is 1.96 for 95 % confidence, p is the expected proportion (0.5 gives the most conservative estimate) and e is the margin of error.

Designing questionnaires

Question design strongly affects the answers. Questions should be clear, neutral, specific and answerable. Leading questions ('Don't you agree that our new design is more attractive?') push respondents toward an answer. Double-barrelled questions ('How satisfied are you with our prices and service?') ask two things at once. Questions about the distant past or hypothetical behavior ('Would you buy this product if it existed?') tend to produce optimistic answers, because saying yes costs nothing. Researchers often discount stated purchase intentions heavily when forecasting.

Closed-ended questions, such as multiple-choice and rating scales, are easy to analyze. A Likert scale asks respondents how strongly they agree or disagree with a statement, often on a five- or seven-point scale. Open-ended questions let respondents answer in their own words, revealing ideas researchers did not anticipate; generative AI tools now help code large numbers of open-ended answers into themes, although human review remains important. The order of questions also matters: general questions usually come before specific ones, and sensitive questions near the end.

Digital analytics and experiments

Digital channels generate huge volumes of behavioral data. Web and app analytics track visits, sources of traffic, pages viewed, time spent, actions taken and conversions. Customer relationship management systems record each customer's purchases and interactions. Marketing automation platforms record email opens and clicks. The challenge is less collecting data than deciding which metrics matter and avoiding vanity metrics, such as raw page views or follower counts, that look impressive but do not connect to business outcomes.

A/B testing applies the logic of experiments to digital marketing. The firm randomly assigns visitors to version A (the control) or version B (the variation), measures a specific outcome such as the conversion rate and checks whether the difference is larger than chance would explain. A common test for comparing two conversion rates is the two-proportion z-test. The p-value is the probability of seeing a difference at least this large if there were really no difference; a p-value below 0.05 is conventionally treated as statistically significant. Statistical significance is not the same as practical importance: a tiny but significant improvement may not be worth the cost of change.

import math, random

random.seed(220)
# Simulated A/B test: 6,000 visitors per version
n_a, n_b = 6000, 6000
true_a, true_b = 0.040, 0.047          # unknown in real life
conv_a = sum(random.random() < true_a for _ in range(n_a))
conv_b = sum(random.random() < true_b for _ in range(n_b))

p_a, p_b = conv_a / n_a, conv_b / n_b
p_pool = (conv_a + conv_b) / (n_a + n_b)
se = math.sqrt(p_pool * (1 - p_pool) * (1 / n_a + 1 / n_b))
z = (p_b - p_a) / se
p_value = 2 * (1 - 0.5 * (1 + math.erf(abs(z) / math.sqrt(2))))

print(f"A: {conv_a} of {n_a} = {p_a:.2%}")
print(f"B: {conv_b} of {n_b} = {p_b:.2%}")
print(f"Relative lift = {(p_b - p_a) / p_a:+.1%}")
print(f"z = {z:.2f}, two-sided p-value = {p_value:.3f}")
print("Significant at 5 %" if p_value < 0.05 else "Not significant at 5 %")

A: 224 of 6000 = 3.73%
B: 307 of 6000 = 5.12%
Relative lift = +37.1%
z = 3.68, two-sided p-value = 0.000
Significant at 5 %

Common A/B testing mistakes include stopping a test as soon as the result looks significant (which inflates false positives), running many tests at once and reporting only the winners, testing with too little traffic to detect realistic differences, and ignoring effects that take time to show up, such as returns or cancellations. Many firms decide sample size and test duration in advance and stick to them.

Estimating market size

Before investing in a product or market, firms estimate how large the opportunity is. A common framework distinguishes three nested figures. The total addressable market (TAM) is total demand for the product category if every possible customer bought. The serviceable available market (SAM) is the part of the TAM the firm's offer can actually serve, given its product, geography and channels. The serviceable obtainable market (SOM) is the share of the SAM the firm can realistically win in the planning period, given competition and its resources. Investors are often skeptical of plans that jump straight from a huge TAM to an assumed small percentage of it; a bottom-up estimate built from customer numbers, purchase rates and prices is more convincing.

Ethics, privacy and first-party data

Marketing research depends on public trust. Research industry codes, such as the Insights Association Code of Standards and Ethics and the international ICC/ESOMAR Code, require researchers to be honest with participants, to obtain informed consent, to protect personal data and to keep research separate from selling. 'Sugging', selling under the guise of research, and 'frugging', fundraising under the guise of research, are considered unethical. In law, there is no single federal consumer privacy statute, but the FTC polices unfair and deceptive data practices, the Children's Online Privacy Protection Act protects children under 13, and a growing number of states, led by California, give consumers rights to know, delete and opt out of the sale or sharing of personal information. BUS-210 lesson 11 explains the current law.

Changes in technology and regulation have pushed marketers toward first-party data, information customers share directly with the firm through purchases, accounts, loyalty programs, surveys and website interactions, and zero-party data, which customers proactively and intentionally share, such as preferences in a style quiz. Third-party data bought from brokers and collected by tracking people across other sites faces more legal and reputational risk, and browser and phone makers have restricted tracking. Building a trusted value exchange, where customers understand what they share and what they get in return, has become a strategic priority.

Analytics also raise fairness issues. Models that predict who will respond to an offer can unintentionally exclude groups, and price-personalization based on inferred characteristics can feel exploitative. Responsible teams document how data is used, test models for unfair outcomes and ask whether customers would be comfortable if they knew how their data was being used.