Building the business case and measuring value

Dana Leclair, Birchline's CFO, has approved many technology projects and cancelled a few. When Jules brings her the product description pilot, her first question is the one she asks of every project: "What will it cost, what will it return, and how will we know?" She is not hostile to generative AI. She uses Gemini in Gmail every day. But she has read enough vendor slides to distrust numbers that nobody measured. This lesson shows how Jules builds a business case Dana can check, and how the team measures value once the pilot runs.

Why generative AI business cases go wrong

Generative AI business cases fail in a few predictable ways. Some count the time a tool could save without asking whether people will use it, or whether the saved time turns into anything useful. Some leave out costs that appear later, such as reviewing outputs, maintaining the data the model relies on, or training staff. Some promise transformation in the first quarter, then disappoint when the pilot shows a smaller, steadier gain. And some never set a baseline, so nobody can say afterwards whether anything changed.

Farah's advice is to treat the business case as a hypothesis with numbers attached. It states what the team believes, how big the effect might be, what it will cost to find out, and what result would justify going further. A good pilot can then confirm the hypothesis, shrink it or reject it, and each of those is a useful answer.

The benefit side

Jules starts with the copy team's real numbers. Birchline launches about 2,400 new products a year. Each description currently takes a copywriter about 40 minutes, including reading the supplier spec sheet, writing, and fixing the editor's comments. With a grounded first draft, the team estimates 15 minutes per description for review and editing. The fully loaded cost of a copywriter's time, including benefits and overhead, is about 48 dollars an hour.

With those figures, 2,400 × (40 − 15) ÷ 60 gives 1,000 hours a year. At 48 dollars an hour, that is about 48,000 dollars of staff time. Farah immediately asks the question that separates a good case from a vendor slide: what will the team do with the time? The head of merchandising answers that the backlog of products waiting for descriptions, currently about three weeks, delays launches, and the freed time would go to clearing it and writing seasonal buying guides, which the team never has time for. Faster launches have a revenue effect, but it is harder to measure, so Jules lists it as a secondary benefit rather than adding a guess to the total.

Benefits are easier to defend when they are sorted by how firmly they can be measured.

The cost side

Generative AI costs are more than the model's price. Jules lists every cost the pilot and a full rollout would have, including people's time.

• Model usage: most generative AI services charge by the amount of text processed and generated, measured in tokens, or by the number of images or seconds of audio. Each product description needs the spec sheet, style rules and examples as input, plus a few hundred words of output.
• Platform and integration: building the prompt, connecting the catalogue, testing, and the small application editors use. For Birchline this is mostly internal staff time.
• Review: editors still read every draft. That time is part of the after-state cost, not a hidden extra; it is already inside the 15 minutes.
• Data work: cleaning catalogue fields, collecting approved example descriptions, and keeping the style guide current.
• Change and training: teaching the team the new workflow, and the productivity dip in the first weeks while people adjust.
• Ongoing operations: monitoring quality, updating prompts when models change, security reviews, and support.

The model usage cost is often the smallest line for internal productivity tools, and the largest surprises usually come from people's time and from usage growing faster than planned. Jules uses illustrative prices for the estimate and labels them as such, because prices change and depend on the model chosen. The team will replace them with real figures from the pilot's billing export.

# Illustrative figures only. Replace with real pilot data.
items_per_year = 2400
minutes_before, minutes_after = 40, 15
hourly_cost = 48.0                  # loaded cost of a copywriter hour, in dollars

tokens_per_item = 6000              # input and output tokens per description (estimate)
price_per_million_tokens = 2.50     # illustrative blended price, in dollars
build_cost = 22000                  # one-time: integration, testing, training
run_cost_per_year = 9000            # monitoring, prompt upkeep, support

hours_saved = items_per_year * (minutes_before - minutes_after) / 60
benefit = hours_saved * hourly_cost
model_cost = items_per_year * tokens_per_item / 1_000_000 * price_per_million_tokens
yearly_cost = model_cost + run_cost_per_year
net_year1 = benefit - yearly_cost - build_cost
payback_months = build_cost / ((benefit - yearly_cost) / 12)
print(f"Hours saved per year : {hours_saved:,.0f}")
print(f"Benefit per year     : {benefit:,.0f}")
print(f"Model usage per year : {model_cost:,.0f}")
print(f"Net value, year 1    : {net_year1:,.0f}")
print(f"Payback (months)     : {payback_months:.1f}")

Hours saved per year : 1,000
Benefit per year     : 48,000
Model usage per year : 36
Net value, year 1    : 16,964
Payback (months)     : 6.8

The model usage line, at these illustrative prices, comes to only a few dozen dollars a year. The build and run costs dominate. That is typical for a focused internal tool, and it tells Dana where the real risk lies: not in the cloud bill, but in whether the team adopts the tool and whether the 15-minute estimate holds.

Return on investment and payback

Two simple measures summarize a business case. Return on investment (ROI) compares net gain with cost. Payback period is how long it takes for cumulative net benefits to repay the one-time investment.

Costs that grow with success

One more cost line deserves attention because it grows when a project succeeds. If the product description tool works, other teams will want it, the volume of drafts will rise, and the descriptions will be extended to French for Quebec shoppers. Each step adds usage, review time and support. Jules adds a line to the case called growth costs, with a simple rule: every expansion must bring its own small business case, so that success does not quietly turn a cheap pilot into an expensive service nobody budgeted for.

Dana adds two cautions that Jules writes into the case. First, ROI depends on the benefit estimate, so the case shows a range: what happens if editing takes 25 minutes rather than 15? Second, released time is only real value if it is used well, so the case names the buying guides as the planned use and commits to counting them.

build, run = 22000, 9000
for minutes_after in (15, 20, 25, 30):
    benefit = 2400 * (40 - minutes_after) / 60 * 48
    net_month = (benefit - run) / 12
    payback = build / net_month if net_month > 0 else float("inf")
    print(f"{minutes_after} min: benefit {benefit:>7,.0f} a year, payback {payback:5.1f} months")

15 min: benefit  48,000 a year, payback   6.8 months
20 min: benefit  38,400 a year, payback   9.0 months
25 min: benefit  28,800 a year, payback  13.3 months
30 min: benefit  19,200 a year, payback  25.9 months

Even in the cautious case of 30 minutes per description, the tool pays back in under two years. That sensitivity table persuades Dana more than the headline number, because it shows the case does not depend on one optimistic assumption.

Measuring value: baselines and KPIs

A business case is only as good as the measurement that follows. Before the pilot starts, the team records a baseline: how things work now, measured the same way they will be measured later. For the copy team that means timing a sample of descriptions over two normal weeks, counting the backlog, and collecting editor quality ratings on a simple 1 to 5 scale.

Then Jules defines key performance indicators (KPIs) for the pilot. Good KPIs cover more than speed, because a tool that makes work faster but worse is not a success.

Farah adds one technique for fairness. For the first month, the team splits new products at random: half are written the old way and half from AI drafts. Comparing the two groups over the same weeks removes the effect of seasonal workload or a particularly easy batch of products. This kind of controlled comparison, often called an A/B test, is the most convincing way to measure impact when the number of items is large enough.

Measuring what is harder to measure

Some benefits resist neat numbers. A staff policy assistant might reduce the calls to head office, but also give store staff confidence that is hard to quantify. Jules uses a few honest techniques. Proxy measures stand in for the real effect, such as counting calls to the head-office help line. Before-and-after surveys ask users how long tasks take and how confident they feel. Sampled audits check a random set of outputs for accuracy each week. And the team keeps a short log of stories, specific examples where the tool helped or failed, because the steering committee remembers a concrete example longer than a chart.

Measurement also needs a decision point. The pilot charter says that after eight weeks the steering committee will choose one of three paths: expand to all products if editing time is under 25 minutes with no drop in quality; adjust and extend for four weeks if results are mixed; or stop if quality falls or adoption stays low. Agreeing the decision rule in advance stops a pilot from drifting on for months because nobody wants to call it.

Presenting the case

Jules's one-page summary for the steering committee has five parts: the problem in the business's words; the proposed solution and its review step; the costs, one-time and yearly, with assumptions labelled; the expected benefits as a range, with the baseline already measured; and the decision rule at the end of the pilot. It ends with the risks and how each is controlled, a section that later lessons fill in more fully. Dana approves the pilot with one change: the run cost must include a named person who owns the tool after launch. "Projects without an owner," she says, "become problems without an owner."