Costs of AI over the full life cycle

When Lorraine read the first draft of the call-note summarizer proposal, she circled one line in red. The vendor-style estimate said the model would cost "about 150 dollars a year" to run. "That can't be the cost of this project," she wrote in the margin. "What else is there?" She was right to ask. The model calls were indeed cheap. Everything around them was not.

This lesson is about total cost of ownership (TCO): every cost an AI solution creates from the first idea to the day it is retired. The exam guide asks strategists to implement basic cost controls, including cost planning and optimization, and to understand how AWS pricing models affect business cases. Both start with knowing where the costs really are.

The life cycle of an AI solution

Costs arise at every stage of a solution's life. Noor listed them for Andre in six groups.

The operation stage usually lasts longest, so its costs add up most. And within operation, people are often the largest item: the staff who review outputs, follow up on predictions, monitor quality and handle exceptions.

A full TCO for the call-note summarizer

Andre and Noor rebuilt the summarizer estimate from the bottom up, for a three-year period. The emergency hospital takes about 14,600 calls a year. All prices are illustrative and rounded, and AWS prices change, so a real estimate would use the AWS Pricing Calculator with current rates.

Lorraine's red circle was vindicated. The "150 dollars a year" was accurate for the thing it described and almost irrelevant to the decision. This is a common pattern for generative AI used at modest volumes: token prices are small, and the costs that matter are design, integration, evaluation and the people who keep the system safe and useful. At very high volumes, such as a public assistant answering millions of questions, usage costs can become significant, which is why the pricing model still matters.

How AWS prices AI services

The exam guide names three pricing patterns that affect AI business cases. Each behaves differently as use grows.

Choosing the right pattern for a workload is a cost decision in its own right. A workload with spiky, low volume suits consumption pricing. A workload with high, steady volume may be cheaper on reserved capacity. A tool that many people use lightly may be cheaper per seat than per use, or the reverse, depending on how heavily each person uses it.

Commitments and Savings Plans

AWS offers lower prices in exchange for commitments. Savings Plans give a discount on compute in return for committing to a steady amount of hourly spending for one or three years; there are Savings Plans that cover Amazon SageMaker AI usage as well as general compute. Amazon Bedrock offers Provisioned Throughput with optional commitment terms for workloads that need guaranteed capacity. Commitments make sense only when usage is predictable. A pilot whose volume might double or disappear is a poor candidate.

Cost controls in practice

Cost control has two halves: planning, before money is spent, and optimization, while the solution runs. Noor gave Andre a list of basic controls that any AI initiative should have from the start.

• Estimate before building. Use the AWS Pricing Calculator to estimate service costs from expected volumes, and add the people and integration costs it does not cover.
• Tag everything. Apply cost allocation tags, such as project and environment, to every resource so the bill can be broken down by initiative. Cascade Ridge tagged every AI resource with its use case name.
• Set budgets and alerts. Use AWS Budgets to set a monthly budget for each initiative, with alerts when actual or forecast spending passes a threshold.
• Watch the trend. Use AWS Cost Explorer to see spending by service, tag and time, and to spot unexpected growth early.
• Right-size models. Use the smallest model that meets the quality bar. A smaller foundation model can cost a fraction of a larger one per token and respond faster. Test on the evaluation set to find where quality starts to drop.
• Control output and input length. Set a maximum output length and send only the context a task needs; tokens are the meter for most generative AI pricing.
• Batch and cache. Use batch inference for work that does not need an immediate answer, and reuse results or cached prompt prefixes where the platform supports it.
• Limit runaway use. Apply quotas or rate limits to public-facing applications so a surge, an error loop or misuse cannot run up an unlimited bill.
• Review the people costs. Measure the time spent reviewing and acting on AI outputs, because improving precision or confidence routing can reduce it more than any infrastructure change.

Cost per outcome

Total cost is the start, not the end, of cost thinking. A more useful measure for decisions is cost per outcome: what it costs to produce one unit of the result the business wants. For the no-show initiative, the relevant figure is the cost per recovered appointment. With annual running costs of about 34,000 dollars and about 1,480 recovered appointments, that is about 23 dollars each, against a contribution margin of about 90 dollars. Cost per outcome makes initiatives comparable and shows quickly whether a change in design helps or hurts.

It also exposes a trap. Reducing cost per transaction is not the same as reducing cost per outcome. If a cheaper model summarizes calls for less money but produces summaries that technicians must rewrite, the cost per useful note goes up. Lorraine liked to say: "Cheap and wrong is expensive."

Hidden and rising costs

Some costs are easy to miss in early estimates:

• Growth in use. Successful tools are used more. A client assistant that answers 2,000 questions a month in the pilot might answer 20,000 after launch. Estimate costs at the expected scale, not the pilot scale.
• Retraining and updates. Prediction models drift as behavior changes and need retraining. Foundation models are updated or retired by providers, so prompts and evaluations must be revisited.
• Compliance and audit. Logging, retention, reviews and evidence for auditors take time and storage every year.
• Running two systems. During a transition, old and new run side by side, and costs overlap.
• Exit. Leaving a product or partner may involve fees, data export work and a migration project.

Estimating at scale

Pilot costs can mislead in both directions. Some costs grow with use: model usage, transcription, storage and the staff time spent on each output. Others are mostly fixed: integration, monitoring and governance. A good estimate separates the two and projects each to the expected scale. For the client-support assistant, Andre asked the partner to estimate usage costs at three volumes: the pilot volume of about 2,000 questions a month, an expected volume of about 8,000 after launch, and a high case of 20,000 if clients took to it quickly. The usage cost grew tenfold across that range; the fixed costs barely moved.

Projecting at scale also shows when a pricing model should change. At low volume, consumption pricing for the assistant was clearly cheapest. At the high case, the partner suggested that reserved capacity or a smaller model for simple questions might cost less, and that a cache of approved answers to the most common questions would avoid calling a model at all for many of them. None of these changes was needed for the pilot, but knowing the break points in advance meant the group would not be surprised by its bill if the assistant succeeded. Cost planning is partly about preparing for success.

Finally, cost estimates should be revisited after launch. Andre asked Noor to compare actual monthly costs for each initiative against the estimate in its business case, using the tags and Cost Explorer reports, and to explain any gap larger than a fifth. Over time, this comparison improves the accuracy of future estimates, which is one of the less visible benefits of good cost discipline: each business case becomes easier to believe than the last.