AWS-121 · Lesson 3 of 12 · 19 min
Is AI the right fit for this problem
▶ Video coming soon A video version of this lesson is on the way.
One of the ideas that survived the canvas stage looked, at first, like an easy win. The finance team wanted "an AI that checks invoices" from suppliers. Andre was ready to score it until Noor asked what the check actually was. The answer: confirm that each invoice matched a purchase order and a delivery record, and flag any line where the quantity or the price differed. "That's three lookups and a comparison," Noor said. "You don't need AI. You need the accounting system's matching feature switched on."
This lesson is about that moment. The exam guide expects a strategist to determine when AI is not the appropriate solution for a business problem. It is one of the most practical skills in the whole domain, because every AI project that should not have been started wastes money that a better project could have used, and some create risks that a simpler solution would have avoided.
A ladder of solutions
Noor described the options as a ladder. Each rung is more powerful and more flexible than the one below it, and also more expensive, less predictable and harder to explain. The right choice is usually the lowest rung that solves the problem.
| Rung | What it is | Good for | Cascade Ridge example |
|---|---|---|---|
| Fix the process | Change the steps, roles, forms or policies | Problems caused by unclear responsibility, missing steps or bad incentives | Lab reports arriving by fax at some clinics: ask the labs to send electronic results instead |
| Use an existing feature | Switch on or configure something already in a system | Common needs that software vendors have already solved | Three-way invoice matching in the accounting system |
| Rules and automation | Explicit if-then logic, scripts, workflows | Stable, well-defined decisions with clear criteria | Vaccine due dates set by protocol; reminder texts sent two days before every booking |
| Traditional machine learning | Models that learn patterns from labeled historical data | Predictions and classifications from structured data where patterns are too complex for rules | Predicting which bookings are likely to be missed |
| Generative AI | Foundation models that understand and produce language, images or code | Unstructured content: summarizing, drafting, answering questions from documents | Drafting a call-note summary from a transcript |
| Agentic AI | Systems that plan and act through tools with some autonomy | Multi-step tasks across systems where actions can be checked and reversed | A future booking agent that can find and hold a slot, with a person confirming |
The invoice idea moved two rungs down. So did a request to "use AI to decide which clients get the dental-month reminder": the clinic's own rule (every dog and cat over three years old without a dental cleaning in the past year) was clear, auditable and easy to apply with a simple query.
Signs that AI is the wrong tool
Several patterns suggest that AI is not the right solution, or not yet. None is absolute, but each should prompt a hard question.
- The rules are known and stable. If experts can write down the decision as clear criteria that rarely change, rules are cheaper, faster, fully predictable and easy to explain. Vaccine due dates are the classic Cascade Ridge example.
- Every answer must be exactly right and fully explainable. Some decisions require a precise, auditable result: tax calculations, drug dosage tables, legal deadlines. A model that is right 98 times in 100 is not good enough if the other two cases cause serious harm and nobody can explain why they went wrong.
- There is not enough suitable data. Machine learning needs examples, usually many, that represent the situations the model will face. A new clinic service with six months of history is not ready for a prediction model.
- The volume is too low. If a task happens twelve times a year, the cost of building, testing and maintaining an AI solution will rarely be repaid. A checklist or a template may do.
- The cost of errors is high and errors cannot be caught. If a wrong output would cause serious harm before anyone could notice, AI should not act alone, and perhaps should not be used at all.
- The real problem is somewhere else. Long wait times on the phone line may come from staffing at the wrong hours rather than a lack of automation. Fixing the schedule may beat any chatbot.
- The organization cannot support it. If nobody will own the system, monitor it and act on what monitoring shows, the system will drift and fail quietly.
Noor added one more, specific to generative AI: "If the task needs a fact, give the system the fact, don't ask the model to remember it." A model asked for a clinic's opening hours from memory may invent them. The same model given the current hours in its instructions, or retrieving them from an approved document, is far safer. Sometimes, though, the simplest answer is just to show the hours page.
Signs that AI is a good fit
The opposite patterns suggest AI is worth considering:
- The decision depends on many factors interacting in ways too complex for hand-written rules, such as the combination of booking lead time, past attendance, appointment type, weather and day of week behind a no-show.
- The input is unstructured, such as free text, speech, images or documents in many layouts, and the task is to understand, summarize, classify or extract from it.
- The task is frequent enough that small savings per case add up.
- Errors are tolerable or can be caught: a person reviews the output, or the action is easily reversed, or the cost of a mistake is small.
- Suitable data exists, or can be gathered lawfully and affordably, and someone owns it.
- The pattern changes over time, so a system that can be retrained on new data is easier to maintain than rules someone must keep rewriting.
Traditional machine learning or generative AI
Once AI is a reasonable fit, the next question is which kind. A common mistake in 2026 was to reach for generative AI for everything, because it was the most visible. Many business problems are better served by traditional machine learning, which is usually cheaper to run, faster, easier to evaluate against a clear right answer, and often more accurate for numeric prediction from structured data.
| Question | Points toward traditional machine learning | Points toward generative AI |
|---|---|---|
| What is the input? | Rows and columns of structured data | Text, speech, images or mixed documents |
| What is the output? | A number, a category or a ranking | Text, an answer, a summary, an image or code |
| Is there one right answer? | Usually yes, and it can be checked against history | Often several acceptable answers; quality is judged |
| How is it customized? | Trained on your own labeled examples | Prompting, retrieval from your documents, sometimes fine-tuning |
| Running cost pattern | Small per prediction; can run in batches | Priced per token or per use; grows with volume and length |
The no-show idea is a clear case for traditional machine learning: structured booking data, a yes-or-no outcome, and three years of history to learn from. The call-note summary is a clear case for generative AI: the input is speech turned into free text, and the output is a structured summary. The client-support assistant needs generative AI to understand questions phrased in many ways, but its answers should come from approved documents rather than the model's memory, which is why the design uses retrieval from a knowledge base.
Hybrid answers are common
Real solutions often combine rungs. The lab-report idea used a managed document extraction service, which is machine learning, to pull values out of PDFs, and then simple rules to check each value against its reference range and units before a person approved it. The no-show idea used a model to score bookings, a rule to decide who gets an extra reminder (scores above a threshold), and a process change so reception staff offered freed slots to a waitlist. Thinking in rungs does not mean choosing only one; it means using the simplest tool for each part of the job.
Transitions between solutions
The exam guide also asks you to evaluate what matters when moving a business process to an AI-based solution, or from one AI platform to another. The same considerations apply in both directions, and Noor listed four that the guide names explicitly:
| Consideration | Questions to ask | Example |
|---|---|---|
| Business continuity | What happens if the new system fails or performs badly at launch? Is there a fallback? Can the old way be restored quickly? | Reception keeps the manual lab-report process available for the first two months |
| Cost implications | What are the one-time costs of switching, the change in running costs, any overlap period when both run, and exit costs from the old system? | Running the old and new systems side by side for six weeks doubles some costs for that period |
| Data readiness | Is the data the new system needs available, clean, in the right format and lawful to use? Will historical data move with it? | Three years of appointment history must be cleaned before the no-show model can be trained |
| Performance impact | Will speed, accuracy or user experience change? How will you know before the full switch? | A new model gives better recall but slower responses; test both on real workloads before committing |
Moving between AI platforms deserves a specific warning. Prompts, evaluation results and guardrail settings tuned for one foundation model often behave differently with another. A switch that looks like a simple change of supplier can require re-testing the whole application. Keeping a fixed evaluation set, such as the 300 test transcripts Dr. Kaur's team used for the call-note summarizer, makes this much easier, because the new option can be measured against the same standard.
Worked problem
Screening three ideas
Apply the fit screen to three ideas: (a) send every client a text two days before each appointment; (b) estimate next month's demand for each of 400 pharmacy products at each clinic; (c) answer clients' questions about a clinic's parking.
Show solution
- (a) The rule is clear and the same for every booking. Use automation; no AI is needed.
- (b) Demand depends on season, outbreaks, client numbers and past orders, interacting in complex ways, and there is history to learn from. Traditional machine learning (forecasting) fits, once product codes are cleaned.
- (c) The facts are few and stable. Put them on each clinic's web page and in the booking confirmation. A generative assistant could answer, but only from an approved document, and a web page solves most of the need for almost no cost.
Answer: Automation for (a), traditional machine learning for (b) after data cleanup, and a simple content fix for (c), with generative AI optional only as part of a wider assistant.
Questions to ask before saying yes to AI
Noor kept a short list of questions on a card and asked them whenever an idea reached her desk. They work equally well for a strategist reading a scenario on the exam:
- What exactly is the decision or task, and who makes or does it today?
- Could the problem be solved by changing the process, switching on an existing feature or writing a rule?
- How often does it happen, and what does each occurrence cost now?
- What data exists, who owns it, and may it lawfully be used for this purpose?
- What happens when the output is wrong, who notices, and how quickly?
- Who will own the solution after launch, and who will watch whether it still works?
If the answers show a clear, stable rule, a low volume, missing data or errors that would go unnoticed, the idea goes back down the ladder. If they show a frequent, complex task on unstructured or richly structured data, with errors that can be caught and an owner ready to look after it, the idea moves on to scoring. Either outcome is a success, because both save money that would otherwise have been spent finding out the hard way.
Common mistakes
MistakeUsing generative AI because it is the newest option.
InsteadChoose the lowest rung that solves the problem; many needs are met by rules, existing features or traditional machine learning.
MistakeUsing a model for a decision with clear, stable rules.
InsteadWrite the rules; they are cheaper, predictable and explainable.
MistakeAsking a generative model to recall facts it was never given.
InsteadSupply facts in the prompt or through retrieval from approved documents, or simply publish them.
MistakeTreating a platform switch as a simple change of supplier.
InsteadRe-test prompts, guardrails and performance against a fixed evaluation set before switching.
Summary and key terms
- AI is not always the right answer.
- Consider a ladder of solutions from process changes and existing features through rules, traditional machine learning, generative AI and agentic AI, and choose the lowest rung that solves the problem.
- AI is a poor fit when rules are clear and stable, when every answer must be exact and explainable, when data or volume is too low, when errors are costly and cannot be caught, or when the real problem lies elsewhere.
- Traditional machine learning suits structured predictions; generative AI suits unstructured content.
- Transitions to AI or between platforms require attention to business continuity, cost, data readiness and performance.
- Rules-based automation
- Explicit if-then logic that applies known criteria the same way every time.
- Traditional machine learning
- Models trained on labeled historical data to predict or classify.
- Generative AI
- Foundation models that produce text, images or code from prompts.
- Hybrid solution
- A design that combines AI with rules and process changes, each used where it fits.
- Business continuity
- Keeping essential work going when a system changes or fails.
- Fallback
- A way of working that can be restored if a new system fails.
Practice problems
Work each problem on paper first, then reveal the answer to check.
Give three signs that AI is not the right solution.
Show answer
For example: clear stable rules, a need for exact explainable answers, too little data or volume, uncatchable high-cost errors, or a problem that lies elsewhere.
Why might traditional machine learning beat generative AI for the no-show problem?
Show answer
The input is structured, the output is a yes-or-no prediction that can be checked against history, and it is cheaper and easier to evaluate.
Name the four transition considerations from the exam guide.
Show answer
Business continuity, cost implications, data readiness and performance impact.
Lesson quiz · 5 questions
Score 60% or more to complete this lesson.
Create a free account to save your progress, take quizzes and earn a certificate.