Design considerations for foundation model applications

Noor Siddiqui began the first AI hour of the new course with a sticky note on the meeting-room wall in the Calgary head office. It said, in capital letters, WHAT IS THE JOB? Jonah Petrakis, who had spent the last course learning how machine learning models are trained and measured, had brought a list of model names he had read about. Noor set the list aside.

"Every week someone tells me which model they want," she said. "Almost nobody tells me what the model is supposed to do, for whom, how fast, in which language, and what happens when it is wrong. Choose the job first. The model is the last decision, not the first."

Cascade Ridge Veterinary Partners had two generative AI ideas left over from the assessment in AWS-111. The first was Night Notes: the 24-hour emergency hospital takes dozens of after-hours phone calls from worried pet owners, and the call handlers type rushed notes. A model could turn those notes into a tidy summary for the on-call veterinarian, who still decides how urgent each case is. The second was Front Desk Helper: reception staff at eleven clinics keep asking the same questions about cancellation rules, payment plans, kennel cleaning steps and what to say when a client asks for records. The answers are in about 140 standard operating procedures (SOPs) and policy documents that nobody can find quickly. A model could answer staff questions from those documents and show where each answer came from.

This course follows Jonah and Noor as they design both applications on AWS. It matches the third domain of the AWS Certified AI Practitioner exam, Applications of Foundation Models, which AWS gives the largest weight of the five domains. This first lesson covers the decisions that come before any prompt is written: what a foundation model application is made of, how to choose a model, and which constraints shape the design.

What a foundation model application is

A foundation model (FM) is a large model trained on broad data that can be adapted to many tasks. A large language model is one kind of foundation model; others generate images, understand images and text together, or produce embeddings. AWS-112 explained how these models work. This course is about using them inside real applications.

A foundation model application wraps a model in everything else a business needs. Noor drew the pieces on the whiteboard for Jonah:

• The user interface: a chat window, a form, a button inside an existing system, or no interface at all for a batch job that runs overnight.
• Instructions and templates: the fixed text that tells the model its role, rules and output format, with blanks for each request. Lessons 2 and 3 cover them.
• Context: the information the model needs that it was never trained on, such as the clinic SOPs. Retrieval-augmented generation, in lessons 5 and 6, supplies it.
• The model and its settings: which foundation model, and inference parameters such as temperature and maximum output length. Lesson 4 covers the parameters.
• Tools: actions the application lets the model request, such as looking up an appointment. Lesson 7 covers agents and tool use.
• Safety controls: guardrails that check inputs and outputs, plus permissions that limit what the application can reach.
• Measurement: logs, evaluations and cost tracking, covered in lessons 9 to 11.

On AWS the usual starting point is Amazon Bedrock, a fully managed service that offers foundation models from Amazon and other providers through one API, with built-in features for knowledge bases, guardrails, agents, prompt management and evaluation. Teams that need more control can deploy models themselves with Amazon SageMaker AI, for example by picking a model from SageMaker JumpStart and running it on endpoints they size and manage. For staff who simply want an assistant rather than a custom application, AWS also offers ready-made assistants, such as Amazon Quick for business users working with their company's data and documents. Amazon Quick is now AWS's recommended choice for this kind of assistant: Amazon Q Business, the earlier service, closed to new customers in 2026, with existing customers encouraged to move to Quick. The AI Practitioner exam expects you to know when each fits: Bedrock for building applications quickly on managed models, SageMaker AI for full control over training and hosting, and a ready-made assistant when one does the job.

Start with the job

Noor's first worksheet had five questions, and she made Jonah answer them for both ideas before any model was named.

The answers already narrowed the design. Night Notes needs a model that follows a format reliably and handles shorthand and typos; it does not need outside knowledge. Front Desk Helper needs the clinic's own documents, so retrieval matters more than the model's general knowledge. Front Desk Helper needs low latency; Night Notes can wait for its shift-end batch, which also costs less. Neither should make clinical decisions: Dr. Simone Kaur, the medical director, had made that rule in AWS-111, and it applied here too. Night Notes would summarize, never rank urgency, and Front Desk Helper would refuse clinical questions and point staff to a veterinarian.

"Look at what we have not said yet," Noor told Jonah. "We have not said 'the biggest model'. Bigger is slower and costs more per request. We need a model that is good enough for the job, and we will prove it with tests in lesson 9."

Criteria for choosing a foundation model

The exam guide lists the criteria a practitioner weighs when choosing a foundation model. Noor turned them into a checklist.

• Cost. Most managed models charge by the number of input and output tokens. A model that costs five times as much per token may be worth it for a hard task and wasteful for a simple one.
• Modality. Text in and text out, images in, images out, audio, or embeddings. Night Notes is text to text; a future radiograph quality check would need an image-capable model.
• Latency. How long until the first words appear and until the answer is complete. Smaller models are usually faster.
• Multilingual ability. Cascade Ridge writes some client letters in French, so any model used for client-facing text must handle that language well.
• Model size and complexity. Larger models generally reason better but cost more and respond more slowly. Many tasks, such as summarizing to a template, work well with smaller models.
• Customization. Whether the model can be fine-tuned or adapted, if prompting and retrieval are not enough. Lesson 8 covers the options.
• Input and output length. The context window limits how much text fits in one request, and output limits cap how long a response can be. A long SOP may not fit in one prompt, which is one reason for retrieval.
• Prompt caching. Some models on Bedrock can cache a long, repeated beginning of a prompt, such as the same instructions sent with every request, so later requests reuse it at lower cost and latency.

Two more criteria matter in practice even when an exam question does not name them. Availability in the Region: not every model is offered in every AWS Region. Cascade Ridge runs in ca-central-1, Canada (Central), and Hannah Weiss, the privacy lead, wanted client information processed in Canada. Some Bedrock features use cross-Region inference, which can route a request to another Region to find capacity, so the team agreed to confirm where any model or inference profile actually processes data before using it with client details. Licence and terms: each model provider sets terms of use, which the legal team should read once, early.

Matching criteria to the two applications

Jonah built a rough comparison using three placeholder models from the Bedrock catalogue, which Noor insisted they call Small, Medium and Large so they would not get attached to a brand before testing. The scores came from a quick trial on 30 sample notes.

# Illustrative scores (1 = poor, 5 = excellent) from a quick 30-note trial.
models = {
    "Small":  {"cost": 5, "latency": 5, "quality": 3, "multilingual": 3},
    "Medium": {"cost": 4, "latency": 4, "quality": 4, "multilingual": 4},
    "Large":  {"cost": 2, "latency": 2, "quality": 5, "multilingual": 5},
}
# Weights reflect what each application cares about (they add up to 1).
weights = {
    "Night Notes":       {"cost": 0.25, "latency": 0.35, "quality": 0.35, "multilingual": 0.05},
    "Front Desk Helper": {"cost": 0.30, "latency": 0.30, "quality": 0.30, "multilingual": 0.10},
}
for app, w in weights.items():
    scores = {m: round(sum(w[k] * v[k] for k in w), 2) for m, v in models.items()}
    best = max(scores, key=scores.get)
    print(app, scores, "-> best:", best)

Night Notes {'Small': 4.2, 'Medium': 4.0, 'Large': 3.2} -> best: Small
Front Desk Helper {'Small': 4.2, 'Medium': 4.0, 'Large': 3.2} -> best: Small

For both jobs the Medium model came out ahead, but only narrowly, and the trial was tiny. Noor wrote "provisional" beside the result. A weighted score like this is a way to make trade-offs visible and to argue about the weights, not a final answer. The real decision would come from a proper evaluation with hundreds of examples in lesson 9.

Constraints that shape the design

Some constraints have nothing to do with the model and everything to do with the business.

Privacy. Night Notes would handle owners' names, phone numbers and sometimes addresses. Under PIPEDA and Alberta's and British Columbia's private-sector privacy laws, Cascade Ridge must protect that information and use it only for the purposes it was collected for. Hannah's requirements were simple: send the model only what it needs, mask what it does not need, never use client data to train a model without a privacy review, and keep logs secured. On Bedrock, AWS states that prompts and responses are not used to train the base models and are not shared with model providers, which Hannah confirmed in the service documentation and recorded in her notes.

Accuracy and accountability. Generative models can produce fluent text that is wrong, which is called a hallucination. The design must make errors visible and recoverable. Night Notes would always show the original call note beside the summary. Front Desk Helper would always show which SOP and section an answer came from.

Security. The application must not let a clever prompt make the model reveal other clients' data or perform actions it should not. Lesson 3 covers prompt misuse defensively, and the IAM practices from AWS-102 apply: the application's role gets only the permissions it needs.

Cost. Lorraine Beaudry, the CFO, asked for a monthly estimate before any pilot. Lesson 11 shows how to estimate and watch the cost of each interaction.

Build, buy or configure

Before designing anything custom, Noor asked whether a ready-made assistant would do. Amazon Quick, for example, can answer questions over a company's connected documents with permissions applied, without the team writing an application. For Front Desk Helper that was a real option. For Night Notes, which needed a fixed output format inside the emergency hospital's call software, a small custom integration through Bedrock looked simpler. The team decided to prototype Front Desk Helper on Bedrock Knowledge Bases, because it let them learn every moving part, and to compare it with a ready-made assistant before any pilot. Choosing the least complex option that meets the need is itself a design principle.

At the end of the hour, Jonah rewrote the sticky note. Under WHAT IS THE JOB? he added three more lines: WHO READS IT? WHAT IF IT IS WRONG? WHAT DOES IT COST? Noor left it on the wall for the rest of the course.