Deploying a model and using the playground

A model in the catalog is like a product on a shop shelf: you can read about it, but you cannot use it until you take it home. In Foundry, taking a model home means deploying it. Arjun and Maya deploy the chat model they chose in lesson 2, and then spend most of a session in the playground, the part of the Foundry portal where you talk to a deployed model, adjust its settings and write the instructions that turn a general model into Ask Finch. The AI-901 skills outline lists both steps: deploying a model and interacting with it in the Foundry portal, and creating effective system and user prompts.

What a deployment is

A deployment is a named, callable instance of a model in your Foundry resource. When you deploy, you choose the model and its version, give the deployment a name, pick a deployment type and set how much capacity it may use. Applications then call the deployment by its name. Because the name is yours, you can deploy the same model twice under different names, perhaps one for development and one for production with more capacity, or replace the model behind a name later without changing application code.

Arjun names the deployment askfinch-chat rather than using the model's own name. "If we switch models next year," he says, "I want to change one setting, not every line of code that mentions the old model."

Deployment types

Deployment types decide where requests are processed and how capacity is paid for. Names and availability change, and not every type exists for every model, so check the model card, but the main ideas are stable.

Microsoft's documentation distinguishes where data is stored from where it is processed. For global deployments, stored data such as uploaded files stays in the resource's geography, but the prompts and responses may be processed in another region. For Harrow & Finch, Arjun chooses a United States data zone deployment for production, so that processing stays in the country, and a global deployment for the development project, which uses only made-up data.

Capacity for pay-as-you-go deployments is expressed as a rate limit, usually tokens per minute and requests per minute, drawn from a quota your subscription has for each model and region. If an application sends more than its limit, extra requests are rejected with a too-many-requests error and must be retried after a pause. Provisioned throughput instead reserves a fixed amount of processing capacity, which gives predictable speed for a predictable bill.

The chat playground

Once a deployment exists, the chat playground lets you talk to it without writing code. The screen has three main parts. A setup area holds the instructions, often called the system message, that tell the model how to behave. A chat area is where you type user messages and read the replies. And a parameters area holds settings that change how the model generates text. Most playgrounds also show a view-code button that produces sample code for the same conversation, which is a quick way to move from experiment to application in lesson 4.

The playground is for exploring, not for production. Nothing you type there becomes part of an application until you copy the settings into code or into an agent. It is, however, the fastest way to learn how a model responds to different instructions, and Arjun wants Maya to try at least twenty variations before they settle on one.

System messages and user messages

A chat model receives a conversation made of messages, each with a role. The system message, sometimes called the developer message or instructions, comes first and sets the rules for the whole conversation: who the assistant is, what it should and should not do, what tone to use and how to format answers. User messages are what the person types. Assistant messages are the model's earlier replies, which are sent back with each new turn so the model can follow the conversation.

Writing a good system message is the single most effective way to shape a model's behavior, and the AI-901 outline lists it as a skill on its own. Arjun's guidance for Maya is practical.

• Say who the assistant is and who it serves: an assistant for Harrow & Finch customers, not a general chatbot.
• State what it must do and must not do, plainly: answer questions about returns, store hours and orders; do not give legal or medical advice; never promise refunds that the policy does not allow.
• Say what to do when it does not know: admit it and offer a human agent, rather than guessing.
• Describe the tone and length: friendly, concise, plain language, two or three short paragraphs at most.
• Give the format when it matters: a numbered list for steps, a single sentence for yes-or-no questions.
• Include one or two short examples of good answers if the style is hard to describe in words.

You are Ask Finch, the customer-service assistant for Harrow & Finch Home Supply.
Help customers with returns, store hours and the status of their orders.
Use only the information provided to you. If you are not sure, say so and offer to connect the customer with a person.
Never promise a refund, discount or exception that the return policy does not allow.
Do not give legal, medical or electrical-safety advice; suggest a qualified professional instead.
Be friendly and brief: no more than three short paragraphs. Use numbered steps for instructions.

A user prompt is what the customer types, and while Ask Finch cannot control what customers write, Maya can shape the user prompts used in testing and in other internal tools. Good prompts are specific about the goal, give the needed context, and say what form the answer should take. "Summarize this chat" gets a vague paragraph. "Summarize this chat for the next agent in three bullet points: the customer's problem, what has been tried, and what the customer wants" gets something an agent can use.

Parameters

The playground's parameters change how the model picks its words. The names vary slightly between models, and some reasoning models accept fewer of them, but the common ones appear on the exam.

Temperature is the one Maya needs to understand properly. At each step, the model calculates a probability for every possible next token. With temperature near zero, it almost always takes the most probable token, so the same question gets nearly the same answer every time. With a higher temperature, less probable tokens get a better chance, which makes writing more varied and sometimes more creative, and also more likely to drift from the facts. For a customer-service assistant, consistency beats creativity. For drafting marketing copy, Marcus Feldt's team might prefer a higher setting.

# How temperature reshapes next-token probabilities (toy example with four candidate words).
import math
scores = {"Yes": 2.0, "Usually": 1.2, "No": 0.4, "Perhaps": 0.1}   # model's raw scores
def probabilities(temp):
    exp = {w: math.exp(s / temp) for w, s in scores.items()}
    total = sum(exp.values())
    return {w: v / total for w, v in exp.items()}
for t in (0.2, 0.7, 1.5):
    p = probabilities(t)
    print(f"temperature {t}: " + ", ".join(f"{w} {v:.2f}" for w, v in p.items()))

temperature 0.2: Yes 0.98, Usually 0.02, No 0.00, Perhaps 0.00
temperature 0.7: Yes 0.67, Usually 0.21, No 0.07, Perhaps 0.04
temperature 1.5: Yes 0.45, Usually 0.27, No 0.16, Perhaps 0.13

The output shows the effect: at 0.2 almost all the probability sits on the top word, while at 1.5 the alternatives get a real share. The model has not changed, only how adventurous it is allowed to be when it chooses.

Testing in the playground

Maya spends the session working through the fifty test questions, changing one thing at a time. She learns three lessons that every prompt writer learns. First, small wording changes can matter: adding "use only the information provided" cut the number of invented store hours sharply. Second, instructions placed at the end of a long system message are sometimes followed less reliably than ones at the start, so the most important rules go first. Third, the model cannot know facts it has not been given. Without the actual return policy, Ask Finch makes up a plausible one, which is exactly the problem lesson 5 solves with grounding.

She keeps a simple log: the version of the system message, the parameter settings, the question, the answer and whether it was acceptable. That log turns playground experiments into evidence the AI review group can read, and it becomes the starting point for the formal evaluations in lesson 12.

From playground to application

When Maya is happy with the instructions and settings, Arjun shows her the view-code option. It produces a short program, in Python or another language, that sends the same system message and settings to the askfinch-chat deployment. That code is where lesson 4 begins. He also saves the system message and parameter values in the project's shared notes, because the playground does not keep a permanent, versioned record the way source code does.