Review: Techniques to improve model output

The third session takes place in the room where Birchline's review summarizer was tested, and Ravi uses that history on purpose. In October the summarizer wrote that a sleeping bag was rated to minus 30 degrees when no review said so; the team caught it in testing and fixed it with cited claims, a check on every number and a buyer's approval before publishing. "That one near miss covers half of this section," Ravi says. Section 3 of the guide we read covered roughly 20 out of every 100 questions. It asks you to name the limitations of foundation models and the practices that overcome them, to explain the main prompt engineering techniques, and to explain grounding and the settings that shape a model's response. This lesson reviews each objective and then gives you original practice scenarios.

The limitations of foundation models

The guide lists six common limitations. Data dependency: a model is only as good as the data it learned from and the data it is given. Knowledge cutoff: a model knows nothing about events after its training data was collected unless it is given newer information. Bias: a model can reflect and amplify patterns in its training data that disadvantage some groups. Fairness: related to bias, the question of whether outcomes are equitable for different people. Hallucinations: confident, fluent output that is false or unsupported, like the sleeping bag rating. And edge cases: unusual inputs the model handles poorly because it saw few examples like them. A typical question describes a symptom and asks which limitation it shows. An assistant that does not know about a product launched last month shows the knowledge cutoff; an assistant that invents a warranty term shows hallucination.

Practices that overcome them

The guide pairs the limitations with recommended practices. Grounding connects the model to reliable sources at answer time, so its output is anchored in facts; retrieval-augmented generation, or RAG, is the common way to do this, retrieving relevant passages and giving them to the model with the question. Prompt engineering improves the instructions, examples and context in the input. Fine-tuning adapts the model itself on examples, which helps with consistent style, format or specialized behavior. And human in the loop, often shortened to HITL, keeps a person reviewing or approving output where errors would be costly. Knowing which practice fixes which limitation is one of the most tested ideas in this section.

The figure shows why the order matters. Prompt engineering is cheapest and fastest, so try it first. Grounding takes more work, because sources must be indexed and kept current, but it is the right tool whenever facts are the problem. Fine-tuning takes the most effort and needs good training examples, and it does not give the model new facts in a reliable way, so it is rarely the first answer to a factual problem. Human review adds control at any stage, at the cost of people's time. When a question asks for the best first step, the cheapest technique that addresses the actual problem is usually right.

Monitoring and evaluation

Improving output is not a one-time task. The guide lists monitoring and evaluation practices: planning for automatic model upgrades when a provider releases a new version, tracking key performance indicators, applying security patches, versioning models and prompts so that changes can be traced and rolled back, tracking performance over time and monitoring for drift. It also names Feature Store on the platform, which stores and serves consistent data features for models. Birchline's internal policy assistant is tested every month against a fixed set of 200 real questions, and any change to the model, the prompt or the documents is recorded with a version. A question about an assistant whose answers slowly got worse after a model upgrade is testing exactly these ideas.

Prompt engineering techniques

A prompt is the input you give a model. Zero-shot prompting gives an instruction with no examples. One-shot gives a single example of the input and the desired output. Few-shot gives several examples, which helps the model follow a format or style. Role prompting tells the model to act as a particular expert or persona, such as "You are a careful returns specialist". Prompt chaining breaks a task into steps, using the output of one prompt as the input to the next. Chain-of-thought prompting asks the model to work through intermediate reasoning steps before answering. ReAct, short for reasoning and acting, combines reasoning steps with actions such as searching or calling a tool, observing the result and continuing; it is the pattern behind many agents.

# Building a few-shot prompt from examples (no model call)
examples = [
    ("Stove leaked fuel after one trip.", "Product defect"),
    ("Parcel arrived three days late.", "Delivery"),
]
new = "The tent pole snapped on the first night."
lines = ["Classify each customer comment into one category."]
for text, label in examples:
    lines.append(f"Comment: {text}\nCategory: {label}")
lines.append(f"Comment: {new}\nCategory:")
print("\n\n".join(lines))

Classify each customer comment into one category.

Comment: Stove leaked fuel after one trip.
Category: Product defect

Comment: Parcel arrived three days late.
Category: Delivery

Comment: The tent pole snapped on the first night.
Category:

The exam asks you to recognize these techniques from descriptions. A prompt that includes three sample product descriptions before asking for a fourth is few-shot. A prompt that asks the model to list the steps of its calculation before giving a total is chain-of-thought. A workflow where the first prompt extracts facts from a review and a second prompt writes a summary from those facts is prompt chaining. An agent that thinks, searches, reads the result and thinks again is following ReAct.

Grounding and its data sources

The guide distinguishes three kinds of data for grounding. First-party data is the organization's own, such as Birchline's catalog, policies and order history. Third-party data comes from outside partners or providers, such as a supplier's product specifications or licensed industry data. World data is public information, such as the open web. Grounding with first-party data suits questions about the organization itself; grounding with Google Search suits questions about current public events. RAG changes output by putting the retrieved passages into the model's context, so answers can quote and cite them and are far less likely to invent facts. The guide named three Google Cloud grounding routes: prebuilt RAG with Agent Search, RAG APIs for teams building their own pipeline, and grounding with Google Search.

Sampling settings

The last objective covers the settings that shape a response. Output length, or the maximum number of output tokens, caps how long the answer can be, which also limits cost. Temperature controls randomness: low temperature makes the model choose the most likely next words, giving focused and repeatable answers, while high temperature spreads the choice, giving more varied and creative output. Top-p limits the choice to the smallest set of words whose combined likelihood reaches a threshold, another way to control variety. Safety settings control how strictly the model blocks content in categories such as harassment or dangerous content. Token count matters because both input and output are measured and often priced in tokens.

Practice scenarios

Here are original scenarios with reasoning. First: a bank's assistant gives answers about interest rates that were correct last year but not now. Which practice fixes this most directly? Grounding with current sources, because the problem is the knowledge cutoff. Second: a marketing team wants fifty very different slogan ideas for a campaign. Which setting should they raise? Temperature. Third: an insurer needs claim letters in exactly the same structure every time, and prompts with examples have helped but not enough across thousands of letters. What is a reasonable next step? Fine-tuning on a set of approved letters. Fourth: a hospital wants an assistant to suggest discharge instructions, but a mistake could harm a patient. Which practice must be included? Human in the loop, with a clinician approving each output. Fifth: a prompt asks a model to plan a trip by first listing the constraints, then checking each option against them, then recommending one. Which technique is this? Chain-of-thought.

Evaluating whether output improved

Every technique in this section is only worth using if it actually improves the output, and that means measuring. Birchline's habit is a fixed evaluation set: a list of real questions or tasks with agreed good answers, run before and after every change. For the policy assistant it is 200 questions; for the description writer it is 200 descriptions held back from any tuning. Measures can be automatic, such as whether an answer cites a source or whether every number matches the catalog, or human, such as a buyer's rating of accuracy and tone. A question that describes a change made without any before-and-after comparison is pointing at a missing evaluation step.

Evaluation also protects against changes you did not make. When a provider upgrades a model automatically, behavior can shift. A fixed evaluation set run after each upgrade shows whether answers got better, worse or merely different. Versioning the prompt, the model and the grounding sources together means that if results drop, the team can see what changed and roll back.

Safety settings in practice

Safety settings deserve a moment of their own, because questions sometimes pair them with temperature as distractors. They control how strictly the model blocks content in harm categories, such as hate speech, harassment, sexually explicit content and dangerous content. A children's education app would use strict settings; a medical research tool that must discuss injuries frankly might need carefully adjusted ones, with other safeguards around it. Safety settings do not make answers more accurate and do not control variety or length. They are one layer of protection alongside screening of prompts and responses, grounding, and human review.

A last point on this section: the techniques combine. Birchline's shopping assistant uses all of them at once. Its prompt sets a role and gives examples of good answers; it is grounded in the catalog and the policies; it runs at a low temperature with a capped output length; and any answer about a hold or a pickup slot comes from a function rather than from the model's memory. Questions sometimes ask for the single best technique, but in real systems the answer is usually a layered combination, with each technique covering a weakness of the others.

Jules gets nine out of ten on this set. The one miss confused top-p with output length, which goes onto the confusion list for lesson 8.