Fine-tuning and other ways to adapt a model

By early December the description writer is working well with a prompt template and each product's specifications. One complaint remains. The descriptions are accurate, but they do not quite sound like Birchline. The copy team has a distinctive voice, dry and practical with a small joke now and then, built over twenty years of catalogues, and the model keeps drifting back to generic marketing language. Jules asks whether they should "train the model on our catalogue". Ravi says that might be the right answer eventually, but that there is a ladder of options to climb first. This lesson walks up that ladder.

The adaptation ladder

Adapting a model means changing its behaviour for a specific use. The techniques differ in what they change, how much data and effort they need, and what they cost. Ravi draws them as a ladder: start at the bottom and climb only when the rung below is not enough.

• Prompt design (lesson 5): change the instructions. No training, instant iteration.
• Few-shot examples: show the model what good output looks like inside the prompt.
• Grounding and RAG (lesson 6): give the model facts it lacks at request time.
• Prompt tuning: learn a small set of special input values that steer the model, without changing its weights.
• Parameter-efficient fine-tuning: train a small number of extra or selected parameters on task examples.
• Full fine-tuning: retrain all of the model's parameters on task data.
• Training from scratch: build a new foundation model, which almost no business needs.

The key distinction is between techniques that change what the model sees (prompts, examples, grounding) and techniques that change the model itself (the tuning methods). Changing what the model sees is cheaper, faster, easier to update and easier to reverse. Changing the model can teach behaviour that is hard to describe in a prompt, but it needs data, evaluation, versioning and ongoing maintenance.

What fine-tuning is

Fine-tuning continues training a pre-trained model on a smaller, task-specific dataset, adjusting its parameters so it performs that task better. The most common form for business use is supervised fine-tuning: the training data is a set of input and output pairs showing exactly what the model should produce. For Birchline, each pair would be a product's specifications as input and the copy team's approved description as output.

Fine-tuning teaches behaviour: style, tone, format, domain vocabulary and how to handle a specialized kind of task consistently. It is good at making a model write like the copy team, classify emails into Birchline's specific queues with fewer examples in each prompt, or follow a complex output format reliably. It is not a good way to teach facts that change, such as prices, stock or this season's policy. Facts learned in tuning are hard to update and easy to misremember, so changing facts belong in grounding.

Ravi puts the rule on the whiteboard in large letters: tune for behaviour, ground for knowledge. Jules notices that the copy team's voice is behaviour, while the tents' specifications are knowledge. The two techniques work together: a tuned model for the voice, grounded on each product's real specifications for the facts.

Parameter-efficient tuning

Full fine-tuning updates every parameter of a model, which for a large model means enormous compute and a full new copy of the model to store and serve. Parameter-efficient fine-tuning (PEFT) methods update only a small fraction. A widely used method, low-rank adaptation (LoRA), keeps the original weights frozen and trains small additional matrices that adjust the model's behaviour. The trained adapter is tiny compared with the model, cheap to train and store, and can be swapped in and out, so one base model can serve several tuned behaviours.

Most managed tuning services on cloud platforms use parameter-efficient methods behind the scenes. On Google Cloud, the Gemini Enterprise Agent Platform offers supervised tuning of supported Gemini models and tuning of open models such as Gemma; GCP-111 covers the products. For a leader, what matters is that tuning a hosted model is now a matter of preparing good examples and running a managed job, not of building infrastructure.

Prompt tuning, sometimes called soft prompting, is lighter still. Instead of changing the model's weights, it learns a short sequence of numerical vectors that are placed in front of every input, acting like a learned prompt that no person could write in words. It needs less data than fine-tuning and leaves the model untouched. The exam guide lists prompt tuning among core terms, so Jules notes the distinction: prompt engineering writes words; prompt tuning learns vectors.

# Rough comparison of trainable parameters: full fine-tuning vs a LoRA adapter.
# Illustrative model shape, not a real Gemini or Gemma configuration.
layers, hidden = 32, 4096
full = layers * 12 * hidden * hidden          # rough count of a transformer's main weights

rank = 8                                      # LoRA rank
adapted_matrices_per_layer = 4                # e.g. attention projections
lora = layers * adapted_matrices_per_layer * 2 * hidden * rank

print(f"full fine-tuning updates about {full / 1e9:.1f} billion parameters")
print(f"a rank-{rank} LoRA adapter trains about {lora / 1e6:.1f} million parameters")
print(f"that is about 1 in {round(full / lora):,} of the full count")

full fine-tuning updates about 6.4 billion parameters
a rank-8 LoRA adapter trains about 8.4 million parameters
that is about 1 in 768 of the full count

Other ways models are adapted

Three further techniques appear in discussions of adaptation, and leaders should recognize them.

Reinforcement learning from human feedback (RLHF) and related preference methods train a model on people's judgements of which of two responses is better. Model builders use them to make assistants more helpful and safer. Organizations rarely run them themselves, but the idea explains why human review data, such as thumbs-up and thumbs-down ratings on an assistant's answers, is valuable: it can feed future improvements.

Distillation trains a smaller "student" model to imitate a larger "teacher" model's outputs. The student is cheaper and faster to run while keeping much of the teacher's skill on the target task. Birchline might, in time, distil its email-sorting behaviour from a large model into a small one to cut costs at high volume.

Continued pre-training trains a model further on large amounts of unlabelled text from a specialized domain, such as medical literature, before task tuning. It is expensive and suited to organizations with vast domain-specific text and needs a general model does not meet.

What tuning needs

If the ladder leads to tuning, the work shifts to data. Ravi lists what Birchline would need for the brand-voice model.

• Examples: a set of high-quality input and output pairs. Managed services often accept tuning with a few hundred examples, and more good examples generally help; quality matters far more than quantity. Birchline has about 2,000 copy-team descriptions from the last three seasons.
• Cleaning: remove descriptions of discontinued products with outdated claims, fix errors, and exclude anything that should not be learned, such as personal information or confidential supplier terms.
• Held-out evaluation set: keep a portion of examples out of training to test whether tuning actually helped (lesson 9).
• Baseline: measure the best prompt-only result first, so the team knows what tuning adds.
• Ownership and versioning: a tuned model is a new asset. It needs an owner, a version number, a record of the data used and a plan for retuning when the base model is updated or retired.

Jules realizes that the last point is easy to forget. Base models are updated and older versions retired on a published schedule, and a tuned model built on a retired base must be retuned and re-evaluated. Prompt-only and RAG solutions also need retesting when the base model changes, but they carry less to redo.

Costs and trade-offs

Tuning has three kinds of cost. The tuning job itself, charged by the amount of training data processed. The serving cost, which for some tuned models differs from the base model's. And the people and process costs of preparing data, evaluating results and maintaining versions, which are usually the largest. Against those, tuning can reduce costs: a tuned model may need shorter prompts (fewer few-shot examples on every request), and a tuned small model may replace a large one.

# When does tuning a small model pay off against few-shot prompting a large one?
# All prices are illustrative, in US dollars per million tokens.
large_in, large_out = 1.25, 10.00
small_in, small_out = 0.10, 0.40
requests = 20_000                     # emails per month
fewshot_prompt, tuned_prompt, output = 1_200, 250, 20
tuning_job_once = 150.0               # illustrative one-time tuning charge

large_month = requests * (fewshot_prompt * large_in + output * large_out) / 1e6
tuned_month = requests * (tuned_prompt * small_in + output * small_out) / 1e6
print("large model with few-shot prompt, per month:", round(large_month, 2))
print("tuned small model, per month:", round(tuned_month, 2))
print("months to recover the tuning job:", round(tuning_job_once / (large_month - tuned_month), 1))

large model with few-shot prompt, per month: 34.0
tuned small model, per month: 0.66
months to recover the tuning job: 4.5

With these illustrative numbers the tuning job pays for itself in a few months, but only if the tuned small model is as accurate as the large one, which the evaluation must show. Ravi's point is the method, not the numbers: compare total cost, including people's time, against measured quality.

Preparing a tuning dataset

Jules asks what a tuning example actually looks like. Ravi opens a sample: each line of the file is one example with an input, the product's specifications in the same format the production prompt uses, and an output, the approved description. Consistency matters. If half the examples include a heading and half do not, the tuned model will be inconsistent too. The inputs should look exactly like what the model will see in production, so that it learns the real task. And the outputs should be the best examples available, because the model will imitate their quality, including any mistakes. Ravi's team spends more time choosing and cleaning the 2,000 descriptions than it expects to spend on the tuning job itself.