Foundation models and large language models

In the second week Jules brings a question from the steering committee. Dana Leclair, the CFO, had read that some companies spend tens of millions of dollars training their own AI models, and she wanted to know whether Birchline would need to. Ravi laughs. "Almost certainly not. That is the whole point of foundation models." This lesson explains what foundation models and large language models are, how they are built, why they can do so many things, and what that means for a retailer that wants to use them rather than build them.

What a foundation model is

A foundation model is a large machine learning model trained on a very broad collection of data, usually with little or no human labeling, so that it learns general patterns it can apply to many different tasks. The term was popularized by Stanford researchers in 2021 to capture a shift: instead of training a separate model for each job (one for translation, one for sentiment, one for summaries), organizations start from one large, general model and adapt it.

Three properties define a foundation model. It is trained on broad data, such as large portions of public web text, books, code and images. It is trained at large scale, with billions of parameters and enormous computing budgets. And it is adaptable: the same model can answer questions, write, summarize, classify, translate and reason about images, often with nothing more than a well-written instruction. Jules writes "broad, big, bendable" in his notebook as a memory aid.

The economics follow from those properties. Training a frontier foundation model costs a great deal and needs specialized chips, data pipelines and research teams, so only a handful of organizations do it. Using one is comparatively cheap: a business pays per request or per amount of text processed, and it can start in an afternoon. Dana's question has a clear answer for Birchline: buy access to capable models, invest in the data and processes around them, and keep the option of light tuning for later.

Large language models

A large language model (LLM) is a foundation model specialized for language: it is trained on huge amounts of text, and often code, to predict and generate text. LLMs are the models behind chat assistants, writing tools and coding helpers. Some foundation models are not language models at all (an image generator, for example), and many modern models are multimodal, handling text together with images, audio and video, which lesson 3 covers.

The core idea behind an LLM is surprisingly simple. During pre-training, the model reads a vast amount of text and repeatedly tries to predict the next piece of text, called a token, given everything before it. When it guesses wrong, its parameters are nudged so that next time it would guess better. After trillions of such predictions, the model has absorbed grammar, facts, styles, the structure of arguments and a great deal of knowledge about the world, all encoded in its parameters. Lesson 4 explains tokens and parameters properly.

Next-token prediction sounds too small to explain what LLMs can do, and Ravi admits the scale effect still surprises researchers. To predict the next word of a recipe, a model must in some sense represent cooking. To predict the next line of a legal argument, it must represent the argument. Abilities such as summarizing, translating and step-by-step reasoning emerge from that training without being programmed one by one.

# A toy "language model": count which word follows which in a tiny corpus,
# then generate text by repeatedly choosing the most likely next word.
corpus = ("the tent is waterproof . the tent is light . the jacket is waterproof . "
          "the boots are light . the jacket is warm .").split()

follows = {}
for a, b in zip(corpus, corpus[1:]):
    follows.setdefault(a, {}).setdefault(b, 0)
    follows[a][b] += 1

def next_word(word):
    options = follows.get(word, {})
    return max(options, key=options.get) if options else "."

word, out = "the", ["the"]
for _ in range(4):
    word = next_word(word)
    out.append(word)
print(" ".join(out))
print("after 'is', the model has seen:", follows["is"])

the tent is waterproof .
after 'is', the model has seen: {'waterproof': 2, 'light': 1, 'warm': 1}

The toy model only counts word pairs from five short sentences, so it can only repeat patterns it has seen. A real LLM uses a transformer network with billions of parameters that considers the whole preceding context, not just the last word, and it learns from trillions of tokens. The principle, though, is the same: predict what comes next, one token at a time.

From pre-trained model to helpful assistant

A model fresh from pre-training is good at continuing text but not at following instructions. Ask it a question and it might continue with three more questions, because that pattern appeared in its training data. Model builders therefore add further training stages before releasing a model.

• Pre-training on broad, mostly unlabeled data gives the model general knowledge and language ability. This is the expensive stage.
• Instruction tuning (a form of supervised fine-tuning) trains the model on many examples of instructions paired with good responses, so it learns to follow requests.
• Alignment with human feedback uses people's ratings of model responses, through reinforcement learning or related methods, to make answers more helpful, honest and safe.
• Safety training and filters teach the model to decline harmful requests, and providers add separate safety classifiers around the model in production.

The result is what most people now meet as an AI assistant. When Birchline uses a model from Google Cloud, all of these stages have already been done. Birchline's choices start after them: which model to use, how to instruct it, which company data to connect, and whether to tune it further on Birchline's own examples.

Diffusion models and other generators

Not every generative model predicts tokens one after another. Many image and video generators are diffusion models. During training, a diffusion model is shown images that have been progressively blurred into random noise, and it learns to reverse that process step by step. To generate an image, it starts from pure noise and removes noise over many steps, guided by a text description, until a clear image appears that matches the prompt. Google's Imagen image models and Veo video models, which lesson 3 introduces, are built for this kind of generation.

Other architectures exist, and the field changes quickly, but leaders rarely need the internals. What matters is matching the kind of model to the kind of content: language models for text and code, diffusion and related models for images and video, multimodal models when a task mixes kinds of input and output.

Open and proprietary models

Foundation models also differ in how they are offered. Proprietary models are run by the company that built them and reached through an API; customers cannot download or inspect the parameters. Google's Gemini models are offered this way on Google Cloud. Open models, sometimes described as open-weight models, publish their parameters so that anyone can download, run and modify them under a license. Google's Gemma family is an example of open models, and many other organizations release open models too.

Google Cloud's Model Garden offers both kinds side by side: Google's own models, models from partner companies, and open models that can be deployed with a few clicks. GCP-111 covers Model Garden and how to choose. For now, Jules notes that "open" is not the same as "free of obligations": open models still come with license terms, and running them means Birchline takes on hosting and security work.

What foundation models are good and bad at

Ravi is careful not to oversell. Foundation models are remarkable at working with language and unstructured data: drafting, rewriting in a different tone, extracting fields from messy text, answering questions about documents they are given, translating, classifying by meaning rather than keywords, and writing code. They generalize to tasks no one trained them for explicitly.

Their weaknesses are just as important for a leader. They can hallucinate, producing fluent statements that are false. Their knowledge stops at a training cutoff date unless they are connected to current information. They can reflect biases in their training data. They are not databases: they do not reliably recall exact prices, stock levels or policy wording unless that information is supplied. And they are probabilistic: the same prompt can give different answers on different runs. Lesson 10 covers these limitations in depth, and lessons 5 to 9 cover the techniques that manage them.

For Birchline, that list leads straight to design rules. The shopping assistant must look up live prices and stock rather than remember them. The description writer must be given each product's real specifications. Every output that customers see must be checked, either by a person or by automated tests. Jules adds a second page to his notebook headed "Models are not databases."

Choosing a foundation model

Because many capable models exist, choosing one is a business decision with several factors. The exam expects leaders to weigh them rather than to memorize model names, which change every few months.

• Modality: what goes in and comes out: text only, or images, audio and video too.
• Quality on the task: how well the model does on examples of the real work, measured by evaluation (lesson 9).
• Context window: how much text and other input the model can consider at once (lesson 4).
• Latency: how fast responses arrive, which matters for a shopper waiting in a chat window.
• Cost: price per amount of input and output; smaller models are cheaper and faster.
• Customization: whether the model can be tuned or grounded in the ways the use case needs.
• Data governance and location: where requests are processed and how data is handled, which matters for state privacy laws and card-data rules.
• License and support: the terms of use, and the support and service levels the provider offers.

A common pattern is to use a large, capable model for hard tasks and a small, fast, cheaper model for simple, high-volume ones. Birchline might use a larger model to draft product descriptions once and a smaller one to classify thousands of incoming emails every day.