What makes a system an agent

The word agent is everywhere in AI today: coding agents that edit repositories, browser agents that book travel, customer service agents that resolve tickets, research agents that search and summarize, and robots that act in the physical world. The idea is much older than large language models. Since the early days of AI, researchers have studied systems that perceive an environment, decide what to do and act to achieve goals, often over many steps and in the presence of uncertainty and other agents. This course connects the classical foundations, such as search, planning, decision theory, reinforcement learning and game theory, with modern agents built on language models, and it covers how to evaluate and control them. This first lesson defines agents, describes the environments they operate in, and introduces the main agent designs.

Agents and environments

An agent is anything that perceives its environment through sensors and acts on it through actuators. A thermostat senses temperature and switches a furnace; a robot vacuum senses obstacles and dirt and drives its wheels; a software agent receives text, files and tool results and acts by calling tools, writing files or sending messages. Russell and Norvig describe the agent function as a mapping from the history of percepts to an action, and the agent program as the concrete implementation of that mapping on some hardware.

An agent is judged by a performance measure defined by its designer or user: how clean the floor is and how much energy is used, how many tickets are resolved correctly and how satisfied customers are, how many tests pass and how readable the code is. A rational agent chooses actions expected to maximize its performance measure given what it has perceived and what it knows. Rationality is not omniscience: an agent can act rationally and still fail because of information it could not have had. Specifying the performance measure well is surprisingly hard, and many failures of real agents come from measures that reward the wrong thing, a theme that returns in Lesson 12.

Describing tasks: PEAS

A useful way to specify an agent's task is PEAS: performance measure, environment, actuators and sensors. For a warehouse picking robot, performance includes items picked per hour, errors and safety incidents; the environment is the warehouse with shelves, bins, people and other robots; actuators are wheels, an arm and a gripper; sensors are cameras, depth sensors, encoders and barcode readers. For a software engineering agent, performance includes correct, tested changes delivered within budget without breaking other functionality; the environment is a code repository, a terminal, tests and documentation; actuators are file edits and commands; sensors are file contents, command outputs and test results. Writing a PEAS description before building an agent clarifies what success means and what the agent must be able to observe and do.

Properties of environments

Environments differ in ways that determine how hard the agent's problem is:

• Fully or partially observable: can the agent see everything relevant? A chess program sees the whole board; a poker agent cannot see opponents' cards; a coding agent sees only the files it opens.
• Deterministic or stochastic: do actions have predictable results? A file write is deterministic; a web request may fail; a robot's grasp may slip.
• Episodic or sequential: are decisions independent, or does each affect the future? Classifying images is episodic; navigating, negotiating and multi-step tasks are sequential.
• Static or dynamic: does the environment change while the agent deliberates? A crossword is static; traffic and markets are dynamic.
• Discrete or continuous: are states, time and actions countable? Board games are discrete; driving is continuous.
• Single-agent or multi-agent: are there other agents whose actions matter, cooperative or competitive?
• Known or unknown: does the agent know the rules of how the environment works, or must it learn them?

The hardest environments are partially observable, stochastic, sequential, dynamic, continuous, multi-agent and unknown, which describes the real world and many open-ended software tasks.

Agent designs

Agent programs range from simple to sophisticated:

• Simple reflex agents choose actions from the current percept using condition–action rules: if dirty, clean; if obstacle, turn. They work only when the right action can be determined from the current percept alone.
• Model-based reflex agents maintain an internal state, a model of the parts of the world they cannot currently see, updated from percepts and knowledge of how actions change the world.
• Goal-based agents consider the future, choosing actions that lead to states satisfying a goal, using search and planning (Lessons 2 and 3).
• Utility-based agents rank outcomes by a utility function, trading off goals, risks and costs, and handle uncertainty with expected utility (Lesson 4).
• Learning agents improve their performance from experience, adjusting their models, policies or utility estimates (Lesson 5).

Modern language model agents combine these ideas: they keep state in a conversation and memory, pursue goals set in natural language, weigh options implicitly or explicitly, and can be improved through training and feedback.

Reflex versus model-based behaviour

The value of internal state is easy to demonstrate. In the following simulation, two agents clean a corridor of ten squares, each initially dirty with probability one half. The reflex agent sees only its current square: it cleans if dirty and otherwise moves randomly. The model-based agent remembers which squares it has visited and the shape of the corridor, so it sweeps systematically and stops when its model says everything is clean.

import numpy as np
# Reflex versus model-based agents cleaning a 1 x 10 corridor
rng = np.random.default_rng(1)
def run(agent, steps=60, trials=500):
    cleaned, moves = [], []
    for _ in range(trials):
        dirt = rng.random(10) < 0.5
        pos, visited, mv = 0, np.zeros(10, bool), 0
        direction = 1
        for t in range(steps):
            visited[pos] = True
            if dirt[pos]:
                dirt[pos] = False
                continue
            if agent == "reflex":
                step = rng.choice([-1, 1])                      # no memory: wander randomly
            else:
                if visited.all():
                    break                                       # model says everything is clean
                if pos + direction < 0 or pos + direction > 9:
                    direction = -direction                      # remembers the corridor layout
                step = direction
            pos = int(np.clip(pos + step, 0, 9)); mv += 1
        cleaned.append(1 - dirt.mean()); moves.append(mv)
    return np.mean(cleaned), np.mean(moves)
print("agent         share_clean  moves_used")
for a in ("reflex", "model-based"):
    c, m = run(a)
    print(f"{a:12s}  {c:11.2f}  {m:10.1f}")

agent         share_clean  moves_used
reflex               0.90        56.0
model-based          1.00         9.0

The reflex agent cleans about 90 % of the dirt but uses all 56 of its available moves wandering back and forth, and still misses some squares. The model-based agent cleans everything in 9 moves and then stops, because it knows the task is complete. Memory and a model of the environment turn an aimless policy into an efficient one. The same principle applies to software agents: an agent that keeps track of which files it has inspected, which tests have run and what remains to do wastes far fewer steps than one that reacts only to its latest observation.

Autonomy and the spectrum of agency

Agency is a matter of degree. A chatbot that answers one question at a time has little autonomy; a workflow that calls a language model at fixed points in a predefined sequence has more structure than agency; a system that decides for itself which tools to call, in what order, and when it is done, over dozens or hundreds of steps, is strongly agentic. Anthropic's guidance on building agents distinguishes workflows, where code orchestrates model calls along predefined paths, from agents, where the model dynamically directs its own process and tool use. Workflows are more predictable and easier to test; agents are more flexible for open-ended tasks. Good practice is to use the simplest design that solves the problem and add autonomy only where it clearly helps, because each additional degree of freedom adds cost, latency and new ways to fail.

Why agents now?

Language models made general-purpose agents practical because they can interpret instructions in natural language, reason about many kinds of tasks, use tools described in text, read the results and adapt. Benchmarks such as SWE-bench for software engineering, WebArena and OSWorld for web and computer use, and GAIA for general assistant tasks track rapid progress. Yet agents remain unreliable over long horizons, can take harmful or costly actions, and are hard to evaluate. This course aims to give you both the classical tools to reason about agent behaviour and the practical techniques to build, evaluate and control modern agents responsibly.

Agents and their principals

An agent usually acts for someone: a user, a team or an organization, called its principal. The principal's goals, preferences and constraints define what good behaviour means, but they are never fully specified. A user who asks an agent to "clean up the test failures" probably wants the underlying bugs fixed, not the tests deleted; a manager who asks for "the cheapest flight" probably does not want a 30-hour itinerary with three overnight layovers. Good agents infer reasonable intent, respect implicit constraints, and ask when the intent is genuinely unclear and the stakes are significant. This gap between what is said and what is meant, sometimes called the principal–agent problem in economics, runs through the whole course, from reward design in reinforcement learning to approval rules for consequential actions.

A brief history

The agent perspective has deep roots. Early AI systems such as Shakey the robot (1966 to 1972) combined perception, planning and action in one loop. The 1980s and 1990s brought expert systems, distributed AI and multi-agent systems research, including the belief–desire–intention (BDI) architectures that model agents as having beliefs about the world, desires they want to achieve and intentions they commit to. Russell and Norvig's textbook made the rational agent the unifying idea of AI in 1995. Reinforcement learning, game-playing systems and robotics developed agent methods through the 2000s and 2010s, and since 2022 language model agents have brought general-purpose agency to software tasks. Each wave added tools that remain useful: the vocabulary of agents and environments, explicit planning, learning from reward, and coordination among many agents.

Throughout the course, each lesson pairs a concept with a short program you can run and modify. Changing the parameters, such as the number of steps, the probabilities or the costs, and predicting the result before running it is one of the best ways to build intuition for how agents behave and why they fail.

In the lessons that follow, keep the agent–environment loop in mind. Every technique, from search and planning to reinforcement learning, tool use, memory and multi-agent coordination, is a way of improving one part of that loop: what the agent perceives, how it represents the world, how it decides, or how its actions are controlled. Asking which part a technique improves is a quick way to understand what problem it solves and where its limits lie.