Decision making and judgment

Leaders are paid, above all, to make decisions under uncertainty, and they make a great many of them: whom to hire, which project to fund, when to escalate a problem, whether to restructure a team. Most of these decisions are made quickly, with incomplete information, under social pressure. Leo Brandt's next big one is how to add sixty engineers at Kestrel Analytics over the coming eighteen months. His leadership team has been arguing for weeks, and he suspects that the argument is being driven more by who speaks loudest than by evidence. In this session Dr. Sato examines what research on judgment says about how individuals and groups decide, where they go wrong and which practical techniques reliably improve decisions.

Bounded rationality and two kinds of thinking

Classical economics assumed that decision makers identify all options, estimate the consequences of each and choose the one with the highest expected value. Herbert Simon, who won the Nobel Prize in economics in 1978, argued that real decision makers are boundedly rational: limited in information, time and cognitive capacity, they search until they find an option that is good enough, a process he called satisficing. Satisficing is often sensible. The question for a leader is not how to make every decision perfectly but how to match the effort spent to the importance and reversibility of the decision. Jeff Bezos's distinction between one-way doors, decisions that are hard to reverse, and two-way doors, decisions that can be undone cheaply, captures the same idea: careful analysis for the first kind, speed for the second.

Daniel Kahneman, drawing on decades of work with Amos Tversky, popularized a distinction between two modes of thinking. System 1 is fast, automatic and intuitive; System 2 is slow, effortful and deliberate. Intuition is extremely valuable when it has been trained in an environment with regular patterns and rapid, accurate feedback, which is why experienced firefighters, nurses and chess players can make good decisions in seconds. Kahneman and the naturalistic decision researcher Gary Klein, who had long disagreed, wrote a joint paper in 2009 concluding that intuition can be trusted when the environment is sufficiently regular and the person has had the chance to learn its regularities. In domains with poor feedback, such as long-range strategy, hiring or predicting the success of projects, confident intuition is a much weaker guide.

Biases that matter most for leaders

Research has documented many systematic errors in judgment. Some of the most important in organizational life are these. Anchoring is the tendency for an initial number, even an arbitrary one, to pull estimates toward it; the first figure mentioned in a budget or salary negotiation shapes everything that follows. Availability is the tendency to judge frequency or risk by how easily examples come to mind, so recent or vivid events are overweighted. Confirmation bias is the tendency to seek and favour information that supports what we already believe. Overconfidence shows up in estimates that are too precise: when people give ranges they are 90 per cent sure contain the true value, the true value often falls outside them far more than 10 per cent of the time. Escalation of commitment, sometimes called the sunk cost fallacy, is the tendency to continue investing in a failing course of action because of what has already been spent.

The planning fallacy, a term coined by Kahneman and Tversky, is the tendency to underestimate the time, cost and risk of future actions while overestimating their benefits, even when we know that similar projects have overrun. Bent Flyvbjerg's database of thousands of large projects across many countries shows that cost overruns are the norm, with information technology projects, among others, having long fat tails of very large overruns. His recommended remedy is reference class forecasting: instead of building an estimate from the inside, by adding up the tasks in your plan, start from the outside, with the actual distribution of outcomes for similar projects, and then adjust for genuine differences. The outside view is uncomfortable because it ignores what makes your project feel special, which is precisely why it works.

Noise: the other source of error

Bias is a systematic deviation in one direction. Noise is unwanted variability in judgments that should be identical. Kahneman, Olivier Sibony and Cass Sunstein, in their 2021 book Noise, describe an audit at an insurance company in which underwriters were given the same realistic cases. Executives expected the premiums set by two underwriters to differ by about 10 per cent; the median difference was about 55 per cent. Similar variability has been documented in sentencing, medical diagnosis, performance appraisal and hiring. Noise is often larger than bias and much less visible, because each judgment is made separately and nobody sees the spread.

The remedies for noise differ from the remedies for bias. Kahneman and colleagues call them decision hygiene. They include structuring complex judgments into independent assessments of separate dimensions, delaying holistic intuition until those assessments are complete, using relative rather than absolute scales, aggregating independent judgments from several people, and using simple rules or algorithms where they perform as well as experts. Structured interviews, in which every candidate is asked the same job-related questions and rated on predefined scales, are a good example. Meta-analyses of personnel selection, such as the widely cited work by Frank Schmidt and John Hunter and later revisions by Paul Sackett and colleagues, find that structured interviews predict job performance considerably better than unstructured ones.

How groups decide

Groups can make better decisions than individuals because they pool knowledge and catch each other's errors, but they also have characteristic failures. Irving Janis described groupthink, the tendency of cohesive groups under pressure and with a directive leader to seek agreement prematurely, suppress doubts and ignore warnings, drawing on case studies such as the Bay of Pigs invasion. Later research has not supported the full model as Janis described it, but the specific symptoms, such as self-censorship and an illusion of unanimity, are real and common. A more robust finding concerns information sharing. In hidden profile experiments by Garold Stasser and William Titus, group members each held some unique information and some shared information. Discussions focused mostly on the information everyone already had, and groups frequently failed to discover the best option, which was only apparent when the unique information was combined.

Several practical techniques counter these failures. Collect individual views independently before discussion, for example by having each person write down a recommendation and reasons, so that early speakers do not anchor everyone else. Ask the most senior person to speak last. Assign explicit roles: a devil's advocate to argue against the leading option, or a red team to look for ways it could fail. Gary Klein's premortem asks the group to imagine that it is a year later and the decision has turned out to be a disaster, and then to write down the reasons. Research by Deborah Mitchell and colleagues found that this kind of prospective hindsight increased people's ability to identify reasons for future outcomes by about 30 per cent.

Clarity about decision rights is equally important. Many leadership teams argue in circles because nobody knows who actually decides. Frameworks such as Bain's RAPID, which distinguishes who recommends, who must agree, who performs, who provides input and who decides, or the simpler RACI matrix, which identifies who is responsible, accountable, consulted and informed, make roles explicit. A study of more than 1,000 business decisions by Dan Lovallo and Olivier Sibony, reported in 2010, found that the quality of the decision process, such as whether uncertainties were explicitly discussed and dissenting views encouraged, mattered about six times more for the outcomes than the depth of the analysis.

Structuring a choice

For important decisions with several options and several criteria, a weighted decision matrix forces the group to be explicit about what matters and how much. The steps are to agree on criteria and their weights before scoring options, to score each option on each criterion independently, ideally by several people, and then to compute weighted totals. The matrix does not make the decision; its value lies in exposing disagreements about weights and scores that would otherwise stay hidden. Where outcomes are uncertain, expected value, the probability-weighted average of possible outcomes, adds a second lens, and reference class information can be used to correct optimistic inputs.

# Weighted decision matrix plus an expected value check with bias adjustments
criteria = {"time to productivity": 0.30, "cost": 0.25,
            "quality and retention": 0.30, "management load": 0.15}
options = {  # average of five leaders' independent scores, 1-10
    "Hire locally":      {"time to productivity": 5, "cost": 4, "quality and retention": 9, "management load": 8},
    "Satellite office":  {"time to productivity": 6, "cost": 7, "quality and retention": 7, "management load": 4},
    "Outsourced vendor": {"time to productivity": 8, "cost": 6, "quality and retention": 4, "management load": 6},
}
for name, s in options.items():
    total = sum(w * s[c] for c, w in criteria.items())
    print(f"{name:<18} weighted score = {total:.2f}")

# Expected first-year cost (thousands of dollars) with an outside-view correction
plans = {  # (probability, cost) scenarios from the inside view
    "Hire locally":      [(0.6, 7800), (0.4, 8600)],
    "Satellite office":  [(0.5, 6200), (0.5, 7000)],
    "Outsourced vendor": [(0.7, 6800), (0.3, 7600)],
}
overrun = {"Hire locally": 1.05, "Satellite office": 1.25, "Outsourced vendor": 1.15}
for name, scen in plans.items():
    ev = sum(p * v for p, v in scen)
    print(f"{name:<18} inside-view EV = {ev:,.0f}  adjusted = {ev * overrun[name]:,.0f}")

Hire locally       weighted score = 6.40
Satellite office   weighted score = 6.25
Outsourced vendor  weighted score = 6.00
Hire locally       inside-view EV = 8,120  adjusted = 8,526
Satellite office   inside-view EV = 6,600  adjusted = 8,250
Outsourced vendor  inside-view EV = 7,040  adjusted = 8,096

The matrix ranks hiring locally first because the leaders weighted quality and retention heavily, while the satellite office looks cheapest on the inside view. The overrun factors, which Leo takes from the experience of comparable firms, reflect evidence that new offices in unfamiliar markets overrun more than local hiring does. After adjustment, the satellite office's cost advantage over local hiring shrinks from about 1.5 million dollars to under 300 thousand, and the outsourced vendor becomes the cheapest option, although it scores worst on quality and retention. Neither number settles the question, but the conversation moves from who is most persuasive to whether the weights and overrun assumptions are right.

Speed matters as well as quality. Organizations that require every decision to pass through the same heavy process become slow, and slowness has its own costs: opportunities missed, talented people frustrated and problems allowed to grow. A useful discipline is to classify decisions before deciding how to make them. Reversible, low-stakes decisions should be delegated to the people closest to the information, with a bias toward action. Irreversible, high-stakes decisions deserve independent input, explicit uncertainty and a premortem. Recurring decisions, such as approving discounts or hiring junior staff, are candidates for simple rules or checklists that make them faster and less noisy at the same time. Leo realizes that his leadership team has been treating every decision as if it were a one-way door, including many that could have been made in a day by a single person.