Organizational design: structure, spans and layers

When Kestrel Analytics had forty employees, Leo Brandt knew everyone, decisions happened in the kitchen and the structure was a founder surrounded by talented people. At 120 employees the same structure is failing. Leo has eleven people reporting directly to him, product and engineering argue about priorities, two client teams have built separate versions of the same data pipeline, and new hires complain that nobody can tell them who decides what. With sixty more engineers coming, as discussed in the session on decisions, Leo has to redesign the organization. This session is about how to think about that problem: the trade-offs every structure makes, how to size spans and layers, and why there is no perfect design, only designs that fit a strategy better or worse.

Differentiation, integration and fit

Organizational design starts with two opposing forces. As organizations grow they differentiate: they divide work into specialized units, each developing its own goals, time horizons and ways of working. Differentiation brings expertise and efficiency but creates a need for integration: mechanisms that coordinate the specialized units so they work toward common goals. Paul Lawrence and Jay Lorsch's 1967 study of firms in industries with different levels of uncertainty found that the most successful firms in uncertain environments were both highly differentiated and highly integrated, using rich integrating mechanisms such as cross-functional teams and dedicated integrating roles. Successful firms in stable industries needed less of both.

An earlier British study by Tom Burns and G. M. Stalker contrasted mechanistic organizations, with clear hierarchy, specialized roles and rules, which suited stable conditions, with organic organizations, with fluid roles, lateral communication and shared problem-solving, which suited rapid change. Both studies gave rise to contingency theory: the idea that there is no single best structure, and that the right design depends on strategy, environment, technology and size. Jay Galbraith's star model extends this by arguing that structure is only one of five design elements that must fit together: strategy, structure, processes and lateral connections, rewards, and people practices. Changing the boxes on the chart without changing how decisions are made, how people are rewarded and what skills they have usually achieves little.

Basic structural forms

Most organizations group work in one of a few basic ways. A functional structure groups people by expertise, such as engineering, sales and finance; it builds deep skills and economies of scale but makes cross-functional coordination slow, with problems escalating to the top. A divisional structure groups people by product, customer or geography, each with its own functions; it creates clear accountability for results and responsiveness to markets but duplicates resources and can weaken specialist depth. A matrix structure combines two groupings, so that people report both to a functional manager and to a product or project manager; it can balance expertise and responsiveness but creates dual reporting and conflict that must be managed. Network or modular forms rely heavily on partners and contractors for parts of the value chain.

Some organizations have experimented with radically flat structures. The online retailer Zappos adopted holacracy, a system of self-organizing circles with distributed authority, in 2013 and 2014, and in 2015 offered a buyout to employees who did not want to work that way; a significant minority, reported at about 14 percent, accepted at first. The company later moved away from strict adherence. The Chinese appliance maker Haier reorganized into thousands of small microenterprises that contract with each other and with customers. The evidence on such experiments is mostly from case studies, and a common finding is that removing formal hierarchy does not remove the need for coordination and decision rights; it shifts them into rules, processes and informal status, which can be harder to see. Similarly, the widely copied squads, tribes, chapters and guilds model popularized by Spotify was described by people involved as a snapshot of an evolving approach rather than a blueprint, and it was never implemented exactly as described.

Spans of control and layers

A manager's span of control is the number of people who report directly to them. Wider spans mean fewer managers, fewer layers between the front line and the top, lower overhead and faster communication, but less time per direct report for coaching, review and support. Narrower spans allow closer supervision and development but create more layers, more management cost and slower decisions. There is no universally correct span. Wider spans work when work is standardized, employees are experienced and self-directed, the manager has few other duties and tools such as dashboards make performance visible. Narrower spans suit complex, novel or highly interdependent work, inexperienced teams and managers who also carry substantial individual work, often called player-coaches.

For an organization of a given size, the span determines the number of layers. If every manager has s direct reports and there are N front-line employees, the number of management layers is roughly the logarithm of N to base s, rounded up. Doubling the span from 4 to 8 roughly halves the number of layers. The number of managers is the sum of the managers needed at each level. These calculations are simplified, because real organizations have uneven spans and specialist roles, but they show why small changes in average span have large effects on management cost.

# Spans, layers and management headcount for a target organization
import math

def design(front_line, span):
    """Build levels upward from the front line until one leader remains."""
    levels, count = [], front_line
    while count > 1:
        count = math.ceil(count / span)
        levels.append(count)
    return levels  # managers at each level, bottom first

N = 160  # front-line employees Kestrel expects after hiring
print(f"{'Span':>4} {'Layers':>7} {'Managers':>9} {'Mgr share':>10}")
for span in (4, 6, 8, 10, 12):
    lv = design(N, span)
    managers = sum(lv)
    print(f"{span:>4} {len(lv):>7} {managers:>9} {managers / (N + managers):>10.1%}")

# Leo's current direct reports: 11, against a target of about 6 to 8
print("Levels at span 7:", design(N, 7))

Span  Layers  Managers  Mgr share
   4       4        54      25.2%
   6       3        33      17.1%
   8       3        24      13.0%
  10       3        19      10.6%
  12       3        17       9.6%
Levels at span 7: [23, 4, 1]

With 160 front-line employees, a span of 4 needs four layers of management and 54 managers, about one employee in four. A span of 8 needs three layers and 24 managers. A span of 12 needs only 17 managers, but each would have twelve direct reports, which leaves little time for the coaching and development that the engagement data in the fifth session showed Kestrel badly needs. Leo settles on a target span of about 6 to 8 for engineering managers, who are also expected to do some technical work, and 8 to 10 for client delivery managers, whose work is more standardized.

Two further principles matter. First, Conway's law, an observation by the computer scientist Melvin Conway in 1968, holds that organizations design systems that mirror their own communication structure. Kestrel's two separate data pipelines were a direct consequence of two client teams that never talked. Some technology firms deliberately apply what is called the inverse Conway maneuver, arranging teams to match the system architecture they want. Second, cognitive limits constrain informal coordination. The anthropologist Robin Dunbar proposed that humans can maintain stable social relationships with about 150 people, and some firms, such as W. L. Gore and Associates, have kept facilities to around that size. Later statistical reanalyses have questioned whether the figure is as precise or universal as often claimed, so it is best treated as a reminder that informal coordination becomes harder as units grow rather than as a hard limit.

Designing for decisions

A structure is ultimately a way of allocating decision rights and information. When Leo diagnoses the current problems, most turn out to be failures of decision rights rather than of reporting lines. Nobody decides between product and engineering priorities, so both escalate to Leo. Nobody owns shared data infrastructure, so client teams build their own. Designers often begin with the organization chart; Dr. Sato recommends beginning with the five to ten most important recurring decisions and asking who should make each one, with what information and with whom consulted, then designing units and roles around those answers. The RAPID and RACI tools from the session on decisions are useful here.

Integration mechanisms deserve as much attention as units. Options range from light to heavy: shared goals and metrics; regular cross-unit forums; liaison roles; temporary task forces; permanent cross-functional teams; integrating managers with authority over a cross-cutting outcome; and, at the heaviest, a matrix. The rule of thumb is to use the lightest mechanism that will work, because heavier mechanisms add cost and complexity. Leo also needs to plan the transition, because reorganizations are disruptive. Surveys of executives by consultancies have repeatedly found that many reorganizations fail to deliver their intended benefits and that productivity often dips during the transition, so a clear rationale, a short transition and early resolution of who reports to whom are essential.

Centralization and formalization

Two further dimensions describe any structure. Centralization is the degree to which decision authority is concentrated at the top. Formalization is the degree to which work is governed by written rules, procedures and job descriptions. Both have costs and benefits. Centralization gives consistency and coordination but slows decisions and starves front-line managers of authority to respond to local conditions; decentralization gives speed and local responsiveness but risks inconsistency and duplication, which is exactly what happened when Kestrel's client teams each built their own data pipeline. Formalization makes work predictable and scalable and protects against arbitrary treatment, which matters for fairness, but too much of it stifles judgment and initiative.

Paul Adler and Bryan Borys made a helpful distinction between coercive formalization, rules designed to control and replace the judgment of employees, and enabling formalization, rules designed to help employees do their work well, such as checklists written with the people who use them and procedures that can be improved when they do not work. The surgical safety checklist promoted by the World Health Organization, associated in a 2009 study by Alex Haynes and colleagues with substantial reductions in complications and deaths across eight hospitals, is a classic example of enabling formalization. For growing firms such as Kestrel, the aim is to add enabling rules for recurring work, so that people know how things are done without having to ask the founder.