Quality management

Thomas Eberle, Moss's partner and the students' instructor in Professional Practice, takes the seventh week. He begins with a story. Twenty years ago his firm completed a school addition whose drawings showed the new gymnasium floor at a different level from the corridor it joined. The error was found on site after the slab was poured. Fixing it cost the school a ramp, two weeks and about forty thousand dollars, which the firm's insurer paid after a long negotiation. No one had been careless, Eberle says. The structural drawings had been updated after a meeting and the architectural drawings had not. What was missing was a system that would have caught the difference. That system is what this week is about.

Quality assurance and quality control

Eberle distinguishes two ideas that are often confused. Quality assurance is the system a firm uses to make good work likely: its standards, procedures, templates, checklists, training and the way it plans and staffs projects. Quality control is the checking that confirms a particular set of drawings and specifications meets those standards: reviews, coordination checks and back-checks. A firm needs both. Quality control without quality assurance catches errors one at a time but never prevents them. Quality assurance without quality control assumes that good systems produce good work automatically, which they do not.

He describes quality as a loop rather than a final inspection. At the start of each project, the project manager writes a short quality plan that names the reviewers, the review dates and the checklists that apply. The team designs and documents to the firm's standards. At set points, usually about half way through construction documents and again just before they are issued, someone who has not worked on the project reviews them. The team responds to every comment and the reviewer checks the responses. After construction, the firm records what went wrong, from contractor questions to change orders, and updates its standards and checklists so that the same errors are less likely next time.

The firm's standards

Quality begins with standards that everyone follows. Eberle Moss has an office manual that covers drawing conventions, sheet organization, layer and model standards, title blocks, standard details, specification templates, file naming and storage, and the checklists for each phase. It is not long, Eberle says, about forty pages, and it is revised every year from the lessons of completed projects. Standard details are especially valuable: a firm that has a tested detail for a window head in a brick veneer wall does not need to reinvent it, and its contractors learn to trust it.

Moss adds that a new firm can start small. Wren's firm will begin with a one-page drawing standard, a set of templates for its building information model, a dozen standard details for the housing it designs, specification sections adapted from a master specification, and checklists for each phase. The point is not the volume of the manual but the habit of using and improving it.

Reviews that work

Eberle describes what makes a review effective. The reviewer must be independent: someone senior who has not worked on the project and will read the documents as a contractor would. The review must happen early enough for changes to be made without disrupting the schedule; a review the day before issue finds problems no one has time to fix. The reviewer should focus on the issues that cause the most costly errors: coordination between disciplines, dimensions and levels, the building envelope, fire separations and exits, accessibility, and anything unusual about this project. And every comment must be answered and back-checked: a comment that is acknowledged but not corrected is worse than none, because it creates a record that the firm knew.

Building information models have changed the work of coordination. Clash detection finds pipes running through beams, ducts through walls and stairs without headroom, problems that once were found only on site. Eberle cautions that the software finds geometric clashes but not errors of intent: a wall can be modeled perfectly in the wrong place. Human review remains essential.

Measuring quality

Eberle says a firm cannot improve what it does not measure, and that the best measures of quality come from construction. Two are especially useful. The first is the number of requests for information, the formal questions contractors ask when the documents are unclear or incomplete, often expressed per million dollars of construction so that projects of different sizes can be compared. The second is the cost of change orders caused by errors and omissions in the documents, expressed as a share of construction cost. Owners commonly carry a contingency of a few points of construction cost for such changes, and an error rate above about 2% is a sign that something in the firm's process needs attention.

He shows the students the record of five finished projects, including two where the firm had been too busy to hold an independent technical review.

# Cost of quality: errors and omissions on five finished projects (US dollars).
projects = {  # name: construction cost, change orders caused by design errors or omissions, RFIs, QC review hours
    "community hub":    (11_900_000, 119_000, 212, 40),
    "clinic fit-up":    (1_950_000, 49_000, 64, 0),
    "infill housing":   (4_900_000, 64_000, 140, 24),
    "school addition":  (7_500_000, 165_000, 175, 0),
    "seniors co-op":    (8_400_000, 75_000, 160, 60),
}
RATE = 225                     # billing rate of a senior reviewer
print(f"{'project':18}{'E&O changes':>13}{'of cost':>9}{'RFIs per million':>18}{'review hours':>14}")
for name, (cost, eo, rfi, qc) in projects.items():
    print(f"{name:18}{eo:>13,.0f}{eo / cost:>9.1%}{rfi / (cost / 1e6):>18.1f}{qc:>14}")
with_qc = [(c, e) for c, e, r, q in projects.values() if q > 0]
without = [(c, e) for c, e, r, q in projects.values() if q == 0]
rate_qc = sum(e for c, e in with_qc) / sum(c for c, e in with_qc)
rate_no = sum(e for c, e in without) / sum(c for c, e in without)
print(f"error rate with independent review {rate_qc:.2%}; without {rate_no:.2%}")
saved = (rate_no - rate_qc) * 8_400_000
print(f"on the co-op that difference is worth {saved:,.0f}; the review cost {60 * RATE:,.0f}")

project             E&O changes  of cost  RFIs per million  review hours
community hub           119,000     1.0%              17.8            40
clinic fit-up            49,000     2.5%              32.8             0
infill housing           64,000     1.3%              28.6            24
school addition         165,000     2.2%              23.3             0
seniors co-op            75,000     0.9%              19.0            60
error rate with independent review 1.02%; without 2.26%
on the co-op that difference is worth 104,222; the review cost 13,500

The two projects without an independent review had error rates of about 2.5 and 2.2% of construction cost. The three with reviews had rates of about 0.9 to 1.3%. Across the five, the projects with reviews averaged about 1% and those without about 2.3%. Applied to a building the size of the co-op, the difference is worth about 104,000 dollars of change orders, against a review that cost about 13,500 dollars of senior time.

Eberle warns the students not to over-read five projects. The two without reviews were also the two with the tightest schedules, and the clinic was a fit-up of an old building full of surprises. The numbers do not prove that reviews alone explain the difference. But they are consistent with what his firm, its insurer and many studies of construction have found: errors caught in the office are far cheaper than errors found on site, and the cost of a review is small compared with what it can save. That is why his firm no longer issues documents without one, however tight the schedule.

The number of requests for information also varies, from about 18 to about 33 per million dollars. Eberle says that some questions are inevitable and some reflect a contractor who asks about everything, but a project with many more than the firm's usual number is worth a closer look in the close-out review.

The cost of quality

Moss frames quality as an investment. The cost of quality has two parts: the cost of preventing and finding errors, such as standards, training, reviews and checklists, and the cost of errors that are not caught, such as redesign, change orders, delays, damaged relationships and claims. The second is usually far larger than the first, and much of it never appears on the firm's books: a client who does not return or a contractor who prices the firm's next project higher because its drawings are known to be poor. A firm that treats reviews as overhead to be cut when it is busy is saving the cheaper cost and risking the more expensive one.

Formal quality management systems exist for firms that want them. ISO 9001 sets requirements for a documented quality management system and is certified by independent auditors; some larger architecture and engineering firms hold it, and some public clients prefer it. Most small firms do not need certification, but the principles behind it, defined processes, records, review and continual improvement, apply to any firm. Eberle says that the real test of a quality system is whether the same mistake is made twice.

Quality in design, not only documents

Moss reminds the class that quality is more than error-free drawings. A building can be perfectly documented and still fail its users: apartments that overheat in summer, a common room nobody uses, entrances that are hard to find. Design quality is reviewed through design critiques within the firm, through the involvement of users during design, and through post-occupancy evaluation after the building is occupied. Wren's firm plans to visit each building a year after it opens, interview residents and staff, measure temperatures and energy use, and feed the findings back into its standards. Moss says that very few firms do this systematically, and that the ones that do build a reputation clients notice.

Quality during construction

Quality control does not end when the documents are issued. During construction, the architect reviews shop drawings and samples, answers the contractor's questions, observes the work on site and inspects it at substantial completion. Each of these is a chance to catch an error before it is built into the building, and each needs the same discipline as the document reviews: a named reviewer, a checklist for the critical items, a written response and a record. Eberle's firm keeps a log of every shop drawing and request for information with the date received, the reviewer, the date returned and any follow-up. The log protects the schedule, because late responses delay the contractor, and it protects the firm, because it shows what was reviewed and when if a question arises years later.

Learning from projects

The last step of the loop is the one most often skipped. At the end of every project, Eberle's firm holds a one-hour review with the team and records three things: what went well and should become standard, what went wrong and why, and what the firm will change. The changes go into the office manual and the checklists, and the principal responsible confirms within a month that they have been made. Eberle tells the students that this habit, more than any software, is what distinguishes firms whose quality improves over time from those that repeat the same errors on every project.

Exercise 7: a quality plan

• Write a quality plan for a project, with reviews, reviewers, dates and checklists.
• Draft a ten-item checklist for the technical review of a housing project.
• Calculate the error rate and requests for information per million dollars for a project you know.
• Describe how your firm would learn from one completed project.

Self-check

• Can I explain the difference between quality assurance and quality control?
• Are my reviews independent, timely and back-checked?
• Do I measure quality through requests for information and change orders?
• Does my firm change its standards after each project?