Measuring committee effectiveness

A year after the company standard came in, Rachel asked Elena a simple question at the leadership meeting: 'Are the committees working?' Elena could have answered with the injury rate, which had fallen at two sites and risen slightly at the third. But she knew injury rates were a poor measure of a committee. They are small numbers that jump around from year to year, they depend on many things besides the committee, and they say nothing about whether the committee is doing the work that prevents injuries. She asked for a month to give a better answer.

This lesson covers how to measure whether a committee is working: what to measure, how to collect it, how to present it and how to use it to improve. The same approach works for a single committee looking at itself and for a company comparing several committees.

Why measure

Measuring a committee is not about grading it. It is about knowing whether its work is having an effect, and where to focus effort. A committee that measures itself can see that its inspections are on schedule but its recommendations are stuck, or that workers raise concerns but rarely hear back. Without measures, those problems stay hidden until something goes wrong.

Measures also help the committee make its case. A committee that can show management it found 150 hazards in a year, sent 41 recommendations and got 34 accepted and verified is in a much stronger position than one that can only say it met every month.

Leading and lagging measures

Measures fall into two groups. Lagging measures count outcomes after they happen: injuries, lost days, workers' compensation claims. They matter, but they are slow, noisy and influenced by much more than the committee. Leading measures count the activities and conditions that prevent harm: inspections done, hazards fixed, recommendations answered, training completed, concerns raised and resolved. They are within the committee's influence and change quickly when the committee's work changes.

Elena built the committee scorecard mainly from leading measures, with lagging measures shown for context. She chose measures that were easy to count from records the committees already kept, so that measuring would not become a burden of its own.

The last measure is the most unusual, and Elena thought it the most revealing. It comes from a short survey, explained below. A committee that workers cannot name is not one they will go to with a concern.

Collecting the numbers

Most measures come from records the committee already has. The inspection schedule shows what was planned and what was done. The tracking log shows response times, acceptance and verification. The concern log shows how quickly concerns were answered. The training record shows completion. The minutes show attendance and quorum. Grace at the distribution centre spent about two hours a quarter pulling the numbers together, most of it checking the logs were up to date.

# Quarterly committee scorecard for the distribution site.
measures = [
    # (measure, actual, target, higher_is_better)
    ("zone visits on time",          26 / 27, 0.95, True),
    ("class A made safe in 24 h",     5 / 5,  1.00, True),
    ("responses within 21 days",     36 / 41, 0.90, True),
    ("actions verified by due date", 27 / 39, 0.80, True),
    ("concerns answered in 7 days", 112 / 131, 0.90, True),
    ("core training complete",        7 / 8,  1.00, True),
    ("meetings with quorum",         12 / 12, 11 / 12, True),
    ("workers naming a member",      0.68,    0.75, True),
]
met = 0
for name, actual, target, _ in measures:
    ok = actual >= target
    met += ok
    gap = "" if ok else f"  gap {target - actual:.0%}"
    print(f"{'met ' if ok else 'MISS'} {name:<30} {actual:>5.0%} vs {target:.0%}{gap}")
print(f"{met} of {len(measures)} targets met")

met  zone visits on time              96% vs 95%
met  class A made safe in 24 h       100% vs 100%
MISS responses within 21 days         88% vs 90%  gap 2%
MISS actions verified by due date     69% vs 80%  gap 11%
MISS concerns answered in 7 days      85% vs 90%  gap 5%
MISS core training complete           88% vs 100%  gap 12%
met  meetings with quorum            100% vs 92%
MISS workers naming a member          68% vs 75%  gap 7%
3 of 8 targets met

The distribution centre committee met three of its eight targets. Inspections, class A findings and meetings were on track. Responses within 21 days, at 88 per cent, were just short. The weakest areas were verification of actions, at 69 per cent, and concerns answered within a week, at 85 per cent. Training was one member short, because Jasmine's refusal module had been postponed. And only 68 per cent of workers surveyed could name a committee member on their shift.

Asking workers

Records show what the committee did. Only workers can say whether it made a difference to them. Harrowgate Foods Ltd. runs a short survey at each site once a year, taking about three minutes, in the main languages spoken at the site. It asks five questions:

• Can you name a committee member who works on your shift?
• If you raised a safety concern in the past year, did you hear back about what happened?
• Do you believe you could refuse or stop unsafe work without getting into trouble?
• Has the committee made a difference to safety where you work?
• What is one thing the committee should work on next?

The first year's survey at the distribution centre had 214 responses. Most workers believed they could stop unsafe work, which was encouraging. But a third could not name a committee member, and night workers were less likely to have heard back about a concern than day workers. The answers to the last question gave the committee its priorities for the next year: congestion in the pick modules, cold in the freezer and the speed of repairs.

Using the results

Measures only matter if they lead to change. The distribution centre committee spent most of one meeting on its scorecard and survey, and agreed three actions:

• Verification: at each meeting, assign every action due that month to a named member to verify, and add a 'verified' column to the safety board.
• Concerns: make the night co-chair, Owen, responsible for answering night concerns within a week, with a weekly check of the concern log.
• Visibility: put photos of the members on each shift at every time clock, introduce members at every shift start-up meeting once a quarter, and add committee members to the agency orientation slide that Jasmine had rewritten.

A year later, verification was at 84 per cent and 81 per cent of workers could name a member. Concerns answered within a week reached 93 per cent.

Comparing committees

With the same scorecard at every site, the company safety council could compare committees. Elena was careful about how. Sites differ in size, hazards and history, and a league table of committees would encourage gaming the numbers rather than improving. Instead, the council used the comparisons to share what worked. The sauces plant had the best concern response time, and its co-chairs explained their method: a short daily check of the concern log by whichever co-chair was on shift. Two other sites adopted it.

Avoiding bad measures

Some measures do more harm than good. Elena avoided a few on purpose:

• Number of recommendations made. Rewarding volume encourages trivial recommendations.
• Injury rate as a committee target. It is noisy, it can discourage reporting, and committees cannot control it alone.
• Hours spent in meetings. Time in meetings says nothing about what was achieved.
• Zero findings on inspections. An inspection that finds nothing usually was not looking hard enough.

The test she used for every measure was simple: if a committee tried hard to improve this number, would the workplace become safer? If the answer was not clearly yes, the measure did not go on the scorecard.

Looking at quality, not just quantity

Numbers can show that work was done. They cannot easily show whether it was done well. Once a year, Elena and a co-chair from another site review a sample of each committee's work in depth: three inspection reports, two investigations, five recommendations and two sets of minutes. They ask simple questions. Did the inspection find the hazards a careful person would expect in that zone? Did the investigation go beyond the first explanation? Was the recommendation specific and supported by evidence? Did the minutes record clear actions?

The review at the main plant found that its inspection reports had improved greatly since the zone checklists were introduced, but that its investigations still often ended at 'worker did not follow procedure'. That finding would never have shown up on the scorecard, because the investigations were all completed on time. The committee asked for refresher training on investigation methods and paired its members with the distribution centre's more experienced investigators for the next two serious events.

The annual self-assessment

Lesson 1 introduced a self-assessment that each committee completed when the program began. Harrowgate Foods Ltd. repeats it every year, with members rating their committee on the same areas: purpose and support, membership and training, inspections, investigations, concerns and refusals, recommendations, meetings and visibility. Comparing one year with the next shows members where they have improved and where they have stalled.

The self-assessment and the scorecard sometimes disagree, and that is useful. The distribution centre committee rated its handling of concerns as strong, but the scorecard showed concerns answered within a week at only 85 per cent, and the survey showed night workers often did not hear back. Members had been judging by the concerns they handled personally, which were mostly on days. The gap prompted the change that made Owen responsible for night concerns.

Reporting to senior management

Elena's answer to Rachel's question came as a two-page report to the leadership team. The first page showed each site's scorecard side by side, with a short note on what each committee was doing about its gaps. The second page showed what the committees had achieved in the year: hazards found and fixed, recommendations accepted, refusals resolved and the main changes made, with photos. It ended with three requests from the committees to the leadership team: faster capital approval for class A fixes, protected time for night members, and a visit from a senior leader to each committee meeting once a year.

All three were agreed. Rachel attended the distribution centre committee's next meeting herself, and asked members what the company was getting wrong. Jasmine, by then in her second year, told her.

Keeping measurement light

There is a risk that measuring a committee becomes a job in itself. Elena set a rule that the scorecard must take no more than two hours a quarter to prepare at each site, and that any measure needing more effort than that would be dropped or simplified. Two measures she had first proposed, the average age of open recommendations and the share of inspections with a worker from the zone present, were dropped after the first quarter for that reason. The committees could still track them if they found them useful, and one did.