Performance evaluation: monitoring, internal audits and management review

Nine months after the system launched, Elena Novak sat down with a stack of reports. The injury rate had fallen at the main plant and risen slightly at the distribution centre. Hazard reports had tripled. Forty-one corrective actions were open, eleven of them overdue. Two training courses had completion rates below 70 percent. 'Is it working?' Rachel Okoye asked her. Elena realized that the honest answer was that she could not yet tell, because nobody had defined what 'working' meant or checked the system in a structured way. This lesson is about the check stage of plan-do-check-act: how an organization finds out whether its system is working.

Management system standards set out three main tools for performance evaluation: monitoring, measurement, analysis and evaluation; internal audit; and management review. A fourth, evaluation of compliance with legal and other requirements, was covered in lesson 6.

Monitoring and measurement

The organization must decide what needs to be monitored and measured, the methods for doing so, the criteria against which performance will be evaluated, when monitoring will happen, and when results will be analysed and communicated. The standards suggest what to monitor: the extent to which legal requirements are met, activities and operations related to identified hazards and risks, progress toward objectives, and the effectiveness of operational and other controls.

Equipment used for measurement, such as noise meters, gas detectors, air sampling pumps and thermometers, must be calibrated or verified and maintained. At Harrowgate Foods Ltd., the ammonia detectors were tested quarterly and calibrated each year by the manufacturer's technician, and the records were kept with the maintenance system.

Choosing indicators

Lesson 4 introduced leading and lagging indicators. Performance evaluation turns them into a dashboard that tells leaders where to look. Elena rebuilt Harrowgate Foods Ltd.'s monthly report around a small number of indicators, each with a definition, a source, a target and an owner.

The list was deliberately short. Elena had seen dashboards with forty measures that nobody read. Eight indicators, reported the same way every month, showed trends that leaders could act on.

Analysis: looking past the numbers

Numbers alone mislead. The rise in hazard reports looked bad on a simple chart, but it reflected a reporting culture that was finally working. The slight rise in injuries at the distribution centre turned out, on analysis, to come from one task: unloading mixed pallets from a new supplier whose loads arrived poorly secured. The trend pointed to a procurement problem, not a worker problem. Analysis also meant comparing sites, shifts and departments, looking at the types and causes of incidents, and reading the free-text descriptions on near-miss reports for patterns no code would catch.

# Monthly trend check: flag indicators moving the wrong way for three months.
data = {
    "Hazard reports closed in 30 days (%)": [78, 74, 71, 69],
    "Overdue corrective actions":           [14, 12, 11, 9],
    "Critical control verifications (%)":   [88, 91, 94, 96],
}
better_if_higher = {
    "Hazard reports closed in 30 days (%)": True,
    "Overdue corrective actions": False,
    "Critical control verifications (%)": True,
}
for name, vals in data.items():
    steps = [b - a for a, b in zip(vals, vals[1:])]
    worse = all((s < 0) if better_if_higher[name] else (s > 0) for s in steps)
    trend = "WORSENING three months running" if worse else "stable or improving"
    print(f"{name}: {vals[-1]} ({trend})")

Hazard reports closed in 30 days (%): 69 (WORSENING three months running)
Overdue corrective actions: 9 (stable or improving)
Critical control verifications (%): 96 (stable or improving)

The check showed that overdue actions and critical control verifications were improving, but the share of hazard reports closed within 30 days had fallen three months running. Supervisors were receiving more reports than they could deal with. The response was not to tell them to try harder, but to give the joint occupational health and safety committees a role in sorting reports and to give maintenance a weekly slot for small fixes.

Internal audit

Monitoring tells the organization what is happening. An internal audit asks a different question: does the management system conform to the organization's own requirements and to the standard, and is it effectively implemented and maintained? Audits are systematic, independent and documented. They look for evidence, not opinions.

Management system standards require an audit program that sets the frequency, methods, responsibilities, consultation, planning requirements and reporting of audits, taking into account the importance of the processes and the results of previous audits. For each audit the organization defines its criteria and scope, selects auditors who are objective and impartial, makes sure results are reported to the relevant managers, workers and worker representatives, and acts on nonconformities.

Planning the audit program

Harrowgate Foods Ltd. trained twelve internal auditors in the first year, including four workers nominated by the joint occupational health and safety committees. Auditors never audited their own areas. Aisha Rahman, who had run food safety audits for years, helped Elena design a three-year program that covered every clause of the standard and every site at least once a year, with high-risk processes, such as lockout, ammonia, contractor management and forklift traffic, audited more often.

Audits followed the process approach. Rather than auditing clause by clause in an office, auditors picked a process, such as receiving at the distribution centre, and followed it end to end: the risk assessment, the procedures, the training records of the people doing the work, what they actually did on the floor, the inspection records, the incidents and the corrective actions. A single trace often tested half a dozen clauses at once.

Conducting the audit

An internal audit at the main plant's lockout process looked like this. The audit team of two reviewed the lockout procedure, the equipment-specific procedures and the training records the week before. On the day, they held an opening meeting with Marcus Bell and the maintenance supervisor, then spent most of their time on the floor. They watched two lockouts, asked workers to talk them through their steps, checked that the lock and tag records matched the locks on the boards, and compared the equipment list with the procedures on file. They interviewed a new sanitation worker and a twenty-year maintenance mechanic. At the closing meeting they presented their findings.

Findings fall into three kinds. A nonconformity is a failure to meet a requirement, with objective evidence. An observation or opportunity for improvement points to a weakness that is not yet a nonconformity. A positive finding records good practice worth sharing. The lockout audit found one nonconformity, that three machines installed in the past year had no equipment-specific procedure, two observations and one positive finding: the sanitation crew's group lockout box, designed by the workers themselves.

Writing good findings

A good nonconformity statement has three parts: the requirement, the evidence, and the gap. 'Lockout procedures are poor' is not a finding. 'The lockout program requires an equipment-specific procedure for each machine with more than one energy source. The new slicer, the second packaging line and the tray washer, installed between March and September, have no equipment-specific procedures; maintenance staff confirmed they use the generic procedure' is a finding. It can be checked, it names the problem without blaming individuals, and it tells the owner what to fix.

Management review

Management review is where top management steps back and asks whether the system remains suitable, adequate and effective. It is the single most important meeting in the management system, and it is easy to waste. Harrowgate Foods Ltd.'s first management review, a two-hour meeting each year with quarterly shorter reviews, used a standard agenda drawn from the inputs the standards list:

• status of actions from previous reviews;
• changes in external and internal issues, the needs of interested parties, legal requirements and risks and opportunities;
• the extent to which the policy and objectives were met;
• performance: incidents, nonconformities, corrective actions and continual improvement, monitoring results, compliance evaluation, audit results, consultation and participation of workers, and risks and opportunities;
• adequacy of resources;
• relevant communications with interested parties; and
• opportunities for continual improvement.

The outputs of management review are decisions: on the continuing suitability of the system, on improvement opportunities, on changes needed to the system, on resources, on actions, on integration with business processes, and on implications for strategic direction. A review that produces no decisions has not done its job.

A management review in practice

At Harrowgate Foods Ltd.'s first full review, Elena presented the dashboard and the audit results; Tanya Morris and Sam Pellerin, the worker co-chairs, presented the joint occupational health and safety committees' view, including what workers were saying on nights. Grace Whitfield, the distribution centre manager, explained the pallet problem. The leadership team made seven decisions, recorded with owners and dates: tighten load-securing requirements in supplier contracts, fund a second maintenance technician on nights, add the four new machines to the lockout procedure backlog with a deadline, change the hazard report process to involve the committees, set an objective for the next year on psychosocial risk, schedule an external pre-assessment audit, and add worker auditors at the distribution centre.

Worker participation in evaluation

Workers are not only the subjects of performance evaluation; they are a source of evidence and judgement that no dashboard can replace. Management system standards expect workers to be consulted on what needs to be monitored and measured, and to take part in audits and in investigating incidents and nonconformities. At Harrowgate Foods Ltd., the joint occupational health and safety committees reviewed the draft indicator set before it was adopted and added one that management had missed: the number of times the night shift ran below planned staffing, which the committees believed drove both injuries and shortcuts. It turned out to correlate closely with near misses on the packaging lines.

Audit results were shared with the joint occupational health and safety committees in full, not as a summary. Worker auditors brought something the managers could not: workers spoke to them more openly, and they knew where the informal workarounds were. One worker auditor at the main plant found that a guard on the tray washer was routinely removed to clear jams because the clearing tool had been lost months earlier, a fact no record would have shown.

Preparing for external audits

Internal audits also prepare an organization for external ones, covered in lesson 11. Aisha Rahman advised Elena to treat the internal program as a rehearsal: same evidence standards, same finding categories, same follow-up discipline. By the time a certification body or a recognition program auditor arrived, every element of the system should have been audited internally at least once, the nonconformities found should have been corrected and verified, and the most recent management review should have recorded its decisions. An external auditor who finds problems the internal program missed is a sign that the internal program needs to dig deeper.