Internal control over financial reporting: certification, testing and deficiencies

ACCT-331 explained how auditors understand internal control and decide whether to rely on controls when testing the financial statements. This lesson goes further, to engagements and regimes in which internal control over financial reporting (ICFR) is itself the subject of reporting: management's certification or assessment of its controls, the auditor's tests of controls at the depth needed to support an opinion on them, and the evaluation of control deficiencies. The regimes differ sharply between Canada and the United States, but the auditor's techniques for testing controls and evaluating deficiencies are largely shared.

The reporting regimes

In Canada, National Instrument 52-109, Certification of Disclosure in Issuers' Annual and Interim Filings, requires the chief executive officer and chief financial officer of a reporting issuer to certify that they have designed disclosure controls and procedures and ICFR, and, for non-venture issuers such as Tamarack, that they have evaluated the effectiveness of ICFR at year end and disclosed their conclusion and any material weakness in the management's discussion and analysis. NI 52-109 does not require an auditor's report on ICFR. The external auditor of Tamarack therefore audits the financial statements under CAS, tests controls where it chooses to rely on them, and communicates significant deficiencies to those charged with governance under CAS 265; it does not express an opinion on ICFR. An auditor can be engaged to report on ICFR under CSAE 3000 or a related assurance standard if a company or its lenders want one, but this is rare. Because Canadian practice draws heavily on US methods for evaluating ICFR, and many Canadian companies with US listings file under SEC rules, Canadian auditors need to understand the US regime as well.

Both regimes rely on a suitable framework, and in practice nearly all issuers in both countries use the COSO 2013 Internal Control-Integrated Framework, with its five components (control environment, risk assessment, control activities, information and communication, and monitoring activities) and 17 principles. Management's evaluation, and the auditor's, consider whether each principle is present and functioning.

A top-down, risk-based approach

An integrated audit or a thorough management evaluation does not test every control. It uses a top-down approach: start with the financial statements and the risks of material misstatement; identify entity-level controls, such as the control environment, the CFO's monthly results review and the period-end financial reporting process; then identify significant accounts and disclosures and their relevant assertions; understand the likely sources of misstatement through walkthroughs; and select the controls that address the risks for testing. The emphasis is on controls that matter, which often means fewer controls tested more thoroughly.

Walkthroughs follow a transaction from origination through the company's processes and systems until it is reflected in the financial records, using inquiry, observation, inspection and reperformance. They confirm the understanding of the flow, identify points where a misstatement could occur, and evaluate the design of controls at those points. For Tamarack, walkthroughs of revenue, trade promotions, purchasing, payroll, inventory and the financial close each identify key controls, which are then tested for operating effectiveness.

Testing operating effectiveness

Tests of operating effectiveness evaluate whether a control operated as designed and whether the person performing it had the necessary authority and competence. The amount of evidence needed depends on the risk associated with the control: the nature and materiality of the misstatements it prevents or detects, whether it is manual or automated, its complexity, the judgment involved, and the results of prior tests. A manual control performed many times a day requires a larger sample than a quarterly review; an automated control may need only one test, combined with effective ITGCs.

Firms translate these principles into methodology tables that set minimum sample sizes by control frequency. The program shows a typical table and then evaluates a sample of the three-way match control, performed many times a day, using a binomial upper limit on the deviation rate.

# Tests of controls: sample sizes by control frequency and evaluation of deviations.
# The frequency table is a typical firm methodology (illustrative, not taken from a standard).
import math

frequency_table = {"Annual": 1, "Quarterly": 2, "Monthly": 2, "Weekly": 5, "Daily": 20, "Many times a day": 25}
print("Minimum sample sizes for a control relied on, lower risk of failure:")
for f, n in frequency_table.items():
    print(f"  {f:<18}{n:>3}")

def upper_deviation_rate(n, k, conf=0.90):
    """Smallest p with P(X <= k | n, p) <= 1 - conf (one-sided binomial upper limit)."""
    lo, hi = 0.0, 1.0
    for _ in range(60):
        p = (lo + hi) / 2
        cdf = sum(math.comb(n, i) * p**i * (1 - p)**(n - i) for i in range(k + 1))
        lo, hi = (p, hi) if cdf > 1 - conf else (lo, p)
    return hi

tolerable = 0.10
print("\nThree-way match control (many times a day), tolerable rate 10%, 90% confidence:")
for n, k in [(25, 0), (25, 1), (40, 1), (60, 2)]:
    u = upper_deviation_rate(n, k)
    verdict = "supports reliance" if u <= tolerable else "does not support reliance"
    print(f"  n={n:>2} deviations={k}: upper limit {u:5.1%}  {verdict}")

Minimum sample sizes for a control relied on, lower risk of failure:
  Annual              1
  Quarterly           2
  Monthly             2
  Weekly              5
  Daily              20
  Many times a day   25

Three-way match control (many times a day), tolerable rate 10%, 90% confidence:
  n=25 deviations=0: upper limit  8.8%  supports reliance
  n=25 deviations=1: upper limit 14.7%  does not support reliance
  n=40 deviations=1: upper limit  9.4%  supports reliance
  n=60 deviations=2: upper limit  8.6%  supports reliance

With 25 items and no deviations, the team can say with 90 per cent confidence that the deviation rate does not exceed about 8.8 per cent, within a tolerable rate of 10 per cent, so the sample supports reliance. One deviation in 25 pushes the upper limit to 14.7 per cent and does not; the team could extend the sample to 40, and if no more deviations are found, the upper limit of 9.4 per cent again supports reliance. Many firms' methodologies do not allow sample extension for manual controls once a deviation is found unless the deviation is understood, isolated and explained; a deviation caused by a systematic flaw, such as a clerk who never performs the match for small invoices, is a control failure regardless of sample size. The confidence levels and tolerable rates here are illustrative; each firm sets its own.

Timing matters too. Tests performed at an interim date need roll-forward evidence for the remaining period, by inquiry, observation and further testing. Controls that operate only at year end, such as the period-end close, can only be tested then. And for an opinion on ICFR as of the year end, controls must be effective at that date; a control remediated in November can support a year-end conclusion if it operated for long enough to test.

Evaluating deficiencies

A control deficiency exists when the design or operation of a control does not allow management or employees, in the normal course of performing their functions, to prevent, or detect and correct, misstatements on a timely basis. Deficiencies are evaluated for severity.

Severity depends on two factors: whether there is a reasonable possibility that the control will fail to prevent or detect a misstatement, and the magnitude of the potential misstatement, not the misstatement actually found. A deficiency can be a material weakness even if no misstatement occurred. Compensating controls that operate at a level of precision sufficient to prevent or detect a material misstatement can reduce severity. Indicators of a material weakness include fraud by senior management, a restatement to correct a material misstatement, a material misstatement identified by the auditor that the controls did not catch, and ineffective oversight by the audit committee.

The program applies a simplified version of this reasoning to the deficiencies identified so far in the Tamarack audit, using group materiality of 1.4 million. The thresholds in the code are a teaching device; in practice the evaluation involves judgment about likelihood, magnitude, compensating controls and qualitative factors, and is documented deficiency by deficiency.

# Evaluating control deficiencies at the year end (illustrative logic, thousands of dollars).
materiality = 1_400
deficiencies = [
    # name, potential misstatement if the control fails, reasonably possible?, compensating control?, account
    ("Pay rates not compared to master file (2 deviations)", 260, True, True, "Payroll expense"),
    ("Terminated ERP user still active and used access", 900, True, False, "Journal entries"),
    ("No monthly payroll clearing reconciliation", 1_150, True, False, "Accrued payroll"),
    ("Inventory count instructions not signed", 40, False, True, "Inventory"),
    ("Management review of impairment model lacks precision", 3_800, True, False, "Goodwill"),
]
def classify(amount, possible, compensating):
    if possible and amount >= materiality and not compensating:
        return "material weakness"
    if possible and amount >= 0.5 * materiality and not compensating:
        return "significant deficiency"
    return "deficiency"
for name, amt, poss, comp, acct in deficiencies:
    print(f"{classify(amt, poss, comp):<23}{amt:>6,}  {acct:<16}{name}")
print("\nNext: consider whether deficiencies affecting the same accounts combine into a more severe one.")

deficiency                260  Payroll expense Pay rates not compared to master file (2 deviations)
significant deficiency    900  Journal entries Terminated ERP user still active and used access
significant deficiency  1,150  Accrued payroll No monthly payroll clearing reconciliation
deficiency                 40  Inventory       Inventory count instructions not signed
material weakness       3,800  Goodwill        Management review of impairment model lacks precision

Next: consider whether deficiencies affecting the same accounts combine into a more severe one.

The pay rate deviations at the payroll provider, from lesson 5, are a deficiency: the potential misstatement is small and Tamarack's own reviews partly compensate. The terminated user who retained access, from lesson 4, and the missing payroll clearing reconciliation are significant deficiencies under these assumptions. The weakness in management's review of the goodwill impairment model is assessed as a material weakness: the potential misstatement, 3.8 million, exceeds materiality, there is a reasonable possibility that the review would not detect an error in the discount rate or cash flow assumptions, and no other control compensates. The team also considers whether deficiencies in combination are more severe, for example several weaknesses in access and journal entry controls that together could allow an unauthorized entry to go undetected.

For Tamarack in Canada, the material weakness must be disclosed by management in its MD&A under NI 52-109, with a description, its impact and remediation plans; the CEO and CFO certifications are qualified accordingly. Hollis Ward communicates significant deficiencies and the material weakness in writing to the audit committee under CAS 265, considers the effect on the financial statement audit, for example by expanding substantive procedures on goodwill, and reads the MD&A disclosure for consistency under CAS 720.

Remediation and the period-end close

Management usually tries to remediate deficiencies before the year end. For the goodwill review, Tamarack hires a valuation adviser and redesigns the review so that the CFO compares each key assumption with external data and documents the comparison. If the redesigned control operates for a sufficient period before year end and the team tests it, the material weakness may be remediated as of the year end; if it is implemented only in the last weeks, there may not be enough instances to test, and the weakness would still be reported.

The financial close and reporting process deserves particular attention in both regimes. It includes the procedures for entering transaction totals into the general ledger, selecting accounting policies, initiating and recording journal entries, recording recurring and non-recurring adjustments, and preparing the financial statements and disclosures. Many material weaknesses relate to this process, especially in smaller companies with few accounting staff, where segregation of duties is hard to achieve and complex transactions are handled infrequently.