The accreditation body ANAB is reported to be beating up its certification bodies (CBs) for failing to flow down “proper root cause analysis” to ISO 9001 certified companies. It sounds good on paper, but it’s based on a fallacious stance that reveals even ANAB doesn’t really know how to conduct objective audits.

Here’s the scenario: during a typical ISO 9001 certification audit, a CB auditor writes a nonconformity against the client for some failure or other. The client submits a corresponding corrective action plan, with a root cause analysis. The CB signs off on the fix, the nonconformity is closed, and the certification is maintained. Later, ANAB conducts its regular “office audit” of the CB, and at random selects the records of that nonconformity, and declares — on the spot, without any background, context or in-person witnessed evidence — that the original client’s root cause was inadequate. ANAB thus cites the CB with a nonconformity for failing to ensure its clients are performing proper root cause analysis, and then the CB has to go through some symbolic hoops to placate ANAB.

The reality, of course, is that ANAB has no intention of yanking the CB’s accreditation over this minor matter; it’s kabuki. But this obsession with root cause leads CB auditors to become paranoid on the subject; they start to reject clients’ responses to their findings on the basis of “poor root cause analysis,” and worse, they start to over-analyze the root causes in clients’ internal corrective action systems, and write up nonconformities during routine surveillance audits, just to prove to ANAB they are “tough” on the issue.

Here’s the thing: the entire “poor root cause” complaint is bogus, because it’s based on subjective auditor opinion, whether that of the CB or ANAB. Here’s why.

Gotta Have Evidence, Mate

Recall that ISO 17021 and ISO 17011 — the two standards governing the practices of CBs and Accreditation Bodies — both require that findings be supported by objective evidence. Objective evidence must be supported by data, direct observation or measurement, and must have a component to it that can be independently verified by a third party later.

Blurting out “poor root cause analysis!” does not meet the criteria for “objective evidence.” In fact, it’s nearly always subjective, and you can tell because the term “poor root cause analysis” starts with an adjective (“poor”). If a finding has an adjective in it, it’s likely subjective, and thus garbage.

There is a way for CBs and ABs to write up findings against root cause, however; they just never do this because it requires additional work. Let’s say a CB auditor finds an internal corrective action that they think is poorly executed based on what they think is a poor root cause analysis. Let’s even go to “the well” and say that the CAR indicated “operator error,” the root cause that sends all auditors into an apoplectic fit automatically. (Real world hint: yes, operator error can exist. See? I said it, and the world did not collapse into itself.) So the auditor writes up a nonconformity indicating “CAR # 243 showed poor root cause analysis.”

That appears to include objective evidence; it lists the actual CAR number for example. But in fact, the nonconformity is impossible because it includes only one data point. It’s like a one-sided coin, and can’t exist in the known universe.

To properly write a nonconformity against root cause, the auditor needs at least two data points, if not three or more. To prove that the original root cause was “poor,” the auditor would have to find additional examples of the same exact problem recurring and recurring only after the root cause in question was assessed. If the problem recurred, only then could the auditor write a nonconformity on the original root cause. Therefore, the proper nonconformity might look like this:

CAR # 243 was filed dealing with an issue related to maintenance on the lathes; the same issue recurred as indicated by CAR # 255 and CAR # 265, indicating the original root cause was inadequate at preventing recurrence.

Now I have a legitimate nonconformity, based on solid objective evidence, and which has removed all auditor opinion. It can be verified easily by anyone who was not involved in the original conversation. This is easy for a CB to do, because they are on-site, and have access to the CAR records.

In the case of ANAB, to do this requires a lot more work. ANAB is typically assessing how the CB accepts client corrective actions during an audit at the office of the CB, not on-site with the client; and even if it was done during an on-site witness audit, rules prohibit ANAB from rifling through the client’s CAR files; it can only silently witness the CB doing it. As a result, ANAB cannot obtain additional evidence; they are auditing only the evidence they have on file from the CB.  To actually write a nonconformity against a CB for not enforcing proper root cause analysis, ANAB would have to have at least two data points showing the issue had recurred after an initial root cause failed. It’s doable, for sure — ANAB need only check previous CB audits of that same client to see if the same nonconformity was reported earlier — but it’s not easy, and often not likely to result in any meaningful answer.

So keep this in mind: if an auditor — whether of a registrar or accreditation body — writes a nonconformity against root cause without at least two data points showing recurrence of a problem, it is subjective and can be rejected wholesale. And frankly, having three data points would be better, since sometimes a problem can recur before the action has taken hold, and it’s not an indicator of a poor root cause analysis, but just latency.

Auditors’ statements such as “operator error is never a proper root cause” or “that doesn’t seem to resolve the problem to me” or “you didn’t go deep enough” are all opinions, and not admissible in an official nonconformity. In fact, ISO 17011 and ISO 17021 specifically prohibit these types of findings. Likewise, demanding a client or CB engage in “Five Whys” — or as I call it, “Why Five?” — is a subjective opinion and the provision of a “specific solution” prohibited by the accreditation rules, too.

Certification bodies need to start pushing back on ANAB auditors who, themselves, don’t understand the concept of truly objective, evidence-based auditing. They also need to train their CB audit teams on writing proper evidence-based findings, and their internal review committees on how to spot opinion-based findings and toss them out. Failing to do so creates a never-ending cycle of circulated opinions, which often contradict each other, creating confusion among ISO 9001 standards users, and thus increasing ISO 9001’s already damaged reputation.



    About Christopher Paris

    Christopher Paris is the founder and VP Operations of Oxebridge. He has over 25 years' experience implementing ISO 9001 and AS9100 systems, and is a vocal advocate for the development and use of standards from the point of view of actual users. He is the author of Surviving ISO 9001:2015. He reviews wines for the irreverent wine blog, Winepisser.