the_invisible_man1The future of how certification bodies (CBs) will audit the new ISO 9001:2015 “risk based thinking” language is already setting, like a wet clay in the oven. In short, they are going to default to traditional risk management techniques and impose FMEA on their clients. It won’t matter if you are making tanks for the military or muffins in a bakery, you are going to be doing FMEA because they auditor’s don’t know any other way. That contradicts what ISO 9001’s authors intended, but it’s what is going to happen.

But if the CBs are wrong, then what is the right way to audit something like RBT, which doesn’t actually have any requirements? How do you audit “thinking”? Some have argued it’s impossible, but it’s not; it’s just challenging. Let’s take a look.

The Audit Process

First, let’s understand what auditors are supposed to do. They are supposed to gather objective evidence to assess the conformity of a company against ISO 9001 requirements. That means there needs to be two inputs into the audit process: requirements and evidence.

Let’s take that in reverse order and understand what “objective evidence” is. In short, objective evidence is evidence that is gathered independent of the auditor’s opinions and biases, and which can be confirmed at a later date by any third party. For decades, auditors have been trained to believe that the only acceptable forms of evidence are documents and records. This is not true. Other forms of objective evidence include:

  • Direct observation of work by multiple parties
  • Direct observation of work by a single party (when a job is only performed by one person)
  • Gathering of intellectual evidence (i.e., conversations)
  • Sounds, smells, tactile evidence
  • … and so on.

When auditing RBT it will be imperative for CB auditors to avoid the reliance on documents, records and procedures since the standard specifically does NOT require them. Auditors who demand to see such documentation because “I can’t audit otherwise” should be shown the door. And I mean that in the Old Western saloon bar fight way.

So the second input is the requirement itself. For RBT, as I said, there are no firm requirements for documents, records, processes or resources. One need merely “think” about risk when crafting and managing a QMS. The idea behind this approach was to, according to one TC 176 representative, “address risk according to the context of the organization.” 

Some organizations might be required to take a heavy, formal approach in order to provide the necessary level of confidence in their ability to provide consistent conforming product. In the automotive context, design and process FMEA would be expected, and possibly other risk-based things like sampling criteria etc. In the Food context we have HACCP; and so on. Clearly though, it wouldn’t be appropriate for a small mom & pop store selling innocuous hardware products to have to go through a full FMEA. So we came up with the “risk-based thinking” phrase as a way of diluting the push for out-and-out risk management.

OK, that’s simple enough. But what does ISO 9001 actually require? From this standpoint, the 9001:2015 DIS calls out the need to assess risks in the following areas:

  • Determining the context of the organization
  • Determining the processes needed for the QMS
  • Risks associated with assuring product or service conformity
  • Post-delivery risks

There’s also a general theme running that risks should be considered throughout the QMS regardless of whether it’s specifically called out in a clause or not. It appears, then, that we have very loose, vague language defining how RBT is to be implemented, and (again) that’s by TC 176’s design. The intent was for the company to determine the level of rigor to be used.

So that leaves us with the need to find “hard” evidence of a “soft” intangible. Impossible? Only if you are one of those badly trained auditors who probably should have retired 10 years ago.

The Good

Instead, this is a situation that requires gathering of intellectual evidence, typically through conversations. Auditors, we find, actually don’t know how to document a conversation, so here is what an audit report might look like:

Re: risk based thinking, held interviews with Bob J. the VP Engineering and Jim S. the President. Management indicated that during the development of the QMS risks specific to customer product requirements, local labor force availability and previous issues with utilities were taken into account. This, they reported, resulted in the current process set and related objectives. For example, the “Maintenance” process was formerly embedded in production, but now is a standalone process in order to better manage utilities.

In that example, the names of the people interviewed (“Bob J. and Jim S.”) are the objective evidence, since they can be confirmed later by a third party. The rest of the notes are the supporting evidence to show what they said, and to give some idea that it was acted upon. There’s no risk registry, no procedure, no records to prove it. Nothing that approaches any common approach to “risk management.” And none would be required, yet the company complies with risk-based thinking just fine.

The Bad

So what might a nonconformance look like in such a scenario? In real life we can expect to see all sorts of nightmarish NCs written up, as auditors go nuts trying to invent risks from thin air and then play “gotcha!” with the client by writing them up for not thinking of them; or worse, for not completing an FMEA on each risk. None of that is required, but some evidence or risk-based thinking must be present, or a nonconformity can be issued. Here’s an example:

Re: risk based thinking, held interviews with John D. the CEO and Fester B. the President. There was little understanding of risk and management admitted it did not consider risk when developing the QMS. The management could not name any risks it might face, nor any actions it took to address those risks.

What we see is that you have to be pretty ignorant of risk to get such a finding. Which is also by TC 176’s design; they didn’t intend for companies to be written up for having a different view of risk than their auditors, but they did intend a complete absence of any risk-based thinking to be noncompliant. That’s good news for users, since you don’t have to do too much to comply with RBT, especially since TC 176 says it’s been “implicit in ISO 9001” all along. But for companies that ignore it entirely, blow it off, or fail to execute something, it will be a nonconformity. And it should be.

The Ugly

So, to recap, here’s how it should work:

  1. Determine how the company has interpreted the requirements for risk-based thinking.
  2. Determine how the company has implemented risk-based thinking
  3. Conduct interviews with key management to confirm; capture names and discussions as evidence.
  4. If available, capture documents and records to support. If not, stop at # 3.

This idea of auditing intangibles may be frustrating, and yes, ISO 9001’s risk-based thinking is a mess. But it’s not un-auditable, and auditing it doesn’t require imposing specific solutions on clients simply because an auditor lacks the imagination to audit something other than a document or record.

And the thing is, auditors have already been doing this, without a peep. In the 2000/2008 versions of 9001, the standard required the management to show “commitment” and “a customer focus,” neither of which are tangible things. Auditors typically relied on documentation to check these off, and if the words were on paper, that satisfied them. They were idiots, of course, and should instead have done the same thing I am suggesting here: have conversations with management and key staff, and document that as the evidence.

So we find that auditing RBT isn’t the great mystery that many are claiming. If auditors can stop trying to be consultants, stick to the rules, and understand that they should stop their fixation on auditing documents, we might see some benefit from this after all.

At least when RBT hits the fan, we know who to blame.



About Christopher Paris

Christopher Paris is the founder and VP Operations of Oxebridge. He has over 30 years' experience implementing ISO 9001 and AS9100 systems, and is a vocal advocate for the development and use of standards from the point of view of actual users. He is the author of Surviving ISO 9001 and Surviving AS9100. He reviews wines for the irreverent wine blog, Winepisser.


Aerospace Exports Inc