by Christopher Paris

There has been a flurry of controversy over AS9100’s new Process Effectiveness Assessment Report, or PEAR form, being used by accredited AS9100 registrars. In fact, the form has nothing to do with AS9100 Rev C, but instead is a mandatory form to be completed by registrar auditors working under their rules, AS9101 Rev D.

The gripes to date in a nutshell:

  • The PEAR form is hard to understand.
  • There is disagreement on the purpose of the PEAR. Registrars are using them as part-demographic gathering tools, to capture information on processes and company metrics. The AS9101 standard defines the PEAR as a evidence record, to contain evidence… not just demographics. (ANAB is planning a Heads Up ruling on this, but early signs show they are relying on the input of the market and industry, rather than simply adhering to the language in AS9101, which is never a good sign.)
  • Registrar auditors are requiring (or, at least “strongly requesting:) that clients fill them out before an audit actually occurs. Clients are balking because of the extra workload placed on them, and others are claiming this is, in effect, having auditees fill out their own audit evidence before the audit even occurs.
  • Despite the PEAR’s attempt to reduce subjectivity when having third-party auditors “assess effectiveness” of a process, actual on-the-street evidence shows that subjectivity has increased. No sense yet as to whether the registrar’s required Audit Review Committees will bounce these back or not. So far, they haven’t.

Of course, the biggest stumbling block hindering good PEAR work is the fact that since ISO 9001:2000, there has never been a universal consensus on what constitutes a QMS “process” to begin with. ISO 9001 and AS9100 clearly state that the ORGANIZATION defines it’s processes, but registrar auditors routinely break from this rule, and apply their own vision of what constitutes a process, scrubbing the client’s QMS entirely. Since registrar Review Committees only see the auditor’s report, and never compare it against the clients Quality Manual listing of processes, this problem never gets caught.

If we can’t decide what a process IS, how can we judge its effectiveness?

There’s also been some equally noisy support for the PEAR, coming from various sectors. The most obvious source of this is from individuals working with the IAQG, of course; after all, this is their work output we’re talking about. A secondary line of defense comes from the CB’s themselves, who — while they are unaware that they are each disagreeing with each other’s interpretation on what the PEAR is actually supposed to accomplish — nevertheless support their own unique vision of it.

Some end users are embracing the report, seeing its value. And that’s where this column today comes from. We’ve griped about the PEAR… but putting aside the flaws and misunderstanding, what’s the benefit of it?

NOT ALL PEARS ARE ROTTEN

The IAQG’s rationale for rolling out the PEAR was to ensure that third party auditors went beyond just assessing compliance to the “shall clauses” of AS9100, but also verified that the resulting QMS processes were effective. Recognizing that “effective” can mean a lot of things to a lot of people, the IAQG knew it had to put some bookends around this open-ended term, and help clarify it. The PEAR form was thus developed to suit two purposes: to document the evidence gathered by the auditor of process efficacy (or lack thereof), and then to provide a rating system which could take an auditor’s subjectivity out of the picture, and boil down the evidence into one of four possible Process Effectiveness ratings.

Thus, a completed PEAR form would not only document clearly and with evidence the decision of the auditor, but also provide valuable, traceable data back to the client as to what was right — or wrong — with each of the organization’s processes.

Used properly, the PEAR can do this.

A completed PEAR form, provided by your registrar’s auditor, should include evidence that they have gathered during the audit on how you measure each of your processes, and — based on YOUR data — whether it is effective or not.

EFFECTIVE VS. NON-EFFECTIVE

According to the PEAR, a process may be deemed effective if either of the following conditions is met:

  1. The process has been implemented and is meeting its goals. Simple.
  2. The process has been implemented, but some of its goals are not met; however, there are records of actions being taken to bring the process on track and this is being watched closely by management.

A process would be considered NOT effective if:

  1. The process has not been implemented at all, isn’t meeting any goals (or goals aren’t set) and there are no actions in place to fix any of this.
  2. The process is implemented, but goals are not met and there are no plans underway to correct this.

The four ratings above rely on the word “implemented” which is understood to mean a little more than simply being documented in a Quality Manual. The process should be underway, and meeting the requirements (shall clauses) of AS9100 without any major nonconformities.

USING THE INFORMATION

Once you have a PEAR returned from an auditor, assuming it’s completed properly and objectively, you have a valuable tool at your disposal. With it, you can quickly understand which of your company processes are effective, which are only semi-effective, and which ones are off the rails entirely. With this information, Top Management should then analyze the impacts — both going back a few months, and projecting forward into the future — of the ineffective process on products, the QMS, and other processes. Formal, documented corrective actions must be filed to develop plans to bring the offending process back on track.

Cost of Quality guys can use this information as valuable inputs into CoQ studies, comparing the CoQ of “effective” processes against those found to be “not effective”.  Studies should be performed to determine if the low efficacy is due to resources being unfulfilled, procedures and methods being more complex than current training levels allow, problematic relationships with suppliers, subcontractors or partners, equipment issues, etc. A standard FMEA approach would be a great way to fish down to find some root causes for process inefficiency.

One thing is true: a process found to be “not effective” is likely costing your organization a flood of money, leaking unseen through the cracks and perhaps unnoticed even by the most hardcore accountant. Plugging those leaks, but using the PEAR feedback, can result in a sudden upsurge in “recovered revenue” – and that’s never a bad thing. In my experience, this almost always pays for itself many times over, when considering the low cost of having a few staffers do the studies and process review activities necessary to find those leaks.

FORGET THE WALLPAPER

So never mind the AS9100 certification for a minute. Use the data on the PEAR to drive your continual improvement for purely selfish reasons: you want your company to make more money and succeed. You want to improve your processes and grow your company. You want to be a job creator.

Ensure your registrar completes robust PEAR reports (preferably relying on information they gathered during the audit, and not what your employees “fed” the auditor before the audit even began), and then use the data to improve.

For a free PEAR file you can use in your own internal audit system, click here.

Advertisements

Traditional Tri-System