One UL Auditor is Basically Crafting the AS9100 QMS for Every Company on the Planet

A while back you may remember a controversy over the PEAR (Process Effectiveness Assessment Report), which is a mandatory document that every accredited certification body auditor must fill out when doing AS9100 audits. A few years ago, I was flooded with clients asking me to fill out the PEAR for them, and when I asked how they got it, they said their CB auditor had asked that they fill it out before the audit. There’s one wrinkle with this: per accreditation rules in ISO 17021-1, the CB auditor is supposed to fill out  —  you know — the CB auditor’s paperwork. They are not supposed to have the client do the one thing they are being paid to do.

Not only does it smack of malpractice and near theft — you’re paying the CB to do the CB’s work, not to have him make you do it — it injects tremendous distrust and uncertainty into the resulting AS9100 certificates. Nevertheless, ANAB, the IAQG and IAF all ruled that it was perfectly fine. They allow CBs to effectively ignore the accreditation rules, have clients fill in the audit paperwork, and then simply have the CB auditor “verify” it. This destroys the idea of objective third-party auditing and replaces it with a first-party audit, where one audits one’s self, but where this then gets blessed with a third-party certificate. The resulting certificate, of course, claims the entire thing was independent and objective, meaning they are fraudulent.

Mutant Ninjas

The unwitting source of the problem appears to be a single auditor, Michelle Barton of UL-DQS. In July of 2011, at an official AAQG “Auditor Workshop” in Boston, Ms. Barton gave official training on “Use of PEAR’s Auditing Application of AS9101” alongside UL Aerospace Program Manager, Henry Gamboa. The presentation gives step-by-step instructions on completing the PEAR, and while the presentation makes no claim that CBs are allowed to have clients fill it out, that appears to have nevertheless been the main takeaway. According to one course attendee:

Someone raised the question on whether it was OK to have the client help in filling the PEARs out ahead of time, and the answer was, “yes.” But to be honest, the instructors probably didn’t mean to imply the client should fill out the entire thing and it sounded like the person asking meant during the audit, not before it.

For whatever reason, attendees came away with the idea that offloading PEARs back to the client for completion was now AAQG-sanctioned advice. Things then went from bad to worse. Further on in the presentation, Ms. Barton quoted from AS9101, the rules governing AS9100 audits:

Collection of Data (4.3.2.2 of AS9101D)

NOTE 1: The processes can be depicted in various ways [e.g., process maps, turtle diagrams, SIPOC method (breakdown of supplier, inputs, process steps/tasks, outputs, customer), octopus].

While the rules clearly list “various ways” to depict processes, Ms. Barton elected to show only one — a Turtle diagram — and included training on that. She projected a screenshot of a Turtle diagram she had made in MS Word, and walked the attendees through it, effectively providing them instruction on only one of the possible process models.

 

By providing training on the Turtle, it was natural that the attendees would think that they were responsible for filling it out; what other reason would there be to train auditors on the form? Thus Barton appears to have fundamentally misunderstood AS9101, because that standard requires the client to provide some form of process flow, but leaves the format up to the client. Specifically, the clause’s actual language — which was not included in the Barton presentation — reads as follows:

The audit team leader shall require the organization to provide the necessary information and documentation for review, including the following:

• description of processes showing their sequence and interactions, including the identification of any outsourced processes.

NOTE 1: The processes can be depicted in various ways [e.g., process maps, turtle diagrams, SIPOC method (breakdown of supplier, inputs, process steps/tasks, outputs, customer), octopus].

Under this reading, it is clear that the client may opt to use any method it chooses, which is consistent with AS9100, which is ultimately the standard that clients are actually held accountable to. The intent here was that the client should have already defined its processes in some way, and that this must be provided to the CB auditor as part of the “document review.” The intent was not to impose any specific method on the client, nor to imply that CB auditors were to complete a Turtle diagram and embed it in the eventual CB audit report.

Again, for reasons which no one is admitting to, that’s not what the attendees took away. Instead, attendees were left with the impression that AS9101 required the CB auditor to fill out a process description, and that the AAQG was indicating a preference not only for a Turtle diagram, but for Ms. Barton’s specific model of a Turtle. And so, immediately after the 2011 training, sure enough, CB auditors began requiring clients to fill out both PEAR forms and Turtle diagrams prior to Stage One audits.

Pandemic

Fast forward, and everyone contacted regarding the matter at both UL-DQS and AAQG now insist this wasn’t the case, and they have no explanation as to why auditors began, in lockstep, to demand these things. Worse, they claim the “Barton Turtle” was not distributed to anyone, and yet CB auditors have been discovered using the exact, original MS Word file of Barton’s Turtle, including her name in the file metadata as the original author. Those contacted claim they have no idea how a slide in a MS Powerpoint presentation became a MS Word file, spreading through the entire AS9100 CB community.

As recently as this year — some five years after the Barton presentation — Oxebridge clients are still being required to fill out the Barton Turtle. This example comes from The Registrar Company, who — like the other CBs — is probably unaware that they are not only violating AS9101 by forcing these on clients, but that they are also effectively stealing UL-DQS’ intellectual property:

As we can see, the file is exactly the same as Barton’s, right down to the fonts used and the colored gradiant shading used for each text box. Furthermore, checking the file’s properties in Word shows Michelle Barton as the author and UL-DQS as the file’s owner:

Again, the “aerospace experts” claim to have no explanation how Ms.Barton’s Turtle diagram leaked to the entire industry, nor how it is being used by UL-DQS’ competitors across the world some five years later. Obviously, Occam’s Razor tells us that Barton shared the file, and probably did instruct the class to use it, but that’s been denied, and we lack definitive proof.

If Barton and UL really didn’t know back in 2011, they certainly found out when Oxebridge filed major complaints about the problem years back. Yet neither UL nor the AAQG ever issued a statement to the auditing pool to stop the practice, or to stop distributing Barton’s Turtle. Worse, we are supposed to believe that ANAB auditors never noticed that AS9100 clients all over the world just happened to be using the exact same Turtle diagram, so a red flag was never raised by them either. Given their predilection to literally sleep during witness audits of registrars, that’s not unexpected. For clients to be using a form provided by UL-DQS, this constitutes a violation of ISO 17021-1 which prohibits any CB from providing documentation or “client-specific solutions” to its certification clients; ANAB would have to close its eyes very, very tightly in order to overlook this. Of course,

Worse still, AS9100 auditors impose the Turtles even when the client has implemented a superior representation tool, like a process map, or written instructions. No, say the auditors, the clients must use the Barton Turtles.

To date, we have seen auditors from NQA, NSF-ISR, The Registrar Company, UL-DQS and Intertek using the Barton Turtles. Obviously, Oxebridge’s first hand experience is anecdotal and statistically small, meaning the problem is much more widespread.

The Impact

The end result is that nearly the entire AS9100 auditing pool has violated both AS9101 and ISO 17021-1, while simultaneously making AS9100 certifications effectively worthless. If AS9100 users cannot customize their quality systems to reflect their actual processes and needs, and instead are forced to adopt a model developed by a single auditor who hasn’t worked under an actual AS9100 system ever — Barton’s last practical experience was with Boeing in 1999 — then it reduces AS9100 compliance to parody.  The entire notion of objective third party auditing is destroyed if the CB auditor provides clients procedures and documentation which they created, and which they will later allege to objectively audit. ISO 17021-1 calls this a “threat to impartiality due to familiarity.” The promise of AS9100 is that companies implement the standard to reflect their actual business practices, and then submit that to an auditor for compliance assessment; CB auditors — notoriously lazy and unqualified — instead try to make their jobs easier by ensuring every system looks the same, so they don’t have to think during audits. The Barton Turtle fiasco all but enshrines this as official practice.

It’s not the first time the AAQG and its “Registration Management Committee” have blundered into malpractice. Remember, this is the same organization that thought it was fine to have official training on how to hypnotize aerospace audit clients without their consent, in order to make them more susceptible to auditor suggestions.

Not only have UL-DQS and the AAQG ignored the problem, and in fact endorsed these practices, Michelle Barton was then promoted to full Director of Aerospace Programs at UL-DQS, so there’s that. We can only wait to see what AAQG training Ms. Barton will be giving now that the latest AS9101 rules have been released, and they are more confusing than ever.

The AAQG, meanwhile, continues to refuse access to its meetings and RMC members by folks such as myself who could actually help them avoid such minefields. So if things get worse, they will have to defend their decision to insulate themselves from advice of industry stakeholders, while pushing forward on practices that can only lead to increasing risk in aerospace, not reducing it.

And things might get worse. The scandal points acts as more evidence that the IAQG AS9100 scheme is simply that — a scheme — to flow money from consumers to a handful of non-profit organization executives, moving through the intermediaries of AS9100 users, registrars and accreditation bodies. Without proper independent oversight and legal accountability, we are likely to see further degradation of the system so that more and more ineligible and incompetent companies achieve AS9100 regardless, thus putting the health of airline passengers, military personnel and astronauts in danger.