The IAQG gave a webinar on two important new changes to the AS9100 certification scheme, both of which aim to fix long-standing problems, but which now appear to be more kabuki theater. There were two accidental revelations, though: one put forth by what the IAQG team said, and another by what they refused to say.
The two new concepts are the Organization Certification Analysis Process (OCAP) and Performance-Based Surveillance/Recertification
Program (PBS/RP). Let’s tackle them one at a time.
OCAP shows the most promise, as it aims to provide an improved layer of validity to the issuance of AS9100 certificates. The thinking mirrors a bit of what Oxebridge did with its Q001 certification scheme but pre-dated it (so, no, they didn’t steal anything from us.) OCAP aims to throttle audit intensity based on past performance, an interesting concept.
To do this, IAQG finally acknowledges that “audit time” and “audit duration” are two different things. The duration of an audit is the time calculated “between the opening meeting and the closing meeting,” while total audit time must include all paperwork and report writing as well. The intent here is to free up AS9100 auditors to do more comprehensive auditing, without worrying about cutting things short so they can finish their audit reports during the 8-hour day. This should result in more orderly, improved audits, with less clock-watching. It should also allow auditors to actually go back to the hotel and chill, rather than rushing reports over soggy microwaved burgers and $10 mini-bottles of vodka. The downside is that billable time for audits, and overall audit length, will increase; that means clients can expect AS9100 audit costs will tick up a bit.
OCAP then will adjust audit duration based on various factors. First of these are PEAR scores from prior AS9100 audits; you might see a reduction in audit duration if your last AS9100 audit found each process to have a PEAR score of 5, the maximum. A single PEAR score of 1 (the lowest) might result in a higher audit duration.
Another factor will be customer satisfaction. Allegedly, if you have outstanding customer complaints or (gasp!) regulatory issues, audit duration will be increased. So the companies that have current scandals underway may find their audit duration jacked up a bit. (Compared to Q001’s “Incident Investigation” rules which deny certification outright if such problems are not addressed, and OCAP does seem like weak sauce.)
Finally, OCAP will consider your internal metrics, including on-time delivery and product/service quality. At last, AS9100 audits will tie into actual performance, and not just the rambling silliness of whatever an auditor happens to observe on audit day.
The biggest problem with OCAP, however, is the amount of audit duration tweaking IAQG is talking about. Consistently, the IAQG speakers (Tim Lee of Boeing, and Darrell Taylor, formerly of Raytheon) mentioned adjusting audit duration by “10%.” That’s laughable on its face. During the presentation, the speakers kept referring to massive organizations “with 5000 employees or more,” where such a thing might have an actual impact. But that’s a demographic that represents a tiny fraction of the AS9100 user base (which shows what the organizers’ mindsets are locked into, coming from companies of that size.) Instead, a typical small-to-medium enterprise (SME) is likely to have only a few dozen employees, and thus undergo an audit comprised of a few days at most. Adding 10% to such an audit thus only adds hours, not days, an arrangement which not only will result in no real improvements to audits, but also raises all sorts of questions for CBs.
For example, if a company is scheduled to undergo a 3-day audit, that’s 24 audit-hours. If OCAP deems them a risk and adds “10%” to the audit duration, that only means another 2.4 hours. Not exactly a big threat. This prompts the next set of questions:
- Does the CB round up, to 2.5 hours?
- CBs typically bill in half-day increments. Will they now have to bill in hours? Minutes?
- If 2.5 hours are added, do these get added to a single 8-hour audit day, making it a 10.5 hour day?
- Do AS9100 auditors know they may be asked to work 10.5 hour days? Did anyone ask them?
- If the extra hours are rolled over to the next day, did anyone ask the users if they want to pay a whole extra day’s hotel and rental car fee for only two and a half hours of audit time?
It’s clear that IAQG didn’t think of any of this stuff. And despite Taylor repeatedly claiming the IAQG spoke with a lot of stakeholders, we all know that’s bullshit: they spoke to the same ten people who have run this show since the beginning, with the bulk of them coming from Boeing, Lockheed, and other massive firms. In my opinion, a high-risk organization should face a lot more than a few extra hours of a fat dude eating donuts in their conference room. Meanwhile, this also increases the likelihood of auditors pulling wink-wink stunts with their clients. “I’m gonna put down 10.5 hours of auditing on the report, but will leave early today. You don’t have a problem with that, right?“)
So while OCAP has some possibilities, it’s hindered by some poorly thought-out, pie-in-the-sky ideation.
It sounds like a public television documentary on larping, but “PBS/RP” is an attempt to throw a bone to high-performing companies by reducing their audit time fairly dramatically if they can meet certain hurdles. It’s a reboot of the prior “ASRP” program, which the IAQG speakers were forced to admit no one ever really adopted.
Under PBS/RP, if a company meets certain criteria, it could see its audit time reduced by half. That’s huge.
Here are the criteria, though:
- The company must have already completed one three-year AS9100 certification cycle
- The company must have an advanced internal audit program, based on some specific PBS/RP requirements
- The company’s internal auditors must have taken the Aerospace Experienced Auditor (AEA) course
- The company must have a standalone Ethics Policy
- The company must not have any “externally reported major nonconformities” from customers, CBs, or regulatory bodies, in the past 12 months
- The company must not have had an AS9100 certificate suspension in the prior 6 years
- The company must be current on all its customer satisfaction objectives
To be honest, nearly any Oxebridge client — and probably the client of any serious consultant — would meet all but one of the requirements. That lingering requirement is also the most controversial: that internal auditors must take the AEA class. This is a gross conflict of interest, aimed at lining the pockets of Plexus, which just so happens to sell those classes. This scam should never have passed any committee, much less one that is telling everyone else they need to have “an Ethics policy”.
(I’m also not thrilled that I’m going to have to take the class now, and sit for five days listening to some guy who knows less about aerospace QMS than I do, grilling me like I’m a schoolboy. My skin crawls just thinking about it.)
Again, whereas Oxebridge Q001 tries to roll these things into reasons to outright deny certification, the new AS9100 approach only uses them to throttle audit time. Wimpy, wimpy, wimpy.
The Question They Accidentally Answered
Throughout the one-hour presentation, Lee and Taylor repeatedly beat one single drum: that the new AS9104-1 edition, which codifies OCAP and PBS/RP, was driven by an intent to align with ISO accreditation standards (like ISO 17021-1) and the associated IAF “MDs” (Mandatory Documents.)
So what does this mean?
Simple: IAQG has gone all-in on its relationship with ISO, and the debate over whether AS9100 will decouple from ISO 9001 is settled (at least for the next decade or so.)
For years, there’s been scuttlebutt both in and out of IAQG that AS9100 would follow the footsteps of the automotive standard IATF 16949 and part ways with ISO. The current arrangement cripples AS9100’s ability to improve, as it ties the standard to whatever text ISO may dream up for ISO 9001, and dramatically limits IAQG’s ability to improve certification audits. It’s also denied IAQG the pot of gold waiting for them if they kicked the entire IAF scheme out, and formed their own CB/AB monolith, similar to what the CMMI Institute had done with the CMMI program. (Now run by ISACA, but that’s another story.)
But someone at IAQG clearly never saw the big picture and has their cart tied firmly to the ISO train, regardless of how far off the tracks it appears to be heading. That’s good news for AS9100 users in that it means no major shakeups. It’s bad news in the long run, because it means AS9100 audits will continue to suck, and the AS9100 standard will remain stuck with ISO 9001’s baffling, incomprehensible language.
The Question They Intentionally Avoided
But Lee and Taylor failed to answer one question I put to them directly, which — by their act of omission — actually provided an answer. My question was intended to address the elephant in the certification scheme, and was exactly this:
The approaches rely on truthful reporting of nonconformities by CBs. But softgrading of nonconformities and artificial inflation of PEAR scores remains a serious problem. CBs also know they face very little risk of de-accreditation. What controls are in place to improve CB objectivity so that OCAP/PBSRP can work?
My point here is that IAQG can dream up OCAP and PBS/RP and ICOP and any other number of acronyms and abbreviations all day long, but what’s the point if no one follows them? I mean, seriously, do you think BSI or Intertek or NSF are the least bit afraid of being de-accredited from the AS9100 scheme?
IAQG has ignored a formal complaint filed in OASIS against BSI for nearly two years. They work hard to protect bodies like BSI. What, they’re gonna start playing tough guy now?
We know it’s a fact that auditors routinely soft-grade nonconformities, and have for a decade or more. Here’s an article on the practice from 2008. Here’s another one from 2013. Heck, auditors even brag about the practice on social media. Major nonconformities are reduced to minors, and minors are reduced to “OFIs”, all to keep the customer certified and paying their annual dues to the CB. Worse, more and more companies now undergo AS9100 audits and come out with “zero nonconformities,” something that wouldn’t have happened with such regularity just 10 years ago.
With the NCs soft-graded, auditors can then give fake PEAR scores, to ensure — again — that their clients don’t fire them, and transfer to a competitor.
OCAP relies on NCs and PEAR scores, but what’s to ensure the CBs will start playing by the rules? Would the IAQG suddenly find its balls have dropped, and are filled with the fortitude to de-accredit a bad-acting CB? Might we see a day when a CB like Bureau Veritas is forced to actually go back in and deny certification to a client after, say, the US Dept. of Defense finds dozens of major nonconformities at one of their clients?
But Taylor and Lee didn’t just ignore the question, they actively refused to answer it. While the Plexus moderator said they simply ran out of time, that appears to have been a fib. At the end of the meeting, I did a copy and paste of all the Q&A questions, and then noticed this in the resulting text file:
That tag didn’t show up on the screen itself, but the copy-and-paste picked up the metadata. Whereas the prior questions showed the timestamp and response by Tim Lee, mine was the only one tagged as “Dismissed by host.” “Dismissed by host” is also consistent with Zoom webinars, too, according to the official instructions on the Zoom Q&A feature:
Meaning, they didn’t intend on answering the question at all. They actually had to do something to dismiss it, and not merely run out the clock.
(At least they didn’t kick me out of the seminar as soon as they saw my name, like the TC 176 clowns.)
So, no, the IAQG has no plan on how to enforce any of this.
And, yes, it’s all kabuki theater. CBs won’t be punished for soft-grading NCs or inflating PEARs, because the IAQG has no balls. That means OCAP can’t work, since it will become another way that CBs can “ensure customer satisfaction” by falsifying audit data in order to win that “10% reduction in audit time.” With everybody passing every AS9100 audit, the number of companies eligible for the massive 50% reduction, through the PBS/RP program, will skyrocket, too… provided you pay your “AEA” training fee to Plexus and its pals. If overall quality in the AS9100 scheme declines, so what? Plexus made its money.
I really wanted to think that IAQG was on the right path here, and was sincerely trying to fix the problems facing the AS9100 scheme. I even gave a class the other day where I said I might have been wrong about Tim Lee all these years, and that maybe he knew what he was doing after all.
But, nope. Lee’s wheezing, slurping, unprofessional performance (I thought he was suffering an embolism in the first five minutes) just confirmed all my suspicions, and the guy shouldn’t be put within 100 yards of any certification scheme. Likewise, the IAQG is just another ISO-coddling paper tiger, a make-work project for a handful of old white dudes who work for companies that don’t even implement AS9100 themselves.
So, alas, OCAP and PBS/RP will become more meaningless exercises. Oxebridge clients could probably coast into the PBS/RP scheme and win a 50% reduction in audit time, but it will be because upon close examination, the bar is pretty low, and CBs are more than willing to spike audit data to keep their clients paying their annual fees.
This is no way to run an aerospace industry.
About Christopher Paris
Christopher Paris is the founder and VP Operations of Oxebridge. He has over 30 years' experience implementing ISO 9001 and AS9100 systems, and is a vocal advocate for the development and use of standards from the point of view of actual users. He is the author of Surviving ISO 9001 and Surviving AS9100. He reviews wines for the irreverent wine blog, Winepisser.