The problem with allowing uninformed people to have power is that authority doesn’t improve performance; instead, they just gain the ability to make things worse for everyone else.
The new AS9104/1 standard is evidence of this truism. This standard sets the rules for how the various bodies within the IAQG’s aerospace certification program must operate, allegedly to ensure fair and trustworthy audits to standards like AS9100. We need to trust these certifications because, you know, airplanes can crash. So this stuff matters.
But the authors of AS9104/1 are so divorced from reality, you’d wonder if they have ever actually seen a plane, much less flown on one. This updated standard (now at Rev. A, released earlier this year) represents a bloated, labyrinthine spaghetti-map of bureaucratic rules, contradictory nonsense, and pompous arrogance, that is nearly incomprehensible.
There’s one silver lining, however, which I’m going to put up front, rather than dangle it near the end. They’re never gonna enforce these rules anyway, so this is all performance art.
AS9104/1 and the other related “ICOP” and “OPM” or whatever-the-hell-they’re-calling-it-now standards exist not to actually ensure the trust and validity of AS9100 certifications, and the performance of auditors or certification bodies. They are make-work projects that enable official IAQG “partners” to sell expensive training courses. Nearly immediately, the accreditation body ANAB began selling courses on AS9104/1 “revisions”:
Then, the IAQG will use its own publishing accounts to advertise those courses on behalf of the bodies it’s supposed to be overseeing. Here’s just one recent example of a post from LinkedIn showing IAQG promoting BSI’s training:
Remember, now, that IAQG’s actual role is not only to develop the AS9100 series of standards, but also enforce them. They have an entire OASIS ticket system which pretends to process complaints against CBs like BSI and ABs like ANAB. Yet, over ten years of history show that when complaints are filed, the IAQG does nothing. (As I’ve reported, we have complaints filed against BSI that had languished in OASIS for nearly a decade.) The OASIS code, however, automatically prompts teh complainant to close the ticket even though no actions have been taken.
So we’ve reported issues related to bribery, audit malpractice, CBs performing consulting during audits, and CBs issuing certificates without actually showing up. Some of these were reported in OASIS, others via traditional methods. In 100% of the cases, the IAQG did nothing, and the complaints were shut down without action.
It makes no sense why IAQG would spend so much effort on updating rules that it doggedly refuses to enforce — despite the risks to public safety and accusations of criminal fraud — unless you follow the money.
This handout to the CBs and ABs is further illustrated by the content of the new AS9104/1 rules themselves.
Too Big To Oversee
The chart below shows the sampling plan defined in AS9104/1 which is to be used by AS9100 Accreditation Bodies (ABs) when assessing their client Certification Bodies (CBs). In what can only be described as a blatant gift to ABs like ANAB and UKAS, take a look at what the AS9104/1 authors think constitutes “statistics.” The number of CB “client files” to be assessed by the AB is based on the number of clients the CB has, and drops precipitously as the number of clients increases.
Now, that chart was created by me, so here’s the actual chart from AS9104/1 itself. You can see that the authors applied no statistics at all, and just came up with some random employee count ranges to fit into a linear sample plan.
This means that ABs will apply the least amount of oversight over the largest CBs that need it the most, like BSI and NSF and Intertek. Smaller CBs, meanwhile, will have to undergo oversight at rates 100 times that of their competitors.
Now, remember that the IAQG then uses its budget to help advertise the training services of those giant CBs, like BSI, and you start to see a pattern. If I was a smaller CB, I’d sue them so dry, their kidneys would turn to powder. There is nothing at all fair and impartial about this corrupt arrangement.
Clearly, this sampling plan was not based on any actual science. It was based on the IAQG’s reality that ABs operating in this space don’t really have enough resources to do a proper job when it comes to the large CBs selling AS9100 certificates. Instead of addressing that resource problem, they just throw the giant ABs a big, fat Christmas present, and grant them staggeringly low levels of oversight.
But what does the AS9100 standard demand of the companies pursuing certification, the actual users? For them, the the AS9100 Rev. D standard’s clause 8.5.1 says (emphasis added):
… when sampling is used as a means of product acceptance, the sampling plan is justified on the basis of recognized statistical principles and appropriate for use….
So once again we see the IAQG — led largely by Boeing — telling other people to do things that it, itself, refuses to do.
Witness Audit Madness
Throwing another bone to the big ABs, the IAQG then goes insane on witness audit requirements of CBs. This appears to contradict the point above, but stick with me… it is entirely consistent.
“Witness audits” are when the AB tags along on a CB’s AS9100 audits of their clients. Many of my readers have probably experienced these, as you are forced to act as a host not only for the CB auditors, but their AB “handlers” from ANAB or UKAS. The AB sits quietly during the audit, observing and largely prohibited from saying anything, while eating your donuts and drinking your coffee. The purpose is that the AB gets to see, first hand and in real-time, if the CB is actually competent and following all the various rules.
This chart from AS9104/1 shows how many CB “witness audit” days are required to be performed by the AB based on the number of audit days the CB generally books in a year. Here we see that larger CBs get brutalized, in what appears to be a reverse of the client file review discussed above.
Here, a large CB could expect to have an AB witnessing them for over a month.
Compare that to the same table in the prior 2012 version of AS9104/1 and the jump is startling:
So, why does IAQG have such a weird disparity between the low numbers of “client files” to be reviewed (which is typically done off-site) and the staggeringly high number of (on-site) witness audits?
Again, this is a handout to the big ABs. You see, ABs generate tremendous income from witness audits. In fact, witness audits are the number one source of revenue for ABs, far exceeding their income for licensing or training services. These fees are paid by the CBs (who then pass that down in their pricing to clients), and it’s a major profit center for the ABs. A reduction in witness audit days would cripple the ABs’ financials, while a decrease in the off-site “document reviews” of client files has little impact.
In reality, these ridiculous witness audits are wholly ineffective, and everyone knows it. In one ANAB witness audit, the AB auditor literally slept in the corner, ignoring violations by the CB auditor. In another case, ANAB ignored an auditor’s emotional meltdown — so bad, he had to be walked out to the parking lot to “cool off” — and then later lied about the matter when a formal complaint was raised, saying the incident never happened.
My clients have reported similar experiences with other ABs, such as UKAS (UK), INMETRO (Brazil), ONAC (Colombia), DAkkS (Germany) and SCC (Canada).
The reality is that an AB will never de-accredit a CB, since doing so would cut off this witness audit revenue stream. So witness audits are performed as a theatrical event, to suggest to the world that the IAQG is actually on the job, and things are working. In reality, it’s a make-work project for ABs, with no actual effective oversight at all.
Using Initials for Everything Confuses Everyone
I’ll just put this up and let you form your own opinion:
Project Planning on Crack
In an attempt to (pretend to) respond to the fact that AS9100 has had no demonstrable effect on actual aerospace quality, the IAQG ate a tremendous quantity of psilocybin and came up with the most convoluted, complicated, and full-on chaotic plan for how CBs should quote AS9100 audits.
The complexity of the new rules is so outlandish, it’s clear that no CB will ever comply with this new method. Fortunately, as I said, the ABs don’t actually enforce these rules, so it’s just more fantastickal accreditation cosplay.
I can’t possibly explain these new methods in detail, so let me offer up the rough outline:
- The CB must first perform an analysis of the client organization’s “context,” to include its scope of work, customers, applicable AS standards (including more than just AS9100), and a host of other topics.
- Then, the CB must determine the “certification audit program,” which will not only be the number of audit-days to perform, but the locations and scope of the certification itself. This means analyzing facilities, covered entities, processes, and more.
- Next, the CB must perform a “risk analysis” of the client organization. This is the newest part of the process, and is itself comprised of many steps:
- Assessing the “complexity” of the client’s quality system
- Assessing the maturity of the client’s internal audit program
- Assessing the client’s “performance
This is then supported by a number of tables and figures in the AS9104/1 standard, such as this one:
And this one:
And then, finally, a risk score is developed using this monstrosity:
All of this is to determine the required audit time, which will mean some very slight alterations from the standard audit duration defined in the usual IAF documents. For many small to medium companies, the end result may be an entirely meaningless reduction (or increase) in audit time that is calculated in hours — not days. Here’s the associated table:
So let’s say a company of 50 employees is pursuing AS9100. The normal number of audit days for a company of that size would be 6 days. If they happen to get a good risk rating, their audit duration is reduced to … wait for it … 5.4 days. Assuming a day is 8 hours, that’s a reduction of about 4 hours and 48 minutes.
I’ve run the numbers on companies of various sizes, and the application of reductions in percentages results in largely meaningless calculations. Because audits are typically performed in full-day or half-day increments, any reduction by “x hours and 48 minutes” will be subject to rounding. AS9104/1 then undermines the entire complex process by saying, “The calculated audit duration shall be rounded to the nearest half day.”
That’s for initial certification audits, which are typically the longest. For surveillance audits, which are usually might only be 1 or 2 days long before any adjustments, applying the risk adjustments become nearly comical. A 50-person company undergoing a surveillance audit could see a reduction of 15 minutes. Which, after rounding, means NOTHING.
The entire exercise was pointless. It results in no changes whatsoever.
Even a large company of 1000 employees might only see a half-day reduction in audit time during a normal surveillance audit. So things don’t get particularly better for the larger industry players, either.
Now consider that all of this planning and calculations happen before the CB has even got a contract! This is just for quoting. This dramatically increases the CB’s costs for business development, and those costs will be buried into the normal day rate and “administrative fees.” So we can expect AS9100 audit costs to increase, without any corresponding increase in the quality of audits, trust of certification, or assurance of results.
All so BSI and ANAB can sell training courses.
So how will CBs react? We know already. I’ve seen CBs in India and parts of Asia already creating template spreadsheets that are pre-filled with fake client data, formed to look just like the various tables of AS9104/1. They just toss in the client’s name and employee count, and it spits out a boilerplate risk analysis. The CBs won’t actually assess any of the details nor perform any real risk analysis — the sheets are pre-filled with bogus information on “context“, “internal audit maturity,” and “on-time delivery” figures. none of the numbers are real, but the CBs know that the ABs are not going to check, and the IAQG certainly does not care.
Planes Are Still Going To Crash
For decades, we have reported on how simple adherence to oversight rules by organizations like the IAQG and ABs could help ensure that planes don’t crash, and defective products are not delivered to market. For decades, the bodies have fought back to stop this reporting, threatened to sue, even made physical threats.
So to whatever extent industry professionals can ignore AS9104/1, they should. This is absolute theatrics. I know that CBs will be required to undergo training, but no one else will. So please spend your money wisely, by not spending it at all on this nonsense.
For user organizations, understand that nothing much will actually change, except you can see an entirely unnecessary increase in AS9100 audit fees. The CBs will pass the costs of their training and the quoting oversight to you, and there’s nothing you can do about it.
I also know I am cannibalizing my own business by saying this, but I’d rather be an honest-but-broke broker than a shill, so here goes: pursue AS9100 if your clients demand it. If you’re looking for mere process improvements, consider ISO 9001 (and there’s no need to even certify to it, if no one is demanding it.) But AS9100 adds unnecessary complexity and costs, without any corresponding benefits. So unless you stand to earn a lot of customer dollars by meeting a requirement to have AS9100, you may want to sit it out.
If you absolutely have to do it, be sure to contact me at Oxebridge. At least you won’t get ripped off for consulting, too.
About Christopher Paris
Christopher Paris is the founder and VP Operations of Oxebridge. He has over 30 years' experience implementing ISO 9001 and AS9100 systems, and is a vocal advocate for the development and use of standards from the point of view of actual users. He is the author of Surviving ISO 9001 and Surviving AS9100. He reviews wines for the irreverent wine blog, Winepisser.