As reported earlier, ISO has just released its latest ISO Survey report, with data up to 2017. This report collects the total certifications for ISO 9001, 14001 and other standards, and is typically trotted out by ISO as a marketing piece, and typically with some editorial spin surrounding it. Every year since 2002, Oxebridge has published its annual analysis of the ISO Survey data.

That this release comes on the deadline for ISO 9001:2015 transitions is ironic. ISO has largely let this deadline come and go without a word, whereas they had been beating the drum of panic up until now. Now, on the day itself, they release a report that gives evidence that people really, really hate the new standard.

In prior years, the report was released by ISO’s then-PR flack Roger Frost, who would spin breathless and often dubious claims about the numbers, which often were not born out by even a casual analysis of the numbers themselves. Frost would later quit ISO and tear them a new blowhole in an editorial in the Nikkei Asian Review, and then promptly disappear forever. The report is now released by ISO’s Laurent Charlet, who is much more tightlipped than his predecessor. The numbers are so bad this year — as expected — it’s likely we won’t be getting any press releases from Charlet or the new ISO Secretary-General, Sergio Mujica, the latter of whom suffers otherwise from microphone and Twitter addiction. If we do, however, they are going to have to work very hard to try to spin this in their favor.

The numbers are appalling, which won’t surprise Oxebridge readers since I have predicted this. Back in 2013 you may recall I issued a “Public Call” white paper, asking ISO to put its development of ISO 9001:2015 on hold in order to address stakeholder concerns, specifically about Annex SL and the then-burgeoning “risk-based thinking” nonsense. ISO refused, and then threatened to sue me for copyright infringement. They withdrew their complaint shortly after, but the results bore out that Oxebridge white paper and, sure enough, ISO 9001 users are abandoning the standard in droves. The same happened when ISO partnered with the IAF to create totally arbitrary “transition deadlines” for the 2000 and 2008 releases, as well. In short, user organizations don’t like being forced to buy a copy of the new standard and update their quality systems just because ISO and its attendant apologists demand you do so. ISO failed to make a case for the “process approach” in 2000 (something I actually like) and repeated that disaster with Annex SL and risk-based thinking.

So now we see the first of what is likely a 3-4 year abandonment of ISO 9001 by users, which may or may not be recouped in the years thereafter. My calculations project that in the US we are likely to drop below 10,000 certificates nationwide because of the Annex SL fiasco by either 2020 or 2021. That would be the lowest the US has seen since 1995.

Let’s crack the numbers, though.

Overview

Let’s get this out of the way: ISO lost over 101,000 certificates in a single year. That’s a drop of over 9%. They regained another 53,000 to bring the overall losses down, but 79% of those certificates came from China. And you wonder why ISO and ASQ have gone all-in on China. If you ignore China, the average gain (of those countries adding certs) was only 176 per country. Meanwhile the average loss (of those dropping certs) was -874. Huge difference.

On nearly the exact day of the ISO 9001:2015 transition deadline, ISO has released its 27th cycle of The ISO Survey. (ISO used to reference these by “cycle” number, but stopped at least a decade ago; Oxebridge continues the naming just for nostalgia.) This presents the data for all ISO 9001 (and other) certificates as of 2017; ISO typically lags a year behind, obviously, since the data for the current year is not yet available.

The total ISO 9001 certificates for 2017 came in at 1,058,504 worldwide. This means that overall, ISO 9001 lost around 47,853 ISO 9001 certificates between 2016 and 2017, or a drop of 5%. I have to say “around” because ISO’s figures are not consistently reported year to year, and they have fudged the 2016 numbers (a year later) to make it appear as if they only lost 4%. I’ll get into that in a second.

But the numbers are heavily fudged.  To get to the 47K number, ISO had to edit last year’s numbers.

The report came with a (very) brief and grimly somber comment (presumably by Charlet) saying, “The results for ISO 9001 show an overall decrease of the number of certificates compared to 2016. This is explained by factors related to the participation.” This then links to an equally somber one-and-a-half page PDF file which offers little insight into the decline. Charlet tries to lay the blame of the decline in the certification bodies (CBs) who ISO relies on gathering the data from, typically through the use of AC Nielsen, the polling company. Here’s how Charlet tried to brighten up the funeral:

Each year, it is the large international certification bodies that report a high proportion of the number of certificates issued, however, for a few of them, their data fluctuates from one year to another to a relatively large extent. Before publishing the results, ISO asks them to explain those fluctuations and, while some are real changes coming from the market, others are due to changes in the way the data was reported. This year, the majority of those fluctuations originate from changes in the reporting and have had a significant impact on the overall results, in particular for ISO 9001. This explains the substantial decrease in the number of certificates for Italy and Germany and, to a lesser extent, to many other countries that, in turn, have led to a decrease in the total figures for ISO 9001 compared to 2016.

It’s likely the first part of that explanation, about “large” CB’s having data fluctuations, was due to Oxebridge uncovering that at least one major US-based CB had provided intentionally false data to artificially inflate the figures in prior years. ISO has been adjusting the US numbers since then, but without any IAF-enforced requirement for accredited CBs to provide accurate data, it’s been like a game of blindfolded whack-a-mole.

But Charlet then says the major fluctuations — such as a dramatic loss of over 97,000 certificates in Italy alone — was due to “changes in the way the data was reported.” He doesn’t’ explain that at all, and it’s not clear what he means. the data is reported simply: how many certificates did the CBs in a given country issue that year. It’s not hard math, but something when awry in Italy and their usual astronomical growth rates (always dubious to begin with) suddenly took a nosedive. Charlet says it’s dues to “reporting changes,” but then doesn’t explain what reporting actually changed. I’ll reach out to him and ask, and update this article if I hear back. But back in 2014, when I asked for a similar explanation, Charlet responded with typical ISO discourtesy by saying, “We are not entitled to provide more details than is published with the results of the survey.” Essentially, he said that he’s not going to do his one job and explain anything for ISO.

It doesn’t really matter. ISO has used a rotating set of explanations to shush away declining numbers. For the 2014 Survey, ISO claimedgrowth has plateaued.” For the 2011 Survey, ISO claimed the standard had “reached maturity.” For the 2007 Survey, ISO said, “anticipation for the new standard has slowed down.” You have to ignore these dubious and unsupported editorial comments for what they are — marketing spin by a private publishing company — and instead look at the numbers themselves.

Making long-term trend analysis sticky is the fact that ISO’s data is not consistently reported from year to year, as ISO makes silent edits to prior years’ numbers in current reports. For example, the ISO Survey for 2016 reported a total of 1,106,356 certificates to ISO 9001 in that year, but the latest 2017 report quietly reduces the total for 2016 down to 1,105,937. Without any mention, more than 400 certificates simply disappeared from the two reports. This minor tweak allows ISO to claim only a 4% loss, whereas had they used the original figure from last year’s ISO Survey, the actual loss would have been 5%. That’s the number I’m using, by the way, since I haven’t done a line-by-line, country-by-country analysis yet to determine where ISO’s math went wrong. I’m largely basing my data on what they published when they published it.

But some of the year-to-year report discrepancies can be shrugged off as minor errors, or in changes in the reporting of data when one nation splits into two, or a nation changes its name, causing some confusion in a report that goes back to 1993. But some of the changes are just too huge to ignore. Picking just the numbers for 2010, take a look at how ISO has edited the 2010 figures in subsequent ISO Surveys:

2010 ISO Survey: 1,109,905 certs reported in 2010 (original report)
2011 ISO Survey: 1,118,510 certs reported in 2010 (8,605 certs added)
2016 ISO Survey: 1,076,525 certs reported in 2010 (41,985 certs removed)

Between the original 2010 report and the latest 2017 report, the figures for 2010 have swung back and forth without any explanation. This means for years ISO was reporting higher figures for 2010 than they would later report, with presumably corrected figures.  A loss of nearly 42,000 certificates, quietly removed from the report, represented a 4% drop that ISO didn’t bother to mention. Which begs the question: how can we trust this year’s numbers, knowing they may silently “correct” them in some coming year when no one is paying attention?

Where this impacts the greatest is in the one-page PDF that the majority of readers likely download: the annual “Executive Summary.” Unfortunately for ISO, I have copies of the ISO Survey data report going back to when it was called the Mobil Survey in 1995 (5th Cycle.) But most users don’t have that data, nor would they ever look at it. So ISO can say whatever it wants in the Executive Summary, without anyone knowing; few ever compare the annual Executive Summary reports to see how ISO is fiddling with the numbers each year, nearly always to its advantage.

Keep in mind: these US and worldwide declines occurred even as the quality press and massive ASQ marketing machine are pushing ISO 9001 and refusing to publish information critical of it. This includes the (possibly illegal) threat by ISO and IAF to de-certify anyone who doesn’t buy a copy of ISO 9001 and update their certification. Even with the deck stacked entirely in their favor, they can’t get the numbers up.

Analysis by Industrial Sector

The ISO Survey provides reporting of certificates per the forty IAF Industry Codes as well, and the data is likewise shaming, this time not only for ISO but also for the IAF itself. According to the 2017 ISO Survey report, all but three industrial sectors experienced a loss. Those three industries barely registered in positive territory, with Mining and Quarrying adding 1,209 certs, Fishing gaining 181 certs, and Electricity Supply adding only 65.

Click to enlarge

The biggest loser — dropping over 22,000 certs — was Construction, followed by ISO 9001’s main customer base Metal Manufacturing, which lost over 20,000 itself. The service sector fared poorly, too, dropping about 44,000 certs when combining all the service-related IAF sector codes, robbing ISO of the ability to claim that ISO 9001 is picking up in the service industry.

The numbers are not, again, fully accurate. The total certs covered under the Industrial Sector report only come to about 746,000, and remember the total certs for 2017 was over 1 million, so there are a lot of certs not reported in the Industrial Sector figures. ISO offers no explanation for this, but we can assume many CBs didn’t bother to keep this information, so never passed it onto ISO.

Analysis by Countries

The real interesting part of the ISO Survey is the figures reported by country. ISO doesn’t crunch these numbers, but fortunately for you, Oxebridge does. The chart below compares each nation’s reporting from 2016 vs 2017, and as you can see the majority of nations showed a decline.

Click to enlarge (VERY big graphic)

The top five nations reporting the greatest losses represent a shift from prior reports, but this also points to ISO’s refusal to vet the data it gets. This year’s top losers:

Italy: lost 52,497 certificates
USA: lost 5,387 certificates
Japan: lost 4,399 certificates
Brazil: lost 3,743 certificates
Spain: lost 2,454 certificates

Consider the shift in this list since 2016, when the top five losers were Romania, France, Russia, Belarus, USA and Turkey. In 2015, the list was China, Italy, Australia, India and Spain.

This year’s loser list shows the entry of Japan, a darling of ISO, which bases its marketing these days on growth in “Asia.” ISO has repeatedly hung its hat on growth in Korea, China, India and Japan, and the loss of certs in Japan is a bellwether.

The top five gainers this year were China, Belarus, Sweden, South Korea, and Cyprus. Yes, Cyprus made the list only by adding 547 certs since last year; the bar is that low.

The presence of China is no surprise here, but the inclusion of Belarus — who, as I said, was on the top 5 loser list just last year, shows they are fudging their numbers.  But whereas the average number of certificates gained by top five gainers in 2017 was just under 10,000, this is still a dramatic decline from 2016, when the average gain amongst the top five was nearly double that. There’s just no good news here.

Let’s Talk About Italy

Italy has been an outlier in Europe for many years, with a staggering number of certificates added every year as compared to its neighbors, many of whom were shedding ISO 9001. To many, Italy’s phenomenon never made much sense, since even Italians themselves admitted they didn’t see many companies bearing ISO 9001 flags on their lawns, or see much marketing for it. Take a look at how Italy’s numbers compared to that of its neighbor France over the past few decades:

That’s a pretty massive gap. Italy does have a reputation for producing high-quality products but not to the level of, say, Japan or the USA, and yet its numbers defied belief. Now, Italy faces what may be the beginning of a massive course correction, as its number plummet. One could argue this is to correct what were artificially inflated numbers to begin with, driven by poor CB oversight by Italy’s Accreditation Body, ACCREDIA, which allowed its CBs to routinely provide false data; or it could be real, and Italy is just waking up to ISO 9001’s declining significance years after its neighbors.

ISO tries to explain it away with a tortured note, embedded in its Excel sheet, similar to prior years’ notes regarding data fluctuations in countries like Russia. This year, ISO says:

Decrease of overall number of certificates compared to 2016 due to a reduction in the number of certificates reported by some data providers that changed their way of reporting their data. The decrease can be seen for several countries and is significant for Italy and Germany. The change accounts for the decrease in the number of certificates.

It’s not clear why ISO (presumably M. Charlet) singled out Germany to pick on here, since Spain showed far more hemorrhaging than Germany, so it’s probably worth ignoring that needless slam. Unless ISO knows something about Germany’s defective data that it’s not sharing?

China

It’s always interesting to view China separately, since ISO has put so much stock in that country it literally hired a Chinese national to be its president, and now has a new president with ties to the Chinese government as well. As I have reported, nearly everyone involved in standardization, from ASQ to ANSI to IAF (which also has a Chinese CEO) is going all-in on China, even as journalists report China is “weaponizing” standards development as an overall military strategy. ISO and its partners are complicit in this, just to sell standards that ISO intends on eventually producing domestically anyway.

So ISO has, as I said, hung its hat on China. Here’s where they stand right now, after a slight course correction in 2015.

Impressive numbers no matter what you think, but there’s some caution to be had here, as well. China is thought to be the world’s top producer of fake ISO 9001 certificates, with many of those published with the official logo of China’s national accreditation body, CNAS; the latter has a vested interest to follow the government’s Made in China 2025 policy, which aims to convince the world that Chinese quality is improving. Boasting the world’s highest number of ISO 9001 certificates plays into that policy, and it matters little whether the certs are real or not. CNAS is not beholden to anyone other than the Chinese central government, and that means they are not going to put the IAF ahead of President Xi Jinping.

Remember, the majority of ISO’s gains this past year were due to a single country that is very likely reporting false data. Whereas ISO was okay with correcting tens of thousands of prior entires from countries like Italy (this year), Russia (in past years) and others, it’s unlikely ISO will ever correct anything from China. China can, at the drop of a pin, take its ball and go home, and make its own standards. Using its global buying power, China can then start to demand countries adopt “China9001” (or whatever) and put ISO out of business, forever.

And ISO knows this.

USA Analysis

Now we come to our motherland, the United States. Yet again, despite all the work of our beleaguered ANSI TAG 176 leaders like Paul Palmes, Lorri Hunt, Jack West, Charles Cianfrani and the rest, the US continues to reject ISO 9001 and send them a message that… well …, they suck at their jobs.

The USA lost a jaw-dropping 5,387 certs in 2017, which doesn’t sound like a lot until you realize that the US only had just over 30,000 certs the year prior. Keep in mind, US census data estimates that there are some 30 million commercial entities in the US, meaning that ISO 9001 has only penetrated about 0.19% of that market. If you factor in non-commercial potential users of ISO 9001, that figure plummets by one or more orders of magnitude. By every measure, ISO 9001 remains unpopular the United States, and the work of the TAG 176 is consistently reviled.

The Oxebridge model estimates the US decline will hit bottom in 2020/2021. This model utilizes the data we have from the last major “update” of ISO 9001, and the declines shown from 2006 onward, to estimate the reaction to the new ISO 9001:2015 standard, and then adjusts those numbers given additional factors such as the public’s negative reaction to Annex SL, Oxebridge user feedback on risk-based thinking, overall impact of press scandals, etc.

This assumes that ISO does not come to its senses and trigger an early revision of ISO 9001 to correct the mistakes of Annex SL and RBT. There is no evidence at all that they intend to do so, and rather the opposite: ISO continues to roll out Annex SL to standards for which it barely applies, and is pushing the term “risk-based thinking” into the worldwide industrial lexicon, even if they just invented it out of thin air in 2012. In short, ISO has gone all-in on their new approach and is not letting data or poor sales get in their way.

(Whether this is a move by China to cripple ISO from the inside, while their British and Swedish counterparts remain blithely clueless, is a theory that’s been bandied about, but I’m not quite ready to put that much tin foil on my head yet.)

How did the US stack up against its North American neighbors? It certainly still leads the pack, but we do see Canada continuing its decline, despite the influence that country wields through its SCC accreditation body, and the role of its citizen Elva Nilsen, who essentially runs the IAF. (Perhaps, once again, Ms. Nilsen should look at the data and, like the US TAG members, do some self-reflection on her effectiveness.)

Withdrawal Rationale Analysis

In prior years, on and off, ISO reported on the reasons for withdrawals. They tend to do this in years when things are good, and then suddenly omit that report when things make ISO look bad. This year it’s the latter: ISO did not provide any reporting on withdrawals, so we can’t analyze the data.

Impact of Sector Standards

For this report, the sector-specific standards such as AS9100 would not have any impact on the data, since AS9100 includes ISO9001; companies that achieve AS9100 simultaneously achieve ISO 9001, so would appear in this report. If AS9100 ever decouples, then that may not be true; for now, however, it is.

Messy Data, But Data Still

So we’ve ascertained that ISO’s data is problematic, but even accounting for the 50,000 certs they routinely get wrong, the numbers are still sufficient to show trends. To date, ISO hasn’t entirely lied about the numbers, and magically claimed that ISO jumped to 2 or 3 million overnight. I’m not suggesting they aren’t capable of that, though, and do expect to see some very strange numbers coming out of ISO in the next two or three ISO Surveys. I also think there’s the possibility that ISO may just cancel publication of the Survey entirely, once it becomes evident that publishing the data is working against their main raison d’etre, which is to sell books.

So even though the data is squirrelly, we can still trust the basic trends, which are downwards in all ways that matter. ISO should be apply the principles of ISO 9001 and using the data to improve its product, but ISO does not eat its own dog food. Expecting them to apply any kind of self-reflective thought and do anything other than churn out PDF documents is an exercise in naivety.

Having said that, the IAF could fix this overnight. There is nothing at all stopping the IAF from making accurate reporting of ISO cert data a mandatory part of accreditation. Any CB who fails to provide accurate reports is cited with a nonconformity from their attendant Accreditation Body. Multiple violations means they lose their accreditation entirely. At the same time, IAF could mandate obligatory participation in its upcoming “ISO Certificate Database.” That effort is also doomed because the IAF is too afraid of those it’s tasked with overseeing to require them to do anything; they still believe they can rely on CBs to volunteer their data, even after they have fought such universal databases for decades.

If IAF would tweak their rules — and they can do this quickly, no matter what they say — it would fix both the ISO Survey data and ensure users had a single source to verify ISO certs. Ironically, ISO and IAF would see their revenue increase since these actions would have the side effect of improving trust in accredited certificates. But neither ISO nor the IAF can see more than five minutes ahead of them, so we shouldn’t hold our breath.

In the meantime, if you’re frustrated with the scheme you can write to the following people, who are largely responsible for this mess:

(If you enjoy Oxebridge’s free annual analysis of the ISO Survey, as well as out other reporting, consider donating to the ISO 9001 Users Legal Defense Fund, a fund used to help defend the rights of ISO 9001 standards users, as well as to ensure Oxebridge can continue to provide the reporting it does.)

 

    About Christopher Paris

    Christopher Paris is the founder and VP Operations of Oxebridge. He has over 25 years' experience implementing ISO 9001 and AS9100 systems, and is a vocal advocate for the development and use of standards from the point of view of actual users. He is the author of Surviving ISO 9001:2015. He reviews wines for the irreverent wine blog, Winepisser.