I’m not even exaggerating with that headline. What’s worse is that this has been in an ISO standard for over a decade now, and I only just noticed.
As I study the ISO 42001 standard on AI Management Systems, I am finding a few troubling things. I wrote about one of them here, discussing how ISO just took the information security management standard, ISO 27001, and copied it over as the core text for ISO 42001. That’s a problem because AI and cybersecurity are two different things, but the “experts” at ISO apparently don’t think so.
What carried over from ISO 27001, however, is something I had never noticed before; that’s largely because I don’t work with ISO 27001, so have not had a lot of reason to delve deep into that standard. (I do work with AI, so that’s why I’m poking around 42001 and the EU’s new AI Act.)
ISO Demands Godhood
I came across this mind-boggling bit in 42001 and realized it was copied over from 27001 without so much as a batted eyelash. Read it carefully.
And here is the same section from the ISO 27001 standard on information security management systems:
That’s right, according to (now) two ISO standards, risk assessments can be designed in such a way as to produce “consistent, valid, and comparable results.”
I really hope I am not the only person about to say the obvious: this is absolute bullshit. As in, it can’t happen in any dimension or universe known to man. It is both physically and intellectually impossible.
You cannot design a risk assessment that is infallible. It will never happen. How this got into ISO 27001 is already a mystery, but then how it got copied over into 42001 is just criminal.
Because all risk assessments are based on estimates and guesswork, no matter how many Excel formulas you write, none will ever be 100% “consistent” nor will they product “comparable results” 100% of the time. And yet ISO 42001 says you not only need to develop a system that does this, you need to “ensure” it works! Then, an auditor is going to come in and check. (But of course, they won’t because if they did, they’d quickly be confronted with the fact that they were asking for the equivalent of a perpetual motion machine or a tree that grows dollar bills.)
Risk Management = Divination
In case you’re one of the few crayon-eaters who think ISO is onto something here, let’s break this down. The origin of risk management is — factually — divination. As in oracles and tea leaves and Tarot cards. I am not being symbolic, not being metaphorical, and not being rhetorical. It literally started with mysticism.
Man’s first risk managers were the “soothsayers” who would predict the future for their leaders or others in their tribes or clans. Without any form of long-distance communication, written language, or message-sending ability, early man could only work with what he had. Man had a need to predict the future, and the mystics stepped in to fill that gap. The historical records point to oracles and fortune-tellers practicing their craft as early as 4,000 BC, but may have even preceded that. Various methods were utilized, many of which took the form of interpreting patterns in fallen objects (bones, rocks, dice), reading seemingly random objects (entrails, tea leaves), and analyzing the movement of the stars. No matter the method, all these practices had one common purpose: to read the future.
This became an important skill — no matter how dubious the results — once mankind began warring with one another. Military leaders, from the original clan chiefs, to later warlords and generals, utilized oracles and soothsayers to predict how a battle might ensue so they could alter their strategy and improve the predicted outcome. If that sounds a lot like modern risk mitigation, where we determine the likelihood of a thing happening and then try to predict the severity if it does, well, that’s by design.
Over the centuries, Norse runes, Tarot cards, and chicken bones were replaced with statistical estimates, actuarial studies, and lots of spreadsheets. But the purpose was, and still is, the same: to predict the future so we can alter the outcome in our favor.
But, news flash: mankind still cannot see into the future. We use mathematics and statistics to rely less on outright mystical superstitions, but we have not removed the inherent fallibility of any fortune-telling method. We may have reduced it, but we have not eliminated fallibility.
For ISO to not only suggest but to demand, as a firm requirement, that companies develop a fortune-telling method that is 100% accurate and repeatable is the stuff of both fantasy and severe ego fever. Someone wrote that and inserted it into 27001, never thinking twice that it was complete hogwash, and was probably even praised for it by ISO’s inept Technical Management Board. Or, more likely, no one ever read it, and it got printed anyway.
The original 2005 version of ISO 27001 did not have this language, so that means it was added with the 2013 version. We will likely never know who added it, but it’s worth pointing out that cybersecurity experts are not risk management experts, so I doubt an actual risk manager was consulted on it. This is a troubling feature of ISO standards committees: the narcissistic consultants in one committee think they are experts over the narcissistic consultants in the other committee, and they all compete with each other. God forbid ISO committees should work together towards the best interests if users.
(The authors of ISO 27001 are these guys, in case you are interested.)
What this means is that every company certified to ISO 27001 since 2013 has been issued a fraudulent certificate. There is no way any of them — not one! — has complied with this requirement, since it’s akin to asking someone to paint the sky green or walk on water while reciting Hamlet backwards. Read that again: 100% of all ISO 27001 certificates were issued to companies that did not comply with this standard, making the certificates fake.
And, now, this will apply to ISO 42001 as well. No one who receives a certificate will actually have a risk assessment method that is infallible, but everyone will get certified anyway. The best we can do is exactly what I am doing for my ISO 42001 clients: develop a risk assessment method that does an adequate job of peering into the future, but without any pretense that it’s anything more than 21st century fortune-telling.
Well, at least I’m honest about it.
Christopher Paris is the founder and VP Operations of Oxebridge. He has over 30 years’ experience implementing ISO 9001 and AS9100 systems, and helps establish certification and accreditation bodies with the ISO 17000 series. He is a vocal advocate for the development and use of standards from the point of view of actual users. He is the writer and artist of THE AUDITOR comic strip, and is currently writing the DR. CUBA pulp novel series. Visit www.drcuba.world
Trackbacks/Pingbacks