A study by a government-funded research team found that risk-based audits of UK healthcare facilities had “little measurable impact” on the quality of care, and that patients rarely used the results of such audits to make healthcare decisions anyway.

Currently, UK healthcare facilities are audited through a program developed by the Care Quality Commission (CQC). According to an article appearing in the British Medical Journal (paywall), a study commissioned by the Department of Health and Social Care’s Policy Research Programme found that the CQC’s “resource-intensive and very high-profile system of inspection and rating does not seem to have had more than quite small and mixed effects on available performance indicators.”

Risk-based auditing has gained new prominence since the release of ISO 9001:2015, which invented the concept of “risk-based thinking” and under which companies can be audited for third-party certifications. The dubious results of ISO 9001 certification and risk-based thinking are forcing similar conversations in the quality management certification scheme, as more and more companies are found holding such certifications while engaged in deadly product recalls, financial scandals, or other malpractice.

One of the criticisms levied against the CQC audit program is that target healthcare facilities are notified in advance of such audits, much like those of companies undergoing ISO 9001 audits. This allows the company to “clean house” just prior to the audit, as well as to hide evidence of nonconformities or malpractice.

The report also cited problems with inspections forcing CQC to de-prioritize other concerns:

Inspection and rating have dominated CQC’s regulatory model, consumed most of its available regulatory resources, and may have crowded out some other potential regulatory activities that might be more impactful.

In 2014, CQC was forced to publicly apologize for failings within its risk-based scheme.

ISO still maintains that “risk-based” audits and management is effective, but provides no evidence to support its claim; instead, it relies on a broad network of consultants and auditing bodies to influence industry opinion through one-sided journal articles, blogs and social media posts.

Similar to problems identified with auditors under the ISO certification scheme, the UK report called for improvement of CQC’s auditors, specifically:

… invest in recruitment and training to create an inspection workforce with the credibility and skills necessary to foster improvement through close relationships, while maintaining consistency and objectivity.

A major criticism of the ISO scheme auditors is poor training, lack of objectivity, and tendencies to provide consulting during audits, despite this being prohibited. These actions are then ignored by oversight accreditation bodies such as UKAS, even after official complaints are filed against offending registrars. UKAS is funded by income derived from the auditing bodies, reducing their incentive to manage its registrars.

The CQC program is not part of the ISO certification scheme, but borrows much from the ISO auditing model which was launched in the late 1980’s. Prompted by raw marketing by ISO and other related bodies, other sectors of society have adopted ISO-style auditing practices, despite increasing evidence the methods are ineffective and often help cover up problems, rather than highlight them.

Recently, it was revealed that Equifax held third-party, accredited certification to ISO 27001, the information security management system standard, during the massive hack of millions of consumers’ personal financial data. In a related incident, the ISO 27001 auditing body Alcumus ISOQAR itself announced it had accidentally sent a phishing email campaign, ironically exposing its ISO 27001-certified clients to information security risks.

Companies involved in high profile disasters and scandals were all found to be holding accredited ISO 9001 quality certifications, including those responsible for the Takata airbag deaths, Deepwater Horizon oil rig disaster, Kobe Steel test falsifications, VW emissions scandal, and others. In recent years, the US Dept. of Defense has audited multiple companies holding AS9100 certification for aerospace manufacturing, and found hundreds of nonconformities despite the companies’ auditing bodies have issued certificates anyway. In the UK, an ISO registrar granted an ISO 9001 certificate to a company in Leicester later found to be a false “front” for an illegal heroin smuggling operation; despite the jailing of the company officers, the registrar still allows the drug smuggling ring to promote their ISO 9001 certificate and their logo.

The oversight bodies, including ISO and the IAF, have refused to comment or take action on their roles in the scandals, and largely remain immune from involvement in post-disaster investigations, prompted largely by their obscure roles, which are poorly understood by regulators and journalists.

The full report on the CQC study can be downloaded here (PDF). A summary may be found on the King’s Fund website here.