There is much discussion over whether the current third party conformity assessment structure is effective for ISO 9001 end users and their customers. Complaints have been lodged that audits are too short, or that audits are too long. Many say Certification Body (CB) auditors don’t properly assess real outputs, allowing certified companies to continue to produce subpar products or services. They don’t really get into the details of a company’s processes. They “soft-grade” nonconformities to keep clients. The list goes on.
Currently, the audit structure is locked down into the “Stage 1 / Stage 2” format we see codified in ISO 17021-1, which was written largely by CB reps themselves, and designed to make their jobs easier rather than to ensure a competent audit.
But let’s spitball a little… what might a more robust audit program look like, and what would it cost end users? To start, let’s keep the “stage” structure as currently used, since so many people are familiar with the phrase. But those steps would fall into three major activities: application, verification and validation.
Stage 0 – Application
As it stands now, the application process for ISO 9001 certification is pretty weak, while those for AS9100 or ISO/TS 16949 are laborious. A middle ground is needed, but one that accurately captures the applicant company’s current level of documentation and process maturity, as well as what their products are, and who they serve. A standardized Application Form should be used by all CBs, allowing clients to fill it out once, and then submit to multiple CBs when gathering and comparing quotes.
The Application Form would also require the company to not only list their current documentation, but also their processes. This, along with employee count and scope of work, can help the CB decide the exact days needed for the Stages to follow, using minimum audit tables issued by CASCO, as usual. But reductions in audit days, or increases, could be negotiated and justified at the Application Review step, rather than later. NDA’s would have to be signed during the Application step.
Stage 1 – Verification of Implementation
Stage 1 of our “Dream Audit” would comprise an off-site “Verification of Implementation” of the client’s process documents. As we said, the exact length of this review would be determined during the Application step above.
In addition to a traditional document review, this stage should also include phone conferencing of at least 2 hours, so the auditor can come to understand the processes in theory, and to ensure the documentation aligns with them. Too often, companies have not really implemented the process approach, and CBs must flag this early. A failure to implement the process approach should stall any future Stages until evidence of compliance is provided.
Stage 1 would also verify that at least one round of internal audits has been completed, and one management review. There should also be a review of CARs filed, to ensure that no significant issues remain open before the CB auditor arrives.
Auditors would then request additional records and documents as needed to confirm the implementation of the standard, with the purpose of identifying any gaps in implementation, where requirements have not yet been addressed, or have been addressed in a completely noncompliant manner.
Auditors would also verify that the company had a means of assessing each of its identified processes to ensure process outputs met requirements. This would be critical, since this would be physically verified during Stage 2; if a client did not have metrics (or equivalents) for each of the identified processes, this should prevent the assessment from moving forward.
This approach does away with the on-site Stage 1 currently in play, but may extend the off-site work by at least an additional day, given that so much documentation and so many records would be reviewed. The Stage 1 should be firmly scheduled, and then result in a full Stage 1 report. As it stands now, companies pay for document reviews, and are never sure they actually happened.
The auditor should not lock down a date for Stage 2 until the report shows the system has been implemented. Then a full Stage 2 audit schedule can be developed, based on the client’s actual processes. A matrix showing how the ISO 9001 clauses align to those processes would be developed (if not provided by the client) and the audit schedule arranged accordingly. The client would be invited to participate in development of the schedule, and invited to add names of points of contact for each process.
Stage 2 – Verification of Conformity
Following on the verification that the standard has been implemented, Stage 2 would seek to verify that it is in conformity with the standard. The on-site Stage 2 would then commence as usual, except the common notions of a “clause-based audit” or “process based audit” would be thrown out. (No one ever agreed on what they meant anyway.) Instead, a “hybrid audit” would be conducted, where the auditor assesses each process (as outlined in the schedule from Stage 1), verifying the appropriate ISO 9001 clauses applicable to that process as he/she does so. At the conclusion of each process audit, the auditor would verify if the process was meeting its objectives and metrics, as identified during Stage 1.
This would be a full assessment of the company’s level of compliance to requirements, as well as a verification that each process was meeting internal goals and objectives. If a given process met all the ISO 9001 requirements, but was not meeting process objectives, ISO 9001 certification would not be granted. In short, each process would have to satisfy both the ISO 9001 requirements and internal process objectives.
Stage 3 – Validation of Conformity
Here’s where things get interesting; the final stage would go far beyond the current endgame of merely issuing a dubious certification if the company appears in compliance, and validate conformity. Remember: verification and validation are two different things.
Either immediately after the Stage 2 event, or scheduled as a separate event, the registrar would validate the actual product or service measurements to ensure the resulting products or services met requirements. This wouldn’t be a product certification, but a system certification that validated it was working as intended; that’s an important distinction. This would require some new, and potentially radical, steps:
First, the client would have to justify its product or service acceptance criteria against the requirements of customers and other stakeholders. A part manufacturer would have to show how its inspection, testing and acceptance methods and criteria were capable of proving that parts shipped to customers actually met customer requirements. For example, a company making red birdhouses would have to show that its inspections verified the color of the birdhouse (red) and the physical dimensions.
Next, the registrar would assess the methods and inspection data to validate that — in theory — they appear to prove compliance with customer and stakeholder criteria. Depending on the complexity, this may require a formal (off-site) validation of the methods by a professional and independent statistician. If the methods were found to be statistically nonsensical, or not to address customer requirements, ISO 9001 certification would not be awarded.
That sounds complicated, but it’s blindingly easy. Companies already have inspection criteria. The validation step merely ensures their criteria reflects inspections of what the customer cares about, and that the methods make sense, mathematically. For many companies, the statistical review might not even be necessary.
Finally, the client would submit at least six months’ of inspection data, showing the shipped product’s acceptance rate has been within acceptable limits. Companies that lacked six months of data would achieve a “provisional” ISO 9001 certification, to be upgraded only when the data came in; if it never came in, the provisional certification would be withdrawn entirely. Companies shipping poor quality parts would not become ISO 9001 certified at all, no matter how compliant their documentation was.
This means that ISO 9001 would only be awarded to companies in full compliance with the ISO 9001 clauses, with their processes under control, and with evidence that they are reliably producing a product or service that meets requirements.
Surveillance
The current ISO 17021-1 structure requires an initial assessment, followed by two years of truncated “surveillance audits” and then a longer “re-certification audit” at the three year mark. This is stupid, and is costing companies a lot of money for nothing.
Instead, this “Dream Audit” plan does away with the three-year re-certification entirely, and after the initial certification, surveillance continues indefinitely. Each surveillance audit would focus on half the processes, so that a full process set was re-verified after every two surveillance audits. But at every surveillance audit — no matter what — the auditor would assess all the process objectives and re-validate the product/service quality data.
Only if a company switched registrars would the process re-start, going back at the Application Stage.
Costs
So what would this cost? In fact, not much more than current schemes… and maybe even less.
The increased Application process would likely have CBs invoking a strict Application Fee that isn’t likely to get waived like the current ones often are; say, $500.
Next, it’s likely that Stage 1 would add a full day from what is currently done, so let’s add another $1300. Now deduct the travel expenses currently charged, since our proposed Stage 1 is entirely off-site. You now have a net reduction in costs, not an increase.
Next, Stage 2 would largely remain the same, based on current audit tables. So add $0.
Finally, the Stage 3 would be entirely new, so would be an increase. This might include at least another half day on site, plus expenses, and then a half day for the independent statistician’s review. To be safe, consider adding $3,000.
But this would be offset by the elimination of the extra audit days at the 3-year “re-certification” mark!
Which means, this “Dream Audit” would likely only increase certification costs by about a few thousand dollars each year at most, but which would be offset by removing the expenses currently required by the 3-year re-cert scheme.
This “Dream Audit” would thus result in a certification that actually meant what it promised: that certified companies not only complied with rote requirements, but also were capable of reliably producing high quality parts or services. The resulting program would flatline costs, while improving the trust in the end result.
[This article was updated since its original publication in March 2015.]
Christopher Paris is the founder and VP Operations of Oxebridge. He has over 30 years’ experience implementing ISO 9001 and AS9100 systems, and helps establish certification and accreditation bodies with the ISO 17000 series. He is a vocal advocate for the development and use of standards from the point of view of actual users. He is the writer and artist of THE AUDITOR comic strip, and is currently writing a pulp novella called DR. CUBA.
