Audit programme management, a risky business?

Written by Paul Simpson, Bywater associate and quality expert.

All of the management systems standards (MSS) produced by the International Organization for Standardization (ISO) are required to follow the harmonised structure (HS) that is described in Annex SL. The HS describes common elements and text required to be included in all ISO MSS and, to the subject of this blog, requires organisations to manage an internal audit programme.

The good practice that each MSS requires is: 

9.2.2 Internal audit programme

The organization shall plan, establish, implement and maintain (an) audit programme(s), including the frequency, methods, responsibilities, planning requirements and reporting.

When establishing the internal audit programme(s), the organization shall consider the importance of the processes concerned and the results of previous audits.

Focusing on one area of this: How do we demonstrate that we have considered the importance of, for example, our quality management system (QMS) processes? My experience of preparing and auditing this requirement shows there is a lot of variability in demonstrating that we have taken importance into consideration. Many audit programmes audit each process once a year and have little to demonstrate that they have covered the important processes to the depth necessary. The same requirement exists for other MSS and the same principles can be applied to, for example, Environmental, Health & Safety and Information Security systems but I’ll focus on quality here.

Before presenting a method to do this, I’m going to introduce a new word, ‘Risk’. It’s obviously not a ‘new’ word and will be very familiar to most people involved in developing their organisation’s QMS. When we want to determine importance, a risk management tool is one way of doing this and the tool I’ll be covering neatly also covers off the process risk evaluation that is called up in the current ISO 9001, clause 4.4.1 f) address the risks and opportunities as determined in accordance with the requirements of 6.1. Apart from anything else, this is just common sense. All organisations should be aware of the relative importance of what they do in different areas of the organisation and/or the risk of things going wrong in those processes and customers being affected and reputations damaged.

The tool that I’m going to recommend is one that should be very familiar to anyone working in the aerospace and automotive sectors. Its use has spread widely and I’ve been using it for a range of applications since I discovered it in 1980. It is Failure Mode and Effects Analysis (FMEA).

So, how does this work in an audit programme context? Firstly we need to have an overview of all the processes we have within our QMS, something we should have already to meet the requirements of 9001 clause 4.4.1., as previously mentioned. For each process we need to evaluate its risk priority based on the Severity of a process failure, the likelihood of Occurrence of that failure and our ability to Detect a failure before it affects the customer. 

Put simply:

• Severity – is the importance of the process to the customer and includes its impact on our legal compliance
• Occurrence – what is the current level of process performance: have we had any complaints; what have our previous audits told us; have we detected any nonconformities ourselves?
• Detection – how good are our current in-process controls: how do they compare with the internal and external feedback after the process has completed?

To help decide on the ratings for each element of the FMEA, the organisation should establish tables to guide the risk scorers. There is an attached workbook (audit risk tables) that provides examples of how that might work. There is no right or wrong answer here; the ratings systems should be tailored to the organisation.

We can rate each of these three elements and the product of the three ratings (S x O x D) is the Risk Priority Number (RPN). Easy to remember, as it can be a SOD to do!

When we have produced an RPN by multiplying the three scores together we will have a number per process between 1 and one thousand. By itself the number is meaningless but we can use it to compare process risk (importance), and the resulting RPN can be used to define the frequency and/or the duration of the process audit within the programme. The higher the number the more frequent or the more in-depth the audit needs to be.

Organisations can establish their own rules, for example:

• All processes with an RPN over 125 will be audited at least once per year due to the overall perceived risk
• All processes with a severity rating of 8 or more will be audited at least once per year as they have a significant impact on the customer and/or legal compliance
• All processes with an RPN less than 25 will be audited only once in a three year certification cycle due to the low perceived risk