Auditing Improvement

How much improvement is “enough”?

It should be emphasised that the requirement in ISO 9001 is for improvement of the effectiveness of the QMS.

Improvement emanates from the objectives set by top management, which should (at least) address:

• the improvement of internal efficiency (for the organization to remain economically competitive),
• individual customer needs, and
• the level of performance that the market normally expects.

For example, in the aeronautical sector, the “acceptable rate” of nonconforming delivered product is zero percent, so it would not be useful for the organization to set objectives for an “improvement" in this rate.

However, it would be useful for the organization to have objectives aimed in improving its internal efficiency and its competitiveness (e.g. through innovation).

The auditor should seek to determine if the auditee has attempted to set objectives that establish the correlation between the 3 factors of:

• corporate objectives,
• customer needs, and
• market expectations.

Thereafter, it is up to the organization to balance the need for improving internal efficiency and the need to progress with external performance (although the two are very often closely related).

No one in isolation can ever be considered as being “enough” or “not enough”.

One area which can be problematic for the auditor to know is what a reasonable market benchmark is. Continuing the above aeronautical example, if the organization announced that it had improved from a level of 50% nonconforming product delivered to 40%, this would demonstrate improvement, but would hardly be acceptable, given the industry sector's zero percent normal rate.

However, if it announced that it had set an objective to improve its performance from 0.50% to 0.40%, this would be much nearer the market norm.

The only real solution for the auditor is to verify how the organization has determined this proposed rate of improvement, how it has evaluated the associated risks, and how this relates to customer requirements and the monitoring of feedback on customer satisfaction.

It would be almost impossible to issue an NCR that stated: “There was not enough improvement”.

What sort of information is relevant and where can we find it?

The auditor has to verify how the overall corporate objectives have been translated into internal requirements throughout the appropriate processes, and how these requirements are communicated and monitored.

So, the auditor should look for evidence that the organization is analysing data from process monitoring, and is then taking the results forward for evaluating process efficiency and/or improving process output.

One point that should be specifically examined is the consistency of the way in which the improvement of any one process contributes to meeting the overall objectives, in order to ensure that this will not cause conflict in the achievement of other objectives.

ISO 9001 lists a number of areas that an auditor can assess to obtain evidence of both planning and actually implementation of improvement.

Examples of areas where the quality management system can be improved include, but are not limited to:

• internal communications,
• follow-up activities,
• documented procedures,
• the effectiveness of management review meetings,
• customer feedback systems, and
• training programs (e.g. for management or for internal auditors).

The type of information that an auditor needs to look for, is evidence of how the corporate objectives are translated into specific QMS objectives. For example: an organization could set an objective to reduce customer complaints by 30%.

The top management analysis shows that 50% of the complaints concern overdue deliveries.

The auditor should then look for evidence that the organization is monitoring and analysing key aspects of its scheduling and planning activities, throughout its processes, and the process interfaces, to reduce delays.

Improvement of the product, process or improvement of the QMS?

It is important to understand that improvement doesn’t necessarily just mean improvement of product or process, but can and should also apply to the quality management system itself.

An auditor should remember that it would be unrealistic to expect an organization to make progress on all potential improvements simultaneously.

What it means is when opportunities for improvement are identified and when such improvements are justified, an organization needs to decide how they are to be implemented, based on the available resources.

Each improvement will require the commitment of resources, which may need prioritisation by top management, especially where investments are needed. Instead, the auditor should seek to ensure that the improvement objectives are consistent overall, and are coherent with the trilogy of factors mentioned above.

However, an organization that does not have a policy and objectives relating to improvement is clearly not complying with the standard.

Similarly, the absence of any evidence of improvement on at least one of these aspects may be considered as indicating that an organization's quality policy is not in line with ISO 9001.

One caution

There is no requirement that the organization should set objectives for improvement of all its processes at any one time.

As in the above example on reducing customer complaints, some processes may not be deemed by top management to contribute significantly to the reduction of delays, and it is only normal therefore, that the organization would not concentrate on these areas.

If the top management has set a (realistic) objective for a process, and there is no evidence of improvement, this information must be fed back into the management review so that top management can decide what type of action is appropriate - for example, re-adjusting the objective or providing other means to impact on the process.