Do an assessment of your Asset Management Assessments

How to do Value Based Asset Management Assessments & Decision Making.

This article dares to offer better ways to do Asset Management assessments. The first improvement focusses on improving the current qualitative assessments. The second improvement takes assessments into the realm of quantitative analyses using value driver trees or techno-economic simulation. I was a Certified Asset Management Assessor (CAMA under WPiAM) for three years and performed more than 50 Asset Management assessments at various industries (mostly mining operations) over my 17 years of experience. I am guilty of doing everything I will be talking against in this article, but I have something worthwhile to contribute.

The Status Quo

Assessments are the tools organizations use to identify gaps, risks, and areas that need to be improved. i.e. the assessments are used for decision making, so they’re important to get right.

In the world of Asset Management there seems to be mostly generic assessments using a 5-tier approach to show where you stand on various elements. The results are displayed on a spiderweb diagram or XY-axis. (such as these below). The results are aggregated through averages or weighted-averages based on scores that were designed for various dimensions.

These assessments are maturity assessments - innocence to excellence usually - that compares you to a best practice or standard. There have been diverse best practices over the years, of which standardization for various elements was achieved by PAS 55 and then ISO 55000. The attempt of standardization was taken even further with GFMAM that covers 39 subjects, as well as the SMRP and IAM that have even more best practices. These additional bodies of knowledge (BOK) however are not adopted universally, and there is still vastly more to Asset Management than what these standards and BOKs cover. Hence the flood of best practices.

My experience is that most best practices for Asset Management are in a circular reference. The organizations and consultants using best practices are recycling them among each other, and even many of the standards were developed by the people who were recycling them among each other. This is not to say that the best practices are trash or irrelevant, but they should be viewed with caution. There is precious little research to test whether these best practices are indeed best practices.

A mentor and I discussed the merits of adopting standards and best practices addressing specifically the issues raised above. The policy my company decided upon back then was to only implement s standards or best practices where value could be demonstrated.

The trouble is of course: how do you demonstrate that something is of value?

We’ll get to that, but let’s distinguish two approaches first.

I wrote an article about risk management (see the link at the bottom of the article), that spoke about qualitative and quantitative approaches. I would like to do the same here. The spiderweb or XY-axis scoring out of 5-tiers is qualitative. It is based on subjective interpretations of the elements, falls into the trap of mathematical compression (i.e. it is mathematically incorrect), and most of the questions only asks you to identify whether an element is implemented at an operation, seldom how well it is implemented.

The biggest problem with the current approach is the 5-tier scoring. Not the elements (standards and best practices) themselves. ISO 55000 and PAS55 gives great elements to evaluate, but not how to score and evaluate them holistically. The next section will address how to improve the presentation of results.

Improved Qualitative Assessments

Traditionally the elements (or aspects) are only evaluated by their existence and often by their quality. This does not ensure that they are embedded though. I’d like to propose 6-dimensions of the aspects in Asset Management, namely

• Existence - does the aspect exist formally through approved documentation or registers?
• Quality - is the quality of the aspect adequate? Design, content, application, style is in line with policies, standards, or best practices.
• Control - is the aspect controlled well? Ownership assignment, content controls, change controls.
• Awareness - is the awareness of the aspect suitable? Stakeholder identification, communication, roles and responsibilities, competence management.
• Embedment - is the aspect implemented and used? Business integration (into processes and systems), auditing of the aspect (compliance).
• Effectiveness - is the aspect reviewed? Performance & cost analyses of aspect, review & improvement.

These dimensions ensure that the aspect is doing what it is supposed to do sustainably. Each aspect and dimensions is ‘scored’ by a simple colour used as indicator of state. The indicator colours are interpreted as follows:

• Green – Okay, no significant findings
• Amber – findings that causes risk
• Red – findings that are causing issues

Note: these three scores indicate clearly where action should be taken. The maturity ratings do not show clearly where action should be taken.

The results can be plotted as follows:

Okay, so this is not exactly a score, but a heatmap instead. The heatmap shows you quickly where there are issues (currently affecting the organization) and risks (can affect the organization). This approach allows you to cluster issues to be put together as focus areas. The assessment above for instance shows a lack of the SAMP, policies, objectives, as well as reporting in general.

One could argue that this approach is still subjective. What is a risk and what is an issue? The risk component was addressed in the article linked at the end of this article, but the issue component is something that can be addressed next, as well as plans and objectives in general.

Value Based (Quantitative) Assessments

I’ll start this section with a case study, showing how Techno-Economic Simulation or Value Driver Tree (VDT) Analysis demonstrates that something is of value (to answer the hanging question earlier in this article).

The case study is an existing mining operation. There are various scenarios already loaded, but we’ll be looking at two new scenarios and compare them with each other.

First we do a sensitivity analysis to determine where we should be focusing (see figure below). Of what we can control is the cycle time of the ore haulers is the highest influence on the EBITDA.

We’ve been given two possible things we can do:

• CASE 1 is where we improve the cycle time by 2% over the next 20 years, but the cost is a tremendous R40m pa.
• CASE 2 is where we reduce the downtime for each truck by 100hpa (about 17% reduction) for R10m pa.

The question is: Are they adding value?

We can see in the figure below that they are indeed adding tremendous value. Both NPVs (@ 16% discount) are well over R250m, the cycle time improvement is even reaching R400m.

So we can emphatically demonstrate value, and even rank them and put their evaluations through WACCs and hurdle rates to determine what we should be focusing on. As we can see in the figure above, there are many other plans that could be executed, and these two projects, even though great, may have to wait for even more valuable projects.

My premise is that this quantitative approach can also be used on assessments in Asset Management. For instance, the results may as well be as presented by improvements in the application of ISO standard as shown in this figure:

Quantitative models are much more work than qualitative assessments. That is the biggest issue to overcome. But it should be evident that it can add tremendous value once the model is developed.

Conclusion

So should you do qualitative or quantitative?

Unlike the vitriol I spewed on qualitative risk analyses, I'm more soft on qualitative AM Assessments.

Quantitative analyses require effort (and costs) initially to compile the model - AM is simply too far reaching throughout an organization to keep it down to only a few areas of influence - therefore the lower costs of qualitative assessments are attractive... and they're not completely useless for decision making like qualitative risk analyses. Qualitative assessments however should make it clear to the decision maker where to take action, and which areas require attention. The current 5-tier assessments do not adequately do this.

The current 5-tier assessments also do not adequately check whether something is properly embedded and reviewed within an organization. Existence and even quality of something good is not proof of it necessarily doing what its supposed to. I think it is important to check whether the aspect is controlled, people know what they're supposed to do, the thing is compliant to how it is functioning, and it is reviewed to see whether it is achieving the expected results.

The current 5-tier assessments also displays the scores incorrectly. When arbitrarily weighted scores are aggregated the result suffers from mathematical compression and is incorrect. At least using quantitative approaches you are prevented from working with incompatible units of measure.

Quantitative models give much better insight to the value of something, but they rely on careful calibration in the model, and careful assumptions applied when creating a scenario for comparison. Validation is crucial to prevent cowboy assumptions from given inaccurate business cases for plans.

Thanks for reading, here's some more reading and videos...

If you want to discuss the VDT or Techno Economic simulations, let me know. I’m passionate about the subject, and even developed a solution that was used for the figures above.

Here's the link to the article about risk management:

Here's also some links to YouTube clips about the Techno-Economic simulations (VDT). Turn the captions on if the audio is not clear