Reliability Automation

Reliability and maintenance management is still too manual. Several of the tasks can be automated. Many, but not all, tasks can be automated. For those that can, you should, because there are many benefits of doing so. Since 2015 I call the 24th of July (24/7) the?World Reliability Day to recognize the women and men who keep things running 24 hours a day, 7 days a week (24/7). Let’s give them the right tools to do their job. That is, plants, old and new, need to be equipped with more automation. Some of it is software, but a lot of it is hardware like sensors and network infrastructure. For a new greenfield plant built to the Industry 4.0 paradigm this additional automation shall be deployed from the very beginning. For an existing plant, deploying this additional automation is part of its digital transformation. Many plants have already started. Improved availability from reliability automation can help towards another 2 more weeks of production per year, and reduced maintenance cost. But, there are those that promote approaches which are more complicated than necessary. So let’s take a look at how easy it can be with particular attention to autonomous operation. Here are my personal thoughts:

Autonomation (not a spelling error)

"Autonomous" plus "automation" gives "autonomation" (my italics). Another interesting term and concept from Toyota. But the concept can be applied to process plants as well. It is particularly relevant for sites looking for autonomous operation. A truly autonomous plant would require no human intervention. It would produce product and maintain itself. But robots are still not able to perform lubrication, laser alignment, bearing replacement, and other maintenance tasks. However, you can today detect operating problems in the process and process equipment so the tasks up to the point where you need to turn a wrench can be automated and monitored remotely from a central location. So the key piece I see in autonomation is to detect operating problems that could lead to failure, process upsets, or fouling and inefficiency etc. which is what some plants have now started doing with additional sensors and analytics apps. Autonomation is striking a balance of what is cost effective to automate and what is not. For instance, if taking the extra step of automatically computing Remaining Useful Life (RUL) time before failure, with a great level of confidence, has little or no additional cost and effort, then add automation for that. But if the additional cost and effort for a trustworthy RUL is very high, then descriptive or prescriptive analytics is sufficient provided it is predictive (advance notice). Autonomation detecting operating problems enables rapid correction of the problem before the process or equipment is affected. And it relieves reliability personnel from continuously inspecting and judging the health of the equipment. Similarly it helps the production operators judge if the process (continuous or batch) is doing OK.

Key to autonomation detecting operating problems is installing permanent sensors to pick up on early symptoms of trouble to come, such as increased vibration, rising temperature, louder acoustic noise, thinning of pipe walls, pressure drop or increase, and fluid level drop or increase. So this is the recommendation. Now you get reliable information based on actual data once a minute or once an hour. This is how you become truly predictive, and can have greater confidence in the diagnostics (analytics).

What used to be inspected monthly, with automation gets checked hourly or once a minute

Keep Digitalization Simple

Robots for inspection sounds futuristic but I don’t think it is the right way. Trials have demonstrated a frame on continuous tracks that crawl the plant point by point with a camera and other sensors. But just like humans, robots are not there at every inspection point measuring all the time. They visit periodically, and in between, sudden developments of interest will be missed. Depending on the number of inspection points each robot is expected to cover there could be days or weeks between inspections which is not predictive. The same problem as with humans today. Some locations may be hard to access. Moreover, drawbacks of the installed mechanical gauges remain. Robots are themselves moving parts that require maintenance. That is robots are not an ideal solution for data collection. But sure, crawler robots may be used for inspection in places where sensors cannot be installed and humans cannot go.

Similarly drones have applications where sensors cannot be installed, but if a sensor can be permanently installed, that is generally a better solution.

A permanent electronic sensor is a much better solution to automate the collection of reliability and other data because it measures more frequently; daily, hourly, by the minute, or second as required. The plant becomes more predictive. So this is another recommendation. Your I&C engineers have a knack for figuring out what sensors are required to take the places of manual inspection and portable testers, and what the appropriate update period should be.

Design for Reliability Management

Most plants are built low budget where only the most critical equipment like turbomachinery is instrumented for condition monitoring. However, this sets the plant up for higher operational cost in the long run since a lot of manual inspection is required, and with time-based preventive maintenance procedures there are still surprise failures.

Instead the recommendation for new Industrie 4.0 plants is to design with reliability programs in mind with design input from operations & maintenance to be setup with new condition monitoring technology to enable predictive maintenance procedures. And this should be already in the pre-FEED and FEED stages. This will require a higher capital project budget but will result in lower maintenance cost. Analytics software together with sensors can detect developing problems early. These could be problems originating in poor design decisions, installation or commissioning mistakes, or mistakes during maintenance activities like lubrication, alignment, or balancing. By detecting these problems early, they can be corrected before they cause premature failure and downtime. Automating monitoring reduces the burden on manual inspection. The value of digital transformation is to uncover equipment issues sooner, allowing maintenance personnel to act, to keep the plant running. An analytics app or a sensor doesn’t make the failures go away – but you can detect developing problems sooner and do something about it.

Design plant automation with reliability programs in mind

Existing plants need to digitally transform, including condition monitoring to be competitive with other plants globally which may be lower wage, lower energy cost, lower feedstock price etc. The principles of condition monitoring and analytics with software and sensors is already proven on the critical equipment in the plant. Now we are simply applying those same principles to second tier essential equipment to bring an older plant to the Industry 4.0 level. Second tier equipment like motors, pumps, and gearboxes do not need as sophisticated monitoring equipment as large turbomachinery, so the cost is also much lower. Thanks to wireless vibration transmitters and other sensors, it is economically feasible to do this for large numbers of equipment.

Design for reliability management must include developing a vision for reliability and maintenance in the plant, and then defining the equipment monitoring strategy. With such a template the condition monitoring details can be worked out: the software, the wireless infrastructure, and the sensors. Contrary to what many tell you, software is not enough. Additional sensors are required to pick up on symptoms of developing problems early. The Digital Operational Infrastructure (DOI) stack at the top has the visualization such as higher-level dashboard of the overall health of the plant. Below that is the predictive analytics apps. The apps in turn get their data from the data management platform; typically the historian (mostly process data) extended with a data lake (mostly reliability data). All this rests on a foundation of sensors for data collection.

That is, you need more than one type of app in your reliability software toolbox. Yet, all these apps can have a common look and feel, so even people that use more than one app in their daily work can move seamlessly from one to the other.

So clearly software can help increase plant uptime. Yet, the software should not have long and costly implementation periods or lock the plant into costly dependencies.

New Reliability Tools

Tools make all the difference to successfully completing work. Without tools or without the right tools you can’t do the job well. Therefore it is important to equip reliability engineers and technicians with new powerful digital tools. Don’t expect better job performance without providing better tools. Wireless data collection is transforming inspection practices by removing the manual data collection step. Similarly, readymade analytics apps are transforming interpretation by removing the manual data interpretation step. There are readymade apps for condition monitoring of common equipment types found in plants. Your automation vendor can tell you about all the apps available.

The principle of operation of the software plus sensor tools is:

• Measure symptoms more frequently with sensors
• Analyze to diagnose health with software
• Follow the health trend to predict failure
• Show description and prescription (recommended actions)
• Alarm on exception

The recommendation is to start with the required sensors. The description, and in some cases prescription, for each failure mode help the maintenance technician service the equipment correctly.

Avoid Cloud and App Trap

There are lots of reliability, maintenance, and other tasks to be automated in the plant. You can’t do all of them at the same time. You will have to roll out digital transformation one or a few use-cases at a time. However, it is also important to think of the big picture; what the final system will look like. If each use-case is solved in isolation, you will end up with a piecemeal solution.

Potential pitfalls:

• Each vendor has their own cloud service, only for their sensors
• Each vendor has their own mobile app, only for their sensors
• Each vendor has their own data logging software, only for their sensors

Imagine one cloud site, one mobile app, and one data logging software for one brand of vibration sensor. Another cloud site, another mobile app, and another data logging software for another brand of vibration sensor. Another cloud site, mobile app, and data logging software for UT corrosion. Another for ER/LPR corrosion, valve position, temperature for heat exchanger monitoring, and yet another for pressure for plugging of filters a.s.o. It would fast become a clutter of mobile apps hard to navigate. A plethora of cloud sites for which access must be individually managed. And islands of data logging software not integrating with existing data lake, historian, or DCS. This is not hypothetical. This is happening in some plants.

By thinking ahead, by thinking big about the many other use-cases that will be solved over time, yet allowing you to start small, these problems can be avoided. The recommendation is to select tools built on international standards:

• WirelessHART (IEC62591) sensors
• HART-IP backhaul network from wireless gateways to software
• Modbus/TCP and OPC-UA (IEC62541) server for gateways
• OPC-UA (IEC62541) software such as data lake, historian, analytics, and operator graphics

By using hardware and software components based on these standards all the pieces fit together as a single system even when bought separately, from different vendors, at different times. Reliability, integrity, and performance of multiple types of process equipment can be managed using a single common wireless sensor network, in a single application framework, with mobile notifications in a single mobile app, sent through a single cloud subscription. This is a lot easier to manage. Yet, since it is based on standards, it is not proprietary, and not monolithic. This recommendation also means avoid non-IEC and non-IEEE wireless hardware. Avoid custom coding of apps and APIs for all software.

For cloud there are additional considerations. Plants are concerned with being stuck with dependencies on third-party cloud service providers. The most interesting fact is that most plants prefer to use on-premises edge computing instead of cloud for several reasons:

Criticality: Plants don’t want certain functions to be reliant on the Internet connection working properly

Lag: Sending sensor data across the Internet cause delay which is not acceptable for certain use-cases

Confidentiality: Plants don’t want to expose equipment condition and other data to external suppliers

That is, cloud is not a must for digital transformation. Many plants deploy the automation on-premises.

CMMS 4.0

A Computerized Maintenance Management System (CMMS) is found in many plants, used to keep a record of all plant equipment, schedule preventive maintenance, maintain records of maintenance performed etc. However, in most plants data is entered manually, either through a desktop computer in the office or on a tablet or a mobile computer in the field. This includes for instance manually opening a ticket if an inspected piece of equipment needs overhaul including a description of the problem first seen as well as findings from detail troubleshooting and overhaul.

Computers and tablets do not automate data collection because you still need a human to take the reading and type it in. Only a sensor automates data collection.

Scheduling in the CMMS has until now been very much about calendar-based preventive maintenance which works for age-related failures. But Nolan and Heap found age-related failures is only 11% of failures. The other 89% of failures are random, and for these failures, condition-based maintenance works better.

The cause of this challenge is lack of information. The only way to see what is going on with most equipment at any moment is to go look. This is a fundamental limit of twentieth-century CMMS - still in use today. Almost all the data entered into the CMMS is typed in by people. Maintenance and reliability technicians do not have time to check equipment hourly or even daily, then type what they found, so CMMS is blind. Maintenance and reliability engineers are not the first to discover that equipment was degrading and about to fail; production operators are; the equipment fails, and the process unit trips. Maintenance systems without sensors are like brains without senses.

Therefore the recommendation is to focus on predictive analytics and the enabling advanced sensors. Predictive analytics does not require a CMMS, but predictive analytics can be integrated to the CMMS making the CMMS more effective. For instance, when predictive analytics is integrated to the CMMS, the analytics can automatically trigger a workorder in the CMMS when trouble is predicted in a piece of equipment. The workorder is approved by a maintenance scheduler before it is released.

Manual Data Quality

The usefulness of maintenance records in the CMMS can be undermined by poor manual data entry practices. When free-form text input is used the same problem will invariably be described differently by each person. Even with a set of ‘failure-codes’ for each type of problem, poor housekeeping will result in creation of multiple additional failure-codes for the same problem. In other cases people simply select ‘others’ when they are in a hurry. There are also cases where people forget to log the problem and the fix. This makes it hard to use CMMS data for analytics such as to establish correlation between cause and effect. Even if you don’t have experience with logging maintenance activities yourself, I’m sure you make payments online where you must select “purpose of transfer” to classify the payment to enable the app to report how you spend the money. Payment classes may include credit card, medical, dental, rent, phone bill, transport, utilities, education, investments, loan repayment, tax, and ‘others’. But we often do not have time to scroll through the list to pick the right one so we select ‘others’. Or worse, accidentally classify it wrongly. And as a result the spending analytics and report is not accurate. The same thing happens when failures are not categorized, or are categorized incorrectly, in the CMMS.

Fortunately, CMMS data is not required for predictive analytics on equipment. Readymade analytics apps use rule-based Artificial Intelligence (AI) based on well-known cause-and-effect for each failure mode of each type of equipment. Since the apps are ready-made, you don’t need to train the algorithm on large datasets from the historian or DCS. Instead, the analytics simply works based on real-time data direct from sensors on the equipment. That is, data need not be brought from the CMMS to the condition monitoring analytics. This makes system integration much easier because the analytics simply push equipment status up to the CMMS at level 4.

Automation 4.0

The Fourth Industrial Revolution (4IR) and Digital transformation of the plant means a new era in automation. Plants are deploying DOI. The automation software and hardware makes the plant more predictive. Additional plant automation is required to reduce the manual tasks required for reliability and maintenance management, as well as many other tasks. Schedule a meeting for 24/7, the world reliability day, or today.

Share this essay with your reliability and maintenance managers now, as well as your I&C team.

And remember, always ask for product data sheet to make sure the software is proven, and pay close attention to software screen captures in it to see if it does what is promised without expensive customization. Well, that’s my personal opinion. If you are interested in digital transformation in the process industries click “Follow” by my photo to not miss future updates. Click “Like” if you found this useful to you and to make sure you keep receiving updates in your feed and “Share” it with others if you think it would be useful to them. Save the link in case you need to refer in the future.