A life cycle of an OT network, how to prevent errors?

Preamble

Isn't it strange? When we order a job to install a new network connection in our office. We demand a measuring protocol that proves that the work we ordered is fulfilled according to the specifications. Also, in our IT network we have full control of the quality, as we have invested in system for permanent network monitoring. But when we order a rebuild in our automation network, what measurements and protocols do we then demand for proofing the quality of the work? What systems do we have in place to detect malfunctions before they affect our production?

Sadly, in most cases the answer is none. We often trust our intuition, Green lights in masters and I/O modules and no communication alarms in our control systems. Measurements done in new installations have shown that many new installations only have 70% network quality when started up. Why?

The life cycle of an industrial network.

A short description about what happens over time and how to prevent errors that result in production losses. Information that enables you to do preventive maintenance. below in the graphs we see the network quality over time.

#1 Machine is put into operation

When a machine is put into operation at time 0. it takes some time before the first error occurs. This is because the network quality decrease overtime. The quality is getting bad due to problems such as wiring, aging contact, aging modules, interference from new installations etc. The time to first failure varies depending on the system and how well the installation is done from the beginning. Some installations have problems after a month, others only after several years.

#2 First errors causing production loss

The errors lead to communication failures, we reach the critical zone. Production can no longer proceed. Maintenance people solves the problem, but after a while a new problem appears, that is solved, and so on. Normally, without any network monitoring, it is when you reach the critical zone that you notice the problem. You get communication errors in your controls. The result of this errors is that the network communication fails. In many cases, this results in costly downtime in the production. It is often also very difficult for maintenance personnel to pinpoint and understand the cause of the error.

From measurements made in new installation we can see that many of the installation doesn’t start at 100% network quality. It is more like 70%. Why?

#4 Measurements during commissioning to secure quality.

If you after the installation makes measurements to prove the quality, you can make sure to start at 100%. You can pinpoint all the problems before you set the machine or line into full production. You can now delay the time for the first error to occur, Green line below. To accomplish this, you can measure quality of cables, do EMC analyzes and connect tool that can analyze the telegram traffic. Everything to be able to determine the communication quality.

It is also recommended to do measurements to check the quality before and after rebuilds. By doing this you can prove that you have not build in any new errors. But still you will over time have errors bringing down the quality to the critical level, causing production losses.

#5 Permanent network monitoring – a warning level

By implementing permanent monitoring in your network, you can take this one step further. This by introducing a warning level. Within the warning Zone, your communication is still running. You will have green light in your master and IO modules. But there are measurable disturbances. This disturbance is a so-called grey zone, normally not detected by our control system. So, what can cause these disturbances. It can be EMC problems e.g. disturbances from other installed equipment, Bad cabling and contacts, To high netload, Aging of electronics. Or that someone did swap out components, resulting in components having different firmware.

#6 To Summarize.

By having a standard that seeds for the installation to be correct from the beginning and routines for measuring during commissioning, FAT, SAT. We can make sure to start at 100% network quality. If we have a permanent network monitoring system installed, it can warn us when we reach the waring level. The warnings can be sent out by mail or visualized in HMI applications etc. We can then sleep good at night because we know how good our networks preform. The warning system gives us the possibility to do preventive maintenance without affecting the production.

By installing a permanent monitoring device an Inspektor you can monitor the quality of the communication. The Inspekor has an informative and easy to understand web interface. If it is permanently installed between the master and the first switch you will get notified if there is any change in the network quality. By mail, By OPC UA connection or by a simple relay contact. You get notification before the network fails and causing production disturbances.

You can also install a central server that collects data from all your networks. A system with a simple interface where maintenance personnel can get an overview of all networks in the facility. The PROmanage? NT is a software that can be used for this purpose.

 Are you responsible for buying in new machinery?

As a buyer of production machinery or lines that contains industrial networks or bus systems, you should always demand documentation that proves that the network is working. You should make sure to have an updated standard and routines for measuring.

If the machine is a critical part of your production, you should demand that there is a system for permanent monitoring installed. Or at least that the system is prepared for connecting measurement tools. We can help you.

 Do you want a demo, are you interested and need more information?

Contact Mattias Lindh for more information.

[email protected]

+46 (0) 70- 512 27 05