Conjunction of the Spheres: Secure Integration of IT and OT in Modern Industrial Environments

Imagine a factory grinding to a halt due to a cyberattack, causing millions in losses and potentially endangering lives and destroying the environment around it. Sadly this is not fiction, but real risks for modern OT environments. In the modern industrial landscape, Operational Technology (OT) environments are essential in the day-to-day operations of companies from manufacturing plants and energy grids to transportation and utilities. OT systems control physical processes and equipment, on which our modern societies and businesses are built. IT systems on the other hand are used to handle the information and business processes, which are required to direct what is done with the OT environments. In any company that must run physical process as a part of its business, these two environments must work in harmony for the company to conduct its business. They are however traditionally kept separate in many ways both logically and operationally and interconnectivity, the defining feature of our IT environments, took time to begin to emerge on the OT side.

This separation has historically been beneficial as it has isolated critical control systems from the cyber threats, that have proliferated in the IT world. However, as companies seek to leverage digitalization for greater efficiencies and competitive advantage, integration between the IT and OT environments is exposing the OT environments to the IT side cyber threats.

?

Differences Between IT and OT

Major risk lies in the operational separation between the two environments and the convergence of IT and OT introduces several new challenges. While IT systems evolve quickly, with constant software updates and improvements, and the lifespan of the equipment is usually counted in years, OT systems are more static and, once installed, stay in place for decades with minimal changes. The systems are specified and designed to do few things efficiently in a rather unchanging environment, and even the contracts may prohibit any installation of customer’s software on the OT system related computers, or limit for example anti-virus or EDR solutions to a specific product. The devices can also be so delicate that even a simple network scan can cause serious disruption in the OT environment.

The security practices on both sides are completely different and, in many cases, incompatible. The OT environment is usually designed with focus on reliability and safety over security with minimal focus on threats coming in over network. The devices may have been intended to only communicate in their own local network with no outside connectivity, or a simple perimeter firewall is expected to protect the equipment and everything inside the firewall is trusted. Which is not unlike the IT environment 10 – 15 years ago.

Another difference is that the workforce managing the respective environments typically work independently of each other, possess different skill sets, and even different lingo. IT professionals are versed in network and server management, cybersecurity practices, and data protection, whereas OT professionals are experts in maintaining the physical equipment and operations and ensuring safety and compliance with industrial standards. Both also nowadays rely heavily on external vendors and service providers, which complicates the picture. On the IT side the company’s IT systems are built over time by the IT department or external IT service providers. Different systems, such as HR, Financial, or document management systems may also be provided by external vendors as on-prem installations or as cloud services. On the OT side the industrial systems are rarely built from the scratch by the company itself, but procured from vendors such as ABB, Honeywell, or Siemens as complete solutions, that are delivered by the vendor. In both cases the support by the vendor after installation depends on the contract signed by the company and it is the purchasing company’s responsibility to make sure updates, maintenance, monitoring, backups, etc. are either arranged by the vendor or by the purchasing company, either by itself or by some third party.

A key blind spot in OT environments are the PC’s delivered by the vendor, as a part of a turn-key solution, but maintenance is not included. The company IT is also not included in the procurement process, and no-one considers how those PCs are going to be administered, updated, or even backed up. No-one therefore is responsible for them, and if there is a maintenance contract for the complete solution, there is likely a clause that prohibits any changes to the PCs under threat of voiding the support contract. Fast forward 10 - 20 years and we now have industrial environments full of PC’s running old Windows XP installations, running vital systems, have never been updated, without even an antivirus software running, and no-one knows who will fix them in case any issues are encountered.

?

The below table highlights some differences in focus between the two environments based on the NIST Cybersecurity Framework.

As seen in the list, both sides strive to safeguard business continuity, but their focus, goals, and operating culture are different. There is also major pressure for businesses to cross the gap technologically by interconnecting the two environments to realize business benefits. The cost-benefit analysis is however woefully incomplete if only the maintenance and business analytics benefits of connecting the old steel furnace to the internet are considered without taking notice of the cybersecurity implications and budget requirements brought by the new introduced risks. Multiple incidents, such as the Colonial Pipeline Ransomware Attack in 2021, and the company opting to pay the hacker group the demanded $4.4 million USD ransom, attest to this.

?

Bridging the Gap?

When IT and OT systems are connected to each other, they cannot be thought of as two separate environments, that are expected not to affect each other. Anything that happens in the OT side has the potential to impact IT environment and vice versa. A maintenance engineer inadvertently carrying a crypto locker into the OT environment in one factory in an unprotected environment, can lead to the OT and IT environment of the factory getting encrypted, halting the operations, or worse causing damage to the equipment, people, and the environment. If the factory is not isolated from the rest of the company environment, the worm can spread to other sites over Wide Area Networks (WAN) and continue to infect other factories, causing the same damage in all of them, as happened to the world’s largest shipping giant A.P. M?ller-Maersk in 2017 eventually costing the company between $250 and $300 million USD according to conservative estimates.

?

NIST Cybersecurity Framework

The updated NIST Cybersecurity Framework 2.0 with its new “Govern” function is a good way to manage this “Conjunction of the Spheres” (I couldn’t resist a Witcher reference here). Of course, Rome is not built in a day, but all companies should form holistic idea of where they want to be regarding the cybersecurity of their entire environment, start with identifying the low hanging fruits with the greatest impacts, and continue working from there.

Govern

Create a unified cybersecurity and safety strategy and ensure leadership commitment to support and allocate resources for promoting an integrated approach to both IT and OT security. Based on the strategy, create governance policies that address both environments, considering their unique requirements and different stakeholders.

As the two different sides have likely always worked separately and have completely different priorities, this is likely to be a major undertaking requiring serious commitment, planning, and change leadership from the top leadership.

?

Identify

Asset management and risk assessments. Create a unified inventory of all assets across your IT and OT environments, including hardware, software, network settings, configurations, interdependencies, update and backup schedules, etc. and even more importantly, who owns each system and device in the environment and who can fix it, if there is anything wrong with it. Set up a Configuration Management Database (CMDB) with procedures to keep it up to date. Use industry standards such as ITIL and ANSI/ISA-95 in creating the structure of your CMDB database.

Carry out regular risk assessments that consider both IT and OT perspectives and the potential impacts on the entire organization, including operational, safety, and cybersecurity risks. Assign priorities to the different assets based on your risk assessments and record them in the CMDB database.

?

Protect

Implement network segmentation to isolate the OT systems from the IT networks. Use the Purdue model to understand the role of the different components in the network and to help in the segmentation. Set up firewalls, Identity and Access Management (IAM), security monitoring, and rules for where and how connections between the different networks are allowed, in order to control access and identify attempts at lateral movement in the environment. Use role-based access control and Multi-Factor Authentication wherever possible.

Also consider the separation of your IT and OT Active Directory domains and connections to the cloud environment to avoid eroding segmentation (The OSI level 3 segmentation doesn’t matter, if the IT and OT environments are integrated together on the higher OSI layers). The ideal situation from security perspective would be to deny all connections to the cloud from your OT environment, but as vendors are moving their backends to the cloud, this strategy is unlikely to be feasible in the long run. Consider setting up a separate Azure tenant for your OT environment, in case you want to use for example Microsoft Defender for Endpoint in your OT environment. Also don’t set up trust between the two Active Directory domains.

Set up an update and patch management program that applies to both IT and OT environments, as far as possible, and install EDR on all devices that technically support it. Take these requirements into account when negotiating purchases and contracts with vendors and don’t allow any contracts or systems in your environment, that cannot comply with your security strategy and policies. Any exceptions need to be accompanied by risk assessments and mitigation plans.

?

Detect

Plan and implement monitoring and alerting in your entire environment. Collect the alerts and logs to a Security Information and Event Management (SIEM) system, create rules for alerting, that are based on your security policies, and set up a Security Operations Center (SOC) for monitoring and responding to the alerts. Integrate the alerts to your CMDB database so that the SOC will be able to pull up-to-date technical, priority, and contact information directly from your CMDB database when investigating incidents.

Describe what is usual and unusual behavior in your environment, so that for example access to the OT environment from your management subnets or management workstations is allowed, but a security alert is generated for any access attempt from any other source.

?

Respond

Develop and implement an Incident Response (IR) and Major Incident Management (MIM) plans, that cover both environments, are general enough to work in any situation, but detailed enough to give structure for the incident response when the situation occurs.

The most important part of the response plan is the up-to-date contact information of all stake holders such as the SOC, IT managers and engineers, helpdesk, plant managers and maintenance engineers, control room, security team, vendor support contacts, etc. If you have nothing else, have the contact list as a physical copy on every site in case the company systems go down. Also establish clear communication protocols for the incident response, also considering back up options in case the company email, chats, and phones are unavailable.

Create playbooks starting from your highest risk scenarios, set up cross-functional response teams in all your shifts and conduct regular cybersecurity trainings and drills as a part of your regular safety training to ensure that the response teams are prepared to act quickly and effectively during an incident regardless of whether it is a stroke, a fire, or a cybersecurity incident.

?

Recover

Consider both IT and OT environments in your disaster recovery and business continuity plans and ensure that the plans, backups, and recovery procedures are tested regularly. All IT and OT systems in your environment should have defined update schedules and recovery instructions. The time and costs required for restoring the systems need to also be considered in your risk management.

?

Conclusion

Implementing everything is likely to take a long time and require profound changes in an organization, that has traditionally had IT and OT people firmly taking care of their own turfs without letting the other come disturb their environment. Therefore, I believe the “Govern” function is the most important one in the NIST framework as it lays the foundation for everything else to succeed and much like any other improvement initiative, improving OT cybersecurity is also fundamentally about change leadership.

#OTCybersecurity

#ITandOTConvergence

#IndustrialCybersecurity

#CyberThreats

#IndustrialTechnology

#NISTCybersecurityFramework

#OperationalTechnology

#SmartManufacturing

#NetworkSecurity