Part # 3 - OT / ICS Network Security Architecture & Segmentation

This is Part 3 - OT / ICS Network Security Architecture & segmentation of “The OT Security Dozen - The OT Security Dozen – a 12-part series on building an OT / ICS Cyber security Program ”?– an essential part of building an OT/ICS Cyber security / Management Systems (OT CSMS) Program for an industrial operations environment.

Note: you may have noticed that OT/ICS cybersecurity awareness is a common theme across "The OT Security Dozen," and hence no exclusive part on awareness itself. The aim for this series is to raise awareness on each type of controls covered, and therefore is considered an essential/integral necessity across this 12-part series.        

This part is to help end user (owner/operator) organizations understand typical options for working towards designing & building a secure OT/ICS network architecture and its awareness for the technical staff. Ultimately, the goal is to familiarize oneself with execution flow for designing & building an OT/ICS network security architecture and segment the network via enforcements around different zones.

Assuming: after performing Part # 1 OT/ICS Cyber security Assessment / Reviews against your industrial network environment and establishing Part # 2 OT / ICS Cyber security Policy & Governance ; hopefully by now you have a understanding of network architectural issues in terms of lack of properly segmenting the network into zones/conduits (between IT and OT networks and more within the OT/ICS or production environment), along with relevant policies and setting the goal towards establishing a secure OT/ICS network architecture. The next steps are to design, plan and execute a short to long term plan for re-architecting the OT/ICS network and continue building and executing an OT/ICS Cyber security program and strategy.

OT/ICS Cybersecurity Program & Mapping to Industry Standards

OT/ICS Network Architecture Basics

Whether an industrial manufacturing organization is operating at level of industry 3.0 and or at industry 4.0 or in between, having a secure network architecture is an essential first line of defense. As OT/ICS networks, if compromised, poses a higher level of risk to the organization, safety of its employees and in few cases to the public (for critical infrastructures).

A typical method adapted is separation or division of the systems into two distinct networks i.e., (a) enterprise / business / IT network and (b) process control / control / automation or simply OT/ICS network. Most organizations establish a strong perimeter around OT/ICS network by segmenting the two networks via a next-gen firewalls and or data diodes, minimizing the possibility of intrusions in case of a compromise on the IT network.

Though, there are still many manufacturers (specially in APAC) that are either in the process or still don’t have any reasonable separation between the IT and OT systems and merely manage such separation at best with a port based VLANs or assigning different subnet mask (only limiting broadcast domain) and believing it to be sufficient enough.

While segmenting between IT and OT network is a good starting point, it isn’t enough. There’s a need for defining additional sub-perimeters / zones / conduits within the OT/ICS network to place additional preventive or detective controls, and to have a better contextual visibility and protection to contain potential compromises.

Enhance Purdue Enterprise Reference Architecture (PERA) / Purdue Model

Analyze the environment and its traffic flow against ISA Purdue Reference Model (PRM) – which is a method of grouping of technologies based on their criticality to cyber-physical process.

A common approach that could be adopted is, organizing the network architecture using industry reference models like Purdue Enterprise Reference Architecture (PERA) or simply Purdue levels, ISA/IEC 62443, a 3 tier Industrial IoT Consortium, ENISA, NIST OT security guidance, SANs ICS 410 and or a combination of these models to organize OT/ICS network segmentation. PERA is a reference architecture that can model the enterprise in multiple layers and in multiple stages of the architectural life cycle:

• Level 0?— Physical process (defines actual physical processes) / field devices / equipment control (e.g., valves, pumps, motors, etc.).
• Level 1?— Local control (controllers for local cell, line, process, DCS controllers, PLCs, RTU, etc.) & Intelligent devices (sensing and manipulating the physical processes e.g., like process sensors, actuators, analyzers, and related instrumentation) (for cell, line, process).
• Level 2?— Control systems — Supervising, monitoring, and controlling the physical processes. Real-time controls and software; DCS, human-machine interface (HMI); supervisory and data acquisition (SCADA) software, local alarm servers, process analytic systems, and other similar systems as level 3, but not just plant wide.
• Level 3?— Manufacturing operations systems — Managing production workflow to produce the desired products. Batch management; manufacturing execution/operations management systems (MES/MOMS); laboratory, maintenance, and plant performance management systems; data historians and related middleware. Time frame: shifts, hours, minutes, seconds.
• Level 4?— Business logistics systems — Managing the business-related activities of the manufacturing operation. ERP is the primary system; establishes the basic plant production schedule, material use, shipping, and inventory levels. Time frame: months, weeks, days, shifts.
• Level 5 – Enterprise / IT Networks – Cloud, Managing servers, financial, ERP type systems/applications, Internet DMZ (email, web, proxy etc.).
• IIOT – Industrial Internet of Things – spans and interact across layers 1-to-5 (highlighted above).
• Safety Systems – a recommended best practice is isolating safety systems to the greatest degree possible in their own segments with major enforcement boundaries in place.?

Secure Network Architecture Reference

An OT/ICS network security architecture reference provides a blueprint or a template for a site network implementation with a common set of standards vocabulary to refer to, for design, build and implement an either a greenfield (new) and or a brownfield (existing) network environment.

Note: Having a reference architecture does not guarantee a secure or compliant implementation, nor it is systems (SCADA/DCS, etc.) specific. It’s just a means to design an OT/ICS system implementation to achieve a certain secure state and or in terms of ISA/IEC 62334 standards, achieve the right target “security level”. They are defined to achieve an objective of securing the OT/ICS networks against different type of threat actors and attacks tactics as defined below:

• SL 1 – non targeted attackers
• SL 2 – hacktivist / hobbyist hackers
• SL 3 – Professional hackers
• SL 4 – Nation states

An end user organization need to decide what target security level is desirable. IEC 62443 # 5 is the only functional requirements that applies to the network reference architecture.

Note: More on the security levels to define segments and zones / conduits in a later multi-part series on OT/ICS network architectures. This is an introduction only!        

Impact of IIOT / Industry 4.0 on OT/ICS Secure Network architectures

The accelerated adoption of IIOT, IOT, analytics, cloud, 5G, increased hyper connectivity towards Industry 4.0 have a great impact on traditional OT/ICS secure network architectures. There’s been a great debate about whether PERA/Purdue model is dead or not when it comes to industry 4.0 or IIOT based implementations. HERE is an example debate among few others.?

Organizations can begin the process of characterizing and segmenting the devices/assets based on data flows, location, critical functionality, level of trust, management ownership and or other logical combinations. Also, consider how the configuration of zones and isolation impacts the day-to-day operations, safety, and response capabilities.

Create an ICS/ industrial Demilitarized Zones (iDMZ) as an enforcement boundary (major) between IT & OT network segments by utilizing levels, tiers or zones while ensuring operational performance and safety.

Enforcement capabilities or controls to segment and isolate can be achieved using devices such as ACLs on layer 3 switches, routers, firewalls, and unidirectional gateways/data-diodes. Firewalls are typically deployed as boundary protection and to control information flows and connections between network segments. For example, implementing firewall rules and connection flows that prevent Level 4 devices from directly communicating with Level 2, 1, or 0 devices. Allowing outbound connections from lower levels / tiers / zones may represents a significant risk if unmanaged. Ensure outbound rules are as stringent as inbound rules to reduce these risks. On other hand, a unidirectional gateway, or data diode, allows traffic to flow in only one direction and acts as an additional protection against system compromises at higher levels or tiers. For example, a unidirectional gateway between layers 2 and 3 may protect the devices in layer 0, 1, and 2 from cybersecurity attacks that occurs at Layers 3, 4, or 5.?

Typical IIOT implementations have a tier model from edge to cloud, have different requirements for connectivity, traffic flows, and use of different communications channels and security. It also has specific needs for architecting the implementation of Unified Name Space (UNS), MQTT, Sparkplug B, 5G and other analytics and cloud technologies.

Organizations have an option to control both north-south and east-west traffic flows using advance techniques for OT/ICS micro-segmentation (network security technique that further segments an environment for lateral visibility of all assets in the same broadcast domain) and zero trust security models (assumes breach, verify all identities/devices, use least privileges, have continuous monitoring & response capabilities) using specialized tools and solutions. More on these in later posts.

An example representation of an enhanced industry 4.0/IIOT ready secure OT/ICS network architecture is depicted in the figure below:

Figure 1- Enhanced Purdue Model - IIOT, Wireless & Security Enforcement Boundaries

Figure 1, highlights an enhanced Purdue reference model including an implementation of IIOT, Cloud, wireless and other traditional IT and OT systems in a layered model with certain enforcement boundaries (major/minor). Few key things to know:

1. Purdue levels - have different components, services, and functions and can comprise of multiple subnets with network defenses placed between subnets, even in the same Purdue levels.
2. Example Attack Surface - components at different Purdue level have slightly different attack surface with few examples highlighted in figure 1 above.
3. Enforcement boundary – includes functions and cybersecurity defenses necessary to segment and protect the various levels.
4. Internet DMZ Boundary & IT Enforcement Boundary includes level 5 and enterprise WAN components. Defines internet perimeter
5. ICS/iDMZ Enforcement Boundary is a major ICS/OT perimeter boundary between Level 4 and Level 3 (sometimes known as Level 3.5 – IT & OT separation) and can have one or more sub-zones (depending upon the size/needs of environment):

• Level 4 to 3 sub-zone (limits all in-bound traffic)
• Level 3 to 4 sub-zone, (limits all out-bound traffic)
• Cloud Access sub-zone,
• Remote Access sub-zone,
• IIOT/MQTT broker/UNS sub-zone.

1. Enforcement Boundary (Minor): between Level 3 and Level 2 - a lesser enforcement boundary for restricting flow
2. Airgap Enforcement Boundary: is a major perimeter between Safety systems and rest of the OT network
3. Secure communications flow – across ICS/iDMZ enforcement boundary are:

• Level 4/5 pushes to and Level 3 pulls from ICS/iDMZ (level 3.5)
• Level 3 pushes to and level 4/5 pulls from ICS/iDMZ (level 3.5)
• Level 0-1-2 (if required) pushes to ICS/iDMZ (level 3.5)

Applying IT & OT Cybersecurity Best Practices – (few examples only below)

• Level 4 - Limit email, Internet access and enterprise Active directory at this level and explicitly deny this type of traffic to lower levels.
• Level 3.5 - Devices/Servers placed in ICS/iDMZ, is where all inbound and outbound traffic must stop.
• Updates for patches and endpoint signatures should be pushed through ICS DMZ
• Simple proxies to be avoided at all costs since they can blindly pass on exploits.
• Limit and monitor traffic using Next-Gen firewalls/IPS/IDS and Network Behavior Anomaly detection (NBAD) or via modern data diodes.
• Level 3 – can have an ICS specific cybersecurity operations subnet (say SecOps zone) for patch management servers, endpoint security solutions and SIEM management. Use ACLS to prevent compromise of this subnet against other layer 3 subnets. And if needed, place an OT active directory (separate domain) with no trust relationships to the enterprise/IT.
• For Cloud connectivity, assume a state of an attack and isolate traffic patterns by moving on-prem server communicating with cloud to ICS/iDMZ Cloud access sub-zone and limit the server to specific only system in ICS network.
• For IIOT solutions, consider them as a single process, isolated from other processes. ?

OEMs & Vendors – Secure Network Architecture References

Almost all the major OEMs/hardware manufacturers and or product vendors have some documented reference OT/ICS secure network architecture – few examples are below: (this is not an exhaustive list)

Example OEMs/Vendors & their OT/ICS Network Security Architecture Reference:

• #fortinet - Securing OT in the Face of IIOT and 5G
• #cisco - Industrial Automation reference architecture
• #paloaltonetworks - Reference Architecture for OT and IoT Device Security in Industrial Control Systems
• #Yokogawa - OT Architecture Design
• #Siemens - Industrial Network Security Architecture
• #schneiderelectric - Practical Overview of Implementing IEC 62443 Security Levels in Industrial control applications
• #abb - ABB Reference Architecture
• #rockwellautomation - Industrial Network Architectures
• #HiveMQ - MQTT-Based Manufacturing Reference Architectures
• #microsoft - MS Azure IOT Reference Architecture
• #aws - Industrial IOT Architecture Patterns

Recommendations

For Owner & Operators: Start by using a preferred reference architecture using both general industry best practices from international standards (e.g., IEC 62443) and or industry guidance (e.g., NIST Special Publication (SP) 800-82r3, Guide to Operational Technology (OT) Security, Europe NISA etc.) or your specific industry sector guidance (e.g., Energy / Gas, Utility / Power, Maritime, Automotive, Railway, Aviation etc.)* along with close coordination with OEMs/vendors. Start with separating IT and OT networks between level 4 and 3, defining industrial DMZ, then go to lower Purdue levels to start defining zones and conduits for separation between level 3 and level 2.

* Yes, almost each major industrial sector mentioned above and more have different types of connectivity needs, uses a variety of solutions, have different automation machine types, and cover different distances; therefore, each have some form of their own reference architecture documented in respective sector specific cyber security guidance standards/documents. So, it’s highly recommended to refer your sector/regulatory specific guidance as well when designing or re-architecting your OT/ICS networks.        

For vendors: It’s essential to provide detailed reference architecture guidance on how the products/systems developed by OEMs/vendors can support designing and implementing a secure OT/ICS network architecture layered model. ?

Key Takeaways

For Owner & Operators:

• Start segmenting - do not operate a flat or unsegmented OT/ICS network architecture.
• Limit or restrict all incoming traffic from IT network (especially Internet, email, web,) to OT/ICS network utilizing technical controls like firewalls and or data diodes / unidirectional gateways etc...
• Understand, document all the traffic flows between levels or zones/conduits while allowing only traffic that is required for OT processes along with business and technical justifications in place.
• Define a target goal to achieve a desired security and protection level for OT/ICS network.
• For greenfield implementations (new setup) – incorporate latest methods for secure network architecture, utilizing latest techniques like Zero Trust, micro/macro segmentation, SDWAN, 5G, secure IIOT architecture, etc.
• For brownfield environments (existing setup), use a phased approach to re-architecting OT/ICS networks with minimal business disruptions to production operations.?

Next Steps

If you are unsure where to start, engaging an expert is your best bet to perform an OT/ICS network architecture review / re-design and or get in touch.

For your industrial operations design, build and maintain a secure OT/ICS network architecture - It’s a great day to start “#SecuringThings”.

References:

• Tech and OEM vendors reference links provided above
• ISA/IEC 62443 Standards
• Industrial IoT Consortium (IIC)
• NIST Guidelines
• SG CII CCOP v2
• SANS

Look out for future three part series on OT/ICS Secure Network Architectures in 2023 / post this series.

Stay tuned for?the?Part 4 – OT / ICS Asset Discovery, Vulnerabilities & Threat Detection (or OT IDS) + Tool Selection & Implementation (part of Phase 2 - Implement | Deploy (Predict, Protect/Prevent & Detect)?-?coming soon….

"A properly segmented & secure OT/ICS network architecture is first line of defense against attackers and sets the tone for defining and achieving the appropriate security and protection levels to ensure defense against growing list of IT & OT cyberattacks for reducing cyber risks and start #SecuringThings on the OT/ICS production environment". (Yousuf)

About the Author:

M. Yousuf Faisal (EMBA, GICSP, ISO 27001 LA, CISSP, CISM, CISA) has two+ decades of technology & IT/OT Cyber security-related industry experience, helping organizations worldwide (specially across APAC) securing their digital transformation journey with secure-by-design principles. He has served both as an end user and mostly as an independent consultant/advisor across multiple industrial sectors and enterprise organizations. Currently, he is doing business development, pre-sales/solution, and consulting delivery for emerging technologies in IT & OT, GRC/PCI, and other Cyber security services globally (especially across APAC region). He holds a B.E. Electrical and an Executive MBA degree.