My Experience with Network and Service Automation

I have been in the Network/Service Automation space among other areas like - Service Order Management, Inventory, Provisioning, Fault Management etc - for quite some time and worked on auto provisioning and activation of residential and enterprises services in provider networks. Of late I see a lot of discussions and traction around this area with events like - Autocon1/Autocon2 - gaining a lot of momentum from different organizations and individuals alike. This article is to put forth my experience in the area of Service/Network automation and to list the different products - both commercial and open source - that are currently in the market that are addressing this area.

? NSoT (Network Source of Truth) is the buzz word these days and there are a lot of discussions and debates about what NSoT really is and there are references to order history, telemetry data, network data, excel sheets and so on and the reconciliation among them etc etc. While all this holds true depending on the specific context in hand, I want to touch upon my experience on a product which relied on only one source of truth to do its job and do it better.

? Between 2004 and 2014, I was working on a product which was primarily used for service provisioning and activation at a majority of tier 1’s in Europe and the US and this product used only the Network data as a Source of Truth and nothing else. One of the prime goals is to reduce fall outs during provisioning and activation and thus save time and resources (on redesigning and repeating the tasks for failed orders) and this can happen only if the application has a real-time copy of the Network data. The application DB was populated with network data by discovering all the information that is required so that provisioning can use the discovered data to identify the available resources that can be used for a successful activation. “The Network is the Database”.

? Vendor specific adapters were built to discover network data either via direct interaction with the device or via the vendor EMS/NMS. A wide variety of protocols (SNMP, TL1, CORBA, SSH, Telnet, TeMIP etc.) or standards based (TMF814, MTOSI)/proprietary interfaces (APIs) were used to interact with the southbound systems and extract, transform and load the data into the application DB. On successful discovery, a copy of the network is present in the DB which can then be used for tasks like “Service Discovery” (stitching the network data to identify end-to-end services so that MACD operations can be performed), “Service Optimization” (Re-grooming of Services based on network augmentations that have occurred post service creation, migrating to newer devices or different OEMs etc.), “Reconciliation” with other data sources to align intent with implementation or vice-versa and so on.

? Provisioning of residential and enterprise services involved path finding between customer endpoints (UNIs) with matching QoS and identifying the required resources like ATM VPI/VCI, FR DLCI, VLAN IDs, Timeslots, IP Addresses, VRF Target ID’s, LSP Labels etc using out of the box functionality and optionally inducing operator specified business rules to influence the path finding and number management algorithms to align with the operator policies. Once all the required resources are identified across the multi-vendor network, network wide transactions were created to configure the devices directly or via EMS/NMS. The transactions are atomic in nature i.e., the provision and activation request ?either succeeds or fails in entirety with no scope for any partial result. Resources part of the activation are assigned activation and rollback sequence numbers such that the configuration is applied in a certain order complying with the dependencies imposed by the end system. For ex:, if a VLAN has to be created before it can be applied to an interface, the VLAN creation happens first followed by the interface modification and the reverse during rollback. Any number of devices and any number of objects within each device can be part of a single transaction and the configuration is applied concurrently across devices.

? Broadband services using DSL, GPON, Ethernet etc and Enterprise services using SDH (Ex: IPLC, NPLC) , Ethernet (Ex: VPWS, VPLS), MPLS-TP, MPLS, IP (Ex: GVPN, ILL) etc were provisioned automatically. ?Per domain Service applications were built for provisioning services pertaining to the specific domain. Ex: DSL Service Application, ATM Service Application, Ethernet Service Application, VPN Service Application etc. ?Each Service Application had the corresponding domain knowledge built into it in the form of Service objects. Each Service object caters to a specific aspect of the Service. Take Ethernet Service Application for example, it would contain objects like - Ethernet Subscriber (Refers to the Subscriber of the Service at the UNI), VLAN Service (Refers to the configuration of VLAN across the UNI, NNI, the Metro Access Rings and the PE), QoS Service (Refers to the QoS to be configured for the Service being provisioned) etc. All these Services were intent based i.e., the attributes part of these Service objects are vendor agnostic and depict the intent rather than the specifics. Attributes like - Ethernet port, Subscriber Name, VLAN ID, CBR, PCR, CBS etc are modeled in a vendor agnostic manner. How these get translated into the vendor specific attributes or commands is the responsibility of the underlying layers.

The overall application stack at a very high level would look like the below -

Even after so many years, the core concepts remain the same and new products keep coming to address the automation aspect in different ways. For example - Intent based data input, Orchestration - We had this object called “Connectivity Service” which was used to tie the different Service Objects that form an end-to-end service, Network wide transactions, Rollbacks, Network Discovery, Service Discovery etc.

? Whether it’s configuration management tools like - Unimus, Nautobot?or Kubernetes based Network Automation tool like Kubenet, a Passive Inventory/Documentation tool like - NetBox?or monitoring tools like - SuzieQ, Kentik?or any other; they all help in Network Automation in one way or the other. The key to network automation success depends on how individuals/enterprises leverage these applications or any other, one at a time, to solve the problems they face on a day to day basis. There is no single solution that fits all or a specific product/solution to start with. Some start with deploying Assurance systems as they think that Customer Experience is key and then follow up with Inventory and Fulfillment. Some do the other way around. A lot also depends on whether you are a greenfield operator or brownfield.

? Building adapters during those times required access to vendor labs or shipping vendor equipment to our own labs. This was a very painstaking and time consuming process for the organization. Now a days we have a wide range of options to experiment and work with devices remotely - starting from Cisco Networking labs, Containerlab, netlab, eve-ng, GNS3?and many more. These labs give the flexibility to experiment at will and helps in the learning process.

? See you at Autocon2?in Denver and hope to have some good discussions on the future of Network Automation.