Dos and Don’ts with NOC Automation
Yet, despite all the challenges, it is doable to automate big portion of the business processes by paying attention to several key points. I believe that even if smaller parts would after all be automated, still greater efficiency can be gained just by detecting anomalies in the current process as an outcome of the process analysis, learning the lessons and understanding that it’s not just technology but a mindset, a working model of an organization that can change.
Don’t take written procedures for granted
Start by mapping the today’s workflows. Don’t just look at the cookbooks and instructions that your team formally follow but look at it, literally, visit NOC, accompany field engineers. You would be surprised to find how the information they use, the decisions they make, the time that it takes them, are different than planned. Such visits repay themselves in gold when results influence investments as they are found to be either simpler, shorter, a fault of a vendor, or not disciplined. Sometimes short interviews to holders in the process chain can gain that results.
Secure your business first, then make it more efficient
The parts to be prioritized for automation are not necessarily the heaviest time and budget consumers but those with most impact on the business, namely the most customer facing ones. Naturally these are the most dynamic areas with frequent changes in the process and that makes their automation even more complex but with the highest ROI.
A good automation design should be flexible enough to minimize the changes overhead and not be strictly tailored to the specific vendors. Some examples can be with the use of standard rule engine where policies can be managed and are out to be easily maintained.
Other parts that are more static may be optimized as well but many times slight improvements can be good enough.
Make information available
One of the barriers in business processes is the access to information. Major part of the processes that cross departments are around the same data sets (e.g. service details, resource state) that kept in different formats and accuracy level. Many of the organizations are built around specific expertise and tools where all what they do is to retrieve data, process it and take decisions. Making the data available and in the same model and convention is the first act to ease further automation steps and derive changes in the organization.
Making the data accessed to all, via BI or common APIs is a key and inevitably costly part of automation project. Definitely its infrastructure. There is no tool on earth that can fit at first and is always a long-time project with tiring integration process.
With giving the right attention and priorities, data management project can be managed in phases, it can starts with small and most critical data and then be leveraged to more sources. Make sure to have the design of other parts considered, even when first phase is considerably small.
Leave the decision points to later stage
One of the concerns we discussed in my last post was the lack of confidence in automation tools where wrong, automatic decisions, may take the network or service down. A way to gradually get this confidence is to distinguish between trivial actions, like information retrieval or routine configuration, to making decisions like starting OAM test or optimizing service path. Let the automation environment inform the NOC or planning team on the change to be done, the information that led to the decision and to get their approval. At the bottom line the time and complexity were saved and user can anytime block the operation, or check manually before approval. One of the reasons to work with workflow or BPM tools is their inherent human intervention as part of the process (here is another point to remember when bench-marking such tools, get to notify user by various ways, not only emails)
Improve communication
Tons of words were spent about the inefficiency of Email in organizations (here is one example), and still email is daily in use for troubleshooting, managers approval, information request and more. Unlike other topics, adopting better ways like Slack or any IM that combines voice, text and attachments has reasonable costs and comparably short learning curve. The best way is to integrate IM and notes in the working environment and make messages visible to all stakeholders, which can shorten dramatically the response and acting time (“hey all, I’m on this ticket. Steve”), rather than unorganized, uncategorized inbox. And I didn’t mention at all telephone calls, yes, they are still there in certain cases as the only way to communicate with field engineers.
Work with your vendors
Almost every aspect we discussed about automation requires work with the equipment and software (NMS, OSS, BSS) vendors. They are the partners to support data access, work with APIs, and above all the awareness to the integration and the use of interfaces (luckily when all is there, documented and self-contained). Remember that your processes, after transformation, should still be sustainable and with minimal maintenance costs. Hence every potential change and risk should be planned in advance. A vendor may declare end-of-life to certain products and may deprecate interfaces in a short notice. To be more proactive when approaching them, they may even have better ways to simplify your processes. Remember that an atomic function by the vendor can save higher costs by reducing the number of APIs you need to sync with its NBI (and avoid dealing with sync/async, response time, etc).
There is a wider aspect of whether to adopt an automation solution by one of the equipment vendors, claiming to be multi-vendor, taking external BPM/BPA, or approach the OSS vendor, all compared to an in-house project. This is a full topic that worth a post by its own.