Root Cause Analysis in a GPON Network Alarm Manager

Root Cause Analysis (RCA) is a systematic approach for identifying the core causes of a system fault, and organizations can take effective actions to address these problems and prevent recurrence by revealing these root causes. As traditional methods have limitations, innovative approaches are essential. Through data mining, this paper enables organizations to efficiently analyze alarm data, enhance RCA, and ensure continuous service. Leveraging data mining can create more resilient systems that proactively tackle issues and maintain uninterrupted operations.

Keep reading to learn more about Root Cause Analysis and Altice Labs’ solution to revolutionize its application in complex systems or download the full white paper.

RCA is vital for system reliability and is essential for uncovering and addressing system faults in complex environments like fiber optic networks and large-scale industries, but conventional RCA has trouble handling data volume and complexities, which results in time-consuming and subjective analysis. To solve this issue, this paper examines data mining techniques to increase RCA accuracy and efficiency, aiming to develop a data mining-based model that autonomously identifies and clusters alarm floods (Afs), which simplifies analysis. The model automates alarm data analysis, detects recurring patterns, and prioritizes critical issues, all through advanced algorithms. This study will explore the potential of data mining to revolutionize RCA in complex systems.?

Alarm Manager

Before diving into data mining techniques, it's important to introduce the alarm manager’s central role in this process. Developed by Altice Labs, Alarm Manager is the RCA tool in use in this case study and is partly responsible for generating and storing alarm instances related to equipment failures for customers. Therefore, understanding its functionality and data storage is vital for grasping project details.

Alarm management involves monitoring, analyzing, and responding to alarms from various systems. The goal is to handle alarms effectively and reliably to prompt appropriate actions during abnormal situations.

AGORA from Altice Labs serves as a comprehensive Fault Management tool. It handles alarm reception, storage, treatment, and correlation. Through diverse collection channels, this central hub manages alarm parameters, cycles, and processing and it streamlines fault monitoring, detection, and ticket initiation by drawing on telecom insights, increasing operational efficiency and adaptability.

AGORA delivers a user interface with a more intuitive and practical real-time visualization of active alarms in the network, and it enables users to create reports on the historical record of occurred alarms, which enhances accessibility and usability.

Thanks to Alarm Manager, organizations can quickly adapt to changing needs, improving operations and agility.

Data Mining

Alarm data mining involves extracting valuable insights and patterns from alarm data across several industries such as manufacturing, telecommunications, and energy. By analyzing large alarm datasets, alarm data mining seeks to reveal patterns, trends, anomalies, and correlations. This approach attempts to improve the comprehension of alarm behavior, optimize alarm settings, and increase the efficiency of alarm management systems. Organizations can get information on system behavior, streamline operations, and enhance decision-making regarding alarm management by using data mining.

The objectives of alarm data mining include alarm optimization; fault detection and diagnosis; performance enhancement; and predictive analytics. Pursuing these goals empowers organizations to optimize their alarm management systems and enables operators to effectively manage alarms in intricate network environments.

Multiple techniques are employed in data mining, including statistical analysis, data visualization, machine learning, pattern similarity, and pattern mining algorithms. In this study, the focus is on pattern matching of AF sequences, using the Smith-Waterman Algorithm for a more in-depth analysis. This approach promises to enhance the understanding of Afs, contributing to more effective root cause analysis and system optimization.

Pattern Matching

Pattern matching involves recognizing and analyzing repeating patterns or alarm sequences within a system or network. Alarmistic data mining aims to reveal important connections and correlations from extensive alarm data produced by complex systems such as telecommunication networks, industrial operations, or fiber networks.

When it comes to alarmistic data mining, pattern matching includes the search for specific alarm sequences that might represent systemic problems or origins of issues within the system. This empowers network operators and makes them better equipped to understand system behavior and identify potential anomalies or irregular states.

Different methods for pattern matching have been extensively employed in the analysis of AFs similarities. Among these methods, the Dynamic Time Warping (DTW) algorithm is notable, as it enables flexible alignment of sequences, accommodating time variations and distortions, rendering it suitable for time-series data such as alarm sequences. The Smith-Waterman algorithm is also notable, as it handles the temporal aspect of Afs and it computes the similarity score between two alarm sequences, considering both the sequence order and their temporal relationships. These methods, among others, contribute to further identifying recurring alarm patterns and uncovering latent patterns in large-scale alarm datasets.

Similarity of Alarm Floods

The challenge in AF pattern matching involves assessing the similarity between segments to categorize floods. An AF is a sequence of alarms within a time frame, and the goal is to find shared patterns among them for differentiation.

Through a similarity index, we quantify the likeness between the sequences. If floods have substantial shared alarms or similar patterns, they’re grouped. These common segments act as distinguishing features to classify new floods. This process enhances analysis and decision-making.

Modified Smith-Waterman Algorithm

The Smith-Waterman algorithm identifies similar pairs within two sequences, initially for molecular sequences but now used in analyzing industrial alarm sequences.

This algorithm focuses on local alignment and its limitation lies in disregarding the temporal order of alarms, which can lead to inaccurate results. To overcome this, a modified version for alarmistic data mining can consider temporal order, aligning alarms based on timestamps for accurate matching of similar alarms.

This modified approach involves pre-processing to include time information, accommodating variations in reporting order, and identifying alarm pairs with significant temporal differences, enhancing the understanding of alarm patterns.

Data from Altice Labs’ alarm manager is used. A 7-day dataset with over 5.5 million alarms is employed to assess the algorithm’s real-world effectiveness. Pre-processing techniques are crucial to efficiently manage this substantial data volume.

Pre-processing

In data analysis and machine learning, pre-processing is vital for accurate results. It converts raw data for algorithms, including feature selection, addressing chattering, and arranging data into temporal sequences.

Feature selection identifies key features, reducing complexity and enhancing model performance. This study focuses on alarm correlations, using location, timing, and nature of the problem for a multidimensional analysis, which enhances precise issue identification.

Chattering, caused by errors, disrupts accuracy. Techniques like smoothing improve real-time RCA, crucial for applications requiring prompt responses and removing 2.7 million repetitive alarms in the analyzed dataset.

Data is organized into time-based segments by the ISA18.2 standard, enhancing predictive power. Alarms are grouped into AFs for focused analysis of related alarm sequences, enhancing issue understanding.

Proposed Algorithm

This innovative approach extends the principles of the Smith-Waterman Algorithm by including temporal information. This interprets alarm sequences more precisely. For instance, when two alarms occur closely, this method treats them interchangeably, accommodating variations in reporting order. It also identifies less correlated alarms with significant time gaps. This enriches insights into alarm patterns and relations, enhancing complex system alarm management accuracy.

To gain a better understanding of the algorithm, download the full white paper.

Post-processing

After calculating similarity indexes among AFs, post-processing and hierarchical clustering are used to refine, enhance, and transform data output. This technique groups similar AFs, enabling deeper insights.

Hierarchical clustering organizes data into clusters based on similarity, facilitating pattern identification. The process transforms similarity values into a distance matrix. Initially, each AF forms its own cluster, and clusters with the smallest distance are merged until a defined maximum distance is reached.

The Complete-linkage method determines cluster distances based on the largest distance within the merged cluster. Clusters maintain high similarity which ensures coherent patterns.

Archetype AFs are selected to represent clusters, reducing computational load. The medoid method is used for complex cases. This results in 25 distinct clusters, each representing various problem types with unique root causes. This showcases the model’s ability to identify diverse issues, helping network operators to efficiently diagnose and resolve problems.?

Test Model

After selecting archetype AFs from training clusters, the next phase is to analyze AFs within a validation dataset. Instead of comparing each AF with all others, they are matched against the training-generated archetypes using the proposed similarity algorithm.

Our observations show that 542 AFs in the test dataset share a similarity index of 0.6 or higher with at least one cluster archetype, covering 71.3% of AFs tied to known clusters. This means that a significant portion of the new dataset already includes alarm sequences represented by archetypes. As a result, these AFs can be recognized as familiar, which allows network operators to focus on deciphering the root causes of the remaining unfamiliar patterns, streamlining resource allocation, and root cause identification.

Discover more about the proposed algorithm and conclusions on our website.

Authors

• Lucas Leite Baptista, Altice Labs

Keywords:?Data Mining, Pattern Matching, Alarm Floods, Similarity Analysis

Contact us?if you want to engage in a deeper discussion on this topic!