AIOps: Time series analysis – Forecasting

Balakrishnan Saravanan Kesavan, Upendra Sharma and Arun Ayachitula

Measures by Business Objectives (MBOs) in IT Service Management (ITSM) refer to specific key performance indicators (KPIs) that are aligned with the strategic goals of an organization. These measures are designed to track and assess the effectiveness of IT services in meeting the desired business outcomes. MBOs are often tailored to reflect the organization's priorities, such as improving customer satisfaction, reducing operational costs, or increasing service delivery efficiency. They enable organizations to make data-driven decisions and adjust and govern their IT processes and services to drive operational efficiencies.

Examples of Key MBOs:

[Note: The thresholds indicated in the above metrics change occasionally per governance policies.]

Uptime: This metric shows the percentage of IT devices with less than 95% uptime. The uptime for a server is measured over 90 days. Both the progress and projection lines should stay below the trendline.

Incident Ticket Budget: The incident ticket budget is based on how many incident tickets an account can accumulate before exceeding the 0.8 incident per server per month threshold. The total number of incident tickets is calculated by multiplying the number of servers by 0.8 (incident per server per month threshold) and evenly spreading it over the month. The progress line indicates the cumulative amount of incident tickets during the month and should stay below the incident ticket budget line.

Automation progress: This metric provides an overview of the impact of automated corrective actions on the total number of incident tickets. It is calculated by dividing automation-resolved incident tickets (Remediation with Corrective Closure) by all incident tickets. The progress and projection line should stay above the trendline of 35%.

Best Practices Deployment: This metric provides an overview of where there are deviations from the identified industry best practices. When Best Practices are not followed, there is a higher risk of issues and impact on business services. Both Best Practice progress and projection line should stay above the trendline of 90%. The “Must Fix” progress line indicates the percentage of essential deviations that must be addressed.

Forecasting Considerations for time series analysis

Time series analysis is a method for analyzing sequences of data points ordered chronologically. Fundamental techniques include ARIMA (p, d, q), a model incorporating Autoregressive (AR), Moving Average (MA), and Autoregressive Moving Average (ARMA) components. Time series can be decomposed into trends and seasonality with techniques like Exponential Smoothing (Simple, Double, Triple) and Dynamic Regression Models, which include external variables. Advanced forecasting methods such as Holt-Winters formulation (Triple Exponential Smoothing) and the Prophet model developed by Facebook illustrate a range of approaches for analyzing and predicting time series data.

The key considerations in choosing a forecasting technique includes:

• We need to train a model on the fly for each tenant/Client and each metric(so thousands of models)
• Should be able to train the model on the data rapidly
• Missing values are a high possibility in our use case, and the model should naturally handle it.
• Large seasonal patterns as we have daily data, so seasonality length could be as large as 365 or even weekly seasonality (seasonal length of ~52, actually non-integer)
• It should be suitable for various time series, i.e., those include holiday effects.
• Easy to interpret and easily tunable based on domain experience
• Trustworthy, which means easily explainable and automated comparison to detect the poor performance of a large number of models

Exponential Smoothing techniques cover critical concepts and equations related to exponential smoothing methods in time series analysis. It starts with Simple Exponential Smoothing, explaining its essence as a weight decay mechanism for older samples and the role of the α parameter. Double Exponential Smoothing, or Holt’s Method, is discussed, detailing its forecast, level, and trend equations. Triple Exponential Smoothing is touched upon, highlighting its decomposition into seasonal, trend-cycle, and remainder components. The document also evaluates the effectiveness of additive models over multiplicative ones in specific scenarios, like multiple seasonality or non-integer seasonality. It delves into the Holt-Winter’s Additive and Multiplicative Methods. Additionally, it explains various parameters like lt, bt, α, β*, γ, and their roles in these models.

Simple Exponential Smoothing:

Double Exponential Smoothing:

Triple Exponential Smoothing (Decomposable TS Model [Harvey amp; Peters, 1990])

Holt-Winter’s Additive and Multiplicative Methods?

The equations below are part of Holt-Winter's forecasting methods for time series data, which account for level, trend, and seasonality. The level equation, denoted as lt, is a weighted average of the current observation adjusted for seasonality and the sum of the last level and trend, aiming to smooth out the data's fluctuations. The trend equation, bt, smooths the series' trend component. The seasonal equation adjusts for seasonality by taking a weighted average of the current seasonality index and the seasonality of the same period in the previous cycle. The parameters α, ?β?, and γ, ranging between 0 and 1, are the smoothing parameters for the level, trend, and seasonal components. Parameter m represents the seasonal frequency, and k adjusts the forecasting horizon to the seasonal cycle. These equations combine to adjust the forecast based on the data's recent level, trend, and seasonal pattern.

Thumb rules for picking up ETS models

General Idea of Time Series Decomposition

Time series decomposition is an approach to model data by identifying and separating it into three distinct patterns. The 'trend' reflects a consistent long-term upward or downward movement in the data. 'Seasonality' captures regular fluctuations based on time intervals like days or months. 'Cyclic' patterns denote variations without a set frequency, often combined with trends to form a 'trend cycle'. Lastly, 'remainder' represents random, unpredictable noise in the data. This method is favored for its straightforward interpretability compared to ARIMA. Modern, efficient techniques, such as the Prophet model, offer enhanced estimation of these components.

Trend Identification: Recognizes long-term directional movements in data, indicating overall increase or decrease.

Seasonality Detection: Finds repetitive patterns at regular intervals like weekly, monthly, or quarterly.

Cyclic Analysis: Identifies fluctuations without a fixed pattern, often intertwined with a trend, termed a trend cycle.

Remainder Assessment: Isolates the 'noise' or random components the model cannot predict or explain.

Model Preference: This decomposition is favored for its simplicity and ease of explanation compared to ARIMA models.

Advanced Techniques: Utilizes more sophisticated and quicker methods, such as the Prophet model, for better estimation.

The Prophet Forecasting Model

The Prophet model is a forecasting tool designed by Facebook for handling time series data that features strong seasonal effects and several seasons of historical data. It works well with daily observations that display non-linear trends and can accommodate missing data and shifts in the trend. Prophet is robust to outliers, handles end-of-week and holiday effects, and provides a flexible framework that can be tuned with domain-specific knowledge. It's particularly user-friendly, requiring no detailed forecasting model background, making it accessible for analysts to produce high-quality forecasts.

Prophet’s growth function and seasonality

The Prophet model's growth function is depicted, showing two models: the logistic growth model, which captures growth with a natural saturation limit, and the constant growth-rate model for unfettered growth. The logistic model's growth is moderated by a carrying capacity C(t) and change points γj that adjust the growth rate over time. The constant growth-rate model linearly projects growth with adjustments at change points. Seasonality is modeled using a Fourier series, summing sinusoidal components to fit periodic changes, with coefficients and an and bn determining each seasonal component's amplitude and phase shift.

ARIMA forecasting models

ARIMA, which stands for Autoregressive Integrated Moving Average, is a class of statistical models for analyzing and forecasting time series data. It is especially useful for non-stationary data, where statistical properties such as the mean and variance change over time. An ARIMA model is characterized by three terms: p, d, and q:

• p: the number of lag observations included in the model, also called the lag order (autoregressive part).
• d: the number of times the raw observations are differenced, also called the degree of differencing (integrated part).
• q: the size of the moving average window, also called the order of moving average.
• The model aims to describe the autocorrelations in the data and can be fitted to time series data to better understand or predict future points in the series.

Acknowledgments

Thanks to Girish Mohite, Krishna Sumanth Gummadi, Subbareddy Paturu, Amar Mollakantalla, Murali Batthena, Vamshi Gummalla, Nitik Kumar, Kishore Maalae, Saravanan Kumarasamy, Divakar Reddy Doma, Rainy Moona, Muhammad Danish, Suryakanth Barathi, Shakuntala Prabhu, Pradeep Soundarajan, Hyder Khan, Godwin Dsouza, Prameela S, Vipin Sreedhar, Abhishek Gurav, Santosh Kumar Panigrahi, Diwakar Natarajan, Shivam Choudhary and Sander Plug for their contributions to AIOps development.