Optimizing Forecasting Models with Root Mean Squared Logarithmic Error (RMSLE)

Forecasting demand accurately is crucial for industries like retail and grocery, where both stockouts and overstocks can lead to financial losses and customer dissatisfaction. A robust forecasting model should balance these challenges while minimizing critical forecasting errors. In this article, we explore how Root Mean Squared Logarithmic Error (RMSLE) can enhance demand forecasting models and how to align models and loss functions with this evaluation metric.

---

What is RMSLE and Why Use It?

RMSLE measures the logarithmic difference between predicted and actual values, focusing on the relative error rather than absolute deviations. It is especially effective in scenarios where:

1. Larger errors are more acceptable for high-demand products.

2. Smaller errors matter more for low-demand items, where even minor shortages can disrupt operations.

3. Under-predictions are penalized more heavily than over-predictions, making it ideal for inventory-sensitive sectors like grocery retail.

The RMSLE formula is:

\[

RMSLE = \sqrt{ \frac{1}{n} \sum_{i=1}^{n} \left( \log(1 + \hat{y}_i) - \log(1 + y_i) \right)^2 }

\]

Building a Forecasting Model Optimized for RMSLE

1. Model Architecture: LSTM + Prophet Ensemble

We leverage Long Short-Term Memory (LSTM) networks, which excel in time-series prediction, and Prophet, a model from Meta, designed to capture trends and seasonality. Using an ensemble approach combining these two models provides more accurate forecasts, accounting for both time-series dynamics and external events like holidays or promotions.

---

2. Custom RMSLE Loss Function in LSTM

Since traditional loss functions (like Mean Squared Error) may not align with RMSLE, we replace them with a custom RMSLE loss function in the LSTM model. Below is the implementation in Python using TensorFlow:

```python

import tensorflow as tf

def rmsle_loss(y_true, y_pred):

# Add 1 to avoid log(0) errors

y_true_log = tf.math.log1p(y_true)

y_pred_log = tf.math.log1p(y_pred)

# Calculate the squared difference

square_diff = tf.square(y_true_log - y_pred_log)

# Compute mean and return the square root

return tf.sqrt(tf.reduce_mean(square_diff))

```

This function ensures the model minimizes errors in a way that aligns with the RMSLE metric, focusing more on reducing critical under-predictions.

---

3. Logarithmic Data Transformation for Alignment

To further align the model’s predictions with RMSLE, we apply a logarithmic transformation to the sales data during preprocessing:

```python

import numpy as np

# Apply log transformation to the sales data

log_sales = np.log1p(sales_data)

```

After predictions, we apply the inverse transformation to return the values to their original scale:

```python

# Inverse log transformation for predicted sales

predicted_sales = np.expm1(log_predictions)

```

This ensures the model works in a space where differences in small sales volumes are emphasized, preventing stockouts of crucial items.

---

Bias Correction to Handle Stockouts and Overstocks

Given that RMSLE penalizes under-prediction more than over-prediction, we introduce a bias correction layer in the LSTM model to slightly favor overestimating demand. This bias helps avoid stockouts for essential products, ensuring high customer satisfaction.

---

## Integrating Reinforcement Learning for Order Optimization

Beyond forecasting, we integrate a Reinforcement Learning (RL) agent to optimize order quantities. The RL agent learns to balance stockouts and overstocks by adjusting orders based on forecasted demand.

- Reward Function: The reward is structured to minimize both stockouts (which harm customer satisfaction) and overstocks (which lead to waste).

- Adaptive Learning: The RL agent adapts to demand shifts, reducing penalties from under-predictions.

---

Model Evaluation Workflow Using RMSLE

We follow a structured process to ensure our models are aligned with the RMSLE metric:

1. Training the LSTM model using the custom RMSLE loss function.

2. Generating forecasts for each product and store over a specified time horizon.

3. Applying inverse transformations to convert predictions back to their original scale.

4. Calculating RMSLE on the validation data to evaluate model performance:

```python

def rmsle(y_true, y_pred):

return np.sqrt(np.mean(np.square(np.log1p(y_pred) - np.log1p(y_true))))

```

This evaluation ensures that the model’s performance meets real-world expectations, with minimal forecasting errors for critical low-demand products.

---

Continuous Model Improvement and Real-Time Adaptation

To maintain forecasting accuracy, the model will be continuously retrained with real-time data. This allows the system to adapt to sudden changes, such as promotions, weather changes, or holidays, ensuring optimal stock levels.

---

Conclusion: Enhancing Forecasting Accuracy with RMSLE

By using RMSLE as the primary evaluation metric, the forecasting model becomes more aligned with real-world needs, especially in industries like grocery retail. The combination of custom loss functions, logarithmic transformations, bias correction layers, and reinforcement learning ensures that the system performs well under real-world conditions.

This approach not only helps businesses minimize waste and avoid stockouts but also enhances customer satisfaction by ensuring product availability. In a competitive landscape, such advanced forecasting models can provide a significant operational advantage.