N-BEATS: The Unique Interpretable Deep Learning Model for Time Series Forecasting

Introduction

In various sectors such as finance, retail, and meteorology, time series forecasting is pivotal. Traditional models often grapple with flexibility and interpretability, particularly when dealing with complex patterns. Enter N-BEATS, a groundbreaking model developed by researchers at Element AI and the Montreal Institute for Learning Algorithms (MILA). Renowned for its interpretability and robust forecasting capabilities, N-BEATS is a true game-changer.

What is N-BEATS?

N-BEATS (Neural Basis Expansion Analysis for Time Series Forecasting) revolutionizes the approach to time series predictions. Distinct from typical models that depend on recurrent neural networks (RNNs), N-BEATS employs a series of feed-forward neural networks. This structure not only enhances performance but also circumvents the complexities and instabilities often associated with RNNs.

Key Features of N-BEATS

• Interpretability: A major highlight of N-BEATS is its interpretability, offering insights into which data components influence predictions — a rare feature in deep learning models.
• Modularity: The architecture includes multiple blocks that can be configured in various ways to suit different applications, allowing extensive customization without altering the core framework.
• Generalization: Designed to handle a diverse array of time series data, N-BEATS can adapt without needing specific adjustments for different datasets.

Architecture Deep Dive

• Block Structure: At its core, N-BEATS comprises blocks each tasked with capturing specific data patterns like trends or seasonalities. These blocks forecast and predict backcast values, helping the model focus on different aspects of the time series.
• Stacking Blocks: The strength of N-BEATS lies in stacking these blocks. Each block layer refines the forecast by addressing the residuals — the differences between actual values and previous predictions. This refinement enhances the accuracy of the final forecast.

How N-BEATS Works

• Training: N-BEATS trains by alternating between forecasting future values and reconstructing past values (backcasts). It minimizes the error between its predictions and actual data, sharpening its forecasting ability.
• Forecasting: In forecasting, N-BEATS aggregates predictions from all blocks, ensuring that each data aspect, from general trends to specific seasonal patterns, is considered in the final prediction.

Example: Daily Temperature Data

To demonstrate N-BEATS, let’s use the daily temperature dataset, a common benchmark in time series modeling. It’s a sine wave with a period of one year (365 days) and an amplitude of 10. This example will show how to implement N-BEATS using PyTorch and interpret the model’s outputs to understand the influence of different time series components.

Lets explore the main parts for the exercise: (Link to the full code is in the reference section)

# Step 1: Function to create a synthetic temperature dataset
def create_temperature_dataset(length, num_samples):
    np.random.seed(0)
    x = np.linspace(0, length, num_samples)
    seasonal = 10 + 10 * np.sin(2 * np.pi * x / 365)
    noise = np.random.normal(0, 2, num_samples)
    y = seasonal + noise
    return x, y

# Create dataset
x, y = create_temperature_dataset(365, 3650)

# Plot the dataset        

# Step 2: Define a PyTorch Dataset
class TimeSeriesDataset(Dataset):
    def __init__(self, data, backcast_length, forecast_length):
        self.data = data
        self.backcast_length = backcast_length
        self.forecast_length = forecast_length

    def __len__(self):
        return len(self.data) - self.backcast_length - self.forecast_length

    def __getitem__(self, index):
        x = self.data[index : index + self.backcast_length]
        y = self.data[index + self.backcast_length : index + self.backcast_length + self.forecast_length]
        return torch.tensor(x, dtype=torch.float32), torch.tensor(y, dtype=torch.float32)

# Parameters
backcast_length = 30
forecast_length = 7

# Create dataset
dataset = TimeSeriesDataset(y, backcast_length, forecast_length)
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)        

The TimeSeriesDataset class in PyTorch is designed to handle time series data by leveraging backcasting and forecasting techniques. It requires three parameters for initialization: data (the time series data), backcast_length(the number of past time steps to consider), and forecast_length (the number of future time steps to predict). The class has methods to return the length of the dataset and to fetch data samples, converting slices of the time series into tensors. In this example, if you set backcast_length to 30 and forecast_length to 7, you can create an instance of the dataset and a DataLoader to handle batching and shuffling of the data as follows: dataset = TimeSeriesDataset(y, backcast_length, forecast_length) and dataloader = DataLoader(dataset, batch_size=32, shuffle=True). This setup facilitates the training of models by providing efficient data handling.

# Step 3: Define and Train the N-BEATS Model
device = 'cuda' if torch.cuda.is_available() else 'cpu'

# Define N-BEATS model
model = NBeatsNet(
    device=device,
    stack_types=(NBeatsNet.GENERIC_BLOCK, NBeatsNet.GENERIC_BLOCK),
    forecast_length=forecast_length,
    backcast_length=backcast_length,
    hidden_layer_units=128
).to(device)

# Define loss and optimizer
criterion = torch.nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters())

# Training loop
epochs = 50

for epoch in range(epochs):
    for x_batch, y_batch in dataloader:
        x_batch, y_batch = x_batch.to(device), y_batch.to(device)
        optimizer.zero_grad()
        backcast, forecast = model(x_batch)
        loss = criterion(forecast, y_batch)
        loss.backward()
        optimizer.step()

    if (epoch + 1) % 10 == 0:
        print(f'Epoch {epoch + 1}/{epochs}, Loss: {loss.item()}')        

The N-BEATS model uses a series of fully connected layers organized into blocks, each with sub-networks for backcasting and forecasting. Configured with parameters like backcast and forecast lengths and the number of hidden units, the model captures complex data patterns.

The loss function is set to mean squared error (MSE), and the Adam optimizer adjusts the model’s parameters to minimize this loss. Training involves iterating over the dataset for several epochs, processing data batches, computing forecasts, and updating parameters via backpropagation. Progress is monitored by checking loss values to ensure effective learning.

# Step 4: Visualize Backcast, Forecast, and Model Weights
model.eval()
x_batch, y_batch = next(iter(dataloader))
x_batch, y_batch = x_batch.to(device), y_batch.to(device)

with torch.no_grad():
    backcast, forecast = model(x_batch)

# Convert to numpy for plotting
backcast = backcast.cpu().numpy()
forecast = forecast.cpu().numpy()
x_batch = x_batch.cpu().numpy()
y_batch = y_batch.cpu().numpy()

# Plot results
Redracted

# Visualize model weights
weights = [param.cpu().data.numpy() for param in model.parameters()]

# Plot weights of the first few layers for illustration
Redracted        

Visualize Backcast, Forecast, and Model Weights

To visualize the backcast, forecast, and model weights, set the model to evaluation mode and obtain a batch of data from the DataLoader. Use this batch to compute the backcast and forecast without updating model parameters. Convert these results to NumPy arrays for plotting.

Create plots to show the backcast input, actual forecast, predicted backcast, and forecast. Use matplotlib to display these results, showing the time series and predictions.

The plot visualizes the N-BEATS model’s performance.

• The blue line represents the backcast input, which is the historical data used by the model to make predictions.
• The orange line indicates the actual forecast, the real future values we want to predict.
• The green line shows the model’s backcast, representing how well the model reconstructs past data points.
• The red line depicts the model’s forecast, the predicted future values. Comparing the actual forecast with the model’s forecast helps assess the model’s accuracy and effectiveness in predicting future trends based on historical data.

Visualize model weights

Next, extract and visualize the model weights. Convert the model parameters to NumPy arrays and plot the weights of the first few layers for illustration. This helps in understanding the model’s learned features and patterns. The visualizations of the N-BEATS model’s weights reveal the following:

1. Layer 1 Weights: This layer exhibits weights ranging from -0.4 to 0.4, with a dense distribution and numerous small fluctuations, indicating it captures detailed patterns in the data.
2. Layer 2 Weights: These weights have a similar range but are more sparsely distributed, suggesting this layer identifies broader trends and variations.
3. Layer 3 Weights: The weights in this layer range from -0.4 to 0.4, densely packed like in Layer 1, highlighting the layer’s role in capturing intricate details.
4. Layer 4 Weights: This layer shows weights ranging from -0.10 to 0.15, with more pronounced fluctuations, indicating it captures different patterns and features compared to the previous layers.

Key Hyper-parameters in the N-BEATS Example

Your model will always not give you the best performance, which you can benchmark with metrics such as MSE, MAE. Hyper parameter tuning is a great way to adapt your model to your particular use case. Here are the common parameter which can be tuned for optimal performance.

1. Learning Rate

• Description: This controls how quickly the model adjusts its weights in response to the error it sees. A higher learning rate makes larger adjustments, while a lower rate makes finer adjustments.
• Typical Use: Start with a common value like 0.001 and adjust based on performance. If the model trains too slowly or gets stuck, consider increasing the learning rate. If the model’s training loss fluctuates widely or it learns too fast (which can skip optimal solutions), lower the learning rate.

2. Number of Epochs

• Description: This refers to how many times the model will see the entire dataset. More epochs mean the model has more opportunities to learn and adjust.
• Typical Use: If the model is underfitting, try increasing the number of epochs. If it’s overfitting, reduce the number of epochs or employ techniques like early stopping to halt training when the validation performance degrades.

3. Batch Size

• Description: This determines how many data points the model sees before it updates its weights. Large batch sizes can lead to faster computation but might impact the generalization ability of the model.
• Typical Use: A smaller batch size often provides a more robust convergence, at the cost of increased computation time. If your model’s performance is too variable between epochs, consider reducing the batch size to improve stability.

4. Number of Layers and Neurons per Layer

• Description: In your model, this relates to the fc1 and fc2 layers' configurations. These layers’ size and number determine the model’s capacity to learn complex patterns.
• Typical Use: More neurons and layers can model more complex functions but can also lead to overfitting. Start with simpler architectures and increase complexity as needed. If the model is too simple to capture the trend or seasonality in the data, increment the number of neurons.

5. Network Architecture Adjustments

• Description: The choice of activation function can affect how well the network models non-linear relationships. In your example, ReLU is used, which is a common choice for many tasks.
• Typical Use: ReLU works well in many situations due to its simplicity and efficiency. However, if you suspect that your model suffers from dying ReLU problems (where neurons consistently output zeros), consider variants like Leaky ReLU or ELU.

# Define hyperparameter grid
hyperparams = {
    'backcast_length': [15, 30, 60],
    'forecast_length': [5, 7, 10],
    'hidden_layer_units': [64, 128, 256],
    'num_blocks': [2, 3, 4],
    'learning_rate': [0.001, 0.01, 0.1],
    'batch_size': [16, 32, 64],
    'epochs': [50, 100, 200]
}        

Conclusion

N-BEATS is more than just a forecasting tool; it’s a robust, interpretable framework that democratizes deep learning for time series analysis. Its capability to provide clear insights into what drives forecasts makes it invaluable for businesses reliant on accurate, explainable predictions.

Call to Action

For those looking to push the boundaries of time series forecasting, implementing N-BEATS provides a perfect blend of performance and interpretability, transforming decision-making processes. Try applying N-BEATS to your data and delve into the depth of insights it can offer.

Helpful Resources

• The Full Code can be accessed here: Google Collab N-BEAT
• Explore the original research paper for a deeper dive into N-BEATS.
• Check out an implementation of N-BEATS on GitHub to start experimenting with the model.
• Further reading on time series forecasting can expand your understanding and application of these models in real-world scenarios.

This article bridges the gap between complex theoretical models and practical, actionable insights, helping you harness the power of N-BEATS in your forecasting endeavors.