Unlocking Tomorrow's Trends Today: MOIRAI Pioneers Precise Time-Series Forecasting!

?

Moirai (Masked EncOder-based UnIveRsAl TIme Series Forecasting Transformer) is a foundational ?probabilistic forecasting model for time series forecasting developed by Salesforce. It is designed as a universal model capable of predicting a wide range of time series. To achieve this flexibility, the model addresses several challenges associated with time series data, including the ability to:

·?????Handle all kinds of data frequencies (hourly, daily, weekly, etc);

·?????Accommodate any number and types of covariates (ex include factors like holidays, special events, and economic indicators), whether they are unknown in the future or known;

·?????Generate a probabilistic forecast using a flexible distribution that can be adapted to several cases.

?

This model has three main novel concepts: Multi Patch Size Projection Layers, Any-Variate Attention, and Mixture Distribution which makes this model unique.

Multi Patch Size Projection Layers

Patching?serves to divide the time series data into patches of size?P, which are shorter subsets of the original series.

Any-Variate Attention

Any-Variate Attention to allow Moirai to process multiple sequences. This is possible due to two distinct approaches:

1.???? Rotary Positional Embeddings (RoPE) [8]?ensures permutation equivariance by how its encoding works. It encodes the positional information by rotating the representation of tokens in the embedding space.

2.???? Binary attention bias?allows the model to be invariant — treating the variates as unordered.?

Mixture Distribution

Moirai is a probabilistic forecasting model, which means it learns the parameters of a distribution rather than merely providing a single point prediction. The output, being a distribution, enables decision-makers to evaluate the uncertainty of the predictions, as wider intervals indicate greater uncertainty from the model.

?For more details plz read: “Unified Training of Universal Time Series Forecasting Transformers” by Gerald Woo, Chenghao Liu, Akshat Kumar, ?Caiming Xiong, Silvio Savarese, Doyen Sahoo

?

For more details plz read: "Unified Training of Universal Time Series Forecasting Transformers" by Gerald Woo, Chenghao Liu, Akshat Kumar, ?Caiming Xiong, Silvio Savarese, Doyen Sahoo