Out-of-sample forecasting challenges in time series data

Achieving "reliable" out-of-sample forecasts remains a formidable challenge, notwithstanding the significant strides made in forecasting methodologies. We delve into the nuances of forecasting out-of-sample data for housing starts in Canada, scrutinizing a time series that spans annually from 1948 to 2023.

This analysis juxtaposes two distinct methodologies: the traditional econometric approach, embodied by the Autoregressive Integrated Moving Average (ARIMA) models, and the more contemporary Prophet model, initially developed by Facebook (now Meta). The intention behind minimal adjustments to the modelling parameters is to underscore the inherent variability in results that can arise from different modelling choices. Both methodologies are applied to univariate time series data, deliberately excluding exogenous variables to maintain focus on forecasting beyond 2023 without the need to predict external inputs.

Utilizing RStudio for the analysis, we explore the ARIMA model through the Auto ARIMA function within the forecast package. This process automatically recommended an ARIMA(0,1,0) configuration, which, when visualized, yielded an out-of-sample forecast that closely resembled a straight line, underscoring a potential limitation in capturing the dynamics of the series.

Seeking a more nuanced forecast, we experimented further with the ARIMA framework, settling on an ARIMA(1,1,1) model augmented with a drift term. This model offered a slightly nuanced and, arguably, more "realistic" forecast compared to a straight, horizontal line extension, illustrating the potential benefits of incorporating a trend component in the model. The question remains: how does one capture the variance observed in the data in out-of-sample forecasts?

The investigation then shifted towards the Prophet model, a tool designed for intuitively handling seasonal trends and the ability to accommodate holidays and other special events easily. Initially, the default Prophet model settings did not yield insightful forecasts. However, after refining the model specifications, including adjustments for seasonality and trend components, the resultant forecasts for both in-sample and out-of-sample periods appeared considerably more plausible.

The wide confidence interval (shaded in blue) should not give one much confidence in the forecast.

These explorations into time series forecasting for housing starts in Canada highlight the critical role of model selection and parameterization and emphasize the inherent uncertainties associated with predicting future trends. We are particularly eager to engage with the broader community on this topic. We invite you to share your experiences with out-of-sample forecasting in time series data, including any insights on alternative tools or methodologies that have proven effective in your univariate time series forecasting endeavours.