Introduction to Time Series Forecasting

There are many prediction problems that involve a time component such as forecasting some yield each year, forecasting some price each day, forecasting some rate each hour etc., which makes the problems more difficult to handle. This blog will introduce machine learning techniques to better analyze and predict time series.

Time Series

A time series can be decomposed into four constituent components: level (baseline value), trend (linear behavior), seasonality (the periodic behavior) and noise. According to the number of observations recorded at each time, the dataset can be marked as univariate time series and multivariate time series. We will focus on univariate time series in this blog, which is the simplest to understand and work with.

We can restructure the time series dataset as a supervised learning problem by using the value at the prior time step to predict the value at the next time step, which is called the sliding window method or lag method. The number of previous time steps is called the window width or size of the lag. According to the number of time steps ahead to be forecasted, there are one-step forecast and multi-step forecast.

ARIMA

ARIMA (Autoregressive Integrated Moving Average) model is a popular and widely used statistical method for time series analyzing and forecasting. ARIMA combines three parts: AR, Differencing (I), and MA. An AR (autoregressive) model is when a value from a time series is regressed on previous values from the same time series. The MA (moving average) model can be described as the weighted sum of noise of an observation and lagged observations. The differenced value is equal to the present value minus the value at the next lag. A time series which needs to be differenced to be made stationary is said to be an Integrated (I) time series. ARIMA has three main parameters: p stands for the number of AR terms or the order of AR model, d stands for the number of times differencing the data, and q stands for the number MA terms or the order of MA model. ARIMA can be extended considering seasonality and eXogenous variables, which is called SARIMAX model.

ACF (AutoCorrelation Function) and PACF (Partial AutoCorrelation Function) plots can give intuitive understanding of AR (p) and MA (q). The ACF plot for the AR(p) time series is strong to a lag of p and remain stagnant for subsequent lag values, trailing off at some point as the effect is weakened. The PACF describes the direct relationship between an observation and its lag with no correlation for lag values beyond p. The ACF for the MA(q) process shows a strong correlation with recent values up to the lag of q, then an immediate decline to minimal or no correlation. For the PACF, the plot shows a strong relationship to the lag and then a tailing off to no correlation from the lag onwards.

ARIMA model are well understood and effective on many problems. However, it suffers from some limitations as a linear method, such as nonlinearity, multivariate inputs and multi-step forecasts etc. Therefore, advanced time series forecasting methods have been developed to deal with these issues.