The time series analysis algorithm has not been updated for a while, and the traditional time series forecasting algorithm is almost coming to an end. In the order of presentation of our series of articles, there are four more algorithms not mentioned:

Stationary time series forecasting algorithms are all big and difficult to explain. However, if you read this series of articles from the beginning to the end and carefully study the research, you will find that the time series forecasting algorithm is doing one thing from beginning to end, that is, how to make better use of historical data and mine the historical data contained in it. A cyclical pattern or trend. After reading the above articles in this series, it is not difficult to understand the stationary time series forecasting algorithm. The real difficulty is the principle and derivation, as well as the practical application method, which is very important. The key is how to understand such algorithms, so let's start step by step to complete the derivation of this algorithm and the application of code writing to practical cases.

Bloggers will maintain blog posts for a long time. If you have any mistakes or doubts, you can point them out in the comment area. Thank you for your support.

1. Moving Average Model (MA)

The moving average model MA is somewhat related to the autoregressive model AR. It is not a linear combination of historical time series values but a linear combination of historical white noise. The biggest difference from AR is that the impact of historical white noise in AR models indirectly affects current forecast values (by affecting historical time series values). The AR model simplifies the random part and retains the historical sequence part, then dually, the MA model simplifies the historical sequence value and retains the random part.

By comparing the principles and formulas with the AR model, we can better understand the MA model.

Model principle

According to the basic content of our previous article and the explanation of the AR model: time series analysis algorithm for stationary time series prediction algorithm and autoregressive model (AR) detailed explanation + Python code implementation

autoregressive

Autoregression is only suitable for predicting phenomena related to its own earlier period. The mathematical model is expressed as follows:

where is the current value, is the constant term, is the order, is the autocorrelation coefficient, is the error, and at the same time

to conform to a normal distribution. The model reflects that there is a linear relationship between the target value at time t and the previous t-1~p target values, namely:

moving average model

The moving average model (MA) is concerned with the accumulation of error terms in the autoregressive model. The mathematical model is expressed as follows:

The model reflects that there is a linear relationship between the target value at time t and the first t-1~p error values, namely:

Finding the expected variance of both sides, we get:

MA models are always weakly stationary. In particular, when = 0, it becomes a centralized MA(q) model .

autocorrelation coefficient

The autocorrelation coefficient of the MA model is truncated to order q.

Autocorrelation coefficients of commonly used MA models

Universal:

MA(1) model :

MA(2) model :

autocovariance function

Second, Python case implementation

Stationary Time Series Modeling Steps

According to the properties of the autocorrelation , a suitable model is selected.

The properties of the autocorrelation and partial autocorrelation coefficients of the AR, MA and ARMA models are as follows:

	ON(p)	MA(q)	ARMA(p,q)
model equation	$\varphi (B)=a_{t}$	$z_{t}=\Theta (B)a_{t}$	$\varphi (B)z_{t}=\Theta (B)a_{t}$
Stationarity Condition	$\varphi (B)=0$ The root of is outside the unit circle	none	$\varphi (B)=0$ The root of is outside the unit circle
reversibility condition	none	$\Theta (B)=0$ The root of is outside the unit circle	$\Theta (B)=0$ The root of is outside the unit circle
autocorrelation function	trailing	Q-step truncated	trailing
Partial correlation function	P-step truncated	trailing	trailing

Stationarity Test

Unit root test (Unit Root Test) Unit root test is a special method of stationarity test proposed for macroeconomic data series, monetary and financial data series with certain statistical characteristics. There are many methods for unit root test. Including ADF test, PP test, NP test, etc.

#导入包
from statsmodels.tsa.stattools import adfuller

def test_stationarity(timeseries):
    #Determing rolling statistics 
    rolmean=timeseries.rolling(7).mean()
    rolstd=timeseries.rolling(7).std()
    #Plot rolling statistics:
    orig=plt.plot(timeseries,color='blue',label='Original') 
    mean=plt.plot(rolmean,color='red',label='Rolling Mean') #均值
    std=plt.plot(rolstd,color='black',label='Rolling Std') #标准差
    plt.legend(loc='best')
    plt.title('Rolling Mean & Standard Deviation')
    plt.show(block=False)
    
    #Perform Dickey-Fuller Test:
    print('Results of Dickey-Fuller Test:')
    dftest=adfuller(timeseries,autolag='AIC')
    dfoutput=pd.Series(dftest[0:4],index=['Test Statistic','p-value','#Lags Used','Number of Observations Used'])
    for key,value in dftest[4].items():
        dfoutput['Critical Value (%s)'%key]=value 
    print(dfoutput)

#检验结果
test_stationarity(df_test['客流量'])

Results of Dickey-Fuller Test:
Test Statistic                  -4.608773
p-value                          0.000124
#Lags Used                      15.000000
Number of Observations Used    483.000000
Critical Value (1%)             -3.443962
Critical Value (5%)             -2.867543
Critical Value (10%)            -2.569967
dtype: float64

Output content analysis:

ADF: float, test statistics.
pvalue: float, probability value: MacKinnon's approximate p-value based on MacKinnon (1994, 2010).
usedlag: int, the number of lags used.
NOBS: int, number of observations to use for ADF regression and calculation of critical values.
critical values: dict, critical values for test statistics at 1%, 5% and 10%. Based on MacKinnon (2010).
icbest: float, maximizes the information criterion if autolag is not None.
resstore: ResultStore, optional, a dummy class whose result is attached as a property

Returns
    -------
    adf : float
        The test statistic.
    pvalue : float
        MacKinnon"s approximate p-value based on MacKinnon (1994, 2010).
    usedlag : int
        The number of lags used.
    nobs : int
        The number of observations used for the ADF regression and calculation
        of the critical values.
    critical values : dict
        Critical values for the test statistic at the 1 %, 5 %, and 10 %
        levels. Based on MacKinnon (2010).
    icbest : float
        The maximized information criterion if autolag is not None.
    resstore : ResultStore, optional
        A dummy class with results attached as attributes.

Additional instructions:

The model requires that the series must be stationary time series data. By analyzing the t value, analyze whether it can significantly reject the null hypothesis that the series is not stationary
If it is significant (p<0.05 or 0.01), it means that the null hypothesis is rejected, and the series is a stationary time series; otherwise, it means that the series is an unstable time series
Comparison of the critical value (1%, 5%, and 10%) and the ADF Test result, if the ADF Test result is less than 1%, 5%, and 10% at the same time, it means that the hypothesis is rejected very well.
Lag value: optimal lag order
Difference order: It is essentially the next value, minus the previous value, mainly to eliminate some fluctuations to make the data become stable, and the non-stationary sequence can be converted into a stationary sequence by difference transformation
AIC value: a standard to measure the goodness of the fit of a statistical model, the smaller the value, the better
Critical value: A critical value is a fixed value corresponding to a given significance level

The results of the series test show that, based on the field 客流量:
when the difference is 0 order, the significant P value is 0.000**significant, and the null hypothesis is rejected, and the series is a stationary time series.

MA prediction model

After the data is adjusted, there are generally two cases:
1) Strictly stationary series, there is no correlation between the values, this is a simple example where we can model the white noise of the residual. But very rare.
2) Sequences with significant correlation between values. Some statistical models need to be used.

Moving average models focus on the accumulation of error terms in autoregressive models. It can effectively remove random fluctuations in forecasting.

Eliminate trends and seasonal changes

Differential Differencing

Take the difference between the observed value at a certain moment and the observed value at the previous moment.

Decomposition Decomposition

Model trends and seasonality and remove them from the model

Take the difference between the observed value at a certain moment and the observed value at the previous moment:

df_diff=df_test['客流量']-df_test['客流量'].shift()
df_diff.dropna(inplace=True)
test_stationarity(df_diff)

ACF autocovariance and PACF partial autocorrelation function

ACF: The auto-covariance of a time series Xt is the covariance between a signal and its time-shifted signal. For example compare t1, t2 with t1-5 and t2-5. Comparing an ordered sequence of random variables with itself, it reflects the correlation between the values of the same sequence at different time series.
PACF: Partial autocorrelation function (PACF) calculates the strict correlation between two variables, which is the degree of correlation between two variables obtained after eliminating the interference of intermediate variables. For a stationary AR§ model, when the autocorrelation coefficient p(k) with lag k is obtained, the actual result is not the correlation between x(t) and x(tk). This is because there are k-1 variables between these two variables, which will have a series of effects on the autocorrelation coefficient, and the k-1 variables themselves are related to x(tk). This is not a small disturbance to the calculation of the autocorrelation coefficient p(k). The partial autocorrelation function can eliminate these interferences.

#ACF and PACF plots:
from statsmodels.tsa.stattools import acf,pacf

lag_acf=acf(df_diff,nlags=20)
lag_pacf=pacf(df_diff,nlags=20,method='ols')

#Plot ACF：
plt.subplot(121)
plt.plot(lag_acf)
plt.axhline(y=0,linestyle='--',color='gray')
plt.axhline(y=-1.96/np.sqrt(len(df_diff)),linestyle='--',color='gray')
plt.axhline(y=1.96/np.sqrt(len(df_diff)),linestyle='--',color='gray')
plt.title('Autocorrelation Function')

#Plot PACF：
plt.subplot(122)
plt.plot(lag_pacf)
plt.axhline(y=0,linestyle='--',color='gray')
plt.axhline(y=-1.96/np.sqrt(len(df_diff)),linestyle='--',color='gray')
plt.axhline(y=1.96/np.sqrt(len(df_diff)),linestyle='--',color='gray')
plt.title('Partial Autocorrelation Function')
plt.tight_layout()

On this graph, the two dashed lines around 0 are confidence intervals that can be used to determine 'p' and 'q'

Model building

df_diff=df_test['客流量']-df_test['客流量'].shift()
import statsmodels.api as sm
model = sm.tsa.arima.ARIMA(df_test['客流量'],order=(0,1,2))
results_MA=model.fit()
plt.plot(df_diff)
plt.plot(results_MA.fittedvalues,color='red')
plt.title('RSS: %.4f'% ((results_MA.fittedvalues-df_diff)**2).sum())
print((results_MA.fittedvalues-df_diff).sum())

Comparison of MA and AR models

Pay attention, prevent getting lost, if there are any mistakes, please leave a message for advice, thank you very much

That's all for this issue. I'm fanstuck. If you have any questions, feel free to leave a message to discuss. See you in the next issue.

See:

Using Python to build a time series (ARIMA, MA, AR) forecasting model

The MA model of the nature of the ARMA model - Programmer Sought

A quick study - detailed explanation of moving average model (MA) of time series analysis algorithm + Python example code

foreword