Summary of Stationarity Test Methods for Time Series

The time series stationarity test methods can be divided into three categories:

Graphical Analysis Methods
Simple Statistical Methods
Hypothesis Testing Methods

1. Graphical analysis method

Visualize data

Visualizing the data is to draw a line graph of the time series to see if the curve fluctuates around a certain value (to judge whether the mean value is stable), to see whether the fluctuation range of the curve changes greatly (to judge whether the variance is stable), and to see the frequency of fluctuations in different time periods of the curve [~Compact degree] does not change much (judging whether the covariance is stable), so as to judge whether the time series is stationary.

Let’s draw a few pictures below, and let’s intuitively judge which ones are stable and which ones are non-stationary.

import numpy as np
import pandas as pd
import akshare as ak
from matplotlib import pyplot as plt

np.random.seed(123)

# -------------- 准备数据 --------------
# 白噪声
white_noise = np.random.standard_normal(size=1000)

# 随机游走
x = np.random.standard_normal(size=1000)
random_walk = np.cumsum(x)

# GDP
df = ak.macro_china_gdp()
df = df.set_index('季度')
df.index = pd.to_datetime(df.index)
gdp = df['国内生产总值-绝对值'][::-1].astype('float')

# GDP DIFF
gdp_diff = gdp.diff(4).dropna()


# -------------- 绘制图形 --------------
fig, ax = plt.subplots(2, 2)

ax[0][0].plot(white_noise)
ax[0][0].set_title('white_noise')
ax[0][1].plot(random_walk)
ax[0][1].set_title('random_walk')

ax[1][0].plot(gdp)
ax[1][0].set_title('gdp')
ax[1][1].plot(gdp_diff)
ax[1][1].set_title('gdp_diff')

plt.show()

a. White noise, the curve fluctuates up and down around the 0 value, and the fluctuation range is consistent before and after, which is a stable sequence.
b. Random walk, the curve has no definite trend, the mean and variance fluctuate greatly, and the sequence is non-stationary.
c. The trend of GDP data is rising, the mean value increases with time, and the series is non-stationary.
d. The data after the seasonal difference of GDP, the curve fluctuates roughly on a horizontal line, and the fluctuation range changes little before and after, which can be considered stable.

Visualize statistical features

Visualizing statistical features refers to drawing autocorrelation and partial autocorrelation diagrams of time series, and judging whether the sequence is stable or not based on the performance of the autocorrelation diagram.

Autocorrelation, also known as serial correlation, is the degree of correlation of a signal with itself at different points in time, or with delayed copies of itself -- or lags -- as a function of delay. The autocorrelation coefficients obtained at different lag periods are called autocorrelation plots.

(There is a default assumption here that the sequence is stationary, and the autocorrelation of a stationary sequence is only related to the time interval k and does not change with the change of time t, so the autocorrelation function can be called a function of delay (k))

Stationary series usually have short-term correlation. For stationary time series, the autocorrelation coefficient tends to degenerate rapidly to zero (the shorter the lag period, the higher the correlation, when the lag period is 0, the correlation is 1); for non-stationary time series Data, degradation will occur more slowly, or there will be changes such as first decrease and then increase, or periodic fluctuations.

import statsmodels.api as sm
X = [2,3,4,3,8,7]
print(sm.tsa.stattools.acf(X, nlags=1, adjusted=True))

> [1, 0.3559322]
where the first element is the autocorrelation when the lag is 0, and the second element is the autocorrelation when the lag is 1

When the lag k autocorrelation coefficient is obtained from the ACF, it is not actually a simple correlation between X(t) and X(tk).

Because X(t) is also affected by the intermediate k-1 random variables X(t-1), X(t-2), ..., X(t-k+1), and these k-1 random variables All random variables have a correlation with X(tk), so the autocorrelation coefficient is actually doped with the influence of other variables on X(t) and X(tk).

After eliminating the interference of the intermediate k-1 random variables X(t-1), X(t-2), ..., X(t-k+1), the effect of X(tk) on X(t) The degree of correlation is called the partial autocorrelation coefficient. The partial autocorrelation coefficients obtained at different lag periods are called partial autocorrelation plots. (The calculation of the partial autocorrelation coefficient is more complicated, and will be introduced in detail later)

Let's take a look at a few practical cases (the data in the above figure will look at their autocorrelation diagram and partial autocorrelation diagram):

# 数据生成过程在第一个代码块中
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

fig, ax = plt.subplots(4, 2)
fig.subplots_adjust(hspace=0.5)

plot_acf(white_noise, ax=ax[0][0])
ax[0][0].set_title('ACF(white_noise)')
plot_pacf(white_noise, ax=ax[0][1])
ax[0][1].set_title('PACF(white_noise)')

plot_acf(random_walk, ax=ax[1][0])
ax[1][0].set_title('ACF(random_walk)')
plot_pacf(random_walk, ax=ax[1][1])
ax[1][1].set_title('PACF(random_walk)')

plot_acf(gdp, ax=ax[2][0])
ax[2][0].set_title('ACF(gdp)')
plot_pacf(gdp, ax=ax[2][1])
ax[2][1].set_title('PACF(gdp)')

plot_acf(gdp_diff, ax=ax[3][0])
ax[3][0].set_title('ACF(gdp_diff)')
plot_pacf(gdp_diff, ax=ax[3][1])
ax[3][1].set_title('PACF(gdp_diff)')

plt.show()

(1) The autocorrelation coefficient of white noise quickly decays to near 0, which is an obvious stationary sequence. When the lag period is 0, the autocorrelation coefficient and partial autocorrelation coefficient are actually the correlation between the sequence itself and itself, so it is 1; when the lag period is 1, the autocorrelation coefficient is 0, indicating that white noise has no autocorrelation.
(2) In the random walk, the autocorrelation coefficient decreases very slowly, so it is a non-stationary sequence; from the partial autocorrelation coefficient, we can see that the random walk is only related to the previous item.
(3) It can also be seen in the autocorrelation diagram of GDP data that there is a certain periodicity. The autocorrelation coefficients such as lags 4, 8, and 12 are relatively large and decrease slowly. After the difference, the decrease has a certain effect. is stable. Like visual data, intuitive judgment is highly subjective, but it allows us to have a more intuitive understanding of the data.

2. Simple statistical methods

The method of calculating statistics is only as a supplement, and you can understand it. There are two conditions for wide stability: the mean and variance are unchanged. We can see it intuitively in the visual data. In fact, we can also calculate it in detail.

Very interesting logic, directly split the sequence into two sequences before and after, calculate the mean and variance of the two sequences respectively, and compare them to see if the difference is obvious. (In fact, many time series anomaly tests are also based on this idea. If the distribution before and after is consistent, there is no anomaly, otherwise there is an anomaly or mutation)

Let's calculate the mean and variance of white noise and random walk sequences in different time periods:

import numpy as np

np.random.seed(123)

white_noise = np.random.standard_normal(size=1000)

x = np.random.standard_normal(size=1000)
random_walk = np.cumsum(x)

def describe(X):
    split = int(len(X) / 2)
    X1, X2 = X[0:split], X[split:]
    mean1, mean2 = X1.mean(), X2.mean()
    var1, var2 = X1.var(), X2.var()
    print('mean1=%f, mean2=%f' % (mean1, mean2))
    print('variance1=%f, variance2=%f' % (var1, var2))

print('white noise sample')
describe(white_noise)

print('random walk sample')
describe(random_walk)

white noise sample:
mean1=-0.038644, mean2=-0.040484
variance1=1.006416, variance2=0.996734

random walk sample:
mean1=5.506570, mean2=8.490356
variance1=53.911003, variance2=126.866920

The white noise sequence mean and variance are slightly different, but roughly On the same horizontal line;
the mean and variance of the random walk sequence are quite different, so it is a non-stationary sequence.

3. Hypothesis Testing Methods

The current mainstream hypothesis testing method for stationarity is the unit root test, which tests whether there is a unit root in the sequence.

Before introducing the inspection method, it is necessary to understand some related supplementary knowledge, so that the understanding of the subsequent inspection method will be clearer.

what is a unit root

import numpy as np
from matplotlib import pyplot as plt

np.random.seed(123)

def simulate(beta):
    y = np.random.standard_normal(size=1000)
    for i in range(1, len(y)):
        y[i] = beta * y[i - 1] + y[i]
    return y

plt.figure(figsize=(20, 4))
for i, beta in enumerate([0.9, 1.0, 1.1]):
    plt.subplot(1, 3, i+1)
    plt.plot(simulate(beta))
    plt.title('beta: {}'.format(beta))
plt.show()

Testing method

DF test

ADF test (Augmented Dickey-Fuller Testing) is one of the most commonly used unit root test methods. It is used to test whether the sequence has a unit root to determine whether the sequence is stationary. The ADF test is an enhanced version of the DF test. Before introducing the ADF, let's take a look at the DF test.

Dickey and Fuller (1979) roughly classified them into three categories based on the basic characteristics of non-stationary series and proposed DF test:

(1) When the basic trend of the series shows an irregular rise or fall and repeats, it is classified as a drift-free autoregressive process;
(2) When the basic trend of the series shows an obvious increase or decrease with time and the trend is not too steep , which is classified as an autoregressive process with a drift term;
(3) When the basic trend of the sequence increases rapidly with time, it is classified as a regression process with a trend term.

If the test statistic is greater than the critical value (the p value is greater than the significance level), the null hypothesis cannot be rejected, and the series is non-stationary;
if the test statistic is less than the critical value (the p value is less than the significance level), the null hypothesis is rejected and the series is considered stationary of.

ADF test

The test formula of DF is a first-order autoregressive process. In order to be suitable for the stationarity test of high-order autoregressive processes, Dickey et al. modified the DF test in 1984 and introduced a higher-order lag term. The test regression correction is:

import numpy as np
from matplotlib import pyplot as plt

np.random.seed(123)

y = np.random.standard_normal(size=100)
for i in range(1, len(y)):
    y[i] = 1 + 0.1*i + y[i]

plt.figure(figsize=(12, 6))
plt.plot(y)
plt.show()

Check for stationarity:

from arch.unitroot import ADF
adf = ADF(y)
# print(adf.pvalue)
print(adf.summary().as_text())

adf = ADF(y)
adf.trend = 'ct'
print(adf.summary().as_text())

Description:
The ADF test in the arch package can specify trend as
'n' (excluding intercept term and time trend term)
'c' (including intercept term)
'ct' (including intercept term and time trend term)
'ctt' (including intercept term, time trend term and quadratic time trend term)
correspond to tests of different stationary types respectively. (The lags defaults to the smallest AIC)

in the first text output above, if trend is not specified, the default is to test whether the intercept term is stable, the significance level is p=0.836>0.05, the null hypothesis is not rejected, and it is non-stationary;
the second above In each text output, specify trend to test whether the intercept term and the time trend term are stable, the significance level is p=0.000<0.05, and the null hypothesis is rejected, so the trend term is stable.

Let's see if the data before and after the seasonal difference of GDP is stable:

# 数据在第一个代码块中
from arch.unitroot import ADF
adf = ADF(gdp)
print(adf.summary().as_text())

adf = ADF(gdp_diff)
print(adf.summary().as_text())

It can be seen that the p value before the difference is 0.998>0.05, the null hypothesis cannot be rejected, and the data is non-stationary; the p value after the difference is 0.003 < 0.05, so the null hypothesis can be rejected at the 5% significance level, and the data after the difference is stable of.

# 数据在第一个代码块中
from arch.unitroot import ADF
adf = ADF(gdp)
adf.trend = 'ct'
print(adf.summary().as_text())

The specified test stationary type is stationary with intercept term and time trend term, and the p-value is 0.693>0.05. Also, the null hypothesis cannot be rejected, so the trend is not stationary before the difference.

PP inspection

Phillips and Perron (1988) proposed a nonparametric test method, mainly to solve the potential serial correlation and heteroscedasticity problems in residual terms, and the asymptotic distribution and critical value of the test statistic are the same as the ADF test. It also appeared earlier, with the same assumptions and similar usage, and can be used as a supplement to the ADF test.

Also construct a trend stationary series, look at the PP test results:

import numpy as np
from arch.unitroot import PhillipsPerron

np.random.seed(123)

y = np.random.standard_normal(size=100)
for i in range(1, len(y)):
    y[i] = 1 + 0.1*i + y[i]


pp = PhillipsPerron(y)
print(pp.summary().as_text())

pp = PhillipsPerron(y)
pp.trend = 'ct'
print(pp.summary().as_text())

If trend is not specified as the default test for whether it is a stationary process with an intercept term, the p-value of the test result is 0.055>0.05, and the corresponding test statistic is -2.825 greater than the critical value -2.89 at the 5% significance level, so 5% significance However, the test statistic is less than the critical value -2.58 at the 10% significance level, so the null hypothesis can be rejected at the 10% significance level, and it is considered to be a stationary series.

Specify trend='ct' to test whether it is a stationary process with an intercept term and a time trend term. The p value of the test result is 0.000<0.05, so the trend is stable; in fact, the test statistic is -10.009 less than 1% significance level The critical value of -4.05, so it is stationary even at the 1% significance level.

Based on the above test results, it can be determined that the series is trend-stationary.

DF-GLS inspection

The DF-GLS test is a unit root test method proposed by Elliott, Rothenberg, and Stock in 1996. The full name is Dickey-Fuller Test with GLS Detredding, that is, "the test for removing the trend using the generalized least squares method". It is currently the most effective. unit root test.

The DF-GLS test uses the generalized least squares method. First, perform a "quasi-difference" on the data to be tested, and then use the quasi-difference data to detrend the original sequence, and then use the model form of the ADF test to detrend the data. A unit root test is performed, but at this time the ADF test model no longer contains constant terms or time trend variables.

Also construct a trend stationary series to see the test effect:

import numpy as np
from arch.unitroot import DFGLS

np.random.seed(123)

y = np.random.standard_normal(size=100)
for i in range(1, len(y)):
    y[i] = 1 + 0.1*i + y[i]

dfgls = DFGLS(y)
print(dfgls.summary().as_text())

dfgls = DFGLS(y)
dfgls.trend = 'ct'
print(dfgls.summary().as_text())

If trend is not specified, the null hypothesis cannot be rejected, and it is not stationary; when trend='ct' is specified, the p value is less than 0.05, the null hypothesis is rejected, and the intercept term and the time trend are stable.

Let's construct a non-stationary series with a unit root to see the test results:

import numpy as np
from arch.unitroot import DFGLS

np.random.seed(123)

y = np.random.standard_normal(size=100)
for i in range(1, len(y)):
    y[i] = 0.1 + y[i-1] + y[i]

dfgls = DFGLS(y)
print(dfgls.summary().as_text())

dfgls = DFGLS(y)
dfgls.trend = 'ct'
print(dfgls.summary().as_text())

One p-value is 0.645 and the other is 0.347, both greater than 0.05/0.1. If the test type is not specified, all of them fail to pass the test, so the series is a non-stationary series. (DF-GLS test trend can only be specified as 'c' or 'ct')

KPSS inspection

Another well-known test for the existence of the unit root is the KPSS test proposed by Kwiatkowski, Phillips, and Shin in 1992. Compared with the above three test methods, the biggest difference is that its null hypothesis is a stationary sequence or a trend stationary sequence, while the alternative hypothesis is that there is a unit root.

Null hypothesis: The series does not have a unit root (time series is stationary or trend stationary)
Alternative hypothesis: The series has a unit root (the time series is non-stationary)
```
import numpy as np
from arch.unitroot import KPSS

np.random.seed(123)

y = np.random.standard_normal(size=100)
for i in range(1, len(y)):
    y[i] = 0.1 + y[i-1] + y[i]

kpss = KPSS(y)
print(kpss.summary().as_text())

kpss = KPSS(y)
kpss.trend = 'ct'
print(kpss.summary().as_text())
```
Note that the null hypothesis in the KPSS test is that there is no unit root. The p value under the default test trend type is 0.000, the null hypothesis is rejected, there is a unit root, and the series is non-stationary. After specifying trend='ct', the p value is 0.115>0.05, the null hypothesis is not rejected, the series trend is considered to be stable, and the test is wrong. None of the above tests can be 100% guaranteed to be correct. The PP test can be considered as a supplement to the ADF test. The KPSS test can also be used together with other tests. When both are considered to be stable or the trend is stable, it is determined to be stable.

In addition to the above test methods, there are Zivot-Andrews test, Variance Ratio test and other test methods.

The above code implementation uses the arch package in Python, and a common package statsmodels also implements the unit root test method, and the results are the same.

Big thanks to Python Data Science

I am going to do financial statistics next, I need to turn to python, and I need a tutorial. This official account is very useful to me.