【论文阅读】Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction [将交通作为图像学习: 用于大规模交通网络速度预测的深度卷积神经网络]（1）
Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction
将交通学习为图像:用于大规模交通网络速度预测的深度卷积神经网络
注: 阅读原文请转至link.
Abstract（摘要）
Abstract: This paper proposes a convolutional neural network (CNN)-based method that learns traffic as images and predicts large-scale, network-wide traffic speed with a high accuracy. Spatiotemporal traffic dynamics are converted to images describing the time and space relations of traffic flow via a two-dimensional time-space matrix. A CNN is applied to the image following two consecutive steps: abstract traffic feature extraction and network-wide traffic speed prediction. The effectiveness of the proposed method is evaluated by taking two real-world transportation networks, the second ring road and north-east transportation network in Beijing, as examples, and comparing the method with four prevailing algorithms, namely, ordinary least squares, k-nearest neighbors, artificial neural network, and random forest, and three deep learning architectures, namely, stacked autoencoder, recurrent neural network, and long-short-term memory network. The results show that the proposed method outperforms other algorithms by an average accuracy improvement of 42.91% within an acceptable execution time. The CNN can train the model in a reasonable time and, thus, is suitable for large-scale transportation networks.
Keywords: transportation network; traffic speed prediction; spatiotemporal feature; deep learning; convolutional neural network
摘要: 本文提出了一种基于卷积神经网络(CNN)的方法，该方法将交通作为图像学习，并以较高的精度预测大规模、全网络的交通速度。通过二维时空矩阵将交通流的时空动态转换为描述交通流时空关系的图像。将CNN应用于图像，经过两个连续步骤:抽象交通特征提取和全网交通速度预测。以北京市二环交通网络和东北交通网络为例，与普通最小二乘、k近邻、人工神经网络和随机森林四种常用交通网络算法进行比较，评价了该方法的有效性。三种深度学习体系结构，即堆叠自编码器、循环神经网络和长短期记忆网络。结果表明，在可接受的执行时间内，该方法的平均精度比其他算法提高了42.91%。CNN可以在合理的时间内对模型进行训练，因此适用于大型交通网络。
关键词: 交通网络; 交通速度预测; 时空特征; 深度学习; 卷积神经网络
1.Introduction
Predicting the future is one of the most attractive topics for human beings, and the same is true for transportation management. Understanding traffic evolution for the entire road network rather than on a single road is of great interest and importance to help people with complete traffic information in make better route choices and to support traffic managers in managing a road network and allocate resources systematically [1], [2].
预测未来是人类最具吸引力的话题之一，交通管理也是如此。了解整个路网的交通变化，而不是单一道路上的交通变化，对于帮助拥有完整交通信息的人们更好地规划路线，并支持交通管理者系统地管理路网和分配资源具有重要意义[1], [2]。
However, large-scale network traffic prediction requires more challenging abilities for prediction models, such as the ability to deal with higher computational complexity incurred by the network topology, the ability to form a more intelligent and efficient prediction to solve the spatial correlation of traffic in roads expanding on a two-dimensional plane, and the ability to forecast longer-term futures to reflect congestion propagation. Unfortunately, traditional traffic prediction models, which usually treat traffic speeds as sequential data, do not provide those abilities because of limitations, such as hypotheses and assumptions, ineptness to deal with outliers, noisy or missing data, and inability to cope with the curse of dimensionality [2]. Thus, existing models may fail to predict large-scale network traffic evolution.
然而，大规模网络流量预测对预测模型的能力要求更高，比如应对网络拓扑结构带来的更高计算复杂度的能力，形成更智能、更高效的预测以解决二维平面上道路交通的空间相关性的能力，以及预测更长期未来以反映拥堵传播的能力。不幸的是，传统的交通预测模型通常将交通速度视为顺序数据，但由于假设和假设、处理离群值的能力、噪声或缺失数据的能力、以及无法处理维数 [3]的变化等限制，无法提供这些能力。因此，现有的模型可能无法预测大规模网络流量的变化。
In the existing literature, two families of research methods have dominated studies in traffic forecasting: statistical methods and neural networks [3].
在现有的文献中，交通预测的研究主要有两类研究方法:统计方法和神经网络[3]。
Statistical techniques are widely used in traffic prediction. For example, according to the periodicity of traffic evolutions, nonparametric models, such as k-nearest neighbors (KNN), have been applied to predict traffic speeds and volumes [4–6]. More advanced models were employed, including support vector machines (SVM) [7], seasonal SVM [8], Online-SVM [9], and on-line sequential extreme learning machine [10], to promote prediction accuracy by capturing the high dynamics and sensitivity of traffic flow. SVM performance in large-scale traffic speed prediction was further improved [8] [11]. Multivariate nonparametric regression was also used in traffic prediction [12,13]. Recently, a wealth of literature leverage multiple hybrid models and spatiotemporal features to improve traffic prediction performance. For example, Li et al. [14] proposed a hybrid strategy with ARIMA and SVR models to enhance traffic prediction power by considering both spatial and temporal features. Zhu et al. [15] employed a linear conditional Gaussian Bayesian network (LCG-BN) with spatial and temporal, as well as speed, information for traffic flow prediction. Li et al. [16] studied the chaotic situation of traffic flow based on a Bayesian theory-based prediction algorithm, and incorporated speed, occupancy, and flow for accuracy improvement. Considering the correlations shown in successive time sequences of traffic variables, time-series prediction models have been widely employed in traffic prediction. One of the typical models is the autoregressive integrated moving average (ARIMA) model, which considers the essential traffic flow characteristics, such as inherent correlations (via a moving average) and its effect on the short future (via autoregression). To date, the model, and its extensions, such as the seasonal ARIMA model [17,18], KARIMA model [19], and the ARIMAX model [20], have been widely studied and applied. In summary, statistical methods have been widely used in traffic prediction, and promising results have been demonstrated. However, these models ignore the important spatiotemporal feature of transportation networks, and cannot be applied to predict overall traffic in a large-scale network. SVM usually takes a long time and consumes considerable computer memory on training and, hence, it might be powerless in large data-related applications.
统计技术广泛应用于交通预测。例如，根据交通变化的周期性，应用k-nearest neighbors (KNN)等非参数模型来预测交通速度和流量[4–6]。采用支持向量机[7]、季节性支持向量机[8]、在线支持向量机[9]和在线序流极值学习机[10]等更先进的模型，通过捕捉交通流的高动态性和敏感性来提高预测精度。进一步提高了SVM在大规模交通速度预测中的性能[8] [11]。多变量非参数回归也用于交通预测[12,13]。近年来，大量文献利用多种混合模型和时空特征来提高交通预测性能。例如Li et al.[14]提出了一种混合ARIMA和SVR模型的策略，通过考虑空间和时间特征来提高交通预测能力。Zhu等人的[15]采用具有时空信息和速度信息的线性条件高斯贝叶斯网络(LCG-BN)进行交通流预测。Li等[16]基于基于贝叶斯理论的预测算法研究交通流的混沌状态，并结合速度、占用率和流量来提高精度。由于交通变量在连续时间序列中表现出的相关性，时间序列预测模型被广泛应用于交通预测中。最典型的模型之一是自回归综合移动平均(ARIMA)模型，该模型考虑了交通流的本质特征，如内在相关性(通过移动平均)及其对短期未来的影响(通过自回归)。到目前为止，该模式及其扩展，如季节性ARIMA模式[17,18]、KARIMA模式[19]和ARIMAX模式[20]，已经得到了广泛的研究和应用。综上所述，统计方法在交通预测中得到了广泛的应用，并取得了良好的效果。然而，这些模型忽略了交通网络的重要时空特征，无法用于大规模交通网络的整体交通预测。支持向量机通常需要很长的时间和大量的计算机内存用于训练，因此，在大型数据相关的应用中可能是无能为力的。
Artificial neural networks (ANNs) are also usually applied to traffic prediction problems because of its advantages, such as their capability to work with multi-dimensional data, implementation flexibility, generalizability, and strong forecasting power [3]. For example, Huang and Ran [21] used an ANN to predict traffic speed under adverse weather conditions. Park et al. [2] presented a real-time vehicle speed prediction algorithm based on ANN. Zheng et al. [22] combined an ANN with Bayes’ theorem to predict short-term freeway traffic flow. Moretti et al. [23] developed a statistical and ANN bagging ensemble hybrid model to forecast urban traffic flow.
由于人工神经网络具有处理多维数据的能力、实现的灵活性、通用性以及较强的预测能力[3]等优点，因此也常被应用于流量预测问题。例如，Huang和Ran[21]使用人工神经网络预测恶劣天气条件下的交通速度。Park等人[2]提出了一种基于ANN的实时车速预测算法。Zheng等人[22]将神经网络与贝叶斯定理相结合来预测短期高速公路交通流。Moretti等人[23]开发了一种统计和人工神经网络bagging集合的混合模型来预测城市交通流。
However, the data-driven mechanism of an ANN cannot explain the spatial correlations of a road network particularly well. In addition, compared with deep learning approaches, the prediction accuracy of an ANN is lower because of its shallow architecture. Recently, more advanced and powerful deep learning models have been applied to traffic prediction. For example, Polson and Sokolov [24] used deep learning architectures to predict traffic flow. Huang et al. [25] first introduced Deep Belief Networks (DBN) into transportation research. Then, Tan et al. [26] compared the performance of DBNs with two kinds of RBM structures, namely, RBM with binary visible and hidden units (B-B RBM) and RBM with Gaussian visible units and binary hidden units (G-B RBM), and found that the former outperforms the later in traffic flow prediction. Ma et al. [27] combined deep restricted Boltzmann machines (RBM) with a recurrent neural network (RNN) and formed a RBM-RNN model that inherits the advantages of both RBM and RNN. Lv et al. [28] proposed a novel deep-learning-based traffic prediction model that considered spatiotemporal relations, and employed stack autoencoder (SAE) to extract traffic features. Duan et al. [29] used denoising stacked autoencoders (DSAE) for traffic data imputation. Ma et al. [30] introduced a long short-term memory neural network (LSTM NN) into traffic prediction and demonstrated that LSTM NN outperformed other neural networks in both stability and accuracy in terms of traffic speed prediction by using remote microwave sensor data collected from the Beijing road network.
然而，神经网络的数据驱动机制并不能很好地解释道路网络的空间相关性。此外，与深度学习方法相比，神经网络的结构较浅，其预测精度较低。近年来，更先进、更强大的深度学习模型被应用到流量预测中。例如，Polson和Sokolov[24]使用深度学习架构来预测交通流。Huang等人[25]首先将深度信念网络(Deep Belief Networks, DBN)引入交通研究。然后，Tan等人[26]比较了dbn与两种RBM结构的性能，即具有二进制可见和隐藏单元的RBM (B-B RBM)和具有高斯可见和二进制隐藏单元的RBM (G-B RBM)，发现前者在交通流预测方面优于后者。Ma等人[27]将深度限制波尔兹曼机(RBM)和循环神经网络(RNN)结合起来，形成了继承RBM和RNN优点的RBM-RNN模型。Lv等人[28]提出了一种考虑时空关系的基于深度学习的新型交通预测模型，并采用堆栈自动编码器(SAE)提取交通特征。Duan等人的[29]使用去噪的堆叠自动编码器(DSAE)进行交通数据imputation。Ma等人[30]将一种长短期记忆神经网络(long - short-term memory neural network, LSTM神经网络)引入到交通预测中，利用北京路网中收集的远程微波传感器数据，证明LSTM神经网络在交通速度预测的稳定性和准确性方面优于其他神经网络。
Deep learning methods exploit much deeper and more complex architectures than an ANN, and can achieve better results than traditional methods. However, these attempts still mainly focus on the prediction of traffic on a road section or a small network region. Few studies have considered a transportation network as a whole and directly estimated the traffic evolution on a large scale. More importantly, the majority of these models merely considered the temporal correlations of traffic evolutions at a single location, and did not consider its spatial correlations from the perspective of the network.
深度学习方法利用了比人工神经网络更深、更复杂的体系结构，可以获得比传统方法更好的结果。然而，这些尝试仍然主要集中在某一路段或小网络区域的交通预测上。很少有研究将交通网络作为一个整体来考虑，直接对大规模的交通演化进行估计。更重要的是，这些模型大多只考虑了单一地点交通演化的时间相关性，而没有从网络的角度考虑其空间相关性。
To fill the gap, this paper introduces an image-based method that represents network traffic as images, and employs the deep learning architecture of a convolutional neural network (CNN) to extract spatiotemporal traffic features contained by the images. A CNN is an efficient and effective image processing algorithm and has been widely applied in the field of computer vision and image recognition with remarkable results achieved [31,32]. Compared with prevailing artificial neural networks, a CNN has the following properties in extracting features: First, the convolutional layers of a CNN are connected locally instead of being fully connected, meaning that output neurons are only connected to its local nearby input neurons. Second, a CNN introduces a new layer-construction mechanism called pooling layers that merely select salient features from its receptive region and tremendously reduce the number of model parameters. Third, normal fully-connected layers are used only in the final stage, when the dimension of input layers is controllable. The locally-connected convolutional layers enable a CNN to efficiently deal with spatially-correlated problems [31,33,34]. The pooling layers makes CNNs generalizable to large-scale problems [35]. The contributions of the paper can be summarized as follows:
- The temporal evolutions and spatial dependencies of network traffic are considered and applied simultaneously in traffic prediction problems by exploiting the proposed image-based method and deep learning architecture of CNNs.
- Spatiotemporal features of network traffic can be extracted using a CNN in an automatic manner with a high prediction accuracy.
- The proposed method can be generalized to large-scale traffic speed prediction problems while retaining trainability because of the implementation of convolutional and pooling layers.
为了填补这一空白，本文引入了一种基于图像的方法，将网络流量表示为图像，并利用卷积神经网络(CNN)的深度学习体系提取图像包含的时空流量特征。CNN是一种高效、有效的图像处理算法，已广泛应用于计算机视觉和图像识别领域，取得了显著的效果[31,32]。与现有的人工神经网络相比，CNN在特征提取方面具有以下特点:首先，CNN的卷积层是局部连接的，而不是完全连接的，即输出神经元只与附近的局部输入神经元连接;其次，CNN引入了一种新的层构建机制，称为池化层，它只从接受区域中选择显著特征，大大减少了模型参数的数量。第三，在输入层尺寸可控的情况下，仅在最后阶段使用正常的全连通层。局部连接的卷积层使CNN能够有效地处理空间相关问题[31,33,34]。池化层使CNN可泛化到大规模问题[35]。本文的贡献可以总结如下:
- 利用所提出的基于图像的方法和CNN的深度学习体系结构，将网络流量的时间演化和空间依赖性同时应用到流量预测问题中。
- 利用CNN可以自动提取网络流量的时空特征，具有较高的预测精度。
- 由于实现了卷积层和池化层，该方法可以在保持可训练性的前提下推广到大规模的交通速度预测问题。
The rest of the paper is organized as follows: In Section 2, a two-step procedure that includes converting network traffic to images and a CNN for network traffic prediction is introduced. In Section 3, four prediction tests are conducted on two transportation networks using the proposed method, and are compared with the other prevailing prediction methods. Finally, conclusions are drawn with future study directions in Section 4.
本文的其余部分组织如下:在第2节中，介绍了将网络流量转换为图像和用于网络流量预测的CNN的两步过程。在第3节中，利用该方法对两个交通网络进行了4次预测试验，并与其他常用的预测方法进行了比较。最后，在第4节得出结论，并指出未来的研究方向。
参考文献
1. Zhang, J.; Wang, F.-Y.; Wang, K.; Lin, W.-H.; Xu, X.; Chen, C. Data-driven intelligent transportation systems: A survey. IEEE Trans. Intell. Transp. Syst. 2011, 12, 1624–1639.
2. Park, J.; Li, D.; Murphey, Y.L.; Kristinsson, J.; McGee, R.; Kuang, M.; Phillips, T. Real Time Vehicle Speed Prediction Using a Neural Network Traffic Model. In Proceedings of the International Joint Conference on Neural Networks, San Jose, CA, USA, 31 July–5 August 2011; pp. 2991–2996.
3. Karlaftis, M.G.; Vlahogianni, E.I. Statistical methods versus neural networks in transportation research: Differences, similarities and some insights. Transp. Res. Part C Emerg. Technol. 2011, 19, 387–399.
4. Davis, G.A.; Nihan, N.L. Nonparametric Regression and Short-Term Freeway Traffic Forecasting. J. Transp. Eng. 1991, 117, 178–188.
5. Chang, H.; Lee, Y.; Yoon, B.; Baek, S. Dynamic near-term traffic flow prediction: Systemoriented approach based on past experiences. IET Intell. Transp. Syst. 2012, 6, 292–305.
6. Xia, D.; Wang, B.; Li, H.; Li, Y.; Zhang, Z. A distributed spatial–temporal weighted model on MapReduce for short-term traffic flow forecasting. Neurocomputing 2016, 179, 246–263.
7. Wu, C.H.; Ho, J.M.; Lee, D.T. Travel-time prediction with support vector regression. IEEE Trans. Intell. Transp. Syst. 2004, 5, 276–281.
8. Hong, W.C. Traffic flow forecasting by seasonal SVR with chaotic simulated annealing algorithm. Neurocomputing 2011, 74, 2096–2107.
9. Castro-Neto, M.; Jeong, Y.S.; Jeong, M.K.; Han, L.D. Online-SVR for short-term traffic flow prediction under typical and atypical traffic conditions. Expert Syst. Appl. 2009, 36, 6164–6173.
10. Ma, Z.; Luo, G.; Huang, D. Short term traffic flow prediction based on on-line sequential extreme learning machine. In Proceedings of the 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI), Amnat Charoen Chiang Mai, Thailand, 14–16 February 2016; pp. 143–149.
11. Asif, M.T.; Dauwels, J.; Goh, C.Y.; Oran, A.; Fathi, E.; Xu, M.; Dhanya, M.M.; Mitrovic, N.; Jaillet, P. Spatiotemporal patterns in large-scale traffic speed prediction. IEEE Trans. Intell. Transp. Syst. 2014, 15, 794–804.
12. Clark, S. Traffic Prediction Using Multivariate Nonparametric Regression. J. Transp. Eng. 2003, 129, 161–168.
13. Haworth, J.; Cheng, T. Non-parametric regression for space-time forecasting under missing data. Comput. Environ. Urban Syst. 2012, 36, 538–550.
14. Li, L.; He, S.; Zhang, J.; Ran, B. Short-term highway traffic flow prediction based on a hybrid strategy considering temporal-spatial information. J. Adv. Transp. 2017.
15. Zhu, Z.; Peng, B.; Xiong, C.; Zhang, L. Short-term traffic flow prediction with linear conditional Gaussian Bayesian network. J. Adv. Transp. 2016, 50, 1111–1123.
16. Li, Y.; Jiang, X.; Zhu, H.; He, X.; Peeta, S.; Zheng, T.; Li, Y. Multiple measures-based chaotic time series for traffic flow prediction based on Bayesian theory. Nonlinear Dyn. 2016, 85, 179–194.
17. Tran, Q.T.; Ma, Z.; Li, H.; Hao, L.; Trinh, Q.K. A Multiplicative Seasonal ARIMA/GARCH Model in EVN Traffic Prediction. Int. J. Commun. Netw. Syst. Sci. 2015, 8, 43–49.
18. Williams, B.M.; Hoel, L.A. Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical Results. J. Transp. Eng. 2003, 129, 664–672.
19. Voort, M.V.D.; Dougherty, M.; Watson, S. Combining kohonen maps with arima time series models to forecast traffic flow. Transp. Res. Part C Emerg. Technol. 1996, 4, 307–318.
20. Williams, B. Multivariate Vehicular Traffic Flow Prediction: Evaluation of ARIMAX Modeling. Transp. Res. Rec. J. Transp. Res. Board 2001, 1776, 194–200.
21. Huang, S.H.; Ran, B. An Application of Neural Network on Traffic Speed Prediction Under Adverse Weather Condition. In Proceedings of the Transportation Research Board 82nd Annual Meeting, Washington, DC, USA, 12–16 January 2003.
22. Zheng, W.; Lee, D.H.; Zheng, W.; Lee, D.H. Short-Term Freeway Traffic Flow Prediction: Bayesian Combined Neural Network Approach. J. Transp. Eng. 2006, 132, 114–121.
23. Moretti, F.; Pizzuti, S.; Panzieri, S.; Annunziato, M. Urban traffic flow forecasting through statistical and neural network bagging ensemble hybrid modeling. Neurocomputing 2015, 167, 3–7.
24. Polson, N.; Sokolov, V. Deep Learning Predictors for Traffic Flows. arXiv 2016, arXiv:1604.04527.
25. Huang, W.; Song, G.; Hong, H.; Xie, K. Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning. IEEE Trans. Intell. Transp. Syst. 2014, 15, 2191–2201.
26. Tan, H.; Xuan, X.; Wu, Y.; Zhong, Z.; Ran, B. A comparison of traffic flow prediction methods based on DBN. In Processedings of the 16th COTA International Conference of Transportation Professionals (CICTP), Shanghai, China, 6–9 July 2016; pp. 273–283.
27. Ma, X.; Yu, H.; Wang, Y.; Wang, Y. Large-scale transportation network congestion evolution prediction using deep learning theory. PLoS ONE 2015, 10, e0119044.
28. Lv, Y.; Duan, Y.; Kang, W.; Li, Z.; Wang, F.Y. Traffic Flow Prediction With Big Data: A Deep Learning Approach. IEEE Trans. Intell. Transp. Syst. 2014, 16, 1–9.
29. Duan, Y.; Lv, Y.; Kang, W.; Zhao, Y. A deep learning based approach for traffic data imputation. Intell. Transp. Syst. 2014, 912–917.
30. Ma, X.; Tao, Z.; Wang, Y.; Yu, H.; Wang, Y. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp. Res. Part C Emerg. Technol. 2015, 54, 187–197.
31. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012.
32. Oquab, M.; Bottou, L.; Laptev, I.; Sivic, J. Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks. In Processedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1717–1724.
33. Lawrence, S.; Giles, C.L.; Tsoi, A.C.; Back, A.D. Face recognition: A convolutional neural-network approach. IEEE Trans. Neural Netw. 1997, 8, 98–113.
34. Ji, S.; Yang, M.; Yu, K. 3D Convolutional Neural Networks for Human Action Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 221–231.
35. Karpathy, A.; Toderici, G.; Shetty, S.; Leung, T.; Sukthankar, R.; Li, F.F. Large-Scale Video Classification with Convolutional Neural Networks. In Processedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1725–1732.