Known, CNN machine vision problems in the outstanding performance, because it performs convolution operation, extracting a feature from the local input tile, modular and can be expressed, while efficient use of data.
These properties allow convolutional neural network with outstanding performance in the field of computer vision, also made him particularly effective sequence processing. Time can be viewed as a spatial dimension, like height or width of the two-dimensional image.
For some sequences deal with the problem, the effect of one-dimensional convolution neural network can be comparable RNN, and calculate the cost is usually much smaller. Recently, one-dimensional convolution neural network [usually eaten with empty convolution kernel (dilated kernel)] has achieved great success in the audio field and generate machine translation. For simple tasks text classification and time series prediction, a small one-dimensional convolution neural networks can be used instead RNN, and faster.
A pool of sequence-dimensional data
CNN mentioned dimensional pooling operation, such as two-dimensional and two-dimensional pooling average maximum pooling, for downsampling image tensor space convolutional neural network. You can do the same one-dimensional pooling operations: extracting from the input sequence of a short-dimensional (i.e., sequence), and then outputs the maximum value (maximum pooling) or average value (average pooled). And two-dimensional convolution neural networks, this operation is used to reduce the length of one-dimensional inputs (sub-sampling).
CNN and binding to handle long sequence RNN
A dimensional convolutional neural network processes each input sequence segment, the sequence of steps it is not sensitive to time (here said range is greater than the order of the local scale, i.e., larger than the size of the convolution window), which is different from the RNN. In order to identify the longer-term mode, a number of layers and the convolution cell layer may be stacked together such that the upper layer can be observed in the original input sequence segments longer, but this is still not a good method for introducing susceptibility sequence. CNN all positions in the input time series seek mode, it does not know the time position of a pattern seen (from the beginning how long, how long, etc. from the end). For some prediction problem, its interpretation of the latest data point to the earlier interpretation of the data points are not the same, so the convolution neural network can not get meaningful results. CNN's sentiment classification for this limitation is not a problem, because for positive emotions or negative emotions associated with the keyword mode, and what position in the input sentence appears regardless of the amount of information he included in the same.
To convolutional neural network in conjunction with the speed and sensitivity of the order RNN lightweight, one approach is to use a one-dimensional convolution neural network as the front RNN pretreatment step, as FIG. Those very long, so that the sequence can not handle RNN (comprising thousands of time steps in sequence), this method is particularly useful for. Product rolls neural network can be long input sequences to shorter sequences composed of advanced features (downsampling). Then, these sequences composed of extracted features become RNN network input. These sequences then extracted features become network consisting of an input RNN.
to sum up:
1) In general, a two-dimensional convolutional neural network architecture and a computer vision dimensional convolution neural network is very similar, and it Conv1D layer MaxPooling1D layers stacked together, the final calculation of a global pool or flattening operations.
2) Because the cost of the process RNN computationally very long time sequence, but the calculation cost of a one-dimensional convolution neural network is small, it is used before RNN dimensional convolution neural network as a pre-treatment step is a good idea, so It allows shorter sequence, and to extract useful representation RNN processed.
