Article Directory
Conference: CVPR 2019
标题：《 Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networks 》
Papers Link: https://arxiv.org/pdf/1903.02193.pdf
Code links: -
Summary
In unmanned and driver assistance systems, lane detection is a very important module. In recent years, we made a lot of complex lane detection method. However, many methods mainly detect lane lines on a single frame images, in complex scenarios such as heavy shadow, severely degraded tag line, traffic jams and other scenes in poor performance. In fact, the lane lines on the road is a continuous line structure. Thus, not only accurately detect the lane mark on the current one. To this end, we studied consecutive frames lane detection by multiple roads, proposed a hybrid combination of depth framework of CNN + RNN. In particular, information of each frame is extracted module CNN, CNN wherein a plurality of consecutive time frames to maintain continuity module as the input to the RNN lane line prediction and learning features. Extensive experiments were performed on two large data sets, junction consequences show that the proposed method in this lane detection, particularly outstanding performance on the lane line detection in complex scenes.
Related work
Over the past two centuries, there is a lot of research in the field of lane line detection and prediction. These methods can be roughly divided into two categories. The traditional method and the method based on the depth of learning.
Traditional methods
Before learning method developed depth, lane detection is primarily used to check the geometry model based on linear and linear fitting, the main features of the first gradient, color and texture and energy minimization algorithm.
1. 几何模型
2. 能量最小化
Based on the depth of learning
1. Encoder-decoder CNN
2. FCN with optimization algorithms
3. ‘CNN+RNN‘
4. GAN model
Different from the above four methods based on the depth of learning, we propose a training network for combined RNN lane detection and CNN end to end.
The proposed method
System Overview
Lane line is a broken line configuration implemented or on the road, on a single frame or image geometry can be detected semantic segmentation method. But in ADAS (intelligent driver assistance) system, due to the complexity of the scene, a single image information can not meet robust lane detection algorithm.
We propose a combination of CNN and RNN successive frames lane detection algorithm. RNN continuous signal processing, feature extraction and timing of the benefits of integration, and can be used to predict lane detection. CNN the advantage that can handle a large number of images, and by pooling operations such as convolution, into the input picture feature extracting FIG small size. These features of FIG comprising successive frames from a timing properties that can be well treated RNN module.
In order to better RNN and CNN will be combined into an end to end training network, we designed a (encoder-decoder) codec framework. The frame shown in FIG. CNN CNN encoding and decoding are two fully connected network. As a series of consecutive input images, the incoming encoding module CNN, CNN encoding timing to obtain post-processing features of FIG. Wherein FIG LSTM network as input the incoming information is used to predict lane line. LSTM output of the decoding network and then passed to the CNN network, to obtain the predicted probability lane line in FIG. This probability plots and consistent input and picture size.
Network design
- LSTM network
Alternatively ConvLSTM LSTM using matrix multiplication of each door, for combining the end of the train the network.
A common cell activation function ConvLSTM time t may be expressed as follows:
- Encoder-decoder network
Network codec similar semantic segmentation lane detection task model.
Training Strategies
1) The network uses SegNe and UNet re-model pre-trained right on the ImageNet.
2) the road driving scene a number of consecutive image frames as the input image N
3 designed loss function)
4) using different optimization methods on different training levels. When first started training, using Adam optimization, training to a very high precision, using SGD algorithm
Experiments and results
data set
TuSimple lane combined data set and our own data set
in conclusion
This paper proposes a complex neural network CNN and one binding RNN robustness lane line detection. This network architecture based codec framework for a plurality of consecutive frames as an input, and in a manner predicted lane line semantic segmentation in the current frame. In this framework, each frame of the input image in the feature extraction will be first CNN encoder, and wherein all sequence encodes an input frame into ConvLSTM (LSTM convolution) processing. Finally, ConvLSTM output incoming information reconstruction and lane line decoding CNN to predict. Two data sets contain a continuous lane picture composed for performance evaluation.
Contrast those using only a single image as a reference input architecture, the new proposed architecture is obviously good results, justify the use of continuous multi-frame as very effective input. Meanwhile, the results show that there are obvious advantages ConvLSTM to learn than in sequence features FcLSTM lane line detection and target information to predict. This model better performance in precision, recall, and accuracy. Further, in this model on a set of very challenging test data set, verify its robustness, the results showed that in this model can be very stable under a variety of road lane line is detected, while avoiding erroneous recognition. In the parametric analysis, the longer the input sequence will have better performance, and a further indication of the lane detection, a multi-frame images more helpful than the single-frame images.
The future, we plan to further by adding a curve fit to mention lane detection in this framework. In this case, the detected vehicle large depression will be more smooth and more complete. In addition, there is a dark environment of strong interference, SegNet-ConvLSTM performance will be a little better job than UNet-ConvLSTM, more research is needed in this area.