Table of contents
A lane line detection algorithm based on traditional image processing methods in 2008.
Summary
The author proposes a fast and stable lane line detection algorithm for street scenes. The processing flow is to generate IPM images - use customized Gaussian kernel filtering - use Ransac method to fit Bezier curve parameters - post-processing. The algorithm can process 50 frames of images per second, and can detect lane lines in various scenes, and the effect is not bad compared with previous methods.
Introduction
One of the goals of the evolution of autonomous driving is to reduce the incidence of car accidents, among which lane line detection is a technology that has attracted much attention since the 1980s. The techniques used include monocular/binocular based, morphological operations, probabilistic grouping, B-snakes, etc. However, most of these algorithms are aimed at highways. In contrast, street scenes are more challenging, including the impact of vehicles, broken lane lines, background shadows, more curved/strange/intersecting lane lines, pedestrians, lighting environments, etc., as follows As shown in the figure.
This paper proposes a fast and robust method for detecting lane lines in street scenes. The process is to use the IPM method to generate a bird's-eye view - use a customized Gaussian filter to detect vertical lines - binarize - Hough transform to extract straight lines ——Fit the curve in the original image based on Ransac.
The innovations of the article are as follows: 1) The speed is fast, and the running frame rate of the 640x480 image on the Intel Core2 2.4GHz machine reaches 50Hz; 2) The number of straight lines detected is not limited; 3) A new curve fitting method based on Ransac is proposed ;4) Evaluate the actual effect of the algorithm by manually labeling the data and establishing an automatic scoring method.
accomplish
Generate IPM
The author chose to generate a bird's-eye view first, and then do detection in the bird's-eye view. This has two advantages: 1) Due to the influence of perspective projection in the original image, the lane lines are not parallel to each other, but they will be close to parallel in the bird's-eye view; 2) A bird's-eye view of the region of interest of a specified size can be generated, reducing Handles the dimensions of the image.
The internal and external parameters of the camera need to be used in the processing flow. Here, the author does not consider the distortion effects of roll and camera in the external parameters. The imaging model of the camera can be used to establish the pixel mapping relationship between the bird's-eye view and the original image, and then the bird's-eye view can be generated through interpolation calculation, as shown in the figure below.
filtering and filtering
The lane lines in the IPM image are basically distributed vertically, and the lane lines are brighter in the horizontal direction than the two sides; based on this, the author designed a filter along the x direction, which is similar to the combination of Gaussian + sobel, specifically for Detection of edges in the horizontal direction; at the same time, it is a trick to specify the sigma value in the filter to control the distribution of the template by assuming that the width pixel value of the edge in the image is known. The shape of its kernel is shown on the left side of the figure below, the middle is the operation result after applying the kernel, and the right side is the effect after thresholding
There is noise in the thresholded image obtained in the previous step, and its influence needs to be reduced. The author arranges the pixel values of the image from large to small, selects the top 2.5% as possible edge points for subsequent calculations, and discards all other pixel points.
Line detection
First, the above result image is used as input, assuming that the lane lines are roughly vertically distributed; the cumulative sum of pixel values in the column direction of the image is calculated to obtain a histogram, and then processed by Gaussian filtering; the local maximum point of the histogram is obtained, and its relative Apply parabolic interpolation to two adjacent points to obtain a more accurate position, which is used as the initial coordinate of the horizontal direction of the line, as shown in the figure below.
Then apply a rectangular frame near the above coordinates, consider that the pixels in the rectangular frame belong to the same straight line, and use the RANSAC method to fit the straight line, as shown in the figure below.
Curve Fitting
The above straight line fitting is only considered as a preliminary result. Since the lane line may be curved in the actual scene, it is necessary to continue to perform the curve fitting operation. The author here selects a third-order Bezier curve, which has 4 control points, which are divided into a start point, an end point, and two points that control the shape. The equation and schematic diagram are shown below.
The author implements curve fitting based on the idea of RANSAC, and the calculation process is shown in the figure below.
1) getRandomSample(): The input image is the result image of the straight line detection, select a point from the area near the line obtained after the straight line fitting, the higher the pixel value of the point, the greater the probability of being selected
2) fitSpline(): After getting some points, the coordinates of the four control points of the Bezier curve can be obtained by solving the least square method, and then the Bezier curve equation can be obtained
3) computeSplineScore(): Here the author did not use the sum of the distances from each point to the fitted Bezier curve as an evaluation indicator, because the amount of calculation is too large; instead, a new evaluation standard is proposed, including the use of fitting curves The pixel sum of the points (the higher the pixel value, the more likely it belongs to the lane line), the length of the curve (the longer the better), the angle of the curve (calculated using the two angles of the control points, the closer to the vertical the better).
The fitting output effect is shown in the figure below.
Post-processing
In order to obtain better curve fitting results, the author proposes the following operations, which are applied to both the bird's-eye view and the original image.
1) Get better edge points
Discretize the curve obtained in the previous step to obtain a series of points, and then calculate the normal direction of these points, along this direction, find a point with a larger nearby pixel value as the updated point, and re-fit the curve
2) Extended curve length
Along the direction of the head and tail of the curve obtained in 1), try to find a point that matches the pixel value and its normal vector is the extreme value to add in to extend the length of the curve
3) Result verification
Re-check the newly obtained curve, it cannot be too curved, and the length cannot be too small, and the direction in IPM should basically remain vertical
The effect after retesting is shown in the figure below.
experiment
The author collected some images of street scenes and manually marked some data, compared the detected curve results with the marked real values, and defined the evaluation criteria for whether it is correct or not, and the details will not be expanded.
in conclusion
A robust and fast lane line detection algorithm based on traditional image processing is proposed, specifically:
1) Processing in the bird's-eye view, using a customized Gaussian kernel filter, and then performing a straight line fitting, and proposing a new third-order Bezier curve fitting method based on the RANSAC method, and a post-processing method
2) There is no limit to the number of lane line detections, and it can process up to 50 images per second on the PC side
3) Experiments show that the detection accuracy of the algorithm is not bad