1. Research Background

With the rapid development of unmanned vehicle driving technology, higher requirements are put forward for the real-time performance of lane line detection technology. To detect lane lines based on OpenCV, firstly, the images collected by the vehicle system are cropped, grayscaled, Then filter the grayscale image to enhance the image quality, then expand the enhanced image and perform edge detection on it, and finally select the region of interest, probability Hough line transformation and fitting, so as to obtain the expected lane Line detection image. Experiments show that cutting the collected image twice can make the vehicle lane line image detect efficiently, real-time and with high precision, which is suitable for lane line detection of vehicles driving at high speed outdoors.

2. Picture demonstration

3. Video demonstration

Real-time lane detection system based on OpenCV (source code & tutorial)_哔哩哔哩_bilibili

4. Algorithm flow chart

5. Gaussian filtering

Gaussian filtering is often used to remove some Gaussian noise in the image. At present, the commonly used image filtering technology is mainly divided into spatial domain filtering technology and frequency domain filtering technology. Due to the high real-time requirements for lane line detection in this paper and the high The frequency signal will not be polluted, so the Gaussian filter in the frequency domain is selected. The Gaussian filter is essentially a low-pass filter, as shown in the figure.

The comparison before and after noise reduction is shown in the figure:

6. Image Adaptive Edge Extraction

The solution proposed by this blog , after grayscale and Gaussian smoothing filtering of the collected graphics, the obtained image is enhanced, but its grayscale level is still 256, which will have an impact on the subsequent lane line detection work. Therefore, this paper uses the maximum inter-class variance technology and binarization technology to process the image, so as to eliminate some interference in the image, so that the accuracy and detection efficiency of the image in lane line detection can be improved. The selection of the threshold is determined by the collected images. In this paper, the maximum inter-class variance method (otsu) and the function CV2.Threshold(img,0,255,cv2.THRESH_BINARY+ CV2.OTSU) in the OpenCV library are used, as shown in the figure. Image edge extraction techniques mainly include edge detection algorithms such as Sobel, Laplace, Canny, Roberts, and Prewitt. Since the Canny operator can detect more lane line edge information, this paper uses the Canny edge detection operator to perform edge detection on the image processed with the maximum inter-class variance. The detected image is shown in the figure:

7. Lane line area of interest division

Because the image pixels taken by the car camera are generally 616×808, the resolution is good, but it contains non-lane information such as vehicles, sky, street trees, houses, and vulnerable traffic. In order to improve the efficiency and accuracy of lane line detection and reduce the detection time, this paper divides and selects the regions of interest in the images collected by the vehicle camera. According to the actual situation and the analysis of a large number of experimental results, the effective information of the lane line is generally distributed in the lower part of the image, so this paper cuts the collected picture and only keeps the lower part of the image, which is beneficial to reduce the later detection calculation time and Detection accuracy. The image is cropped using the crop() function in the PIL library. After the edge detection of the image, it is necessary to further select the region of interest, and perform a bitwise AND operation on the selected region of interest information and the selected mask to obtain more accurate lane line information ,as the picture shows:

8. Hough transform line detection

After the region of interest is divided, in order to detect the straight lines in the region and realize the conversion of the image of the region of interest from the Cartesian xy coordinate system to the Hough kb coordinate system. In this paper, the probability Hough transform function is selected for straight line detection. In order to detect straight lines more clearly, refer to this blog to improve the probability Hough transform as follows :
(1) Accept the minimum length of straight lines. If the detected line is very short, the line is suppressed.
(2) Maximum pixel pitch allowed when straight lines are accepted. If a straight line is formed by multiple pixels, but the pixel distance between the pixels forming the straight line is large, it is not acceptable.
The principle of Hough transform is as follows:
(1) The essence is to detect all the pixel points on the straight line, that is, to find all the points (x0, y0) conforming to the unary linear function y=kx+b; (2) For the Cartesian coordinate system
y =kx+b is transformed into the form of b=-xk+y, and the points on the straight line in the Cartesian coordinate system all reflect the straight line passing through (k,b) in the Hough coordinate system, so that the detection point becomes a detection line;
(3) All points in the image represent various straight lines in the Hough coordinate system. If the intersection of the straight lines is found, then the straight line in the Cartesian coordinate system is found. The essence of Hough transform is to express any straight line in the plane in polar coordinate system
, and its formula is:

9. Code implementation

import cv2
import numpy as np


def make_coordinates(image,line_parameters):
    slope, intercept = line_parameters
    y1 = image.shape[0]  # Height
    y2 = int(y1 * (3/5))
    x1 = int((y1 - intercept)/slope)
    x2 = int((y2 - intercept)/slope)
    return np.array([x1,y1,x2,y2]) 

def average_slope_intercept(image,lines):
    left_fit = []
    right_fit = []
    for line in lines:
        x1,y1,x2,y2 = line.reshape(4)
        parameters = np.polyfit((x1,x2),(y1,y2),1)
        slope = parameters[0]
        intercept = parameters[1]
        if slope<0:
            left_fit.append((slope,intercept))
        else :
            right_fit.append((slope,intercept))
    left_fit_average = np.average(left_fit,axis=0)
    right_fit_average = np.average(right_fit,axis=0)
    left_line = make_coordinates(image,left_fit_average)
    right_line = make_coordinates(image,right_fit_average)
    return np.array([left_line,right_line])


def canny(image):
    gray = cv2.cvtColor(image,cv2.COLOR_RGB2GRAY)
    blur = cv2.GaussianBlur(gray,(5,5),0)   # Kernel size is 5x5
    canny = cv2.Canny(blur,50,150)
    return canny

def display_lines(image,lines):
    line_image = np.zeros_like(image)
    if lines is not None:
        for line in lines:
            x1,y1,x2,y2 = line.reshape(4)  # Reshaping all the lines to a 1D array.
            cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),10) # Draw a Blue Line(BGR in OpenCV)
    return line_image