1. Introduction and installation of OpenCV
This part is to understand the introduction of OpenCV (Open Source Computer Vision Library). OpenCv can run on multiple platforms. It is lightweight and efficient. It consists of a series of C functions and a small number of C++ classes. It provides Python, Ruby, MATLAB, etc. The interface of the language, so when learning, pay attention to the language implementation related issues of consulting the information.
In addition to installing OpenCV related libraries at this stage, it is recommended to collect official websites, official manuals, and official introductory tutorials. These are the best learning materials.
After the module is installed, it is necessary to focus on testing whether OpenCV is installed successfully, and the installed version can be queried through Python.
2. Introduction to OpenCV module
First grasp the modules of OpenCV from a global perspective. For example, the following modules, you need to find the application scenarios and introduction of the following modules.
core、imgproc、highgui、calib3d、features2d、contrib、flann、gpu、legacy、ml、objdetect、photo、stitching。
Organize the core functions of each module, and complete the first OpenCV case, read and display pictures.
3. OpenCV image reading, displaying and saving
After installing OpenCV, start learning from image acquisition, including local loading of pictures, camera acquisition of pictures, video acquisition, image creation and so on.
Only after the image is acquired first, can the image be manipulated, information extracted, result output, image displayed, and image saved.
For an image, the steps of reading and displaying it in OpenCV are as follows, and you can correspond to the code.
image reading;
window creation;
Image display;
image storage;
Resource release.
The functions that need to be learned are cv2.imread() , cv2.namedWindow() , cv2.imshow() , cv2.imwrite() , cv2.destroyWindow() , cv2.destroyAllWindows() , cv2.imshow() , cv2. cvtColor() , cv2.imwrite() , cv2.waitKey() .
4. Camera and video reading, saving
The first one is to focus on learning the VideoCapture class. The commonly used methods of this class are:
open() function;
isOpened() function;
release() function;
grab() function;
retrieve() function;
get() function;
set() function;
In addition to reading video, you also need to master the VideoWriter class provided by Opencv to save video files.
After learning the relevant knowledge, you can conduct such an experiment to save a video as a picture frame by frame.
5. OpenCV common data structure and color space
The classes to be mastered in this part include Point , Rect , Size , and Scalar . In addition, numpy is used to operate images in Python , so it is recommended to learn numpy -related knowledge points in advance for better results.
Commonly used color spaces in OpenCV include BGR color space, HSV/HLS color space, and Lab color space, all of which need to be understood, and BGR color space is given priority.
6. OpenCV common drawing functions
After mastering the usage of the following functions, you can skillfully draw graphics in Opencv.
cv2.line()；
cv2.circle()；
cv2.rectangle()；
cv2.ellipse()；
cv2.fillPoly()；
cv2.polylines()；
cv2.putText()。
7. Mouse and slider for OpenCV interface event operation
The first function to master is the mouse operation message callback function, cv2.setMouseCallback() , and the slider involves two functions, namely: cv2.createTrackbar() and cv2.getTrackbarPos() .
After mastering the above content, two cases can be realized, one is to drag the mouse on a picture to select a frame area to take a screenshot, and the other is to use the slider to make the video play at double speed.
8. Image pixel, channel separation and merging
Understand the image pixel matrix, familiar with the pixel composition of the picture, you can access the pixel value of the specified pixel and modify it.
Channel separation function cv2.split() , channel merging function cv2.merge() .
9. Image logical operation
To master the calculation between images, the functions involved are as follows:
cv2.add()；
cv2.addWeighted()；
cv2.subtract()；
cv2.absdiff()；
cv2.bitwise_and()；
cv2.bitwise_not()；
cv2.bitwise_xor()。
You can also study image multiplication and division.
10. Image ROI and mask mask
This part belongs to the key knowledge in OpenCV. The first one is the region of interest ROI, and the second one is the mask mask (mask) operation.
While working on the ROI section, you can also learn about light and dark copies of images.
11. Image geometric transformation
Image geometric transformation is still the learning and understanding of basic functions, and the content involved is as follows:
Image scaling cv2.resize();
Image translation cv2.warpAffine();
Image rotation cv2.getRotationMatrix2D();
Image transposition cv2.transpose();
image mirroring cv2.flip();
Image remapping cv2.remap().
12. Image filtering
Understand what is filtering, high-frequency and low-frequency filtering, and image filtering functions.
Linear filtering: box filtering, mean filtering, Gaussian filtering,
Nonlinear filtering: median filtering, bilateral filtering,
Box filter cv2.boxFilter();
mean filter cv2.blur();
Gaussian filter cv2.GaussianBlur();
Median filter cv2.medianBlur();
Bilateral filtering cv2.bilateralFilter().
13. Image fixed threshold and adaptive threshold
Image thresholding is an important basic part of image processing. It is widely used. It can segment different parts of the image according to the difference in grayscale. The image processed by thresholding is generally a single-channel image (grayscale image). There are two core functions to master:
Fixed threshold: cv2.threshold();
Adaptive threshold: cv2.adaptiveThreshold().
14. Image dilation and erosion
Dilation and erosion are morphological operations, which are a series of image processing operations based on the shape of the image.
Dilation and corrosion are based on the highlighted part (white). Expansion is to expand the highlighted part, similar to "field expansion", and corrosion is to corrode the highlighted part, similar to "field is eroded".
Applications and Functions of Dilation Corrosion:
eliminate noise;
Split independent elements or join adjacent elements;
Find the obvious maximum and minimum value areas in the image;
Find the gradient of the image;
The core functions that need to be mastered are as follows:
Dilate cv2.dilate();
Erode cv2.erode().
Other morphological operations, such as opening operation, closing operation, top hat, black hat, and morphological gradient, are all based on expansion and corrosion, and are operated using the cv2.morphologyEx() function.
15. Edge detection
Edge detection can extract important image contour information, reduce image content, and can be used for image segmentation, feature extraction and other operations.
General steps for edge detection:
Filtering: filter out the influence of noise on the detection edge;
Enhancement: It can highlight the intensity changes of the pixel neighborhood - gradient operator;
Detection: Threshold method to identify edges;
Commonly used edge detection operators:
Canny operator, Canny edge detection function cv2.Canny();
Sobel operator, Sobel edge detection function cv2.Sobel();
Scharr operator, Scharr edge detection function cv2.Scahrr();
Laplacian operator, Laplacian edge detection function cv2.Laplacian().
16. Hough transform
Hough Transform (Hough Transform) is a feature extraction technique in image processing. In this process, in a parameter space, by calculating the local maximum value of the cumulative result, a set conforming to the specific shape is obtained as the result of Hough Transform. .
Functions to learn in this part:
Standard Hough transform, multi-scale Hough transform cv2.HoughLines();
Cumulative probability Hough transform cv2.HoughLinesP();
Hough circle transformation cv2.HoughCricles().
17. Image histogram calculation and drawing
First master the related concepts of the histogram, and then master the core functions, and finally draw the histogram through the matplotlib module. The function used to calculate the histogram is cv2.calcHist() .
Histogram related applications:
Histogram equalization cv2.equalizeHist();
Histogram comparison cv2.compareHist();
Backprojection cv2.calcBackProject().
18. Template matching
Template matching is the technique of finding the most matching (similar) part of an image to another template image.
The functions used in the core are as follows:
Template matching cv2.matchTemplate();
Matrix normalization cv2.normalize();
Find the most value cv2.minMaxLoc().
19. Contour search and drawing
The core is to understand that in OpenCV, finding contours is like finding white objects on a black background.
Common functions:
Find contour cv2.findContours();
Draw contours cv2.drawContours().
Finally, you should master the operation for each contour.
20. Contour feature attributes and applications
This part of the content is more important, and there are more knowledge points. The core content and functions are as follows:
Find the convex hull cv2.convexHull() and convexity detection cv2.isContourConvex();
Outline circumscribed rectangle cv2.boundingRect();
Contour minimum circumscribed rectangle cv2.minAreaRect();
Contour minimum circumscribed circle cv2.minEnclosingCircle();
Contour ellipse fitting cv2.fitEllipse();
Approximate polygonal curve cv2.approxPolyDP();
Calculate the contour area cv2.contourArea();
Calculate the contour length cv2.arcLength();
Calculate the distance and position relationship between the point and the contour cv2.pointPolygonTest();
The shapes match cv2.matchShapes().
21. Advanced part - watershed algorithm and image inpainting
Master the principle of watershed algorithm and the core function cv2.watershed() .
It can expand and supplement the image inpainting technology and related functions cv2.inpaint() . After learning, you can try the application of portrait freckle removal.
22. GrabCut & FloodFill image segmentation, corner detection
This part of the content requires some image professional background knowledge. First master the relevant concept knowledge, and focus on learning related functions.
GrabCut algorithm cv2.grabCut();
Flood filling algorithm cv2.floodFill();
Harris corner detection cv2.cornerHarris();
Shi-Tomasi corner detection cv2.goodFeaturesToTrack();
Sub-pixel corner detection cv2.cornerSubPix().
23. Feature detection and matching
The detection and matching of feature points is one of the very important technologies in computer vision, and it is widely used in object recognition, visual tracking, 3D reconstruction and other fields.
OpenCV provides the following feature detection methods:
“FAST” FastFeatureDetector；
“STAR” StarFeatureDetector；
"SIFT" SIFT(nonfree module) Opencv3 removed, need to call xfeature2d library;
"SURF" SURF(nonfree module) Opencv3 removed, need to call xfeature2d library;
"ORB" ORB Opencv3 removed, need to call xfeature2d library;
“MSER” MSER；
“GFTT” GoodFeaturesToTrackDetector；
“HARRIS” (compound Harris detector);
“Dense” DenseFeatureDetector；
“SimpleBlob” SimpleBlobDetector。
24. Moving object tracking and face recognition in the application part of OpenCV
Understand what moving object detection is. Commonly used moving object detection methods in OpenCV include background subtraction, frame difference method, and optical flow method . Commonly used tracking algorithms include meanShift , camShift , particle filter , optical flow method , etc.
meanShift tracking algorithm cv2.meanShift();
CamShift tracking algorithm cv2.CamShift().
If you learn face recognition, the knowledge points involved are:
Face detection: Find and identify the face position from the image;
Face recognition: distinguish the name or other information of the person from the localized face area;
machine learning.