Note: There are already many tutorials and books about OpenCV, so I don't want to repeat what others have already done. How to systematically and comprehensively master OpenCV? I think this is what everyone who learns OpenCV wants to do. After all, OpenCV is just a digital image processing function library. To fully master the use of OpenCV, you only need to understand what functions there are and how to use each function. What this series of tutorials want to accomplish is to let everyone systematically understand the functions of OpenCV and become proficient in the role and use of each function. Therefore, I won't take the time to go into detail about the algorithmic principles of each function. I focus on what each function does and how to use it, while demonstrating it with detailed sample code. Sample codes include c++, python, emgucv, and OpenCVsharp.
1. Introduction and functional modules of OpenCV
1. Introduction to OpenCV
OpenCV (Open Source Computer Vision Library) is a widely used open source computer vision library designed to provide rich image and video processing functions. It was originally developed by Intel in 1999 and has evolved into a global open source project with contributions and support from many developers. OpenCV can be called through C++, Python, Java and other programming languages, allowing developers to develop image processing and computer vision applications on different platforms. As a comprehensive and powerful computer vision library, OpenCV contains hundreds of functions and tools for image processing, feature detection, object recognition, video analysis and other fields. Whether you want to read and display images, perform image filtering, edge detection, image segmentation, feature extraction, or target tracking, OpenCV provides corresponding feature-rich APIs.
For image processing, OpenCV provides a variety of functions, such as color space conversion, image filtering (smoothing, sharpening), morphological operations, image transformation (rotation, scaling), image registration, etc. These functions are useful for different image processing tasks. In addition, OpenCV also supports advanced functions such as histogram operation, binarization, shape descriptor calculation, image contour extraction, etc. for more complex image processing and analysis. In terms of feature detection and description, OpenCV provides the implementation of a series of classic algorithms, such as Harris corner detection, SIFT (Scale Invariant Feature Transform), SURF (Speed Feature Transform), ORB (Oriented FAST and Rotated BRIEF), etc. These algorithms can be used to detect key points in images, generate descriptors, and perform feature matching and object recognition. When it comes to video processing, OpenCV also provides rich functionality. It can read video data from cameras or video files and perform real-time analysis and processing. You can use OpenCV to capture video streams, extract video frames, perform video encoding and decoding, calculate optical flow, track moving targets, etc.
In addition, OpenCV also provides advanced computer vision functions such as camera calibration, stereo vision, motion estimation, and optical flow analysis, as well as support for real-time image processing and video processing. Developers can use these functions to build various applications, such as robot visual navigation, autonomous driving, security monitoring systems, virtual reality, etc. In recent years, with the widespread application of deep learning, OpenCV also provides advanced functions for neural networks and deep learning. By integrating the Deep Learning module, OpenCV supports common deep learning frameworks such as TensorFlow, PyTorch and Caffe, allowing developers to use neural networks for tasks such as face detection, object recognition and semantic segmentation. This brings more possibilities for the development of computer vision.
OpenCV has been widely used in various fields. In the medical field, OpenCV can help with medical image analysis, lesion detection and diagnosis. In the industrial field, OpenCV can be applied to quality control, product inspection and machine vision systems. In the entertainment field, OpenCV can be used to implement augmented reality, virtual reality and interactive games. OpenCV has strong cross-platform capabilities and can run on Windows, Linux, macOS and other operating systems. It can also be integrated with other popular computer vision libraries and tools, such as NumPy, SciPy, Matplotlib, etc., providing developers with more choices and richer functions.
In short, OpenCV is a powerful and widely used computer vision library that provides rich image processing and computer vision functions to help developers solve various image processing and computer vision problems. Whether it is academic research, education or commercial applications, OpenCV is an indispensable tool to create a better visual experience and intelligent applications for people.
2. OpenCV function module
The organizational relationship of OpenCV is shown in the figure below:
The functional modules of OpenCV are divided into two parts, one is the basic module and the other is the extension module.
(1) The basic module mainly has the following functional modules
calib3d : camera calibration related
core ; core functional module, mainly including the basic structure and basic operations of the opencv library.
dnn : deep learning module
features2d : two-dimensional feature detection point detection, description, matching, etc.
flann ; nearest neighbor matching, clustering, etc.
gapi : This module accelerates the image processing algorithm.
highgui : image display, button, mouse and other operations.
imgcodecs : responsible for image file reading and writing, such as image reading and saving
imgproc : image processing function
ml : machine learning related algorithm
objdetect : target detection, including Cascade face detector; latent SVM; HOG, etc.
photo : used for computational photography processing and recovery of photos Algorithm
stitching : Image splicing, using image feature points for image splicing
video : Video processing, including background segmentation, video tracking, etc.
videoio : Responsible for reading and writing video files.
Among the basic modules, the core functional module is the one used most. , image processing module, feature detection module; followed by target detection, camera calibration, machine learning and other functions. The image display function is usually used to view the processing results after the above functions are completed.
(2) Extension module
In addition to the above basic modules, OpenCV provides more extension modules. The
extension modules are included in contrib. The extension modules are supplements to the basic modules and also include some patented algorithms. Extension modules may differ from version to version, and new versions may add some new functions. Even so, the usage of each module is similar. The extension modules provided by OpenCV4.0 are as follows:
aruco : ArUco and ChArUco tags. Includes ArUco markers and ChArUco markers for augmented reality
bgsegm : background segmentation. Improved adaptive background blending model for real-time human tracking under different lighting conditions.
ccalib : Custom calibration. Modes for 3D reconstruction, panoramic camera calibration, random pattern calibration and multi-camera calibration.
cnn_3dobj : Deep object recognition and pose. Build, train, and test CNN models for visual object recognition and pose using the Caffe deep neural network library.
cuda : A module that uses GPU to process images, including the following parts:
cudaarithm : a library that performs mathematical calculations on the CUDA architecture
cudaabgsegm : performs background segmentation on the CUDA architecture
cudacodec : performs encoding and decoding on the CUDA architecture
cudafeatures2d : performs features on the CUDA architecture Detect
cudafilters : Image filtering on CUDA architecture
cudaimgproc : Image processing on CUDA architecture
cudalegacy: Traditional processing support on CUDA architecture
cudaobjdetect : Object detection on CUDA architecture
cudaoptflow : Optical flow calculation on CUDA architecture
cudastereo : A module for stereo vision tasks based on CUDA technology, which can accelerate the stereo vision algorithm Calculation process, improve processing speed and efficiency
cudawarping : Use NVIDIA's CUDA technology to perform image distortion correction and image reprojection calculations to achieve more efficient processing speed and better performance
cudev ; provides certain functions in OpenCV CUDA implementation to accelerate these operations. It can be used with other OpenCV modules to take advantage of the parallel computing power of the GPU to increase the speed of image processing and calculations.
cvv : debugger, pops up a graphical user interface (GUI), capable of interactive visual debugging computer vision program
datasets : code used to read existing computer vision databases, and shows how to use these readers to process data Set of training, testing, and running examples.
dnn_objdetect : A deep learning module for object detection and recognition
dnns_easily_fooled : A code or technique capable of fooling deep neural networks (DNNs). This method uses activation values in the network to trick the network into identifying the wrong target or identifying one target as another.
dpm : an algorithm for object recognition based on Felzenszwalb's cascade detector and adding deformable object detection and recognition
face : Face recognition, including Eigen, Fisher and Local Binary Pattern Histogram (LBPH) methods.
freetype : Function module for drawing text on images. It provides some functions and classes that can be used to add text to images, including selecting fonts, setting font size, specifying color, position and alignment, etc.
fuzzy : Image processing with fuzzy mathematics
hdf ::hierarchical data storage. This module contains input/output (I/O) routines for hierarchical data formats designed to store large amounts of data.
hfs : hierarchical feature selection, an algorithm for feature selection and dimensionality reduction, which can be used in computer vision and pattern recognition tasks
img_hash : hash algorithm to calculate the hash value of the image, used to compare the similarity of two images
line_descriptor : Function module for line segment extraction and matching. Provides methods to extract, describe and match line segments from images, using binary descriptors to represent line segments
matlab : MATLAB interface
optflow : Optical flow algorithm. Algorithms for running and evaluating deepflow, simpleflow, sparsetodenseflow and motion templates (contour optical flow).
ovis : Toolkit for rendering, visualization and interactive manipulation. It is a functional module based on OpenGL for creating and displaying 3D scenes, models and virtual reality (VR) environments
phase_unwrapping : a functional module
plot for phase unwrapping: Allows plotting data in one or two dimensions.
reg : image registration. Pixel-based image registration for precise alignment
rgbd : RGB depth processing module. Linemod 3D object recognition.
saliency : saliency API. Used to determine where the human eye focuses in a scene. Computational routines with static, kinematic, and "objectness" salience.
sfm : This module contains algorithms for 3D reconstruction from 2D images. The core of this module is a lightweight version of Libmv
shape : providing some shape-related functions and operations
stereo : stereo matching. Stereo matching using different descriptors: Census, CS-Census, MCT, BRIEF and MV.
structured_light : Generating structured light patterns, projecting structured light, decoding structured light images, three-dimensional reconstruction, etc.
superres : Super-resolution image processing technology, a process designed to restore low-resolution images to high-resolution by using algorithms and image processing methods
surface_matching : Functions for three-dimensional object detection and localization
text : Visual text matching, detecting text, segmenting words, and recognizing text in a visual scene tracking : Vision-based object tracking
videostab
: Video stabilization function. Used to reduce severe shaking or jittering in videos caused by camera shake or motion.
viz: Provides three-dimensional visualization function. This module allows the creation, rendering and interactive display of three-dimensional scenes and objects in applications.
xfeatures2d : Extended capabilities for feature detection and descriptors, including some experimental and paid patented feature detector/descriptor algorithms
ximgproc : Extended image processing. Including algorithms such as structured forests, domain transformation filters, guided filters, adaptive manifold filters, joint bilateral filters, and superpixels.
xobjdetect : Enhanced 2D object detection. Compute local binary patterns as integral features for 2D object detection using Waldboost cascade sums.
xphoto : Extended computational photography. Provides additional photo processing algorithms: color balance, noise reduction, repair processing and other functions.
2. Commonly used function directory
Although OpenCV provides numerous image processing functions, many of them may not be used by most people. OpenCV is an image library that includes multiple visual directions. For each person's own direction of interest, just learn to use the corresponding function function. For example, people who work in the field of industrial vision inspection may pay more attention to conventional image processing methods, machine learning, deep learning and other functions; people who work in 3D vision may pay more attention to the 3D functions of OpenCV; people who focus on visual algorithms may pay more attention to Focus on the latest image processing algorithms in a certain direction. People who do face recognition may focus on the face recognition function of OpenCV.
However, no matter what functions are implemented using OpenCV, some commonly used image processing functions still need to be familiar with how to use them. The following is a list of commonly used OpenCV functions. If you are familiar with the use of these functions, you can quickly implement the functions you want.
三、imread、imwrite、imshow、VideoCapture、VideoWriter
imread, imwrite, imshow, and VideoCapture are the most basic functions provided by OpenCV, including reading, displaying and saving images and videos.
1. imread function
Used to read images, defined as follows:
Mat imread( const String& filename, int flags = IMREAD_COLOR);
Parameters:
Filename: file name
Flags: read identification, enumeration type, can take the following values
enum ImreadModes { IMREAD_UNCHANGED = -1, //Return the original channel image IMREAD_GRAYSCALE = 0, //Return the grayscale image IMREAD_COLOR = 1, //Return the three-channel BGR image IMREAD_ANYDEPTH = 2, //Return the 16-bit/32-bit image when the input has the corresponding depth, otherwise it will It is converted to 8-bit IMREAD_ANYCOLOR = 4, //Read the image in any possible color format IMREAD_LOAD_GDAL = 8, //Use the gdal driver to load the image IMREAD_REDUCED_GRAYSCALE_2 = 16, //Always a grayscale image, and the image size is the original 1/2 IMREAD_REDUCED_COLOR_2 = 17, //Always a three-channel BGR image, and the image size is 1/2 of the original IMREAD_REDUCED_GRAYSCALE_4 = 32, //Always a grayscale image, and the image size is 1/4 of the original. IMREAD_REDUCED_COLOR_4 = 33, //Always a three-channel BGR image, and the image size is 1/4 of the original.
IMREAD_REDUCED_GRAYSCALE_8 = 64, //Always a grayscale image, and the image size is 1/8 of the original.
IMREAD_REDUCED_COLOR_8 = 65, //Always a three-channel BGR image, and the image size is 1/8 of the original.
IMREAD_IGNORE_ORIENTATION = 128 //The image will not be rotated according to the EXIF direction flag.
};
It is more common for the flags parameter to be "0" or "1". If "0" is set, the image obtained is a grayscale image. Take "1" to get a three-channel color image.
2. imwrite function
Used to save images, defined as follows:
bool imwrite( const String& filename, InputArray img, const std::vector& params = std::vector()); Parameters:
Filename
: file name
img: image to be saved
params: expressed as specific The parameter encoding saved in the format usually uses the default value directly.
3. imshow function
Used to display images, often used together with the namedWindows function. The namedWindows function is used to create a display form. If the namedWindows function is not used, the imshow function will use a form to display the image by default. The function is defined as follows:
void imshow(const String& winname, InputArray mat)
Parameters:
winname: The name of the window that displays the image. You can use the namedWindow function to create a display window.
mat: Mat image to be displayed.
If you use the namedWindow function to create a form, the namedWindow function is defined as follows:
void namedWindow(const String& winname, int flags = WINDOW_AUTOSIZE);
where the parameter winname is used to set the window name, and the flags parameter represents the style of the window. There are the following values to choose from:
WINDOW_NORMAL, WINDOW_AUTOSIZE, WINDOW_FREERATIO, WINDOW_KEEPRATIO, WINDOW_FULLSCREEN, WINDOW_GUI_EXPANDED, WINDOW_GUI_NORMAL.
Sample code:
c++: read and write images, display images
#include <opencv2/opencv.hpp>
#include <iostream>
int main(int argc, char** argv)
{
Mat ReadImage = imread("E:/TestImage/data/aloeL.jpg", 1);
namedWindow("ReadImage", 0);
imwrite("E:/TestImage/test.png", ReadImage);
imshow("ReadImage", ReadImage);
waitKey(0);
return 0;
}
python: read, write and display images
import cv2.cv2 as cv2
#如果支持中文路径,可以采用以下方式
#ReadImage=cv2.imread("E:/TestImage/data/aloeL.jpg",1)
#如果不支持中文，请采用以下方式
ReadImage = cv2.imdecode(np.fromfile("E:/TestImage/data/aloeL.jpg", dtype =np.uint8),1)
#如果支持中文路径,可以采用以下方式
#cv2.imwrite("E:/TestImage/test.png",ReadImage)
#如果不支持中文，请采用以下方式
fileName ="E:/TestImage/test.png"
cv2.imencode('.png',ReadImage)[1].tofile(fileName)
cv2.namedWindow("ReadImage",cv2.WINDOW_NORMAL)
cv2.imshow("ReadImage",ReadImage)
cv2.waitKey(0)
Note: If reading and writing images does not support Chinese paths, OpenCV also provides two functions, imdecode and imencode, for reading and writing images using encoding and decoding methods.
emgucv: read and write images, display images
using Emgu.CV.CvEnum;
namespace test
{
class Program
{
static void Main(string[] args)
{
Mat ReadImage = CvInvoke.Imread("E:/TestImage/data/aloeL.jpg", (ImreadModes)0);
CvInvoke.Imwrite("E:/TestImage/test.png", ReadImage);
CvInvoke.NamedWindow("ReadImage", 0);
CvInvoke.Imshow("ReadImage", ReadImage);
CvInvoke.WaitKey(0);
return;
}
}
}
OpenCVsharp: read and write images, display images
using OpenCvSharp;
namespace OpenCVSharpTestFunction
{
class Program
{
static void Main(string[] args)
{
Mat ReadImage = Cv2.ImRead("E:/TestImage/data/aloeL.jpg", (ImreadModes)0);
Cv2.ImWrite("E:/TestImage/test.png", ReadImage);
Cv2.NamedWindow("ReadImage", 0);
Cv2.ImShow("ReadImage", ReadImage);
Cv2.WaitKey(0);
return;
}
}
}
4. VideoCapture class
VideoCapture is a class provided by OpenCV for reading videos. The camera or video file can be opened through the open function. This class also provides the set function, which is a function for setting the camera image collection parameters.
c++: read video
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
cv::VideoCapture cap; //声明相机捕获对象
cap.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('M', 'J', 'P', 'G'));
cap.set(cv::CAP_PROP_FRAME_WIDTH, 640); //图像的宽，需要相机支持此宽
cap.set(cv::CAP_PROP_FRAME_HEIGHT, 480); //图像的高，需要相机支持此高,图像分辨率640×480
int deviceID = 0; //相机设备号
cap.open(deviceID); //打开相机，如果打开视频文件，直接输入视频文件路径和文件名
if (!cap.isOpened()) //判断相机是否打开
{
std::cerr << "ERROR!!Unable to open camera\n";
return -1;
}
cv::Mat img;
while (true)
{
cap >> img; //以流形式捕获图像
cv::namedWindow("example", 1); //创建一个窗口用于显示图像，1代表窗口适应图像的分辨率进行拉伸。
if (img.empty() == false) //图像不为空则显示图像
{
cv::imshow("example", img);
}
int key = cv::waitKey(30); //等待30ms
if (key == int('q')) //按下q退出
{
break;
}
}
cap.release(); //释放相机捕获对象
cv::destroyAllWindows(); //关闭所有窗口
return 0;
}
python: read video
import numpy as np
import cv2.cv2 as cv2
#读取摄像头图像
# cap = cv2.VideoCapture(0)
cap = cv2.VideoCapture('E:/TestImage/test.mp4')
while cap.isOpened():
ret, frame = cap.read()
if ret:
# 在这里进行对帧的处理，比如展示、分析等
cv2.imshow("test",frame);
# 按下 'q' 键退出循环
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap.release()
cv2.destroyAllWindows()
emgucv: read video
using Emgu.CV;
using System;
namespace test
{
class Program
{
static void Main(string[] args)
{
// 视频文件路径
string videoPath = "E:/TestImage/test.mp4";
VideoCapture capture = new VideoCapture(videoPath);
if (!capture.IsOpened)
{
Console.WriteLine("无法打开视频文件！");
return;
}
// 创建一个窗口显示视频帧
CvInvoke.NamedWindow("Video", 0);
while (true)
{
// 从视频中读取下一帧
Mat frame = capture.QueryFrame();
if (frame == null)
break;
CvInvoke.Imshow("Video", frame);
if (CvInvoke.WaitKey(30) == 27)
break;
}
// 关闭视频窗口
CvInvoke.DestroyWindow("Video");
}
}
}
OpenCVsharp: read video
using OpenCvSharp;
using System;
namespace OpenCVSharpTestFunction
{
class Program
{
static void Main(string[] args)
{
string videoPath = "E:/TestImage/test.mp4";
VideoCapture capture = new VideoCapture(videoPath);
if (!capture.IsOpened())
{
Console.WriteLine("无法打开视频文件！");
return;
}
Cv2.NamedWindow("Video", WindowMode.AutoSize);
while (true)
{
Mat frame = new Mat();
if (!capture.Read(frame))
break;
Cv2.ImShow("Video", frame);
if (Cv2.WaitKey(30) == 'q')
break;
}
// 关闭视频窗口
Cv2.DestroyWindow("Video");
}
}
}
5. VideoWriter class
VideoWriter is a class provided by OpenCV for saving videos. VideoWriter is similar to VideoCapture.
c++: writing video
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
VideoCapture cap;
cap.open(0); // 打开摄像头，如果打开视频文件，直接输入文件路径和文件名
if (!cap.isOpened())
{
cout << "Couldn't open capture." << endl;
return -1;
}
double fps = 30;
cv::Size size(
(int)cap.get(cv::CAP_PROP_FRAME_WIDTH), (int)cap.get(cv::CAP_PROP_FRAME_HEIGHT)
);
cv::VideoWriter writer;
writer.open("E:/test/test.avi", VideoWriter::fourcc('M', 'J', 'P', 'G'), fps, size);
if (!writer.isOpened())
{
cout << "Could not open the output video file for write" << endl;
return -1;
}
cv::Mat bgr_frame;
for (;;)
{
cap >> bgr_frame;
if (bgr_frame.empty()) break;
writer << bgr_frame;
char c = cv::waitKey(10);
if (c == 27) break;
}
return 0;
}
python: write video
import cv2.cv2 as cv2
cap = cv2.VideoCapture('E:/TestImage/test.mp4')
output_file = 'E:/TestImage/output.mp4'
codec = cv2.VideoWriter_fourcc(*'mp4v')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
# 创建VideoWriter对象
out = cv2.VideoWriter(output_file, codec, 24,(frame_width, frame_height))
cv2.namedWindow("test", 0)
while cap.isOpened():
# 在这里获取当前帧
ret, frame = cap.read()
if frame is not None:
# 将帧写入视频
out.write(frame)
cv2.imshow("test", frame)
# 按下 'q' 键退出循环
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
out.release()
cv2.destroyAllWindows()
emgucv: write videos
using Emgu.CV.CvEnum;
using Emgu.CV;
using System;
using System.Drawing;
namespace test
{
class Program
{
static void Main(string[] args)
{
string videoPath = "E:/TestImage/test.mp4";
VideoCapture capture = new VideoCapture(videoPath);
if (!capture.IsOpened)
{
Console.WriteLine("无法打开视频文件！");
return;
}
string outputPath = "E:/TestImage/output.mp4";
int width = (int)capture.GetCaptureProperty(CapProp.FrameWidth);
int height = (int)capture.GetCaptureProperty(CapProp.FrameHeight);
VideoWriter writer = new VideoWriter(outputPath, 24, new Size(width, height), true);
if (!writer.IsOpened)
{
Console.WriteLine("无法创建视频文件！");
return;
}
// 创建一个窗口显示视频帧
CvInvoke.NamedWindow("Video", 0);
while (true)
{
using (Mat frame = capture.QueryFrame())
{
if (frame == null)
break;
writer.Write(frame);
CvInvoke.Imshow("Video", frame);
if (CvInvoke.WaitKey(30) == 'q')
break;
}
}
}
}
}
OpenCVsharp:Write video
using OpenCvSharp;
using System;
namespace OpenCVSharpTestFunction
{
class Program
{
static void Main(string[] args)
{
string videoPath = "E:/TestImage/test.mp4";
VideoCapture capture = new VideoCapture(videoPath);
if (!capture.IsOpened())
{
Console.WriteLine("无法打开视频文件！");
return;
}
string outputPath = "E:/TestImage/output.mp4";
int width = capture.FrameWidth;
int height = capture.FrameHeight;
VideoWriter writer = new VideoWriter(outputPath, VideoWriter.FourCC(@"MP4V"), 24, new Size(width, height));
if (!writer.IsOpened())
{
Console.WriteLine("无法创建视频文件！");
return;
}
Cv2.NamedWindow("Video", WindowMode.FreeRatio);
try
{
while (true)
{
Mat frame = new Mat();
if (!capture.Read(frame))
break;
writer.Write(frame);
Cv2.ImShow("Video", frame);
if (Cv2.WaitKey(30) == 'q')
break;
}
// 关闭视频窗口
Cv2.DestroyWindow("Video");
}
catch
{
}
}
}
}
The OpenCV learning platform is an interactive and visual operating platform designed to allow everyone to quickly master OpenCV. The code is automatically generated and you can quickly master the use of OpenCV without writing code. Supports c++, python, opencvshape, emgucv, and you can use it by directly copying the generated code.
Link: https://pan.baidu.com/s/1XyotEmXLfD-XCNCYOiVUYg
Extraction code: aj9d