1. Let’s first talk about the relationship between opencv and ffmpeg
1.1 The connection between them
We know that opencv is mainly used for image processing, but it also includes video decoding functions. In the video decoding part, opencv uses ffmpeg. So they are both codecs that can process images and videos. I personally feel that the focus of the two is different.
1.2 Differences between them
This is why I mentioned above that their emphasis is different.
OpenCV focuses on processing images and image-related processing applications. It can be loosely considered as PhotoShop. Opencv mainly does some recognition and tracking machine vision applications.
FFmpeg focuses on processing video and audio encoding, decoding, conversion, etc. It can be loosely considered as a combination of Format Factory and PotPlayer. The main applications are encoding and decoding and various format conversions.
2. Opencv and ffmpeg versions match
Before building the environment, I focused on the problem of matching versions between the two, because when I built the environment myself, I ran into too many pitfalls due to version mismatch, and I was afraid. So if you deal with these interference factors well at the beginning, you can save a lot of trouble later.
2.1 What problems may arise if the versions do not match?
When compiling opencv, there will be various errors that cannot recognize certain definitions, resulting in failure to compile.
Specifically:
2.1.1 Question 1
error: ‘CODEC_ID_H264’ was not declared in this scope
{ CODEC_ID_H264, MKTAG('H', '2', '6', '4') }
The solution given by netizens is to not enable ffmpeg compilation when compiling:
-D WITH_FFMPEG=OFF I have tried it and it can be compiled and passed, but at the same time it is like breaking an arm, which brings huge trouble to the use of ffmpeg function (video processing class), or simply cannot be used. Therefore, I think the best way is to match the version, compile it perfectly, and use it perfectly. This is the fundamental way to use it with peace of mind.
2.1.2 Question 2
error: 'CODEC_FLAG_GLOBAL_HEADER' was not declared in this scope
This solution is to add undefined macros. If you don't understand the underlying principles and mechanisms deeply, most people won't think of it, and only big guys can understand and find this method.
In short, basically the problem is the matching of ffmpeg and opencv versions, why don't we do it well from the beginning, right?
2.2 How to know the matching versions of ffmpeg and opencv
In order to solve this problem, I walked a long way to figure it out. It may only take a few sentences in the end, but the process is very tortuous, but it is through this process that I become more profound. At the same time, I also exercised my patience and strengthened my spirit of research, which is not bad! Let’s get to the topic!
Enter the opencv source code directory according to the method provided by netizens
It stands to reason that there should be a file called ffmpeg_version.cmake, which will list the ffmpeg version number corresponding to opencv, similar to the following figure:
Those with numbers are the versions corresponding to each component under a certain version of ffmpeg. We can go to the ffmpeg official website to download the corresponding version source code of FFmpeg .
The above picture is just to illustrate the problem, and does not specifically match the version number.
The problem now is that there is no ffmpeg_version.cmake in my file. I was confused at the time. There is no other way, so I will open ffmpeg.cmake to see if I can find any clues.
Seeing the content inside, there are several clues, which have been marked on the picture, here is a summary:
1. You can see the tag tag corresponding to the ffmpeg version number required by opencv, as well as the commit id number;
2.ffmpeg.cmake is actually downloaded from the above URL ***raw.git***;
Why is it not downloaded on my computer? After a series of inquiries, I found that the website cannot be accessed. Netizens also suggested adding IP to the host file. My solution is to use github.com instead, which is actually the same.
Enter additional links:
GitHub - opencv/opencv_3rdparty: OpenCV - 3rdparty
Click to enter
You will find that the commit id of opencv also matches.
That's it. So much about version matching. Once you understand the whole process, it will be much simpler. But if you don’t figure it out step by step, you won’t be able to experience the anxiety of encountering a problem and the feeling of enlightenment after solving the problem. .
The next step is to download the source code of ffmpeg and opencv respectively. The link to the official website of ffmpeg is provided above. Releases - OpenCV is the source code download link of opencv. Choose the version yourself. Remember that the versions between them match, otherwise I will nag so much above. What is it for? Then compile and install it, and let's continue.
3. Installation process of ffmpeg
Installation process
It is recommended to install ffpmeg first and then opencv, because installing opencv will use ffmpeg.
First unzip the ffmpeg source code, then enter the source code directory and enter the following command:
Let’s talk about it here, install the support library before entering the command
sudo apt-get install -y autoconf automake build-essential git libass-dev libfreetype6-dev libsdl2-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texinfo wget zlib1g-devThen enter the parameter configuration command
./configure --disable-x86asm --enable-shared --prefix=/usr/local/ffmpegThe meaning of the parameter is to install ffmpeg into the /usr/local/ffmpeg directory, and then enter
make
中间编译的时间会有点长，依电脑的性能而定
sudo make installAfter everything is done, you will see the ffmpeg folder in the /usr/local directory, as follows:
The tools and dynamic library of ffmpeg are all inside. In order to find the dynamic library of ffmpeg when compiling, the following processing needs to be done:
创建文件ffmpeg.conf
sudo vi /etc/ld.so.conf.d/ffmpeg.conf
输入如下内容(ffmpeg动态库的路径)
/usr/local/ffmpeg/lib
最后使能生效
sudo ldconfigWhen you see the following content, the installation is successful.
You can test to play a video using the ffplayg tool
/usr/local/ffmpeg/bin/ffplay **/**/***.mp4 (视频文件目录)Of course, we can add ffmpeg's bin to the global variable, so that it can be called at any time without adding an absolute address. Edit the profile file (sudo vi /etc/profile) and add at the end of the file:
export FFMPEG_HOME=/usr/local/ffmpeg
export PATH=$FFMPEG_HOME/bin:$PATH
The installation of ffmpeg ends here. Next is the installation of opencv. Keep going!
4. Opencv installation process
Installation process
Similarly, unzip the opencv compressed package, enter the source code directory, and create a pc_build (if you want to use cross-compilation in the future, just change to arm_build, which is too far) folder:
Here I use cmake to compile graphically. First install a cmake tool:
sudo apt-get install cmake cmake-qt-gui cmake-curses-guiThen execute it in the pc_build directory
cmake-guiThe interface appears
Here we only demonstrate the ubuntu environment, not cross-compilation.
Click finish, then click configure to configure some parameters, as shown below
The version information framed in the picture below appears, indicating that ffmpeg has been recognized. What should I do if it is not recognized? Leave it in suspense and talk about it later.
Finally click generate.
hint:
If you encounter a stuck situation during the cmake process, there are missing files that need to be downloaded, but they cannot be downloaded.
For example: opencv source code installation file download problem: ippicv_2017u3_lnx, face_landmark_model.dat, tiny-dnn
配置：打开${opencv_folder}/3rdparty/ippicv/ippicv.cmake，
第47行 "https://raw.githubusercontent.com/opencv/opencv_3rdparty/${IPPICV_COMMIT}/ippicv/"，
改成："file://${path}"，比如我的就是"file:///home/downloads/"Download address: https://github.com/opencv/opencv_3rdparty/tree/ippicv/
Download according to the version you are missing. If the download is slow, you can copy the link to Thunder to download.
Start compiling
Back to the command interface, don’t rush to type make yet. First, add the #define HAVE_PTHREAD macro definition on line 33 of the file 3rdparty/protobuf/src/google/protobuf/stubs/common.cc in the source code directory before it can be compiled. The specific reason is that the HAVE_PTHREAD macro defines the pthread library.
cd ..
// 返回 opencv 源码顶层目录
vi 3rdparty/protobuf/src/google/protobuf/stubs/common.cc
Then enter the pc_build directory and enter the command
make -j 16
漫长的等待编译编译完之后，安装
sudo make installAfter all success, the corresponding files will be generated in the /usr/local directory.
It contains header files and dynamic libraries
Also in order to find the dynamic library of opencv, do the following processing
创建文件opencv.conf
sudo vi /etc/ld.so.conf.d/opencv.conf
输入如下内容(ffmpeg动态库的路径)
/usr/local/lib
最后使能生效
sudo ldconfigNow the opencv compilation and installation is complete!
The hole dug in front
One pitfall mentioned earlier is what to do if the version of ffmpeg does not appear when running cmake. My approach is to compile first and ignore it. After the compilation is passed, copy all the pc files in ffmpeg's pkgconfig to /usr/local/lib/ In pkgconfig, this file contains the pc file of opencv.
sudo cp /usr/local/ffmpeg/lib/pkgconfig/*.pc /usr/local/lib/pkgconfigThen recompile opencv according to the above operation. Does it crash a little? There is no way.
Of course, I also thought about some operations. Some netizens said that after installing ffmpeg, do the following operations, and then compile opencv
sudo vi /etc/profile
添加
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/ffmpeg/lib/pkgconfig
让环境变量生效
source /etc/profileI have dealt with it like this, but it seems useless, and I don’t know what’s going on. If there are netizens who know, please leave a message to let me know.
Now that ffmpeg and opencv have been installed, you can't hold back your excitement and are eager to give it a try. Next, take a few small chestnuts and sharpen your knife, hahaha.
5. Practical operation
It's a mule or a horse that comes out to run around. I have done so much before just to learn and practice the operation. Next, I will write a few small programs to run the function.
5.1 Display a picture
#include "opencv2/objdetect.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/opencv.hpp"
#include "opencv2/core/core.hpp"
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc,char** argv)
{
Mat image = imread("11.png", 1 );//加载
cv::namedWindow("picture",CV_WINDOW_AUTOSIZE);
cv::imshow("picture", image);//显示图片
waitKey(5000);//等待
return 0;
}
5.2 Play a video
#include<opencv2/opencv.hpp>
using namespace cv;
int main()
{
//从摄像头读取视频
VideoCapture capture("video.mp4");
//循环显示每一帧
while (1)
{
Mat frame;//定义一个Mat变量，用于存储每一帧的图像
capture >> frame;//读取当前帧
imshow("读取视频帧", frame);//显示当前帧
waitKey(30);//延时30ms
}
system("pause");
return 0;
}
5.3 Use the laptop’s built-in camera to shoot a video and save it
#include "opencv2/objdetect.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/opencv.hpp"
#include "opencv2/core/core.hpp"
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc,char** argv)
{
//打开电脑摄像头
VideoCapture capture(0);
if(!capture.isOpened()){
cout<<"error"<<endl;
waitKey(0);
return 0;
}
//获得分辨率
int w = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
int h = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));
cout<<"w="<<w<<endl;
cout<<"h="<<h<<endl;
Size videoSize(w,h);
VideoWriter writer;
writer.open("video.mp4",CV_FOURCC('M','J','P','G'),25,videoSize);
if(!writer.isOpened()){
cout<<"fail"<<endl;
return -1;
}
Mat frame;
int key;
char startorstop=1;
char flag=0;
while(1){
capture >> frame;
if(key == 32){//按下空格开始录制、暂停录制 可以来回切换
startorstop = 1-startorstop;
if(startorstop == 0){
flag = 1;
}
}
if(key == 27){//按下ESC退出整个程序，保存视频文件到磁盘
cout << "exit" << endl;
break;
}
if(startorstop == 0 && flag == 1){
writer << frame;
cout << "recording" << endl;
}else if(startorstop == 1){
flag = 0;
cout << "end recording" << endl;
}
imshow("picture",frame);
key=waitKey(100);
cout<<"key="<<key<<endl;
}
capture.release();
writer.release();
destroyAllWindows();
return 0;
}
compile script
g++ test.cpp -o test -L/usr/local/lib -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lopencv_imgcodecs
Okay, it comes to an end here. I have stepped on most of the pits that I feel I have to step on along the way. It is not easy! So feel free to use your creativity on the road ahead!
If there are any errors, please correct me!