One, ffmpeg introduction
FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. Use LGPL or GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library libavcodec. In order to ensure high portability and codec quality, many codes in libavcodec are developed from scratch.
The block diagram is as shown:
Reference materials: Add group 832218493 can receive related materials for free, C++, linux,
FFMPEG video and audio codec zero-based learning method
FFmpeg uses a small note
FFmpeg official document
How to install FFmpeg program on Windows
ffmpeg tutorial (1) Basic installation
Get the blog update reminder as soon as possible, and share more technical information, welcome to follow me,
1. Directly help you answer ffmpeg related questions c++. linux, TCP.
2. Obtain technical articles in more than ten fields in the industry in the first time
3. Ask questions for doubtful points in the article, reply to you as soon as possible, and help you patiently answer
4. Let you and the original author become good friends and expand your network resources
2. Basic knowledge of encoding and decoding
(1) Package format
The so-called encapsulation format refers to the combination of audio and video formats. For example, the most common encapsulation formats are mp4, mp3, flv, etc. Simply put, the audio and video files with a suffix that we usually come into contact with are all a kind of encapsulation format.
(2) Encoding format
Take mp4 as an example, it should usually contain video and audio. The video encoding format is YUV420P, and the audio encoding format is PCM. Take the YUV420 encoding format as an example. We know that the usual image is displayed as RGB (red, green and blue three primary colors). When the video is compressed, the RGB representing each frame will be compressed to YUV, and then according to the key frame (I frame), transition frame (P frame or B frame). Frame) for calculation and encoding. The decoding process is just the opposite. The decoder reads the I frame, calculates and decodes the P frame and the B frame according to the I frame. And finally restore the RGB data of each frame according to the preset FPS of the video file. Finally pushed to the graphics card. So usually the encoding process we talk about includes: image capture, transcoding, encoding and then packaging.
(3) What is the difference between video decoding and audio decoding
FPS is a definition in the field of graphics, which refers to the number of frames transmitted per second, in general, refers to the number of frames of animation or video. If the FPS is too low, the picture will feel that the flicker is not consistent enough. The higher the FPS, the better the performance of the graphics card. The capture speed of some high-speed cameras can reach 11000 frames per second, so do we need to play this kind of movies at 11000 frames per second? Of course not, we usually set the FPS value of the image at 25 frames/sec or 60 frames/sec. However, due to the difference between the key frame and the transition frame in the video, the key frame saves the complete picture while the transition frame only saves the changed part of the previous frame, which needs to be obtained through key frame calculation. Therefore, we need to decode each frame, that is, get the YUV data of the picture. At the same time, only the pictures we really need to display are transcoded, that is, YUV data is converted into RGB data, including the calculation of the width and height of the picture.
Three, code implementation
(1) Register FFmpeg components
//Register and initialize FFmpeg wrapper and network equipment
(2) Open files and create input devices
AVFormatContext represents a wrapper,
When reading multimedia files, it is responsible for saving contextual information related to packaging and encoding and decoding.
(3) Traverse the stream and initialize the decoder
Various streaming media channels are stored in the encapsulator, usually the video channel is 0 and the audio channel is 1.
In addition, it may also include subtitle stream channels, etc.
Steps 2 and 3 are basically the main steps to open multimedia files.
All parameters of decoding and transcoding can be obtained here.
Next, we need to read, decode, and transcode in a loop until the playback is complete.
(4) Read compressed data
/*The reason why it is called compressed data is mainly to distinguish the two structures of AVPacket and AVFrame.
AVPacket represents a picture after key frame or transition frame encoding,
AVFrame represents a complete YUV picture after decoding of an AVPacket*/
(5) Decoding
(6) Video transcoding
// 720p output standard
/*
Here need to explain the reason for calculating outWidth * outHeight * 4:
The 720p standard video screen contains 720 * 480 pixels,
Each pixel contains RGBA4 types of data, and each type of data is represented by 1 byte or 8 bits.
Therefore, the size of a complete picture is outWidth * outHeight * 4.
(7) Audio transcoding
Fourth, the code address
FFmpeg client based on qt (Linux version):
The server can use the LIVE555 server, refer to the blog post:
Zero background knowledge
This chapter mainly introduces where FFMPEG is used (here are only a few that I know, in fact, it is far more than this). To put it bluntly is to illustrate: FFMPEG is very important.
Use FFMPEG as the core video player:
Directshow Filter using FFMPEG as the core:
Use FFMPEG as the core transcoding tool:
In fact, FFMPEG's video and audio codec function is really too powerful, covering almost all existing video and audio coding standards, so as long as you do video and audio development, you can hardly do without it.
1.1 Use of
ffmpeg program FFmpeg project consists of the following parts:
FFmpeg video file conversion command line tool, also supports real-time TV card capture and encoding into video files;
ffserver is a multimedia server based on HTTP and RTSP for real-time broadcasting. It also supports time shifting;
ffplay is a simple development using SDL and FFmpeg libraries Media player;
libavcodec a library that contains all FFmpeg audio and video codecs. In order to ensure the best performance and high reusability, most codecs are developed from scratch;
libavformat is a library that contains all the parsers and generators of common audiovisual formats.
1.2 Who is using ffmpeg
Use FFMPEG as the core video player: MPlayer, ffplay, shooter player, Baofeng Yingyin, KMPlayer, QQ audio and video...
Directshow Filter using FFMPEG as the core: ffdshow, lav filters...
Transcoding tool using FFMPEG as the core: ffmpeg, format factory …
2. How to install
FFmpeg can be installed and used in various operating systems such as Windows, Linux, and Mac OS.
FFmpeg is divided into 3 versions: Static, Shared, Dev
The first two versions can be used directly on the command line. Contains three exe: ffmpeg.exe, ffplay.exe, and ffprobe.exe
. The exe in the Static version is larger because the related Dlls have been compiled into the exe.
The size of the exe in the Shared version is relatively small, because when they run, they also need to call the corresponding function
Dev version in the related dll for development, which contains the library file xxx.lib and the header file xxx.h
3. How use
3.1 Use of command line tools
3.11 ffmpeg.exe
Applications used for transcoding:
ffmpeg -i input.avi -b:v 640k output.ts
Specific usage reference: ffmpeg parameter Chinese detailed explanation
Detailed instructions (English): http://ffmpeg.org/ffmpeg.html
3.12 ffplay.exe
Mainly used for playback applications
Play test.avi
ffplay test.avi
For specific usage, please refer to: ffplay shortcut keys and options
3.13 ffprobe.exe
ffprobe is an application for viewing file formats.
3.2 Use ffmpeg library for development
To Be Continue…
FFMPEG advanced chapter
FFmpeg basic environment construction and compilation
Basic processing of FFmpeg's decapsulation
FFmpeg is used for key extended data processing of hardware device decoding
FFmpeg is used for the basic control of the track mode of the hardware
Virtual machine installation
Since I was in my home and not working environment, the computer was still installed with windows7 system, so I started to install the virtual machine. I always like to use the VMWare virtual machine. I have not installed it since I bought a new computer last time. Install it. First, I installed the latest VMWare Workstation 11 + Ubuntu 14.04, and found that it runs extremely slow. Is it because my computer is out of date? There is no way, and I don’t want to toss about it. I just changed to a VMWare Workstation 10.0.1 build-1379776 + Ubuntu-12.04.4-alternate-i386. It’s better to run it just fine. The specific virtual machine and Ubuntu installation process will not be detailed, there are a lot of online articles.
Configure and compile
Download the latest version of ffmpeg, the current stable version is ffmpeg-2.6.1. Enter the virtual machine to decompress:
Configuration
Then compile with make, and finally an error appears:
It turned out to be neglected to directly in the windows shared directory, decompress it to the Linux directory, this problem will not occur. Of course, if we compile a static library instead of a shared library, this problem will not occur because there is no link file for Linux.
In the Linux directory, the compilation is successful and our dynamic library is generated:
In this way, we have produced the dynamic library and header files we need.
You may be confused. I am compiling on a pc now and cannot be used on embedded devices. I know this, I can't help it. Now I don't have the previous ARM compiler environment. If you are in the actual cross environment, you specify the cross compilation parameters when configuring ffmpeg's configure, it is estimated that it is probably like the following configuration:
./configure --prefix=./install --disable-static --enable-shared --enable-gpl --enable-pthreads --cross-prefix=arm-none-linux-gnueabi- --enable-cross-compile --target-os=linux --extra-cflags="-mcpu=arm9 -W -Wall -Wpointer-arith -Wstrict-prototypes -Winline -Wundef -O2 -Wall" --arch=armv4l --cpu=arm9
Of course, these are modified according to the specific environment. Not much to say here, this article is only explained on the PC.
FFmpeg summary
video size
ntsc: 408P, the corresponding resolution is 720*480, the recommended bit rate is 1800Kbps
snits: corresponding resolution 640*480
hd720: 720P, the corresponding resolution is 1280*720, the recommended bit rate is 3500Kbps
hd1080: 1080P, the corresponding resolution is 1920*1080, the recommended bit rate is 8500Kbps
Variable bit rate (VBR): Dynamic bit rate coding means that the output bit rate of the encoder can be adjusted adaptively according to the complexity of the input source signal of the encoder. At present, the output quality remains unchanged. VBR is suitable for storage, but not for streaming. It can effectively use limited space.
Fixed bit rate (CBR): Refers to the encoder output bit rate is fixed, not suitable for storage, CBR may not have enough bit rate to encode complex content,
Frame number: The number of pictures played per second (fps). A high frame rate can get a smoother and more realistic picture. However, a large frame rate will waste graphics processing power, because the display cannot be updated at a faster speed, and the frame rate exceeding the refresh rate will be wasted. In the case of the same video with a uniform bit rate, the more frames, the worse the picture quality. Because each picture will share the limited file volume per second, if there are more pictures, the content that each picture can express is limited. Generally, 30fps is basically smooth, and 50fps has a sense of flowing water. It is difficult to distinguish between 60fps and 100fps.
Resolution: The size of the screen, in pixels. The relationship with the encoding rate: the higher the resolution, the higher the encoding rate. Because the image has many details, the file size is large. At the same bit rate, the larger the picture, the more obvious the degree of image mosaic.
Sampling rate: The number of times the audio signal is sampled per second. The higher the sampling rate, the higher the sound reproduction and the more natural the sound, in Hz. The general audio file sampling rate is 44100Hz, that is, 44100 samples per second. Below this value, there will be a significant loss of sound, above this value, the human ear is difficult to distinguish, and it will increase the space occupied by the audio file.