1. Introduction to FFmpeg
FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. Use LGPL or GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library libavcodec. In order to ensure high portability and codec quality, many codes in libavcodec are developed from scratch.
FFmpeg provides a powerful command line tool, which is very convenient for users to use and secondary development.
Official website: http://www.ffmpeg.org/.
The Github project address of ffmpeg is: https://github.com/FFmpeg/FFmpeg
FFmpeg is developed under the Linux platform, and it can also be compiled and run in other operating system environments, including Windows, Mac OS X, etc. This project was first initiated by Fabrice Bellard, and was mainly maintained by Michael Niedermayer from 2004 to 2015. Many FFmpeg developers are from the MPlayer project, and currently FFmpeg is also placed on the server of the MPlayer project team. The name of the project comes from the MPEG video coding standard, the "FF" in front stands for "Fast Forward"
A great software to produce it must have a legendary history, FFmpeg is the creator of the French genius programmer Fabrice Bellard (Fabrice Bella) in 2000 is to develop the first edition;
Fabrice Bella:
Developed the famous executable compression program LZEXE during high school
and created the FFmpeg project in 2000
In 2011, he wrote a Linux virtual machine (JSLinux) in JavaScript.
He is also the author of QEMU, TinyCC
In 2004, Michael (Michael Niedermayer) took over FFmpeg, and Michael added the filter mirror system libavfilter to the FFmepg project, making FFmpeg's multimedia processing more diverse and more convenient. After the release of version 0.5 of FFmpeg, no new version was released for a long time. Later, after FFmpeg adopted Git as the version control server, it continued to update the release version.
In March 2011, because some submitters in the FFmpeg project were inconsistent with the project management method and development direction of FFmpeg, the FFmpeg team was divided into two factions. One of them believed that FFmpeg should focus more on audio and video development and research and update advanced audio and video technology. , These people recreated a new project called Libav. The other group thinks they have to take into account the old FFmpeg technology, and then gradually develop.
In August 2015, Michael took the initiative to resign as the head of the FFmpeg project. Michael transplanted a lot of codes and functions from Libav to FFmpegZhong. Since then, Libav and FFmpeg have merged and developed together.
2. What can FFmpeg do
The multimedia video processing tool FFmpeg has very powerful functions including video capture function, video format conversion, video capture, and watermarking of videos.
Video capture function:
-
The ffmpeg video capture function is very powerful. It can not only capture images from a video capture card or USB camera, but also screen recording. At the same time, it also supports RTP to transmit the video stream to a streaming server that supports RTSP, and supports live applications.
-
On the Linux platform, ffmpeg provides very good support for V4L2 video devices, such as: ./ffmpeg -t 10 -f video4linux2
-s 176 144 -r 8 -i /dev/video0 -vcodec h263 -f rtp rtp: //192.168.1.105:5060> /tmp/ffmpeg.sdp The
above command means: capture 10 seconds of video, capture video4linux2 video device, capture QCIF (176 144) video, 8 frames per second, video device is /dev /video0, the video encoding is H263, the output format is RTP, the IP address and port are defined later, the SDP file corresponding to the stream is redirected to /tmp/ffmpeg.sdp, and the SDP file is uploaded to the streaming server You can live broadcast. -
There is very little information about ffmpeg video capture under windows, but ffmpeg still supports video capture under windows. ffmpeg supports video
capture of video forwindows (VFW) devices under windows , but VFW devices are outdated and are being replaced by WDM video devices, but ffmpeg has no plans to support WDM, but there seems to be a tool to convert WDM to VFW, so ffmpeg can still capture video under windows.
Video format conversion function:
- ffmpeg video conversion function. Video format conversion, such as converting multiple video formats to flv format, is not video signal conversion.
- ffmpeg can easily realize the mutual conversion between multiple video formats (wma, rm, avi, mod, etc.), for example, it can convert the recorded video avi, etc. into the flv format used by the video website.
Video screenshot function:
-
For the selected video, a thumbnail of the specified time is captured. Video capture, get static images and dynamic images, it is not recommended to capture gif files; because the captured gif files are large and the playback is not smooth.
-
Add watermark function to video
-
Use ffmpeg video to add watermark (logo).
[Article Welfare] I recommend my Linux and C/C++ technology exchange group: [960994558] I have compiled some learning books and video materials that I think are better for sharing, and you can add them if you need them! ~
Quickly learn FFmpeg video https://www.bilibili.com/video/BV1G5411G7XY/
3. FFmpeg architecture module composition
libavutil
-
libavutil: Contains some public utility functions;
-
AVUtil is the core tool library of FFmepg. This module is one of the most basic modules. Many other modules below will rely on this library to do some basic audio and video processing operations.
libavformat
-
libavformat: used for the generation and analysis of various audio and video packaging formats, including functions such as obtaining information required for decoding to generate decoding context structure and reading audio and video frames;
-
AVFormat
is a file format and protocol library, which encapsulates the Protocol layer and Demuxer and Muxer layers, making the protocol and format transparent to developers. AVFormat realizes most of the media encapsulation formats in the current multimedia field, including encapsulation and decapsulation, such as MP4, FLV, KV, TS and other file encapsulation formats, RTMP, RTSP, MMS, HLS and other network protocol encapsulation formats. -
Whether FFmpeg supports a certain media packaging format depends on whether the packaging library of that format is included when compiling. According to actual needs, the media packaging format can be expanded, adding your own customized packaging format, that is, adding your own packaging processing module in AVFormat.
libavcodec
-
libavcodec: used for various types of sound/image encoding and decoding;
-
AVCodec is a codec library. This module encapsulates the Codec layer, but some Codecs have their own licenses. FFmpeg will not add libraries like libx264, FDK-AAC, lame, etc. by default, but FFmpeg is like a platform. You can add other third-party Codecs in the form of plug-ins, and then provide developers with a unified interface.
-
AVCodec implements most of the current multimedia encoding and decoding formats, supporting both encoding and decoding. In addition to supporting MPEG4, AAC, MJPEG and other built-in media codec formats, AVCodec also supports third-party codecs, such as H.264 (AVC) encoding, which requires x264 encoder; H.265 (HEVC) encoding , Need to use x265 encoder; MP3 (mp3lame) encoding, need to use libmp3lame encoder. If you want to add your own coding format, or hardware codec, you need to add the corresponding codec module in AVCodec.
libavfilter
- AVFilter: It
is an audio and video filter library. This module provides processing including audio features and video special effects. In the process of encoding and decoding using FFmpeg API, it is very convenient to directly use this module to do special effects processing for audio and video data. A very efficient way.
libavdevice
- AVDevice:
Input and output devices. For example, if you need to compile the tool ffplay for playing sound or video, you need to make sure that the module is turned on, and libSDL is pre-compiled, because the device module uses libSDL for sound and video playback. Library.
libswscale
- libswscale: used for video scene scaling and color mapping conversion; SWScale
module is a module for format conversion of images, for example, it can convert YUV data to RGB data.
libpostproc
- libpostproc: used for post effect processing;
- The PostProc module is used for post-processing. When we use AVFilter, we need to turn on the switch of this module, because some basic functions of this module will be used in Filter. If it is an older version of FFmpeg, then it is possible to compile and process the avresample module. This module is actually used to resample the original audio data, but it has now been abandoned. It is no longer recommended to use this library, but to use swrresample The library is substituted.
libswrressample
- The SwrRessample module can be used for audio resampling, which can convert digital audio channels, data format, sampling rate and other basic information.
ffmpeg
- ffmpeg: a tool provided by this project, which can be used for format conversion, decoding or instant encoding of TV cards, etc.;
ffsever
- ffsever: an HTTP multimedia instant broadcast streaming server;
ffplay
- ffplay: It is a simple player that uses ffmpeg library to parse and decode, and display through SDL;
4. Install ffmpeg under Linux
ffmpeg is a very powerful audio and video processing tool, the official website is: FFmpeg official website introduction ffmpeg is: a complete, cross-platform solution that can record, convert and transmit audio and video. ffmpeg can not only play videos, but also provide command-line tools to process videos. In addition, there is a powerful video processing library for development. The following is a simple command line transcoding operation of the video that introduces the installation process of ffmpeg using Linux as an example. It is the easiest entry content in ffmpeg.
First go to the official website to download the source package, here is ffmpeg-3.3.1.tar.bz2, after downloading, upload it to Linux for installation, first unzip the installation package:
tar -xjvf ffmpeg-3.3 .1.tar.bz2
cd ffmpeg-3.3.1/
If you execute configure directly now, the following error may be reported: The
error means that the yasm/nasm package does not exist or is very old, you can use –disable-yasm to disable this Option compilation, yasm is an assembler, and completely rewrites the nasm assembly environment, accepts nasm and gas syntax, supports x86 and amd64 instruction sets, so you can install yasm here. The download address is: The Yasm Modular Assembler After entering the Project, download the 1.3.0 source code package and install it by executing the following command:
tar -xvzf yasm-1.3.0.tar.gz
cd yasm-1.3.0/
./configure
make
make install The
compilation parameters are all default, install directly Just go to the system, after the installation is successful, go back to the directory where ffmpeg is decompressed, execute the following command to compile and install:
./configure --enable-shared --prefix=/monchickey/ffmpeg
make
make install
The compilation process is a bit long. After waiting patiently, execute cd /monchickey/ffmpeg/ to enter the installation directory and check that there are 4 directories: bin, include, lib, and share, where bin is the binary directory of the ffmpeg main program, and include is C/C++ Header file directory, lib is the compiled library file directory, share is the document directory, then enter the bin directory, and execute ./ffmpeg -version to view the details of the current version. By default, it will generally report libavdevice.so.57: cannot open shared object file: No such file or directory, the reason is that the lib directory is not loaded to link to the system library, the system ld directory list is in /etc/ld.so.conf, open the file, you will find that it references /etc/ld .so.conf.d/All the .conf files below, such as mariadb-x86_64.conf, we only need to create a file and write it into the lib path, execute the command: vim /etc/ld.so.conf.d/ffmpeg .conf Then add a line of content: /monchickey/ffmpeg/lib and then save and exit, then execute ldconfig to make the configuration effective, and now execute again. /ffmpeg -version shows that it is normal.
Then you can add the bin directory to the environment variable as needed. To ensure that you can use the ffmpeg command at any time, let's test the transcoding of the video:
first, a video file in avi format, the size is 37M, now use ffmpeg to convert to mp4 format, execute the following command:
ffmpeg -i test.avi out
After confirming the .mp4 , you see the screen scrolling to start processing, and the video is converted after about half a minute. After the conversion, the size of the mp4 video is 17M. You can download it for details.
Now ffmpeg installation and the simplest command-line video conversion are completed. In fact, the ffmpeg command-line tool can have many parameters without programming to achieve powerful functions.