FFmpeg is an open source multimedia framework that includes a library for audio and video encoding and decoding. It can perform various multimedia operations such as format conversion, video clipping, audio processing, etc. An open source computer program that can be used to record, convert, and stream digital audio and video.
The structure of FFmpeg
The default compilation produces 4 executables and 8 static libraries. The executable files include ffmpeg for transcoding, streaming, and dumping media files, ffplay for playing media files, ffprobe for obtaining media file information, and ffserver as a simple streaming media server. The 8 static libraries are actually 8 modules of FFmpeg, including the following:
- AVUtil: Core tool library. This module is one of the most basic modules. Many other modules below will rely on this library to perform some basic audio and video processing operations.
- AVFormat: File format and protocol library. This module is one of the most important modules. It encapsulates the Protocol layer, Demuxer, and Muxer layers, making the protocol and format transparent to developers.
- AVCodec: Codec library. This module is also one of the most important modules. It encapsulates the Codec layer. However, some Codecs have their own licenses. FFmpeg does not add libraries such as libx264, FDK-AAC, lame, etc. by default. But FFmpeg is like a platform that can add other third-party Codecs in the form of plug-ins, and then provide developers with a unified interface.
- AVFilter: Audio and video filter library. This module provides processing including audio and video special effects. In the process of encoding and decoding using FFmpeg's API, it is very convenient and very convenient to directly use this module to perform special effects processing on audio and video data. An efficient way.
- AVDevice: Input and output device library. For example, if you need to compile the tool ffplay for playing sound or video, you need to ensure that the module is open, and you also need to pre-compile libSDL, because this device module uses both sound and video playback. libSDL library.
- SwrRessample: This module can be used for audio resampling, and can convert various basic information such as the number of channels, data format, and sampling rate of digital audio.
- SWScale: This module is a module for image format conversion, for example, it can convert YUV data to RGB data.
- PostProc: This module can be used for post-processing. When we use AVFilter, we need to turn on the switch of this module, because some basic functions of this module are used in Filter.
Simple use of FFmpeg
- Convert input.avi to output.mp4:
ffmpeg -i input.avi output.mp4
- Add some requirements:
ffmpeg -threads 4 -i input.avi -ab 32 -flags +loop -vol 200 -vf yadif input.mp4
-ab: set bitrate
-vol: Increase volume to 200%
- Modify size when transcoding:
ffmpeg -i input.avi -s 640x320 input.mp4
- Convert video to m3u8 on-demand file
ffmpeg.exe -i input.avi -strict -2 -hls_time 10 -hls_list_size 0 -c:v libx264 -c:a aac -f hls input.m3u8
-hls_time 10: About 10 seconds for each ts file. This will be based on the specific situation, try to maintain a ts within 10 seconds
-hls_list_size 0: keep all ts paths in the m3u8 index
- Convert videos to encrypted m3u8 on-demand files
ffmpeg -i input.mp4 -c copy -bsf:v h264_mp4toannexb -hls_time 10 -hls_list_size 0 -hls_key_info_file key_info input.m3u8
key_info needs to be replaced with your own path, because it needs to be encrypted, so you need to prepare two files: key.key and key_info. The details are not introduced here. If you are interested, you can watch the video to learn how to use it in detail.
- screenshot:
ffmpeg -ss 00:02:06 -i input.mp4 -f image2 -y poster.jpg
-ss: time point of screenshot
- Continuous screenshots:
ffmpeg -y -i input.mp4 -vf “fps=1/2,scale=iw/4:-1,tile=10x10” -an %d.png
fps=1/2: take a picture every 2 seconds, if it is a picture every second, it should be fps=1
Scale: The size of the captured image. The above code sets the width to 1/4 of the original size and the height automatically, which is equivalent to iw/4:ih/4. It can also be set to a fixed value such as 120:80.
tile: grid, automatically merge 100 images into one large image
- Another way to take continuous screenshots
ffmpeg -i my_dream.mp4 -vf fps=1/2 -q:v 2 -s 120x67 preview/%d.png
fps=1/2: take a picture every 2 seconds, if it is a picture every second, it should be fps=1
-s: size
This method cuts out a small picture, which needs to be stitched together again.
This article is mainly a brief introduction to FFmpeg technology in audio and video development and its principle and simple use. There are many other technologies about FFmpeg. You can refer to the following technical route: Abstract "Audio and Video Getting Started Mastery Manual" can be viewed for details.
final note
FFmpeg is a world-leading multimedia framework that can decode, encode, transcode, multiplex, demultiplex, transcode, mix, stream, filter and play on most devices. FFmpeg can run and compile on various platforms (Linux, Mac OS X, Microsoft Windows, BSDs, Solaris, etc.) and architectures (x86, arm, mips, etc.). It is worth noting that FFMpeg is not directly used for encoding and decoding of various videos, it is just a framework. Other codecs are usually used to do the actual codec work.