FFmpeg is a leading multimedia framework and a powerful open source library for processing audio and video, capable of decoding, encoding, transcoding, muxing, stream separation, filtering, and playing audio and video content in almost all formats. It can use API to process audio and video, and can also use FFmpeg commands to edit audio and video files.
Contains multiple decoders and encoders for audio, video, and subtitle streams, as well as multiple bitstream filters.
FFmpeg directory and what it does
libavutil: is a library of functional utilities used to simplify programming, including random number generators, data structures, math routines, core multimedia utilities, and more.
libavcodec: is a library containing codecs for audio/video.
libavformat: is a library containing compositors and splitters for multimedia container formats.
libavdevice: is a library containing input and output devices for grabbing and rendering many common multimedia input/output software frameworks, including Video4Linux, Video4Linux2, VfW, and ALSA.
libavfilter: is a library that contains media filters.
libswscale: is a library that performs highly optimized image scaling and colorspace/pixel format conversion operations.
libswresample: is a library that performs highly optimized audio resampling, rematrixing, and sample format conversion operations.
FFmpeg tools and what they do
ffmpeg: ffmpeg is a command line tool used to implement the functions that FFmpeg has on the command line.
ffplay: ffplay is a very simple and portable media player using FFmpeg library and SDL library. It is mainly used as a testbed for various FFmpeg APIs.
ffprobe: ffprobe is a multimedia stream analysis tool. It gathers information from multimedia streams and prints it out in human and machine readable form. It can be used to detect the container type of multimedia streams, as well as the format and type of each multimedia stream. It can be used as a standalone application or combined with text filters to perform more complex processing.
After understanding the FFmpeg directory and its tools, the following article focuses on how we use the FFmpeg command line to process audio and video files in our daily life.
FFmpeg processing audio and video function description
FFmpeg commands can be roughly divided into several major modules, such as recording commands, audio and video synthesis and separation, audio and video cutting and merging, and mutual conversion of pictures and videos. Below we introduce how to use the FFmpeg command line to process audio and video to achieve the above functions.
This article takes the Ubuntu operating system as an example, and installs it through a simple command sudo apt-get install ffmpeg.
You can also execute the following commands after installing FFmpeg on Windows, MacOS, and other Linux operating systems.
1) Recording
ffmpeg -f avfoundation -list_devices true -i ""
2) Record screen
ffmpeg -f avfoundation -i 1 -r 30 out.yuv
-
f Specifies to use avfoundation to acquire data.
-
-i specifies where to collect data, it is a file index number. On my computer, 1 represents the desktop (the device index number can be queried with the command above).
-
-r specifies the frame rate. According to the official ffmpeg document, -r and -framerate have the same effect, but the actual test found that it was different. -framerate is used to limit input, and -r is used to limit output.
3) Screen recording + sound
ffmpeg -f avfoundation -i 1:0 -r 29.97 -c:v libx264 -crf 0 -c:a libfdk_aac -profile:a aac_he_v2 -b:a 48k out.flv
-
-i 1:0 The "1" before the colon represents the screen index number. The "0" after the colon represents the phase number of the sound.
-
-c:v Same as parameter -vcodec, it means video encoder. c is the abbreviation of codec, v is the abbreviation of video.
-
-crf is a parameter for x264. 0 Tabular lossless compression.
-
-c:a Same as parameter -acodec, it means audio encoder.
-
-profile is an argument to fdk_aac. The aac_he_v2 form uses AAC_HE v2 to compress data.
-
-b:a Specifies the audio bitrate. b is the abbreviation of bitrate, a is the abbreviation of audio.
4) Record video
ffmpeg -framerate 30 -f avfoundation -i 0 out.mp4
-
-framerate Limit the capture frame rate of the video. This must be set according to the prompt requirements, if not, an error will be reported.
-
-f Specifies to use avfoundation to collect data.
-
-i Specifies the index number of the video device.
5) Video + Audio
ffmpeg -framerate 30 -f avfoundation -i 0:0 out.mp4
6) Recording
ffmpeg -f avfoundation -i :0 out.wav
7) Record audio raw data
ffmpeg -f avfoundation -i :0 -ar 44100 -f s16le out.pcm
8) Extract the audio stream
ffmpeg -i input.mp4 -acodec copy -vn out.aac
-
acodec: Specifies the audio encoder, copy indicates only copying, not encoding and decoding.
-
vn: v stands for video, n stands for no, which means no video.
9) Convert to MP3 format
ffmpeg -i input.mp4 -acodec libmp3lame out.mp3
10) Extract video stream
ffmpeg -i input.mp4 -vcodec copy -an out.h264-
vcodec: Specifies the video encoder, copy indicates only copying, not encoding and decoding.
-
an: a stands for video, n stands for no, which means no audio.
11) Video conversion format
ffmpeg -i video_test.mp4 -vcodec copy -acodec copy out_convert.flv
The above command table audio and video are directly copied, but the package format of mp4 is converted to flv.
12) Remove video sound
ffmpeg -i video_auido.mp4 -vcodec copy -an video.mp4
13) Video Compression
1）ffmpeg -i test_ffmpeg.mp4 (压缩的文件更大更清晰，一般情况下不用）
2）ffmpeg -i out.MP4 -b:v 500k 512k_out.mp4(减小视频码率，压缩的更模糊）14) Audio and video merge
ffmpeg -i out.h264 -i out.aac -vcodec copy -acodec copy out.mp4
15) Extract YUV data
ffmpeg -i input.mp4 -an -c:v rawvideo -pixel_format yuv420p out.yuv
播放
ffplay -s wxh out.yuv-
-c:v rawvideo specifies to convert the video to raw data
-
-pixel_format yuv420p specifies the conversion format as yuv420p
16) Video YUV to H264
ffmpeg -f rawvideo -pix_fmt yuv420p -s 640x480 -r 30 -i out.yuv -c:v libx264 -f rawvideo out.h264
17) Extract audio PCM data
ffmpeg -i out.mp4 -vn -ar 44100 -ac 2 -f s16le out.pcm
播放
ffplay -ar 44100 -ac 2 -f s16le -i out.pcm18) PCM to WAV
ffmpeg -f s16be -ar 8000 -ac 2 -acodec pcm_s16be -i input.raw output.wav
19) Add watermark image watermark:
ffmpeg -i out.mp4 -vf "movie=logo.png,scale=128:72[watermask];[in][watermask] overlay=96:54 [out]" water_img.mp4
-
movie in -vf specifies the logo location. scale specifies the logo size. overlay specifies where the logo should be placed.
Text watermark:
ffmpeg -i out.mp4 -vf "drawtext=fontfile=FZBaoHTJW_Xi.TTF: text='anyRTC':x=128:y=72:fontsize=24:fontcolor=red:shadowy=2" water_text.mp4
20) Video scaling
fmpeg -i out.mp4 -vf scale=iw/2:-1 scale.mp4
-
-vf scale specifies to use a simple filter scale, iw in iw/2:-1 specifies the width of the video as an integer. -1 means the height changes with the width.
Reduce video resolution and convert video format
ffmpeg -i input.avi -vf scale=640:360 out.mp421) Video cropping
ffmpeg -i VR.mov -vf crop=in_w-300:in_h-200 -c:v libx264 -c:a copy -video_size 1280x720 vr_new.mp4
crop 格式：crop=out_w:out_h:x:y
-
out_w: The width of the output. The width of the video can be entered using the in_w form.
-
out_h: The height of the output. The height of the video can be entered using the in_h form.
-
x : X coordinate
-
y : Y coordinate
If x and y are set to 0, the clipping starts from the upper left corner. If not written, it is cropped from the center point.
22) Video left and right (up and down) flip
ffmpeg -i out.mp4 -filter_complex "[0:v]pad=w=2*iw[a];[0:v]hflip[b];[a][b]overlay=x=w" duicheng.mp4
-
hflip horizontal flip
-
If you want to modify it to flip vertically, you can use vflip
23) Crop clips
ffmpeg -i out.mp4 -ss 00:00:00 -t 10 out_cut.mp4
-
-ss specifies the start time of the crop, accurate to seconds
-
-t The length of time to be trimmed.
24) Audio and video merge First create a videolist.txt file with the following contents: file 'test1.mp4' file 'test2.mp4' and then execute the following command:
ffmpeg -f concat -i videolist.txt -c copy output.mp4
25) HLS Slicing
ffmpeg -i out.mp4 -c:v libx264 -c:a libfdk_aac -strict -2 -f hls out.m3u8
-
-strict -2 specifies that the audio uses AAC
-
-f hls convert to m3u8 format
26) Video to JPEG
ffmpeg -i video_test.mp4 -r 1 -f image2 image-%3d.jpeg
27) Image format conversion
ffmpeg -i input.bmp out.jpg
ffmpeg -i input.bmp out.png28) Video frame-by-frame screenshots are mainly used for users to manually take screenshots or upload videos to generate thumbnails
Here: -ss position searches for the specified time [-]hh:mm:ss[.xxx] The format is also supported, -vframes sets how many frames (frame) of video to convert, the example command is to get the first frame of the first second screenshot.
Note: The generated screenshots are best in jpg format, which takes up less space. If you need to use other formats, you can set them according to your business needs.
ffmpeg -ss 00:00:01 -y -i video_test.mp4 -vframes 1 snap.jpg
29) Video to GIF
ffmpeg -i video_test.mp4 -r 1 -f image2 image-%3d.jpeg
30) Image to video
ffmpeg -f image2 -i image-%3d.jpeg out_img_video.mp4
31) Add subtitle srt to video
ffmpeg -i video_test.mp4 -vf subtitles=subtitle.srt out_subtitle.mp4
32) Add music + subtitles to video
ffmpeg -i video_test.mp4 -i audio_bg.mp3 -vf subtitles=all_mp3_srt.srt out_mp3_subtitle.mp4
FFmpeg is a powerful library dedicated to audio and video processing, and many players are developed based on FFmpeg. The functions mentioned in the article are only part of the functions that can be used by FFmpeg listed by the editor. More FFmpeg command lines can realize the functions of processing audio and video. You can go to the FFmpeg official website to check related documents.