Hi everyone, and welcome to the Stop Refactoring channel.
We've discussed live streaming protocols before, and in this issue we're discussing video formats for online on-demand .
That is, the commonly used formats of network video files and short videos such as MP4, FLV, HLS, etc.
We will discuss in detail the advantages and disadvantages of these video formats and why in the online on-demand scenario.
We discuss in this order:
1. The role of packaging format
2、MP4
3、 FLV
4、 HLS
The role of packaging format
First of all, we need to make it clear that MP4, FLV, and HLS are all encapsulation formats .
As discussed in the previous "Working Principles of Audio and Video Transcoding", a video file is essentially divided into three layers , encapsulation, encoding, and basic data .
The player needs to support these 3-layer formats at the same time to play the video.
The encoding and basic data format supported by most browsers are not scalable, so even if the same MP4 file has a different encoding format, the browser may not be able to play it.
In addition, most packaging formats do not limit the basic parameters of audio and video data, such as frame rate, resolution, sampling rate, etc.
Therefore, claims such as MP4 files are clearer than FLV files are essentially rumors.
So what is the use of the package format ?
Simply put, the encapsulation format is just arranging data such as audio data, video data, and basic information into files according to certain structural rules .
Although the data arrangement will not affect the audio and video data itself, it will affect some specific scenarios. For example, the MKV package format supports external subtitles, and subtitles in different languages can be switched.
In the scene of online video playback, the performance of FLV and HLS videos will be better than that of MP4 files.
The excellent performance is not in terms of picture and sound quality, but in the time required to play the first frame and the jump before loading, etc. The performance is better. The following will introduce why this is the case in detail.
MP4
MP4 is the most common encapsulation format, with a wide range of applications, high flexibility, and strong compatibility.
MP4 even supports custom cover art, and almost all players and even browsers support this package format.
An MP4 file is a tree structure in which many data blocks are nested with each other . MP4 supports dozens of data block types, and different types of data blocks store different information.
However, many data blocks are unnecessary. The data block structure and data of specific MP4 files can be viewed with the tool mp4info.
We don't need to understand all the technical details of MP4 , but we need to know the functions of the three data blocks of ftyb, mdat, and moov . These three data blocks are necessary data blocks for MP4 files.
The ftyb data block stores information such as encoding format and standards , and the player needs to select the corresponding decoder based on these information.
The mdat data block stores specific audio and video data , and the audio and video data recorded here generally has no time stamp.
The moov data block stores metadata , including the mapping relationship in the mdat data. The player needs to calculate the playback timestamp of the audio and video frames based on the moov data.
It often happens that when the network is normal, it takes a while for the MP4 files played online to be played out .
Often it is because the moov data block has not been obtained , and the playback time of the audio and video frame cannot be calculated and cannot be played.
Therefore, if the MP4 file is used for online playback , it is generally necessary to place the MOOV data block before the mdat data block , which is conducive to faster playback. Many tools can do this operation. The FFmpeg command is shown in the figure.
Some people say that MP4 files will be larger than FLV files, which may be caused by the fact that MP4 files record a lot of unnecessary information. Under normal circumstances, for the same video, the size of MP4-encapsulated files is similar to that of FLV-encapsulated files.
If online playback is not the main business of the website system , and the video files are not large and the duration is not long , you can use MP4 files directly . After all, browsers and many players support the MP4 encapsulation format, and users do not need to troublesomely upload them. .
However, if there are video files with a large file size of several GB or a long duration, try not to use the MP4 format: first, the CDN acceleration effect is not good, and second, MP4 files may be interrupted during long-term online playback. .
FLV
The FLV format is similar to RTMP and HTTP-FLV in the previous "Live Streaming Protocol". After all, they are standards led or launched by Adobe.
FLV is a format designed for online video, and the web page can be played by adding the flv.js plug-in.
Each audio and video data block of an FLV file can be marked with a time stamp , so FLV performs better in scenes such as first frame playback and unloaded jumps.
What needs to be specially explained here is that sometimes when playing FLV video files online, there will be an error reporting an unloaded jump , that is, an error will be reported and the playback will be interrupted when the jump is not loaded.
Generally, it is because the FLV file does not have a key frame index , and the FFmpeg command adds a key frame index as shown in the figure.
Generally, online videos use FLV format files . In our past test experience, large files and long-term playback are more stable than MP4. Of course, ordinary CDN acceleration is not effective for large FLV files, but you can access a large file CDN to alleviate this problem.
HLS
The last is the HLS format. HLS has been discussed in detail in the previous "Live Streaming Protocol".
In the on-demand scenario, HLS essentially divides a large video file into many fragmented video files .
The file in HLS format consists of two parts , one is multiple .ts fragmented video files with a length of only a few seconds, and the other is a .m3u8 index file that records the addresses of these video files.
The HLS viewing address is actually the .m3u8 index file. After the client obtains the index file, it can download the corresponding fragmented video file and start playing it.
The .m3u8 index file of HLS supports secondary indexing , that is, multiple viewing addresses such as high-definition, standard definition, and smooth can be integrated into one index file, and the player can automatically switch between different viewing addresses according to the current bandwidth. "It's also because of this.
Since HLS cuts a video file into many fragmented files, compared with the FLV format, HLS is better in terms of unloaded jumps, CDN acceleration, and multi-threaded preloading when playing long videos online .
However, in the short video scene, the advantage of HLS is not obvious , and the video fragment ts file itself is also a complete video package format, which can also be played independently, so some unnecessary traffic will be generated.
In addition, since HLS videos are essentially composed of many files, HLS videos generally require the website system to transcode them by themselves. This is also a troublesome part of HLS videos. However, public clouds also have video file transcoding services that generally provide automatic transcoding functions.
As a supplementary note, the HLS protocol is a standard introduced by Apple, and the MPEG-DASH protocol is similar to the HLS protocol. The working principles of HLS and MPEG-DASH are similar, but some standards are different, so I won’t expand here.
Summarize
Generally speaking, if there are not many videos in the website system and online playback is not the main business, then the regular MP4 format for video files is fine.
If playing video online is the main business, you need to consider FLV and HLS formats.
If there are many short videos, the FLV format is recommended, and if there are many long videos and large files, the HLS format is recommended.
