technical analysis

With the popularity of gigabit fiber optics, video has gradually become the mainstream media. Now, many websites need to provide the function of video playback, so before we start, let’s investigate how the websites that already provide video playback function do it.

Xuetangonline
It is obvious that MP4 files are directly used here.
insert image description here

What can be seen from Chinese MOOCs
is that Chinese MOOCs, like Xuetangonline, use direct whole block mp4 files. Therefore, you can directly click to download, as shown in the figure below:
insert image description here

This transmission method is more traditional, and the user needs to cache the entire mp4 file.

Then, let's look at other sites.

MOOC
insert image description here
Leecode

AI University

And the above few are more intriguing. The address actually starts with blob? wtf? Then you directly click on this link, and a 404 similar to the picture below will appear? ? ?

So what is this blob?
Blob is Binary Large Object binary large object. Our usual File is a subset of it. There are such objects in the JavaScript language. And the above url with blob. It can be regarded as a binary stream. That is to say, the url with blob is generated after reading the binary stream at the front end . In JavaScript, the url of this blob can be used like a normal url. One of the purposes of using blob is to anti-crawl. The other is because blobs support streaming protocols very well.

The js code to realize this function is roughly as follows.
insert image description here
Other major entertainment video websites also use similar technology to anti-crawl.

Tencent Video
insert image description here
B station

iQiyi Youku

But

the blob is only used to hide the problem of downloading URLs, and does not actually solve the problem of real-time video transmission. Because solving this problem does not only depend on the front end, but also requires the cooperation of the protocol. Therefore, the streaming media protocol is the topic we are going to discuss today.

streaming protocol

According to the above analysis. However, the url of the blob above does not solve the problem of playing while caching, but only hides the url. We know that the traditional video website needs to download the entire video file to play the video, but it is impossible to download and play the video on demand, so there is a streaming media protocol.

The advantages and disadvantages of several common streaming media protocols:

RTMP : A private protocol developed by Adobe for the transmission of audio and video data between Flash/AIR and the server, and it is also the most commonly used streaming media transmission protocol at present.
Advantages: Based on TCP long connection, it does not need to establish multiple connections, and the delay is low, usually only 1~3s; the technology is mature and the supporting facilities are complete.
Disadvantages: It can only be used through Flash in PC browsers, and cannot be used in mobile browsers; since Flash is about to exit the stage, RTMP will basically not be used for streaming on the web player side.
HLS : An HTTP-based streaming media network transmission protocol proposed by Apple. Its working principle is sliced transmission, which cuts the live stream into countless slices. When the user watches the video, the client can only download a part each time.
Advantages: Based on the HTTP protocol, it is easier to access CDN, rarely blocked by the firewall, and comes with multi-bit rate adaptation; as a protocol proposed by Apple, it has great advantages under macOS/iOS, and Android also provides Corresponding support; it can be said that this protocol is very suitable for use on mobile devices.
Disadvantages: The delay is relatively large, usually not less than 10s. A large number of TS slice files will cause pressure on server storage and requests.
HTTP-FLV : Encapsulate audio and video data into FLV, and then transmit it through HTTP connection. Compared with RTMP, only the transmission protocol has changed. For the web player side, Flash is still required to play, but the emergence of "flv.js" makes up for this defect.
Advantages: low latency, the overall effect is very close to RTMP; compared with RTMP protocol, it can effectively avoid the influence of firewalls and proxies.
Disadvantages: Its transmission characteristics will cause streaming media resources to be cached on the local client, which means that the confidentiality is not very good; until now it is still not compatible with iOS browsers.
WebRTC : Based on Google's open source technology, a protocol for streaming media on the Web.
Advantages: Both RTMP and HLS are protocols in the hands of large companies, and WebRTC has been included in the W3C standard; there is no need to install plug-ins, and more and more browsers are supported.
Disadvantages: The manufacturer's customization of the browser or system may cause usability problems, and there is a lack of server-side design and deployment solutions; the transmission quality is difficult to guarantee, and the optimization methods are limited; the compatibility on Android devices is not good; in addition, this agreement mainly Oriented to the Web side, there is insufficient support for native development.

DASH : DASH (MPEG-DASH) is the abbreviation of Dynamic Adaptive Streaming over HTTP, which is a technical standard launched by the international standard group MPEG in 2014. DASH is a dynamic adaptive bit rate streaming technology based on HTTP, and the transmission protocol used is TCP (some old client live broadcasts will use UDP protocol live broadcast, such as YY, Qiqi Video, etc.). Similar to HLS and HDS technologies, both It is to divide the video into a small segment, transmit it through the HTTP protocol, and play it after the client gets it; the difference is that MPEG-DASH supports MPEG-2 TS, MP4 and other formats, and the video can be cut and downloaded according to various codes The downloaded media format can be either a ts file or an mp4 file, so when the client loads a video, it loads the corresponding video segment for playback according to the current network speed and supported encoding.
Advantages: small delay and high resolution.
Disadvantages: IOS does not provide native support.

Comparison of DASH and HLS

In general, DASH can provide substantially lower end-to-end latency compared to HLS. This is important for live streaming workflows. In addition, the template-based MPD of MPEG-DASH does not need to be updated and can be cached on the network edge server, while HLS needs to be periodically updated and propagated multiple times. HLS is supported on almost all devices and browsers, either natively or with JavaScript built into the player. Chrome and Android provide native support for DASH. On Apple products, additional software packages are required to support DASH. Safari browser does not support
codecs by default: DASH supports almost all codecs, while HLS only supports H.264 or HEVC / H.265 codecs.
Resolution: DASH has a higher resolution than HLS, but since Apple updated HLS to support 4K resolution at the end of 2017, the resolution advantage of DASH is not obvious. On the contrary, the compatibility advantage of HLS is more obvious (Apple: DASH? What does it matter to me?)
insert image description here

Technology solutions used by educational websites

MOOC
insert image description here

As you can see, the MOOC has m3u8 files, and the video files are mp2t video files in ts-ending format. Therefore, the HLS protocol is used.
In order to improve the video loading speed and save bandwidth resources, the current video websites all adopt HLS technology. The background of the website makes the video into small video slices, and these slices are recorded in the m3u8 file. Each slice is a ts file, and each ts file can be played directly in hundreds of KB for about 10 seconds, and the browser requests each ts file while playing.
Also note that the request parameter contains Alibaba Cloud's CDN, so it can be judged that the MOOC is a video-on-demand implemented using Alibaba Cloud's oss service.

Leecode
insert image description here

can see that although leecode hides the file format ts, the parameters of viu of leecode also contain m3u8 files. In addition, the file format is also m2pt (maybe I didn’t take a screenshot here before). It can be judged that it is also using the HLS protocol. In addition, seeing the leecode access address, it is obvious that it uses the video cloud service of Hangzhou Alibaba Cloud.

AI University
insert image description here

AI University is as obvious as MOOCs, using the HLS protocol. In addition, it is impossible to judge whether other OSSs are used. It is possible that AI University built its own CDN as a video streaming server.

Technology solutions used by entertainment websites

Segmented flv

iQIYI Youku
insert image description here

HLS

insert image description here
It can be seen that Tencent also uses the HLS protocol.

DASH

Station B
insert image description here
abandoned segmented FLV at the end of 2018 and officially entered the ranks of DASH. In this regard, the technicians of Station B have an article: Why did we choose DASH
Youtube

insert image description here

Netflix also uses HASH

It can be seen that large video websites basically use segmented FLV, HLS or DASH. DASH technology is relatively new. In addition, foreign companies such as Netflix have been exposed to streaming media earlier, and its own movies must support 4K resolution, so it is widely used abroad. It has been the end of 2017 that HLS supports 4K resolution. Due to the common 1080 resolution in China and considering many factors such as IOS compatibility, HLS was chosen. Only the younger station B chose DASH. Segmented FLV may be the reason why iQiyi and Youku have their own built-in frameworks, and no changes have been made.

flow media services

Generally speaking, current cloud service vendors will provide corresponding video cloud services. However, if you need to build your own streaming media server, there are several options below

Open source streaming media server
For general enterprise users or companies with certain technical capabilities, it is recommended to choose open source products. There are still many open source streaming media server software options, including: SRS, NGINX-RTMP, EasyDarwin, CRTMP SERVER, etc.
SRS is the work of a genius programmer named Winlin in China. The starting point is to develop a simple and efficient RTMP live broadcast server. This open source project is relatively active. Winlin has always had a high enthusiasm and has always adhered to the idea of open source. The code can be downloaded from GitHub It is freely available on the Internet, and the open source license used is MIT. Perhaps due to the rapid development progress, there were many bugs in the use of this open source software in the past few years (before 18 years). From the source code point of view, the design idea of the project seems not so firm, and there are often major changes. However, there are indeed a number of companies in China that are improving and developing based on this project, which also proves that the code quality of the project is still good. Adhering to the open source route is also the reason why it has won the favor of many users. The author has also been paying attention to this project, and I like this project more, which is regarded as the leader of domestic open source projects.
NGINX-RTMP is the most widely used and most mature open source streaming media server system. It is provided as an Nginx module. The author ARUT uses the BSD open source protocol. Due to the looseness of authorization, many commercial software manufacturers around the world have improved and released it on this basis. Due to the use of the Nginx development framework and the extremely high attainments of the author, the quality of the program is quite high and the performance is also very good. If you have linux or nginx development experience, it will be a good choice to expand based on this open source system. The problem is that the author himself seems to have no enthusiasm for further improvement of the code, and there has been almost no improvement in more than 3 years, but some branch versions continue to evolve.
EasyDarwin is improved based on Apple's open source project and is maintained by a company in Anhui, China. This project is a collection of many open source projects to create a set of its own solutions, which requires high development investment and code organization capabilities. Due to the clear price tag and strong commercial atmosphere, it can no longer be regarded as a free open source project. Many development languages are used, including go, nodejs, c, java, etc. Most users do not have the ability to improve based on these, and there is limited room for improvement in continuous optimization and performance. Because the project integrates too many open source project codes and has many functions, it also brings some confusion to users in terms of ease of use, security and maintainability. The author believes that the project team should still think about the development direction of the project. Whether to continue to open source or make commercial software, there should be a clear idea.
CRTMP SERVER is an open source software abroad in the early years. It is written in C++ language and has good performance. The company of several friends of the author is still developing based on this software. The problem is that there has been no follow-up update for several years. The commercial version was launched before, but it was not successful, so the free and open source version was put on hold. But the overall code quality is good, the structure is clear, and the protocol support is relatively complete.

Commercial streaming media server
If the project is more demanding in terms of stability, security and responsibility constraints, it is recommended to choose a commercial product. The choice of commercial streaming media server software is not large, basically it is made by several professional companies at home and abroad, and the optional products include: Wowza, Adobe Flash Media Server and domestic NTV Media Server G3, etc.
Wowza is a product of Wowza Media Systems in the United States, and it is also the most widely used streaming media server product, and it also has its agents in China. Authorize by issuing software authorization certificates, you can purchase licenses on a monthly or annual basis, and there are clear quotations on the official website. Domestic agents add their own packaging and technical support fees, and the prices vary.
Adobe Flash Media Server is the most widely used product in the early years. The product is mature and the price is relatively high. With Adobe's withdrawal from the Chinese market, some domestic agents are mainly responsible for technical support. Like Wowza, due to the support of domestic manufacturers, coupled with issues such as language, technical capabilities, and time differences, the technical support is not satisfactory.
NTV Media Server G3 is a product of the domestic technology company Yunshi Ruibo. This product competed with similar foreign products under the banner of commercial products at the beginning. The cooperation with OPEN Education and other companies has obtained high development support in technology, and the products are excellent in maturity and performance, and have a large customer base in large enterprises, government agencies, scientific research, media and other fields. When the author’s friends talk about the company, they generally say that their after-sales service is very good, and they can provide you with valuable consulting services before and after sales. The problem with this product is that the interface standard has changed too much in several key versions, which makes it inconvenient to upgrade from the old version to the new version.

Summarize

I believe that after reading this article, you already have a certain understanding of the current popular streaming media technology solutions. From the perspective of practicality and speed, BitTao recommends using the current easy-to-implement Apple HLS protocol. If the video function business If it is heavy in your website, or if you value 4K or higher resolution, you can also consider using DASH.