01
Background of the project
The vast majority of iQIYI mobile users spend their time watching videos, and a large number of users play hundreds of millions of times a day. The speed and experience of broadcasting are particularly important to the length of consumption and retention of users. With the improvement of the quality of audio and video, the resources required to start playing the video are also gradually increasing, which also puts forward new requirements for the experience of starting the broadcast. However, the overall playback link is very complex, mainly including the network environment, CDN, P2P, terminal device performance, content clarity, etc.; how to optimize and reduce the time-consuming of each link of the start-up link, and overcome uneven device performance, network fluctuations, and system version differences It brings new challenges, and it is also the direction of our continuous optimization.
02
Time-consuming demolition of audio and video on demand
Before introducing the optimization plan, let's take a look at the complete process of broadcasting. A complete broadcast usually starts from the time when the user triggers the playback, and ends when the video sound and picture appear.
The picture below is a complete video playback process
As shown in the figure above, it mainly includes three stages: business layer interface creation, data download, and decoding and rendering, which constitute the time-consuming for broadcasting. For different stages, we need to customize different strategies to shorten the overall time. The following will introduce some optimization practices that we are doing or have already launched in detail.
03
optimization practice
Preloading
Preloading is a common time-consuming solution for optimizing the network. It refers to finding the right time to trigger the download of some video data in advance if the network capacity allows before playing the video. Preloading is equivalent to reducing the user-perceivable network loading and network connection time.
Preloading also needs to pay attention to efficiency, hit rate after preloading (starting broadcast and using preloaded data/number of times of starting broadcast), preloading utilization rate (starting broadcast and using preloaded data/number of preloaded data). In order to balance efficiency and utilization, it is not possible to preload all the videos displayed on the page. To this end, we selected a few scenarios to focus on trying.
Home page drama hot comprehensive card:
There are a total of 8 videos in the popular drama comprehensive card. The CTR of the first 2 videos is relatively higher than that of the latter ones. After the test and comparison, the preloading of the first 2 videos is enabled. The playback rate triggered by the popular drama card is increased by 5pp
pre-rendered
Preloading data better optimizes the user's waiting time that may be caused by data download, but it does not improve the time consumption of decoding and business layers. Considering that data decoding and business display can be separated, we preload data in some scenarios. On top of that, a pre-decoding mechanism is introduced.
After the preloading data is completed, continue to PreDecode, and cache the decoded picture frames.
pre-created
Preloading or pre-decoding saves the time-consuming data download and rendering, and the time-consuming UI creation of the business layer has not been optimized. In order to pursue the ultimate broadcasting experience, we consider using these time-consuming The time is pre-empted as a whole, so that users do not feel the time-consuming start of the broadcast, that is, it is infinitely close to the start of 0S, which leads to our pre-created solution.
This is a player multi-instance solution. In addition to the currently playing video, a player instance pool (extra player instance) needs to be initialized to support the playback of the next video.
The picture above shows our execution process in the vertical full-screen scene. In the vertical full-screen scene, the upper and lower videos are pre-created after the current video starts broadcasting, and the two videos are muted and broadcasted until they are paused immediately after broadcasting. When the user slides to the corresponding position and resumes playing, 0S broadcasting can be realized. Multiple instances have certain requirements for device performance. We support enabling pre-creation of one (lower) or two (upper and lower) device controls for different performances. The vertical full-screen scene in the app is not the first multi-instance hit rate reaches 97%.
Initialization streamlined
There is also a very important initialization preparation link before the start of the broadcast process, especially for the scene where the user starts playing immediately after entering the iQiyi APP and externally pulls up the iQiyi APP to play, etc. With the iteration of the playback business, there are more and more aggregated functional services and forms, such as advertisements, interactive videos, super resolution, vertical full screen, and sub-screen and so on. There are also more and more initialization dependencies before playback, and the time-consuming pre-dependencies for broadcasting become specific blocking items in specific scenarios.
For this reason, on the Android system, we start the playback initialization thread early by identifying specific scenarios, increase the priority of playback initialization, migrate the business layer configuration data to MMKV, use a streamlined ffmpeg library, and do not load libraries and APIs that are not used for the first time. This optimization shortens the average time for launching the iQIYI APP by 1.5 seconds.
Most of the previous measures are optimized across multiple stages such as business and decoding, which can basically ensure that the user side has no obvious perception of network, decoding, and rendering time-consuming in extreme cases.
Network time-consuming optimization
As mentioned above, a core stage of broadcasting is data download, which can be divided into request start, domain name resolution, scheduling, priority strategy, etc. After the app is started, the IP address of the commonly used domain name for broadcasting is pre-analyzed and cached. When broadcasting, it can directly connect to the server IP, saving time-consuming DNS resolution. At the same time, the closed-loop control system composed of self-built CDN, third-party CDN service and peak feedback, scheduling server, client SDK parallel multi-connection screening strategy, and real-time access feedback statistics can obtain better access quality and lower bandwidth in real time.
Multi-source scheduling : The scheduler allocates multi-source nodes and parameters as options to clients based on self-built and third-party historical peaks and bandwidth costs as control targets;
Screening strategy : The client records the speed of multiple connections during use, and selects one or more suitable nodes from the scheduling results according to the cost, speed, and currently configured working mode;
Access quality feedback : the client feeds back information such as the speed, hit status, and error code of the current connection node to the real-time access feedback background; the real-time access feedback background counts the access speed, success rate, and hit rate of each CDN in each operator's region, The data is injected into the scheduling server to control the scheduling of the next cycle.
Optimization of decoding start-up strategy
Decoding is a necessary step before the video screen appears. If you do not optimize and directly rely on ffmpeg or MediaCodec for processing, it will take a long time. At the same time, it is also critical to choose a suitable starting cache. There are mainly three optimization points in this area.
Intelligent selection of model capabilities : Comprehensive evaluation maximizes the use of device hardware decoding capabilities, optimizes HEVC streams, and supports intelligent stream start-up. According to device-side network speed conditions, taking into account the balance between clarity and playback quality, real-time intelligent calculation starts the broadcasting point Code stream required;
Moov front-end : At present, iQIYI’s MP4 data is all produced with the moov front-end production strategy, which speeds up the efficiency of data processing on the end;
Low buffer buffer start on the playback side : Adopt the separate control strategies for the start buffer and the freeze buffer, and set the minimum buffer water level when starting the broadcast, so as to decode the first frame in the fastest way. And according to the capabilities of different models of equipment, intelligent management configuration to achieve the best results.
04
Summary and Outlook
Committed to creating the ultimate playback experience, we try to measure the pain points of watching movies from the perspective of users, solve their pain points in real playback scenarios through technical deconstruction, refinement, and ensure a comprehensive lead over competing products through continuous optimization Long video broadcasting experience.
In the future, we will continue to look for new directions to optimize the broadcasting experience, such as reducing the time-consuming request for independent audio tracks, intelligent downloading, reducing the data required for broadcasting, and cooperating with hardware manufacturers to increase link priority and reduce network transmission delays, etc., while optimizing and improving the link Monitoring, ensuring quality and stability contributes to "steady growth".
maybe you want to see
GC pause investigation tour under Spring Cloud Gateway