Recently, Tencent multimedia laboratory design based on full reference video quality assessment algorithm depth study of DVQA officially open source, performance of the algorithm model has now achieved industry-leading performance in the open test data sets.
Audiovisual era, audio and video more widely: live, a short video, video programs, audio and video calls ...... due to the recent epidemic caused by a new crown online collaborative office, the rise of online educational products, but also brought online audio and video the outbreak of the demand, users increasingly strong demand for audio and video quality.
Throughout the video link, the modules can be most accurately measure, such as gathering, uploading, pretreatment, transcoding, distribution. However, some of them unknown precisely the most critical part of that user's video viewing experience in the end how. Currently video quality assessment method within the industry is divided into two categories: objective and subjective quality assessment quality assessment. The former is calculated video content, but also according to whether the use of high-definition video for reference, the source video is further subdivided professional video or user-generated video; the latter is mainly dependent on the human eye and scoring, can directly reflect the feelings of the audience on video quality. However, these methods are still time-consuming, costly, deviations and other problems of subjective perception.
Video quality assessment solutions proposed Media Lab, combined with business needs first, "Online subjective quality evaluation platform" to build large-scale database subjective quality, while using subjective data collected to evaluate training based on objective quality depth learning algorithm Finally, the quality assessment algorithm is trained to deploy to the lines of business, the closed-loop monitoring possible quality problems. From these three perspectives, DVQA able to take into account the different services, under the premise of the scene, efficiency and precision to meet two requirements.
DVQA contains multiple quality assessment algorithm model, this is for open source algorithm C3DVQA PGC video. This project uses Python development, deep learning modules PyTorch. Code uses a modular design for easy integration of newer deep learning technology and flexible custom models, new training and testing data sets.
In the algorithm design, the network structure C3DVQA used as shown below. Its inputs are damaged video and video residuals. A network comprising two dimensional convolution spatial feature extraction frame by frame. After four cascaded three-dimensional space-time convolution layer learns joint feature. It described three-dimensional video output of the convolution temporal masking effect, and then use it to simulate the human eye's perception of video residuals: where the weak masking effect, a residual perceived more easily; strong masking effect where, more complex background You can mask picture distortion.
Finally, the network layer and the cell layer fully connected. Input cell layer is the result of a residual effect of masking frame is processed, it represents the residual perceptible to the human eye. Fully connected layer to learn the overall perception of non-linear regression relationship quality and the target quality score range.
In the evaluation results, Tencent multimedia laboratory to verify the performance of the proposed method in two CSIQ LIVE video quality and data sets. And using standard PLCC and SROCC as a quality criterion to compare the performance of different algorithms. The proposed C3DVQA with a conventional full reference quality assessment algorithm comparison, including PSNR, MOVIE, ST-MAD, VMAF and DeepVQA, the results shown in the following table.
▲ LIVE CSIQ the two databases and compare the performance of a full reference algorithm
At present, the evaluation algorithm has been verified using external Tencent in a variety of products, such as Tencent meeting on the standard indicators of foreign aid hundreds of laboratories in line with ITU / 3GPP / AVS and other judge, closed-loop monitoring of the quality of the user experience of the whole network , starting from the user real-life experience, to optimize product performance.