Weekly, overview the dry goods in the field of audio and video technology.
News contribution: [email protected].
Content recommendation
In the era of fragmentation, don't let users worry about it
In this lecture interview, we invited the lecturer of this LiveVideoStackCon 2020 Beijing station and the technical director of Moxiang Technology Liu Zhimei to do the interview. As an Internet practitioner, especially an expert in the field of service content production, I think this interview Derived from an event that everyone is paying more attention to recently.
https://mp.weixin.qq.com/s/fk_GYljlOfHCeyAf1bmBxw
Architecture
Understanding onicecandidateerror
How to deal with common 701 and 401 STUN / TURN server errors.
https://webrtccourse.com/course/webrtc-codelab/module/fiddle-of-the-month/lesson/understanding-onicecandidateerror/
WHIP-Make WebRTC easy to use for media platform extraction
The new IETF draft is a simple transmission protocol for extracting video broadcast scenes. More information about Medooze and Meetecho.
http://webrtcbydralex.com/index.php/2020/09/30/whip-make-webrtc-easy-to-use-for-media-platforms-ingest/
8 components of WebRTC security architecture
An overview of WebRTC security.
https://www.red5pro.com/blog/webrtc-security-architecture/
transporting network
Chrome is deploying HTTP/3 and IETF QUIC
QUIC is a new network transmission protocol that combines TCP, TLS and other functions. HTTP/3 is the latest version of HTTP, which carries the vast majority of web traffic. HTTP/3 only runs on QUIC.
https://blog.chromium.org/2020/10/chrome-is-deploying-http3-and-ietf-quic.html
Implement a custom protocol based on Netty
This article first compares the custom protocol with the HTTP protocol, and explains some of the advantages of the custom protocol. Then a custom protocol was defined and the meaning of each byte in the protocol was explained. Finally, the custom protocol is implemented through Netty, and the heartbeat function based on the custom protocol is realized.
https://mp.weixin.qq.com/s/Usf6x4hd-Ku6edkTyfibvw
LinkedIn's content delivery strategy
This article comes from a presentation at Content Delivery Summit 2020. The speaker is Bhaskar Bhowmik from LinkedIn. The main content of the presentation is LinkedIn’s content delivery strategy.
https://mp.weixin.qq.com/s/jjbttww4r4rl0WYO0zWcWw
More efficient transmission through better congestion control
This speech is from Demux-SF Video Technology July 2020. The speaker is Nitin Garg, a software engineer at Facebook, and introduced how to use better congestion control for more effective transmission and improve video quality.
https://mp.weixin.qq.com/s/yWXiU-yMTY2dcT8Y6j19tQ
Codec
FFmpeg code architecture
The learning process of FFmpeg is difficult. If you sort out the structure, the overall code context is clearer, but the core modules such as libavfilter are not covered in this article.
https://mp.weixin.qq.com/s/elNw6nxc0caN9z76uLJaSg
ijkplayer
There are many places where ffmpeg is used in audio and video development. Play scenes/audio and video editing scenes will use ffmpeg core modules. The introduction of this article is to tell you that we can seal different functional modules in the same ffmpeg library. We save space.
https://mp.weixin.qq.com/s/6kfu-U61E-MTmwoydmNAVw
Audio and video technology
Audiovisual speech enhancement in YouTube Stories
Although people put great effort into improving the quality of videos taken with smartphone cameras, the audio quality in the videos is often overlooked. For example, the speech on a certain topic in the video may cause confusion, distortion or incomprehension, in which multiple people are speaking, or the background noise is high.
https://ai.googleblog.com/2020/10/audiovisual-speech-enhancement-in.html
Audio and video editing and processing through FFmpeg commands in Android (open source)
Sometimes we want to process audio and video, such as video editing, adding subtitles, cropping and other functional processing. Although some good projects are open source on Github, if we want to carry out secondary development on this project, for example, I want to The dynamic library of this project is based on OpenH264 to encode YUV. At this time, it is possible that the dynamic library does not integrate the OpenH64 library, so for extensibility, I made a set of universal libraries, which basically contains all commonly used libraries. Audio and video processing library, you do not need to compile.
https://mp.weixin.qq.com/s/K23qVq_qwmc0JY7dmTeVRg
AI intelligence
74KB pictures are also high-definition, Google uses neural network to create a new image compression algorithm
Still worrying about image loading? The latest good news is that the Google team has adopted HiFiC, an image compression method that combines GANs and neural network-based compression algorithms, which can still restore images with high fidelity even when the bit rate is highly compressed.
https://mp.weixin.qq.com/s/WO_r7aSVaTP0Idzhj8kq0w
Lian Zhouhui: Research Progress of Text, Image and Graphics Generation Technology
CSIG document image analysis and recognition.
https://mp.weixin.qq.com/s/1JOmHPu8I31Sw-Ad9RejqQ
Fun with the StyleGAN2 model: teach you to generate anime characters
Generative Adversarial Network (GAN) is a generative model that can generate new content. Due to its interesting applications, such as generating synthetic training data, creating art, style conversion, image-to-image translation, etc., this topic is very popular in the field of machine learning.
https://mp.weixin.qq.com/s/SvsXNuhym0gYBDdBztbFaQ
image
GIMP 2.10.22 released
GIMP began to support AV1's image container AVIF.
https://www.gimp.org/news/2020/10/07/gimp-2-10-22-released/
Summary of picture/file upload scheme based on business scenarios
The image/file upload group is one of the indispensable links in the development of enterprise projects, but all user modules will have image/file upload requirements. It is also a basic component in many third-party component libraries (ant desigin, element ui) One. Next, the author will take you to implement an image/file upload component from scratch and expand a more powerful upload component.
https://juejin.im/post/6880802339835133966
Resource recommendation
How to improve professional work efficiency-Tencent Technology Engineering
In daily development work, sometimes you find that when writing code 6 to fly smoothly, but when it comes to needs follow-up, meeting participation, communication with people, goal setting and other work scenarios, there are always no rules and unclear expressions , Unable to grasp the key points, surrounded by trivial matters and unable to escape, low efficiency of the communication process, etc. If there is, it is obvious that you need to pay attention to the problem of work efficiency at this time. This article attempts to make some generalizations from many aspects, which may bring you some awareness and thinking.
https://mp.weixin.qq.com/s/zH9kFjJQ5zE9mKGEiwEYAA