Weekly, overview the dry goods in the field of audio and video technology.
News contribution: [email protected].
Architecture
WebRTC first frame display optimization strategy full analysis
The display of the first frame of audio and video real-time calls is an important user experience standard. This article mainly analyzes the receiver to understand and optimize the display time of the first frame of the video.
https://juejin.im/post/6875681198316716045
Analysis of low-latency Camera architecture
Broadcasting, machine vision, and video surveillance are the traditional application directions of video camera products. In recent years, with the development of the Internet, the web live broadcast industry has emerged as a new hot application direction. In fact, these seemingly unrelated fields have always faced a common technical challenge, which is the smoothness and real-time of video.
https://mp.weixin.qq.com/s/6VyVeKFs9nJSGHvZbnW3xQ
transporting network
What do you know about instant messaging technology?
From the beginning of the browser, the mode in which the client requests the server and the server returns the result has not changed.
https://juejin.im/post/6876602732464537608
Codec
Demystifying wasm+h265 live broadcast player
The IVWEB team is responsible for the business development of live broadcast scenes such as NOW live broadcast. Under the background of exploring the landing of wasm technology, a live broadcast player based on wasm has been implemented.
https://juejin.im/post/6877058224424976392
Audio and video technology
Audio and video applet development notes
First of all, you need to know how to "Quick Run through Demo". The implementation of this cloud game applet is based on TRTC two-person call. The custom component of TRTC intercommunication is realized through the <trtc-room> tag, which is based on the two media components of the WeChat applet native <live-pusher> and <live-player>. The live-player is mainly used here, which is used to play video streams of cloud games.
https://tuaran.github.io/views/2020/cgamemp.html#%E5%B0%8F%E7%A8%8B%E5%BA%8F%E9%9F%B3%E8%A7%86%E9%A2%91%E5%8E%9F%E7%90%86
Immersive light field video represented by "layered grid"
Usually the videos we see are flat or two-dimensional, and the more three-dimensional is the 3D effect of the left and right eyes that we see in the cinema with polarized glasses. But that is not really immersive video. So what is it? It is possible to change the viewing point of view in the video.
https://mp.weixin.qq.com/s/fzqoAPsSpL1dSLsSTwoMEg
Pure web video editing/conversion tool
Our product hopes to be able to edit the highlights of the anchor during the playback. As the front end, we should listen to it. After all, video cutting work has been done in the background for a long time. However, this time, as the front end of IVWEB, we decided to take Start wasm to try it.
https://juejin.im/post/6877054091357487118
How far is 8K from us?
8K is currently moving towards the application stage: 8K TVs of major brands are now available; YouTube and Vimeo have thousands of 8K short videos; 8K has begun to be used in major sports events.
https://mp.weixin.qq.com/s/Wm3gFLRmRFaCJs5vAgPiiw
Super-resolution technology based on machine learning
This article is compiled from Bitmovin's VideoTech Deep Dive: Super-Resolution with Machine Learning blog series. The author is Adithyan Ilangovan. It introduces the basic concepts of super-resolution and related technologies.
https://mp.weixin.qq.com/s/p-hBXu7sLeqRYphZ04zUhw
AI intelligence
A breakthrough in the field of image enhancement! To process 4K images at a speed of 1.66ms, Hong Kong Polytechnic proposes image-adaptive 3DLUT
This article is another subversive achievement of Hong Kong Polytechnic University's Zhang Lei and his students in the field of image enhancement. It combines deep learning technology with traditional 3DLUT image enhancement technology to obtain a more flexible and efficient image enhancement technology. The proposed method can enhance 4K resolution images at a speed of 1.66ms (hardware platform: Titan RTX GPU).
https://mp.weixin.qq.com/s/JFwroAiHYMwvgvUYXY21kg
Take a shot, Fu Jian: AI can already generate comics!
The project used AI to create a new comic called "PHAEDO", which was published in the weekly magazine "Morning". In order to generate new characters and stories in comics, the project team used NVIDIA StyleGAN to analyze hundreds of Tezuka’s classic works, including Phoenix, Black Jack, and Astro Boy, and used these comics as training materials to generate new comic characters. .
https://mp.weixin.qq.com/s/5FVmFTgWH-WegyKWYogafw
100-year-old photo restoration algorithm, those high-value parents!
Blurred, wrinkled photo repair, is Microsoft's latest CVPR article in 2020: "Bringing Old Photo Back to Life"
Coloring black and white photos is a classic old coloring algorithm (2018): "DeOldify"
https://mp.weixin.qq.com/s/hsMppcLioJbXSZ2ZL8bnug
7 Papers & Radios | Huawei Noah's new method of end-to-end lane line detection; overview of network representation learning
This week’s important papers include the new lane line detection method proposed by Huawei's Noah’s Ark Laboratory in conjunction with Sun Yat-sen University, and a review of network representation learning published in the journal Computer Science.
https://mp.weixin.qq.com/s/sC24h_xgSR6-0oE3ti1uHA
image
iOS image processing technology tracking-Core Image
Core Image is an image processing framework officially provided by Apple. It efficiently processes static pictures, dynamic pictures or videos through a rich built-in (built-in) or custom Filter (filter). Developers can also achieve richer effects by constructing a Filter chain or customizing the Core Image Kernel.
https://juejin.im/post/6876268620134121480
Android OpenCV: Image perspective transformation
Perspective transformation (Perspective Transformation) refers to the use of the three-point collinear condition of the perspective center, the image point, and the target point to rotate the bearing surface (perspective surface) around the trace (perspective axis) by a certain angle according to the perspective rotation law, destroying the original Some projection light beams can still maintain the constant transformation of the projection geometry on the shadow bearing surface.
https://juejin.im/post/6876014483777519629
Consultation Quick Fact
Regarding that Apple charges too much, foreign manufacturers have begun to team up to resist
What we should be curious about is not why developers choose to attack at this time, but why developers did not attack earlier.
https://mp.weixin.qq.com/s/uzvcQtTZjl9gCdcew83sEg