Article directory
1. What is WebRTC
WebRTC, whose name comes from the abbreviation of Web Real-Time Communication (English: Web Real-Time Communication), is an API that supports web browsers for real-time voice or video conversations. It was open sourced on June 1, 2011 and was included in the W3C recommendation standard of the World Wide Web Consortium with the support of Google, Mozilla, and Opera.
WebRTC (Web RealTime Communication) was purchased by Google from Global IP Solutions for US$68.29 million in 2010, and it was
open sourced in 2011, aiming to establish a platform for real-time communication between Internet browsers, making WebRTC technology one of the H5 standards one.
The official website ( https://webrtc.org ) is introduced as follows:
From the description on the official website, we can know that WebRTC is a free and open project that provides real-time communication (RTC) functionality for browsers and mobile applications through a simple API
.
2. WebRTC framework introduction
We can see all the codes of webrtc from https://webrtc.googlesource.com/
The description of the color identification of the architecture diagram:
(1) The purple part is the Web developer API layer; \color{purple}{The purple part is the Web developer API layer;} The purple part is the Web developer API layer; (2) The blue solid line part is for the
browser The API layer of the manufacturer (that is, the module in the red frame, which is also the part I focus on research) \color{blue}{The blue solid line part is the API layer for browser manufacturers (that is, the module in the red frame, which is also the part I focus on) The part that focuses on research)} The blue solid line part is the API layer for browser manufacturers (that is, the module in the red box is also the part that I focus on research) (3) The
blue dotted line part can be customized by browser manufacturers\ color{blue}{The blue dotted line part of the browser manufacturers can customize the implementation} The blue dotted line part of the browser manufacturer can customize the implementation
Detailed component introduction
(1) Your Web App
is a program developed by web developers. Web developers can develop real-time communication applications based on video and audio based on the web API provided by the browser integrated with WebRTC.
(2) Web API
The WebRTC standard API (Javascript) for third-party developers enables developers to easily develop web applications similar to online video chat. The latest standardization process can be viewed here.
(3) WebRTC Native C++ API
The native C++ API layer makes it easy for browser manufacturers to implement the WebRTC standard Web API and abstractly process the digital signal process.
(4) Transport/Session
The session
layer components are implemented using some components of the libjingle library, without the need to use the xmpp/jingle protocol
a.RTPStack protocol stack \color{blue}{a.RTP Stack protocol stack}a.RTPStack protocol stack Real Time Protocol
b. STUN / ICE \color{blue}{b. STUN/ICE } b. STUN/ICE
can establish call connections between different types of networks through STUN and ICE components.
c. Session Management \color{blue}{c. Session Management} c. Session Management is
an abstract session layer that provides session establishment and management functions. This layer protocol is left to the application developer to customize the implementation.
(5) VoiceEngine
The audio engine is a framework that includes a series of audio multimedia processing, including the entire solution from the video capture card to the network transmission end.
PS: VoiceEngine is one of the most valuable technologies of WebRTC, which was open sourced after Google acquired GIPS. On VoIP, the technology is leading in the industry, and the following articles will learn more about it
-
a. iSAC
Internet Speech Audio Codec
is a wideband and ultra-wideband audio codec for VoIP and audio streaming. It is the default codec for the WebRTC audio engine.
Sampling frequency: 16khz, 24khz, 32khz; (the default is 16khz)
The adaptive rate is 10kbit/s ~ 52kbit/;
adaptive packet size: 30~60ms;
algorithm delay: frame + 3ms -
b. iLBC
Internet Low Bitrate Codec
VoIP audio stream narrowband voice codec
Sampling frequency: 8khz;
20ms frame bit rate 15.2kbps
30ms frame bit rate 13.33kbps
The standard is defined by IETF RFC3951 and RFC3952 -
c. NetEQ for Voice
is a voice signal processing component NetEQ algorithm implemented by audio software
: adaptive jitter control algorithm and voice packet loss concealment algorithm. This enables it to quickly and high-resolution adapt to changing network conditions, ensuring great audio quality with minimal buffering delays.
It is the unique technology of GIPS, which can effectively deal with the impact on voice quality due to network jitter and voice packet loss.
PS: NetEQ is also a very valuable technology in WebRTC. It has a significant effect on improving the quality of VoIP. It can be integrated with modules such as AEC\NR\AGC, and the effect will be better. -
d. Acoustic Echo Canceler (AEC)
is a software-based signal processing component that can remove the echo collected by the mic in real time. -
e. Noise Reduction (NR)
is also a software-based signal processing element used to remove certain types of background noise associated with VoIP (hiss, fan noise, etc...) -
f. Opus: Supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame size from 2.5 ms to 60 ms, various sampling rates from 8 kHz (4 kHz bandwidth) to 48 kHz (
20 kHz bandwidth, which replicates the entire hearing range of the human auditory system). Defined by IETF RFC 6176.
(6) VideoEngine
WebRTC video processing engine
VideoEngine is an overall framework that includes a series of video processing, from camera video capture to video information network transmission to video display solutions for the entire process.
-
a. VP8
video image codec is the default codec of the WebRTC video engine.
VP8 is suitable for real-time communication application scenarios because it is mainly a codec designed for low latency.
PS: The VPx codec was open sourced after Google acquired ON2, and VPx is now part of the WebM project, which is one of the HTML5 standards that Google is committed to promoting -
b. Video Jitter Buffer
The video jitter buffer can reduce the adverse effects caused by video jitter and video packet loss. -
c. Image enhancements
Image quality enhancement module
Processes the images collected by the network camera, including functions such as brightness detection, color enhancement, and noise reduction processing, to improve video quality.
3. Detailed explanation of the module
WebRTC has three modules, Voice Engine (audio engine) , Video Engine (video engine) , Transport .
-
Voice Engine includes iSAC/iLBC Codec (audio codec, the former is for broadband and ultra-wideband, the latter is for narrowband), NetEQ for voice (handling network jitter and voice packet loss), Echo Canceler (echo canceller), Noise Reduction (noise suppression);
-
Video Engine includes VP8 Codec (video image codec), Video jitter buffer (video jitter buffer, processing video jitter and video packet loss), Image enhancements (image quality enhancement).
-
Transport includes SRTP (secure real-time transport protocol for audio and video streaming), Multiplexing (multiplexing), P2P, STUN+TURN+ICE (for NAT network and firewall traversal).
Secure transmission may also use DTLS (Datagram Secure Transmission) for encrypted transmission and key agreement. The entire WebRTC communication is based on UDP
Domestic solution manufacturers
Companies such as Shengwang, Jigou Technology, Huanxin, and Rongyun are all developing their own audio and video call solutions based on WebRTC.
Sound Network https://www.agora.io/cn/
Instant Technology https://www.zego.im/
WebRTC Development Prospects
Although WebRTC is named "web", it is not limited to the terminal operating environment of traditional Internet applications or browsers. In fact, regardless of whether the terminal operating environment is
a browser, desktop application, mobile device (Android or iOS) or IoT device, as long as the IP connection is reachable and complies with the WebRTC specification, they can communicate.
This releases the real-time communication capabilities of a large number of smart terminals (or apps running on smart terminals), and opens up the imagination of many application scenarios that require high real-time interactivity, such as
online education, video conferencing, video social, remote Assistance, remote control, etc. are suitable fields of application.
Technavio, the world's leading technology research and consulting company, recently released a report titled "Global Web Real-Time Communication (WebRTC) Market, 2017-2021"
. According to the report, during the period from 2017 to 2021, the global web real-time communication (WebRTC) market will grow at a compound annual growth rate of 34.37%, which is a
very rapid growth. The growth came mainly from North America, Europe and Asia Pacific.
Article reference:
- https://blog.csdn.net/fishmai/article/details/69681595
- https://blog.csdn.net/u011077027/article/details/86225524
Reprinted: https://blog.csdn.net/u011077027/article/details/99836025