1. What is WebRTC
WebRTC, whose name is derived from the abbreviation of Web Real-Time Communication (English: Web Real-Time Communication), is an API that supports web browsers for real-time voice conversations or video conversations. It was open sourced on June 1, 2011 and was included in the W3C recommendation of the World Wide Web Consortium with the support of Google, Mozilla, and Opera.
WebRTC (Web RealTime Communication) was purchased by Google from Global IP Solutions for $ 68.29 million in 2010 and was
open sourced in 2011. It aims to build a real-time communication platform between Internet browsers and make WebRTC technology a standard of H5. One.
The official website (https://webrtc.org) is as follows:
From the description on the official website, we can know that WebRTC is a free open project that provides real-
time communication (RTC) functions for browsers and mobile applications through a simple API .
2. Introduction to WebRTC framework
We can see all the codes of webrtc from https://webrtc.googlesource.com/
Explanation of the color identification of the architecture diagram:
(1) The purple part is the web developer API layer; \ color {purple} {The purple part is the web developer API layer;} The purple part is the web developer API layer;
(2) The blue solid line Part is the API layer for browser manufacturers (that is, the module inside the red box, which is also my focus on research) \ color {blue} {The solid blue part is the API layer for browser manufacturers (that is, the red box The inner module is also a part of my focus on research)} The blue solid line is the API layer for browser manufacturers (that is, the red framed inner module is also the part of my focus)
(3) The blue dotted line is the browser manufacturer Can be customized to achieve \ color {blue} {Blue dotted line browser manufacturers can customize the implementation} Blue dotted line browser manufacturers can customize the implementation
Detailed component introduction
(1) Your Web App
The program developed by the Web developer. The Web developer can develop real-time communication applications based on video and audio based on the web API provided by the browser integrated with WebRTC.
(2) Web API
is a WebRTC standard API (Javascript) for third-party developers, which enables developers to easily develop web applications similar to online video chat. The latest standardization process can be viewed here.
(3) WebRTC Native C ++ API The
local C ++ API layer makes it easy for browser manufacturers to implement the WebRTC standard Web API and process digital signal processes abstractly.
(4) Transport / Session
transmission / session layer The
session layer components are implemented using some components of the libjingle library, without using the xmpp / jingle protocol
a.RTPStack protocol stack \ color {blue} {a.RTP Stack protocol stack} a.RTPStack protocol stack
Real Time Protocol
b.STUN / ICE \ color {blue} {b. STUN / ICE} b. STUN / ICE
can establish call connections between different types of networks through STUN and ICE components.
c.SessionManagement \ color {blue} {c. Session Management} c.SessionManagement is
an abstract session layer that provides session establishment and management functions. This layer of protocol is left to the application developers to customize their implementation.
(5) VoiceEngine
The audio engine is a framework that contains a series of audio multimedia processing, including the entire solution from the video capture card to the network transmission end.
PS: VoiceEngine is one of the most valuable technologies of WebRTC. It is open source after Google acquired GIPS. In VoIP, the technology industry is leading, later articles will learn more
a. iSAC
Internet Speech Audio Codec is
a broadband and ultra-wideband audio codec for VoIP and audio streaming. It is the default codec of the WebRTC audio engine.
Sampling frequency: 16khz, 24khz, 32khz; (default is 16khz)
Adaptive rate is 10kbit / s ~ 52kbit /;
adaptive packet size: 30 ~ 60ms;
algorithm delay: frame + 3ms
b. iLBC
Internet Low Bitrate Codec
Narrowband voice codec for VoIP audio stream
Sampling frequency: 8khz;
20ms frame bit rate 15.2kbps
30ms frame bit rate 13.33kbps
standard defined by IETF RFC3951 and RFC3952
c. NetEQ for Voice NetEQ algorithm
for audio signal processing components implemented by audio software
: adaptive jitter control algorithm and voice packet loss hiding algorithm. This enables it to adapt to changing network environments quickly and with high resolution, ensuring beautiful sound quality and minimum buffer delay.
It is a unique technology of GIPS, which can effectively deal with the impact of voice quality caused by network jitter and voice packet loss.
PS: NetEQ is also a very valuable technology in WebRTC. It has a significant effect on improving the quality of VoIP. It is better to use AEC \ NR \ AGC and other modules for integration.
d. Acoustic Echo Canceler (AEC)
is a software-based signal processing component that can remove the echo collected by mic in real time.
e. Noise Reduction (NR)
is also a software-based signal processing component used to eliminate certain types of background noise (hiss, fan noise, etc ...) associated with VoIP.
f. Opus: supports constant and variable bit rate encoding from 6 kbit / s to 510 kbit / s, frame size from 2.5 ms to 60 ms, and various sampling rates from 8 kHz (4 kHz
bandwidth) to 48 kHz (20 kHz bandwidth, which can replicate the entire hearing range of the human hearing system). Defined by IETF RFC 6176.
(6) VideoEngine
The WebRTC video processing engine
VideoEngine is an overall framework that includes a series of video processing, from the camera to capture video to the video information network transmission to the entire process of video display solutions.
a. VP8
video image codec is the default codec of WebRTC video engine.
VP8 is suitable for real-time communication application scenarios, because it is mainly a codec designed for low latency.
PS: The VPx codec was open sourced after Google acquired ON2. VPx is now part of the WebM project, which is one of the HTML5 standards that Google is committed to promoting
b. Video Jitter Buffer
video jitter buffer can reduce the adverse effects caused by video jitter and video packet loss.
c. Image enhancements
image quality enhancement module
to process the image collected by the webcam, including lightness and darkness detection, color enhancement, noise reduction processing and other functions, used to improve video quality.
3. Modules explain in detail
WebRTC has three modules, Voice Engine (Video Engine), Video Engine (Video Engine), Transport.
Voice Engine includes iSAC / iLBC Codec (audio codec, the former is for broadband and ultra-wideband, the latter is for narrowband), NetEQ for voice (processing network jitter and voice packet loss), Echo Canceler (echo canceler), Noise Reduction (noise suppression);
Video Engine includes VP8 Codec (video image codec), Video jitter buffer (video jitter buffer, to deal with video jitter and video packet loss), Image enhancements (image quality enhancement).
Transport includes SRTP (secure real-time transport protocol for audio and video streaming), Multiplexing (multiplexing), P2P, STUN + TURN + ICE (for NAT network and firewall traversal).
Secure transmission may also use DTLS (Datagram Secure Transmission) for encrypted transmission and key agreement. The entire WebRTC communication is based on UDP
Domestic solution manufacturers
Sound Network, Instant Technology, Huanxin, Rongyun and other companies are all developing their own audio and video call solutions based on WebRTC.
Soundnet https://www.agora.io/cn/JiGo
Technology https://www.zego.im/
Development Prospects of
WebRTC Although WebRTC is called "web", it is not limited to the terminal operating environment of traditional Internet applications or browsers. In fact, no matter the terminal operating environment is a
browser, desktop application, mobile device (Android or iOS) or IoT device, as long as the IP connection is reachable and meets the WebRTC specification, it can communicate.
This releases the real-time communication capabilities of a large number of smart terminals (or apps running on smart terminals) and opens up
imagination spaces for many application scenarios that require high real-time interactivity , such as online education, video conferencing, video social networking, and remote Assistance, remote control, etc. are all suitable applications.
The world's leading technology research and advisory firm Technavio recently released a report entitled "Global Network real-time communication (WebRTC) market, 20,172,021," the newspaper
reported. The report shows that during 20172021, the global Web Real-Time Communication (WebRTC) market will grow at a compound annual growth rate of 34.37%, which is a
very rapid growth . The growth is mainly from North America, Europe and Asia Pacific.
