I. Introduction
WebRTC technology has been widely used in various industries and scenarios, but for most developers, real-time audio and video and related technologies are relatively uncommon.
As a web developer, it took a lot of time to understand the concept of WebRTC. First, WebRTC itself has many unique concepts. Second, although it has the word "Web", it relies on the underlying concepts and the network. But it is rarely touched by Web development;
This article uses the perspective of audio and video developers with 0 experience, and compares commonly used Web technologies, hoping to help you get started with WebRTC technology easily. After reading this article patiently, you will:
- Understand what is WebRTC
- Master the principles of WebRTC calls
- Use Chrome to debug WebRTC applications
Suitable for reading: Web development, students who have js foundation and are interested in WebRTC
2. Example of use
Students who have not been exposed to WebRTC technology can first experience ZEGO's GoEnjoy product, which contains the standard usage scheme of WebRTC in the browser, including but not limited to: device detection, compatibility detection, weak network disconnection coping strategies, etc., application It’s free, you can stamp it— > Example Demo Portal
Before entering the text, let us have a basic impression of it!
figure 1
3. Brief introduction
After experiencing the demo, it is necessary to learn more about the development history and application scenarios of the technology, which will let us know why it is excellent, which aspects are excellent, and what are its shortcomings.
Programmers often use the 5W1H analysis method, so this article will introduce you according to this idea:
What
WebRTC (Web Real-Time Communication), a framework (APIs) that allows users to use their own traffic to realize audio and video real-time communication , supports browsers (Firefox, Chrome, Safari) and native systems of iOS and Android.
When
It became a W3C draft in December 2017, and the domestic WeChat browser only supported it in the second half of 2019. There are still many compatibility issues with the browsers that come with domestic mobile phones. On January 26, 2021, it became an official W3C standard.
Who
In 2011, Google acquired several sub-projects (GIPS, On2, VPx) and established the current WebRTC project, which is currently an open source project of Google.
Where
It can be applied in social/entertainment/education/tools and other scenarios that require real-time audio and video to communicate efficiently, for example: the metaverse that is very popular recently.
Why
W3C standard, open source, plug-in, good overall effect.
How
It is also the focus of this article, and the ultimate goal is to let everyone know how to use it.
Before the official code is explained, there are some concepts that need to be popularized** ( you can also read the code first, and then come back to this paragraph to deepen your understanding. ):**
- MediaStream: Streaming media object, an encapsulation format of audio/video data, mounted on the video or audio tag for playback;
- **RTCPeerConnection: ** Session control, network and media information sending and receiving, the function is similar to http object;
- **SDP: ** Mainly used for media negotiation between two session entities, similar to configuration items in http.
It will be easier to understand with the following analogy:
figure 2
4. Pre-thinking questions
Before explaining the code, you need to think about the following questions, otherwise you will not know why SDP, cadidate, etc. need to be exchanged in the code (you can also read the code first, and then come back to this paragraph to deepen your understanding).
The two parties use browsers to communicate, and the inconsistency of browser capabilities and network conditions will have a great impact on communication. Let's think about the following two questions:
1. Different video encoding capabilities?
Peer-A and peer-B are browsers on both sides of the video interaction. Before they communicate, they must reach an agreement on the video encoding capability, as shown in the figure below, and finally negotiate a common H264. If they cannot reach an agreement, the communication will fail.
image 3
2 Between computers, most of them are in a local area network, which requires NAT (Network Address Translation, Network Address Translation), so they cannot communicate directly;
The display is as follows:
Figure 4
A popular analogy: A Zhai is 30 this year (not me, don't guess) and was forced to marry by his parents. He can only ask a matchmaker for help before he can be known by another Zhai.
The matchmaker solves the problem of social terror in the house. NAT also needs a way to bypass, so that the two parties can establish communication. We need to use STUN and TURN.
Five, code explanation
Finally, we have come to our code explanation part. The following code will explain the APIs needed for each step in stages according to the order of the push section ( if you are looking at the code directly, it is recommended to go back and read the third and fourth after reading the code. Part introduction, the understanding will be more profound).
Step 1: Create a data source
localStream as the sender's local preview screen:
// 创建数据源
const localStream = await navigator.mediaDevices.getUserMedia({
video: true,
audio: true,
});
// 显示数据源，localVideo 是 html 中的 video 标签
localVideo.srcObject = localStream;
Step 2: Create an instance of sending data
For sending the data created in step 1:
// 本地实例
const pc1 = new RTCPeerConnection();
// 对端实例
const pc2 = new RTCPeerConnection();
Step 3: Configure the instance
The purpose of this step is to exchange information at both ends**: icecandidate** and SDP
- icecandidate: includes communication protocol (TCP/UDP) and communication IP, STUN and TURN protocols describe the format specification of network information , and solve the problem of network connection between the two parties;
- SDP: Browser capabilities, including but not limited to audio and video encoding formats, bandwidth, flow control strategies, etc.; to solve the problem of mismatching capabilities between the two parties in the pre-thinking, by exchanging the SDP browsers of both parties, the browsers will automatically select the video encoding format supported by both parties.
// 告诉对端，本端地址
pc1.addEventListener('icecandidate', async (e) => {
// 发送给对端
// 对端添加本端地址
if (e.candidate) {
await pc2.addIceCandidate(e.candidate);
}
});
pc2.addEventListener('icecandidate', async (e) => {
// 发送给本端
// 本端添加对端地址
if (e.candidate) {
await pc1.addIceCandidate(e.candidate);
}
});
// 创建本端SDP,告诉本端浏览器支持哪些能力
const offer = await pc1.createOffer();
pc1.setLocalDescription(offer);
// 创建远端SDP,告诉远端浏览器支持哪些能力
const answer = await pc2.createAnswer();
pc2.setLocalDescription(answer);
// 。。。。发送远端SDP给本端
// 接收远端sdp,告诉远端浏览器支持哪些能力
pc1.setRemoteDescription(answer);
// 接收客户端sdp,告诉远端浏览器支持哪些能力
pc2.setRemoteDescription(offer);
Step 4: Send data
localStream.getTracks().forEach(
(track) => pc1.addTrack(track, localStream)
);
Step 5: Complete and simplified Typescript code
Note that the typescript used here is written, and the actual operation needs to be converted into js first.
const pc1 = new RTCPeerConnection();
pc1.addEventListener('icecandidate', async (e) => {
if (e.candidate) {
await pc2.addIceCandidate(e.candidate);
}
});
pc1.addEventListener('iceconnectionstatechange', (e) => {
console.log('pc1: iceconnectionstatechange', e);
});
const pc2 = new RTCPeerConnection();
pc2.addEventListener('icecandidate', async (e) => {
if (e.candidate) {
await pc1.addIceCandidate(e.candidate);
}
});
pc2.addEventListener('iceconnectionstatechange', (e) => {
console.log('pc2: iceconnectionstatechange', e);
});
pc2.addEventListener('track', (e) => {
if (e.streams.length > 0) {
remoteVideo.srcObject = e.streams[0];
}
});
const remoteVideo = document.querySelector('#remoteVideo') as HTMLVideoElement;
const localVideo = document.querySelector('#localVideo') as HTMLVideoElement;
async function pushStream(answer: RTCSessionDescriptionInit) {
pc1.setRemoteDescription(answer);
}
async function pullStream(offer: RTCSessionDescriptionInit): Promise<void> {
pc2.setRemoteDescription(offer);
const answer = await pc2.createAnswer();
pc2.setLocalDescription(answer);
console.warn('answer', answer);
pushStream(answer);
}
window.onload = async () => {
const localStream = await navigator.mediaDevices.getUserMedia({
video: true,
audio: true,
});
localVideo.srcObject = localStream;
localStream.getTracks().forEach((track) => pc1.addTrack(track, localStream));
const offer = await pc1.createOffer();
pc1.setLocalDescription(offer);
console.warn('pc1 offer', offer);
pullStream(offer);
};
6. Application examples
After learning the theoretical knowledge, let's practice together to deepen our understanding of the knowledge - debug the WebRTC application through the Chrome browser. Learning to debug can not only deepen your understanding, but also an essential skill for subsequent code writing. Don’t miss this step:
1. Click to open the sample DEMO
2. Open another tab page and enter: chrome://webrtc-internals/
3. Enter relevant information in DEMO to start the live broadcast, and switch back to the tab page in 2, as shown below:
Blue part: corresponds to the processing of SDP in the code
Green part: Corresponding to the network link situation
Figure 5
4. Continue, you can see the real-time data changes in the push stream:
Blue part: pull streaming real-time data, including resolution, bit rate, packet loss rate, etc.
Green part: real-time streaming data, including resolution, bit rate, packet loss rate, etc.
Figure 6
For more field understanding, you can click here for in-depth study: Identifiers for WebRTC's Statistics API
Seven, the end
With the upgrading of hardware and network, we have gone through the process of changing from text->picture->video carrier. With the popularization of 5G, the integration of audio and video technology into the invisible is becoming a reality. WebRTC is one of the most important frameworks, and the maturity of browser support is also rapidly improving. Continue to learn WebRTC technology and pay attention to ZEGO instant technology!
Attached are the author's commonly used tools, which are recommended for collection:
1 WebRTC samples . Google official website demo, including the latest features
2 MediaDevices - Web APIs | MDN specific interface introduction
3 WebRTC 1.0: Real-Time Communication Between Browsers (W3C standard introduction)
Digression
Many people who are new to the computer industry or graduates of computer-related majors have encountered obstacles everywhere due to lack of practical experience. Let's look at two sets of data:
-
The 2023 national college graduates are expected to reach 11.58 million, and the employment situation is severe;
-
According to the data released by the National Network Security Publicity Week, by 2027, the shortage of network security personnel in my country will reach 3.27 million.
On the one hand, the employment situation of fresh graduates is severe every year, and on the other hand, there is a gap of one million cyber security talents.
On June 9, the 2023 edition of the Employment Blue Book of MyCOS Research (including the 2023 Employment Report for Undergraduates in China and the Employment Report for Higher Vocational Students in China in 2023) was officially released.
Top 10 Majors with Higher Monthly Salary for 2022 College Graduates
The monthly income of undergraduate computer science majors and higher vocational automation majors is relatively high. The monthly income of the 2022 class of undergraduate computer science and higher vocational automation majors is 6,863 yuan and 5,339 yuan, respectively. Among them, the starting salary of undergraduate computer majors is basically the same as that of the 2021 class, and the monthly income of higher vocational automation majors has increased significantly. The 2022 class of overtaking railway transportation majors (5295 yuan) ranks first.
Specifically, depending on the major, the major with a higher monthly income for undergraduates in 2022 is information security (7579 yuan). Compared with the class of 2018, undergraduate majors related to artificial intelligence, such as electronic science and technology, automation, performed well, and their starting salaries increased by 19% compared with five years ago. Although data science and big data technology are newly added majors in recent years, they have performed well, and have ranked among the top three majors with higher monthly income half a year after graduation for the 2022 class of undergraduates. The only humanities and social science major that entered the top 10 undergraduate high-paying list five years ago-French has dropped out of the top 10.
"There is no national security without cybersecurity". At present, network security has been elevated to the height of national strategy and has become one of the most important factors affecting national security and social stability.
Characteristics of the network security industry
1. The employment salary is very high, and the salary rises quickly. In 2021, Liepin.com released the highest employment salary in the network security industry, which is 337,700 yuan per capita!
2. There is a large talent gap and many employment opportunities
On September 18, 2019, the official website of the "Central People's Government of the People's Republic of China" published: my country needs 1.4 million cyberspace security talents, while major schools across the country train less than 1.5 million people each year. Liepin.com's "Cyber Security Report for the First Half of 2021" predicts that the demand for cyber security talents will be 3 million in 2027, and there are only 100,000 employees currently engaged in the cyber security industry.
The industry has a lot of room for development and many jobs
Since the establishment of the network security industry, dozens of new network security industry positions have been added: network security experts, network security analysts, security consultants, network security engineers, security architects, security operation and maintenance engineers, penetration engineers, information security management Data Security Engineer, Network Security Operations Engineer, Network Security Emergency Response Engineer, Data Appraiser, Network Security Product Manager, Network Security Service Engineer, Network Security Trainer, Network Security Auditor, Threat Intelligence Analysis Engineer, Disaster Recovery Professional , Actual combat offensive and defensive professionals...
Great career potential
The network security major has strong technical characteristics, especially mastering the core network architecture and security technology in the work, which has an irreplaceable competitive advantage in career development.
With the continuous improvement of personal ability, the professional value of the work will also increase with the enrichment of one's own experience and the maturity of project operation, and the appreciation space is bullish all the way, which is the main reason why it is popular with everyone.
To some extent, in the field of network security, just like the doctor profession, the older you are, the more popular you become. Because the technology becomes more mature, the work will naturally be valued, and promotion and salary increase are a matter of course.
Hacking & Cyber Security How to Learn
Today, as long as you give my article a thumbs-up, I will share my private collection of online security learning materials with you for free, so let’s see what is there.
1. Learning Roadmap
There are also many things to learn in attack and defense. I have written all the specific things to learn in the roadmap above. If you can learn them, you will have no problem getting a job or taking private jobs.
2. Video Tutorial
Although there are many learning resources on the Internet, they are basically incomplete. This is a video tutorial on cyber security recorded by myself. I have a supporting video explanation for every knowledge point in the above roadmap.
The content covers the study of network security law, network security operation and other guarantee assessment, penetration testing basics, detailed explanation of vulnerabilities, basic computer knowledge, etc., which are all learning contents that must be known when getting started with network security.
(It’s all packed into one piece and cannot be unfolded one by one. There are more than 300 episodes in total)
Due to limited space, only part of the information is shown, you need to click the link below to get it
CSDN spree: "Hacker & Network Security Introduction & Advanced Learning Resource Pack" free sharing
3. Technical documents and e-books
The technical documents are also compiled by myself, including my experience and technical points of participating in large-scale network security operations, CTF and SRC vulnerability mining. There are also more than 200 e-books. Due to the sensitivity of the content, I will not show them one by one.
Due to limited space, only part of the information is shown, you need to click the link below to get it
CSDN spree: "Hacker & Network Security Introduction & Advanced Learning Resource Pack" free sharing
4. Toolkit, interview questions and source code
"If you want to do a good job, you must first sharpen your tools." I have summarized dozens of the most popular hacking tools for everyone. The scope of coverage mainly focuses on information collection, Android hacking tools, automation tools, phishing, etc. Interested students should not miss it.
There is also the source code of the case and the corresponding toolkit mentioned in my video, which can be taken away if needed.
Due to limited space, only part of the information is shown, you need to click the link below to get it
CSDN spree: "Hacker & Network Security Introduction & Advanced Learning Resource Pack" free sharing
Finally, there are interview questions about Internet security that I have sorted out in the past few years. If you are looking for a job in Internet security, they will definitely help you a lot.
These questions are often encountered in interviews with Sangfor, Qi Anxin, Tencent or other major companies. If you have good questions or good insights, please share them.
Reference analysis: Sangfor official website, Qi Anxin official website, Freebuf, csdn, etc.
Content features: clear organization, including graphic representation, which is easier to understand.
Summary of content: Including intranet, operating system, protocol, penetration test, security service, vulnerability, injection, XSS, CSRF, SSRF, file upload, file download, file inclusion, XXE, logic vulnerability, tool, SQLmap, NMAP, BP, MSF…
Due to limited space, only part of the information is shown, you need to click the link below to get it
CSDN spree: "Hacker & Network Security Introduction & Advanced Learning Resource Pack" free sharing