#Simple analysis of webrtc, combined with easyrtc and coturn (stun, turn) services to achieve audio and video chat
- The full name of webRTC is Web Real-Time Communications, that is, web real-time communication
- The premise of audio and video chat is different from that of live broadcast~ There are many types of APIs, and some are outdated. If you see something different, you can browse MDN and try another solution. Google needs https or localhost to enable media permissions, but Firefox does not. , but it takes a lot of work to be compatible. Open two web pages for testing under localhost, and I will talk about easyrtc later
Understanding websocket connection/websocket is the most prosperous field of node.js, http://socket.io is simple and rude https://github.com/socketio/socket.io.git
understands the workflow of webrtc juejin.cn/post/ 684490…
- The role of the stun server, to obtain the ip of both parties to assist in establishing a connection
- The role of the turn server, turn the voice video
- The role of the signaling server is realized by using http://socket.io , mainly to transfer the software and hardware information ip and device information of both parties in the voice and video chat, and assist in direct connection or video forwarding
Simplify the introduction of the article content, webrtc process, the following can run in the intranet
- 1. Obtain the user's audio and video media. After completing this step, you can see your own camera image
navigator.mediaDevices.getUserMedia({
audio: true,
video: true,
}).then((stream) => {
let video = document.querySelector('#video')
video.srcObject = stream // mediaDevices.getUserMedia获取到的音视频流捆绑在video标签上
video.onloadedmetadata = () => video.play() // 读取数据时进行播放
})2. Make a peertoer point-to-point connection,
- 2-1. In the homeowner createOffer callback, the homeowner’s information desc, audio and video structure software and hardware information can be obtained, stored locally, and sent to the signaling server to assist in forwarding
let peer = new RTCPeerConnection(servers)
pc.createOffer(sendOffer, function (error) {
console.log('发送 offer 失败')
})
function sendOffer (desc) {
console.log('sendOffer')
pc.setLocalDescription(desc);
socket.emit('offer', JSON.stringify({
data: {sdp: desc}
})
)
}2-2. When the homeowner creates Offer, it will trigger the onicecandidate event of its own connection to obtain the candidate, user address, etc.
pc.onicecandidate = function(event) {
if (event.candidate !== null) {
socket.emit("_ice_candidate",JSON.stringify({
type: '_candidate',
data: {
candidate: event.candidate
}
})
)
}
};2-3. When the other party receives the offer in the socket, set it as the remote description, and the offer corresponds to the answer. When the user is created, the onicecandidate event of the user will be triggered, and the user address information can be obtained. At this time, the description will be sent to the homeowner
socket.on('offer', function(e) {
getUserMedia(function (stream) {
let vid1 = document.getElementById('vid1')
vid1.srcObject = stream
vid1.onloadedmetadata = function(e) {
vid1.play();
};
pc.addStream(stream)
var json = JSON.parse(e)
pc.setRemoteDescription(new RTCSessionDescription(json.data.sdp));
pc.createAnswer(sendAnswer, function (error) {
console.log('创建answer失败')
})
}, function (error) {
console.log('摄像头获取失败', '接听视频失败');
})
})
function sendAnswer(desc) {
pc.setLocalDescription(desc);
socket.emit('answer', JSON.stringify({
type: '_answer',
data: {
sdp: desc
}
}))
}- 2-4. The homeowner sets it as the remote description in the socket event receiving the answer
socket.on('answer', async function(e) {
let data = e.message
await peer.setRemoteDescription(data)
})Implement point-to-point audio and video chat ICE on the external network (combine stun, turn to penetrate to obtain the addresses of both audio and video parties)
- STUN analysis
- But in reality, our video call cannot be limited to the intranet, it may be behind the firewall or NAT, and before the point-to-point connection, we need to check whether it can be performed. This technology is called NAT penetration, usually based on the UDP protocol, such as RFC3489. TCP penetration is supported in the new RFC5389
- TURN analysis
- TURN uses relays to penetrate NAT: the extension of STUN, like STUN, achieves the effect of NAT penetration by modifying the private network address in the application layer. The similarity and difference are that TURN achieves penetration through the "middleman" method of two-party communication . It's a bit like the meaning of the server translating audio and video streams. When STUN fails, it will go TURN
What is the role of NAT
- In order to alleviate the shortage of global ip, NAT can convert the private address of the internal network to the public address of the external network, so that Internet access can be performed
- Prevent external hosts from attacking internal hosts
Of course, front-end development is not familiar with these. After reading many RTC blogs, I learned that Google has open-sourced services related to stun and coturn. You only need to install the dependencies and encrypt the account password to use it. Reference https://www.cnblogs.com /idignew/p/7440048.html
After configuring the coturn, test the address, and return the relay attribute as the server address to prove success https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/
easyRTC
- After many tests, the compatibility of my own RTCdemo is poor, and I will not write it by hand at the current testing stage. Using easyRTC, except for safari and antique ie, other aspects are still ok. github.com/priologic/e…
Note that the beta branch is used for the latest project
The original webrtc principle and related api usage logic - Nuggets
★The business card at the end of the article can receive audio and video development learning materials for free, including (FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, srs) and audio and video learning roadmaps, etc.
see below!