I. Introduction
WebRTC (Web Real-Time Communication) aims to introduce real-time communication functions to browsers. Users can perform real-time communication functions between browsers without installing any other software or plug-ins. This article introduces the implementation of one-to-one audio and video real-time chat room function based on WebRTC. The browser requests the Web server front-end page running elements (HTML, CSS, JS) through HTTP, and then interacts with the signaling server. The signaling service provides room management. For functions such as message forwarding, media data is transferred through the STUN/TURN server.
2. Function display
As shown in the picture above, it is a 1v1 chat room. After joining the room, users can text chat with other users in the room, or click audio and video calls< a i=2> button to make audio and video calls.
3. Architecture description
The above is a description of the one-to-one audio and video chat room architecture. The left side is the Call initiator process, and the right side is the Call callee process. The initiator first detects audio and video equipment and collects audio and video equipment data, then creates a PeerConnection connection (peer-to-peer connection) with the STUN/TURN server, and generates an offer SDP and sends it to the callee. The callee receives the initiator's SDP. After that, it also starts to detect audio and video equipment and collect audio and video data, and then generates answer SDP and sends it to the initiator.
4. Code implementation
1. Server-side implementation
a. Web+Signal Server implementation
First we need to build a Web server for the browser to pull the front-end running elements. This article chooses the nodejs express module, which can easily build static files. Resource server, the server only needs to expose the path where the front-end code is located, and the browser can request it through HTTP.
Secondly, we also need to build a signaling server Signal Server, which maintains room status and provides functions such as message forwarding. We can receive user requests and process them via HTTP.
The following shows part of the server (Web+Signal Server) code, which exposes the public directory through express. The user obtains the front-end code and sends request messages to signaling in combination with user operations, such as joining a room request join_req, exiting a room request leave_req and some other needs. The signaling server forwards control messages to other users in the same room.
// 注意：如下代码不完整，仅提供关键部分
var logger = log4js.getLogger();
var app = express();
app.use(serveIndex('./public'));
app.use(express.static('./public'));
var options = {
key : fs.readFileSync('./certificate/server.key'),
cert : fs.readFileSync('./certificate/server.pem')
}
var httpsServer = https.createServer(options, app);
httpsServer.listen(443, '0.0.0.0');
var httpsIO = socketIO.listen(httpsServer);
httpsIO.sockets.on('connection', (socket) => {
socket.on('join_req', (roomId, userName) => {
socket.join(roomId);
var room = httpsIO.sockets.adapter.rooms[roomId];
var memberNumInRoom = Object.keys(room.sockets).length;
if (memberNumInRoom > MAX_NUMBER_IN_ROOM) {
socket.emit('member_full', roomId, socket.id);
socket.leave(roomId);
return;
}
fetchUserInRoom(socket, roomId);
addUserToRoom(roomId, userName);
logger.info(`${userName} join room: (${roomId}), current number in room is ${memberNumInRoom+1}`);
socket.emit('join_res', roomId, socket.id);
socket.to(roomId).emit('other_joined', roomId, socket.id, userName);
});
socket.on('leave_req', (roomId, userName) => {
socket.leave(roomId);
var room = httpsIO.sockets.adapter.rooms[roomId];
if (!room) {
return;
}
var memberNumInRoom = Object.keys(room.sockets).length;
delUserFromRoom(roomId, userName);
logger.info(`${userName} leave room: (${roomId}), current number in room is ${memberNumInRoom-1}`);
socket.emit('leave_res', roomId, socket.id);
socket.to(roomId).emit('other_leaved', roomId, socket.id, userName);
});
socket.on('ctrl_message', (roomId, data) => {
logger.debug('recv a ctrl message from ' + socket.id);
socket.to(roomId).emit('ctrl_message', roomId, socket.id, data);
});
socket.on('start_call', (roomId, data) => {
logger.debug("recv start_call from " + socket.id);
socket.to(roomId).emit('start_call', roomId, socket.id, data);
});
socket.on('message', (roomId, data) => {
logger.debug('recv a message from ' + socket.id);
socket.to(roomId).emit('message', roomId, socket.id, data);
});
});
b. Build a STUN/TURN server
As shown in the above architecture diagram, we also need to build a STUN/TURN server. The STUN server mainly provides the function of notifying the client of the NAT mapped address. The client knows its NAT mapped address and forwards it to the peer. The peer will also send the NAT mapped address to the client. Both ends try to communicate with each other. If the P2P connection is unsuccessful, the data needs to be transferred through the TURN server. The TURN server It is generally deployed on a machine with a public IP to ensure that data can be transferred through the TURN server when P2P fails between two clients in different network environments.
If you are not familiar with the working principle of STUN/TURN, you can refer to the working principle of STUN and the working principle of TURN. You can use coturn to build a STUN/TURN server. Please refer to this blog. 2. Client implementation
a. Create RTCPeerConnection
RTCPeerConnection is a component of WebRTC that implements point-to-point communication. It provides a way to connect peers and communicate data. Peer-to-peer connections need to listen for events and execute callbacks when events occur. For example, when an icecandidate event is received, local candidates need to be sent to the peer, when a media track event is received, etc.
var ICE_CFG = {
'iceServers': [{
'url': 'stun:stun.l.google.com:19302'
}, {
'url': 'turn:xxx.xxx.xxx.xxx:3478',
'username': 'xxx',
'credential': 'xxx'
}]
};
var pConn = new RTCPeerConnection(ICE_CFG);
stun:stun.l.google.com:19302 is the STUN server address provided by Google for P2P traversal. After you complete the TURN server setup, you can specify the TURN service address so that you can use the TURN service when P2P traversal fails. Perform relay forwarding.
b. Get the local stream and add it to PeerConnection
WebRTC provides the getUserMedia API, so users can easily access audio and video devices to obtain stream data.
Basic usage: var promise = navigator.mediaDevices.getUserMedia(constraints);
For the use of getUserMedia API, please refer to this blog.
function GetLocalMediaStream(mediaStream) {
localStream = mediaStream;
localAv.srcObject = localStream;
localStream.getTracks().forEach((track) => {
myPconn.addTrack(track, localStream);
});
if (isInitiator == true) {
var offerOptions = {
offerToReceiveAudio: 1,
offerToReceiveVideo: 1
};
myPconn.createOffer(offerOptions)
.then(GetOfferDesc)
.catch(HandleError);
}
}c. Media negotiation
For the call initiator, after adding the audio and video tracks to RTCPeerConnection, you can generate local SDP information through the createOffer method and set it to RTCPeerConnection using setLocalDescription. Then the SDP information is sent to the callee buddy through signaling. After receiving the offer SDP, the buddy uses RTCPeerConnection setRemoteDescription to set the SDP to remote sdp, then generates and sets the local SDP message through the RTCPeerConnection createAnswer method, and finally sends it to the initiator through signaling. .
For an introduction to SDP, please refer to this blog, and for an introduction to createOffer, please refer to this blog.
if (isInitiator == true) {
var offerOptions = {
offerToReceiveAudio: 1,
offerToReceiveVideo: 1
};
myPconn.createOffer(offerOptions)
.then(GetOfferDesc)
.catch(HandleError);
}
function GetOfferDesc(desc) {
myPconn.setLocalDescription(desc);
socket.emit('ctrl_message', roomId.value, desc);
}
function CreateAnswerDesc() {
myPconn.createAnswer().then(GetAnswerDesc).catch(HandleError);
}
function GetAnswerDesc(desc) {
myPconn.setLocalDescription(desc);
socket.emit('ctrl_message', roomId.value, desc);
}socket.on('ctrl_message', (roomId, socketId, data) => {
if (data) {
if (data.hasOwnProperty('type') && data.type === 'offer') {
HandleOfferDesc(data);
CreateAnswerDesc();
} else if (data.hasOwnProperty('type') && data.type === 'answer') {
HandleAnswerDesc(data);
} else if (data.hasOwnProperty('type') && data.type === 'candidate') {
HandleCandidate(data);
} else {
console.log('Unknow ctrl message, ' + data);
}
}
});d. Exchange candidate
As follows, when a PeerConnection is created, the onicecandidate callback function will be monitored. For example, for the call initiator, candidate collection will be started after calling setLocalDescription. When the collection is completed, The onicecandidate method will be called back, and then the local candidate information will be sent to the peer. After the peer receives the local candidate, it can start creating connections and select available connections from many connections for calls.
For a detailed description of candidate, please refer to this blog, which introduces host candidate, srflx candidate, relay candidate, etc.
When creating PeerConnection, the ontrack callback function is also set, which is the processing after receiving the peer stream data. This demo sets the stream to the <video> tag.
function CreatePeerConnection() {
if (myPconn) {
console.log('peer connection has already been created.');
return;
}
myPconn = new RTCPeerConnection(ICE_CFG);
myPconn.onicecandidate = (event) => {
if (event.candidate) {
socket.emit('ctrl_message', roomId.value, {
type: 'candidate',
label: event.candidate.sdpMLineIndex,
id: event.candidate.sdpMid,
candidate: event.candidate.candidate
});
}
}
myPconn.ontrack = GetRemoteMediaStream;
}function HandleCandidate(data) {
var candidate = new RTCIceCandidate({
sdpMlineIndex: data.label,
sdpMid: data.id,
candidate: data.candidate
});
myPconn.addIceCandidate(candidate);
}function GetRemoteMediaStream(event) {
if(event && event.streams.length > 0) {
remoteStream = event.streams[0];
remoteAv.srcObject = remoteStream;
}
}Original text Implementing browser-side audio and video chat room based on WebRTC_ice candidate exchange construction-CSDN blog
★The business card at the end of the article allows you to receive free audio and video development learning materials, including (FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, srs) and audio and video learning roadmap, etc.
See below! ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓