OWT Server信令分析 (下) [Open WebRTC Toolkit]
目录
- 信令分析因为包含一些代码和格式，文章很长，所以分成上下两篇记录，OWT(Open WebRTC Toolkit)相关文章。
相关文章：
1. OWT Server信令分析
- OWT信令协议分为RESTful API和SocketIO长连接两部分，RESTful API由Management API提供，SocketIO长连接则由WebRTC Portal提供。
- 不过不是直接调用Management API，而是调用Conference Sample Server的接口，它是对Management API的一个封装，源码在owt-client-javascript https://github.com/open-webrtc-toolkit/owt-client-javascript/tree/master/src/samples/conference项目中。
- OWT Server WebRTC的信令交互过程如下：
A POST /tokens/
A SocketIO connect
A SocketIO login
A SocketIO publish
A SocketIO soac offer
A SocketIO soac candidate
Portal SocketIO soac answer
B POST /tokens/
B SocketIO connect
B SocketIO login
B SocketIO subscribe
B SocketIO soac offer
B SocketIO soac candidate
Portal SocketIO soac answer
SocketIO logout
2. OWT Server信令交互过程解析
- 以owt-client-javascript作为客户端，owt-server作为服务端为例。
- owt server安装完成后，可以在https://localhost:3004或者https://ip:3004看到通话界面。
- owt-client-javascript入口函数为src/samples/conference/public/scripts/index.js的window.onload（通话界面按F12可见），简化代码如下：
window.onload = function() {
var simulcast = getParameterByName('simulcast') || false;
var shareScreen = getParameterByName('screen') || false;
myRoom = getParameterByName('room');
var isHttps = (location.protocol === 'https:');
var mediaUrl = getParameterByName('url');
var isPublish = getParameterByName('publish');
createToken(myRoom, 'user', 'presenter', function(response) {
var token = response;
conference.join(token).then(resp => {
...
}, function(err) {
...
});
}, serverUrlBase);
};
- 可以看到，客户端首先会创建token，拿到token后再加入会议室。
- 信令分析上篇解析了获取token过程（OWT(Open WebRTC Toolkit) Server信令分析 (上)），接下来分析socketio信令交互过程。
1. Http POST创建token
2. SocketIO登录
- SocketIO连接成功后，客户端需要主动发送login事件，携带的数据格式为：
{
"token":"eyJ0b2tlbklkIjoiNjM2OWMyNGI1MTI2YWMzZTM0NjFiMzdjIiwiaG9zdCI6IjUyLjgxLjIwMS4xOTA6ODA4MCIsInNlY3VyZSI6dHJ1ZSwic2lnbmF0dXJlIjoiWW1JeU56SXdaakJsWkdJMU16VmlPV1ExT0RSa1l6Um1NREk0WTJWak1tTmlZbU14TnpZME1ETmtZVEV3TkRrek9ERXhZMlJoTkdReFpqRTJNR0UxTXc9PSJ9",
"userAgent":{
"sdk":{
"version":"5.0",
"type":"JavaScript"
}
},
"protocol":"1.2"
}
- token字段是创建token接口返回的base64编码字符串，其他的字段用来描述客户端的属性，以便服务端在需要时进行检查和区分处理。
- 登录成功后，SocketIO的ACK格式为：
{
"user":"user",
"role":"presenter",
"permission":{
"publish":{
"video":true,
"audio":true
},
"subscribe":{
"video":true,
"audio":true
}
},
"room":{
"id":"62b95bd27ff8054480cbb73a",
"views":[
"common"
],
"participants":[
{
"id":"IqCKvfpG402ooz_oAAAZ",
"user":"user",
"role":"presenter"
}
],
"streams":[
{
"id":"62b95bd27ff8054480cbb73a-common",
"type":"mixed",
"media":{
"tracks":[
{
"type":"audio",
"format":{
"codec":"opus",
"sampleRate":48000,
"channelNum":2
},
"optional":{
"format":[
{
"codec":"isac",
"sampleRate":16000
},
{
"codec":"isac",
"sampleRate":32000
},
{
"codec":"g722",
"sampleRate":16000,
"channelNum":1
},
{
"codec":"pcma"
},
{
"codec":"pcmu"
},
{
"codec":"aac",
"sampleRate":48000,
"channelNum":2
},
{
"codec":"ac3"
},
{
"codec":"nellymoser"
},
{
"codec":"ilbc"
}
]
},
"status":"active"
},
{
"type":"video",
"format":{
"codec":"vp8"
},
"parameters":{
"resolution":{
"width":640,
"height":480
},
"framerate":24,
"keyFrameInterval":100
},
"optional":{
"format":[
{
"codec":"h264",
"profile":"CB"
},
{
"codec":"h264",
"profile":"B"
},
{
"codec":"vp9"
}
],
"parameters":{
"resolution":[
{
"width":480,
"height":360
},
{
"width":426,
"height":320
},
{
"width":320,
"height":240
},
{
"width":212,
"height":160
},
{
"width":160,
"height":120
},
{
"width":352,
"height":288
}
],
"bitrate":[
"x0.8",
"x0.6",
"x0.4",
"x0.2"
],
"framerate":[
6,
12,
15
],
"keyFrameInterval":[
100,
30,
5,
2,
1
]
}
},
"status":"active"
}
]
},
"data":null,
"info":{
"label":"common",
"activeInput":"unknown",
"layout":[
],
"origin":{
"isp":"isp",
"region":"region"
}
}
}
]
},
"id":"IqCKvfpG402ooz_oAAAZ",
"reconnectionTicket":"eyJwYXJ0aWNpcGFudElkIjoiSXFDS3ZmcEc0MDJvb3pfb0FBQVoiLCJ0aWNrZXRJZCI6IjJmdmdqYm52NW82Iiwibm90QmVmb3JlIjoxNjY3ODc1NDAzNjkwLCJub3RBZnRlciI6MTY2Nzg3NjAwMzY5MCwic2lnbmF0dXJlIjoiTWpZeVlqbGpNMk5sWXpkalpERmxOV0k1TkdNNFlXVmpPREpsWW1FM09UQTNZemcxWW1Sak9EWmpaR1E0TnpCalpERXdZV0l4TW1ZMlpUTTFNV0l4TUE9PSJ9"
}
- ACK里room字段，包含了房间内的用户和流的信息，分别对应participants字段和streams字段。
- ACK里的user、role、id字段含义和participants的子字段含义一致，分别是用户id、角色、服务器分配的唯一标识。之后用户离开的消息只包含id字段，因此如果要用user字段标识用户，就需要维护id到user的映射关系。
- streams：
- type用来表明流的类型：
- mixed为MCU合成的流。
- forward为SFU转发的流。
- id为流的id，订阅时需要用到。
- media和info字段表明流的属性。
- type用来表明流的类型：
- reconnectionTicket字段用于重连时的重新登录。
1. 客户端代码
- SocketIO连接成功后，客户端需要主动发送login事件：
- 位置：src/sdk/conference/signaling.js
/**
* @function connect
* @instance
* @desc Connect to a portal.
* @memberof Oms.Conference.SioSignaling
* @return {Promise<Object, Error>} Return a promise resolved with the data returned by portal if successfully logged in. Or return a promise rejected with a newly created Oms.Error if failed.
* @param {string} host Host of the portal.
* @param {string} isSecured Is secure connection or not.
* @param {string} loginInfo Information required for logging in.
* @private.
*/
connect(host, isSecured, loginInfo) {
return new Promise((resolve, reject) => {
const opts = {
'reconnection': true,
'reconnectionAttempts': reconnectionAttempts,
'force new connection': true,
};
this._socket = io(host, opts);
...
//SocketIO连接成功后，客户端需要主动发送login事件
this._socket.emit('login', loginInfo, (status, data) => {
if (status === 'ok') {
this._loggedIn = true;
this._onReconnectionTicket(data.reconnectionTicket);
this._socket.on('connect', () => {
// re-login with reconnection ticket.
this._socket.emit('relogin', this._reconnectionTicket, (status,
data) => {
if (status === 'ok') {
this._reconnectTimes = 0;
this._onReconnectionTicket(data);
} else {
this.dispatchEvent(new EventModule.OwtEvent('disconnect'));
}
});
});
}
handleResponse(status, data, resolve, reject);
});
});
}
2. 服务端代码
-
位置：socket.on函数在source/portal/socketIOServer.js中。
-
SocketIO服务器的逻辑在socketIOServer.js和v11Client.js（除了v11Client，还有v10Client和legacyClient，用于老版本的兼容）中。
- v11Client监听了其他所有OWT Server信令协议的事件，比如publish、subscribe等
-
socketIOServer监听login、relogin、refreshReconnectionTicket、logout、disconnect和connection这6个事件。
- 前4个是OWT Server信令协议定义的。
- 最后两个由长连接客户端的断开和接入触发。
-
在处理login和relogin事件时，会根据protocol字段的取值来决定使用的Client版本（V10和V11）
socket.on('login', function(login_info, callback) {
...
client_id = socket.id + '';
var client;
if (login_info.protocol === undefined) {
protocol_version = 'legacy';
client = new LegacyClient(client_id, that, portal);
} else if (login_info.protocol === '1.0' ||
login_info.protocol === '1.1' ||
login_info.protocol === '1.2') {
//FIXME: Reject connection from 3.5 client
if (login_info.userAgent && login_info.userAgent.sdk &&
login_info.userAgent.sdk.version === '3.5') {
safeCall(callback, 'error', 'Deprecated client version');
return socket.disconnect();
}
protocol_version = login_info.protocol;
client = new Client(client_id, that, portal, protocol_version);
} else {
safeCall(callback, 'error', 'Unknown client protocol');
return socket.disconnect();
}
return validateUserAgent(login_info.userAgent)
.then((reconnEnabled) => {
reconnection.enabled = reconnEnabled;
return new Promise(function(resolve){
resolve(JSON.parse((Buffer.from(login_info.token, 'base64')).toString()));
});
}).then((token) => {
return client.join(token);
}).then((result) => {
...
});
});
- socketIOServer监听login事件，根据protocol字段创建对应客户端，然后会进行校验token以及完成其他操作，涉及代码较多，这篇主要为信令交互内容，后续再补上相关。
- 调用栈为：
source/portal/socketIOServer.js socket.on('login', function(login_info, ... =>
source/portal/v11Client.js that.join = (token) =>
source/portal/portal.js that.join = function(participantId, token) =>
source/portal/rpcRequest.js that.join = function(controller, roomId, participant) =>
source/agent/conference/conference.js that.join = function(roomId, participantInfo, callback) =>
source/agent/conference/conference.js initRoom = function(roomId, origin) => //保存房间信息，创建roomController、accessController
source/agent/conference/conference.js addParticipant = function(participantInfo, permission) => //保存用户信息
- 其中：
- accessController负责接入逻辑，比如WebRTC的信令处理等。
- roomController负责房间内各种状态的维护和功能的控制，比如流、订阅关系等，以及发布（publish）、订阅（subscribe）、混流（mix）、设置混流布局（setLayout）等。
- 也有很多处理工作是向其他组件的node进程发起RPC完成的。
3. SocketIO发布流
- 登录成功后，就可以发布音视频流了，发布时客户端发送publish事件，携带的数据格式为：
{
"media":{
"tracks":[
{
"type":"audio",
"mid":"2",
"source":"mic"
},
{
"type":"video",
"mid":"3",
"source":"camera"
}
]
},
"transport":{
"id":"cbeae5478b684b5e8f639fa84aafde9b",
"type":"webrtc"
}
}
- 用来指明音视频数据的格式。
- 发布成功的ACK格式为：
- 其中id为streamid
{
"id":"55aa9331e2b741338639d282f0bffa0d",
"transportId":"eceda0bafe3e4a1eab1a75056ede3cd8"
}
1. 客户端代码
- 部分代码如下：
async publish(stream, options, videoCodecs) {
...
return this._signaling.sendSignalingMessage('publish', {
media: {
tracks: trackOptions},
attributes: stream.attributes,
transport: {
id: this._id, type: 'webrtc'},
}).catch((e) => {
// Send SDP even when failed to get Answer.
this._signaling.sendSignalingMessage('soac', {
id: this._id,
signaling: localDesc,
});
throw e;
});
}).then((data) => {
...
this._signaling.sendSignalingMessage('soac', {
id: this._id,
signaling: localDesc,
});
}).catch((e) => {
...
}
- publish客户端代码调用栈为：
src/samples/conference/public/scripts/index.js conference.publish(localStream, ... =>
src/sdk/conference/client.js this.publish =>
src/sdk/conference/channel.js async publish =>
src/sdk/conference/channel.js _signaling.sendSignalingMessage('publish', =>
2. 服务端代码
- 服务端会接收publish信令，部分代码如下：
socket.on('publish', function(pubReq, callback) {
if(!that.inRoom){
return safeCall(callback, 'error', 'Illegal request');
}
//FIXME: move the id assignment to conference
var stream_id = uuidWithoutDash();
var transportId;
return adapter.translateReq(ReqType.Pub, pubReq)
.then((req) => {
if (req.transport && req.transport.type == 'quic') {
req.type = 'quic';
if (!req.transport.id) {
req.transport.id = uuidWithoutDash();
}
transportId = req.transport.id;
} else {
req.type = 'webrtc'; //FIXME: For backend compatibility with v3.4 clients.
if (!req.transport || !req.transport.id) {
req.transport = {
type : 'webrtc', id : stream_id };
}
}
transportId = req.transport.id;
return portal.publish(clientId, stream_id, req);
}).then((result) => {
safeCall(callback, 'ok', {
id: stream_id, transportId});
}).catch(onError('publish', callback));
});
- 调用栈：
source/portal/v11Client.js socket.on('publish', function(pubReq,
source/portal/portal.js that.publish = = function(participantId, streamId, pubInfo) =>
source/portal/rpcRequest.js that.publish = function(controller, participantId, streamId, Options) =>
source/agent/conference/conference.js that.publish = function(participantId, streamId, pubInfo, callback) =>
source/agent/conference/accessController.js that.initiate = function(participantId, sessionId, direction, origin, sessionOptions, ... =>
source/agent/conference/rpcRequest.js that.getWorkerNode = function(clusterManager, purpose, ... => //发起RPC，拿到WebRTC Agent的nodeId
source/agent/conference/rpcRequest.js that.initiate = function(accessNode, sessionId, ... => //向WebRTC Agent的node发起publish RPC
- WebRTC Agent的RPC服务定义在webrtc/index.js中。
source/agent/conference/rpcRequest.js that.initiate = function(accessNode, sessionId, ... => //向WebRTC Agent的node发起publish RPC
source/agent/webrtc/index.js that.publish = function (operationId, connectionType, ... =>
source/agent/webrtc/index.js createWebRTCConnection = function (transportId, ... => //创建WrtcConnection（`webrtc/wrtcConnection.js`），它负责和C++ 代码进行交互
- 在创建WrtcConnection时会执行webrtc/wrtcConnection.js的initWebRtcConnection函数，其中会监听C++代码发送的事件（由webrtc/connection.js做了一层翻译，变成status_event事件），收到事件后会回调给webrtc/index.js的notifyStatus函数，向Conference Agent的node发起onSessionProgress RPC。
/*
* Given a WebRtcConnection waits for the state CANDIDATES_GATHERED for set remote SDP.
*/
var initWebRtcConnection = function (wrtc) {
wrtc.on('status_event', (evt, status) => {
if (evt.type === 'answer') {
processAnswer(evt.sdp);
const message = localSdp.toString();
log.debug('Answer SDP', message);
on_status({
type: 'answer', sdp: message});
} else if (evt.type === 'candidate') {
let message = evt.candidate;
networkInterfaces.forEach((i) => {
if (i.ip_address && i.replaced_ip_address) {
message = message.replace(new RegExp(i.ip_address, 'g'), i.replaced_ip_address);
}
});
on_status({
type: 'candidate', candidate: message});
} else if (evt.type === 'failed') {
log.warn('ICE failed, ', status, wrtc.id);
on_status({
type: 'failed', reason: 'Ice procedure failed.'});
} else if (evt.type === 'ready') {
log.debug('Connection ready, ', wrtc.wrtcId);
on_status({
type: 'ready'
});
}
});
wrtc.init(wrtcId);
};
4. SocketIO发送Offer和ICE Candidate
- 对接OWT Server时，无论是发布还是订阅，都需要客户端创建Offer。拿到Offer后发送soac事件，携带的数据格式为：
- id是发布流成功ACK里返回的streamid。
- signaling的"type":"offer"表示发送的是Offer消息。
- 剩下sdp字段为Offer内容。
{
"id":"eceda0bafe3e4a1eab1a75056ede3cd8",
"signaling":{
"type":"offer",
"sdp":"v=0
o=- 7416128173695703170 3 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1 2 3
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 60916 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 10.110.133.203
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:2722325311 1 udp 2122265343 fc00:2::e19b:dc32:443d:dd84 55478 typ host generation 0 network-id 2
a=candidate:3962993534 1 udp 2122194687 10.110.133.203 60916 typ host generation 0 network-id 1
a=ice-ufrag:gdMS
a=ice-pwd:1yWra8IAyw4YqqXE6JLbrfDI
a=ice-options:trickle
a=fingerprint:sha-256 07:95:5D:7A:A8:FD:53:D3:B9:8B:CC:8C:29:54:08:89:83:63:5E:CE:A6:DE:01:EF:5F:0A:06:96:ED:E1:8A:9F
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 102 122 127 121 125 107 108 109 124 120 39 40 41 42 43 44 45 46 47 48 123 119 114 115 116 49
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:gdMS
a=ice-pwd:1yWra8IAyw4YqqXE6JLbrfDI
a=ice-options:trickle
a=fingerprint:sha-256 07:95:5D:7A:A8:FD:53:D3:B9:8B:CC:8C:29:54:08:89:83:63:5E:CE:A6:DE:01:EF:5F:0A:06:96:ED:E1:8A:9F
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:35 VP9/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 profile-id=1
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:37 VP9/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=rtcp-fb:37 ccm fir
a=rtcp-fb:37 nack
a=rtcp-fb:37 nack pli
a=fmtp:37 profile-id=3
a=rtpmap:38 rtx/90000
a=fmtp:38 apt=37
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 H264/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=124
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:41 H264/90000
a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:43 H264/90000
a=rtcp-fb:43 goog-remb
a=rtcp-fb:43 transport-cc
a=rtcp-fb:43 ccm fir
a=rtcp-fb:43 nack
a=rtcp-fb:43 nack pli
a=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f
a=rtpmap:44 rtx/90000
a=fmtp:44 apt=43
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:47 AV1/90000
a=rtcp-fb:47 goog-remb
a=rtcp-fb:47 transport-cc
a=rtcp-fb:47 ccm fir
a=rtcp-fb:47 nack
a=rtcp-fb:47 nack pli
a=fmtp:47 profile=1
a=rtpmap:48 rtx/90000
a=fmtp:48 apt=47
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=123
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=rtpmap:49 flexfec-03/90000
a=rtcp-fb:49 goog-remb
a=rtcp-fb:49 transport-cc
a=fmtp:49 repair-window=10000000
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:gdMS
a=ice-pwd:1yWra8IAyw4YqqXE6JLbrfDI
a=ice-options:trickle
a=fingerprint:sha-256 07:95:5D:7A:A8:FD:53:D3:B9:8B:CC:8C:29:54:08:89:83:63:5E:CE:A6:DE:01:EF:5F:0A:06:96:ED:E1:8A:9F
a=setup:actpass
a=mid:2
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=sendonly
a=msid:fbiSmx6n3xm4yA1FQtKj8zH9yGBv7vmGoVai b2bbec5a-1452-4840-9171-30a10d300741
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3856090643 cname:L8DTZMOJip3IXAHs
a=ssrc:3856090643 msid:fbiSmx6n3xm4yA1FQtKj8zH9yGBv7vmGoVai b2bbec5a-1452-4840-9171-30a10d300741
m=video 9 UDP/TLS/RTP/SAVPF 96 97 102 122 127 121 125 107 108 109 124 120 39 40 45 46 98 99 100 101 123 119 114 115 116
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:gdMS
a=ice-pwd:1yWra8IAyw4YqqXE6JLbrfDI
a=ice-options:trickle
a=fingerprint:sha-256 07:95:5D:7A:A8:FD:53:D3:B9:8B:CC:8C:29:54:08:89:83:63:5E:CE:A6:DE:01:EF:5F:0A:06:96:ED:E1:8A:9F
a=setup:actpass
a=mid:3
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:fbiSmx6n3xm4yA1FQtKj8zH9yGBv7vmGoVai 1327213e-5d74-461f-be50-ed78ede6a453
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 H264/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=124
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=123
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=ssrc-group:FID 3686128283 1675289484
a=ssrc:3686128283 cname:L8DTZMOJip3IXAHs
a=ssrc:3686128283 msid:fbiSmx6n3xm4yA1FQtKj8zH9yGBv7vmGoVai 1327213e-5d74-461f-be50-ed78ede6a453
a=ssrc:1675289484 cname:L8DTZMOJip3IXAHs
a=ssrc:1675289484 msid:fbiSmx6n3xm4yA1FQtKj8zH9yGBv7vmGoVai 1327213e-5d74-461f-be50-ed78ede6a453
"
}
}
- 拿到ICE Candidate后也发送soac事件，携带的数据格式为：
- id是发布流成功ACK里返回的id。
- signaling的"type":"candidate"表示发送的是ICE Candidate消息。
- sdpMLineIndex、sdpMid和candidate字段分别是IceCandidate类的sdpMLineIndex、sdpMid和sdp成员（在前面拼接a=）。
{
"id":"eceda0bafe3e4a1eab1a75056ede3cd8",
"signaling":{
"type":"candidate",
"candidate":{
"candidate":"a=candidate:2722325311 1 udp 2122265343 fc00:2::e19b:dc32:443d:dd84 55478 typ host generation 0 ufrag gdMS network-id 2",
"sdpMid":"0",
"sdpMLineIndex":0
}
}
}
1. 客户端代码
- 部分代码如下：
async publish(stream, options, videoCodecs) {
...
return this._signaling.sendSignalingMessage('publish', {
media: {
tracks: trackOptions},
attributes: stream.attributes,
transport: {
id: this._id, type: 'webrtc'},
}).catch((e) => {
// Send SDP even when failed to get Answer.
this._signaling.sendSignalingMessage('soac', {
id: this._id,
signaling: localDesc,
});
throw e;
});
}).then((data) => {
...
this._signaling.sendSignalingMessage('soac', {
id: this._id,
signaling: localDesc,
});
}).catch((e) => {
...
}
- 客户端调用栈：
src/samples/conference/public/scripts/index.js conference.publish(localStream, ... =>
src/sdk/conference/client.js this.publish =>
src/sdk/conference/channel.js async publish =>
src/sdk/conference/channel.js _signaling.sendSignalingMessage('soac', =>
2. 服务端代码
- 服务端会接收soac信令，接收代码：
socket.on('soac', function(SOAC, callback) {
if(!that.inRoom){
return safeCall(callback, 'error', 'Illegal request');
}
return validateSOAC(SOAC)
.then((soac) => {
return portal.onSessionSignaling(clientId, soac.id, soac.signaling);
}).then((result) => {
safeCall(callback, 'ok');
}).catch(onError('soac', callback));
});
};
- 相关调用栈：
source/portal/v11Client.js socket.on('soac', function(SOAC, callback) =>
source/portal/portal.js that.onSessionSignaling = function(participantId, ... =>
source/portal/rpcRequest.js that.onSessionSignaling = function(controller, sessionId, ... =>
source/agent/conference/conference.js that.onSessionSignaling = function(sessionId, signaling, ... =>
source/agent/conference/rtcController.js onClientTransportSignaling(transportId, signaling) =>
source/agent/conference/rpcRequest.js that.onTransportSignaling = function(accessNode, ... =>
source/agent/webrtc/index.js that.onTransportSignaling = function (connectionId, msg, callback) =>
source/agent/webrtc/wrtcConnection.js that.onSignalling = function (msg, operationId) =>
- source/agent/webrtc/wrtcConnection.js的that.onSignalling函数中，会根据msg的type是offer/candidate/removed-candidates进行不同操作。
- 如果type是offer，会处理Offer SDP，最后发送answer类型的on_status消息，在initWebRtcConnection函数中接收。
that.onSignalling = function (msg, operationId) {
var processSignalling = function () {
if (msg.type === 'offer') {
log.debug('on offer:', msg.sdp);
processOffer(msg.sdp);
} else if (msg.type === 'candidate') {
wrtc.addRemoteCandidate(msg.candidate);
} else if (msg.type === 'removed-candidates') {
wrtc.removeRemoteCandidates(msg.candidates);
}
};
if (wrtc) {
processSignalling();
} else {
// should not reach here
log.error('wrtc is not ready');
}
};
5. SocketIO接收Answer
- OWT Server收到客户端发送的Offer后，会通过progress事件返回Answer，携带的数据格式为：
- id是发布流成功ACK里返回的id。
- “status”:"soac"表明是返回的SDP消息，还有其他类型的消息，比如流发布或订阅成功的ready消息、错误error消息等。
- data的"type":"answer"表示发送的是Answer消息。
- sdp字段为Answer内容，内容较长，这里省略了。
[
{
"id":"62b95bd27ff8054480cbb73a",
"status":"soac",
"data":{
"type":"answer",
"sdp":"v=0\ro=- 7416128173695703170 3 IN IP4 127.0.0.1s=\r-t=0 0...省略"
}
}
]
6. SocketIO订阅流
- 客户端B创建token、登录的过程和客户端A一样，但是B登录ACK返回的room字段的participants子字段会包含A、B两个用户的信息，streams子字段除了MCU混合的流，还会有SFU转发A的流。拿到A用户的这两个信息之后，我们就可以订阅A的流了。
- 订阅流需要客户端发送subscribe事件，携带的数据格式为：
- audio和video的from都是想要订阅的流的id，即登录ACK返回的room.streams的id字段。
{
"media":{
"tracks":[
{
"type":"audio",
"mid":"0",
"from":"62b95bd27ff8054480cbb73a-common"
},
{
"type":"video",
"mid":"1",
"from":"62b95bd27ff8054480cbb73a-common"
}
]
},
"transport":{
"type":"webrtc"
}
}
- 订阅成功的ACK的格式为：
{
"id":"55aa9331e2b741338639d282f0bffa0d"
}
- id是订阅（subscription）的id。
- 之后发送Offer和ICE Candidate，以及接收Answer，格式和发布流时的格式都一样，但使用的id值都是这里订阅ACK的id。
- 客户端B如果要发布流，那么它的信令流程和A的流程完全一致。发布成功后，A会收到消息通知，这样A也就可以订阅B发布的流。A订阅B的过程和上面描述的B订阅A的过程完全一致。
1. 客户端代码
- 部分代码如下：
window.onload = function() {
var simulcast = getParameterByName('simulcast') || false;
var shareScreen = getParameterByName('screen') || false;
myRoom = getParameterByName('room');
var isHttps = (location.protocol === 'https:');
var mediaUrl = getParameterByName('url');
var isPublish = getParameterByName('publish');
createToken(myRoom, 'user', 'presenter', function(response) {
var token = response;
conference.join(token).then(resp => {
...
var streams = resp.remoteStreams;
for (const stream of streams) {
if(!subscribeForward){
if (stream.source.audio === 'mixed' || stream.source.video ===
'mixed') {
subscribeAndRenderVideo(stream);
}
} else if (stream.source.audio !== 'mixed') {
subscribeAndRenderVideo(stream);
}
}
console.log('Streams in conference:', streams.length);
var participants = resp.participants;
console.log('Participants in conference: ' + participants.length);
});
}, serverUrlBase);
};
- publish客户端代码调用栈为：
src/samples/conference/public/scripts/index.js subscribeAndRenderVideo(stream) =>
src/samples/conference/public/scripts/index.js conference.subscribe(stream) =>
src/sdk/conference/client.js this.subscribe =>
src/sdk/conference/channel.js async subscribe(stream, options)
src/sdk/conference/channel.js _signaling.sendSignalingMessage('subscribe', =>
2. 服务端代码
- 服务端接收消息代码如下：
socket.on('subscribe', function(subReq, callback) {
if(!that.inRoom){
return safeCall(callback, 'error', 'Illegal request');
}
var subscription_id = Math.round(Math.random() * 1000000000000000000) + '';
return validateSubReq(subReq)
.then((req) => {
req.type = 'webrtc';//FIXME: For backend compatibility with v3.4 clients.
return portal.subscribe(clientId, subscription_id, req);
}).then((result) => {
safeCall(callback, 'ok', {
id: subscription_id});
}).catch(onError('subscribe', callback));
});
- 调用栈：
source/portal/v11Client.js socket.on('subscribe', function(subReq, =>
source/portal/portal.js that.subscribe = function(participantId, ... =>
source/portal/rpcRequest.js that.publish = function(controller, participantId, streamId, Options) =>
source/agent/conference/conference.js that.subscribe = function(controller, participantId, ... =>
source/agent/conference/rtcController.js initiate(ownerId, sessionId, direction, origin, ... =>
source/agent/conference/rpcRequest.js that.initiate = function(accessNode, sessionId, ... => //向WebRTC Agent的node发起publish RPC
- WebRTC Agent的RPC服务定义在webrtc/index.js中。
source/agent/conference/rpcRequest.js that.initiate = function(accessNode, sessionId, ... => //向WebRTC Agent的node发起publish RPC
source/agent/webrtc/index.js that.subscribe = function (operationId, connectionType, ... =>
source/agent/webrtc/index.js createWebRTCConnection = function (transportId, ... => //创建WrtcConnection（`webrtc/wrtcConnection.js`），它负责和C++ 代码进行交互
- 在创建WrtcConnection时会执行webrtc/wrtcConnection.js的initWebRtcConnection函数，其中会监听C++代码发送的事件（由webrtc/connection.js做了一层翻译，变成status_event事件），收到事件后会回调给webrtc/index.js的notifyStatus函数，向Conference Agent的node发起onSessionProgress RPC。
/*
* Given a WebRtcConnection waits for the state CANDIDATES_GATHERED for set remote SDP.
*/
var initWebRtcConnection = function (wrtc) {
wrtc.on('status_event', (evt, status) => {
if (evt.type === 'answer') {
processAnswer(evt.sdp);
const message = localSdp.toString();
log.debug('Answer SDP', message);
on_status({
type: 'answer', sdp: message});
} else if (evt.type === 'candidate') {
let message = evt.candidate;
networkInterfaces.forEach((i) => {
if (i.ip_address && i.replaced_ip_address) {
message = message.replace(new RegExp(i.ip_address, 'g'), i.replaced_ip_address);
}
});
on_status({
type: 'candidate', candidate: message});
} else if (evt.type === 'failed') {
log.warn('ICE failed, ', status, wrtc.id);
on_status({
type: 'failed', reason: 'Ice procedure failed.'});
} else if (evt.type === 'ready') {
log.debug('Connection ready, ', wrtc.wrtcId);
on_status({
type: 'ready'
});
}
});
wrtc.init(wrtcId);
};
7. SocketIO接收其他用户的通知
1. 其他用户登录通知
- 其他用户登录后，服务端会通过participant事件发布通知，携带的数据格式为：
{
"action":"join",
"data":{
"id":"1d6pEzr67tkUhEJeAAAn",
"user":"user",
"role":"presenter"
}
}
- 调用栈：
source/portal/socketIOServer.js socket.on('login', function(login_info, ... =>
source/portal/v11Client.js that.join = (token) =>
source/portal/portal.js that.join = function(participantId, token) =>
source/portal/rpcRequest.js that.join = function(controller, roomId, participant) =>
source/agent/conference/conference.js that.join = function(roomId, participantInfo, callback) =>
source/agent/conference/conference.js initRoom = function(roomId, origin) => //保存房间信息，创建roomController、accessController
source/agent/conference/conference.js addParticipant = function(participantInfo, permission) => //保存用户信息
- addParticipant函数在保存用户信息后会向其他参与者发布通知。
2. 其他用户发布stream通知
- 其他用户发布流后，服务端会通过stream事件发布通知，携带的数据格式为：
{
"id":"55aa9331e2b741338639d282f0bffa0d",
"status":"add",
"data":{
"id":"55aa9331e2b741338639d282f0bffa0d",
"type":"forward",
"media":{
},
"info":{
"owner":"kYmJHa8XsioNXhKUAAAJ",
"type":"webrtc",
"inViews":[
]
}
}
}
- “status”："add"表明是其他用户新发布了流。data字段的各个子字段、含义和登录ACK的streams的各个子字段含义一致。
3. 用户退出登录通知
- 其他用户退出登录后，服务端也会通过participant事件发布通知，携带的数据格式为：
{
"action":"leave",
"data":"I-Fv7fQ8Qe9_9qa2AAAG"
}
- “action”："leave"表明是其他用户退出登录了。
- data字段为登录ACK的participants的id子字段。
- 如果是用user字段标识用户，就需要维护id到user的映射关系。
8. SocketIO退出登录
- 如果需要停止发布和订阅所有的流，就可以发送logout事件退出登录，携带的数据为空字符串。退出登录后，其他客户端将会收到通知。通知携带的数据为：
{
"action":"leave",
"data":"kYmJHa8XsioNXhKUAAAJ"
}
1. 客户端代码
- 客户端部分代码：
/**
* @function leave
* @memberOf Owt.Conference.ConferenceClient
* @instance
* @desc Leave a conference.
* @return {Promise<void, Error>} Returned promise will be resolved with undefined once the connection is disconnected.
*/
this.leave = function() {
return signaling.disconnect().then(() => {
clean();
signalingState = SignalingState.READY;
});
};
- 当离开页面时会触发，然后调用leave函数，最终会向owt server发送logout事件。
/**
* @function disconnect
* @instance
* @desc Disconnect from a portal.
* @memberof Oms.Conference.SioSignaling
* @return {Promise<Object, Error>} Return a promise resolved with the data returned by portal if successfully disconnected. Or return a promise rejected with a newly created Oms.Error if failed.
* @private.
*/
disconnect() {
if (!this._socket || this._socket.disconnected) {
return Promise.reject(new ConferenceError(
'Portal is not connected.'));
}
return new Promise((resolve, reject) => {
this._socket.emit('logout', (status, data) => {
// Maximize the reconnect times to disable reconnection.
this._reconnectTimes = reconnectionAttempts;
this._socket.disconnect();
handleResponse(status, data, resolve, reject);
});
});
}
- 调用栈：
src/samples/conference/public/scripts/index.js window.onbeforeunload = function(... =>
src/sdk/conference/client.js this.leave = function() =>
src/sdk/conference/signaling.js disconnect()
2. 服务端代码
- owt server会接收logout事件，代码如下：
socket.on('logout', function(callback){
reconnection.enabled = false;
state = 'initialized';
if (client_id) {
forceClientLeave();
safeCall(callback, okWord());
} else {
return safeCall(callback, 'error', 'Illegal request');
}
});
- 调用栈为：
source/portal/socketIOServer.js socket.on('logout', function(callback) =>
source/portal/socketIOServer.js forceClientLeave = () =>
source/portal/v11Client.js that.leave = () =>
source/portal/portal.js that.leave = function(participantId) =>
source/portal/rpcRequest.js that.leave = function(controller, participantId) =>
source/agent/conference/conference.js that.leave = function(participantId, callback) =>
9. Offer和ICE Candidate顺序
- OWT Server服务端也没有处理Offer和ICE Candidate的先后顺序，所以需要客户端确保先发送Offer再发送ICE Candidate，否则先收到的ICE Candidate会被丢弃，进而可能导致连接失败。
- 当网络连接发生错误时，可能SocketIO会断开连接，或者PC会断开连接，比较简单的做法是在断开连接时自动关闭所有PC，重新建立SocketIO连接，连接成功后发送relogin事件并带上reconnectionTicket的内容。relogin的ACK和login的ACK格式相同，这样之后的流程就可以按照login处理了。
- reconnectionTicket只有一定的有效期，我们需要在过期之前进行刷新。发送refreshReconnectionTicket事件，携带的数据为空字符串，返回的ACK格式为：
[
"ok",
"<新的reconnectionTicket>"
]
- 重连逻辑可以进一步优化，比如没有发生错误的连接就无须断开重连了。
HTTP GET用户和流列表
- 除了通过SocketIO接收其他用户加入或离开的事件，还可以通过HTTP接口主动获取完整的用户和流列表，获取用户列表的接口为GET /rooms//participants，响应的格式为：
[
{
"id": "XHDK7_8QXJjazcwuAAAx",
"user": "user",
"role": "presenter",
"permission": {
"publish": {
"video": true,
"audio": true
},
"subscribe": {
"video": true,
"audio": true
}
}
},
{
"id": "nOqTKNJrCgHEUAlaAAAz",
"user": "user",
"role": "presenter",
"permission": {
"publish": {
"video": true,
"audio": true
},
"subscribe": {
"video": true,
"audio": true
}
}
}
]
- 获取流列表的接口为GET /rooms//streams，响应的格式为：
[
{
"id":"62b95bd27ff8054480cbb73a-common",
"type":"mixed",
"media":{
"audio":{
"status":"active",
"format":{
"codec":"opus",
"sampleRate":48000,
"channelNum":2
},
"optional":{
"format":[
{
"codec":"isac",
"sampleRate":16000
},
{
"codec":"isac",
"sampleRate":32000
},
{
"codec":"g722",
"sampleRate":16000,
"channelNum":1
},
{
"codec":"pcma"
},
{
"codec":"pcmu"
},
{
"codec":"aac",
"sampleRate":48000,
"channelNum":2
},
{
"codec":"ac3"
},
{
"codec":"nellymoser"
},
{
"codec":"ilbc"
}
]
}
},
"video":{
"status":"active",
"optional":{
"format":[
{
"codec":"h264",
"profile":"CB"
},
{
"codec":"h264",
"profile":"B"
},
{
"codec":"vp9"
}
],
"parameters":{
"resolution":[
{
"width":480,
"height":360
},
{
"width":426,
"height":320
},
{
"width":320,
"height":240
},
{
"width":212,
"height":160
},
{
"width":160,
"height":120
},
{
"width":352,
"height":288
}
],
"bitrate":[
"x0.8",
"x0.6",
"x0.4",
"x0.2"
],
"framerate":[
6,
12,
15
],
"keyFrameInterval":[
100,
30,
5,
2,
1
]
}
},
"format":{
"codec":"vp8"
},
"parameters":{
"resolution":{
"width":640,
"height":480
},
"framerate":24,
"keyFrameInterval":100
}
}
},
"data":null,
"info":{
"label":"common",
"activeInput":"961f75a4fac8435f9a54506d25150784",
"layout":[
{
"stream":"961f75a4fac8435f9a54506d25150784",
"region":{
"id":"1",
"shape":"rectangle",
"area":{
"left":"0/1",
"top":"0/1",
"width":"1/1",
"height":"1/1"
}
}
}
],
"origin":{
"isp":"isp",
"region":"region"
}
}
},
{
"id":"961f75a4fac8435f9a54506d25150784",
"type":"forward",
"media":{
"audio":{
"status":"active",
"source":"mic",
"format":{
"codec":"opus",
"sampleRate":48000,
"channelNum":2
},
"optional":{
"format":[
{
"codec":"isac",
"sampleRate":16000
},
{
"codec":"isac",
"sampleRate":32000
},
{
"codec":"g722",
"sampleRate":16000,
"channelNum":1
},
{
"codec":"pcma"
},
{
"codec":"pcmu"
},
{
"codec":"aac",
"sampleRate":48000,
"channelNum":2
},
{
"codec":"ac3"
},
{
"codec":"nellymoser"
},
{
"codec":"ilbc"
}
]
}
},
"video":{
"status":"active",
"source":"camera",
"optional":{
"format":[
{
"codec":"h264",
"profile":"CB"
},
{
"codec":"h264",
"profile":"B"
},
{
"codec":"vp9"
}
],
"parameters":{
"resolution":[
{
"width":480,
"height":360
},
{
"width":426,
"height":320
},
{
"width":320,
"height":240
},
{
"width":212,
"height":160
},
{
"width":160,
"height":120
},
{
"width":352,
"height":288
}
],
"bitrate":[
"x0.8",
"x0.6",
"x0.4",
"x0.2"
],
"framerate":[
6,
12,
15,
24
],
"keyFrameInterval":[
100,
30,
5,
2,
1
]
}
},
"format":{
"codec":"vp8"
},
"parameters":{
"resolution":{
"width":640,
"height":480
}
}
}
},
"info":{
"owner":"XHDK7_8QXJjazcwuAAAx",
"type":"webrtc",
"inViews":[
"common"
]
}
}
]
- 其他REST API接口请看README：OWT REST API