background
Recently the company needs to add functionality to support audio and video communications in IM from research projects, not only to achieve the basic version of the two micro-letter functions as call, but also support multiplayer audio and video conference calls as well. See here, we might think,
increase the audio and video capabilities is not a very simple thing it? ? Direct access from the cloud communications platform sdk not get away with it? There is no difficulty in it.
In fact, general business applications like this to achieve a big problem, but the company has always been a relatively rigid requirements, all things are self-study requirement, which means that you can use open source code, but can not use sdk (as for what, there is not much to say ...). Including the current self-development of IM projects, all developed based on research from the engine and open source, no other use of over-the sdk.
Technology Selection
Since I can not use sdk, you can only bite the bullet and realize each function of their own. Few teams have done before taking into account the relevant audio and video projects, lack of combat experience, this is a project from a 0-1, and the team, it is a very big challenge, technically speaking, is a pair high technical requirements, the underlying technical system partial program. Therefore, to realize a direct pass their own audio and video systems, we chose to embrace open source, select the secondary development on the basis of open source projects. So, how do we choose the right team of open source projects do? Considering that the project is necessary to support multiplayer audio and video capabilities, so we conducted a series of research in the market, and ultimately come to a few options:
▸ the WebRTC
▸ h323plush
▸ live555
▸ OpenMeetings
1 WebRTC
Google open source real-time communications based on open-source browser project, known as the world's most advanced real-time audio and video communications technologies, which the network transmission module, acquisition module, audio and video encoding and other packaged done very leading projects including the android / ios realization / web and other mainstream side, its biggest feature is capable of audio and video communications directly in the browser. Cons: But this project does not support multiplayer audio and video conferencing.
2 h323plus
It can be considered the most mature older set of open source audio and video system, including all of the H.323 protocol functions, including audio and video conferencing capabilities, it has a reference example of an implementation of a large number of video conferencing, such as MCU servers. The project application in conventional LAN audio and video conferencing, a typical application such as a local government conference on school applications, it needs to be used with supporting hardware together.
3 live555
c ++ famous streaming media server, including not only the transport protocol (SIP, RTP), audio and video encoders (H.264, MPEG4), etc., also includes examples of streaming media server, streaming media of choice for the project, which the transmission module video conferencing is well worth developing as a reference, but it only supports servers, clients have to go to achieve
4 OpenMeetings
OpenMeetings mainly based on flash client open source projects, the core technology is based on another open source Red5, including audio and video conferencing, whiteboard and other development language is java
项目名称 | h323plus | webrtc | live555 | OpenMeetings
------- | --------------- | --------- | -------- | --------------
使用场景 | 基于局域网高清联机会议| 互联网行业 | 只用作服务器 | 互联网行业
缺点 | 对手机端支持不够 | 暂无 |没有实现客户端 | 只支持flash客户端
开源语言 | c | c++/node.js/go | c++ | java
Seen from the above comparison, we compared the webrtc priority and h322plus two open source projects, although h323plus LAN in the current mainstream high-definition video conferencing with very much more mature, but the field of view of our application is the Internet, the need to provide 5-terminal (android / ios / h5 / pc / mac) client functionality, so we decided to use audio and video webrtc project development.
As mentioned earlier, the advantage is that the sound webrtc project between single-point (p2p) video communications, people do not support audio and video communications, but the project requires more than is necessary to achieve audio and video functions required to achieve multiperson video conference must be a need to develop server-side program can support (this team before eating a lot of losses, we spent a lot of time on how to support multiplayer audio and video in the study webrtc, but in the end that is not supported by research)
in order to achieve more than people call function, to understand the three possible modes, namely: Mesh mode, MCU mode, SFU mode, mesh model is mainly dependent on client implementation, the latter two models are dependent on server implementation, their advantages and disadvantages are different .
mesh pattern
Also known as mesh mode, we have established a connection between each client, on the one hand their audio and video data equally distributed to other clients, on the other hand also accept data from other clients. The disadvantage is that after more than a certain number (5) client participation, the client becomes very difficult.
mcu mode
After fusion, also known as MCU model server, the server for each client and mixed media stream combined into a fusion coding compression stream (Audio Visual distinction) forwarded to each client. Relatively mesh mode, one client only needs to send and receive data stream all the way to greatly reduce the pressure on the client, of course, at the cost of a corresponding increase in server costs.
sfu mode
SFU (Selective Forwarding Unit) called selective forwarding service, the server selectively forwarding the data stream to the client as needed, compared to mcu mode client receives a single stream, the client receives from multiple servers forwarded over the stream, the business layer more easily extensible.
Comparison of several models
General conventional Internet applications, design principles are taken to re-light the client service side, mainly to expand the system better performance and functionality, support more complex scenarios; therefore generally behind the choice of two modes to achieve more than audio and video capabilities.
in conclusion
Combined with their own team, the final choice to implement audio and video capabilities based on webrtc technology system. P2p achieve audio and video features by webrtc. In order to support multiplayer audio and video, independent research and development team needs webrtc server products project requires multiple screen display and even dynamic mic operation, so select the server mcu / sfu mixed mode as a support multiplayer audio and video capabilities. More support mcu / sfu mode of open source projects, the content will be more, it will be introduced in the next article. .
Think
Suppose you make to achieve a real-time audio and video systems, you are the technical research and technology selection by what means it? Is Baidu google or other more efficient way through it? ? Welcome Message exchange