With the development of the mobile Internet, there are more and more video communication usage scenarios, such as video chat, video conference, online live broadcast, etc. However, the requirements for front-end design and the requirements for back-end servers are also increasing. Therefore, how to build a perfect server for video communication is the concern of every video communication R&D personnel.
A complete audio and video server needs to solve the problems of high concurrency, low latency, NAT penetration and scalable load balancing. We have been paying attention to this direction for more than a year. The following are some of our experiences:
About high concurrency
This paper mainly introduces how to design the Internet distribution architecture to improve the concurrency capability of the system.
There are two commonly used methods: vertical expansion (Scale Up) and horizontal expansion (Scale Out).
1. Use vertical expansion to improve the processing capacity of a single machine.
When writing code, the optimizations we can do fall into three categories: architectural optimizations, algorithmic optimizations, and language optimizations.
1 Architecture optimization: such as using asynchronous IO to increase the throughput of a single server, and improving server performance by reducing the use of locks when multi-threaded.
2. Algorithm optimization: It is relatively rare in the server, because there is usually no cumbersome algorithm in the server logic code, but if there is room for optimization, it should be optimized.
3 Language optimization: The more common optimization methods, such as const and reference passing parameters, such as pre-add and post-add when complex object traversal, etc., see "effective c++" and "more effective c++" for details. Of course, if there is a database, then the optimization of the SQL statement also counts.
Now that the code has been written and it is running, I find that the performance is still unsatisfactory. What should I do?
Look at the bottleneck! Just like a doctor treating a disease, you must first find the cause. Integrated performance analysis can be performed with gperftools under Linux, vs under Windows, and xcode under Mac.
These profiling tools can help you roughly locate which line of code is taking up CPU time. After finding the cause, it is the right medicine.
Of course, for this kind of server performance that is mainly caused by CPU usage, insufficient memory and network card, it is very effective to upgrade the hardware.
However , the performance of a single machine is always limited, and it will be limited by the technology of the times. Therefore, the design of high concurrency solutions for the distributed architecture of the Internet still relies on horizontal expansion.
2. Use horizontal expansion to increase the number of servers to expand system performance
This is related to the layered architecture of the Internet. In the layered organization of the Internet, the practice of horizontal expansion at each level is different. For example, the reverse proxy layer uses the "DNS round-robin" method; the site layer uses nginx, The server mainly relies on the service connection pool. After horizontal expansion is implemented at each layer, by increasing the number of servers, the theoretical system performance can be improved infinitely.
Simply put, when there is too much work, one person cannot finish it, and multiple people work together. Then involving multiple people, it will inevitably involve the issue of scheduling allocation management.
Related terms are: CDN, load balancing, Hadoop, cloud computing, etc.
After solving the problem of high concurrency, let's talk about low latency.
About low latency
Low latency is a point that all video communication R&D personnel will pay attention to, and lower latency will definitely improve user experience. But how to achieve low latency?
Tuya's product adopts the following method: the server uses udp protocol to transmit audio and video data , and tcp protocol to transmit control signaling, so as to ensure reliable control signaling and low audio and video data transmission delay.
As we all know, the UDP protocol is compared with the TCP protocol: using the UDP protocol to transmit data may lead to data loss, but the client receives information with a low delay; while the TCP protocol has a packet loss retransmission policy, but the speed is not fast. When our product is in use, we combine the two to ensure that UDP can also unpack normally when the received packet is incomplete.
After solving the problems of high concurrency and low latency, we have to consider the problem of traffic cost. Reducing the traffic cost is the demand of every video communication user, and it is also the problem that every programmer who studies video communication needs to solve. Here I briefly introduce NAT traversal.
About NAT Traversal
(NAT penetration)
A major technical difficulty of video chat is that the network bandwidth of the server is too high. Using the NAT traversal scheme, the client can use the P2P method when communicating. The P2P solution enables the video data of client A to reach client B without passing through the server, ensuring low transmission delay while reducing server bandwidth consumption.
So how to achieve NAT penetration? We first need to know the characteristics of NAT: NAT will reject packets from unfamiliar sources. Simply put, if there is no host behind NAT that has sent data to a host outside the NAT, then the external host cannot actively send data packets to the host behind the NAT.
A feasible solution is to use a signaling server to first obtain the IP:PORT information exposed by the client on the NAT, and then coordinate two clients to send information to the IP:PORT exposed on the NAT. Since the machine behind NAT (set as NATA here) sends information to the device behind another NAT (set as NATB here), NATA will allow the data of the host behind NATB. vice versa.
But there is another point to note: the first packet of the client that sends the data first will be discarded by the NAT , because the other NAT does not know how to distribute this unknown source packet. The complete solution of NAT traversal can refer to the RFC5389 protocol and implement it according to the protocol.
About strong scalability and load balancing
(a load balancing method)
After dealing with the above problems, you can consider how to balance the load of the entire server.
Here, Tuyajun mainly introduces the design of the master-slave node cluster : the client obtains the forwarding server address from the load balancing server, and the load balancing server intelligently assigns the client the appropriate The server address, while ensuring the efficient use of resources, avoids server overload.
Such a design can seamlessly join the master node from the slave node while improving scalability.
Summarize
In fact, the above mentioned are the basis for building high-performance servers . A good architecture is the foundation, and only good optimization can build tall buildings . For how to better optimize the server? A reasonable approach is to analyze the running status of the server, and perform special optimization on the parts that are executed frequently and consume too much resources.
That is to say, for the specific analysis of the specific server, only in this way can the server be optimized.