On a Novel High Concurrency Technology
Li Wanhong 2020-7-12
High concurrency is a key problem of Internet applications. When there are hundreds of millions of concurrent, the traditional java system is no longer able to do it. Therefore, new ideas need to be innovatively used to solve this problem. Here is a novel based on Go, Kafka, Redis, and RPC. The high concurrency technology architecture supports hundreds of millions of concurrent numbers, which can effectively solve this problem.
Making full use of the CPU's multi-core, message mechanism and efficient RPC can significantly improve the concurrency capability. According to this principle, this solution is designed.
The Go language uses coroutine technology to provide high concurrent processing capabilities. A CPU supports 10,000 coroutines running concurrently. You can use Go to develop web and provide users with access to it. Kafka is a high-performance messaging middleware with excellent performance. It is widely used in big data scenarios. You can send user request data to Kafka in Go applications, and send the returned offset to Redis, and send it with the message Lpush statement The offset is to the message queue, the background processing program is developed, and the offset is taken out from the message queue of Redis using Rpop from Redis, and then processed, and the result is stored in the database. This is an effective way to solve the Iot Internet of Things, efficiently processing the massive data sent by hundreds of millions of devices. For example, in heart monitoring, data from hundreds of millions of sensors who have ill at the same time are sent to the cloud platform, and the number of concurrency is very large, so this method is needed to solve it.
For Internet applications, the offset is sent to the background program by sending an RPC request. The RPC program quickly processes it through the thread pool, obtains data from Kafka to complete the business, and returns the result. Go's web program returns the result to the user. For large-scale Internet multi-users such as Taobao and Jingdong with high concurrent access, this can solve the concurrency problem. Using Zookeeper and Netty to achieve high-efficiency RPC has played a miraculous effect in actual projects and achieved success. This structure is clever and novel, and has practical value.
In short, according to the basic principles of computers and software, a variety of novel high-concurrency solutions can be designed. This method cleverly uses the most advanced technology to support hundreds of millions of high-concurrency. It is effective. You can try it. Please give us valuable suggestions for discussion, and jointly eliminate the obstacle of high concurrency, thank you!