ZuulException REJECTED_SEMAPHORE_EXECUTION is a frequently encountered exception in performance testing recently. The query data is found because Zuul defaults that each route is directly isolated by semaphore, and the default value is 100, that is, when the semaphore of a route request is higher than 100, it will refuse service and return 500.
Semaphore isolation
Since the default value is too small, increase the semaphore of each route in the gateway configuration and then experiment.
The semaphore of the two routes is raised to 2000 and 1000 separately. Let's test it again with gatling.
1 |
setUp(scn.inject(rampUsers(200) over (3 seconds)).protocols(httpConf)) |
This is our model, launching 200 users within 3s, accessing 5 APIs sequentially. So there will be 1000 requests. The machine configuration has only 2 cores and 16G, and it is a dockerized database. So the overall performance is not high.
Looking at the results, there are still 57 KOs, but it is much better than the previous ratio of 900 KOs for 1000 Requests.
thread isolation
Edgware版本的spring cloud提供了另一种基于线程池的隔离机制。实现起来也非常简单，
use-separate-thread-pools的意思是每个路由都有自己的线程池，而不是共享一个。
thread-pool-key-prefix会指定一个线程池前缀方便调试。
The part of hystrix mainly sets the size of the thread pool. Here, 10000 is set. In fact, the bigger the better. The larger the thread pool, the more significant the effect of cutting peaks and filling valleys, that is, time for space. The overall load of the system will increase, resulting in longer and longer response times. When the response time exceeds a certain limit, the system is actually unusable. The data can be seen later.
This time there is no 500, and 1000 Requests are returned normally.
Compare
Compare the effects of the two isolations from several pictures. The upper picture is semaphore isolation, and the lower picture is thread isolation.
response time distribution
Intuitively, it can be found that the distribution using thread isolation looks better, and the response within 600ms will be more.
QPS
The two figures show the number of Requests and Responses at the same time.
First look at the semaphore isolation scenario, the response per second is gradually increased, but after reaching an order of magnitude, the gateway starts to refuse service. Guess the semaphore limit is exceeded or it times out?
The thread isolation is more interesting. You can see that Request per second rises faster than the above, indicating that the system is trying to receive more requests and distribute them to the thread pool. At a certain point in time, Response per second began to decline, because the continuous creation of threads consumed a lot of system resources and the response became slower. After that, because there were fewer requests, the load decreased, and the Response began to rise again. Therefore, the thread pool is not as big as possible, and it is necessary to constantly debug to find a balance point.
Here I recommend an exchange and learning group to everyone: 697579751, which will share some videos recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, principles of microservice architecture, JVM performance Optimizing these becomes an essential knowledge system for architects. You can also receive free learning resources, which are currently benefiting a lot:
summary
The thread pool provides a better isolation mechanism than the semaphore, and it is found from the actual test that more requests can be completed in high-throughput scenarios. However, the overhead of semaphore isolation is smaller. For a system within 10ms itself, semaphore is obviously more suitable.