Recently a pressure test project, the performance of typical problems encountered, the process of record, to give you locate tuning reference;
Appearance:
Single interface load test, QPS up to 300, the response time of 200ms, the application cpu reached 90%, 8c machine, as shown, there may be some students write here to say: the processing capability can also, not to add the machine, expanded node !
Of course, this is a solution, but I think if to do so directly, this is one of the most low of programs, but can not find the essence of the problem; back to just say, I only describe the state of the application server, from complete performance testing, the entire link needs to monitor various indicators, the link line and again after the application layer to the data flow is greater in the following figure (please do not say expanding the bandwidth)
Found from these two problems monitoring, applications continue to look cpu, see deployment details, the server docker deployed about 10 nodes, node status docker each view, one of which reaches 623.59% (* Audit) FIG,
The focus of the investigation found, enter the relevant container, jstat view gc state, ygc up to three 1s, is also possible, just said what the traffic, after Iftop found mainly in the redis server application to interact with, to see from the above description, we can to summarize redis to get large amounts of data, resulting in higher traffic and large volumes of data can lead to frequent ygc surge of applications cpu, so nothing wrong interpretation, the truth is through, but this is just your speculation, but also to do further validation that a lot of data, that is what business data, not the code in the case of commuting, I usually just dump, get jedis method cpu consumption hot spot method, dump to a file found in the user information with a large number of coupons (FIG),
ok, to this point has been very clear fundamental problem, and with the development confirmed that indeed the operational level to obtain a large number of invalid coupon information leading to the development for business layer filtered, qps of 1500, network, cpu return to normal.
No more exchanges of public attention: Ape school desk