After experiencing countless days and nights, working from nine to nine, and overcoming countless difficulties, I finally completed the development of the system's scheduled functions, passed the test, and deployed it online. Do you feel complacent, have reached the pinnacle of life, and how lonely it is to sing about being invincible.
The reality is that if your system is not up to business and no one uses it, then it will be gone. If there are more and more people, keep using it. With the increase of users and business data, the system will definitely expose some performance problems. The optimal solution and monitoring of these problems often require higher and more comprehensive technical qualities and capabilities than the development of specific functions.
1. Performance problem monitoring
Uncle Guoyun: Performance problems are hidden. Good server hardware performance does not mean good system service performance . It may be difficult for inexperienced students to understand. Isn't it just that the system is slow, users will tell us? We have server monitoring. If the CPU or memory is full, there will be a monitoring alarm?
First of all, users may not tell you that the system is slow. If it is much slower than your competing products, users will vote with their feet. Then, if you develop a system that is lightning fast before running the hardware to the bottleneck, please accept the uncle's knees-_-||
Common performance problems are often caused by lack of performance considerations. While the response is extremely slow, the hardware utilization rate may be less than 5%. This kind of problem is also the main discussion of Uncle Guo.
According to experience, monitoring and early warning of performance problems is difficult to solve specific performance problems. In practice, we need some daily mechanisms to screen system performance problems and avoid terminal illness.
Database slow query log - is an important monitoring method, which records long-term database operation records. Slow query logs can be manually or automatically analyzed to screen for possible performance problems. All mainstream databases support the generation of slow query logs. The specific configuration methods are not described here. Non-statistical data operations should usually be less than 100ms.
Interface performance monitoring - the background system usually provides services through the service interface, and the front-end page or mobile device completes the corresponding operation by calling the server interface. The granularity of the interface is greater than that of the database operation, and an interface call may contain many database calls. The interface performance is closer to the user experience, because most of the time, an operation action by the user corresponds to a service interface (such as saving, confirming). When there is no slow query, the interface performance may not meet the requirements. The possible reasons are, for example, an interface calls 1000 database operations in a loop, or calls some third-party interfaces. In principle, the performance monitoring of the interface is to calculate the total time consumption by subtracting the entry time point from the call end time, and record this time consumption for timely screening. Spring's slicing capability can easily meet this requirement. Non-reports, export class interface, Uncle Guo thinks it should be completed within 1 second.
Message Queuing—— If your system uses a mechanism similar to message queuing, you should also monitor the number of messages queued in the queue. If messages accumulate, it means that at this stage, the overall consumption capacity of the system can no longer meet the requirements. Enter the request.
The above three monitoring dimensions are non-overlapping and should be carried out simultaneously.
2. Positioning of performance problems
In addition to the database slow query log that can clearly indicate that the specific SQL is slow, the other two situations above can directly indicate that the granularity of the location is relatively large, corresponding to an interface or a piece of processing code.
The specific method of locating more detailed problems is also easy to think of, that is, the time-consuming of each stage of segment output . For example, for a 100-line method, you can calculate and log the execution time of these three segments at the 30th, 60th, and 100th, respectively. Repeatedly, you can finally locate the problem.
The location of performance problems requires some common sense of performance . The so-called common sense of performance means that the time required to complete a task usually does not need to be very accurate, but the magnitude must be clear. For example, a database operation usually takes tens of milliseconds, the interaction with memory takes nanoseconds or microseconds, and the interface with a third-party system may take hundreds of milliseconds to several seconds. With these experiences, we can have a basic judgment on whether the time-consuming is reasonable. For example, for a database operation of 50 milliseconds, it takes 5 seconds to loop 100 times, which is already very slow. You can consider whether it can be combined into a batch operation.
3. Application performance optimization method
Coalescing remote calls - From experience, in practice looping database operations, or looping calls to third-party system interfaces, is a very, very common cause of performance problems. The optimization method for such problems is usually the way to optimize calls, using batch operations. For example, if you need to query Uncle Guo’s check-in records in the past year in the database, you can use the exact date to query 365 times, or you can use the time range to query 365 records at a time. The latter method is definitely much faster than the previous one. If the method is complex and lengthy, you can extract the required public data from it, perform a unified batch query, and put it in memory for backup, which will be much faster than searching where it is used. Similarly, write and modify operations should be performed in batches as much as possible.
Use cache - for data with high access frequency, it can be stored in memory, and the access speed can be accelerated by using the feature that memory access is much faster than hard disk. Common scenarios include various counts—how many times Uncle Guo’s articles have been viewed.
Multithreaded Parallelism— Modify serial to parallel through multithreading. For example, communicating with the device to query the state, querying the device one by one and parallel to the device at the same time, the latter is much faster. In reality, most operations are time-consuming on IO, so multi-threading can effectively improve performance and avoid "empty" and so on. Multi-thread parallelism requires attention to thread synchronization.
4. Database performance optimization
The database is a component that all modern systems rely on, and the solution to its performance problems is also something that developers need to master.
Adding indexes - adding indexes is the preferred solution for database optimization. The principle is to exchange space for time. Now the cost of storage is dropping very fast, and basically all regular business data queries can be indexed. For complex SQL, sometimes it is necessary to analyze how to add indexes, which can be analyzed through the execution plan.
Lock waiting - the database has a lock concept. Before a transaction occupies an X lock and is not committed, other transactions cannot operate on the record and can only wait. A database write operation that does not use a unique index may add a write lock to the entire table, and the entire table record cannot be operated before submission. Therefore, you should be aware of database locks, try to lock them at night, unlock them as soon as possible, and lock them in a small area as much as possible. It can be deduced that in practice, if it is a long transaction, try to move the update operation backward, and try to remove time-consuming non-transactional operations (such as non-dependent third-party interfaces, etc.) from the transaction.
Transaction isolation level - select the appropriate transaction isolation level, the transaction isolation level is related to the database lock, such as the strictest serial isolation level, all transactions will be performed serially. The widely used database MYSQL has a default isolation level of "repeatable read" , but it brings restrictions on gap locks and increases the probability of lock preemption. If nothing special, it can be adjusted to "read commit" according to the actual situation
Intermediate results - the essence can be understood as a cache at the database level. If some results are calculated from the full amount of records, the amount of data will be huge, and it will take a long time. It can be calculated in batches and store intermediate results. to speed up data acquisition. For example, to calculate the total expenditure of Uncle Guo’s family in the past year, you can add up each record, or you can calculate it once a month, so you only need to query the sum of the expenditure in the past 12 months.