Author: Zen and the Art of Computer Programming
Internet services have become a high-growth industry today. Various data such as user visits to websites, click-through rates, and browsing behaviors will generate massive amounts of data. These data usually need to be stored in the database for subsequent processing, such as for analyzing user behavior, user portrait, product recommendation, etc. At the same time, due to different application scenarios, the database will also run on different servers. Different servers may host different business modules, and the data of these modules are also different. Therefore, a single database server may carry a large number of business modules, and its database resources and hardware configurations are also different. In order to ensure the stable and healthy operation of the system, database administrators often need to monitor the running status, performance and resource utilization of the database in real time. In addition, as an important infrastructure service, the daily operation and maintenance of the database also requires a dedicated system to complete. The database system generally has a complete log function, which records various information during the operation of the database, including error logs, slow query logs, etc. Therefore, the information collection, cleaning, analysis, and display of database system logs is also an important task. All in all, database monitoring is the real-time monitoring of the operating status, performance and resource utilization of the database system. This article will illustrate how to monitor the performance of the database and how to collect, clean, analyze, and display the database log information through some examples and actual cases. Hope to be helpful to readers.
2. Explanation of basic concepts and terms
2.1 Database performance indicators
The performance indicators of the database mainly include the following aspects:
- Query Response Time (QPS): The number of queries processed per second, which can be measured by looking at the TPS, latency, and deadlock values of the database.
- CPU utilization: CPU utilization on the database server, which mainly measures CPU load and SQL statement execution efficiency.
- Memory usage: The usage rate of memory on the database server, mainly focusing on the swap partition and cache hit rate.
- Number of IO requests: disk IO requests