Performance testing: actual analysis of database performance problems

Interface stress test analysis
Now let's stress test an interface for obtaining user information, which involves data query of the database. Our project is deployed on the application server, so we need to monitor both the application server and the database server.
Then let's take a look at the tomcat server. The cpu usage rate is not high, including network, disk IO, etc. There are no problems.
Next, let's take a look at the server of the database, and we can see that the CPU has reached 100%. There are very obvious performance issues. Then we can make a preliminary analysis that the problem may appear in the database query, because we only use 10 concurrency, which is not too much.
Database Performance Analysis - Index

Let's take a look back at the phenomena that occurred in our previous stress test:
TPS is very low, the response time is very long, the CPU of the database server is high (close to 100%), and the load of the application server is relatively low.

Usually our database server cpu is very high, which is usually caused by the low efficiency of SQL execution. There may be three reasons:

• 1. There is a lack of necessary indexes in the database table;
• 2. The index does not take effect;
• 3. SQL is not optimized enough;

about index

An index is a structure that sorts the values ​​of one or more columns in a database table, stores key fields in a table, and uses an index to quickly access specific information in a database table.

Then we preliminarily locate the performance problem caused by SQL. Can we monitor SQL?

We need to check whether mysql has enabled slow query

1｜show variables like '%slow_query_log%

We can see that slow sql is not enabled.

Next, we need to configure slow query for mysql. How to configure, you can check my blog: https://blog.csdn.net/weixin_42274846/article/details/128032102

Next, we start to monitor the SQL and enter the log path stored in mysql (the specific log path was returned in the slow query configuration before), because before the stress test, we don’t know whether the data in it is the data we tested before , so we enter > 文件名称，如 > mysql-slow.logto clear the data in it, and then perform the pressure test.

During the stress test, we can see that there are a large number of SQL logs in the log, so we can be sure that the response time of these SQLs has exceeded the expected value of our original configuration during the stress test.
So we already know what slow SQLs are there, how should we analyze these SQLs next?

In the /usr/bin directory, use the command mysqldumpslow that comes with mysql. This tool provides some parameters below. The introduction of the parameters can be introduced in my previous slow sql configuration blog.

Then we use the following command to analyze:

1｜mysqldumpslow –s at -t 5 mysql-slow.log
// 显示出耗时最长的5个

Above we can see that a SQL is returned. Let's take a closer look at what the returned data means:

1. Count: As the name suggests, it is the number of times this SQL is executed
2. Time: SQL execution time. The slow SQL time I configured here is (0.05s). Here, 0.07s is equivalent to 70ms. Many people may think that 70ms is short, and the time is really short, but for an interface, the database It took 70ms, so the entire http request, including business logic, network factors, and front-end rendering, etc., the response that the end user gets is very long.
3. The SQL that times out is shown below

We can see that this is a very basic query SQL, so we already know that SQL is very slow, how should we analyze it? Our mysql provides explaina method.

Introduction to explain
The keyword explain can be used in mysql to obtain (query) the query execution plan of the sql statement. Using the explain keyword, you can simulate the sql statement executed by the mysql optimizer, so as to know how mysql processes the sql statement. Through explain, you can analyze the performance bottlenecks of query statements or table structures.

The role of explain
①, check the reading order of the table

②. Operation type of data read operation

③. Check which indexes can be used

④. Check which indexes are actually used

⑤. View references between tables

⑥. Check how many rows of each table are executed by the optimizer

Open navicatthe database connection tool, friends who have done the test, almost everyone's computer will install this tool. Next, we enter this SQL statement, before the SQL, add EXPLAIN, EXPLAIN refers to the execution plan.

Why is our SQL executed slowly? The main thing to check is the type of the result. Type refers to our usage of the index. The following is the relationship between the TYPE value of the MYSQL execution plan:
Performance: all < index < range < index_merge < ref_or_null < ref < eq_ref < system/constPerformance can basically be tuned under the range

Then we can see the above SQL query, the type value he returns is all, and this performance is very poor. Usually when all appears, it is mainly because the table is not indexed. Next, we open the table design of the table and we can see that there is no index built in the table.

Add an index to him, usually the index is the condition after adding sql where. (It is usually found to be an index problem, and you can ask the development to notify the DB to add it.)
Next, we will add this index value. Above we can see that there is an index type, and the types commonly used in work are NORMALand UNIQUE. The selection type needs to be determined according to the business type. UNIQUE is the only value. If we choose it UNIQUE, then the user_name will be unique. If the same user_name is inserted later, the database will report an error.

Here our company's business is that user_name is unique. So choose UNIQUE. The rear index method is selected by default BTREE.

After saving, let's take a look at the execution plan of sql. We can see that it directly becomes const. From worst to best.
Now we have added an index, and then re-do the pressure test. Let's take a look at the stress test data. The performance has improved significantly, from the original 40 tps to more than 400, and the response time has also changed from the original 200ms to about 20ms.

Next, let's take a look at the database server again. The idle rate of the CPU has now become about 70%.
Let's look at the application server again, but now the pressure is on the application server. Usually, if the performance of the application server is not good, we can expand the capacity, but if it is a performance problem of the database server, it will be more troublesome, which involves an architectural problem.
The results conclude that
if our SQL is a full table scan, it will greatly affect the performance of the server and directly cause the CPU of the database server to soar. Therefore, it can be concluded that the setting of the index directly affects our performance, and the selection of the index is also very different.