Since the configuration has been running for so long and is very stable, it is basically ignored, so this time it is mainly the optimization of sql, and it is concentrated in the personal space of open source China. The following is the optimized database version:
Case 1: Fan query optimization
There are 2 sql for fan query
--查询所有粉丝
SELECT user FROM osc_friends f INNER JOIN osc_users u
ON u.id=f.user AND f.friend=? AND f.user<>? ORDER BY create_time DESC
--查询粉丝数量
SELECT COUNT(friend) FROM osc_friends f INNER JOIN osc_users u
ON u.id=f.user AND f.friend = ? AND f.user <> ?These two queries can be optimized in business. The purpose of inner join to an osc_users table is to remove the userid that comes with osc_friends, but the osc_users table is a relatively large table . Design Notes
Optimization ideas
Simplify sql, and put the logic of the userid that comes with it into the code layer for processing
Optimized
SELECT user FROM osc_friends f WHERE f.friend=? ORDER BY create_time DESC
SELECT COUNT(*) FROM osc_friends f WHERE f.friend = ?SQL simplifies a lot, greatly improving the query speed
summary
Sometimes business processing is placed in the code layer, which can achieve unexpected results
Case 2: Private message optimization
SELECT MAX(id) AS id, COUNT(id) AS msgCount
FROM osc_msgs WHERE user = 12 GROUP BY friend ORDER BY id DESCThe osc_msgs table stores all private message records. As time goes by, the table gradually becomes larger, and the cost of a query becomes higher, which basically takes more than 1 second.
Optimization ideas
Get the latest two-person conversation from the private message table and put it into a new osc_last_msgs table, and update the osc_last_msgs table every time a private message is sent. This table only records the latest private message, so that the optimized private message list sql does not need to find data in the msg table , just go to the osc_last_msgs table to find .
Optimized
SELECT * FROM osc_last_msgs WHERE user=? ORDER BY msg_id DESC
summary
A typical case of reducing the amount of data from large to small
Case 3 Comment optimization
SELECT
l1.id
FROM
osc_opt_logs l1,
osc_opt_logs l2
WHERE
l1.obj_type IN (101, 111, 113, 116, 119, 121)
AND l2.obj_type IN (
100,
110,
112,
114,
118,
120,
123,
124,
122,
125,
126,
127,
99
)
AND l1.parent_id = l2.id
AND l2. USER = 12
ORDER BY
l1.id DESC
LIMIT 20;Try to establish a joint index for optimization, but the effect is not good, because the optlog table is very large, so the query efficiency of the joint table is extremely low, and it takes up a lot of query cache space.
Optimization ideas
Add a reply_user field to mark the response of the reply, so that the entire joint table query operation can be simplified
Optimized
SELECT id FROM osc_opt_logs where reply_user = 12 ORDER BY id DESC limit 20;summary
Appropriate redundant fields can reduce the complexity of SQL
Case 4 Index Optimization
Index optimization mainly relies on the explain command. I believe everyone is familiar with the explain command. For specific usage and field meanings, please refer to the official website explain-output . It needs to be emphasized that rows are the core indicators, and most statements with small rows are generally executed quickly. So the optimization statement is basically optimizing the rows.
Generally speaking.
- rows<1000 is acceptable.
- Rows is between 1000 and 1w, which may cause performance problems during intensive access, but if it is not accessed too frequently (frequency less than once a minute) and it is difficult to optimize, it is acceptable, but it needs to be observed.
- When the number of rows is greater than 10,000, the design of SQL should be carefully considered, and the SQL should be optimized.
There is no absolute value for this reference. Generally speaking, the smaller the better. If a database with 1 million data volumes has 700,000 rows, it can be judged that the query performance of SQL is very poor. If a database with 1 million pieces of data has rows, It is 10,000. From my personal point of view, it is still acceptable.
The other is the extra information, which contains the details of the query that MySQL solves, focusing on the occurrence of keywords:
Using filesort : When the Query contains an order by operation and the index cannot be used to complete the sorting operation, MySQL Query Optimizer has to choose the corresponding sorting algorithm to implement.
Using temporary : When a temporary table must be used in some operations, Using temporary will appear in the Extra information, mainly in operations such as GROUP BY and ORDER BY
When Using filesort and Using temporary appear in the execution plan Extra, you can consider whether to perform sql optimization and adjust the index, and finally adjust the parameters related to sorting or temporary tables in my.cnf, such as sort_buffer_size or tmp_table_size.
For example the following:
The reason is that the mysql query uses only one index. If the where clause uses an index, the columns in the order by will not use the index. Therefore, the conditions of order by also need to be added to the index to form a joint index. After optimization
Another point to note is that the index has the principle of the leftmost prefix : the joint index (a,b,c) can match (a), (a,b), (a,b,c) but not match (b,c)
summary
The explain SQL statement should be a habitual action in daily development. Sometimes the result of explain may be unexpectedly deviating from the design.
Case 5: Focus on database status
When I look forward to a substantial performance improvement after optimization, the reality is always joking with you. After testing and inspection, it is found that it is not a SQL problem, but may be related to the server. Use the top command to see the CPU usage of the MySQL process. It has been around 100%, which is strange. I used show processlist to look at the mysql process, and found that a suspicious sql was being executed all the time. After the kill, the CPU usage dropped immediately.
summary
When you find that the database cpu or io is abnormal, use show processlist to see what the database is busy with
write at the end
After this optimization, the opening speed of personal space has been improved. Summarize a few experiences:
1. Don't expect all SQL statements to be optimized through SQL, and business adjustments will bring unexpected results;
2. All performance optimizations are space for time , and redundancy is used to improve performance. The general idea is to maximize the small and divide and conquer.
3.explain is an introduction to sql optimization
4. The advantages of indexing outweigh the disadvantages, use it more and make good use of it
If it is stuck again, it may start from the aspect of sub-library sub-table, read-write separation.
The mysql before optimization is very different from the optimized mysql. The same mysql is used to support Taobao, Tencent, and Facebook, but it is very difficult to support your business system?
Please point out the above inaccuracies.