1. Problem description
Launched today, the production mysql has a large table with 27 million data lt_integral_detail_info, and we are going to add fields and indexes to this table;
It is time-consuming to execute the command of adding fields and adding indexes. As a result, someone performed multiple select count(*)operations on this table at this time, which directly squeezed out the Navicat session, and the CPU also soared. Other additions, deletions, modifications, and queries to this database also changed. It's very slow, and I can't run out for half a day.
Quickly contact the database management personnel, kill the operations of adding fields and adding indexes, and mark the stuck select count(*)operations as kill, but it has not been killed for a long time.
As shown in the picture:
It can be seen that the card has been stuck for a long time, and it has not been killed.
At this time, if you operate on other tables, there is no problem; but if you operate the lt_integral_detail_info table again, it will be stuck again, and other additions, deletions, modifications, and queries to this database will also become very slow.
2. Online solutions
1. I found such an article on the Internet:
https://cloud.tencent.com/developer/article/1857161?from=15425&areaSource=102001.9&traceId=OjyLyWs7caMy1jo-2hFIA
It roughly means that when mysql executes the kill operation, it first marks it, and then kills it later; the
lower version of mysql has a bug, which is caused by parallel read operations based on the primary key;
if this bug is triggered, the sql statement will not be killed.
2. The solution, according to the Internet, can upgrade mysql to 8.0.22
3. If you cannot upgrade, you can increase the configuration:
innodb_parallel_read_threads=1
Then restart MySQL to solve.
3. Personal solutions
1. Under normal circumstances, a kill sql statement will take effect soon; it is rare that it is marked as kill without being killed for several hours; you can choose to
continue to wait to see if it can be killed normally. (As shown in the figure above, 7 SQLs are stuck, and one is missing after 2-3 hours, which means that one can still be killed normally after 2-3 hours.)
2. Since mysql cannot be easily restarted in the production environment, another table is temporarily built, and the data is synchronized from the standby database, and the stuck table is left alone. (The version must be released as soon as possible, and the normal business cannot be affected)
3. Summary
1. When executing the sql statement for adding fields and adding indexes to a large table, try not to operate this table again, so as to prevent sql from being stuck and unable to be killed.
2. If there is a sql statement to operate a certain table, and it still cannot be killed after being marked as kill, try not to operate this table, otherwise the new operation sql will still be stuck, and will cause the operation of other tables in the library to fail. It changes very slowly. (can only be marked as kill again)
3. It cannot be killed after marking the kill. You can wait for a while (a few hours or a whole night) to see if there is any change. It may be possible to recover by yourself, and then kill it slowly.
4. If allowed, upgrade mysql to 8.0.22
5. Or increase the configuration innodb_parallel_read_threads=1, and then restart mysql to solve it.