Summary and optimization of common faults in Mysql!!

Case pre-knowledge points

To learn how to optimize, you must first have a deep understanding of the logical architecture of MySQL. Figure 8.1 is the MySQL logical architecture diagram, which allows us to understand the operating principle of MySQL more clearly.

The top layer is some client and connection services, including local sock communication and most TCP/IP communication based on client/server tools. Mainly complete some connection processing, authorization authentication, and related security schemes. In this layer, the concept of thread pool is introduced to provide threads for clients that access through security authentication. The SSL-based secure link can also be realized on this layer. The server will also verify the operation authority it has for each client that accesses securely

The second layer architecture mainly completes most of the core service functions, such as SQL interface, cache query, SQL analysis and optimization, and the execution of some built-in functions. All cross-storage engine functions are also implemented at this layer, such as procedures and functions. At this layer, the server will parse the query and create the corresponding internal parse tree, and complete the corresponding optimization, such as determining the order of the query table, whether to use the index, etc., and finally generate the corresponding execution operation. If it is a select statement, the server will also query the internal cache. If the cache space is large enough, the performance of the system can be improved in an environment where a large number of read operations are resolved.

At the storage engine layer, the storage engine is really responsible for the storage and extraction of data in MySQL, and the server communicates with the storage engine through API. Different storage engines have different functions and can be selected according to their actual needs

The data storage layer is mainly to store data on the file system running on the bare device and complete the interaction with the storage engine

MySQL single instance troubleshooting

(1) Failure phenomenon

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/data/mysql/mysql.sock' (2)

Problem analysis: The above situation is generally caused by the database not being started or the database port is blocked by the firewall. Solution: start the database or open the database listening port on the firewall

(2) Failure phenomenon

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

Problem analysis: The password is incorrect or there is no permission to access. Solution: 1 Modify the main configuration file of my.cnf, add skip-grant-tables under [mysqld], and restart the database. The last modification password command is as follows

mysql>use mysql; 
mysql> update mysql.user set authentication_string=password('123456') where user='root' and Host = 'localhost'; 
mysql> flush privileges;

再删除刚刚添加的 skip-grant-tables 参数，重启数据库，使用新密码即可登录。

(3) Symptoms
When using remote connection to the database, occasionally the problem of slow remote connection to the database may occur.

Problem analysis: If the MySQL host is slow to query DNS or there are many client hosts, the connection will be slow. Since the development machine cannot connect to the external network, it is impossible to complete DNS resolution when connecting to MySQL. I also understand why the connection is so slow.

Solution: Modify the my.cnf main configuration file, add skip-name-resolve under [mysqld], and restart the database to solve it. Note that hostname authorization cannot be used in future authorizations

(4) Failure phenomenon

Can't open file: 'xxx_forums.MYI'. (errno: 145)

Problem analysis: The
server is shut down abnormally, the space where the database is located is full, or some other unknown reason causes damage to the database table.
It may be that if the database file is copied and moved directly under the operating system, this error will occur due to the grouping of the file.

Solution: The following two repair methods must back up the database before execution. The following two methods
can be used to repair the data table (the first method is only suitable for independent host users):

1. Use MySQL's own special user data table check and repair tool myisamchk. Generally, the myisamchk command can only be run under the command line. The commonly used repair commands are:
myisamchk -r data file directory/data table name.MYI;

Through phpMyAdmin repair, phpMyAdmin has the function of repairing the data table. After entering a table, click "Operation", and click "Repair Table" in the "Table Maintenance" below.

2. Modify the file group (only suitable for independent host users): During
the process of copying the database file, the database file is not set to be readable and writable by the MySQL running account (generally applicable to Linux and FreeBSD users)

（5）故障现象
ERROR 1129 (HY000): Host ‘xxx.xxx.xxx.xxx’ is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts

Problem analysis: Due to the mysql database parameter: max_connect_errors, its default value is 10. When a large number of (max_connect_errors) hosts connect to MySQL, and the total connection request exceeds 10 times, the new connection can no longer connect to the MySQL service. Blocking caused by too many interrupted database connections in a short period of time with the same ip (exceeding the maximum value of max_connection_errors of the mysql database)

#解决方法：
#使用 mysqladmin flush-hosts 命令清除缓存，命令执行方法如下：

1.mysqladmin -uroot -p -h 192.168.241.48 flush-hosts 
Enter password:

2.修改 mysql 配置文件，在[mysqld]下面添加 max_connect_errors=1000，然后重 启 MySQL

(6) Fault phenomenon The
client reports Too many connections.
Problem analysis: The number of connections exceeds the maximum number of connections limit of Mysql.

#解决方法：
1.在 my.cnf 配置文件里面增大连接数，然后重启 MySQL 服务
max_connections = 10000

2.临时修改最大连接数，重启后不生效。需要在 my.cnf 里面修改配置文件，下次重启生效
set GLOBAL max_connections=10000;

(7) Failure phenomenon

Warning: World-writable config file '/etc/my.cnf' is ignored
 ERROR! MySQL is running but PID file could not be found

Problem analysis: MySQL configuration file /etc/my.cnf has incorrect permissions.

#解决方法：
chmod 644 /et/my.cnf

(8) Failure phenomenon

InnoDB: Error: page 14178 log sequence number 29455369832
InnoDB: is in the future! Current system log sequence number 29455369832

Problem analysis: Innodb data file is damaged

#解决方法：
#修改 my.cnf 配置文件，在[mysqld]下添加 innodb_force_recovery=4, 
启动数据库后备份数据文件，然后去掉该参数，利用备份文件恢复数据

MySQL master-slave troubleshooting

(1) Failure phenomenon
Slave_IO_Running of the slave library is NO

Problem analysis: The server-id value of the main library and the slave library are the same.

Solution: Modify the server-id value of the slave library to be different from the main library. Restart after modification and synchronize again

(2) Symptom 2
Slave_IO_Running of the slave library is NO

Problem analysis: There are many reasons for the slave library thread to be NO. The main reason is that the primary key conflicts or the master database deletes or updates data. The slave database cannot find the record and the data is modified. Usually the status code error is 1007, 1032, 1062, 1452, etc.

#解决方法一：
mysql> stop slave; 
mysql> set GLOBAL SQL_SLAVE_SKIP_COUNTER=1; 
mysql> start slave;

解决方法二：
设置用户权限，设置从库只读权限
set global read_only=true;

(3) Failure phenomenon

Error initializing relay log position: I/O error reading the header from the binary log

Analyze the problem: the relay-bin of the slave library is damaged

#解决方法：
手工修复，重新找到同步的 binlog 和 pos 点，然后重新同步即可

mysql>CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.xxx',MASTER_LOG_POS=xxx;

MySQL optimization

Hardware

When it comes to server hardware, the most important factors are CPU, memory, and disk.

(1) About CPU
CPU For MySQL applications, it is recommended to use SMP architecture multi-channel symmetrical CPU. For example: Two
Intel Xeon 3.6GHz CPUs can be used . It is now recommended to use a 4U server as a dedicated database server, not only
for MySQL

(2) About memory
Physical memory For a Database Server using MySQL, the server memory is recommended not to be less than
2GB, and physical memory above 4GB is recommended. However, memory is a
negligible problem for current servers . When I encounter high-end servers at work, the memory basically exceeds 32G.

(3) About the disk
Disk seek capability (disk I/O). Take the high-speed SAS hard disk (15000 rpm) currently on the market as an example.
This kind of hard disk seeks 15000 times per second in theory. This is determined by physical characteristics and cannot be changed. MySQL
is performing a large number of complex query operations every second , and the amount of reads and writes to the disk can be imagined. Therefore, it is generally considered that disk I/O is
one of the biggest factors restricting the performance of MySQL. RAID-0+1 disk arrays are usually used. Be careful not to try to use
RAID-5. MySQL is not efficient on RAID-5 disk arrays. . If you do not consider the investment cost of hardware, you can also
consider solid-state (SSD) hard disks specifically used as database servers. The read and write performance of the database will definitely improve a lot

MySQL configuration file

Usually the default my.cnf configuration file can not play the highest performance of MySQL, so it needs to be optimized according to different hardware, and the optimization of the configuration file is also the focus. The following are the database optimization parameters with a physical memory of 32G, which are optimized in terms of global, binary log, master-slave, innodb, and myisam, for reference only.

（1）default-time-zone=+8:00
The default MySQL uses the system time zone, modified to Beijing time, which is the so-called Dongba District
（2）interactive_timeout = 120
The number of seconds the server waits for activity before closing an interactive connection.
（3）wait_timeout = 120
The number of seconds the server waits for activity before closing a non-interactive connection
（4）open_files_limit = 10240
The MySQL server has a limit on the number of open file handles.
（5）group_concat_max_len = 102400
MySQL default splicing maximum length is 1024 bytes, because 1024 bytes will be insufficient, modify it according to actual situation
（6）user=mysql
Run as the mysql user.
（7）character-set-server=utf8、init_connect=‘SET NAMES utf8’
Set the character set to utf8
（8）back_log = 600
How many requests in a short period of time can be stored in the stack before MySQL temporarily stops responding to new requests. If the system has many connections in a short period of time, you need to increase the value of this parameter, which specifies the size of the listening queue for incoming TCP/IP connections. Default value 50
（9）max_connections = 5000
MySQL allows the maximum number of process connections. If the Too Many Connections error prompt appears frequently, you need to increase this value.
(10）max_connect_errors = 6000
Set the maximum number of abnormal interruption of each host's connection request. When this number of times is exceeded, the MySQL server will prohibit the host's connection request until the MySQL server is restarted or the host's related information is cleared through the flush hosts command

（12）table_open_cache = 2048
Specify the size of the table cache. Whenever MySQL accesses a table, if there is room in the table buffer, the table is opened and put into it, so that the contents of the table can be accessed faster
（13）max_heap_table_size = 256M
This variable defines the size of the memory table (memory table) that users can create. This value is used to calculate the maximum row value of the memory table. This variable supports dynamic changes, namely set @max_heap_table_size=#.
But there is no use for the existing memory table, unless the table is recreated (create table), modified (alter table) or truncate table. Service restart will also set the existing memory table to the global max_heap_table_size value
（14）external-locking = false
Use skip-external-locking MySQL option to avoid external locking. This option is enabled by default
（15）max_allowed_packet = 32M
设置在网络传输中一次消息传输量的最大值。系统默认值为 1MB，最大值是 1GB，必 须设置 1024 的倍数
（16）sort_buffer_size = 512M
Sort_Buffer_Size 是一个 connection 级参数，在每个 connection（session）第一次需 要使用这个 buffer 的时候，一次性分配设置的内存。Sort_Buffer_Size 并不是越大越好，由 于是 connection 级的参数，过大的设置+高并发可能会耗尽系统内存资源。
（17）join_buffer_size = 8M
用于表间关联缓存的大小，和 sort_buffer_size 一样，该参数对应的分配内存也是每个 连接独享
（18）default-storage-engine = innodb
默认引擎，现在一般都是 innodb 引擎表居多
（20）thread_stack = 192K
设置 MySQL 每个线程的堆栈大小，默认值足够大，可满足普通操作。可设置范围为 128K 至 4GB，默认为 192KB
（21）transaction_isolation = READ-COMMITTED
设定默认的事务隔离级别，READCOMMITTEE 是读已提交。
（22）key_buffer_size = 1024M
指定用于索引的缓冲区大小，增加它可以得到更好的索引处理性能。
（23）bulk_insert_buffer_size = 64M
批量插入数据缓存大小，可以有效提高插入效率，默认为 8M
(24)skip-name-resolve
禁止域名解析，包括主机名.所以授权的时候要使用 IP 地址。

关于 MySQL 二进制日志文件

（1）log-bin=mysql-bin
打开 MySQL 二进制功能。
（2）expire_logs_days = 7
二进制日志只留存最近 7 天，不用人工手动删除
（3）slow_query_log
打开慢查询日志
（4）slow_query_log_file=slow.log
慢查询日志文件位置
（5）long_query_time = 2
记录超过 2 秒的 SQL 查询

关于引擎是 innodb 的优化如下：

（1）innodb_additional_mem_pool_size = 64M
这个参数用来设置 InnoDB 存储的数据目录信息和其它内部数据结构的内存池大小，类 似于 Oracle 的 library cache。这不是一个强制参数，可以被突破
（2）innodb_file_io_threads = 4
文件 IO 的线程数，一般为 4，但是在 Windows 下，可以设置得较大
（3）innodb_thread_concurrency = 8
服务器有几个 CPU 就设置为几，建议用默认设置，一般为 8
（4）innodb_write_io_threads = 8
InnoDB 使用后台线程处理数据页上写 I/O（输入输出）请求的数量。一般设置为 CPU 核数，比如 CPU 是 2 颗 8 核的，可以设置为 8
（5）innodb_log_files_in_group = 3
#为提高性能，MySQL 可以以循环方式将日志文件写到多个文件
（6）innodb_file_per_table = 1
独享表空间（关闭）
（7）innodb_open_files = 8192
innodb 打开文件句柄数

关于引擎是 myisam 的优化

（1）myisam_sort_buffer_size = 128M
MyISAM 表发生变化时重新排序所需的缓冲大小
（2）myisam_max_sort_file_size = 10G
MySQL 重建索引时所允许的最大临时文件的大小(当 REPAIR，ALTERTABLE 或者LOADDATAINFILE)。
如果文件大小比此值更大,索引会通过键值缓冲创建(更慢)
（3）myisam_repair_threads = 1
如果一个表拥有超过一个索引，MyISAM 可以通过并行排序使用超过一个线程去修复
这对于拥有多个 CPU 以及大量内存情况的用户是一个很好的选择
（4）myisam_recover
Automatically check and repair MyISAM tables that are not properly closed