1：服务器异常宕机、mysql、mysql_safe都无法启动
查看mysql报如下错误：
2018-03-30 06:25:12 1649 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future r
elease. Please use the full name instead.
2018-03-30 06:25:12 1649 [Note] Plugin 'FEDERATED' is disabled.
2018-03-30 06:25:12 1649 [ERROR] Function 'innodb' already exists
2018-03-30 06:25:12 1649 [Warning] Couldn't load plugin named 'innodb' with soname 'ha_innodb.so'.
2018-03-30 06:25:12 1649 [ERROR] Function 'federated' already exists
2018-03-30 06:25:12 1649 [Warning] Couldn't load plugin named 'federated' with soname 'ha_federated.so'.
2018-03-30 06:25:12 1649 [ERROR] Function 'blackhole' already exists
2018-03-30 06:25:12 1649 [Warning] Couldn't load plugin named 'blackhole' with soname 'ha_blackhole.so'.
2018-03-30 06:25:12 1649 [ERROR] Function 'archive' already exists
2018-03-30 06:25:12 1649 [Warning] Couldn't load plugin named 'archive' with soname 'ha_archive.so'.
2018-03-30 06:25:12 1649 [Note] InnoDB: Using atomics to ref count buffer pool pages
2018-03-30 06:25:12 1649 [Note] InnoDB: The InnoDB memory heap is disabled
2018-03-30 06:25:12 1649 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-30 06:25:12 1649 [Note] InnoDB: Memory barrier is not used
2018-03-30 06:25:12 1649 [Note] InnoDB: Compressed tables use zlib 1.2.8
2018-03-30 06:25:12 1649 [Note] InnoDB:Using Linux native AIO
2018-03-30 06:25:12 1649 [Note] InnoDB: Using CPU crc32 instructions
2018-03-30 06:25:12 1649 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-03-30 06:25:12 1649 [Note] InnoDB: Completed initialization of buffer pool
2018-03-30 06:25:35 1649 [ERROR] InnoDB: Tried to read 1048576 bytes at offset 1048576. Was only able to read 913408.
InnoDB: Fatal error: cannot read from file. OS error number 17.
2018-03-30 06:25:35 7fcc5aafc780 InnoDB: Assertion failure in thread 140515671525248 in file os0file.cc line 2694
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
22:25:35 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=400
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 175093 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x555930e2e94c]
/usr/sbin/mysqld(handle_fatal_signal+0x3c2)[0x555930b84e62]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7fcc59c81330]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7fcc590c2c37]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7fcc590c6028]
/usr/sbin/mysqld(+0x815583)[0x555930f3f583]
/usr/sbin/mysqld(_Z28buf_dblwr_init_or_load_pagesiPcb+0x272)[0x5559310105a2]
/usr/sbin/mysqld(+0x883b1c)[0x555930fadb1c]
/usr/sbin/mysqld(+0x7c5bbc)[0x555930eefbbc]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x48)[0x555930ab3b28]
/usr/sbin/mysqld(+0x4f19e0)[0x555930c1b9e0]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x928)[0x555930c21608]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0xa92)[0x555930aac9b2]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fcc590adf45]
/usr/sbin/mysqld(+0x375c2d)[0x555930a9fc2d]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Solution:
delete the ibdata*, ib_logfile* files in the data directory,
The database can be started after restarting, because the database is shut down abnormally
****All such situations must be backed up first to avoid damage caused by human operations
tar -zcf mysqlback20180329.tar.gz
After mysql is started normally, mysql can be accessed normally, But the log reports an error
2018-03-30 10:59:59 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the
table exists. See http://dev. mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03-30 10:59:59 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03 -30 11:00:00 15939 [Warning] InnoDB:Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03-30 11:00:00 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03-30 11:00:00 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03-30 11:00:00 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
2018-03-30 11:00:00 15939 [Warning] InnoDB: Cannot open table ambari/QRTZ_TRIGGERS from the internal data dictionary of InnoDB though the .frm file for the ta
ble exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
Open the table below this library and find that the prompt does not exist, but both frm and ibd files exist. The
mysql command line test found that the table status is incorrect
mysql> check tables QRTZ_TRIGGERS ;
+-------------- --------+-------+----------+---------------------- ----------------------+
| Table | Op | Msg_type | Msg_text |
+----------------- -----+-------+----------+------------------------- -------------------+
| ambari.QRTZ_TRIGGERS | check | Error | Table 'ambari.QRTZ_TRIGGERS' doesn't exist |
| ambari.QRTZ_TRIGGERS | check | status | Operation failed |
+------------+-------+----------+----- ---------------------------------------+
2 rows in set (0.00 sec)
So exit directly and check how many tables are damaged in the entire database.
mysqlcheck -A -uroot -ppassworkd > /tmp/unnormaltable
** (check a single database and table: mysqlcheck zabbix users -uroot -ppassword)
Output from the above command and then analyze which tables need to be restored
grep -v ":" /tmp/unnormaltable |awk -F"." '{print $1,$2}'|sort -rn |uniq -c
恢复zabbix库
grep -v ":" /tmp/unnormaltable|grep zabbix |awk -F"." '{print $2}' > tablezabbix
Mv all the files in the zabbix library to other directories and import the empty table structure
mysql -uroot -ppassword < zabbix.sql
To the table space, empty the idb file, leaving only the frm file
while read line;do mysql -uroot -ppassword -e "use zabbix;SET FOREIGN_KEY_CHECKS = 0 ; ALTER TABLE $line DISCARD TABLESPACE ";done < tablezabbix
ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails ()
SET FOREIGN_KEY_CHECKS = 0
Reimport the previously backed up ibd file to the zabbix directory
for i in `find myzabbix/ -name *.ibd `;do mv $i ./zabbix;done
import the imported ibd file into the tablespace
while read line;do mysql - uroot -ppassword -e "use zabbix; ALTER TABLE $line IMPORT TABLESPACE ";done < tablezabbix
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock'
During the import process, if some table space is too large, an error may be reported. You can properly optimize the two parameters
wait_timeout = x The timeout time is as follows 600 seconds
max_allowed_packet = y maximum allowed data size
The overall process:
1. Backup
2. Determine which tables are damaged
3. Remove the data of the damaged tables to the backup directory
4. Create these tables
5. Export the tablespace
6. Move in the ibd files of the tables just moved out
7. Import the tablespace
8 ,test