这是学习笔记的第 1908 篇文章
今天对表数据恢复的脚本进行了补充，反反复复做了测试，确保是一个相对稳定的版本。
目前对于一个3000多万表数据量的文件（3G）从.frm文件解析到.ibd文件导入，整个过程花费的时间在1分钟。
其中数据导入的部分耗时较为明显。
+----------+
| 34795259 |
+----------+
real 0m43.532s
user 0m0.042s
sys 0m4.894s
这种脚本其实还得靠缘分，你如果刚好碰到，能派上用场，那么这就是一个法宝，否则，就是简单看看，提一些不疼不痒的问题。
下面是两个脚本的内容，第1个脚本是通过解析.frm文件来得到表结构信息，这个过程是不需要.ibd文件的，纯粹解析.frm
在脚本的配置中，需要你预先配置一个测试库test_recover,保证这个库是可以反复折腾的，而不是线上业务库，对于恢复数据和模拟问题还是很方便的。在解析后会生成相应的SQL文件，其中对于非关系型字段的解析是存在问题的，常用数据类型是OK的。
#get table ddl from .frm
SOURCE_DUMP_PATH='/data/tmp_bak'
RECOVER_TABLE_NAME='receipt_12149_420'
MYSQL_DATA_DIR=/data/mysql_5727/data
MYSQL_LOG_DIR=/data/mysql_5727/log/mysql.err
MYSQL_CONN_STR="/usr/local/mysql/bin/mysql --socket=/data/mysql_5727/tmp/mysql.sock --port=5727 -Ne"
MYSQL_RECOVER_DB=test_recover
MYSQL_CON_FILE=/data/mysql_5727/my.cnf
SHUTDOWN_MYSQL="mysqladmin shutdown --socket=/data/mysql_5727/tmp/mysql.sock --port=5727"
STARTUP_MYSQL="/bin/sh /usr/local/Percona-Server-5.7.16-10-Linux.x86_64.ssl101/bin/mysqld_safe --defaults-file=/data/mysql_5727/my.cnf "
ls -l ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}
${MYSQL_CONN_STR} "select now()"
${MYSQL_CONN_STR} "create database if not exists ${MYSQL_RECOVER_DB} ; use ${MYSQL_RECOVER_DB};create table if not exists ${RECOVER_TABLE_NAME}(id int);"
ls -l ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}
rm -f ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.ibd
#mv ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm.bak
#mv ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.ibd ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.ibd.bak
/bin/cp ${SOURCE_DUMP_PATH}/${RECOVER_TABLE_NAME}.frm ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm
chown -R mysql:mysql ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm
${MYSQL_CONN_STR} " flush tables;use ${MYSQL_RECOVER_DB};show create table ${RECOVER_TABLE_NAME}\G"
COLUMN_NUM=`tail -5000 $MYSQL_LOG_DIR |grep "contains 1 user defined columns in InnoDB, but" |tail -1|awk -F, '{print $2}'|awk '{print $2}'`
echo 'Generate dynamic columns...'$COLUMN_NUM
echo "create table ${MYSQL_RECOVER_DB}.${RECOVER_TABLE_NAME} (" > $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
for ((i=1; i < COLUMN_NUM; i++))
do
echo "id${i} int," >> $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
done
echo "id$COLUMN_NUM int);" >> $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};drop table ${RECOVER_TABLE_NAME};"
${MYSQL_CONN_STR} "source $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql"
#create table ${RECOVER_TABLE_NAME}(id1 int,id2 int,id3 int,id4 int,id5 int,id6 int,id7 int)"
/bin/cp -f ${SOURCE_DUMP_PATH}/${RECOVER_TABLE_NAME}.frm ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm
chown -R mysql:mysql ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.frm
sed -i "s/^#innodb_force_recovery=6/innodb_force_recovery=6/g" $MYSQL_CON_FILE
${SHUTDOWN_MYSQL}
sleep 10;
$STARTUP_MYSQL &
sleep 5;
${MYSQL_CONN_STR} "show create table ${MYSQL_RECOVER_DB}.${RECOVER_TABLE_NAME} \G" >$SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
sed -i '1d' $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
sed -i '1d' $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
cat $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql
sed -i "s/^innodb_force_recovery=6/#innodb_force_recovery=6/g" $MYSQL_CON_FILE
${SHUTDOWN_MYSQL}
sleep 10;
$STARTUP_MYSQL &
sleep 5;
第2个脚本是加载.ibd文件的内容，因为上一步解析会生成SQL文件，所以在这个步骤里面就是transport tablespace的操作。这个脚本里面反复做了多种测试，最后发现还是flush table xxx for export的操作稳定一些。
#import_data_with_ibd
SOURCE_DUMP_PATH='/data/tmp_bak'
RECOVER_TABLE_NAME='receipt_12149_420'
MYSQL_DATA_DIR=/data/mysql_5727/data
MYSQL_LOG_DIR=/data/mysql_5727/log/mysql.err
MYSQL_CONN_STR="/usr/local/mysql/bin/mysql --socket=/data/mysql_5727/tmp/mysql.sock --port=5727 -Ne"
MYSQL_RECOVER_DB=test_recover
MYSQL_CON_FILE=/data/mysql_5727/my.cnf
SHUTDOWN_MYSQL="mysqladmin shutdown --socket=/data/mysql_5727/tmp/mysql.sock --port=5727"
STARTUP_MYSQL="/bin/sh /usr/local/Percona-Server-5.7.16-10-Linux.x86_64.ssl101/bin/mysqld_safe --defaults-file=/data/mysql_5727/my.cnf "
${MYSQL_CONN_STR} "select now()"
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};drop table ${RECOVER_TABLE_NAME};"
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};source $SOURCE_DUMP_PATH/ddl_${RECOVER_TABLE_NAME}.sql;"
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};flush table ${RECOVER_TABLE_NAME} for export;"
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};alter table ${RECOVER_TABLE_NAME} discard tablespace;"
/bin/cp ${SOURCE_DUMP_PATH}/${RECOVER_TABLE_NAME}.ibd ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.ibd
chown -R mysql:mysql ${MYSQL_DATA_DIR}/${MYSQL_RECOVER_DB}/${RECOVER_TABLE_NAME}.ibd
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};alter table ${RECOVER_TABLE_NAME} import tablespace;"
${MYSQL_CONN_STR} "use ${MYSQL_RECOVER_DB};select count(*)from ${RECOVER_TABLE_NAME} ;"
相关链接：