【MySQL】批量插入大量数据方法

参考了网上的方法。

有说换引擎为myisam，那myisam比innodb快的原因是什么呢。

再复习一下区别：

事务处理：

MyISAM是非事务安全型的，而InnoDB是事务安全型的（支持事务处理等高级处理）
锁机制不同：

MyISAM是表级锁，而InnoDB是行级锁
select ,update ,insert ,delete 操作：
1. MyISAM：如果执行大量的SELECT，MyISAM是更好的选择
2. InnoDB：如果你的数据执行大量的INSERT或UPDATE，出于性能方面的考虑，应该使用InnoDB表
查询表的行数不同：
1. MyISAM：select count() from table,MyISAM只要简单的读出保存好的行数，注意的是，当count()语句包含 where条件时，两种表的操作是一样的
2. InnoDB ： InnoDB 中不保存表的具体行数，也就是说，执行select count(*) from table时，InnoDB要扫描一遍整个表来计算有多少行
外键支持：

myisam表不支持外键，而InnoDB支持

为什么MyISAM会比Innodb 的查询速度快？ INNODB在做SELECT的时候，要维护的东西比MYISAM引擎多很多：

数据块，INNODB要缓存，MYISAM只缓存索引块，这中间还有换进换出的减少；
innodb寻址要映射到块，再到行，MYISAM 记录的直接是文件的OFFSET，定位比INNODB要快
INNODB还需要维护MVCC一致；虽然你的场景没有，但他还是需要去检查和维护

MVCC ( Multi-Version Concurrency Control )多版本并发控制

MyISAM适合：做很多count 的计算；插入不频繁，查询非常频繁；没有事务。

InnoDB适合：可靠性要求比较高，或者要求事务；表更新和查询都相当的频繁，并且行锁定的机会比较大的情况。

为什么是如果你的数据执行大量的INSERT或UPDATE，出于性能方面的考虑，应该使用InnoDB表，但是实际可能却相反呢？？？

I have a table with 17 million rows. I need to grab 1 column of that table and insert it all into another table. Here’s what I did:
INSERT IGNORE INTO table1(name) SELECT name FROM main WHERE ID < 500001
InnoDB executes in around 3 minutes and 45 seconds
However, MyISAM executes in just below 4 seconds. Why the difference?
I see everyone praising InnoDB but honestly I don’t see how it’s better for me. It’s so much slower. I understand that it’s great for integrity and whatnot, but many of my tables will not be updated (just read). Should I even bother with InnoDB?

1

The difference is most likely due to configuration of innoDB, which takes a bit more tweaking than myISAM. The idea of innoDB is to keep most of your data in memory, and flushing/reading to disk only when you have a few spare cpu cycles.

should you even bother with InnoDB is a really good question. If you’re going to keep using MySQL, it’s highly recommended you get some experience with InnoDB. But if you’re doing a quick-and-dirty job for a database that won’t see a lot of traffic and not worried about scale, then the ease of MyISAM may just be a win for you. InnoDB can be overkill in many instances where someone just wants a simple database.

2

InnoDB will be slightly slower because it is ACID compliant, has MVCC and does useful things like actually check foreign keys etc.

3

The reason is very simple. When you insert a row into MyISAM, it just puts it into the server’s memory and hopes that the server will flush it to disk at some point in the future. Good luck if the server crashes.

When you insert a row into InnoDB it syncs the transaction durably to disk, and that requires it to wait for the disk to spin. Do the math on your system and see how long that takes.

You can improve this by relaxing innodb_flush_log_at_trx_commit or by batching rows within a transaction instead of doing one transaction per row.

I highly recommend reading High Performance MySQL 3rd Edition (I am the author).

总结就是 innodb有一系列机制来保证数据安全，所以插入可能会慢但是更安全。